Microrobotics and Control Engineering

Research Group:
Tools and Technologies for Automation in Micro- and Nanorobotics

The research group develops the necessary technologies for micro- and nanoscale manipulation. The scale “nano” does not relate to the robot’s dimensions, but to the capability to position specimens with very high accuracy. Typical specimens are carbon nanotubes (CNTs) with a diameter of 300 nm and a length of less than 10 µm. Imaging sensors such as optical microscopes or scanning electron microscopes (SEM) are often the only way to detect positions of those robots.

The scope of the group includes the research and development of nanorobots, their actuation and control as well as process planning and automation of nanorobotic systems. A second research topic is image processing algorithms for different tasks on the micro- and nanoscale. Robot tracking as well as object detection and classification are of high interest.

Development of Nanorobotic Systems

The research area of micro- and nanorobotics deals with the design and fabrication of components for precise positioning of specimens and end-effectors as well as their validation. The figure shows a prototype of a microrobot. Its dimensions are 22x22x11 mm³ and the robot can be moved in three degrees of freedom (x, y, φ). The velocity is adjustable over a wide range (nm/s to mm/s).

The main focus is on the development and improvement of driving principles which meet the high requirements of micro- and nanopositioning. Some of the most important demands are a high positioning accuracy, no backlash and vacuum capability.

Nanorobot control


Movement following the stick-slip principle

In order to use the developed nanorobotic systems in an automation architecture, they need to be electrically driven and controlled. The most common actuation principle is the so called stick-slip principle (see Figure) using piezoceramic actuators. Initially, the robot rests at a certain position. It is then moved by applying a slowly rising voltage to the actuators. Reaching the maximum voltage, the polarity is quickly reversed resulting in a rapid deformation of the actuator. Due to its inertia, the robot cannot follow the high acceleration of the actuator and the legs slip on the surface. Thereby, the robot performs a small step.This kind of actuation requires signals with large amplitudes (up to 400 Vpp) and sawtooth-like shapes. The behavior of the robot is controlled by the signal’s parameters such as amplitude, frequency and polarity.

Simulation, Validation and Optimization of stick-slip-drives for nanorobotic applications

Piezo-actuated stick-slip microdrives (PASSMDs) or piezo-actuated stick-slip actuators have received much attention due to their main advantages of simple design and noticeable performance. PASSMDs are actuated by piezoelectric actuators (PEAs) and based on the motion of the stick-slip principle, which allow for displacements of the devices with not only very high precision, but also theoretically unlimited range. These attractive characteristics offer a wide range of applications in the field of micro- and nanorobotics. Here, the high precision of miniaturized positioning and manipulation systems is a key requirement for fine positioning.

The focus of this research is the modeling of the contact dynamics between frictional partners, which is believed to play an important role in determining the working characteristics of these driving principles. With this model the physical behavior of the drives will be theoretically and experimentally investigated. From the physical model, parameters of the drives as well as control signals will be optimized so that their operating behavior is improved in general.

Algorithms for object-tracking and -detection for automation on the nanoscale

One research topic is tracking algorithms for the automation of micro- and nanohandling processes. The algorithms are mainly based on active contour models. They are extended to deliver additional information from the image such as z-positions (using defocus analysis) or deformation measures, or to be robust against typical effects (deformations, movement distortions, shadowing, etc.) which occur especially in scanning electron microscopy imaging of automated handling setups. This is achieved by including transformation models for the distortion and designing the active models and feature generators to exhibit greatest robustness against the distortion effects. Considering motion distortions, features are best to be temporally close connected. E.g. line templates are one suitable approach. In general, features need to be unique even if they are distorted, and robust recognition of these features has to be possible.

Additionally, the research includes the development of algorithms for the extraction of 3D positions from images using multiview approaches, defocus analysis and other means. This is necessary for the reliable automation of real world processes. For characterization, also 3D reconstruction algorithms are being researched. These algorithms deliver depth maps or 3D profiles of the scenes observed. In this 3D data, segmentation of objects is a necessary and useful task which encounters difficulties due to occlusion and ambiguities. The segmentation algorithms in development aim at solving these challenges.

High-speed object tracking


Imaging systems are often the only feasible sensor for detection and tracking of objects and tools. A lot of time during automation is consumed by the closed-loop positioning of tools based on image data. To overcome this drawback, systems with high processing speed and predictable runtime are needed. High-speed cameras with optical microscopes are used on the microscale. On the nanoscale, a special designed hardware directly manipulates the electron beam of an SEM.

Image processing performed on off-the shelf PCs has major drawbacks in terms of latency and predictability. Therefore, embedded systems based on field programmable gate array FPGA or microcontrollers are used for high-speed tracking. By using hardware modules for large parts of the image-processing, predictable high-speed processing is possible on embedded systems. The embedded systems directly work on the camera data stream (or SEM data stream respectively) to have lower latency and therefore earlier results for the closed-loop positioning systems of the robots.

Detection and manipulation of particles inside magnet-resonance imaging devices

Magnetic resonance imaging has become a widespread technique in clinical practice over the last decades. There are more than 20,000 installations worldwide. The main advantage of MR-imaging is the high contrast between different soft tissues. Also it does not incorporate ionizing radiation. Generally the magnetic fields used for MR-imaging may also be utilized to move magnetic particles.

A possible application is the minimally invasive delivery of drugs inside the cardiovascular system of a human body exactly to the point where they are needed by combining the drugs with a ferromagnetic microcapsule.

Until now, the imaging sequences of an MRI have been reprogrammed to include a propulsion-phase between each image acquisition, which sets the parameters of the gradient coils to the values needed for the propulsion of the ferromagnetic objects. The detection and tracking of the objects is achieved through the susceptibility artifacts, which can be identified by geometric distortions and signal losses in the resulting MR-image. Detected actuating variables are transferred to the imaging sequence via a feedback-loop. Hence, a closed loop control has been developed which allows ferromagnetic objects to automatically follow predefined trajectories. Future research aims at scaling down the size of propelled objects and bringing the technology towards clinical applications.

Multimodal Image Registration


Overlapping of a SEM and
AFM scan after multimodal
image registration

In many fields of work in which imaging systems are applied, a number of different imaging modalities exist. E.g. in clinical diagnostics magnetic resonance imaging (MRI), computer tomography (CT) and ultrasound are used. In nanotechnology, scanning electron microscopes (SEM) and atomic force microscopes (AFM) are used to display objects on micro- and nanometer scale. Often, one certain imaging modality is most suitable to display a desired image feature. To combine the advantages of two or more imaging modalities, the overlapping (registration) of the images and combined display is essential. For the registration, a similarity measure evaluates the quality of overlapping, and an optimization technique is used to iteratively enhance it until an optimum is reached.

At AMIR, due to the integration of an AFM into the vacuum chamber of a SEM, it is possible to successively acquire AFM and SEM images of a sample. Due to the different techniques the results of AFM and SEM images vary strongly, so the combination of both images is of great interest. Based on this setup, similarity measures and optimization techniques are developed to address the problem of multimodal image registration.

Projects

Current Active Projects

Completed Projects

  • MiCROW
  • µ4CNC
  • RACoNa - Reliable Assembling of Colloidal Nanoparticles in Two and Three Dimensions by Dual-AFM-based Handling inside a Scanning Electron Microscope
  • Powerbonds - Enhancement of Fiber and Bond Strength Properties for Creating Added Value in Paper Products
  • NanoBits - Exchangeable and Customizable Scanning Probe Tips
  • Flexible Piezoantriebe für Massenmarktanwendungen
  • FIBLYS - Building an Analyzing Focused Ion Beam for Nanotechnology
  • HYDROMEL - Hybrid ultra precision manufacturing process based on positional- and self-assembly for complex micro-products
  • NANOHAND - Micro-Nano System for Automatic Handling of Nanoobjects
  • ZuNaMi - Zukünftige Verfahren der Nano-/Mikroproduktion
  • NANORAC - Nanorobotics for Assembly and Characterization
  • ROBOSEM - Development of a Smart Nanorobot for Sensor-based Handling in a Scanning Electron Microscope
  • RoboMat - ROBOter zur Bestimmung von Mikro-MATerialeigenschaften
  • NanoStore - Mikroroboterzelle zur automatisierten Handhabung und Montage von CNTs für die Integration von Mikro- und Nanoobjekten innerhalb eines Rasterelektronenmikroskops

Teaching

Teaching offers are only available in German. For questions please contact us via email.

Most relevant papers

  • Dahmen, C. & Fatikow, S. (2011), Tracking of objects in motion-distorted scanning electron microscope images, in 'Proc. IEEE/RSJ Int Intelligent Robots and Systems (IROS) Conf', pp. 19--24
  • Dahmen, C.; Wortmann, T. & Fatikow, S. (2011), Actuation and Tracking of Ferromagnetic Objects using MRI, in 'Proc. Int Optomechatronic Technologies (ISOT) Symp'
  • Diederichs, C. (2011), Fast Visual Servoing of Multiple Microrobots using an FPGA-Based Smart Camera System, in 'Proc. of the 18th IFAC World Congress'
  • Fatikow, S.; Edeler, C.; Diederichs, C.; Meyer, I. & Jasper, D. (2011), Design and Control of a Nanohandling Robot, in 'Proc. of the 13th World Congress in Mechanism and Machine Science'
  • Jasper, D.; Diederichs, C.; Edeler, C. & Fatikow, S. (2011), 'High-speed nanorobot position control inside a scanning electron microscope', ECTI Transactions on Electrical Eng., Electronics, and Communications 9(1), 177--186
  • Jasper, D. & Fatikow, S. (2010), 'Line Scan-based High-Speed Position Tracking inside the SEM', International Journal of Optomechatronics 4(2), 115--135
  • Fatikow, S.; Dahmen, C.; Wortmann, T. & Tunnell, R. (2009), Vision feedback for automated nanohandling, in 'Proc. Int. Conf. Information and Automation ICIA '09', pp. 806--811
  • D. Jasper and C. Edeler, “Characterization, optimization and control of a mobile platform,” International Workshop on Microfactories, IWMF, 2008.
  • C. Edeler, D. Jasper, and S. Fatikow, “Development, control and evaluation of a mobile platform for microrobots,” in Proc. of 17th IFAC World Congress, 2008.
  •  C. Stolle, S. Fatikow: "Towards automated nanohandling in a scanning electron microscope", Proc. of the 6th IEEE Conf. on Industrial Informatics, Daejeon, Korea, July 13-16, 2008, pp. 160-165
  • T. Wich, C. Stolle, O. Frick, S. Fatikow: "Automated Nano-Assembly in the SEM: Challenges in Setting up a Warehouse", Proc. of the 17th IFAC World Congress, Seoul, Korea, July 6-11, 2008, pp.12751-12756