Industrial or non-industrial application areas for imaging are, for example, automotive, measurement and testing technology, optics, medical technology, photo technology and electronics. Until recently, the processing of 2D image data was the standard. It was used to track moving objects in the image sequence and to determine their position – but only in 2D space, such as products on a horizontal conveyor belt. Hence, it limited the application possibilities of the technology in the automotive production environment. Thus, the use of 3D vision technologies helps to increase the degree of automation in production scenarios.
Key technology of image processing
Camera and image processing systems are ubiquitous in the automotive world. 100% quality control on the production line requires the systems to ‘see’ spatially and detect micrometre variations from the target value at high speed. Intelligent imaging systems are also providing insights when driving, directing traffic or in crash laboratories & wind tunnels. Observers in the crash laboratory reflexively close their eyes and high-speed cameras capture the perfectly illuminated scene from every conceivable perspective, both outside and inside the crashed vehicles, enabled by robust housings and electronics. Thousands of images per second and per camera ensure that engineers can subsequently follow every detail of the deformation of car body in HD quality.
Automotive industry relies on camera systems
High-speed cameras enable detailed fault analysis, especially in very fast-moving manufacturing processes. In the case of high-speed processes, the cameras’ superiority is obvious. But even in the case of mainstream manufacturing speeds, quality and process control is increasingly being transferred to imaging systems. Image processing ensures brilliant surfaces around the clock, micrometre-accurate assembly tolerances and defect-free circuits on the increasingly prevalent chips & micro controllers.
Absence of errors
For automotive manufacturers, errors are synonymous with costs. Camera and laser systems are resulting in automotive production of glass-like transparency, with equally glass-like materials. The optical sensors employed range from the shortwave ultraviolet (UV) range via the most diverse array of cameras, within the visible light spectrum to deep into the infrared (IR) and terahertz (Thz) range. Painters use them to measure micrometre-thick coats of paint. Engine developers are gaining insights into injection and combustion processes that are hidden to the human eye. Aerodynamic engineers use camera systems in wind tunnels to render even the slightest turbulence visible and capable of analysis. Their vehicle interior, tire and vehicle body development colleagues use laser systems to locate and measure the source and extent of vibrations and structure-borne sound. All of them use imaging systems, not just for diagnosis but also for measuring and documenting their measures’ success.
Process optimisation is replacing retroactive inspection
Early optical detection diverts defective components and semi-finished products immediately from the processes, eliminating costs and wasteful material consumption incurred by subjecting them to further processing & defects developing into scrappage. The systematic analyses highlight error patterns at an earlier stage. Processes can be adjusted with the least accumulation of variances. Production managers can define in advance when the systems sound the alarm. Above all, the indefatigable camera systems ensure quality control around the clock.
Ensuring flexibility & higher productivity
Three-dimensional robotic guidance introduces flexibility into fully automated processes. Cameras locate the characteristics; image processing software uses them to calculate the current location and future plan-of-action. This way, the robots orient themselves as required. Here, 3D robotic guidance needs only to be parameterised; modular systems are scalable with various camera sensors and processors to the most diverse processes. Because 3D robotic guidance is accompanied by a precisely controlled and individually documented process that is far superior to manual operation as regards to speed and accuracy, process measurements can be reconciled with the underlying CAD data.
Many roads lead to the third dimension
3D image processing is based on different processes, such as, mono, stereo or clustered multiple camera systems, which are in operation. In addition, there are time-to-flight sensors based on photo mix detectors (PMD). The systems emit high-frequency light waves. As soon as these are reflected by component surfaces, they are guided by special optics onto the pixels of a light sensitive chip. The distance to each point on the surface can be calculated from the ‘flight time’ of the light. The difference relative to a reference beam is determined in order to accelerate the calculation process. The sensor system thereby generates a highly accurate 3D image. Other processes use angular calculation for spatial ‘vision’. Whereas mono cameras calculate all six degrees of freedom using only three distinctive features of a three-dimensional object. Stereo systems, analogously to human vision, can orient themselves spatially, based on the differences in the stream of images from two cameras. Other systems rely on the projection of lines onto objects. These measuring lines are curved to match the components’ contours. Thus, the combination of several cameras within a sensor cluster enables the 3D coordinates of large objects to be determined with a high degree of accuracy.
Special case shiny surfaces
For imaging systems, shiny surfaces are a challenge. To cope with them, developers use a trick: they use special light sources to superimpose patterns on the surfaces and distortion of which then enables them to determine the precise curvature of three-dimensional objects. Deflectrometric techniques are achieving measurement accuracies of a few thousandths of a dioptre. For quality control of the growing diversity of car window shapes or for real-time paint monitoring on the production line, the systems combine the speed of 2D surface inspection with 3D measurement process accuracy.
Powerful impetus for quality control
100 per cent real-time inspection on the production line is an optimal match with the six sigma approaches in automotive industry quality management. Manufacturers and major suppliers are working towards their zero defects objective, according to a control loop: define –measure –analyse –improve –control. Camera systems combined with downstream analysis software are an important aid in this. Moreover, lightweight materials and the electrification of drive systems and auxiliaries entail new tasks for quality inspectors. And finally bonding and forming processes are changing. Gluing or riveting rather than welding are becoming increasingly common. This is transforming test routines, as is the forming of high-strength steels, lightweight alloys and fibre reinforced plastics.
New standards ensure flexibility and bandwidth
In case of multi-camera applications, questions abound as to bandwidths, processor performance, data processing speeds and the quality of the information thus generated, as well as questions regarding interface standards.
Integral to the transformation is also the increasing variety of megapixel sensors and processors. In the latter case, multi-core strategies and the integration of graphics card processors are catching on as a result of the greatly increased data flows that three-dimensional image processing entail. The upshot of all this is that optical inspection and control systems are becoming faster and more accurate. The key to this is the coexistence of interface standards with different strengths. They all have one thing in common: greatly increased bandwidths – be it CameraLink HS, USB3 Vision, Gigabit-Ethernet or CoaXPress, enabling designs between 400 and 3600 megabytes per second (MB/s).
The coexistence of standardised interfaces increases the modularity of the imaging sector as a whole. Because users can select the cameras, processors and software available on the market that are best suited to their specific intended purpose, without being dependent on individual manufacturers in so doing. The camera suppliers have long since responded by offering cameras with all the relevant interfaces and chips that exploit the bandwidth capability. Ever higher resolution megapixel images at ever higher frequencies enable production
robots and measuring systems to see with ever greater acuity.
Imaging contributing to traffic safety
Image processing has not just established itself in automotive production. Reversing cameras protect against car body damage. At night, infrared cameras warn of unlit obstacles and pedestrians wearing dark clothing on the road. Stereo cameras and laser sensors measure the distance to vehicles driving ahead, detect animals suddenly running out into the road, thereby laying the foundation for cruise control systems and emergency brake assistants. Driverless test vehicles are already negotiating their way through city traffic without difficulty, driving in convoys on motorways, including performing overtaking and completely unprepared braking manoeuvres, as well as journeys on secondary routes. Here too, the key to progress is the combination of optical sensors and intelligent image processing. The same goes for the use of imaging systems in traffic management centres and finally, police speed controls as well. The old green ‘speed camera boxes’ with a single camera are being supplanted by modern laser systems capable of monitoring several lanes simultaneously, thereby reining in speeding urban drivers.