- This article is filed under:
- Chemical Manufacturing
- Consumer Goods
- Containers (glass, plastic, metal)
- Energy/Solar/Wind Power
- Fabricated Metal
- Food & Beverage
- Medical Devices
- Miscellaneous Manufacturing
- Primary Metal
- Scientific Imaging
- Semiconductor Equipment
- Waste Management
Machine Vision for Factory Automation
Teledyne DALSA Posted 06/11/2010
Many key tasks in the manufacture of products, including inspection, orientation, identification, and assembly, require the use of visual techniques. Human vision and response, however, can be slow and tend to be error-prone either due to boredom or fatigue. Replacing human inspection with machine vision can go far in automating factory operation, but implementers need to carefully match machine vision options with application requirements.
Nothing fabricated beats human vision for versatility, but other human weaknesses limit their productivity in a manufacturing environment. Boredom, distraction, fatigue, and sometimes even malice can degrade human performance in vision-related factory tasks such as inspection. Factory automation utilizing a machine vision system in such tasks, then, can bring many benefits. Machine vision systems can perform repetitive tasks faster and more accurately, with greater consistency over time than humans. They can reduce labor costs, increase production yields, and eliminate costly errors associated with incomplete or incorrect assembly. They can help automatically identify and correct manufacturing problems on-line by forming part of the factory control network. The net result is greater productivity and improved customer satisfaction through the consistent delivery of quality products.
Implementing a cost-effective machine vision system, however, is not a casual task. The selection of components and system programming must accurately reflect the application’s requirements. In addition, selection decisions need to consider more than the initial component costs. Factors such as the time required for system development, installation, and integration with the factory system, the operator training (and retraining) costs, project management, maintenance, and software upgrades and modification, all contribute to the total cost of ownership for the system and should be evaluated before investing in a specific system design.
Define the Requirements
One of the first places to begin in selecting a machine vision system for a factory automation task is to closely define the requirements. There are a number of critical questions to ask up front:
What task does the system need to perform?
Different tasks may require different vision attributes. Inspection requires an ability to examine objects in detail and evaluate the image to make pass/fail decisions. Assembly, on the other hand, requires the ability to scan an image to locate reference marks (called fiducials) and then use those marks to determine placement and orientation of parts. A machine vision system designed for the one task may not be well suited to the other.
What are the key visual performance criteria?
The vision system’s camera and lens must perform at the right levels. Factors such as the smallest object or defect to detect, the measurement accuracy needed, the image size (field of view), speed of image capture and processing, and the need for color all affect camera and lens choices. (See Box – Image Quality is the Key)
What are the environmental factors?
Some camera choices better suit stationary views while others are more suitable for handling linear object motion. Temperature, humidity, vibration, and the like can impose a need for specific system fabrication and assembly practices. The physical space available for installing the system can restrict camera and lens choices.
Beyond the system’s physical requirements, developers should also consider the operational requirements. Questions to address include:
Who will program the system?
If the expertise to configure the system is not available in-house, the user must depend on third-party support to make changes and correct errors in the vision system’s programming. If the system needs periodic changes, such as to inspect a new product line or to interface with new production equipment, the question of programming becomes particularly important. A system that has been set up for a single task so that the system integrator needs to reconfigure it for new settings can result in production systems being shut down for extended periods while alterations are underway. A system set up with enough flexibility to allow factory personnel to make such adjustments may cost more to create, but will save production time later.
What equipment must the vision system interface with?
A vision system that only activates a solenoid to eject failed parts from a production line is considerably easier to implement than one that also reports results to a quality control network or that controls the operation of production equipment based on inspection results. Similarly, a system that must inform and enable a human operator has different needs than one that interfaces only to other machines.
What information must the system provide?
Machine vision systems in factory automation seldom operate in a stand-alone mode. Instead, they must send information to other parts of the factory enterprise for a variety of purposes. Quality traceability, for instance, requires that the vision system either log or report inspection results to the enterprise. Highly controlled operations, such as pharmaceutical manufacturing, may also require the logging of access to and changes made in the vision system, sending such data to a secure drive on the company network.
What are the operator requirements?
The extent to which human intervention into and control of the machine vision system is required can affect many system elements, particularly software. If operators are required to periodically change inspection criteria, such as the tolerances that will be accepted, the software must support such manipulation. Software may also need to provide security to prevent unauthorized access or parameter manipulation and include safeguards to avoid the introduction of erroneous values. Software design can affect the type and degree of training that operators will require as well as the ease of system maintenance and modification.
Building a Machine Vision System
While the answers to these operational and functional questions depend on the application, all machine vision systems for factory automation share some fundamental attributes and behaviors. Systems all have a need to image or inspect a scene or object, operating on a continuous basis at the fastest practical speed. Systems all operate by using the following steps:
• Position the object or camera so that the camera can view the object or scene
• Capture an image with a camera
• Process the image
• Take action based on the image processing results
• Communicate results to operators and other factory systems
Because of this commonality, examination of a specific application such as inspection of objects on an assembly line will help illustrate the method by which developers can build a suitable machine vision system for their application.
The essential elements of an inspection system include a delivery vehicle, the vision system, the response system, and sensors to trigger image capture and system response. The delivery vehicle positions the object for inspection. The vision system, which includes camera, optics, lighting, and image processor, captures and processes the object image to determine a pass/fail response. The response system takes the required action as well as communicating results to operators or other systems. The sensors serve to trigger the vision and response systems, identifying when the object is positioned properly for the systems to perform their tasks.
A first step in developing an inspection system, then, is to determine how the parts are to be placed in front of the camera for imaging. In this example, the delivery vehicle is a conveyer belt that carries the objects past the vision system at a constant speed. Other possible delivery vehicles include a part feeder, a robotic arm, or humans placing an object in a station for off-line inspections. Choosing a delivery system can often be the hardest part of a factory automation design because delivery choice will place restrictions on the remaining system choices, including camera, lighting, sensors, and response systems.
With the delivery system chosen, developers can determine the most appropriate method for triggering the vision system to capture the image, and triggering the response system to take action. In the case of a conveyer belt delivery vehicle, an appropriate sensor might be electronic photo-eyes that produce a signal when the object passes between them. With other delivery vehicles, sensors such as proximity switches or programmable logic controllers (PLCs) could serve. Manual triggering by a human operator is also an option.
The image capture, processing, and evaluation of results are tasks for the vision system. This system determines if the object being inspected is within acceptable quality tolerances and directs the response system as to what actions to take. A separate vision controller such as the DALSA IPD Vision Appliance may handle the image processing and evaluation, or those functions may be integrated into a smart camera.
The parameters to be evaluated as well as the object characteristics strongly affect the camera, optics, and lighting needed in the vision system as well as the image processing software. Some examples of how some typical applications are handled: The reading of an identification number requires close-up imaging, front lighting, and optical character recognition software. Inspection of packaged water aerators requires an entire package view and color imaging. Inspecting the fill level in a detergent bottle requires back lighting and the ability to detect the position of the liquid’s surface.
With an appropriate vision system chosen and decision criteria determined, the last step is to define how the system is to respond to its decisions. In this example the vision controller triggers a PLC to push rejected parts off the conveyer to another delivery system, allowing acceptable parts to continue undisturbed. The controller may also send decision results to the factory enterprise for quality control and traceability purposes.
Ensuring Factory Integration
While factory and production specialists need to understand the essential steps of vision system design, they seldom have the in-depth knowledge and experience needed to develop the vision system itself. Thus, the selection of a third-party supplier and/or system integrator is an essential step in ensuring the efficient integration of a vision system into factory automation. There are a number of factors to consider, both up-front and long-term, that will affect the effectiveness and total cost of ownership of the vision system.
One of the first factors to consider is resolving the sometimes opposing needs of the system design and the runtime operations. To control design costs the system should have a well-specified and bounded task. This makes development and programming simpler and allows optimization of components such as camera, lighting, and optics. To lower total cost of ownership, however, runtime considerations must also be given priority, and these can complicate the design task.
One key runtime consideration that affects design is system portability: the ability to reconfigure the system for use in a different production line or to accommodate slight variations in environment or specifications. A system that targets a single task with fixed specifications is easier to develop but will require vendor involvement to make even small changes in operation. Designing for flexibility and extendibility of the system requires greater effort, but can give the users the ability to accommodate changing requirements without vendor assistance, saving time and costs.
A flexible vision system, for instance, will offer a number of options for communicating with third-party factory equipment, operators, and the factor enterprise rather than be restricted to the initial requirements. Physical interfaces might include digital I/O for hardwiring to photo sensors, status indicator lights, PLCs, and directional control devices. Serial ports allow communication with PLCs, motion controllers, robotic equipment, and touch-screen displays. The system could offer Ethernet running TCP/IP for interfacing to the factory enterprise and other equipment and also support other protocols and standard languages such as Modbus and Profibus. Similarly, a flexible system design will use modular hardware design so that system elements can be readily replaced, upgraded, or modified.
Along with flexibility, vision system users should consider the system’s extendibility. For instance, it can be valuable to choose system software that includes access to a library of image processing and analysis functions. Having additional functions available beyond the minimum needed to address immediate concerns will simplify future modifications and enhancements to the vision system’s task. Similarly, hardware extendibility such as the ability to add cameras or change camera resolution at a reasonable cost can be a valuable hedge against future requirement changes.
System maintenance is also an important concern, especially software updates. Over time, faster and more accurate algorithms as well as new communications protocols become available, along with operating system enhancements and bug fixes. Ideally, the system vendor makes such updates to the vision system software readily available for little or no cost.
Keeping in mind all these various performance, operational, and future-oriented considerations can seem a daunting task, but the rewards for doing so are substantial. Machine vision is a key technology for improving the quality and productivity of manufacturing lines though factory automation. Carefully defining the vision system’s task, understanding its performance criteria and technology limitations, and planning its integration into the factory flow are all critical steps toward a successful system design. Choosing a solution that will satisfy both current and long-term needs will help minimize the total cost of ownership and maximize the system’s productive lifetime.
Factory automation applications vary widely, but a key factor in all of them is the vision system’s image quality. Image quality directly affects the accuracy and precision with which the system can perform its task, so image quality requirements require careful attention. Otherwise, a system may end up trying to perform a high-precision task using a low quality image. Three main system elements affect image quality: the camera, the optics, and the lighting.
The camera is the image capture element of a vision system. Its key parameters are the size of its sensor, its resolution in pixels, the type of sensor (area, line scan, or TDI: time delay and integration), and the sensor technology: CCD or CMOS. (For more information on camera sensor choices see DALSA Whitepaper: Applications Set Imager Choices). The speed of the sensor, color capability, and sensitivity to non-visible wavelengths may also be important in some applications.
In consumer devices the optics are considered part of the camera, but in vision systems they are a separate element with their own set of specifications. Key specifications for the optics include their working distance, their field of view, their resolution, their speed (light-gathering capability), and the size of camera sensor they support. Other factors that can affect image quality include lens materials and anti-reflective coatings.
Lighting is one of the most challenging aspects of a vision system. Incorrect or inadequate lighting of an object or scene can dramatically increase error rates in vision systems. Yet, the proper lighting for an application depends strongly on the task to be accomplished and the mechanical and optical characteristics of the objects to be imaged.
Fortunately, there are a few general guidelines that vision system developers can follow. One is to strive for consistent, uniform illumination; the vision system may perceive variations in illumination as variations in the objects themselves. Both spatial and temporal consistencies are important. Changes in lighting over time can also cause increased error rates, and can more difficult to detect than non-uniformity within the scene.
A second general guideline is to light the scene in a way that amplifies the things to be detected or measured, such as markers or defects. Amplification for vision systems means increased contrast. Thus, if the system is to detect a fiducial marker on the surface of an object, front lighting that avoids shadows and reflections is appropriate. If the system is to look for defects in a glass panel, on the other hand, rear lighting looking through the glass may be more effective.
Finally, the lighting should attenuate clutter and background effects. Image clutter makes identification and extraction of desired information more difficult and error-prone. Similarly, background effects such as reflections and shadows can prevent recognition of key features or trigger false recognitions. The simpler the image the faster and more reliably image processing can extract desired information.