• Font Size:
  • A
  • A
  • A

Feature Articles

Machine Vision Software

by Nello Zuech, Contributing Editor - AIA

If the hardware used in machine vision systems can be likened to the “heart” of the system, the software can be likened to the “blood” of the system. One needs both to give life to an application. While many configurable vision products come with embedded machine vision software, there are also a number of companies that offer hardware independent machine vision software. In some cases this software came out of universities. In other cases merchant system integrators developed software as a means of minimizing the extent of “reinvention” required whenever they designed a system around a different configurable vision product. Today one will also find suppliers of frame grabbers that have traditionally offered optional machine vision products compatible with their hardware are now offering the software independent of their hardware.

Machine vision software is different than image processing software. In the case of machine vision software, the image processing software is generally transparent to the user. The graphic user interface of the machine vision software will generally refer to applications rather than specific image processing algorithms. More often than not, the underlying image processing software is accessible for those who want to get 'under the hood' so to speak. However, for most applications it is not necessary for the user to get 'under the hood.'

The software tools incorporated into a suite of machine vision software generally permit one to interface with a camera as well as a production line and automatically perform the image enhancement, segmentation and analysis as a function of the specific application tool enabled – 'find' or location analysis, gage, flaw sense, character verification, character recognition, symbol recognition, etc.

Many of those offering hardware independent general-purpose machine vision software also offer application-specific software packages for some of the more common applications: label inspection, date/lot code, alignment, OCR/OCV, BGA inspection, LED/LCD integrity, print quality inspection, fiber optic inspection, etc. If yours is one of these applications it makes sense to purchase the specific software package and the most cost effective hardware.

Today one can develop a host-based application solution employing a digital camera with FireWire or USB connectivity and a PC with a third-party machine vision software package. These digital cameras typically come with SVGA resolution, though many are now available with megapixel imagers. The result can be a very cost effective solution given the relatively low cost of the components.

To catch up with the latest activities in machine vision software, input for this article was canvassed from all known suppliers of third-party machine vision software in the machine vision market. The following submitted responses to the questions that follow.

  • Jan-Philippe deBroeck, Product Manager, Adept Technology
  • Dr. Lutz Kreutzer, Manager PR and Marketing, MVTec GmbH
  • Dr. Robert K. McConnell, President, Wayland Research

1. Can you provide a one or two paragraph general description of the underlying principles of your suite of machine vision software? Image processing/segmentation principles? And Image analysis principles?
[Jan-Philippe deBroeck – Adept Technology] The HexSight machine vision library uses a combination of more than one technology, which results in two types of tools. The first, based on geometric pattern matching, uses contours in order to find previously trained models. The main tool is a geometric object location tool called Locator upon which are built other 'finder' tools that help in finding simpler geometric entities like lines and circles. The second includes tools using a more “classical” grey-scale approach. Those are edge detectors, calipers, blob analyzers, etc. Tools from the two technological approaches can be seamlessly combined to obtain more performance and robustness.

Additionally, a set of image processing tools can perform image filtering, sharpness and histogram analysis, etc.

[Dr. Lutz Kreutzer – MVTec GmbH] HALCON is a comprehensive machine vision software package that is used worldwide. HALCON’s flexible architecture facilitates rapid development of machine vision and image analysis applications. The software provides an extensive library of more than 1100 operators with outstanding performance. HALCON supports a wide range of operating systems and providing interfaces to more than 40 popular image acquisition devices and was designed as a programmer's toolbox.

ActivVisionTools is a quick and efficient application creator with an easy-to-use graphical interface. By directly supporting digital I/O boards, and with more than 40 interfaces to widely used image acquisition devices, ActivVisionTools can be integrated into any production process. ActivVisionTools’ open software architecture is extensible and scalable to always fit your needs. The single tools are based on the established HALCON library. ActivVisionTools was developed for engineers not wanting to write the code themselves.

Techniques:

  • Shape-based Matching
  • 1D Metrology – Measuring
  • 2D Metrology - Subpixel Edge Detection
  • 3D Metrology - Stereo Vision
  • Blob Analysis
  • Morphology
  • 3D Camera Calibration
  • Bar Code & Data Code Reading
  • OCR & OCV (multilayer perceptron neural network)

[Dr. Robert K. McConnell – Wayland Research] Our software, WAY-2C, addresses a specialized market that requires classification based on complex color, multispectral, or multi-sensor images. This market has been poorly served by traditional machine vision systems. WAY-2C has a considerable degree of hardware independence allowing it to process images from a range of image input devices from thermal imagers and small format RGB video cameras to megapixel digital still cameras and a variety of color, multispectral, and hyperspectral image file formats.

Humans, and other creatures use color as a position independent classifier. Useful information is often associated with complex color distributions. WAY-2C uses a powerful robust non-parametric statistical method to classify image regions based on comparisons of the color distributions of the 'test' region with those from a set of one or more reference classes. The results of the classifications may include any or all of the following: digital and serial output for process control, statistical information, and interpreted images.

2. Are the image processing and analysis tools accessible for modification or addition by the user? If so, describe how.
[Lutz]
HALCON can easily be extended with customized functionality via an open interface. The custom operators are completely transparent to the application and can be used in an identical manner as the HALCON operators provided by MVTec in HALCON's rapid prototyping environment HDevelop as well as in all applications. Furthermore, HALCON can also be extended with customized interfaces to arbitrary image acquisition devices via an open interface. These custom interfaces are transparent to the application and behave identically to the interfaces provided by HALCON.

ActivVisionTools provide a generic mechanism to develop customized tools via its ActivGenericInterface tool. Furthermore, the functionality can be extended in an arbitrary manner by using ActivHALCONFoundation, a tool that allows access to a large part of the functionality in HALCON.

[Rob]  No.

[Jan-Philippe] Users have full access to all the images and contour scenes generated by the HexSight tools. They can combine custom pre-processing tools with the standard image analysis tools or do the opposite, i.e. feed custom image analysis tools with the results of the standard image processing tools.

3. Can you provide a one or two paragraph general description of your graphic user interface? What interface steps need to be taken to set up an application?
[Rob]
The WAY-2C user interface is built around an image window, a hierarchical set of menus, and a text output window. One trains the system much as one would train a human inspector: by pointing out and naming reference examples of the objects in example images and indicating the locations to be inspected in the 'test' images. By clicking on menus the user can assign inspection sequences, trigger parameters, digital I/O values, etc. Among the many other capabilities are verifying the adequacy of the training from goodness of fit statistics, and editing logs from interactive sessions into scripts for automating the entire inspection process.

For certain classes of systems such as basic sorting, assembly inspection, and sample card inspection we offer packages which further simplify the training and setup effort. Starting with a few table entries, these packages generate a complete set of application specific scripts, which lead the operator through the training and inspection setup process.

WAY-2C can also be controlled by scripts generated by almost any kind of customer software capable of generating ASCII text files, directly by ASCII commands from external computers, and in the Windows environment by similar commands from user programs.

[Jan-Philippe] The graphical user interface is provided as a set of ActiveX controls property pages. Some tools used to create the property pages are also provided in order to reduce the users’ development time. A good example is a tool called HexSight Display that users can directly paste and use as is.

To build an application with HexSight, a C++, VB or C# project has first to be created. Next, a user has to add a control called HSApplication and open its property pages. The user interface is then used to build a vision sequence. All vision tools are provided as ActiveX controls that can be configured through their property pages. After this, it simply requires compiling before you can run the application. Users can also programmatically invoke the property pages for all HexSight ActiveX controls at runtime for additional tweaking and monitoring of the vision tools.

[Lutz] HALCON offers no graphical user interface as such since it is a programmer's toolbox that is intended to be extremely flexible and customizable. What HALCON offers is its rapid prototyping environment HDevelop, which enables users to quickly develop the machine vision part of their application by providing powerful programming, debugging, and data visualization and inspection mechanisms. Once the machine vision part of the application has been developed the resulting code can be exported to various target languages (C, C++, Visual Basic, Visual Basic .NET, C#) and can thus be integrated easily into the application.

HDevelop comes with its own GUI with dialogs for realtime interactive inspection of image properties and visualization of iconic variables. HDevelop immediately gives feedback on the execution of operators and instantly shows the result of the processing. With ActivVisionTools, the user simply starts Visual Basic (or other COM-aware development environments like Visual C++ or Visual Studio .NET), selects the appropriate tools, places them on a form, and optionally adds a very small number of calls.  With this, the application is ready and only needs to be parameterized.  Hence, applications can be developed extremely quickly.

4. How do you segment the machine vision software market? What are the major types of machine vision software available in the market today?
[Jan-Philippe]
Vendors offer generic machine vision software libraries, which are the most flexible, and also some applications specific software solutions to perform a set of specific and simpler tasks, for instance barcode reading or OCR. Some application-specific machine vision software can also solve more complex problems like vision-guided motion that requires tight integration of vision with motion control. For this application, the use of specific vision guidance software is most often a big advantage to the users. Adept has recognized this requirement and introduced a product called Adept iSight.

[Lutz] We basically have three types of machine vision software today:

I. Library-based systems
The typical solution for PC-based vision: a call of a function of a library.

Background
Software development without using libraries is more or less impossible. Libraries enable the user to develop their own applications by using reliable functions. Instead of being occupied by writing low-level image processing functions, the user can focus on the development of the application itself. The user can expect that a library for machine vision supports the concepts of the host languages as much as possible.

For example, a library used in C++ must have a class hierarchy as well as its own memory management. The most important benefit is as well the most important drawback: the highly flexible library still is one part of an application; the user-interface and the interaction with the
production process needs to be programmed in addition to the actual machine vision solution.

Advantages

  • Very flexible
  • Easy to use on a programming level
  • Both usable on standard hardware and smart or intelligent cameras

Disadvantages

  •  
  • Experience in programming required
  • User interface needs to be programmed

Products

  •  
  • MIL by Matrox
  • CVL by Cognex

II. Closed systems
A closed system is simply an executable program on whatever hardware.

Background
Users that don't like to program their machine vision application might look into closed systems, i.e. executable programs. Closed systems offer a task-oriented approach for solving a machine vision task: the control of the image acquisition, image processing, and result visualization.

Each step usually is easy to parameterize. The drawback here is the fixed control flow of the application, which might not fit into the particular application of the user. Also extensions of closed systems are close to impossible, e.g. the adaptation of the user interface: an OEM customer might not want to see the logo of a machine vision component supplier on his machine.

Advantages

  •  
  • Configuration instead of programming
  • Easy to learn
  • Both usable on standard hardware and smart or intelligent cameras

Disadvantages

  •  
  • Hard to extend the functionality

Products*

  • NeuroCheck by Neurocheck GmbH
  • WiT by DALSA Corporation
  • P400S by Panasonic
    * See advantages in component-based systems

III. Component-based systems
Mostly make use of Microsoft's COM architecture and encapsulate certain task-oriented machine vision functionality.

Background
Since the introduction of software components in the Microsoft Windows world many people got aware of the concepts behind components. The current naming of software components in Microsoft terminology is COM (Component Object Model) and refers to a mechanism to establish interfaces between components enabling a component to share data with other components via a native Windows operating system support.

Advantages

  • COM interface allows closed systems to get accessed and parameterized by other applications
  • Standardized interfaces support network usage

Disadvantages

  • Only in Windows usable: leads to proprietary systems
  • Difficult to use for smart or intelligent cameras / special hardware
  • Programming is still required

Products

  •  
  • VisionPro by Cognex
  • Common Vision Blox by Stemmer

[Rob] In the specialized world of color-based machine vision we would divide the market into the following:

1. Low cost sensors suitable for differentiating simple colored objects.
2. Traditional vision systems, including both smart cameras and PC based systems employing simple color models. Such systems have served both to define and to unnecessarily limit customer expectations for color-based inspection.
3. Advanced systems such as WAY-2C that can be quickly trained to distinguish complex multi-colored distributions with an accuracy comparable to humans but at much higher speed and reliability.


5. What have been some applications for which your machine vision software has been found particularly well suited?
[Lutz]
Applications

  • Board, Wafer & Die Inspection
  • Positioning/Alignment
  • Completeness Inspection
  • Surface Inspection
  • Print Inspection
  • Metrology
  • Identification
  • Medical Image Analysis
  • Remote Sensing & Aerial Image Interpretation
  • Surveillance & Security

[Rob] Assembly and kit inspection including such products as automotive fuse boxes and fabric sample cards; color based sorting of lumber and plastic recyclables; interpretation and quantitative analysis of large fabric and remote sensing images.

[Jan-Philippe] All applications that have to do with parts location, disambiguation of similar parts and inspection of these found parts are applications for which HexSight has been very well suited, even more in harsh industrial environment where the tool’s robustness can be leveraged. The vision guidance segment is certainly one where HexSight and more recently Adept iSight have proven to be quite successful.

6. What are the compatibility issues one must be aware of when purchasing third party, product independent machine vision software?
[Rob]
In the world of color-based machine vision these are too numerous to list here. It is essential to develop a close working relationship with the vendor or manufacturer and an appropriate guarantee that the system will meet your specifications.

[Jan-Philippe] Since this type of software is sold as a standalone product, compatibility has to be considered for the platform it will be running on; this includes the CPU architecture (some software might not have the flexibility to run on various CPU architectures), CPU performance, operating system and, if this applies, the software development environment. Compatibility with hardware acquisition devices like frame grabbers and cameras also needs to be considered. The more extensive use of standards-based devices like IEEE1394 (FireWire) and USB makes this easier for cameras whereas on the frame grabbers side machine vision software vendors usually list the set of boards they support. In some applications, compatibility with other types of devices that will be used in the application should be considered, for instance motion control devices in a vision guidance application. 

[Lutz] Third party machine vision software should have interfaces to the most popular vision hardware (frame grabber and cameras) and furthermore, it must be independent from operating systems. Only in this case, independent software saves your investment and is prepared for future hardware changes.

7. What are the specific application issues that one must be attentive to when developing an application around commercially available third party machine vision software?
[Jan-Philippe] Users should have a clear understanding of how the vision fits in the application with all the other components. Some applications may involve many components to be integrated together with vision or might require knowledge and expertise in other fields to perform a good vision integration. Vision guidance integration can be quite challenging and it might be easier for someone to buy a more complete package than developing the whole application. On the other hand, many inspection applications will be very well handled by the integration of a machine vision library.

[Lutz] The developer of a machine vision application should make sure that one is not limited by the underlying machine vision software. Thus, a third party machine vision software package has to be comprehensive, flexible, and also extensible. Furthermore, the machine vision software should be independent of the used image acquisition device and available on different operating systems. This ensures that the software is ready for future requirements and helps the developers to secure their investment.

Another important issue to consider is that the manufacturer of the third party machine vision software should provide highly qualified support. Naturally, the support should include the assistance while evaluating a new machine vision application.

[Rob] Again, in the world of color-based machine vision issues are too numerous to list here. A close working relationship with the software vendor or manufacturer is essential.

8. What is it that you require from a prospective buyer of a machine vision software suite for an industrial application to assure you will deliver a software package that will satisfy their application?
[Lutz]
A manufacturer of machine vision software must know about prospective users needs. Manufacturers must ensure to develop robust, accurate, and reliable, and thus, sophisticated algorithms under simultaneous observance of an easy-to-use mode of operation. On the one hand, the solution challenges of special industrial applications are rising; and on the other hand, the application areas are increasing. Therefore, the industry needs comprehensive high-performance software suites as well as competitive modular standard software tools.

[Rob] We start off with sample product or product images representing the range of items, which must be distinguished. To this we add information on speed, layout, visual, budgetary, and other known inspection constraints. Together these allow us to set up a realistic test of our system's ability to perform the required tasks. They also serve as a basis for acceptance specs if the project goes ahead.

[Jan-Philippe] We make sure with the prospective buyers we have a clear understanding of their application, which will help us refine their requirements. This will also makes sure we deliver a package that optimally solves the problem. It is very useful to get real-world data from them in the form of a wide set of application images for instance in order for us to help them optimize the use of the tools. Although the tools can easily be integrated by our customers without our help, customers are always impressed by the level of performance that can be reached when we work together at selecting the right tools and optimizing them for their specific application’s challenges.

9. What are the most important differentiators between machine vision software products today?
[Rob]
In the world of color-based recognition they are ease of use and classification reliability.

[Jan-Philippe] Other than the level of integration of the various products (generic library vs. application specific software), if we look at the vision algorithms themselves, the main differentiators are the robustness of the algorithms (consistency, ability to perform well in harsh environment or varying lighting conditions), the performance (time to find parts, ability to quickly differentiate similar parts), the ease of use and configuration of the various tools, and the ability to quickly and effectively integrate the software tools into an application.

[Lutz] Regarding the development of a machine vision application: Flexibility, extensibility, open architecture, and ease of use.

Regarding the execution of the completed machine vision application: Robustness, accuracy, speed, and reliability.

10. In your opinion, what are the main product features in machine vision software products that customers are looking for today?
[Jan-Philippe]
Customers are looking at features that will not only solve their technical challenges but also allow them to bring their own product to market quickly. From that perspective, ease of use and ease of integration into an application combined with a high performance geometric object locator and inspection tools will prove to be very compelling. Robustness of the vision tools will also help save a significant amount of time on the hardware setup, lighting, etc. A robust machine vision solution will enable better time-to-market.

[Lutz] Power, flexibility, pricing, reliability, and ease of use.

[Rob] Most knowledgeable customers are interested in cost-effective systems which are easy to setup and will operate reliably in their particular application.

11. Are there some emerging technology changes associated with the underlying technology used in machine vision systems that will impact your machine vision software products? What will those impacts be?
[Lutz]
Particularly the camera market will move closer to machine vision software because of changed hardware interfaces like IEEE 1394, USB 2.0, and GigE. Thus, standard software products increasingly become a crucial factor for the rapid implementation of cost-effective machine vision systems.

[Jan-Philippe] The hardware technology enhancements definitely have an impact not only on today’s machine vision software performance but also on the type and breadth of products that will be offered in the future. The ability for instance to embed machine vision software in smart cameras provides new exciting opportunities, although it doesn’t replace the use of the PC in many situations.

12. What are major trends you see regarding specifically machine vision software? How will machine vision software change going forward?
[Rob] We could speculate that as understanding of natural perception improves it will result in vision systems that can more closely mimic such perception.

[Jan-Philippe] Machine vision software is likely to evolve on two fronts. First, on the packaging side, we will see more application-specific products that are not as flexible but which help solving problems in areas becoming increasingly complex. This is particularly true for applications in vision guidance. These products will provide benefits in terms of time-to-market but also cost and risk management. We will also see the machine vision software become more and more easy to use, configure and integrate for the same reasons mentioned above.

On the vision algorithms side, the constant performance increase of PCs (especially CPUs) will allow not only performance increases but also the addition of new tools and algorithms that will help solve problems for which vision could hardly be considered before.

[Lutz] The future of machine vision means easier application building. Users want more and more simplification in use, but at the same time high-end features in functionality.

13. Will machine vision software become increasingly unbundled from machine vision hardware? Will closed system, proprietary software development tools increasingly give way to open system (unbundled) machine vision software?
[Jan-Philippe] The application and the specific customer’s requirements for flexibility will dictate the level of integration. Whereas bundled or proprietary solutions tend to take have a smaller footprint by sparing the use of a PC, unbundled machine vision software running on a PC provides unequaled flexibility and constantly leverages the hardware technology enhancements happening in the PC world. The latter also allows the customers to choose the software and hardware solutions independently rather than being tied to a given product. The price pressure being what it is on PCs, it is often also a cost advantage to use a PC, given that in many applications a PC will be used anyway to perform other tasks.

[Lutz] Yes, machine vision software will become increasingly unbundled from machine vision hardware because of the cost/performance ratio. During the past, many companies developed machine vision software themselves because of the lack of appropriate products in the market. However, standard machine vision software offers substantial cost savings – the added value of a software product that has proved itself in tens of thousands of systems worldwide cannot be achieved when engineers develop
the software themselves. Above all, software tools must excel in features such as accuracy, speed and robustness, all at an affordable price.

Both approaches will still coexist in future dependent on the specific requirements of an application. However, with standard hardware and software becoming more and more powerful, unbundled software will become more important. Proprietary systems often cause a higher cost of ownership and are difficult to adapt to hardware changes. Instead of this, standard machine vision software benefits from the continuous development of standard hardware and development environments and is more flexible regarding future hardware trends.

[Rob] We think there will continue to be a mix. The more routine operations where the software costs can be spread over many systems will be bundled. Where more flexibility is required unbundled systems will be used and the buyers will continue to be surprised by how much of the true system cost is in software, non-recurring engineering costs, and training.

14. Any advice you can give to someone investigating the purchase of machine vision software?
[Lutz] Take your time for evaluating different products according to the following scheme or let the manufacturer of the third party software provide you answers on the points:

  • Technical eval
ActivVisionTools by MVTec
User interface difficult to customize
User interface included
HALCON by MVTec
Experience in machine vision required

 

Comments:

There are currently no comments for this article.


Leave a Comment:

All fields are required, but only your name and comment will be visible (email addresses are kept confidential). Comments are moderated and will not appear immediately. Please no link dropping, no keywords or domains as names; do not spam, and please do not advertise.

First Name: *
Last Name: *
Your Email: *
Your Comment:
Please check the box below and respond as instructed.

Search AIA:


Browse by Products:


Browse by Company Type: