• Font Size:
  • A
  • A
  • A

Feature Articles

Machine Vision-Based Proofreading. . . .by Nello Zuech, Contributing Editor

by Nello Zuech, Contributing Editor - AIA

Accuracy, accuracy, accuracy is the name of the game in all printed materials produced for the pharmaceutical industry. This includes labels, shelf cartons, inserts and outserts. The requirements are especially challenging for inserts and outserts. Many of these can be 20 – 24’‘ x 20 – 24’‘ printed in 4 point type. The result is lots of characters and symbols.

Major concerns are omissions, illegible print due to smearing, ink splash, etc. – pretty much any condition that could go wrong in a press run that would lead to inability to read the text accurately or otherwise be subject to misinterpretation. Additional concerns relate to the accuracy of the printed material as compared to original data today found in one or another computer format. The placement of a decimal point is critical. There is a big difference between 1.0 and .10 or 10.0.

In order to detect both subtle and gross concerns a number of companies have developed machine vision-based proofreading systems. Typically these will use a high-resolution scanner to input an image of the printed material. Processing to achieve precise registration within the computer memory is critical as typically image subtraction is performed – the current image being subtracted from the image of the printed material stored in memory. The difference regions between the ‘‘golden standard’‘ and the current image are flagged so an operator can quickly determine by viewing the display and toggling to the scene location where the difference is detected whether or not the difference is real and a genuine concern or a false alarm.

When one is detecting subtle differences in scene changes, any changes stemming from misregistration between the current image and the ‘‘golden standard’‘ will lead to false alarms. In the past the number of such false alarms was so high that it discouraged the adoption of these machine vision-based proofreading systems. Today false alarms are down to more acceptable levels, often less than the number of real regions of concern that are detected. Significantly, no escapes of real regions of concern are acceptable. Consequently, these systems are set to be more sensitive to potential regions of concern, which, in turn, leads to some false alarms.

Using these systems has reduced the amount of time it takes to proofread printed materials from what were often hours to minutes. Significantly, these systems permit proofreading against the actual original computer generated artwork and printed information rather than against an original print, which itself could have been flawed.

To gain insight into proofreading systems that are on the market, companies known to offer such systems were canvassed for input to this article. The following made contributions:

  • Gary Parish, President, Complete Inspection Systems
  • Miriam Glassman, Marketing Communications Manager, Global Vision, Inc.
  • Dr. Mike Negin, President, Mnemonics
  • Dr. Stephan Krebs, President, Nyquist Systems

Dr. Negin in his response makes some comments about what is meant by electronic proofreading: ‘‘Maybe the best way to start this discussion is to ask, What is not a proofreading system?

    • It is not a spell checker.
    • It is not a grammar checker.
    • It is not a style checker.
    • It is not software that treats only a single document.
    • It is not a ‘‘print quality’‘ inspector.
    • It is not an OCR system.
    • It is not an OCV system.

Some of the above features are built into word processors; others are features of vision systems.

So what is a proofreading system?
It is a collection of comparison engines that performs the comparison of two or more documents. The documents can be from an electronic source or a physical source. For example:

    • Electronic
    • Word processor files like Word
    • PDF, PS, EPS
    • TIF, JPG, PNG, images
    • RIP (raster image processing) file from prepress operations
    • Physical
    • Printed pages (from laser printers, inkjet, lithographic)
    • Press sheets, proof sheets, inserts, outserts, labels
    • Printing plates (metal or flexographic plates)
    • Blue lines (or blue prints)
    • Photographs

The goal of a proofreading system is to compare two or more documents to:

    • Find content deviation between files of a mixed format (e.g., a Word file to a PDF file)
    • Content comparison is performed by extracting the internal coded information from the document (e.g., the ASCII or Unicode character representations) and then comparing the strings. Since this is not a vision problem, this proofreading requirement will not be discussed in detail.
    • Find content and format deviations between files of similar format and layout. This is the classical proofreading task required when comparing a source document to a sample document. The documents can be electronic or physical provided they can be converted to have similar layout.

This task is performed by performing image comparison between a standard and a sample.’‘

1.  Can you provide a brief general description of your proofreading system? Basic theory, brief implementation description (cameras, optics, lighting, material handling, file compatibility, online/offline, etc.).
[Gary Parish – Complete Inspection Systems]
We offer a number of automated proofreading systems for both Image and Text Comparison. Our main product called AutoProof Pro can be used to scan, capture, or compare electronic files. We convert the images to a Bitmap and provide inspection up to 1200 DPI. It is an automated system noting the defects. Another system called AccuProof Pro allows users to manually align and compare images including resizing. A third called Inspec is a workstation that allows users to use either a CCD camera or Digital Camera to capture and compare. AutoProof Matrix provides the ability to grid large documents for comparing sections. Another called AutoProof Pharma provides a complete audit trail to meet CFR 21 Part 11 standards.

[Miriam Glassman, Global Vision] Proofreading is a huge challenge for most pharmaceutical and printing companies around the world. It is often done manually. This method is extremely time consuming and lends itself to a great margin of human error. With over 15 years of experience, Global Vision is proud of its history as the pioneers in automated proofreading and inspection solutions.

There are three main ways our inspection solutions are handled. Which type is used depends on the particular needs of the organization (example: working with hardcopy only, degree of technologically savvy users etc…) our solutions include:

  • Camera Based Solutions
  • Scanner Based Solutions
  • Software Solutions
  • Secure Workflow Consultancy Services 

Which system you use depends on the client’s environment. Our systems work in environments where the workflow is:

  • Digital to hardcopy
  • Hardcopy only
  • Digital only

The concept is the same regardless of the tool (Camera, Scanner, or Software Only). By electronically comparing revisions, our solutions automate the proofreading process by locating the difference from one document to the other (whether it be a master and sample or revisions).

Our recent product launch, DOCU-PROOF™ C³ is a breakthrough technology that compares unlike (dissimilar) document layouts in any language. In the past, text inspection was always limited to comparing like (similar) documents, where embedded text and graphics were impossible to inspect and proofread.

[Dr. Mike Negin – Mnemonics] MNEMONICS proofreading system, AVIA (Automated Visual Information Analysis) removes the human visual detection process from the proofreading task, but leaves the decision task to the reviewer. This is a classical separation of feature extraction and decision rule steps found in classical pattern recognition. AVIA finds the flaws, differences, and errors so that the human is left with the task of making the decisions. AVIA’s operator interface is designed so that navigation through the resulting images and tabulated results is integrated and enables several different review paradigms handling exceptions as well as systemic decision-making.

AVIA is fundamentally a software algorithmic image comparison engine. As such, the images can be from any source and can be used for any image comparison application. For proofreading, the imaging devices are usually large format scanners, flatbed scanners, or CCTV cameras. Because of the AVIA capability for general image comparison, the same engine can be used for a wide variety of uses in a variety of scenarios. For example, documents and inspection profiles can be set up off-line (e.g., in a pre-press application, or sampled QA use) and then be moved to on on-line application (e.g., in label inspection). The AVIA engine is the same engine regardless of the application. The configuration of the comparison and the source of the images may change, but the inner working of the comparison are identical.

The basic theory is to compare two ‘‘images’‘ of the two (or more) documents. Documents in this context can be from any source provided the documents can be converted to an image that consists of a two dimensional array of vectors. The vectors themselves may be points (a gray scale image) or they may be color pixels (e.g., C, M, Y, K) or other more complicated vector data structures. The vectors may be of a mixed data type structure. For practical purposes, thinking of the images as gray scale images is the simplest way to understand the process.

The implementation is based on non-linear image comparison techniques that minimize energy differences between the images. Because of physical printing issues such as paper stretch and warp and ink bleed through, the non-linear comparison techniques are required to prevent excessive false alarms.

AVIA is designed to treat an image as a large multidimensional vector. AVIA is therefore independent from the imaging device, as long as the image can be made available in memory. As a practical matter, the image sources should be of high quality (e.g., large format scanners up to 54 inches wide can generate images on the order of 300 megapixels, electronic cameras from 640x480 (300 Kbytes) to megapixel cameras, electronically generated images from documents (e.g., pdf to image conversion). File compatibility is not a significant issue, as long as the file source can be loaded into computer memory.

[Dr. Stephan Krebs – Nyquist Systems] Our company provides inspection from prepress, via press to postpress applications. A critical issue is the validation of the reference image for the inspection on press or on the rewinder. Our proof reading system bridges the gap to the original artwork by comparing the image taken from the camera of the web inspection system back to the artwork pdf image. The images are taken from high-resolution high-speed line scan cameras. Typically, we use cameras with a resolution of 6144 pixels.

Two or three cameras can be used across the web. The high resolution does not allow for color proof, as such cameras are not available today. However, as our systems are mainly used for critical pharmaceutical or security applications where this is not an issue, as grey scale images work very well. Furthermore, we can scan a sample from the press and compare it back to the artwork. In this case we use flat bedscanners(<17’‘) or wide format scanners (up to 48’‘) to obtain the image.

2.  What have been some specific applications of your proofreading systems?
[Miriam]
80% of our business is dedicated to providing automated proofreading solutions to major pharmaceutical companies around the world. Global Visions solutions can be found all around the world and their applications vary in terms of complexity.  Some Departments that apply our solutions in the pharmaceutical industry include:

  • Regulatory
  • Marketing
  • Graphics
  • QA/QC 
  •  Production
  • Pre-Press

[Mike]

  •  
  • Pharmaceutical incoming inspection and document verification
  • Sampled QA inspection from a web press (proofing a scanned press sheet compared to an electronic master or a printed proof)
  • Includes printing plate inspection, blue line checking, proof sign off against an original .pdf file
  • Electronic document comparison and revision level management during document design

[Stephan] Verifying final press make ready to approved artwork by ensuring that all of the information that is printed is correct. Especially crucial is confirmation of small print used in pharmaceutical and security applications. Color is not an issue with these applications.

[Gary] We have a large number of systems installed in the pharmaceutical and healthcare industries as well as the packaging industry. Our customers also include the US Treasury Department and NSA.

3.  What are the market drivers encouraging the adoption of proofreading systems today?
[Mike]

  •  
  • Global markets. Because many products are printed in languages that are difficult to find fluent proofers for, automated systems can perform document comparison paying attention to punctuation and accent marks that are critical to the document’s meanings. Automated proofing ameliorates the difficulty for humans to proof documents in their non-native tongue.
  • Global issues. Document printing may be done at sites remote to the pharmaceutical company. Electronic summary reports generated by automated proofing systems can be shared as web pages, pdf documents, and other web sharing collaboration.
  • Automated proofing systems make the report generation and collaboration easily realizable.
  • Printing costs. Automated proofers can catch print errors during production. (AVIA has this capability). These print errors can be corrected while the print job is in process and corrective action can be taken immediately. This prevents the job from having to be setup multiple times saving considerable costs.
  • Validation. Human proofers cannot be validated … automated systems can be.  

[Stephan] Make sure that the correct print exists and is complete and accurate. Avoid any mix up by providing seamless inspection from the artwork to the outgoing product. Proofing the reference image used in the press inspection or rewinder inspection is crucial.

[Gary] Accuracy, speed, and liability are the main issues.

[Miriam] The two main factors driving the need for our solutions are the need to significantly reduce costs and increase accuracy. Errors on labels or inserts can have a disastrous impact on companies. It can cost them millions of dollars in product recalls. The proofreading process in itself is a challenging & time-consuming process. A lot of mistakes get by even at the early stages.

4.  Are there specific sizes and formats for which your approach is more suitable? Why?
[Stephan]
No

[Gary] We can handle any document up to 50 x 50 inches in a single pass using large sheet scanners. We prefer conversion to Bitmaps and if we are extracting the text, a PDF format.

[Miriam] No. Our solutions can adapt to any workflow environment.

[Mike] AVIA typically handles document sizes from as small as 1 x 1 inch labels to 36 x 48 inches. Using different imaging devices can accommodate smaller or larger sizes. AVIA can handle a variety of physical media and electronic formats. Any document that can be converted into a raster image can be effectively used. AVIA can also compare a single standard to multiple stepped repeat patterns on the physical printed or electronic media.

5.  What sensitivity does your system offer? How would you characterize the performance of your system?
[Gary]
Highly accurate with multiple methods to check the accuracy.

[Miriam] Extremely easy to use, flexible, and reliable. A major advantage to our solutions is in fact the sensitivity issue. The degree of sensitivity as far as detecting differences goes can be adjusted based on the unique needs of the organization. Global Vision is the only provider that can offer this degree of flexibility and offer such high-level custom solutions.

[Mike] AVIA has size and contrast parameters that permit the user to establish performance. By setting these parameters, AVIA can be configured to be more or less sensitive to flaw size. AVIA’s typical performance is such that the proofing process can be completed in less than 10 minutes for a large document (36x48 inches) and less than 5 minutes for letter size documents. For a large document (36x48 inches), AVIA’s processing takes less than 90 seconds.  The processing speed is proportional to the document area. The human review of the results usually takes 5 to 10 minutes.

The system is usually set up by the users to find errors as small as a partial decimal point in 4 or 6-point print. The number of other print flaws that are found to be non-critical are usually less than 20 which is a very manageable number to review. AVIA also has a clustering feature that consolidates small neighboring defects into larger groups, which makes reviewing of the results more efficient.

[Stephan] We use high-resolution, grey scale, line-scan cameras (typically 6144 pixels) and can run up to three cameras in parallel across the web. Optical resolutions vary depending on the application but typically run from 350 to 750 dpi. For applications where we use scanners, the resolution is typically 150-600 dpi. The system’s performance is geared for production throughput.

6.  What has made it possible to achieve the performance you are able to achieve today?
[Miriam]
Technology & Focus: Our technology is developed hand-in-hand with our customers. 15 years of experience has helped us tremendously. We’ve grown into an organization that has a devoted team of developers that spends every single day focused solely on continuously improving our automated proofreading solutions. Our success has depended on staying ahead of the technology, listening to our customers, and having a dedicated team of professionals at every stage (From Support, Development, Implementation, Validation, Training, to Sales).

Customer Relationship: In-depth understanding of the industry and the needs of every department involved in the process.

Compliance: Systems that follow all regulation requirements such as Current FDA 21 CFR PART 11.

[Mike] Proprietary software techniques…

  • The use of non-linear image comparison techniques enables us to achieve very fast performance, with high sensitivity and high specificity. Image alignment techniques in translation and rotation and scale also enable the system to operate automatically. These techniques correct for printing warp and bleed through.
  • Automated clustering techniques are a significant performance factor in reducing ‘‘false alarms’‘.
  • Automated illumination correction allows us to compare documents with brightness variations of more than 50%.
  • Magnification correction of up to 400%.
  • Very friendly user interfaces are very important for the customer to be able to learn to use the system quickly and effectively.

[Stephan] PDF technology and advances in line-scan cameras (high-resolution at high-speed)

[Gary] The introduction of higher resolution scanners and digital cameras. One camera that we use is 5400 x 4800 pixels.

7.  Besides the pharmaceutical industry, are there other industries embracing proofreading systems?
[Mike]
Because AVIA is a general-purpose comparison engine any set of images with a common base can be inspected. The following are areas where emerging interest exists:

  • General printing industry
  • Prepress setup checking
  • Food and tobacco industry
  • High-speed document printing
  • Chemically etched circuit comparison
  • Currency inspection
  • Photographic & satellite imagery comparison (reconnaissance)
  • Sequential CAT Scan comparisons
  • FLAT panel display inspection

[Stephan] Security applications (especially currency) and RFID.

[Gary] Printers and any company that has a liability if the printing is incorrect.

[Miriam] Printing, packaging, publishing, consumer goods manufacturers.

8.  What are the specific application issues that one must be attentive to when deploying machine vision-based proofreading systems?
[Stephan]
The nature of our business is providing tailored solutions to our customers that include prepress, press and postpress. There are therefore no such issues.

[Gary] Like any vision application acquiring a quality image using correct lighting, material handling, and lenses are critical.

[Miriam] Automated proofreading solutions are meant to identify errors once the documents have already been created, circulated, and/or certified. Clients should also consider reviewing their workflow process and practices regardless of what machine vision-based proofreading systems are being used. Global Vision provides analysis and recommendations on how to improve and secure the entire workflow process at every stage. So by the time the automated comparison takes place, errors have already been reduced and total workflow efficiency has been maximized. Having Global Vision participate in securing the entire workflow process is another way our clients have benefited from our expertise.

[Mike]

  •  
  • Site planning
  • Installation
  • Integrating the system into the customers procedures

9.  What is it that you require from a prospective buyer of a proofreading system to assure you will deliver a system that will satisfy them?
[Gary]
We require a User Defined Specification as to the accuracy of the system.

[Miriam] The more information they can provide about their workflow process from departments such as Regulatory, Marketing, QA/QC to Production, the better we will be in a position to recommend the more appropriate solution. It is important to get a high level overview in order to get the most out of our expertise when implementing systems.

[Mike] Samples of the range of proofing materials must be provided. These materials must be reviewed for applicability for our systems approach. We typically process these as a courtesy to the customer and then send the results to the customer via our PDF and HTML (web page) reports. A follow up discussion is then held to review any performance or applicability issues that arise. Electronic documents are also reviewed to guarantee that the customer’s files can be rasterized. Once the samples have been reviewed, a demo is usually arranged so that the customer can get a hands-on, eyes-on review of the system. We often arrange an on site trial for the customer for them to try the system out.

[Stephan] Information on image alignment and possible web stretch can be an issue in some applications. We have experienced this before with some flexible materials and had to adapt our algorithms accordingly. The smallest feature size (usually text size) is important to know in order to specify the system properly.

10.  Are there specific application issues that must be understood insofar as selecting the correct proofreading system?
[Miriam]
Understanding the difference between scanner-based and camera-based systems.

[Mike] Only to the extent that the customer has some unusual applications or expectations are there questions. Generally speaking, our qualification procedure sorts out the issues.

[Stephan] Web stretch issues as covered in question 9.

[Gary] Absolutely. Materials and how they are printed. Inks bleed, paper stretches. In printing, some plates are distorted so that they print accurately on the web.

11.  Are there some emerging technology changes associated with the underlying technology embodied in your proofreading systems that will positively impact your product line? What will those impacts be?
[Mike]
Our nonlinear processing has already impacted our other software products. One of the impacts is the capability to allow for matching images that are not exactly the same with figures of merit and being able to find flaws that can be more easily quantified.

[Stephan] In some applications the addressable image size under 32 bit Windows Systems has been an issue. 64 bit processing PCs will solve these problems.

[Gary] Higher resolution cameras, computer speeds, and software will continue to simplify and improve the process. This will allow us to speed up the process and assure users that they can detect any defect accurately.

[Miriam] FDA enforced PDF environment for submission. Our solutions are ahead of the game by incorporating this regulation into our solution. Now the FDA/HL7 (body that brings forth these guidelines) is enforcing SPL (Structured Product Labeling). Our products are being engineered to comply with SPL already. Global Vision is a fully compliant solution. We stay on top of all guidelines and regulations that might affect the workflow process.

12.  Any advice you can give to someone investigating the purchase of a proofreading system?
[Stephan]
Make sure that the operating system is up-to-date (Windows XP Pro) and that the solution is scaleable.

Test how long the proofing takes for your largest images (may slow down production). Try whether the system finds partially broken characters even down to the missing decimal point. Make sure that the system does not trigger too many false defects when set to a specific sensitivity where it can find the missing decimal point. In general, emphasize what the system does not find and workout the limits with your production material. Make sure the system can be properly integrated into your work-flow. This includes setup of network drives (directories). Brail might be a feature required soon for the European market.

[Gary] Yes, like any vision application, make sure that the hardware and software meet their needs. Run tests and comparisons to assure that the system will find the defects as well. We offer online interactive tests to check the systems as well. Select systems that are simple for the customer to use.

[Miriam] Yes, there are many important items that should be considered:
1) Make sure you learn about the company history and client portfolio. Reputation and experience is key.
2) Company size. Make sure the company has the resources to support your short-term and long-term needs. Find out if there is an established team in place for deployment, training, support, validation, etc.
3) Learn where the products originate. Are they manufactured or developed in-house? Customers need complete security. By designing and manufacturing all our systems in-house, this gives our customers increased security, reliability, and flexibility.
4) Verify quality standards. Global Vision has been able to pass every supplier audit because it complies with the highest sets of standards such as ISO 9001: 2000 and cGMP(Current Good Manufacturing Practices).

[Mike]

  • Use a checklist and ask lots of questions.
  • Ask for the vendor to compare files, documents, etc. and provide a report.
  • Arrange a hands-on demo.
  • Ignore sales hype.
  • Ask for a live demo.

     

  • Ask about an in-plant trial.
  • Ask about a money back guarantee.

 

 

Training of personnel
Cost of proofing. Because of FDA regulations, proofing often has to be a 200% inspection requiring hours of human effort, which is costly and error prone.
Printing facility proofing

 

Comments:

Bilal Muhammad   (April 9, 2019 03:33 AM)

Hi there, I'm looking forward to install electronic verification system for printed packaging components (leaflets, labels, cartons and pharma code) at our site GSK, Pakistan. Dropping my cell number. Cell Number:923132397711

 


Leave a Comment:

All fields are required, but only your name and comment will be visible (email addresses are kept confidential). Comments are moderated and will not appear immediately. Please no link dropping, no keywords or domains as names; do not spam, and please do not advertise.

First Name: *
Last Name: *
Your Email: *
Your Comment:
Please check the box below and respond as instructed.

Search AIA:


Browse by Products:


Browse by Company Type: