How to determine pass/fail criteria for camera image quality?

This post is an excerpt from the article “Managing Supply Chain Image Quality”

The “Quality Staircase” shows how MTF requirements should be reduced throughout the manufacturing process to balance image quality with yield.

Deciding on pass-fail criteria is challenging and involves multiple tradeoffs between system performance, component cost, and manufacturing yield.

Ideally, in the research and development phase of designing your camera system, a lens, sensor, signal processing, and camera body components will be selected that will adequately perform the human and/or machine vision task you need to have performed.

For human vision, you may consider a collection of perceptually related Image Quality factors such as those included in ISO CPIQ or VCX standards.
For machine vision, you may consider image quality factors where if your system does not provide sufficient quality, the computer vision task will fail. Beyond traditional image quality factors consider information metrics as a quality factor that will include sharpness and noise in a single measurement.

Once you have decided on the components that will comprise your imaging system, you should build and test a collection of sample cameras. Depending on the precision of you and your suppliers’ manufacturing and assembly processes, these samples will fall within a distribution of quality.

In manufacturing, a golden sample is a perfectly made component. If you use a single golden sample to set your pass-fail criteria, you would want to repeat several tests with that sample in order to determine its minimum quality in the presence of sources of variation because of sensor noise and other factors. You also should consider where in the production line you are doing the testing and how the Quality Staircase impacts quality loss.

But if you only consider a single “golden sample”, you may find that other cameras that are not quite as good as the golden sample but “good enough” will fail. So we introduce the term “bronze sample” to represent a device that barely meets your needs, but if it was any worse in quality, it should fail. You should select a bronze sample based on your application needs (see above). Then you should select pass fail criteria just below what your bronze sample is able to achieve so that your bronze sample will always pass.

As you ramp to mass production, you should consider the production yield. If the pass-fail threshold threshold you select leads to a production yield that is too low:

you may need to reconsider your selection of components in order to get improved quality (return to R&D).
you may need to increase the requirement for quality from your suppliers and your incoming quality control step.
you may need to improve your internal manufacturing tolerancing when it comes to the final assembly of your camera’s subassemblies.

If you need additional help navigating this process, Imatest consulting services could help you to make better-informed decisions about your camera quality.

Read “Managing Supply Chain Image Quality”