High-contrast edge-SFR test targets produce invalid MTF results
- Henry Koren
The obsolete ISO 12233:2000 standard defines a resolution test target with a high contrast ratio. These are typically produced at the maximum dynamic range of a printer, which can be anywhere from 40:1 to 80:1. The high contrast can lead to clipping of the signal which leads to overstated invalid MTF values. Some camera manufacturers who want better MTF results may take advantage of this anomaly to overstate the quality of the cameras they produce. This is why it is critical to validate cameras with a proper measurement system that includes a low-contrast target.
4:1 contrast is the international standard
The international standards committee (ISO) has addressed this issue by reducing the contrast ratio to a lower 4:1 level. See the following excerpt from ISO 12233:2014 (Page 7, section 0.3):
All other imaging conditions being equal, camera SFRs using different target contrast edge features can be significantly different, especially with respect to their morphology. This is largely due to non-linear image processing operators and would not occur for strictly linear imaging systems. To moderate this behaviour, a lower contrast slanted edge feature (Figure C1) was chosen to replace the higher contrast version of the first edition. This feature choice still allows for acuity amenable SFR results beyond the half sampling frequency and helps prevent non-linear data clipping that can occur with high contrast target features. It is also a more reliable rendering of visually important contrast levels in naturally occurring scenes.
The standard defines the edge-SFR chart with modulation contrast shall be between 0.55 and 0.65. You can raise 10 the power of the contrast to determine the contrast ratio or in other words: The contrast ratio shall be between 3.54: 1 and 4.47:1.
Clipping warnings
A Clipping warning is issued if more than 0.5% of the pixels are clipped (saturated), i.e., if dark pixels reach level 0 or light pixels reach the maximum level (255 for bit depth = 8). This warning is emphasized if more than 5% of the pixels are clipped. Clipping reduces the accuracy of SFR results. It makes measured sharpness better than reality.
The percentage of clipped pixels is not a reliable index of the severity of clipping or of the measurement error. For example, it is possible to just barely clip a large portion of the image with little loss of accuracy. The plot on the right illustrates relatively severe clipping, indicated by the sharp “shoulder” on the black line (the edge without standardized sharpening). The sharp corner makes the MTF look better than reality. The absence of a sharp corner may indicate that there is little MTF error.
Clipping can usually be avoided with a correct exposure– neither too dark nor light. A low contrast target is recommended for reducing the likelihood of clipping: it increases exposure latitude and reduces the sensitivity of the MTF results to errors in estimating gamma.
Advantages of the ISO 12233:2014/2017 Edge SFR chart over the old ISO 12233:2000 chart
The ISO 12233:2000 standard and chart, shown on the right, is referenced in the new ISO 122433:2014 standard, but is no longer an official part of the standard. The new standard specifies three charts, one of which is a slanted-edge (Edge SFR or E-SFR) chart with much lower contrast (4:1; shown on the right, below).
Compared to the old ISO 12233:2000 chart,
The E-SFR chart has much less wasted area, especially in the Enhanced and Extended versions (the Enhanced chart is shown on the right, above).
About 90% of the old ISO chart is covered with patterns that have little value for computer analysis.
ISO 12233:2014 Edge SFR chart
from the ISO documentYou can produce a detailed map of sharpness (MTF) over the image surface with the Enhanced and Extended charts. This cannot be done with the old ISO chart because there are not enough suitable edges— and they are not well-located.
Automated region detection, based on location criteria you enter. This makes the E-SFR chart and eSFR ISO module well-suited for automated testing: With the old ISO chart, regions of interest (ROIs) had to be selected carefully whenever the image framing changed, even slightly.
The 4:1 contrast edges are less likely to clip than the edges in the old ISO chart, whose contrast is specified at ≥ 40:1. The camera operates in a more linear region, and hence results are more consistent and accurate— less affected by overexposure, underexposure, incorrect gamma estimate, or excessive software sharpening. 4:1 contrast edges are more representative of real edges that affect perceived image sharpness.
eSFR ISO can measure additional image quality factors, including lateral chromatic aberration, distortion, gamma (contrast), tonal response, and color accuracy (in charts that have the color pattern).
The old ISO 12233:2000 chart contains no information that can be used to measure gamma (the average slope of the Log pixel vs. log exposure curve. i.e., the contrast), which is needed to linearize the image for the MTF calculation. Accurate gamma measurement is particularly important with high contrast targets. In the E-SFR chart, gamma can be derived either from the individual squares (which have a known 4:1 contrast) or from the grayscale step chart.
Compared to the SFRplus chart,
The map of MTF over the image surface is less detailed, though it’s sufficient for most purposes.
Distortion measurements are slightly less detailed (especially for the Standard (minimal) chart) because there are no distortion bars and there are fewer features for measuring distortion.
eSFR ISO contains a highly detailed noise calculation, including all noise measurements from Multicharts and Multitest.
Framing is more flexible since you don’t need white space above and below top and bottom bars. As long as the registration marks are reasonably well inside the image, region detection should work.
Low contrast targets
These test targets have contrast ratios that comply with the ISO standard: