Detecting traffic lights with RCCB sensors

Owned by Henry Koren
Jun 30, 2023
2 min read

RCCB (Red-Clear-Clear-Blue) sensors are widely used in the automotive industry because their sensitivity and Signal-to-Noise Ratio (SNR) is better than conventional Bayer (RGGB) sensors. But the improved sensitivity comes at a price: reduced color discrimination, which can make it difficult to distinguish traffic light colors.

We have seen an image where the red and yellow colors were indistinguishable. We have determined that the cause was saturation in the Red channel. We don’t know whether this happened during image capture or image processing: it might have been caused by a Color Correction Matrix (CCM) that attempted to replicate normal RGB colors.

We present a simple set of equations for optimal discrimination between Red, Yellow, and Green lights.

Nomenclature and assumptions

Y is for Yellow, rather than the usual Luminance channel.

C stands for Clear (R+G+B), rather than Cyan.

We assume that the channel has been white balanced (the R and B channels multiplied by a coefficient) so that R = G = B is neutral gray or white.

For the equations to be valid, none of the channels (R, C, or B) can be saturated — either during image capture or after image processing. This was not the case in the image we saw where red and yellow were indistinguishable. The red channel was saturated, which caused it to be weaker than expected in relation to the green channel, leading to the detection failure.

Color channel equations

The Red (R) and Blue (B) channels are derived directly from the image sensor, likely with a multiplier (coefficient) for white balance.

Green (G) and Yellow (Y) are derived from equations:

G = C – R – B

Y = R + G = R + C – R – B = C – B

Single color (detection) channel equations

Generally, a single-color channel (i.e., a detectable color) is the color value minus the values of the other channels. Here are the equations for the three channels needed for traffic light detection.

Rdet = R – G – B = R – C + R + B = 2×R – C

Gdet = G – R – B = C – 2×R – 2×B

Ydet = G+R-B – |G-R| = C – R – B + R – B – |C-R-B-R| = C-2×B – |C-B-2×R|

The detected color is the one from the largest value of {Rdet, Gdet, Bdet}.

Some caveats

  • These equations have not been checked by my (NLK’s) colleagues and they have not been tested experimentally.

  • They do not work if any of the channels are saturated. Avoiding saturation can be difficult with standard sensors where the light is much brighter than the surrounding scene.

  • If a realistic result is needed (typically for human vision), it should be passed through a separate channel (often with a CCM applied).