Fast RGB to Spectrum Conversion for Reflectances

$\begin{comment} <a href="https://arxiv.org/abs/1710.05732" target="_blank" rel="noopener noreferrer"><img class="alignright size-full wp-image-1801" src="http://scottburns.us/wp-content/uploads/2015/05/PDF-logo.jpg" alt="PDF logo" width="32" height="32" /></a> \end{comment}$

Published September 26, 2018 by Scott Allen Burns, last updated January 27, 2020

Change Log

9/26/18 Initial publication.
9/27/18 To avoid confusion with established colorimetric terminology, the word “primaries” has been changed to “components.”
9/28/18 Added section of computational speed.
9/28/18 Added section on comparing RGBC to measured reflectances of Munsell color chips.
9/30/18 Added the phrase “gamma correction” to “companding” since it is a more commonly used term.
10/3/18 Added section showing WGM blending of RGBC and ILLSS side-by-side.
7/28/19 Added note saying that clicking on figures enlarges them.
12/21/19 Added section on producing the RGB component curves using the hyperbolic tangent method.
1/27/20 Added 1nm version and CIE 2016 CMFs version.
8/20/24 Added links to past work and clarified some wording.

Introduction

In a previous presentation on generating reflectance curves from sRGB triplets, I presented several algorithms for creating a reasonably realistic spectral reflectance curve associated with a given sRGB color. These algorithms require considerable computational effort, which limit their applicability in computationally intensive tasks such as computer graphics rendering via ray-tracing.

I was approached by an interested reader, Brien Dieterle, who suggested that the computational burden could be lessened by using these algorithms to generate just three reflectance curves, associated with sRGB values of [255, 0 0], [0, 255, 0], and [0, 0, 255], and using them to create additional reflectances for other colors by simply computing a weighted sum of these three reflectance curves.

The three reflectance curves produced by the ILLSS algorithm (see ref) corresponding to the sRGB values of (255, 0, 0), (0, 255, 0), and (0, 0, 255), respectively, have the shapes below:

To find a reflectance curve for an arbitrary sRGB value, Brien’s method first converts the sRGB to linear rgb (by removing the companding/gamma correction), giving the three rgb values in the range 0 to 1. Then the three component reflectance curves above are multiplied by the corresponding r, g, and b values, and then summed together. For example, for sRGB = (125, 150, 100) the linear rgb is (0.2051, 0.3050, 0.1274), and the weighted sum of the three components becomes:

It is important to use linear rgb instead of the companded (gamma corrected) sRGB because linear rgb preserves the mathematical property of additivity, and consequently, the rgb values of the weighted sum will match the original rgb values.

sRGB to rgb conversion

The process of removing the companding (gamma correction) from sRGB to give rgb is:

sRGB=sRGB/255; % convert 0-255 range to 0-1 range

for i=1:3

  if sRGB(i)<0.04045

    rgb(i)=sRGB(i)/12.92;

  else

    rgb(i)=((sRGB(i)+0.055)/1.055)^2.4;

  end

end

To convert rgb back to sRGB:
for i=1:3 if rgb(i)<0.0031308 sRGB(i)=12.92*rgb(i); else sRGB(i)=1.055*rgb(i)^(1/2.4)-0.055; end end sRGB=round(255*sRGB); % convert to 0-255

One downside to the weighted sum of components approach is that the resulting reflectance curve is quite jagged. But as Brien pointed out to me, the more worrisome aspect is that the weighted sum might exceed a value of 1 in some regions. In fact, when applying it to sRGB = (255, 255, 255), which is rgb = (1, 1, 1), the weighted sum is simply the sum of the three components:

Large portions of the composite reflectance curve have values > 1, which can be problematic in many applications.

This brings us to the goal of this presentation. Brien asked me if I could think of some way to design three component spectral distributions so that their weighted sum would never exceed a value of 1. He suggested that perhaps some modification of the optimization approach I used in the previous work could be applied here as well. I thought it sounded like a fun challenge, and I managed to find a solution, as presented in the next section.

Three Optimal RGB Components

Recall that the ILLSS (Iterative Log Least Slope Squared) method is based on the solution of this nonlinear program:

$\begin{split}\mathsf{minimize}\;\;&\sum_{i=1}^{35} (z_{i+1} - z_i)^2 \\ \mathsf{s.t.}\;\;&T e^z = rgb \\ &z \le 0,\end{split}$

where $z=\ln(\rho)$ and $rgb$ is the target linear rgb triplet. The reflectance vector, $\rho$ , has 36 elements, representing reflectance values for wavelengths 380 nm to 730 nm in 10 nm intervals. (Note: the choice of 36 is arbitrary and an example with 1 nm intervals is shown later.) The objective function is a discretized version of the integral of the square of the the slope of $\ln(\rho)$ . The first constraint set enforces that $\rho$ has a corresponding rgb value matching the target color. $T$ is a 3×36 matrix that produces linear rgb (D65 weighted, and not gamma corrected) when premultiplying $\rho$ . The second constraint set enforces $\rho\le$ 1 (note $\ln(1)=0$ ). This nonlinear program has 36 variables and 39 constraints.

To adapt it to find three optimal RGB components, the set of variables is expanded to three $\rho$ vectors, one for each component, named $\rho^r$ , $\rho^g$ , and $\rho^b$ . The corresponding log of $\rho$ values are called $z^r$ , $z^g$ , $z^b$ . The objective function is expanded to minimize the sum of the three log-slope-squared sums. The constraint set is expanded to make sure $z^r$ , $z^g$ , and $z^b$ have linear rgb values of (1,0,0), (0,1,0), and (0,0,1), respectively, and that the sum of the three $\rho$ values is always $\le$ 1. This gives rise to a nonlinear program with 108 variables and 45 constraints:

$\begin{split}\mathsf{minimize}\;\;&\sum_{i=1}^{35} (z^r_{i+1} - z^r_i)^2 + (z^g_{i+1} - z^g_i)^2 + (z^b_{i+1} - z^b_i)^2 \\ \mathsf{s.t.}\;\;&T e^{z^r} = \left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right], \\ &T e^{z^g} = \left[ \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right], \\ &T e^{z^b} = \left[ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right], \\ &e^{z^r} + e^{z^g} + e^{z^b} \le 1. \end{split}$

In the previous work, the ILLSS method relied on an iterative scheme to identify which of the inequality constraints were active at the solution, and subsequently converted to equality constraints. This was done to make the method more computationally tractable, since an entire optimization must be performed for each reflectance curve being created. In this case, the nonlinear program needs to be solved only once, and the three solution vectors ( $\rho^r$ , $\rho^g$ , and $\rho^b$ ) are then used to generate reflectance curves after that. So it is reasonable to pull out the “heavy artillery” to solve the nonlinear program, namely, a general purpose optimization code such as Matlab’s fmincon, or Excel’s Solver add-in.

Here is a Matlab program that solves this optimization. The three solution vectors, rho_r, rho_G, and rho_B, are available here, and are plotted below:

The sum of the three components is also plotted as a gray line, and turns out to be equal to 1 everywhere.

In summary, the RGB Components (RGBC) method is:
1) Given a particular sRGB triplet, convert it to linear rgb, where r, g, and b are each in the range 0 to 1 (if you are already working in a linear rgb environment, you can skip this step). See above for code on how to perform this conversion.
2) Compute the sum of three weighted vectors, rho = r*rho_R + g*rho_G + b*rho_B.
3) Done!

Consider the example presented above, sRGB = (125, 150, 100) and linear rgb = (0.2051, 0.3050, 0.1274). The plot below shows the reflectance obtained from the optimal RGB Components method and from the ILLSS method.

The two compare very favorably. As expected, the ILLSS curve is smoother in the sense of less slope squared, but the RGB components curve is also quite smooth. The next sections will look more in depth at comparing the RGB components method to other methods.

Computational Efficiency Comparison

The RGB Components (RGBC) method requires only the scaling and addition of three 36-element vectors. This is the same effort as is required for the simple matrix multiplication involved in the LLS (Linear Least Squares) and LSS (Least Slope Squared) methods of the previous presentation. The RGBC method guarantees that the reflectance curve will fall within the 0 to 1 range, a claim neither LLS nor LSS can make. It is also roughly 200 times faster than ILLSS. The only other previous method that guarantees the 0 to 1 result is ILSS (Iterative Least Slope Squared), which is about 10 times slower than RGBC.

Comparison to Munsell Color Spectra

This section compares the reflectance curves generated by RGBC to the curves measured spectrophotometrically from all 1296 sRGB-in-gamut 2007 Munsell glossy edition color chips. See the previous publication for more information on this data set. A similarity measure was established, called $RMM$ ,

$RMM = \sum_{i=380}^{730} lum_i \;|(\rho_i^{measured} - \rho_i^{computed})|,$

which weights the difference in reflectance curves by the relative sensitivity of the eye to various wavelengths (i.e., via the luminosity curve).

Name	Max $RMM$	Mean $RMM$
RGBC, RGB Components method	1.39	0.27
ILSS, Iterative Least Slope Squared	1.04	0.16
ILLSS, Iterative Least Log Slope Squared	0.86	0.15

On average, the RGBC results have about twice the deviation from the Munsell measurements as the ILLSS method. Here is a side-by-side comparison of all 1296 $RMM$ values, where black is mapped to 1.39 (the largest deviation) and white is mapped to zero (click figures to enlarge):

Generally speaking, RGBC has the largest deviations from the Munsell measurements in the saturated yellow and blue regions. The two largest deviations are for Munsell colors 2.5B 7/8 and 7.5Y 7/12:

The tendency for RGBC to generate broad waveforms actually works to its advantage, in comparison to ILLSS, which tends to have too narrow of peaks in the yellow range of colors:

It is not too surprising that the RGBC results are not as good as other optimization-based methods. There are only three degrees of freedom in the RGBC method (the three weighting factors on the rho_R, rho_G, and rho_B), whereas ILSS and ILLSS have more degrees of freedom available to shape the curves.

Subtractive Mixture Comparison

In this section, I compare the subtractive blending using the weighted geometric mean (WGM), as described here. ILLSS is on the left and RGBC is on the right. In each plot, two sRGB colors are blended together, and those blends are also tinted to white and shaded to black (actually a reflectance of 0.02 is used for black, as zero is not an appropriate value for WGM mixture). Click on each figure to see an enlarged version.

Overall, the comparison is very positive. There does appear to be a little hue shifting (Abney effect?) in the blue to white blends in the RGBC method that isn’t present in the ILLSS blends. And there seems to be the most difference between the two methods in the central region of the green-magenta blend, but it is still minor.

Acknowledgments

Thanks to Brien Dieterle for coming up with the idea of using ILLSS-generated RGB components to greatly increase the computational efficiency of reflectance curve generation, and for asking me to try to compute the optimal components. I believe this collaboration will make the application of least-slope-squared type algorithms more useful in certain computer graphics applications.

________________________________________

Update 12/21/2019: Hyperbolic Tangent Method

I recently developed a better method than ILLSS for reflectance reconstruction using a hyperbolic tangent transformation (Burns SA. Numerical methods for smoothest reflectance reconstruction. Color Res Appl. 2020;45(1):8-21 and also on this web page, where it is called the LHTSS method.)

It produces reflectance curves strictly between 0 and 1, without the sharp discontinuities at 0 or 1 that can happen with ILLSS. It is also much faster in terms of computation. I was contacted by “Mark” (member “kram1032” of a Blender discussion page), who asked if I had tried using the hyperbolic tangent method with this RGB Components method. I hadn’t, but soon did!

Recall from above that the three reflectance curves produced by the ILLSS algorithm corresponding to the sRGB values of (255, 0, 0), (0, 255, 0), and (0, 0, 255), respectively, don’t sum to 1. Below is a plot of the sum of the three curves and a sum of the three curves produces by the hyperbolic tangent method. The ILLSS curves are blue and the hyperbolic tangent method curves are orange:

The three curves sum to 1 somewhat better, but still not all the way, so I re-ran the optimization above using the hyperbolic tangent transformation. Both the original component curves and this new set are plotted below; they turned out to be nearly identical.

Bottom line: use the three RGB Components curves originally presented above. Thanks for the insightful comments, Mark!

________________________________________

Update 1/27/2020: Higher Resolution Version and CIE 2016 CMFs

I was asked to generate the RGB components for a higher resolution set of color matching functions (with 1nm spacing instead of 10nm). I obtained a 1nm version of the CIE 1931 CMFs and illuminant D65 from here. The span of wavelengths is increased, from 380nm-730nm to 360nm-830nm. The resulting RGB components obtained from the optimization are available here in CSV format, and are also plotted below:

I was also curious to experiment with other CMFs, or standard observers. The sRGB specs call for the CIE 1931 2-degree color matching functions. More recent CMFs have been proposed, such as the 2012 CIE “physiologically-relevant” version transformed from the CIE 2006 cone fundamentals. The 1931 CMFs are known to underestimate human sensitivity at the blue end of the spectrum, and the 2006 fundamentals address this. In creating the 2012 version of the RGB components, I had to reconstruct the T matrix that relates reflectance to RGB, this time with the new CMFs. (Note that we are no longer adhering to the formal sRGB standard (or the Rec BT.709 specification), since the standard specifically requires the 1931 CMFs to be used.) The new RGB components are available here in CSV format, and are also plotted below in comparison to the 1931 version:

________________________________________

Fast RGB to Spectrum Conversion for Reflectances by Scott Allen Burns is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

________________________________________