Impact of New Developments of Colour Science on Imaging Technology

Yesterday afternoon, at the Stanford Center for Image Systems Engineering, Dr. Joyce Farrell hosted Prof. M. Ronnier Luo for an update on the latest activities at the International Commission on Illumination (CIE), of which he is the Vice-President. He focussed on the aspects relevant to imaging.

Division 7, terminology, has been disbanded because it has finished its work. The e-ILV can be accessed at this link.

There is a new CIE 2006 physiologically based observer model with XYZ functions transformed from the CIE (2006) LMS functions. These functions are linear transformations of the cone fundamentals of Stockman and Sharpe, the 10º LMS fundamental colour matching functions. In the plot below, you can see the 2º XYZ CMFs transformed from the CIE (2006) LMS cone fundamentals. Note the different shapes around 450 nm compared to the 1931 and 1964 observer models.

The new model is a pipeline in whose stages the age-related parameters can be set. The 10º LMS functions are corrected for the absorption of the ocular media and the macular pigment, and take into account the optical densities of the cone visual pigments, all for a 10° viewing field, yielding the low-density absorbance functions of these pigments. Using these low-density absorbance functions one can derive, taking into account the absorption of the ocular media and the macula, and taking into account the densities of the visual pigments for a 2° viewing field, the 2° cone fundamentals.

There is also a new luminous efficiency function V(λ), which has changed mostly in the blue region.

There are new scales for whiteness and blackness, which corresponds to those in the NCS system. They are based on the comprehensive CAM16 appearance model. Considering a hue leaf of CIELAB in cylindrical coordinates, the south–east ↘ diagonal scale is whiteness–depth and the north–east ↗ is blackness–vividness. These new scales are particularly useful in imaging for adjusting complexion. The skin colors of Asian and Caucasian people vary along the whiteness–depth scale and those of African people vary along the blackness–vividness scale.

Next, Ronnier explained the new color rendering index (CRI) that works also for LED light sources. He also presented a very compelling demonstration of the apparatus used to develop the standard. The new color rendering index is called CRI 2010 and IESNA-TM40. It is based on the measurement of 99 test samples.

I was a little disappointed that the new CRI is still based on colorimetry and not on spectral data. Using colorimetry is an analytical process and having a much larger number of samples helps. However, it does not allow a full characterization of a light source, as we learned many years ago with the tri-band fluorescent lamps. They use less energy, but at the cost of quality.

In this case, I am not too much of a fan of the energy reduction because in practice when you reduce the cost of running a light, people will just deploy more lights and in the end you do not save energy. This is so in consumer applications and does not hold for industrial applications.

Our environment is not made out of BICRA tiles and usually, we are not in aperture mode. We perceive complex images and the light from a set of spot lamps modulates our ambient. While in the case of OLED or fluorescent lamps we might have diffuse light, with LEDs and conventional halogen spot lamps we have more of a set of directed sources with a rapid fall-off.

The rooms in my house are painted in a fusion Italian and Japanese style. The colors are vivid (Italian style), but the paints have a very peaked spectrum so the color is modulated by the illumination (Japanese style). We use older high-quality LED sources with two different green phosphors (the additional one is based on Europium), which we dim. The visual effect is similar to candlelight, except for the correlated color temperature (CCT).

From my experience, I think that a CRI model should include the difference between the spectral distributions of the light source and the reference illuminant. I would also like to have two different reference distributions, A for mood light and D for work light. For thousands of years, we have evolved performing work in daylight and relaxing in blackbody radiator light from fires, oil lamps, and candles. When we want to be in a cozy mood, we pull out the candles, which is also common in upscale restaurants. Candles are more expensive and dangerous than LEDs in houses built from flammable materials.

Should the new CRI also have a provision for the blue hour? Ronnier concluded his presentation stating that the new research topic is tunable white.

Illusion of a dress

Earlier this week I wrote about color not being a physical phenomenon, but rather an illusion taking place in our mind. I also wrote about Hunt's problem of completing a wardrobe. Hunt's example is a motivation for colorimetry. When we can keep constant the illuminants and observers, we can use CIE colorimetry and a color management system to closely match color scenes involving ordinary dyes and pigments.

When we can control but not keep constant the illuminants, then we can still do a pretty good job at matching the appearance of colors in a reproduction by using a color appearance model. "Control the illuminant" means we have to know what it is, as Randall Munroe suggests in his xkcd cartoon on the dress.

When we do not know the illuminant, we can estimate it if there is an object in the scene whose color we know. In the dress picture sparking the Internet on 26 February, there is no reference object, no complexion is visible. In this sense, the xkcd cartoon is not a faithful abstraction of the problem at hand because it shows a lot of skin. We would need a second picture were the lady is not wearing the dress. Actually, a nude by itself is not sufficient and the lady should also hold a calibration target, at least the white side of a gray card.

Back in the late 80s and the 90s, Robert Hunt used to teach a course on color science at the RIT. After the course, Roy Berns used to take out Dr. Hunt for a dinner. One year, he took him to a fancy Italo-American restaurant. On the East coast, the fancier a restaurant was, the darker it was, because the cultural understanding was that for a romantic date people would be willing to pay a premium price, but would want a low light level.

As they entered the restaurant, they noticed that the light-bulbs were red and the whole restaurant was imbued in pink. When they sat down at the table, they felt extremely uncomfortable, because they were not able to decide whether the tablecloth was white or pink. After a long discussion and the desperate search for a reference white, Roy Berns finally remembered he had his business card in the wallet and he knew it was white. This allowed them to enjoy their dinner.

In their honor, we should introduce a so-called Hunt-Berns effect: Inability of the cognitive factor to decide on a set. Example: When in an environment with colored illumination the brightest object is not known a priori to be white, the cognitive part of chromatic adaptation fails because it is not possible to establish whether that object is white or has a hue similar to that of the illuminant. This is especially so, if the observer is knowledgeable about the Helson-Judd effect.

This would take care of the illuminant problem by having a second photograph of the lady, this time in the nude and with a white reference target. However, this would not necessarily explain the effect seen in the photograph.

It is pretty obvious from the photograph, that the dress is not Lambertian, therefore the geometric appearance has also to be measured. We would need a spectrogoniometer rather than a simple colorimetric device like a camera, whose white balancing algorithm can get completely duped when confronted with an unexpected target.

As everybody who ever tried to touch up a dent in a car with metallic paint knows, not all surfaces have a color made with a simple dye or pigment based colorant. If for example the color is based on pearlescence or iridescence, you cannot reproduce it on a photograph displayed on a screen. At the very least you need a movie. In this end, you have to examine the original.

Color reproduction is about reproducing an illusion. It will always be hard.

Completing a wardrobe with Kokko

Robert Hunt likes to start his color science lessons with the problem of completing a wardrobe. He starts with some observations:

• If you want to buy a skirt or a pair of slacks to match a jacket, you cannot match the color by memory — you have to take the jacket with you
• Just matching in the store light is insufficient, you have to match also under the incandescent light in the dressing room and outdoors
• You always get the opinion of your companion or the store clerk

This leads to the three fundamental components of measuring color:

• Light sources
• Samples illuminated by them
• Observers

When we complete our wardrobe, we are not interested at measuring colors, but into matching colors. This may sound easier than measuring color by making measurements and comparing the above three parameters, but it is not. In fact, color is not a physical phenomenon, so we cannot measure it. Color is an illusion that takes place in our mind. What we really have to do is to predict an illusion based on physical measurements, which is very difficult, because we cannot measure our mind.

With this, color science has more to do with art than with physics: color scientists have to develop a deep intuition of color perception, otherwise they are not able to interpret the values delivered by their instruments. This is even more so, when instead of just matching colors we need to assess things like the readability of colored text on colored background, or when we need to create a palette of colors that go well together.

Even such a mundane task as determining the best foundation for one's complexion requires a lot of science and intuition. Cosmetologists can do it almost completely with intuition, but it takes them a long time to develop this intuition. What scientists and engineers can do, is to try to put the cosmetologist or color consultant in a box, viz. into a mobile device.

This is what Nina Bhatti has set out to do with her new company Kokko. Kokko's scientifically developed color matching technology enables brands and retailers to revolutionize ways to shop online—specifically when color selection matters the most.

Kokko's solution for demystifying online purchasing of color cosmetics is called ColorSisters. By using the camera on any smart-phone with the specially printed color chart, Kokko's proprietary software can precisely measure skin tone and offer personalized makeup recommendations—proven to be as accurate as professional makeup artists' recommendations.

Tic-tac-toe patent 8,770,625 in color

As noted on lines 23 and 24 in column 4 of the printed version of patent 8,770,625,
the U.S. Patent Office procedure discourages the use of color drawings. This makes Fig. 4 a little hard to visualize for the non color scientist (there are no color figures in Wyszecki & Stiles), so here it is in color (right pane):

The invention is relatively simple. The general field is anti-counterfeiting as it applies to packaging. Professional counterfeiters have no problem faking ordinary measures like serial numbers and holograms, so the trick is to embed information that cannot easily be perceived by a counterfeiter, hence is omitted in the facsimile. Fortunately color does not exist in nature, it is just an illusion happening in our minds. Therefore, all we have to do is to create an illusion you can only perceive if you expect it.

As described in patent 8,770,625, a number computed from the—possibly counterfeited—serial number on the package can be encoded positionally in a tic-tac-toe grid. The marking is just above the visual threshold, so the naive counterfeiter will reproduce the same pattern on all packages. The trained inspector can then quickly assert whether an actual positional code corresponds, for example, to the possibly fake serial number.

Patent 8,770,625 is relatively short with just three claims, but reducing it to practice is a little tricky, even when all the steps are disclosed in the patent. The difficult part is to design the tool to determine experimentally the visual thresholds for the print process being used and the light conditions under which the inspections are expected to happen. You need to be skilled in the art.

The above figure is a screen-shot of that tool. To implement it you need to write a spectral color management system with CIE colorimetry to simulate the press on the display and vision colorimetry to model what the actual human visual system perceives. The details of the controls are explained in patent 8,770,625.

Depending on your viewing conditions, the above color version of Fig. 4 might be under the visual threshold. If that is the case, in the figure below we crank up the saliency and decrease the background coverage, so you will see the encoding for sure. If you have aliasing problems, you can click on the figures to display them at the original resolution in which they were created eight years ago, early July 2006 (time flies).

Appearance of flamingos reloaded

A few years ago we mused on the color appearance of flamingos:

Appearance Of A Flamingo from Giordano Beretta

Now Daniel B. Thomas, Kevin J. McGraw, Michael W. Butler, Matthew T. Carrano, Odile Madden and Helen F. James have studied the issue in general for plumed animals and more importantly, over time.

They visually surveyed modern birds for carotenoid-consistent plumage colors. They then used high-performance liquid chromatography and Raman spectroscopy to chemically assess the family-level distribution of plumage carotenoids, confirming their presence in 95 of 236 extant bird families. Using their data for all modern birds, they modeled the evolutionary history of carotenoid-consistent plumage colors on recent supertrees. Results support multiple independent origins of carotenoid plumage pigmentation in 13 orders, including six orders without previous reports of plumage carotenoids. Based on time calibrations from the supertree, the number of avian families displaying plumage carotenoids increased throughout the Cenozoic, and most plumage carotenoid originations occurred after the Miocene Epoch (23 Myr). The earliest origination of plumage carotenoids was reconstructed within Passeriformes, during the Palaeocene Epoch (66–56 Myr), and not at the base of crown-lineage birds.

Link to the paper: Ancient origins and multiple appearances of carotenoid-pigmented feathers in birds

Color Blindness Simulator

Color Oracle is a free color blindness simulator for Windows, Mac OS X and Linux. It claims to take the guesswork out of designing for color blindness by showing you in real time what people with common color vision impairments will see.

Related posts:

• The First Time Ever I Heard Your Face
• The Colorful Blind Painter

color vision challenges in electronic imaging

It has been a long summer and autumn with all interrupts disabled. Now that the product is finished and transferred, we can resurface and catch up with life, which is made easier by doing 脱藩 (dappan).

The SPIE just released a new video explaining how teaching cameras how to "see" as humans do is both an art and a science, and Edwin Land spent his career pursuing that goal. Here is the link: http://spie.org/x83130.xml?ArticleID=x83130

Given the recent events here in Palo Alto, the op-ed linked at the bottom of that page is also a good read.

The appearance of a Flamingo

Once upon a time, a day came when the management at Xerox PARC decided to hold elaborate Open Lab events to share our knowledge and achievements in pursuit of synergies. In the color project we had just finished building a research lab, and our director instructed us to better have a good demo in the Gray Lab, justifying is construction.

In fact, we had achieved quite a bit of notoriety, because we had it painted in gray, which was taken as a joke by our colleagues, who expected us building a colorful room. We even had it painted twice, because the first time, when we instructed the painting company to add pure black to white base and nothing else because we needed a spectrally flat color, they thought they were smarter than us and mixed a multitude of pigments to match the gray Munsell Sheet of Color we gave them as the standard.

When they called us upon finishing their job, their boss proudly held the Munsell Sheet against the wall, but we could see immediately that something was fishy, because the wall had a different color where it was hit by the light from the hallway (the lamps in the room were D50 simulators). We simply showed them their spectrum and they had to repaint the lab at their expense.

Other than the instruments and display monitors, the lab was completely bare, as to avoid contaminating the retina during psychophysics experiments. On the side we had also a small room completely painted in black with a spectroradiometer for the measurements. All lamps were D50, so we did not have to wait to adapt our visual system, and could reset it anytime by staring at a wall.

The announcement of the Open Lab event came with a big surprise: all the other team members would be on sabbatical or vacation that week, so I would have to set up the demo all by myself, including dealing with the crowd.

After some reflection, I concluded this was an impossible task, because all the other demos were very high concept. I decided to instead shoot a video in the lab and then just put in the door to the lab a cart with a big TV and a U-matic tape player. The question now was what experiment could I tape to demonstrate the need for a gray lab?

One Sunday I surveyed the offices of my colleagues working in graphics and imaging, in search of an error possibly due to inaccurate color evaluation. Of course each office had pictures of Utah teapots showing off the occupant's algorithms, but I noticed that images of flamingos were quite common. I was amazed all these flamingos were of a vivid pink, unlike the vermilion I remembered from a zoo visit when I was a child.

So I thought I drive to Marine World/Africa U.S.A., which had just moved from Redwood City (now the site of Oracle) to Vallejo, get a flamingo feather, measure it, and achieve a perfectly matching reproduction on our monitors and printers, showing off the importance of chromatic adaptation and cross-device color reproduction.

My plan was to keep a professional Betacam in my office and just opportunistically record material, so I could make up a story at the end depending on what I was able to gather. When I showed the first drafts to my colleagues, they educated me that when Americans think of flamingos, they do not think of the bird at all, but instead they think of pink plastic lawn flamingos.

Well, so much for a naive boy from the Alps. There was not enough time for a different demo, so I stared at my hours of video sequences and made up this movie:

Appearance Of A Flamingo

View more videos from Giordano Beretta

[If there is a problem with the above stream or you have a slow connection, you can download the movie from this link. If you stream from this link, there will be a buffering delay due to the slow connection.]

Unfortunately, the original U-matic cassette is no longer available, and my VHS copy is all gummed up. Unlike U-matic, VHS does not have SMPTE time code and the signal bandwidth is very narrow, so it took me 2 months of conditioning the tape and attempting replays, until I got most of the frames.

I would have loved to have Peter Schnorf's digital video editor having lost video frame protection, because I would just have run the digitization process a few times and the system would assemble the complete video.

I used a semiprofessional VHS player and first split the signal in separate luma and chroma components. I then adjusted each signal to fill its gamut and after analog-to-digital conversion denoised each signal. Since the signal is pretty bad, I did not try to do any enhancements, as they would amplify the defects: I simply transformed the digital video stream into MPEG-4 using Quicktime.

In the VHS device gamut of the YIQ color space, very little of the bandwidth is allocated to the magenta region, therefore the flamingos look terrible in the movie, washed out and like followed by a ghost.

An now a lame flamingo joke: Why do flamingos stand on one leg?

If they would lift also the other leg, they would fall over.

"The First Pass is Relatively Arbitrarily Picked Colors"

From the summer toPost folder is a hurl from Tim with a link to a Chuck Close interview on Colbert:





Which regardless of your opinion of Colbert, is a remarkable interview. First, Close gets Colbert to say toner. Second, Close describes paintings as "colored dirt on a flat surface". Third, Close checks his hand before revealing he suffers from prosopagnosia.

Mixing powders

A big part of job security for color scientists is that the mixture of colorants is very difficult to predict. This is especially so in color printing with CMYK halftones. It is surprising to learn that this is not so for mixed powders: their color appearance turns out to be easy to predict.

Dealing with Red Lips

An anonymous reader is asking in a comment on the fuchsia post:

My daughter has asked me to paint the wall in her room this color:

http://img710.imageshack.us/img710/6656/img2676l.jpg

The color name on the paint can is "Hot Lips" but that is hardly appropriate. Can you tell me if this is Magenta or Fuchsia?

Here is how you can answer this kind of questions:

To give a "scientific" answer, you would have to measure your daughter's wall with a spectroradiometer and compare it to a measurement of the fuchsine colorant. If the two spectral curves are within a tolerance, you can say the color terms are synonyms. Unfortunately, this is not practical.

Since I happen to have a little FileMaker script on my PC, I can give you an "engineering" answer based on crowd-sourcing. Your photograph is not uniformly illuminated and you probably did not calibrate your camera and turn off the white balancing, so I just average a few pixels in the center of your picture. For the reference colors, I use Nathan's color thesaurus tool and query fuchsia and magenta, and just for the sake of it, I also query hotlips. Hotlips is not in the thesaurus, but it suggests lips, so let us take that. This gives me the following CIELAB values:

CIELAB values

	L*	a*	b*
hot lips (wall picture)	50	56	11
lips (thesaurus)	50	56	20
magenta (thesaurus)	56	46	-21
fuchsia (thesaurus)	60	45	-24

The FileMaker script yields the following CIE94 ∆E matrix:

CIE94 ∆E

	hot lips	lips	magenta	fuchsia
hot lips	0	4.72	18.8	21.98
lips	4.72	0	22.74	25.69
magenta	18.8	22.74	0	4.38
fuchsia	21.98	25.69	4.38	0

From the last post we know that fuchsia and magenta are synonyms. Therefore, the tolerance for crowd sourcing is a little over 4 for these colors. From a crowd-sourcing perspective, you could not call the wall color in your daughter's room fuchsia nor magenta, but people have a pretty good idea the walls are lips colored even if you do not tell them.

So far we have discussed the "scientific" answer and the "engineering" answer, but actually you want a practical answer, i.e., a color name that you think would be more appropriate for your daughter's room walls. It turns out, that this is a very easy problem.

The solution is to turn to Nathan's color thesaurus, which contains the wisdom of crowds: http://www.mostlycolor.ch/2007/10/on-line-color-thesaurus.html. Just enter hotlips and click submit: this will give you four synonyms for lips and you just pick the color term you believe is most appropriate for your daughter.

So, people will know what you mean if you say coral red, carmine, scarlet, or punch.

At this point your daughter might insist to call her wall color as it is labeled on the paint can. If you are willing to spend a few dollars more, you can actually make everybody happy with the "marketing" answer. Many paint manufacturers also have an upscale brand in the name of a famous designer. The can will have the same paint, but the label on the paint can will be some fashionable fancy name evoking something exclusive. This would give you a posh name and your daughter the color she wants.

I can imagine many parent having this kind of problem, so we would be very interested in learning how you ended up solving your problem. Please leave us a comment below!

A neat example of color break up (blue-yellow)

Color break up happens for example when color components of an image to be displayed are presented in a time sequential manner. How would you demonstrate 'color break up' in a coltrolled way? Micheal Bach published a visual illusion in which the observer is asked to focus on a particular point in the image and moves the test object. In addition, once in a while the test disk changes color for an instant and you can watch what happens (just click on this link: http://www.michaelbach.de/ot/mot_flashlag1/index.html):

Testing Colours to Enhance Web Readability

According to the University of Washington’s Department of Ophthalmology, 2.8 million Americans are colour blind, which can express itself in many variations and degrees of severity. Colour perception problems are important considerations when developing web sites to ensure that all users have access to the content and the functionality of site.

Based on the contrast ratio algorithm, Level AA conformance requires text to have a contrast ratio of at least 4.5:1; larger text (18 point font or larger, or 14 point or larger if bold) a contrast ratio of at least 3:1. For Level AAA conformance, text requires a contrast ratio of at least 7:1; larger text a contrast ratio of at least 4.5:1. Incidental text or images of text that are not part of the user interface or are purely decorative and text that is part of a logo or brand are excluded from the colour contrast requirement

DIY conformance testing tools are available for Windoze and Mac OS X.

[Source: dotgov.com]

World Wide Gamma

This is an experiment to estimate an average gamma or display non-linearity for the World Wide Web.

To participate, use the tool shown below to create an equal lightness step ramp. To create this ramp use the '+' and '-' buttons to lighten or darken the corresponding patches above the buttons. A black patch is shown furthest to the left, a white patch is shown furthest to the right and initially the intermediate patches are shown with random lightnesses.

When you are satisfied with your equal lightness step ramp press the 'plot' button to submit your data and view a plot of your results (black dots) versus the current average (red dots). You will also be able to submit optional feedback about your display and viewing conditions.

Thank you for your participation.



Note: The above background for the ramp consists of relatively coarse black and white squares in order to anchor the ramp to an approximate middle gray without using a solid gray, which would vary by display.

Is green a composite hue, modulated by blue-yellow opponent cells?

In the end of 2009, I had the opportunity to re-visit current models of human visual perception investigating color perception as a function of spatial frequency. Based on my engineering background, a simple sinusoidal sweep pattern modulated by different opponent colors appeared to be appropriate to examine some interesting observations.

Figure 1. 4 sinusoidal sweep patterns showing spatial bandwidth of cone bipolar cells; (leftmost) black-white demonstrating highest bandwidth without hue changes (achromatic), (center left) blue-yellow demonstrating lowest bandwidth with hue shift toward black-white, (rightmost) red-cyan demonstrating medium bandwidth with hue shift toward black-white, and (center right) magenta-green demonstrating bandwidth similar to blue-yellow with hue shift toward red-cyan.

Dynamic Iridescence and other Colors Aquatic

From the September-backlog pile is an item about dynamic on-off iridescence in the squid loligo opalescens. Apparently the squid's reflectin proteins are modulated by acetylcholine, a neurotransmitter. Which is quite remarkable, even without considering the applications to camouflage and perhaps communication.

And then one discovers there are over half a dozen octopus camouflage videos out there on the web.



And like potato chips you can't stop with just one.

The Color of Grasped Porcine Liver

From the medical imaging side of color comes the paper "CIELAB and sRGB color values of in vivo normal and grasped porcine liver" by Smita De and co-authors.



The above is actually the in vivo normal porcine liver color.

Journal of Vision Call for Papers: Perception of surface color and material properties

The Journal Vision has sent out a call for papers for a special issue on the perception of surface color and material properties. Guest editors David Brainard and Larry Maloney describe the intent of the issue as follows:

"Researchers have devoted much effort to understanding the perception of color and lightness for simple stimulus configurations, often consisting of flat matte surfaces rendered under diffuse illuminations, or simulations thereof. The objects we look at in daily viewing, on the other hand, are rarely flat, matte, or diffusely illuminated. There is now considerable interest in pushing our understanding into the realm of more complex, three-dimensional scenes, spurred in part by advances in computer graphics that allow physically accurate rendering of a variety of materials and thus permit exploration of interactions between object shape and orientation, object material, and illumination geometry. The Journal of Vision plans a Special Issue to bring together papers that describe recent advances in this area."

The deadline for submissions is March 15, 2010. For more details about this special issue see the call for papers: here.

Drinking a Reward

In a mere 20 weeks previously color blind monkeys appear to have become trichromats through a subretinal injection of a virus containing L-opsin (1).

Nominal Scaling of Print Substrates

Towards the end of Giordano's AIC presentation is a preview of our next paper "Nominal Scaling of Print Substrates".

This paper has recently issued as a technical report.

One way to look at this paper is what to do when you have 100 different types of commercial print media on the floor of your cube. They vary by color, thickness, surface texture, opacity and more. They come from multiple vendors. They have multiple branded names and descriptors.

How can this pile be put into some kind of order?