Extreme color naming experiment finds locus for luma–chroma transformation

From time to time, a new physiological experimentation technique or a significant new instrument is developed, leading to breakthrough discoveries. In the case of color vision, this usually entails a doctoral student and their significant other—and maybe some additional dedicated colleagues or the professor—to undergo the cruel ordeal of having their pupils dilated (mydriasis) and their ciliary muscle paralyzed to avoid accommodation (cycloplegia), then get strapped to a headrest while biting a dental impression mount, to make observations in repeated interminable sessions for months on end, all in the name of science.

In their recent paper The elementary representation of spatial and color vision in the human retina, Ramkumar Sabesan et al. report on a seminal study to locate where in the human visual system (HVS), the luma-chroma encoding occurs in the parvocellular pathway (midget ganglion cells).

This study by Ramkumar Sabesan et al. represents the first time cone photoreceptors of known spectral type have been individually targeted and activated with light in the living human retina.

I cannot believe, it has been thirty years since I drew this diagram:

Although the above diagram looks like a model for the HVS, it was more a plan for my implementation of a color workbench. To keep the head cool and prevent it from overheating, our brain evolved to minimize the usage of energy. This is accomplished by having pipelines where at each stage the information gets recoded to make it more complex but more compact. This is at the cost of speed: while a two-photon catch the shift in electron density takes less than a femtosecond, the entire photo-cycle lasts a picosecond and at the end of the pipeline, adaptation can take seconds and color naming minutes.

An important feature were the bidirectional arrows: we have a feedback loop with control moving down and information moving up. Because of the sequence of recoding and the feedback, the receptors in the retina are not like pixels in a CCD sensor

• Receptive field: area of visual field that activates a retinal ganglion (H.K. Hartline, 1938)
• Center-surround fields allow for adaptive coding (transmit contrast instead of absolute values)
• Horizontal cells presumed to inhibit either its bipolar cell or the receptors: opponent response in red–green and yellow–blue potentials (G. Svaetichin, 1956)
• Retinal ganglion can be tonic or phasic: pathway may also be organized by information density or bandwidth

The last item comes from a table of the parvocellular and magnocellular pathways Lucia Rositani-Ronchi compiled for me at the 1993 AIC meeting in Budapest:

	P–	M–
Originating retinal ganglion cells	Tonic	Phasic
Temporal resolution	Slow (sustained responses, low conduction velocity)	Fast (mostly transient responses, some sustained, high conduction velocity)
Modulation dominance	Chromatic	Luminance
	Adaptation occurs at high frequencies	Adaptation occurs at all frequencies
Color	Receives mostly opponent type input from cones sensitive to short and long wavelengths	Receives mostly combined (broadband) input from M and L cones, both from the center and from the surround of receptive fields
Contrast sensitivity	Low (threshold > 10%)	High (threshold < 2%)
LGN cell saturation	Linear up to about 64% contrast	At 10%
Spatial resolution	High (small cells)	Low (large cells)
Spatio-temporal resolution	When fixation is strictly foveal, extraction of high spatial frequency information (test gratings), reflecting small color receptive fields	Responds to flicker
	Long integration time	Short integration time
Relation to channels	Could be a site for both a lightness channel as for opponent-color channels. The role depends on the spatio-temporal content of the target used in the experiment	Might be a site for achromatic channels because the spectral sensitivity is similar to Vλ, it is more sensitive to flicker, and has only a weak opponent color component
Possible main role in the visual system	Sustain the perception of color, texture, shape, and fine stereopsis	Sustain the detection of movement, depth, and flicker; reading of text

We have four retinal pigments (erythrolabe, chlorolabe, cyanolabe, rhodopsin) attached by a lysine to a protein backbone. These four pigments are sensitized to photons at 4 energy levels (wavelengths): L, M, S, and rods. The energy levels are not numbers but distributions, namely the probabilities for a photon catch with that chromatophore.

A 3-dimensional signal with L, M, S is not efficient because we need a high spatial resolution but the chromatic information can be at a lower resolution. This is reflected in the modulations transfer functions for the HVS and is exploited for example in image encoding, where we transform an RGB signal into a color opponent signal and then down-sample the chroma images:

In 1993, it was not known where this transformation occurs in the HVS. In fact, there is quite a bit of processing in the retina, and many details are still unknown.

In their recent paper The elementary representation of spatial and color vision in the human retina, Ramkumar Sabesan et al. report on a seminal study to locate where in the HVS, the luma-chroma encoding occurs in the parvocellular pathway.

Using an adaptive optics scanning laser ophthalmoscope (AO-SLO), the authors studied 174 L-cone, 99 M-cone, and 12 S-cone samples by stimulating them individually with a 543 nm, 500 ms pulse and asking two subjects to report the perceived color name. The names were restricted to red, green, blue, yellow, white, and not seen.

The subjects reported achromatic sensations 61.8% of the time. When red was reported (22.5% of seen trials), it was more likely to be driven by L- than M-cones, whereas green (15.7%) was more likely to come from the excitation of M-cones. Thus, L-cones tended to signal both white and red, whereas M-cones tended to signal both white and green. The observation that these color percepts roughly align with the predictions of large-field cone-isolating stimuli suggests that the same opponent neuronal circuits may be implicated in both paradigms. This finding also supports the idea that the visual system can learn the spectral identity of individual cones.

The apparent segregation of color categories into distinct populations of cells is suggestive of a parallel representation of color and achromatic sensations. Moreover, these results imply that, for a large number of cones, their individual activation is not sufficient to produce a color. (Remember that in this experiment single cones are excited; in free vision, most cones are activated and the eye saccades, presenting a point in the visual field to several cones.)

The authors found that the cones most likely to generate strong spectral opponency in a parvocellular neuron, that is, those surrounded by cones of opposing type, were not more likely to generate red or green percepts. Rather, all these examples, when stimulated in isolation, drove achromatic percepts on a majority of the trials.

There is little doubt that the long-duration supra-threshold stimulation of individual cones here influences the firing of a number of different ganglion cell types. In particular, a multi-electrode study demonstrated that the activation of a single cone simultaneously evoked responses in both midget (parvocellular) and parasol (magnocellular) ganglion cells. The results may be particularly informative in differentiating proposals about the role of parvocellular neurons in achromatic spatial and color vision.

The study confirms the old result that the red-green system samples the visual world at a lower resolution than the achromatic system. The new results from the studies reported in the present paper are consistent with the idea that the HVS represents these two pieces of information with separate pathways that emerge as early as the photoreceptor synapse: one chiefly concerned with high-resolution achromatic vision and a second, lower-resolution color system.

The luma-chroma transformation with chroma subsampling is very important in image processing. In your opinion, does this new result allow the design of better imaging pipelines? Does this allow us to design better retinex algorithms? Join the conversation in the Trellis group.

Citation: R. Sabesan, B. P. Schmidt, W. S. Tuten, A. Roorda, The elementary representation of spatial and color vision in the human retina. Sci. Adv. 2, e1600797 (2016).

藍 versus 青 compared to 綠

Jingyi Gao has completed the dissertation: "Basic Color Terms in Chinese: Studies after the Evolutionary Theory of Basic Color Terms" which has as in its abstract the following summary:

The present dissertation researches basic color terms in seven historical and three contemporary lects (i.e., language varieties) of the Sinitic (i.e., Chinese) language family (commonly understood as Chinese languages in the West) with reference to the two main conceptions of the evolutionary theory of basic color terms: (1) The evolutionary trajectories of basic color terms (Berlin & Kay 1969: 2–3; Kay et al. 2009: 10–11, 30ff.); and (2) the composite color categories (Kay 1975; Kay et al. 1991: 15). The present studies of this dissertation has two main parts: (1) A philological portion on two themes (1.1) the basic color terms for black, white and red in Chinese lects; and (1.2) the official colors of Chinese regimes; (2) An experimental portion on the basic color terms in Mandarin Chinese.

Color naming 65,274,705,768 pixels

Where slide 6 is an approximate visualization of the spatial extent of the data analyzed.

Color naming 65,274,705,768 pixels from nmoroney

The First Time Ever I Heard Your Face

Neil Harbisson is a color-blind (achromatopsia) artist from Spain who has a cybernetic "third eye" attached to his head that enables him to hear color. This also makes him a genuine cyborg. Since 2005, Harbisson has used his sonochromatic retina to design sound portraits of famous people. The first person he experimented on was the immediate heir to the British throne, HRH The Prince of Wales.

He scans the eyes, hair and skin color of his subjects and works the individual notes into a musical chord. Harbisson explained, “Prince Charles noticed my electronic eye so, I politely asked him if I could listen to his face.” Apparently Prince Charles' lips sounded like a high E [a blue blood? --njg]. But his hair was almost inaudible—musically thin due to male-pattern baldness.

Some notable quotes from a recent BBC Outlook interview:

• Color as words are "really, really, really, impossible to understand"
• "Color has a frequency that we can't hear"
• "Red is between F and F^#" [? I would've thought it was A/A^#. --njg]
• "I hear the color through my bones, not through my ears"
• The BBC interviewer's voice is "between orange and red"
• Neil's fave "color" is Aubergine, which has a high pitched tone
• "White is silent" [? I would've thought that was black. --njg]

And finally, here is Neil with his Eyeborg on daily maneuvers (video).

Related post: The Colorful Blind Painter

Color My Mother Chartreuse

When I was a graduate student in physics, a friend of mine (who was more into art and poetry) invited me over to dinner at his place. While his wife was busy in the kitchen, we were chatting near a window in the late afternoon sunlight. Suddenly, he pointed at a bowl of fruit on a table near the window and asked me why the shadow it cast was colored purple and not black. Looking back on it, I think it was a test—that art vs. science thing. Nonetheless, I was in a profoundly philosophical mood and immediately rejoined: "It's an optical illusion. What's for dinner?" It would be another decade before I would even begin to realize how much I did not understand about color. I had no inkling then that vision and color perception are computational processes, that the brain is a differential analyzer and computed differences carry relative errors (perceptual illusions).

More recently, in a faint reprise of that illuminating but ancient episode, I was amazed to learn from Nathan and Giordano that the naming of colors is a similarly profound process, and since they are both experts, I could not be as dismissive with them as I was with my arty friend. On the contrary, I was forced to reflect on how I personally came to learn certain color names. In that landscape, my mother cuts a dominant figure.

Games of Color Concentration

Please try the game below. There are eleven variations possible, one of which is randomly selected when the page or experiment is loaded. More details on the game to follow. Suffice to say these are games with hypotheses. Thanks and best.

The Business of Color and Bose-Einstein Condensation

Surprised to hear this piece about Pantone on NPR this morning.

"One of the most influential committees is a group of 10 people whose names are a secret. They meet in Europe twice a year (May and November) at the invitation of Pantone, a company based in Carlstadt, N.J., whose only business is color. In fact, Pantone has a hand in the color of roughly half of all garments sold in the U.S."

"Why would any designer want to run with the pack? John Crocco, the creative director for Perry Ellis, calls color forecasts 'a self-fulfilling prophecy.' He says if designers choose to follow such forecasts, then they'll be 'part of what ultimately becomes the trend.' But if designers disregard the trend, they risk irrelevance — just about the worst thing imaginable for any label."

All of which begs the obvious question: Is this is an example of Bose-Einstein condensation (winner takes all) in a scale-free marketplace?

Fun with Collocates: Orange Vodka to Blue Skies

The SCOTS team at Glasgow University has posted a handy tool the BMC ComPair for visualizing collocates in the British National Corpus. Here's an example for orange and blue, also shown below.

The instructions clearly state: "Collocates of both words are shown, together with your search words. The collocates near each extremity have a strong collocational strength with that search word, collocates in the middle are used equally with both your words."



So vodka is more likely to be used with orange, while skies is more likely to be used with blue and light is roughly equally used by both orange and blue. Interestingly, the most common orange collocate is k-type, as in orange dwarf. The results for pink and beige are even more curious.

But first let's try red and green.

Fried Pickles, 3D Color Prints and Mobile HDR

The 18th IS&T/SID Color and Imaging Conference is over.



This year the conference was in San Antonio, Texas where I learned more about fried pickles, among other things.

The Calibrated Lunch

The calibrated lunch is one part ice-breaker, one part quasi-experiment and one part an exploration of the self-selection of non-basic color terms in the context of a corporate social dining event.



To start you'll need individually calibrated and uniquely numbered printed color patches. Then you'll need a couple hundred hungry co-workers. Beer is optional.

Red is the New Black

For those of you that haven't forwarded it to us already, yes we know that that xkcd has conducted a color naming survey and has posted some analysis of their color survey results. Fun stuff, with interesting similarities (and differences) with some of the work we've done over the years relating to color naming. To maintain parity, we offer the following cartoon response to their survey.



And yes, the comic sans font was used purely for it's inflamtory powers.

Please Pass the Chartreuse

In a profoundly scientific discussion about naming colors at HP Labs, the other day, the color name chartreuse came up. Nathan declared he'd never heard of it until he saw it recently used as a response to his ongoing web-based color-naming experiment. I stated that I only knew of it because my mother used that word when referring to a new dress or curtains or something. I told you it was a profound discussion.

Chartreuse probably stuck with me because, from age 12, I became fascinated with organic chemistry in general and, azo dyes, in particular. So, I'm almost sure it went on my list of chemical terms along with malachite green and paranitraniline red.

An On-Line Color Thesaurus

Color names are a powerful means of selecting and communicating colors. There are a variety of color vocabularies and dictionaries available but there has been less work in capturing the similiarities and differences in color naming. This post is a tool post in that the online color thesarus is embedded directly in the post.

Getting Started
To use the online color thesaurus, click on the screen-shot below to get to the thesaurus page. There simply type the color name in the text field. Once you have typed in your color name click on "submit" and then the results will be displayed. You can use the "clear" button to clear the text field.



Results
The results that are returned are a large color square with a rendering of the color if it was found. If the name was not found, for example "greeb" was entered, then the next nearest color name in terms of edit distance, in this case "green", will be returned. So no you won't have to remember how to spell fuchsia.

In addition to the colored square are the corresponding RGB and hexadecimal values. Finally there is a note about how common the color name is. Below this are the color synonyms and antonyms. Each column has smaller color squares rendering the color names and the names with links so that you can easily click-through to these names. The results are based on analysis of 20,000+ color name database in English collected from a 20+ language ongoing online color naming experiment.