Snap, match, shop

It appears I have not been on HP's home page for a month, because I totally missed a color related story, one that got a high reader score of 4.78 out of 5. This story, Snap, match, shop, is about the practical solution to a very difficult problem, namely finding the best shade of foundation for one's complexion.

As can be appreciated from the technical report, this is not the kind of problem that can be solved in a Silicon Valley garage. It requires a good research laboratory that can synergistically leverage deep expertise in color science, imaging and mobile communications.

The research team was led by Principal Scientist Dr. Nina Bhatti portraited below.

In the picture at below, Nina Bhatti holds the specially designed color chart near her skin while using a mobile phone camera to capture and send the photo via MMS to the advisory service for a color recommendation.

The full HP Labs team that helped to create the Color Match technology is presented in the picture below. Top row (left to right) Harlyn Baker, Sabine Susstrunk, Nina Bhatti, Nic Lyons. Bottom row: Mike Harville, John Schettino, Scott Clearwater.

Instead of sitting down with a consultant at a beauty counter, a shopper photographs herself using a mobile phone camera and while holding a specially designed color chart. The person then sends the photo as an MMS (multimedia message) to an advisory service. That system locates the person’s face within the image and color corrects the image for camera and lighting discrepancies.

Skin pixels are extracted from the color corrected image of the person’s face, and then compared to an existing database of previously captured and analyzed images of skin tones of real people. In a matter of seconds, people using the service receive a text message response with a recommendation on the shades of makeup that are best suited to their complexion.

The technology can work with any mobile operator and on any mobile phone equipped with a camera.

The photograph below illustrates that using the specially designed color chart and a mobile phone camera, a shopper can receive an expert recommendation on what shades of make-up are best suited for their complexion.

The photograph below illustrates how the system locates the consumer's face within the image and color-corrects the image for camera and lighting discrepancies.

The image is then compared to an existing database of previously captured and analyzed images of skin tones of real people. In a matter of seconds, the consumer receives a text message response, with a recommendation on the shade of foundation that best matches her complexion.

HP's Jhilmil Jain holds a newer version of the specially designed color chart near her skin while someone uses a mobile phone camera to capture and send the image via MMS to the advisory service for a color recommendation.

A common mobile phone equipped with a camera can help shoppers identify the make-up shades that best suit their complexion. Users receive a text message, suggesting the appropriate shade of make up for their skin tone.