Thursday, June 20, 2013

3-D printers now mainstream

One way to tell a technology has crossed the chasm from the visionary early adopters to the pragmatic early majority is when it is written up in your local newspaper. Although 3-D printers have been around for many years, they have been an expensive and fragile tool for techies.

In today's New York Times, Amy O'Leary has written an article describing several examples of things people can do for their everyday life, complete with model recommendations and prices, as well as a link to Amazon's new on-line store for digital printing.

3-D Printers to Make Things You Need or Like

Tuesday, June 18, 2013

Robots in your eyes

Retinal vein occlusion from glaucoma is only one of several diseases that can decrease the oxygen supply to the retina: Like every tissue of our body the retina needs oxygen. An insufficient supply can cause blindness, sometimes within mere hours. In order to make a fast and correct diagnosis, physicians need to be able to assess oxygen levels within the eye. However, the currently available tools are not very sensitive. Researchers of the multi-scale robotics lab at ETH Zürich have now developed a micro-robot that can measure the retina’s oxygen supply.

News article with link to the paper: Oxygen-sensing microrobots.

Friday, June 14, 2013

Shape Description Experiment: Comments and Discussion

This is a follow-up post to the Shape Description Experiment to allow reader comments and questions. Thanks.

Shape Description Experiment

It looks like your browser doesn't support data objects. Click here to go directly to included content. If you have comments or questions about this experiment: Please post them here.

Thursday, June 13, 2013

Photography or photographs

Tuesday I attended an interesting seminar organized by Joyce Farrell at The Stanford Center for Image Systems Engineering (SCIEN). The presenter was
David Cardinal, professional photographer, technologist and tech journalist who talked about Photography: The Big Picture — Current innovations in cameras, sensors and optics.

To cut a long story short,

  • DSLR systems will be replaced by mirror-less systems, which deliver almost the same image quality for the same cost but are an order of magnitude lighter
  • point-and-shoot cameras will disappear, because there is no need to carry around a second gadget if it brings no value added over a smart phone (the user interface still needs some work)
  • phone cameras will become even better with plenoptic array sensors etc., like the one by Pelican Imaging

An interesting factoid for those in the storage industry is that according to an IDC report cited by David Cardinal, in 2011, people took a total number of digital photographs requiring 7 ZB of storage (a zettabyte is 270 bytes). All those photographs could not end up on social networks, because even if for example with Flickr you get 1 TB of free storage, with the typical American broadband connection it would take 6 months to upload that terabyte of images.

David Cardinal mused that maybe people are no longer interested in photographs, but rather in photography. By that he means that people are just interested in the act of taking a photograph, not in sharing or viewing the resulting image. Therefore, he speculated the only time an image is viewed might be the preview flashed just after pressing the release button.

Is that so? According to Cisco's forecast, globally, Internet traffic will reach 12 GB per capita in 2017, up from 5 GB per capita in 2012. For photographers, these are not big numbers when one considers that a large portion of the Internet traffic consists of streamed movies and to a minor degree teleconferences (globally, total Internet video traffic (business and consumer, combined) will be 67% of all Internet traffic in 2017, up from 52% in 2012).

However, we might not remain stuck with the current miserable broadband service for long, and receive fiber services like the inhabitants of Seoul, Tokyo, Osaka, etc. do get. In fact, just a kilometer from here, Stanford's housing area is already on Google Fiber, and other places will soon be receiving Google Fiber, like Kansas City and others around it, Austin, and Provo (in Palo Alto we have fiber in the street, but it is dark and there is nothing in the underground pipe from the curb to my network interface). According to Cisco, globally, the average broadband speed will grow 3.5-fold from 2012 to 2017, from 11.3 Mbps to 39 Mbps (reality check: currently our 6 Mbps down and 1 Mbps up, $48/month, VDSL service delivers 5.76 Mbps down and 0.94 Mbps up; 1 km west, our residential neighbors at Stanford get 151.68 Mbps down and 92.79 Mbps up—free beta Google Fiber). In summary, there is reason to be optimistic.

Should Flickr plan to open a 20 ZB storage farm by 2017 in case people will be interested in photographs instead of photography? Probably not. The limit is not the technology but the humans on either end. We cannot enjoy 20 ZB of photographs. Just ask your grand-parents about the torture of having had to endure the slide-shows of their uncle's vacation.

The path to the answer to David Cardinal's question about photography vs. photographs is tortuous and arduous, at least it was for me.

In spring 1996, HP's storage division in Colorado started manufacturing CD-ROM drives for writable media. To create a market, the PC division decided to equip its new consumer PC line with the drives. The question was what could be the killer-app, and they went to HP Labs for help, because Quicken files would never justify an entire CD-ROM.

Working on the FlashPix demo with Mr. Hewlett's flower photos

At that time my assignment was to work on a web site to showcase a new image format called FlashPix (see image above, click on it to see it at full resolution; a GIF version of the demo is still here). The folder Web Album at the bottom center of the desktop contained the demo I gave the CD-ROM people from Colorado.

In February 1991 I had a dinner with Canon's president Dr. Yamaji, where we strategized over the transition from AgX photography to digital. At that time, Canon had the $800 Q-PIC for consumers (really a camcorder for still video images), and a $28,000 professional DSLR. By considering the product release charts of both Canon and its suppliers, we figured that it would be 2001 until digital could replace AgX in both quality and price. For the time in-between we decided to promote Kodak's Photo-CD solution as a hybrid analog-to-digital bridge.

The people at Kodak told us the average American family keeps their images in a shoe box, with the average number of photographs being 10,000. By looking at the evolution curve of disk drives, which at the time was twice as steep than Moore's law for micro-processors, we figured that the digital family would accumulate an order of magnitude more photographs, namely hundreds of thousands, because the effort and cost per photograph would be so much lower in digital.

As in 1996 I was assembling collections of a few thousand photographs for the FlashPix project, it quickly became clear that images on a disk are much more cumbersome to browse and organize than prints in a shoe box. I tried several commercial asset management programs, but they were too slow on my low-end PC.

I ended up implementing a MySQL database and maintain a number of properties for each image, like keywords, rendering intent prediction, sharpening, special effects, complexity, size, copyrights, etc. Unfortunately it turned out it was very tedious to annotate the images, and it was also very difficult.

Indexing entails categorization, which is a difficult cognitive task, requiring a high degree of specialization. What makes this worse is that the categorization changes in time as iconography evolves. Categorization has a scaling problem: a typical consumer album in 1996 required more than 500 keywords, which are hard to manage. Hierarchical keywords are too difficult for untrained people and taxonomies (e.g., decimal classification system) are too bulky. We proposed a metaphor based on heaping the images in baskets on a desktop:

A metaphor to efficiently categorize images

The labels on the baskets were sets of properties we called tickets, represented as icons that could be dragged and dropped on the baskets. The task was now manageable, but nothing that could be used by consumers.

In a family there typically was a so-called photo-chronicler who would go through the shoe box and compile a photo-album. Photographs by themselves are just information: to become valuable they have to tell a story in the form of a photo-album.

The Web photo album

The applet on the right side was running in a browser and would interface to the content on the MySQL database running on the public HP-UX server under my desk. The idea was to reduce the work by allowing any family member who would have time to fill in some of the data: we replaced the model of the mother photo-chronicler with a collaborative effort involving the whole family, especially the grand-parents, who typically have more time.

Collaborative annotation introduces a new problem, namely that each individual has their own categorization system. As described in Appendix II of HPL-96-99, there is a mathematical theory that provides a solution to this problem. An A-construct in mathematics is very similar to an ontology in computer science. The solution is to create ontology fragments for each contributor and then map them into each other leveraging the general structure theory.

Unexpectedly I received a Grassroots Grant, which allowed me to hire a student to rig up an interface to Stanford's Ontolingua system. As a difficult example we took photographs from a mixed-race and mixed-religion wedding (a small sample of the images is here).

Guests at Helen's fairy tale wedding

We wrote up a conference paper (a free preprint is available here) in a couple of nights, because that was it. A senior manager had determined that nobody would ever have more than 25 digital photographs. The manager insisted that when people would take new pictures, they would delete old pictures so that only the best 25 photographs survive. That was it and the project was killed.

In a way, David Cardinal's photography would be an extreme form of this assertion, where only the last photograph survives, and that only for an instant.

In my view, people are too fond of their memories to just toss them. The future will not be a pile of 20 ZB in pixels. Value is only where a story can be told, and our job is that of creating the tools making storytelling easy.

In retrospect, my mistake in 1996 was to do the annotation as a separate workflow step to be performed after triage. Today the better approach would be to use voice input to annotate the photographs while they are been taken. Today, the metadata is just the GPS coordinates, the time, and the other EXIF metadata. How about having Siri asking the photographer "I see five people in the image you just took, who are they?" "Is this still a picture of the wedding?" etc. Easier and more accurate than doing face recognition post factum. Last but not least, we have the image processing algorithms to let Siri exclaim "Are sure you want me to upload this photograph, I think it is lousy!"

I am game for uploading my share of 4 GB in photographic stories when 2017 comes around!

Wednesday, June 12, 2013

White House tackles trolls

On June 4 the White House issued a fact sheet that laid out a framework for five recommended executive actions and seven legislative recommendations for both branches of government to address as a means to tackle "patent trolls." For example, the Patent and Trademark Office will draft a proposal to require patent applicants and owners to "regularly update ownership information" as a means of protecting against the creation of shell companies for hiding abusive litigants. Further, it will permit more discretion in awarding fees to prevailing parties in patent cases, providing district courts with more discretion to award attorney’s fees as a sanction for abusive court filings. It also calls on Congress to craft similar legislative language.

Link: Fact Sheet: White House Task Force on High-Tech Patent Issues

Monday, June 10, 2013

The power of crowd sourcing

This morning on the local radio in the transmission Morning Edition there was a short piece on the new NSA data farm in Utah, which is supposed to go on-line this September. The piece stated that the data farm will store 5 zettabytes, and the old data farm in Virginia, which will remain on-line, has about 2/3 of the capacity.

These 8 zettabytes are contributed by us aliens, i.e. non citizens: this makes it crowd sourced data. How does this compare to the data that the best and brightest scientists in the world can create? At CERN, the CERN Data Centre has recorded over 100 petabytes of physics data over the last 20 years; collisions in the Large Hadron Collider (LHC) generated about 75 petabytes of this data in the past three years; the bulk of the data (about 88 petabytes) is archived on tape using the CERN Advanced Storage system (CASTOR) and the rest (13 petabytes) is stored on the EOS disk pool system — a system optimized for fast analysis access by many concurrent users. For the EOS system, the data are stored on over 17,000 disks attached to 800 disk servers; these disk-based systems are replicated automatically after hard-disk failures and a scalable namespace enables fast concurrent access to millions of individual files.

A zettabyte is 270 bytes and a petabyte is a paltry 250 bytes, indicating that crowd sourcing can yield 5 orders of magnitude more data than the best scientists can. And while the scientists use the most powerful particle smasher ever built by human kind, the crowd just uses their fingers on plain old keyboards.

The more mind-boggling data point is that at some point the NSA may want to synchronize the data in the two farms. To get an idea of the required bandwidth, consider that backing up a 1 terabyte (240 bytes) solid state disk to a top-of-the-line external disk over a FireWire 800 connection takes 5:39:39 hours…

CERN data centre

Servers at the CERN Data Centre collected 75 petabytes of LHC data in the last three years, bringing the total recorded physics data to over 100 petabytes (Image: CERN)

Friday, June 7, 2013

Why color naming

Mid June 1993 I was strolling along the Duna river in Szentendre with Antal Nemcsics and Lucia Ronchi. We stopped, looking across the river, while Nemcsics was explaining his Dynamic Color theory. Then he turned around and with a broad sweep of his arm he referred to the cityscape stating "it has just been all repainted in its original colors; is it not beautiful how all these yellows from the local clays harmonize?" He then started calling out the names for the yellows and explained how the restoration was based on the sequence of the color names.

When I interjected that color names are arbitrary conventions between painters, and sequences in a perceptually uniform color space might be better, he countered that the color names were not arbitrary but based on solid psychophysics. He answered my question on how the 15 students typically recruited for psychophysical experiments could define something so complex as the names of colors, with the bold statement that for many years all his students had to contribute their data and his Coloroid system was based on the outcome of over 80,000 experiments. Wow, big data!

I had to turn around to the Duna river and take a deep breath. That instant in Szentendre remained deeply impressed in my memory, and I visualized the color name regions of varying volume in the Coloroid space.

A decade later, when I was working on the readability problem of colored text on a colored background, I first implemented an algorithm based on distances in CIELAB. While the solution worked from an engineering point of view, it was not entirely satisfying, because a fixed distance in the entire CIELAB space did not reflect the reality of readability for the various color combinations.

Szentendre came to mind and I decided to try out a different implementation based on lexical distances. Implementing the Coloroid system was not a piece of cake, because the calculations are numerically instable and not all the details of the space are published. Also, if the color names in the Coloroid atlas are adequate for urbanistic applications, some extensions are required to achieve a bullet-proof application for automatic document publishing.

Boundary color Coloroid luminance

I presented the result at the 2005 AIC meeting in Granada, but I must admit that not many people stopped by at my poster, although it resulted in a collaboration with Silvia Zuffi on the CIELAB based implementation.

Coloroid hue A = 20

Since the color naming solution was going into a product, a patent application was filed, but very reluctantly and with much hesitation. Indeed, color naming and color categorization were very controversial.

Suddenly, in the last few days everything has changed. We now know experimentally that at the quantum level time does not really exists as we perceive it. This new twist on entanglement is one of the tenets for Federico Faggin's new proposal for the concept of awareness. In one arm of the entanglement we have Kirsty L. Spalding, who after a decade of very difficult work studying the hippocampus of people exposed to the fallout of nuclear bomb tests was able to prove experimentally the physiological existence of a locus for categorization. In the other arm we have patent 8,456,694, which thanks to this entanglement is made rock solid.

Thursday, June 6, 2013

Proofing a page for color discriminability problems based on color names

Now that there is a physiological basis for color categorization, we can ask ourselves what this is good for. We cannot eat it, but it might have considerable commercial value in United States Patent 8,456,694 issued two days ago on June 4, 2013. I am not a lawyer, but it appears that if you take two colors, determine their names, and then do anything with it, you might have to license this patent (but you can still keep your hippocampus ☺).

In large American corporations, when a new CEO start their new position, they often begin by putting their mark on the company's branding. They remodel their office and maybe even the HQ entrance, tweak the logo, design a new font, change the corporate palette, etc. These endeavors cost millions of dollars, but big corporations can afford it, especially when as a consequence other big corporations get motivated by the new branding to buy more widgets of the new CEO's company.

The only pity is that often this means that entire forests are wasted when the company has to reprint all its marketing collaterals. Around 2000, my employer at the time had a big warehouse in Campbell with product brochures, but fortunately our manager had been able to convince the company to deploy a document management system and print the marketing collaterals on demand, just when they were needed.

The hard problem came when a year later the CEO decided to change the color palette. Although all brochure chunks were stored digitally, when a brochure was produced by combining chunks with the old palette and chunks with the new palette, the resulting brochure looked inconsistent.

Colleagues Hui Chao and Xiaofan Lin quickly wrote code that could perform a wide range of graphical changes to the collaterals in the repository, and this writer wrote a few lines of code that would replace an old palette color with the perceptually nearest color in the new palette. Unfortunately, already the first test run demonstrated that this was a hack that did not work in practice. For example, many chicklets ended up having bright green text on orange background, something chromatically challenged people with color vision deficiencies could not read.

The solution that worked was to use a model to compute the names of the foreground and background colors, then change one of them to the nearest color in the new palette that was at least to color name categories away from the other color. This solution ended up being very good in practice and we wrote very efficient code that could process a large repository in a very short time.

I guess a sign of good engineering is to have the intuition for an unexpected solution before the scientists have worked out all the facts, …and we did not need nuclear bomb explosions.

Physiology of color categorization

Up to today there have been quite a few people writing off color naming and categorization as unserious hogwash. As of today, we know of a possible physiological basis, giving us a little more credibility.

In fact, now we know that neurogenesis it taking place in the hippocampus of adult humans. Fresh adult neurons have a specific function facilitating cognitive plasticity in the hippocampus—for example, in helping the brain distinguish between things that belong to the same category, or comparing new information to what it has already learned from experience. The ability to distinguish between vermilion and pink, yet still identify both as flamingo colors, is one example of this type of task in humans.

Kirsty L. Spalding et al. have found that a large subpopulation of hippocampal neurons constituting one-third of the neurons is subject to exchange. In adult humans, 700 new neurons are added in each hippocampus per day, corresponding to an annual turnover of 1.75% of the neurons within the renewing fraction, with a modest decline during aging. They conclude that neurons are generated throughout adulthood and that the rates are comparable in middle-aged humans and mice, suggesting that adult hippocampal neurogenesis may contribute to human brain function.

Reference: Dynamics of Hippocampal Neurogenesis in Adult Humans, Kirsty L. Spalding, Olaf Bergmann, Kanar Alkass, Samuel Bernard, Mehran Salehpour, Hagen B. Huttner, Emil Boström, Isabelle Westerlund, Céline Vial, Bruce A. Buchholz, Göran Possnert, Deborah C. Mash, Henrik Druid, Jonas Frisén: Dynamics of Hippocampal Neurogenesis in Adult Humans. Cell, Volume 153, Issue 6, 1219-1227, 6 June 2013.