Monday, April 7, 2014

Sony to bring 4K tech to surveillance cameras

If the recent flow of billions of dollars in VC capital into the data storage and analysis industry is any indication, we have evolved into compulsive data packrats. However, even billions of people cannot type all the data we hoard. It takes color imaging to produce exabytes of data. Millions of selfies and cat movies contribute to the data stash, but only machines can create "new" data at exabyte scale.

One of the most prolific kind of data generation machines are the surveillance video camera systems. With the relentless widening of the social gap, a larger proportion of the population is evolving into desperate sub-proletarians with nothing to lose. This increases home robberies and is triggering a boom for home video security systems.

On February 25 we wrote on purple disks from Western Digital (the corresponding disks from Seagate have a turquoise label) optimized for surveillance video. Unfortunately, the images the police sends to the neighbors asking for help in identifying thieves are often too blurry to clearly recognize a perpetrator.

Around 2015, Sony plans to put its 4K-resolution technology in its surveillance cameras, which will boast significantly improved picture quality. Even zoomed-in images will appear sharp. Larger CMOS sensors will be employed, and software to make effective use of the images will be developed.

This quadrupling of video image resolution will be a bonanza for the data storage industry, as the global market for surveillance cameras will grow from ¥700 billion in 2013 to nearly ¥1 trillion in 2015 ($6.791 billion to $9.402 billion).

Nikkei article

Thursday, April 3, 2014

Half of the United States is glowing a bright pinkish red

Data from satellite sensors show that during the Northern Hemisphere's growing season, the Midwest region of the United States boasts more photosynthetic activity than any other spot on Earth, according to NASA and university scientists.

Healthy plants convert light to energy via photosynthesis, but chlorophyll also emits a fraction of absorbed light as fluorescent glow that is invisible to the naked eye. The magnitude of the glow is an excellent indicator of the amount of photosynthesis, or gross productivity, of plants in a given region. In the image below, the color red was applied to the illustration to represent the glow.

Image Credit: NASA's Goddard Space Flight Center

Research in 2013 led by Joanna Joiner, of NASA's Goddard Space Flight Center in Greenbelt, Md., demonstrated that fluorescence from plants could be teased out of data from existing satellites, which were designed and built for other purposes. The new research led by Luis Guanter of the Freie Universität Berlin, used the data for the first time to estimate photosynthesis from agriculture.

According to co-author Christian Frankenberg of NASA's Jet Propulsion Laboratory in Pasadena, Calif., "The paper shows that fluorescence is a much better proxy for agricultural productivity than anything we've had before. This can go a long way regarding monitoring – and maybe even predicting – regional crop yields."

Unlike most vegetation, food crops are managed to maximize productivity. They usually have access to abundant nutrients and are irrigated. The Corn Belt, for example, receives water from the Mississippi River. Accounting for irrigation is currently a challenge for models, which is one reason why they underestimate agricultural productivity.

NASA press release: Satellite Shows High Productivity from U.S. Corn Belt

PNAS paper: Global and time-resolved monitoring of crop photosynthesis with chlorophyll fluorescence

Friday, March 21, 2014

3D print selfie

Sony Music Communications Inc. started selling the 3-D Print Figure product last year in which a figure is sculpted using full-color 3-D scanners. To create the figure, the scanner first obtains data through the scanning of a person from head to toe.

Then a computer models the data and outputs images through a 3-D printer using color ink, special bonding materials and white plaster powder. The price for a figure ranges from ¥49,000 to ¥120,000 ($600–$1500), depending on the size. According to Yosuke Takuma, who planned this business for Sony Music Communications, these 3-D figures are popular among people who want to mark such special occasions as weddings and matriculation ceremonies.

Koji Iwabuchi and his wife Yumi visited the studio from Suginami Ward, Tokyo, to order figures to commemorate their 20th wedding anniversary. “It’s like photography at the end of the Edo period as we cannot move at all,” Koji Iwabuchi said. “It’s interesting to feel like Ryoma Sakamoto. In the future, it might become an ordinary thing, but it’s fun that few people have experienced this,” he said. Ryoma Sakamoto (1836-1867) is known as the subject of some famous photos from that time.

Article with pictures

Thursday, March 20, 2014

Hyphenation of color compounds

In computer technology, the golden rule for hyphenation of new technology terms is to write them as separate words when they are first coined, as hyphenated words when they are widely used in the technology community, and as monolexemic terms when the terms are widely used by the general population. For example, in the Sixties we had electronic mail, in the Seventies we had e-mail, and around 1993 when the Arpanet was commercialized and renamed to Internet everybody went on email.

This rule is pretty simple to remember. For color compounds the situation is a little sticky, because if changed significantly in the 16th Edition of The Chicago Manual of Style. According to rule 7.85, section 1, under colors (page 375), the new rule is that in the manner of most other such compounds, compound adjectives formed with color words are now hyphenated when they precede a noun. They remain open when they follow the noun.

Examples:

  • emerald-green tie
  • reddish-brown flagstone
  • blue-green algae
  • snow-white dress
  • black-and-white print

but

  • his tie is emerald green
  • the stone is reddish brown
  • the water is blue green
  • the clouds are snow white
  • the truth is not black and white

While we are at it, rule 7.76 regarding the capitalization of “web” and “Internet” also changed. Chicago now prefers web, website, web page, and so forth—with a lowercase w. But capitalize World Wide Web and Internet.

Since files are now more important than colors, Chicago prefers to present abbreviations for file formats in full capitals. Therefore, write PDF instead of pdf, even when usually we use the latter when we actually specify file names.

More Chicago capitalization examples:

  • Macintosh; PC; personal computer
  • hypertext transfer protocol (HTTP); a transfer protocol; hypertext
  • Internet protocol (IP); the Internet; the net; an intranet
  • the Open Source Initiative (the corporation); open-source platforms
  • the World Wide Web Consortium; the World Wide Web; the web; a website; a web page

Returning to the opening, although nobody younger than 21 years of age has ever experienced a world without email, on page 380 the over twenty-one-year-old white-haired Chicago people still prefer e-mail and e-book.

Glass brain flythrough

This video gives viewers a colorful peek into the complex workings of the human brain as it thinks. In this case, we are “flying through” the brain of a volunteer who is been asked to simply open and close her eyes and hands, National Geographic reports. This 3D brain visualization was created by researchers at the University of California (UC), San Francisco, and UC San Diego with a combination of technologies, including an MRI scan, EEG and diffusion tensor imaging, a process that reveals tissue layout. Known as the Glass Brain, the imaging technology works in real time and may be used to learn more about how the human mind processes information.


Click here for more information

Tuesday, March 18, 2014

Traps in big data analysis

When I was a student, I had chosen mathematical statistics as one of my majors. At the time, the hot topics were robust statistics, non-parametric methods and optimal stopping times. Descriptive statistics was not part of the curriculum (PowerPoint did not yet exist and there was no need for meaningless 3-D pie charts).

In the student houses I lived, there were always medical students at the end of their studies who had to get a doctorate. Residencies were grueling and at that time the least effort thesis was to punch in some historical medical data. On their way home from the clinic, these students would spend part of the night in the empty punch card rooms, for about 6 months.

Thereafter, they would bring the punch cards to the data center and get 10 to 20 centimeters of SAS printout—and the desperation of not knowing how to get from hundreds of cryptic tables to a one hundred page thesis.

Many of them ended up knocking on my door with the printout and scratching their head. Because in the data center the students could not tell what analyses they needed—after all, there never was an experimental design—the data center people just ran all and every function available in SAS. Classical garbage-in garbage-out.

So, I had to tell the students to stare at the data and come up with a few hypotheses, then use the ANOVA routines to confirm them and the regression routines to do a few nice graphs.

Unfortunately, after all these years we are not much better off. Indeed, now we have to deal also with "big data hubris," the often implicit assumption that big data are a substitute for, rather than a supplement to, traditional data collection and analysis. Now we have tools like Google Correlate that allow us to correlate tons of apples with megatons of oranges.

A recent interesting paper by David Lazer et al. is a nice summary of how big data analysis allows us to create more statistical garbage: Lazer D, Kennedy R, King G, Vespignani A. Big data. The parable of Google Flu: traps in big data analysis. Science. 2014 Mar 14;343(6176):1203-5. doi: 10.1126/science.1248506. PubMed PMID: 24626916.

The authors conclude: "Big data offer enormous possibilities for understanding human interactions at a societal scale, with rich spatial and temporal dynamics, and for detecting complex interactions and nonlinearities among variables. We contend that these are the most exciting frontiers in studying human behavior. However, traditional 'small data' often offer information that is not contained (or containable) in big data, and the very factors that have enabled big data are enabling more traditional data collection. The Internet has opened the way for improving standard surveys, experiments, and health reporting. Instead of focusing on a 'big data revolution,' perhaps it is time we were focused on an 'all data revolution,' where we recognize that the critical change in the world has been innovative analytics, using data from all traditional and new sources, and providing a deeper, clearer understanding of our world."

Thursday, February 27, 2014

Google+ Auto Backup

Google does not talk to Google, or rather I should say Picasa does not talk to Blogger. Or, whatever.

I put the images for this blog on a folder called Google+Photos on my machine, downloaded the Google+ Auto Backup app and pointed it to this folder. It only took little time, but it was wasted time.

  1. For images, Google+ Auto Backup supports only the file formats jpg, webp and gif. Hello Google, GIF is an ancient file format for 8-bit images encoded with LZW; it has long been replaced by PNG, which uses the superior Flate encoding (LZ followed by Huffman) and supports both 8-bit and 24-bit images (yes, the latter is important because light typefaces can get destroyed by anti-aliasing).
  2. The uploaded JPEG images are visible on Google+, but not in Blogger, although Blogger has a tab called Select a file From Picasa Web Albums. Hello Google, why can your Blogger app not see images uploaded by your Google+ Auto Backup app?

Tuesday, February 25, 2014

Purple disk

Last year I wrote about Western Digital color-coding their hard disk drives to make it easier to find the optimal drive for an application. We saw that blue is for everyday use (about 40 hours per week in a PC), black for high performance, green for low power, and red for NAS (continuous operation, low power, vibration tolerance, error correction, streaming).

Now Western Digital has a new line color-coded purple. Purple is similar to red: it is aimed at surveillance video and meant to be always on, be deployed in bunches (vibration control), and used for streaming (different cache optimization). Although you could use red disks for video surveillance, the purple disks have an AllFrame firmware technology that reduces video frame loss.

Seagate's corresponding Surveillance HDD line has a turquoise label.

Thursday, February 20, 2014

What is your energy footprint?

In the USA, tucked into a 1,500-page budget bill now moving through Congress is a Republican provision that would restore the incandescent light bulbs that were supposed to be phased out in favor of greener lighting technology. Defenders of the traditional bulb say the government is again overreaching, that the marketplace should decide what kind of bulbs are manufactured in the USA. Led in the House by Rep. Michael Burgess, R-Texas, Republicans got the funding cutoff provision inserted into an energy and water spending bill that President Obama signed into law in mid-January 2014.

With this freedom of conspicuous consumption, according to the Wikipedia the average USA person consumes 12,000 watts of energy. By comparison, a person in Bangladesh on the average consumes 300 watts. The world average is approximately 2000 watts. Of course, the people in Bangladesh and elsewhere in the world would like to enjoy the same conspicuous consumption as USA people, so world energy production should increase to 7,148,400,000 × 12,000 = 85,780,800,000,000 watt.

Or not.

As Dr. Marco Morosini wrote in a paper on the 2000 Watt Society, in the last decades some authors suggested to consider the opportunity of a voluntary ceiling to the amount of primary energy used per capita. Wolfram Ziegler proposed a voluntary limit in the use of primary energy in central Europe under the level of 0.16 W/m2 (Ziegler 1979; 1996); this level was based on ecological arguments and was intended to limit the anthropic pressure on biodiversity. Starting from Ziegler's arguments and data, Dürr calculated and suggested a global value of 9 TW as a voluntary limit in the use of primary energy by mankind (Dürr 1993); this level would be around one fifth of the amount of solar energy transformed by terrestrial organisms, estimated by Dürr at 40–50 TW, for a human population of 6 billions at the end of the last century, Dürr suggested consequently the vision of a "1500-watt society." Goldemberg et al. (1985; Goldemberg 2004) claimed that 1000 watt of primary energy per capita would cover "basic needs and much more." Spreng et al. (2002) suggested to steer human societies towards an "energy window," defined by a lower social limit and an upper ecological limit in the use of primary energy. In Switzerland, the idea of setting a ceiling to energy usage was formulated at the beginning of the '90s (Imboden et al. 1992; Imboden 1993). Paul Kesselring (Paul Scherrer Institute, Switzerland), and Carl-Jochen Winter (German Aerospace Research Establishment, DLR) punctually suggested a "2000-watt society" as worldwide plausible vision achievable within 50–100 years (Kesselring and Winter 1994).

Primarily through the tireless efforts of Prof. Dieter Imboden, Switzerland is now on the path of a 2000 Watt Society.

If you want to learn more on the 2000 Watt Society, the best source is the brochure Smarter Living made available by Novatlantis. A 2000 watt person would consume 17,500 kilowatt hours or 1750 liters of petroleum over the course of a year.

What is your energy footprint?

You can easily calculate this number from the information provided by your utility company. For example, in Palo Alto you go to www.CityofPaloAlto.org/HomeEnergyReports.

In my case, from 18 June 2009 to 21 January 2014 I have used on average 308 watt of electricity and 1092 watt of gas for a total of 1400 watt, so at first sight I might have reached the year 2050 goal. However, the standard deviation of my gas energy usage is 1070, so clearly the average is not meaningful and I have to look at all the data.

My problem is that in the cool season I use too much natural gas for heating. The baseline for gas are the on-demand water heater and the cooking stove, while the baseline for the electricity are the refrigerator, lighting, and the electronics. In the cool season electricity use goes up a little because the heating furnace uses an electric fan.

As explained on page 16 in the Smarter Living brochure, only 20% of the overall heating energy demand can be determined by the behavior of the building's occupants. 80% of the ultimate energy needs are determined at the planning stage of a building. In 1960, when the Swiss were using 2000 watt, I was living in a new building made of bricks, cement, insulation, double pane windows, and rolling shutters. In 2014 I live in a typical 1948 California ranch house with no insulation, so I cannot do much better. In fact, according to the utilities department I even use less natural gas than my neighbors.

The reason I use more gas energy is that my neighbor's houses are mostly newer and therefore have solid plywood walls and some insulation. By efficient neighbors, the utility department refers to the most efficient 20 percent of my immediate neighbors.

Also my electricity use is not out of line with that of my neighbors.

This means that even with the 20% savings maximally possible with a retrofit, if I really want to be a 2000 watt person in Palo Alto I would have to tear down my house and rebuild it. This might be possible in rich Switzerland, but not in the impoverished USA.

There could be a potentially different way out. In 1960 we had a very low energy use because we were 5 people living in a two bedroom apartment, while now the occupancy of the current 176 m2 three bedroom house is much lower.

Indeed, my route to work is now 23.5 km and there is no usable public transportation. My car uses about 17 liters per 100 km or 4 liters per trip. This is a lot of gasoline and I keep it lower by car-pooling and dividing the number by 2. Similarly, I could reduce my energy use at home by increasing the number of residents.

In the intrest of full disclosure, here is the data provided by the City of Palo Alto and used in the graphs above.

year month
electricity
gas
total
2009 July
265.0 W
183.3 W
448.3 W
2009 August
286.7 W
117.3 W
404.0 W
2009 September
299.4 W
248.9 W
548.3 W
2009 October
265.2 W
130.4 W
395.5 W
2009 November
300.0 W
989.9 W
1,289.9 W
2009 December
392.6 W
2,042.2 W
2,434.8 W
2010 January
322.7 W
1,095.0 W
1,417.6 W
2010 February
309.7 W
1,627.3 W
1,937.0 W
2010 March
248.9 W
1,216.6 W
1,465.5 W
2010 April
272.9 W
963.7 W
1,236.5 W
2010 May
253.8 W
364.1 W
617.9 W
2010 June
281.3 W
330.0 W
611.2 W
2010 July
321.4 W
202.3 W
523.7 W
2010 August
286.5 W
227.1 W
513.5 W
2010 September
232.9 W
209.5 W
442.4 W
2010 October
251.0 W
242.7 W
493.8 W
2010 November
261.9 W
984.0 W
1,245.9 W
2010 December
289.7 W
1,941.8 W
2,231.5 W
2011 January
322.8 W
2,265.5 W
2,588.2 W
2011 February
287.7 W
2,081.5 W
2,369.2 W
2011 March
298.5 W
1,850.0 W
2,148.5 W
2011 April
241.4 W
849.6 W
1,090.9 W
2011 May
230.0 W
293.3 W
523.3 W
2011 June
166.3 W
146.6 W
312.9 W
2011 July
204.0 W
234.6 W
438.6 W
2011 August
320.0 W
217.3 W
537.3 W
2011 September
163.8 W
293.3 W
457.0 W
2011 October
251.3 W
256.6 W
507.9 W
2011 November
254.3 W
1,466.5 W
1,720.8 W
2011 December
304.3 W
2,555.9 W
2,860.2 W
2012 January
313.8 W
2,639.7 W
2,953.4 W
2012 February
287.1 W
2,178.8 W
2,465.9 W
2012 March
281.4 W
1,969.3 W
2,250.7 W
2012 April
247.5 W
1,136.5 W
1,384.0 W
2012 May
231.4 W
251.4 W
482.8 W
2012 June
220.0 W
195.5 W
415.5 W
2012 July
214.7 W
156.4 W
371.1 W
2012 August
219.3 W
161.8 W
381.1 W
2012 September
233.8 W
110.0 W
343.7 W
2012 October
217.8 W
130.4 W
348.1 W
2012 November
224.8 W
970.9 W
1,195.7 W
2012 December
309.0 W
2,063.2 W
2,372.2 W
2013 January
655.0 W
3,629.6 W
4,284.6 W
2013 February
533.3 W
3,345.8 W
3,879.1 W
2013 March
512.6 W
2,433.3 W
2,945.9 W
2013 April
432.5 W
806.6 W
1,239.1 W
2013 May
374.3 W
377.1 W
751.4 W
2013 June
355.7 W
377.1 W
732.8 W
2013 July
351.5 W
284.4 W
635.9 W
2013 August
362.8 W
283.2 W
645.9 W
2013 September
354.8 W
264.9 W
619.8 W
2013 October
385.8 W
794.7 W
1,180.5 W
2013 November
487.4 W
2,389.8 W
2,877.2 W
2013 December
541.5 W
3,970.8 W
4,512.3 W
2014 January
418.8 W
3,482.9 W
3,901.7 W

Tuesday, February 18, 2014

Quoting in LaTeX

Quotes have always been a little confusing in LaTeX. First there are two different environments for single (quotation) and multiple (quote) quotations. Second, the environments cannot be parametrized. Last but not least, both environments have some glitches.

Thomas Titz introduced a new package called quoting that addresses these problems by being parameterizable. For example, you can set the font and formatting options in the package declaration and you can even specify beginning and ending strings for the quotes, resulting in guaranteed typographic consistency across your document.

The package being new, it still has some shortcomings. For example, you might want to use dingbats for the begin and end texts, but quoting does not yet allow commands as values in the parameter lists, so you have to set them in every quote.

Example:

\usepackage{pifont}

\usepackage[font={itshape,raggedleft}]{quoting}



\begin{quoting}

\ding{125}Here and now, as always and everywhere, invention is the mother of necessity\ding{126}

end{quoting}

Before you try this, you may want to update your installation with TeX Live Utility, because the current version is from a couple of weeks ago.