Wednesday, March 24, 2010

workflows for technical reports

Writing about workflows for scientific books, I recently came across two old documents while I was searching for a completely different document. In 1988 the technical report publishing process was completely broken in PARC's Computer Science Laboratory. While historically tech. reports were a low-overhead method for the quick dissemination of new research results to get feedback from peers in the same field before one would write a conference or journal paper, in CSL they had become an unachievable trophy or a fata morgana.

Essentially, one would have to first publish a paper in a first tier ACM conference and get a best paper award. Then one could beg for permission to turn it into a Blue and White tech. report. The quality standard for a Blue and White was much higher than that of any scholarly journal, and the writing process was very onerous and time consuming.

In fact, in 1988 CSL was still processing Blue and Whites that had been started before the 1983 exodus, by authors who had left PARC five years earlier. The workflow was a gauntlet meant to discourage anybody dreaming of writing a PARC Blue and White. It was worse than running blindfolded in a maze during a moonless night, where the bush leaves are razor blades.

To unclog the publication process, I set myself the research task of figuring out the secret and arcane workflow. The result was a PARC Notebook entry with a new streamlined workflow that would meet all of the gauntlet's true requirements, leaving out the traps. Here is a scan of the document:

The Blue and White Publication Process

My little project was successful, because it unclogged the process and there was a fresh stream of Blue and Whites.

Of the mentioned tools, Interpress has metamorphosed into PDF, while Cedar and Tioga ended up in the historical archives. In a way this is a pity, because there has never been a commercial match to Tioga. Later, FrameMaker came very close, but then John Warnock liked it and bought the company, and when he retired FrameMaker went into maintenance mode. Later QuarkXpress and InDesign where invented, but they are for designers, not for scientists writing technical reports. Today we really have no WYSIWYG document preparation program and our best tool is LaTeX.

In 1988 I actually was no longer in CSL, but in the new Electronic Document Laboratory EDL. We did not know it at the time, but we were inventing digital publishing. Other researchers were working on electronic publishing, where each step of the traditional mechanical publishing workflow was replaced with a digital method. We, however, took the long view and pondered how computer technology allows us to invent much better workflows. We were after an emergent property, a paradigm shift. It was only 10 years later in a discussion at EI that this was realized and the term digital publishing was coined to replace and augment the old term of electronic publishing.

We were working on color printing, so our publications would have color plates showing, for example, the same image with a conventional and a new gamut mapping algorithm. Alas, a printing process capable of maintaining such an illustration was 16 times more expensive than a black and white process.

So I did another little research project studying the color printing workflow to come up with a much lower cost digital printing workflow. Sure, some methods have been around for centuries, like the ganging of print jobs (although lately people have been reinventing it). However, the key insight was to discover that the main cost factor was insurance, i.e., the cost of mistakes that had to be factored into the price calculation.

I created the new workflow simply by taking out the risks from the old workflow. At every turn where something could go wrong, I replaced it with a computer-based method and rearchitected the workflow. The result is in the Notebook entry:

Printing Blue and White Reports with Color Plates