Wednesday, March 2, 2016

Architecture vs. baling wire and chewing gum

Over thirty years ago, I was working in the Dragon project. The full-custom VLSI chips were so complex, that we were not able to use the commercial tools of the time. Thus, part of the team was working on the development of a set of tools design the chips; I was working on the design rule checker.

Since we were concomitantly writing the tools, designing the chips, writing the compiler, and porting the operating system, we all had to work in parallel. For the tools, this was made possible by having each tool have its own file format and the data was moved from one workflow step to the next in a file. For example, the design rule checker parsed in the layout file and emitted the files for the logic and time simulators.

The challenge was, that this was not really a development project but a research project: we were inventing the algorithms while we were implementing the tools using them. One consequence was, that the file formats kept changing and this broke the parsers. This was painful, especially because we had hard deadlines with the MOSIS service we were using.

One day, during another no-so-happy meeting, I asked the question: "why are we using files at all, we are just dealing with different representations of the same chip. There should be just one data structure and the design tools should decorate it, analyze it, or render it."

The idea stuck and two colleagues designed and implemented the Core data structure. The layout editor would add geometry to the Core data structure and the various design rule checkers and simulation programs would analyze the data structure elements and add their own decorations. Almost immediately, a designer added a schematics editor and we wrote a program to generate the documentation by traversing the Core.

In a way, the original system with the parsers was built with baling wire and chewing gum, while the Core data structure introduced a sophisticated architecture. The various programs became simpler and so we were able to do nightly builds of the entire system. The entire design process became much more efficient and the tools more powerful.

Today, it has become very easy to quickly rig up a system based on open source software. A large number of powerful libraries is available and many produce very compelling eye candy.

Not too long ago, I had to build a prototype where a user could take a mobile phone picture of a store receipt and see on the display the receipt with the prices of a competing store. Since the demo for the customer was imminent, I had to work like a beaver to set up an OCR server, figure out how to get through the firewall, build a price translation table for the two stores, and set up a website to display the doctored receipt.

It took me two weeks and the wabi-sabi result looked like a prototype. Concomitantly, over a weekend, a bunch of young programmers rigged up an app that took a picture, replaced the top quarter with the bitmap of the top of the competing store's receipt to change the store block, leaving the item list unchanged, and displayed the result. This was even worse than bailing wire and chewing gum. However, they spent time making the GUI look slick.

At the end of the day, they got lot of compliments, but since there was no architecture, they would never have been able to implement a working system. In particular, doing the price translation is difficult. They would not have been able to build a real system.

Unfortunately, today's plethora of libraries makes it not only possible to write good applications, but also to quickly rig something up without understanding what is going on. There is much less interest in well-architected powerful software than used to be. Fortunately, the economy works in waves and the next downturn will flush out the system. The smart samurai does not fight his enemy: he patiently sits downstream the river to wait for the body of his enemy to float by.

waiting by the river