Construction

Great corrupted quotes

Some poetic and powerful language, desecrated by various forms of repetition—either in perverse interpretation or just banal and commonplace usage.

“The tree of liberty must be refreshed from time to time with the blood of patriots and tyrants” — Thomas Jefferson

Despoiled by the Oklahoma city bomber, who was executed wearing a t-shirt with these words.

“When you look into the abyss, the abyss also looks into you” — Friedrich Nietsche

Despoiled by hackneyed overuse, such as in the film The Abyss.

“Religion is the opium of the people” — Karl Marx

This wonderful quote is injured in multiple ways:

• It is often misquoted as “Religion is the opiate of the masses”, which distinctly implies a condescension not present in Marx’s original prose
• It is a partial quote that misrepresents Marx’s meaning—the full (and quite stirring quote) is:

“Religion is the sigh of the oppressed creature, the heart of a heartless world, and the soul of soulless conditions. It is the opium of the people. The abolition of religion as the illusory happiness of the people is the demand for their real happiness.”

• Much different from saying people are in a stupor caused by a false prophet. He then goes on to say, 

“the criticism of religion … has plucked the imaginary flowers from the chain, not so that man will wear the chain without any fantasy or consolation but so that he will shake off the chain and cull the living flower”

• …meaning that religion is the comfort of a world in shackle, it is the best one can do while personal and spiritual freedom are denied to men.

It’s worth remembering these quotes in their original context.

Pronunciation Pet Peeves

OK I realize I am in mortal danger of becoming one of those people you meet. Those curmudgeonly misanthropes who correct your grammar, pronunciation (and somehow verbal spelling) while all you’re trying to do is have a light chat and talk about your cat. Or whatever.

But please, these things matter. While we’re speaking this language we’ve all mostly agreed on, mostly, let’s try and stick to the compact. And if that argument doesn’t persuade you, then at the very least you should read this to banish that eternal foe of civil thought, discourse and propriety—ignorance.

• id Software, who made Doom, Wolfenstein and Commander Keen is pronounced “id” (rhymes with did). It is not I.D. Software. “Id” comes from Freud’s mental apparatus and represents the base human desires and motivations below the conscious “ego” and the intellectual “superego”
• OS X - Apple’s seminal Unix operating system, is pronounced O.S 10. As in the Roman numeral X (10). There are literally dozens of interviews, keynotes, speeches where Steve Jobs says “OS 10”. He has never once said “OS X”
• Acronym - Not really a pronunciation peeve, but an Acronym is an abbreviation that must be pronounceable as a word, example: REST (REpresentational State Transfer) or VATS (Vault-tec Assisted Targetting System). The following are not acronyms: FBI, XML, HTTP. If you must use a word for them, use initialism. At least that way you don’t sound ridiculous.

OK maybe a bit harsh, but sorry I was raging there for a bit.

Governments have no right to privacy

I have come to realize that governments are typically viewed as benevolent parental overseers rather than the public servants and safety nets they ought to be. This is endemic in our language: “Democrats are in power”, “the ruling party” and so on.

This perversion runs so deep that we have largely abdicated the collective responsibility we share in the health and purpose of the state. The state is now an actor in its own right, not dissimilar to monarchs of old who ruled by power of divine providence.

We tacitly know that governments spy on one another, that they, composed of humans like us, engage in petty jealousies and subterfuges. And that they take poor decisions born of unsound judgment and emotion. Much as a recalcitrant child does. However, the latter has a strong and deliberate consequence awaiting each such mistake; and increasingly the former is excused her corruptions.

The publishing of internal memoranda of a government is a necessary act of returning power to the people’s hands. While each WikiLeaks disclosure may not itself constitute major revelations, it is essential that they exist on the public record and that the government be even a little embarrassed. It will at the very least, make them think twice next time.

A government should never be able to keep from its people, the truth of its actions and intentions. Not for the sake of security, nor defence and especially not out of some perverse claim of sovereign privacy. People have forgotten who the parent and who the child is in this relationship. Worse, they have reversed it with grotesque claims like “they must have a good reason for keeping secrets”, and “we must trust that they know best how to protect us.”

We must never abdicate the civic responsibility of monitoring and scrutinizing the state. We must demand that our public servants act responsibly and transparently. If the tide against this demand is too great, then we must turn to the fourth estate. Even if this entails high levels of national embarrassment, the retarding of diplomatic relations and even internal progress. It is after all what brought down the Soviet autarchy.

Somewhere along the way people have confused their government with their nation, and use their pride in one to defend the grotesque machinations of the other; the integrity of one to shield the privacy of the other’s occult dealings; and have conflated the scrutiny of public processes with an attack on national sovereignty.

Whether you believe that governments are a force for good or not, it is inescapable that they are a concentration of power not available to any individual. A vigil must be kept on this kind of power. Even in times of crisis—especially in such times. WikiLeaks must be defended with the same fervour as called for by the highest ideals of journalistic enterprise. Whether or not they meet the standards of journalistic excellence—especially as they probably do not.

Each disclosure must be protected and lauded, whether it holds great merit or is simply a moderate embarrassment to the state with little material consequence. Because each disclosure is an act of liberty.

Freedom only matters when most people disagree with what you do.

Experimenting with Agile at Google

The engineering discipline at Google can best be described as the Cowboy Methodology. This derisive monicker is used by agile practitioners to chide those who follow no process at all, the connotation being that they produce messy and often untenable results.

There are exceptions of course, many Google teams use various degrees of XP, Scrum, or something else entirely. However, these are few and far between in my experience. The lack of methodology predominates. This is not to say that software does not get written on time, nor is it to say that good code is not produced but the process under which it is, is largely non-existent.

When Wave was canceled, I started a new project with a colleague. We had a 6-person team and had set ourselves a 6 month goal for launch. Given entrenched practices at Google, I decided to try and run this project differently. We were fortunately armed with a product manager who was both very good and previously well versed in Agile methods. (A product manager at Google is equal parts Business Analyst, Iteration/Project Manager and Cheerleader.) It is too early to tell if the experiment has succeeded or failed, but I do have some interesting observations…

At first, everyone was enthusiastic about the project itself (quite an exciting area) and the people involved. Everyone had worked together before and respected one another. It was immediately apparent however, that terminology was a big problem. Most engineers are skeptical by nature and thus unimpressed with common Agile buzzwords: User Stories, Retrospectives, Showcases and Spikes rolled a good number of eyes. We discussed this early on and axed most of the names, replacing buzzwordy terms with less objectionable ones (Spike -> Experiment; Workshop -> Meeting).

Oddly, user stories were the next big problem. Let it suffice to say that you cannot convince rockstar backend engineers that every feature in an application should revolve around user behavior. Consequently, you’re not going to have much success with the vertical slice approach either (UI programming is anathema to many).

Other team members were willing to be more open. I was always extremely conscious of the appearance of too much process—it can make people dependent, and we neither had the resources nor the inclination for a curated, supervisory process. One advantage to our project structure is we don’t have managers, directors or clients telling us what to do. In that sense, we are akin to a startup. So we quickly replaced the story wall with a “task wall”, where stickies of undone tasks sat haphazardly until claimed, completed and discarded. There were no points, estimation or scoring. The theory being we could estimate velocity from the rate of task completion if necessary.

What else? Scheduled pair programming is out of the question. Iteration planning was largely unnecessary as we cut the process down to be very thin. The product itself is consumer focused so there isn’t really a client to manage. For a product like this, system metaphors are hard to come by as well.

So now we’ve ended up with something that doesn’t really resemble the canonical Agile method much. Have we whittled our way back down to the ranch of Cowboys? Not quite. Here is what did survive and what does seem to work in this style of environment:

• Stand-up: We are able to conduct a daily standup where useful if terse communication occurs. More importantly, it is enjoyable, people joke with each other and engage in a team dynamic.
• Retrospectives: I run one of these every 3-4 weeks when major issues need to be discussed and the next stage of the project decided upon. There is plenty of healthy debate and emergent changes in scope. Everyone has a goal after these meetings, and everyone feels heard, this is important.
• Unit tests: Our code is very well covered and tested for how much it changes. While not all the code is written in a test-driven style, the more critical components are.
• Code hygiene: Having an excellent code review/checkin process helps us keep ahead of most issues with code hygiene. Google’s toolset comes in handy here: providing continuous builds, a pre-submit queue and code review tools.
• Verbal communication/Co-location: We sit in a cluster of tables, very close to one another and can shout across questions or simply walk over with a minimum of effort. Engineers sit beside the UX designer and product manager too. So most technical and design doubts are allayed with this approach, quite effectively.
• Showcase: We throw open the demo to all interested parties (usually this includes other engineers in the office), and has been quite successful even prompting some other engineers to ask to join.

Of course, we are far from settled. It may be that we continue to whittle down the process and end up back at the Cowboy ranch. Or maybe we will continue to experiment. So far it has been interesting to see how the Agile process has worked in this kind of environment.

What do you think? Have we strayed too far from the holy path? Can Agile really work at a place like Google?

Comets and Meteors

I am exploring writing an app with Comet (reverse Ajax) aka ‘hanging gets’. I thought I knew how this worked in detail, but after days of research I found my knowledge sorely lacking. There isn’t much good information on the web either, so I thought I’d summarize what I learned here.

You can achieve server-to-client message pushing in several different ways:

• Websockets - HTML5 standard that allows you to establish a full-duplex TCP socket with a high-level Javascript API. Only Chrome/Safari, Opera and Firefox seem to support this. (Maybe IE9?)

• Forever Frame - An iFrame whose content length is infinite. You just keep writing script tags out that invoke a callback in the parent frame with the server’s push data. This is commonly used with IE.

• Hanging GET (Multipart response) - This is a wonderful hack around an occult and obscure behavior introduced by Netscape. It only works in Firefox and Safari/Chrome, but it is brilliant—by reusing the ability to send multiple images back in a single response, you can instead encode JSON packets chunked by message length. The browser processes each JSON packet without ever closing the response stream, which can live forever.

• Hanging GET (Long polling) - A less wonderful but perhaps more effective hack, a long poll is very much like a regular poll except that if there is no data, the server holds the stream open rather than return an empty response. When there is data to push, it is written and the response is closed. The client immediately opens a new request to re-establish this backchannel. A clever scheme will hold open POSTs that the client uses to send data and flip between them. This is the basis for the Bayeaux protocol.

• Other (Flash Socket, Java Pushlet, etc.) - These rely on plugins to open a duplex channel to the server and have their own issues with compatibility and problems working via proxies.

This confused me a fair bit mainly because there are two flavors of hanging GET. Long polling works on all browsers but is somewhat inefficient. Multipart response is very clever and more efficient but does not work with IE.

There are many libraries that magic all this away for you. I caution against using them until you really understand what they do. Most of the ones I checked out do way more than you want and implement every protocol under the sun. IMO this is unnecessary bloat on the JS side and an increase in stack complexity.

You can build a long polling server with very little effort using vanilla jQuery and Jetty, using its continuations API. This is remarkably scalable too, given that Jetty continuations is not a thread-per-request model. Making a server to use with Websockets is similarly straightforward.

My advice? Build a simple RPC abstraction on top of websockets. Test with Chrome or Firefox and then when you really need to support other browsers sub in the hand-over-hand long polling method I described above.

I’ll post any code I come up with.