laugh in the face of fear, tweak the nose of terror

Sometimes I love the job I am in. Last week I was lucky enough to be in a meeting with a technical fellow of one of the really big multi-national automotive manufactures. Within this manufacturer there are only 12 technical fellows world wide. This guy was an absolute authority on statistical processes such as six-sigma. The discussion covered a lot of broad topics, including design, process, failure modes and application of statistical method within the automotive industry. Some of what he said was specific to the automotive industry. Much of what he said wasn't.  He said a couple of thing that put in a nutshell things that I had kicking around in my head for a while, but had not been able to crystallize. Here they are:

Often when you say that you want to put more effort in to doing tests early, the reaction you  get from people is that they don't have enough resources to do that. His reply is that "Resources always go to the point at which failure modes are detected". You don't have the resources because you are always detecting failure modes late in the cycle. So you don't have the resources to design and deply counter measures early in the cycle. However if you put in counter measures early in the cycle, you detect failure modes early in the cycle, and that is where your resources will go. Kind of obvious really.

The other thing he said was "(engineering) design is usually considered to be a set of artifacts, such as drawing. Can we consider it to be instead a set of counter-measures". Counter-measures is an engineering jargon term meaning a thing that you put in place to prevent a fault from happening, i.e. fault proofing. As programmers we might call these "tests".

Im not a huge fan of the Pragmatic Programmers. Often their stuff sounds like it is a rehash of Agile. (btw, Are they trying to imply that everyone else is not pragmatic?)

One thing I really respect about them for though is their loyalty to their favorite language: Ruby. Their Agile counter parts mostly come from a Smalltalk background, and will if pushed cite Smalltalk as a major, if not defining, influence. However most Agilistas put distance between themselves and Smalltalk. Not so the Pragmatic Programmers. They have stuck to their guns, and actively promote Ruby. As a result Ruby is fast becoming a mainstream language.

Object Studio is another dialect of Smalltalk that Cincom have managed to port on top of the VW virtual machine and environment. It still not fully released (and will not be until late 2007/early 2008). However it is impressive that they have managed to do what Parcplace/Digitalk with a huge team and buget failed to do back in the mid 1990s, and that is to finally unify two dialects of Smalltalk.

Not wanting the be-little the tallents of the Cincom team, but I imagine that the technical talents available to Parcplace/Digitalk were huge in comparison. I speculate that Cincom team were sucessful because they managed the project well. 

I go to two kinds of conferences: Agile conferences and Smalltalk conferences. At the Smalltalk conferences, it seems that everyone is doing different and diverse things. There are never two people doing the same thing. With Agile conferences, a lot of people seem to be doing the same thing. Re-implementing the same program in different companies.

Part of the difference is because there aren't that many people doing Smalltalk, so the probablity of two people doing the same thing is lower, but the other reason is that the mainstream enterprise computing is about doing the same old stuff, over and over again, but doing it your way so that you get competitive advantage, which is the attitude of the closed source community.

It used to be the same with operating systeam. Each hardware vendor would implement an operating system that would leverage their hardware to gain competitive advantage. But not any more. They now either use an open source solution, take an open source solution and customize it, or buy a comercial operating system from a vendor.

Whilst at the Cincom Smalltalk user conference, I was reminded that most users of Smalltalk were 'people who program', rather than 'programmers'. These people often use things like Excel, Visual Basic or Matlab to achieve their goals, but have turned to Smalltalk because of its productivity. I don't need to tell you about Excel or Visual Basic, but Matlab is worth looking into.

Matlab is owned by Mathwork. It is, for a company that spreciallises in an IDE, a big company, with revenue in 2004 of 300 million, and 100 employees. They say that they 1 have million licensed users, and with that renvenue, you can believe them. See http://biz.yahoo.com/ic/109/109223.html for more details. So if you think you can't make a living out of inventing and selling programming languages and IDEs, think again. You may just be looking in the wrong place

More recently they also seem to be people using modded versions of Python. Take scipy (http://www.scipy.org/) for example.Scipy adds scientific stuff into Python. It is an open source project sponsored by http://www.enthought.com/ , and enthought seem to be building a pretty compelling business out of it.

I have no direct evidence, but it would not supprise me if VB is loosing market to these kinds of solutions. Its also an area that is not really addressed by the vendors of Java or C#. As for the Smalltalk vendors, we don't take it seriously either.

Janauary 15th, 2010. A bunch of radical bank employees, known as 'The Revolutionaries', figure that they have enough money to live comfortably for the rest of their lives and take early retirement. To pass the time, they begin to write an Open Source banking application.

Unaware of the 'The Revolutionaries', in an unrelated move, the Chinese government formulates a strategy that it believes will give its state owned financial services industry a lead over the ever expanding might of the American and European banking institutions, and instructs The National Bank of Chinese to Open Sources all of its banking software, which it does on October 1st, 2010, Chinese National Day.

The plan appears to works, and the Chinese code base is quickly assimulated by the Open Source community, lead by The Revolutionaires, giving the Bank of China all the benefits of a banking system that surpases all its competiors. Meanwhile, the western banking institutions are unwilling to put their trust in Open Source code and begin to loose their edge.The Chinese government begin to think that it has changed the balance of power.

However what they don't realise is that the bar for entry into the world of banking has been lowered, and now anybody who with a bit of capital and knows how to download and compile a bit of Open Source software can set up as a bank. As hundreds of DIY banks spring up, the old fashioned banking institutions we know and love collapse, and a new world order is established.

Apologies in advance to any war veterans.

What have the tactics of jungle warfare got to do with Agile/Lean methodologies, such as XP and Scrum? Gurrillas fighting in the jungle often had to tackle large patrols of foreign troops who outnumbered them. In some wars, the gurrillas apparently did this very successfully. How did they do it?

Their tactics were to attack the head of the patrol and then immediately drop off. The head of the patrol would then give chase to them. When they had successfully separated the head of the patrol from the rest of the patrol, they would turn around and fight. In this situation the gurrillas now outnumbered the opponent on hand, and they were able to overwhelm their limited numbers easily.

Once they had successfully executed this tactic, they would repeat the same process, in theory until the entire foreign patrol was eliminated.

So to summarize, to fight effectively, you have to reduce the batch size of the opponent at hand, and iteratively tackle each batch until the job is done - sound a bit like lean/agile, doesn't it.

Now here comes the interesting bit: Why is batch size so important?

It turn out that if you have a numerical advantage, the advantage is a better-than-linear function. Because as the large team reduces the size of the smaller team, the smaller team becomes progressively less dangerous. Myself and a colleague were discussing this the other day, and we decided we would work out what the advantage would be using a simple simulation.

Assume you have a group (a) of gurrillas shooting at a group (b) of 25. Suppose also that per unit time, each person has a 0.1 probability of killing someone on the other team. The decay in the size of the two teams can be crudely plotted as follows :-


                        b                        a
                   b                          a   
              b                             a     
         b                                a       
     b                                   a        
 b                                       a  


At the top of the plot, he size of group (b) starts off at half the size of group (a). Group (b)'s size drops off at a rate that approximates a linear function.

In contrast, the size of group (a) starts decaying at half that of group (b). And then rate of decay starts to back off very quickly. At the end, when group (b) has been completely eliminated, the size of group (a) is 42.

In this idealized simulation, for a loss of less than 16%, a force half your size is eliminated. Do this repeatedly, and the batches will accumulate and you will have eliminated a numerically superior force.

The simulation code (in Smalltalk, 'cause thats installed on my computer), with a simplification in the modeling of probability is as follows:

| a b a0 b0 string |
a := 50.
b := 25.
[
    string := '                                                             ' copy.
    string at: a asInteger put: $a.
    string at: b asInteger put: $b.
    Transcript cr; show: string.
    a0 := a.
    b0 := b.
    a := a0 - (b0 *0.1).
    b := b0 - (a0 * 0.1).
    (a asInteger > 0) & (b asInteger > 0)
] whileTrue.

The next question is this:

So back to software development. If I have a team of 10 that needs to complete 10 stories in an iteration, is it better for the team to tackle all 10 stories at the same time. Or is it better for that team to take 5 stories and do those first, before finishing off the last 5 stories?

Im sure with a bit more thought we could simulate the dynamics of the kind of problem solving that occurs when we do software development and get an objective answer to this question. The simulations would be making assumption, but at least these assumptions could be stated up front.

Although we didn't build a model, both myself and my colleague agreed that we could make the mental jump from tactics of jungle warfare to the tactics of deploying a programming team. We also agreed that we were able to make the jump because of this kind of simulation is far more compelling than the conventional explanations we had been offered in the past.