Smalltalk Tidbits, Industry Rants

delivered by Kent Beck. He had an interesting talk, and got a lot of feedback from the audience. My notes:

Keynote - Kent Beck

The Spirit of Extreme Programming Asked the audience for some expectations - questions related to topic of talk

1. How do you deal with the question - How dare you challenge the conventional wisdon?
2. How do you handle personality issues?
3. Do you always find an appropriate system metaphor?
4. How often are clients really involved in the process?
8. How do you convince an organization to use XP?
9. Pete McBreen - Software Craftsmanship - square with XP
11. From a client perspective - what is the value of XP?
13. how to iterations work with project manager schedules?

How did XP get started? (Creation Myth)
Always been a programmer, father was a programmer. Always immersed. Was in the Smalltalk world, where it's easy to learn about programming. Technical expertise is just not enough - projects mostly fail for political reasons. Wanted to expand his viw beyond the programming to the whole context of programming. Question - what is the least I can do (besides programming) and still succeed? Only once - 1994 - got in at the start of a project. typically gets in after deadline, when everyone hates each other, and when the money is gone. This project is where SUnit, lots of testing, and frequent refactoring were born. It was a typical mainframe to mid-range project. After 4 months, much progress - but management liked it so well, that they explained the actual goal, as they say it - get the calculation engine running on the mainframe. So goals are important - but they change Next - brought into Chrysler for the payroll project. The goal was to fix the performance problems. Asked - where are the tests, so I don't break anything? Answer - we don't get correct answers yet. Said - I can make it run fast, so long as you don't care what the answers are. So, half the team left within 3 days, contracters were kicked out. IT chief asks for answers:

1. Cancel it
2. keep going the way you are going
3. Throw away all the code, give everyone a week off, and start over.

picked option 3, so long as Kent was in charge. He objected, as he said he was not a project lead. CEO convinced Kent with large sums of cash. inspiration - a combination of preparation and panic Now had plenty of panic. Interviewed entire staff so as to get to know them better. Came up with iterations (3 weeks), stories (to flesh out iteration goals) with first person. As things progressed, came up with acceptance tests to validate. Then came up with Unit tests as a developer task to help get to acceptance test Kent: I was just making this (expletive deleted) up Open workspace, people sitting together, continuous integration, unit testing - it all just came together on the Chrysler project with the team. Mental Model Do what they would normally do, but at maximum effort took 3-4 months of coaching to get to all those steps at once take everything I knew about software development, and turn it all up Thought this was having the knobs all the way up Since then, have moved to 1 week iterations - fits with the human notion of a unit of time that is imaginable.

All successful ventures have a creation myth

Companies and nations have them - projects can use them as well.

---

Questions from earlier: What about personality issues? Kent - started out shy and introverted. The more people I meet, the more I learn and the better I get at what I do. Having different sorts work together helps. This is why the requirements come from the customer representative.

---

Question from earlier: How often do clients really get involved? Software as sexual reproduction? Male - business representative Female - Developers Male - leaves 9 months later - the software is born Hurts so much, you aren't even going to try that again soon. (Business rep comes in with a new spec immediately...) Answer - How often are males really there all through pregnancy? What about software as ballroom dancing? Male - business Female - partner work - together If one leaves, it's not dancing Using dancing as a metaphor, software doesn't work unless the client gets involved. You need to get this metaphor embedded, and then it guides the process Metaphors are hard to come up with

---

Close by talking about another question - What is the value to the customer? Total Effort - No Excuses. Focus and Intensity. Turn the knobs up. Be willing to recognize and try to kill doomed projects

I listened to Scott Ambler's presentation on data(bases) and agile methods this am - there was a lot of good stuff on the issues. Here are my notes:

Scott Ambler - Agile data, Agile databases

Metaphor from this morning - the data (dba) community is packing for Antarctica. Unfortunately, the rest of us are traveling in the desert or the jungle

1. Agile Data Method
2. Agile database techniques
3. Summary

Data people - want all the info (data model, etc) up front (Antarctica)
Developers - want agility and incremental (Jungle)
Two reactions to the dba
love them - willing to adapt, change
hate them - insists on up front logical data model, paper, validation, slow to change
Same kind of issues between local project teams and the enterprise planners (IT)

---

Premises
The data and developer communities can, in fact, work together
Part of the problem was the early attitude of the OO community towards the data (RDBMS) community
Developers are not working in a vacuum - there is an enterprise architecture that has to be considered
The enterprise staff is theoretically supposed to support, not control, developers
Not them vs. us - only us. have to work together as a team
Need to find the sweet spot - don't fill the backpack to overflowing

Example - Scott's team (Antarctica) died 11 miles from the ship - carrying too much stuff

There is more to modeling than data modeling
Likewise, there's more to software than coding

---

The Agile DBA Application developers need to understand the basics of Data Models
Likewise, an Agile DBA must understand the basics of programming Otherwise, they cannot communicate effectively. Similar to the translation issue of this presentation - the audience is listening to a Portuguese translation, things might get lost. If we were to work together, we would learn a bit of each other's languages.

---

Enterprise Architects
There to support the team, not to produce large documents and models. There to help the team implement a vision that fits into the overall scheme. Should roll up their sleeves and get their hands dirty (like a guide on an expedition). One size does not fit all in methodologies and data

---

Agile Database Techniques for Agile DBA's Database refactoring is, in fact, as necessary as code refactoring but it's much harder What is it?
The act of making a simple change to the database schema The database is highly coupled to your application code, other application code, data import/export routines, reporting tools, persistance frameworks, documentation, etc. Thus, refactoring is hard What do you need?

1. Strong configuration management system
2. Full regression test suite
3. Willingness to work together
4. Acceptance of your situation

There actually are testing tools for databases. Use them.

When I'm home, my wife makes sure I get to bed at a decent hour. When I'm on the road, no matter what my original intentions were, I end up staying up late. Off to bed...

There were other good talks today as well - Lowell Lindstrom of OMA was talking about XP in the organization, and Scott Ambler was talking about Agile Modeling at the same time. Cool quote from Scott:

"I keep telling people that there are two languages - Smalltalk, and things that aren't Smalltalk"

Alas, as he says, no one listens. I decided to go to Scott's talk, and it was interesting stuff - I didn't take notes, but he had little good to say about UML:

"What does does UML say about Databases?"
Nothing "What does UML say about the User Interface?"
Nothing "How many of you have ever worked on a system that didn't have either a UI, a database, or both?" Audience laughs

It was a good talk, and I enjoyed it.

and took notes. He had a good talk, and after the audience opened up, lots of questions. an amusing thing happened (which I only found out about later) - whenever Kent said Smalltalk, the translation came back as personal conversation. That will have to get addressed before I speak, or my talk will be baffling. Anyway, here are the notes I took:

Kent Beck - "Learning to Steer"

1. introductory caveat - software development varies with culture, ymmv
2. Driving metaphor - driving is not about aiming the car, it's about making constant adjustments - speed, direction, etc.
3. asks audience for some questions, will put on paper and address them

Questions:

1. XP in 5 or size words?
2. How was eXtreme programming born (creation myth)
3. Is the experience (inexperience) of the team important?
4. what is the relationship between XP and RUP
5. Conversational dev?
6. applying pair programming?
7. Is CMM relevant, and can it work with XP?
8. Initiation of an XP project?
9. XP - relationship with Q/A and testing?
10. How much planning is enough?
11. Risk management with regards to contracts?
12. How does XP scale by project size; is there an ideal size?
13. stories and project management
14. Documentation?
15. How much of the dev process does XP cover?
16. Partial Adoption of XP?

In the course of the talk, Kent will address the top three

---

System Dynamic - reference to physical design and manufacturing

1. break product into discrete steps
2. keep machines busy building parts - build up an inventory
3. no feedback, so more waste

Premise - this is about software, not hardware. The more inventory, the more waste and the less feedback. This is bad One possibility - keep less inventory in order to reduce waste
Question answered - What is the goal of XP?
What is the goal of Software Development - getting a steady flow of quality Kent now admits that he is not talking about machines or manufacturing - but instead about the design/development process, and all the people providing services (analysts, designers, etc). For the last thirty years, we have been pushing for more inventory (documents) Even in manufacturing, little to no inventory is now the goal Refutes commonly held assumption that XP does not involve design - it just attempts to reduce the inventory Work in Process Inventory

-- Unrecoverable investment (Opportunity cost)
---- Types
	Code
	Tests
	Design decisions
	Specifications
	Documentation
	Defects
-- Detail not yet in production

(Jet lag caught me here - dozed off for about 10 minutes or so)
Very, very difficult to accelerate your development schedules, based on developer expectations alone new style of marketing, sales, customer service, and engineering
what if you have a website, and want to push a release every two weeks?

1. Have to move towards a more continuous release process - one without
2. formal code freezes
C3 - one defect in 9 months of bi-weekly releases LifeWare - What if you want to release DAILY?

3. Ultimate in YAGNI - no code in system that hasn't actually been used by a real customer - workflows not requested have not been implemented
4. You HAVE to believe that your tests actually cover everything
5. Have had one rollback in last two years

Standish Group - 85% of features in typical IT systems are never used. The key is to pick what you don't implement, and you go 7X faster Inexperienced teams will have a larger inventory, since they haven't learned what they need to know yet. "Does XP work for inexperienced teams? NOTHING works for inexperienced teams." "How do we use analysts (experts) to help inexperienced developers? XP has many feedback loops, and test-first is the first one. Pair programming is another" This builds experience quickly by constantly improving skills "Do programmers on XP projects need experience to accomplish anything? By definition, half the developers are below average. You will have this issue regardless of the methodology in use" "What good is the coach? Akin to a sports coach - keeps the team moving and motivated. Not having one is no excuse, but it helps "

---

Part Two, after a break Questions from the audience

1. XP with non-OO?
2. XP and Open Source?
3. How to estimate cost
4. CMM and XP?
5. how and when to trust the customer
6. documention with XP
7. uml and XP?
8. client commitment
9. stories and requirements management
10. overview of XP
11. timeframes for XP projects
12. what is value to customer
13. config management
14. cost estimation for XP projects?
15. how much planning is enough?
16. Gradual implementation of XP?
17. XP around the world?

Addressing the questions directly, not going from slides at this point XP elevator speech: - it's a long elevator...
"Imagine if software development were as simple as magic 3x5 cards. When I write something down on the cards, it should take me only a week to implement each card. First thing - given a ship date. I say - we have 12 weeks of work. You say - We have 8 weeks. I say - which 8 things would you like then? Picks the absolute minimum (9) things. I say - sorry, which 8 do you want first?" We call this a satisfied customer. Every monday, team gathers. The coach asks - What do we want to see next? Team says, "these two". Coach says, pick one. Repeat the next week. Discard new ideas that come up, pushing them back to future. XP is your best effort applied to the customer's greatest needs. Continual give and take between parties as to what we can and cannot do in a reasonable time. "My question - how do you deal with big failures, politically disfficult to kill?" Very hard, usually fail. example - 5 years, $100 million Euros in infrastructure, and no software of value yet. Company is in trouble. Future value has with what you have invested in it thus far. CMM - Copy of the manufacturing maturity model for physical development. But this doesn't correlate well with making money (inventory issue). Increases the volume of documents (inventory) and creates a floor - you cannot go any faster than CMM allows CMM - "How can we write more documents in order to make things run more smoothly?" i.e., increases inventory, while XP attempts to decrease inventory short answer - CMM and XP are incompatible. CMM is about repeatability - but in software, we are always building something different. i.e., repeatability is not that useful "The team owns it's own process" we move people between teams in order to harmonize UML and modeling - Rose (etc) take a long while to start, involves a lot of context switching. UML is also very complex - recommends a simpler method of whiteboard modeling "GML - galactic modeling language ;-)" Line, Box, Label. What more do you need? Semantics - whatever you mean when you draw it Maintenance and UML - NO. Use code as the communications mechanism - and that includes the tests. gradual implementation - ask the team what inventory can be dropped by working backwards through the build/dev process Puzzle - In India (and other low labor cost areas), high tech risk management did not apply, since labor (developers) was so cheap. Wages being low, competing against low wage people doing CMM. Is the higher productivity swamped by the sheer wage differentiator? Can a 10x productivity boost still matter?

All in all, it was a good talk, and I enjoyed it a lot. The local audience liked it as well, and asked many questions once they got warmed up.

I'm on my way to the XP show here in Sao Paolo - if I have network access, I may have some comments from the show. It should be interesting - I speak Friday, so I have plenty of time to watch other speakers and get a feel for the show first.

This story make a lot of good points I agree with. Sun managed to deal with MS every bit as well as Brer Rabbit's enemies dealt with him. No, no, don't throw me into the briar patch...

Sun spent several years in litigation with Microsoft for purposes of nullifying Microsoft's Java license. Vast sums of money (far more than $20 million) were spent on legal fees for purposes of trying to prevent Microsoft from using Java any longer. Sun declared a victory. Now Sun, having gotten a settlement forbidding Microsoft from further work on Java, is back in court. It's suing Microsoft again. It wants a court order forcing Microsoft to include the Virtual Machine in XP distributions.

anyone else getting dizzy from all the twists and turns here?

My hotel here in Sao Paulo has a broadband connection, so I'm able to get on with decent access. I even got some sleep on the way down - the miles needed for a business class seat were well worth it. I'v got some interesting stuff on stagnation in the Java world - the JCP specifically. but I'm waiting for my source to pass me a live link - all I got was an html doc. Here's an excerpt though:

Even as IBM releases its latest version of its J2EE-based Websphere application server with several key supports for web services, a key IBM Websphere exec says it's tough for J2EE vendors to push the web services envelop because Sun and current JCP (Java Community Process) rules are "blocking innovation."

for the next week. I am headed to the XP conference in Brazil, and I have no idea what kind of network connections to expect. I'll have my wireless card; gosh knows if the show is big enough (like OOPSLA) to provide wireless connectivity. So if you see no postings for awhile, you'll know why...

Take a look at this CNET story

"The U.S. Defense Department should think twice before embracing open-source software, a trade association is advising. The Initiative for Software Choice, which counts Microsoft, Cisco Systems and Intel among its backers, said in comments filed Tuesday that the department should 'avoid crafting needless and potentially detrimental IT policy to promote the use' of open-source software. 'Open source' means every software developer can view the source code for software, modify it, and use it for free. The initiative, which launched in May and is chaired by a group called CompTIA, an organization that has close ties to Microsoft, is worried about a recent report that concluded the Defense Department relies on open-source software and recommended its further adoption..."

and then take a look at this from the Register

I am currently doing some new media projects in an emerging market country. As one of the reasons I was sent there was that I care about cost, I decided to develop the projects based on Open Source. The main reason for choosing Open Source software was:

1. Licensing Cost for Server Software
2. Openness, i.e. the ability to change software to fit our purpose
3. Security & Reliability and (last not least)
4. Low hardware requirements.

The fourth reason was very important as I didn't want to buy any new hardware for the servers and instead reuse existing old hardware and extend its lifetime by using Open Source Server software. We decided to Use FreeBSD, Apache, mySQL+PostgreSQL, Perl+PHP The company I am working with is a pure-Microsoft company, i.e. they only used to use Microsoft software, and they even didn't know anything about Open Source. It was a painful but successful transition. But this is not the reason I am writing. The reason is Microsoft itself. When the local Microsoft rep "heard" (someone inside the company tipped them off), they asked to meet my team(!) and discuss the reasons for our Open Source use.

That much attention means that the industry powers are worried...

But I found this rant amusing. Someone break out the Prozac!

Analysts and industry pundits all have the same blinkered view of software development. They say that no one should be doing Smalltalk, because everything will be either Java or .NET in 12 months. That's funny, because they were saying something awfully similar (but without .NET) 2 years ago. In this commonly held view, I presume that all the PERL, PHP, Ruby, and Python development are a mistake as well, since they aren't part of the mainstream. But hey, what's a little fact checking on usage statistics and actual utility to developers if you have the right chair? Why bother thinking about the 70% failure rate of Java projects - as reported by Gartner - at all, so long as you tell people what you think they want to hear? The main reason the analysts give for using Java is that everyone else uses it. Well. What did your mother say about things everybody else did?

Along with a number of others, Alan Knight was interviewed for this book. Check it out.

Update

In case the link doesn't make it clear, this book is in German

You'll want to see this comment from one of the earlier postings on the "analysts". It's absolutely amazing that anyone takes these people seriously

I got news of this lovely bit of prose from the Forrester folks, under the title: Commentary: An app server winner? Not yet

Someone should buy Borland. A couple years ago, BEA acquired Java tool VisualCafe, but did little with the product. At that point, Borland grabbed a healthy share of the Java development community. Its tools now give J2EE developers an easy way to bridge J2EE and .Net development, and although all the application servers on the market integrate with Borland's tools, vendors don't optimize its tools for their platforms.

Never mind the absence of fact checking; who actually okayed that prose? The whole article may be found here. I guess the question end users of these "analytical reports" should ask themselves is: If you can't trust the facts presented by the analysts, why should you trust their conclusions?

Maybe we should start charging? Tip of the hat to Joseph Pelrine for pointing out this clone of the Camp Smalltalk efforts.

Update

The links have been fixed....

You'll want to look at the comments from David Buck to my previous post. One of the fascinating insights into the state of Smalltalk technology from the analysts:

"There are one or two reputable vendors that still support Smalltalk (e.g. IBM) but they do not and will not introduce significant enhancements, recognizing the niche status of Smalltalk."

What this indicates is a failure on the part of said analyst to do even basic research - he obviously spent zero time even with a web browser, instead taking their information from within their own echo chamber. Within a few weeks of these brilliant comments, Cincom had released VW 7, and IBM had released VAST 6. Meanwile, Object-Arts had put out a new release of Dolphin, and the MT guys were busy putting out development builds of their latest work. And that doesn't even get into all the fascinating stuff that goes on over in Squeak, or in dialects like Smalltalk/X, which is mostly used in consulting projects. Have a visit to Why Smalltalk for a rundown of all the links I've managed to omit here. Apparently, being an analyst meeans never having to fact check. You might want tp bring this up the next time you speak to an analyst, and see if they can explain their research "methodology" to you. Make sure to use small words...

The analysts amuse me. They write stuff like this:

MUNICH (COMPUTERWOCHE) - If your company is just going to decide whether to chose Sun's Java or Microsoft's .NET as development architecture then according to Gartner then it is appropriate to ask whether either of them works at all. About 70 percent of all initial Java implementations have failed up to now the researchers say. "An immoderate number of big Java projects failed", said Mark Driver, Research Director for Internet- and E-Business-Technologies. Driver continued - Microsoft shouldn't draw positive conclusions for their own .NET from that. The failure rate for early implementations of the Microsoft architecture will most likely be as big. "The only practical possibility to decrease the risk (of a failing implementation) is to outsource the development", says Gartner.

So, Java and .NET lead to 70% failure rates, and outsourcing (a trend Gartner has been touting for years) is the only way out. Then they say:

Despite the problems of the Early Adopters the institute anticipates, that up to the year 2005 the decision about the Enterprise Architectures has become a head-to-head race of the two rivals J2EE (Java 2 Enterprise Edition) and .Net. Both shall control in three years about 40% of the market each. "Most of the bigger companies will have both platforms", expects Driver. "They have become de-facto-standards."

So - even with massive failure rates - Gartner just blissfully goes along, and tells people to move off things like Smalltalk, and go to things like Java and .NET - knowing full well that 70% of the people taking this advice will fail These are the people that C level folks listen to? Why????

What this might mean for smalltalk (if anything)

Download this patch. Automatic delivery of patches has a small bug, or I would arrange to have this delivered automatically. Once we get the next release out, patches should be delivered that way. For now, just save the file above into the BtfPatch directory, and the system will load the patch at startup.

I still see a number of messages - in email and in newsgroups - that go something like this:

We are using VW 2.5.2 for our application, and would like to use the newer features of VW 7. How do we go about upgrading?

Well, start by reading here. I went through awhile back and documented the changes between various versions of VisualWorks, and posted the results. More usefully, I posted the scripts I used - you will probably want to run the chnage scripts yourself, in order to get any specific changes you made (I used vanilla VW images in all cases, with only the pacakges loaded that were defaults in 2.5.x). It's a lot easier to upgrade than you think - this is Smalltalk, not Java!

Before anyone else is awake (yes, we do sleep in in this house!). I doubt I'll be blogging much today; there will be too much cooking and eating later. So Happy Thanksgiving all!