Smalltalk Tidbits, Industry Rants

The World Series is taking place during the business day for me - but I can at least follow it all via web updates. Go Yankees!

The Massachussetts state government has adopted an open source/open standards policy for IT purchases. Massachussetts was, as I recall, the last holdout state in the MS anti-trust suit, so this is another shot across MS' bow from the state. Watch this whole story unfold nastily over the next few years - and watch politics drive it more than technical merit (and that would be different from the rest of the IT sector how, I wonder?)

ComputerWorld has a spread on MySQL takeup - looks like the little database that could is getting noticed:

MySQL is also upsetting the entire database market. Charles Garry, an analyst at Meta Group Inc. in Stamford, Conn., hails it as "a disruptive technology" that's commoditizing databases"so much so, he says, that "the future of the database market will be the standardization on MySQL."

Strong words, but adherents of the open-source database are passionate supporters, and they number in the millions. These users are drawn to it because it offers high performance, ease of use and a feature set broad enough to handle most of their database development needs. And it's cheap.

Indeed, MySQL's low cost never fails to come up in conversation with users. Mark Cotner, manager of network application development at Cox Communications Inc. in Atlanta, points out that his MySQL-based application cost less than $90,000 from soup to nuts, including the Intel-based servers, programming time and the approximately $4,000 annual license and support payments to MySQL AB, the Uppsala, Sweden-based company that oversees the development and distribution of the open-source database. An Oracle database license for the project would have totaled $300,000 by itself, he says.

I suppose this means that we (Cincom Smalltalk) should take a more serious look at MySQL - although there is a driver for it in the public store. This has got to be the best set of quotes in the article though, first from MS, and then from MySQL AB (the company behind MySQL):

Yet, despite MySQL's progress in the market, "we haven't found very much MySQL out there," says Microsoft's Rizzo.

"That's the best news I could have," retorts Mickos. "As long as Microsoft is in denial, we're fine."

They could both be right - so far.

Nicholas Petreley is usually off on an anti-MS rant; he used to be an OS/2 stalwart, and he's now a Linux backer. I've mostly gotten tired of his columns, but this week's ComputerWorld column is pretty good. He's got this one right; Sun is a deeply confused company right now.

So I'm stuck on this long flight (almost 5 hours left to go now), and I'm catching up on my journal reading - I'm reading the SDTimes columnists, and I come across something interesting - in an article by Steven J. Vaughn-Nichols, I read this:

It (Jackpot) is good, but there's one problem with Jackpot - The program, which was supposed to work with Java in general, hasn't been updatd since the fall of 2002. I fear, like so many other debugging efforts, that it's been placed on the back burner

Well. There are, in fact, great debugging tools available - you just have to use Smalltalk in order to use them. Debuggers for Java aree forensic tools; in Smalltalk, they are surgical tools. Yes, I know there's some "fix and continue" support in Eclips, and in MS' VisualStudio now. I also know, from talking to people that use it - that it's nothing like what you get in Smalltalk. The Smalltalk debugger is a cod browser that lets you step through (and edit) code.

The point about Jackpot being dormant is interesting - there was a lot of buzz about that about a year ago from Gosling. I've always thought that the reason Smalltalk style tools fail to show up in the C language family is that they are so much harder to create in the presence of static typing and weak reflection support; maybe Gosling got tired of pushing the rock uphill. anyhow, here's the thing that popped at me in the column:

Enough is enough. It's time to make debugging code job No. 1. People have put up with bad programs for far too long

He's right - and the answer is over this way

There are some interesting comments on the .NET Guy's blog about the playoffs and world series:

Too bad Fox won't see those ratings in the World Series now that the Cubs and the Red Sox have been eliminated. Quite a few people cared enough to tune in to the *real* reality programming taking place live on their screens. No scripts. No setups. I watched Game 7 NYC vs. Boston. It was a thriller down to that final Boone HR. I would have definitely watched a Cubs vs. Red Sox World Series.

I'd be watching, except I'll be in Tokyo - and a half day off cut from the games - they'll be playing during business hours where I'm going. Just deal with it gang - the Bosox have a monkey the size of a small country on their backs, and the ghost of the Bambino is holding it there. In the meantime, the Yankees should do what they usually do - win.

Tim Bray thinks that browser based UI's hit a sweet spot. Scoble disagrees. I disagree as well. Browser based UIs are functional at best. In general, they stink. A lot. Go use browser based email for a day, and tell me I'm wrong. What the future holds, IMHO, is not one of browser based apps. Rather, it's one of network enabled clients with rich user interfaces - so long as those network enabled apps are capable of dealing with intermittant connectivity.

As I write this, I'm sitting in a cramped coach seat on my way to Tokyo. It's a long trip - direct from Chicago. Fortunately, I have seatback power - which means that I'll have something to do with all this free time (I have a bunch of books with me as well). I have no idea what kind of connectivity I'll have in my hotel either. I could be posting to myself for awhile....

Ted Leung has a post up where he points to an online history of lisp and the lisp machines. Worth looking at

If you want to develop and ship faster, then listen to Keith Ray. Here's the tip that popped at me:

Don't use the conventional solutions that everybody else uses - use the tools that can allow developers to go up to seven times faster. For example, use Smalltalk (see also Cincom) or Python or Cocoa instead of Visual Basic or Java or C++. Use WebObjects instead of EJB. If you do use Java, use Eclipse or buy IntelliJ IDEA.

If you use what everyone else uses, how do you expect to get a leg up?

If you have been using the development stream of BottomFeeder, and have tried using the new global filter capability, you may have seen some odd behavior. If you set the filter such that only new items showed up, and then deleted that filter - none of your old items came back. I made a mistake in the filter redefinition code such that I wasn't accurately resetting the filters. I had at least one report of "disappearing feeds", and then I saw the problem myself. The latest dev feed fixes the problem - if you have disappearing feed items, define a new global filter, save it, then delete the global filter, and save that - after grabbing the update.

So I'm reading this story about WinFS on CNet, and come across this quote from an MS VP:

NTFS is only one component of the revamped storage system in WinFS. Another key building block is the querying capabilities of Microsoft's SQL Server relational database, according to Microsoft. WinFS also will incorporate the data labeling capabilities of Extensible Markup Language (XML), Muglia said.

"Think of WinFS as pulling together relational database technology, XML database technology, and file streaming that a file system has," he said. "It's a (storage) format that is agnostic, that is independent of the application."

So reading between the lines there - does this mean that Longhorn will ship with an embedded SQL Server - and that all file system activity will generate database records (which would then facilitate searching)? That's what it sounds like to me. I could be wrong; it's just the first thing that popped at me. One things for sure - if I'm right, that will require larger disk drives. I wonder if you'll be able to turn that off?

I'm off to Tokyo tomorrow - the Japanese sales team has set up a number of customer and user group meetings for me. I've never been to any part of Asia, so it should be an eye opener. I leave tomorrow morning, and the time change - like when I traveled to Australia a few years ago - is sure to be a big adjustment. Look for my posts to come in at odd times of day for the next week.

After my recent post on the Yankees victory, I got this in email on the whole curse thing:

IMO, the goat curse is bigger. The Bosox' problem is not only their curse, but the fact that they play in the American League, which means they're always going to have to go thru the Bronx to get to the w/s. Plus, the fact that they play in the East, which means their only chance to get into the playoffs is as a wild card, which means they're never going to have home field advantage in ANY series. The Cubs, they have no such issues. They play in a dog**** division in the dog**** league. They're just screwed.

There's the expert analysis of the aituation...

This is mildly amusing. The internal name for what MS now calls COR (Component Object Runtime) was once COM3. They changed that before reasing. Why? Don Box explains:

On at least one version of Windows, if you created a directory called COM3 you could never delete it (COM3 is a reserved name for a serial port).

That's a legacy installed base issue raising its head for you.

The agony in Boston is going to be exquisite tomorrow - The Yankees pulled a game out in the 11th that Boston clearly should have won. Grady Little made a huge error in the 8th when he left Pedro in to face Matsui - Embree was ready - and Matsui doubled. The momentum turned then and there - you could pretty much tell that the Yankees were going to win from there. Interestingly, I was talking about this with a friend in engineering yesterday, and his comment on the Sox win last night was that they were just prolonging the agony, making sure that it would hurt as much as possible. Well, if you're a Red Sox fan, that's pretty much what happened tonight.

Now, the only question left is - which curse is bigger - the goat, or the Bambino?

I have to say that I agree with Danny Ayers about OPML. I spent this morning fixing up the OPML support in BottomFeeder. I'm not going to do better describing the problems than Danny did; go read his

Keith Ray weighs in on the whole exception business with some useful insights

I had a complaint that BottomFeeder wasn't properly handling OPML based feedlists, so I took a look - sure enough, it wasn't handling folders (embedded outline elements) at all. I retooled the code for that this morning - the dev stream now supports OPML based feedlists.

This post shows that, without a doubt, Joel just doesn't get exceptions.

Keith Ray posts some thought provoking analysis of outsourcing - relating it to the movement of the garment industry and the manufacturing job losses in the US. The whole article is worth reading - here's one of the quotes keith pulls:

For example, see Johanna Rothman's blog: "I'm convinced that the reasons outsourcing works is that it forces organizations to document requirements and the outsourcers work on only one project at a time. The outsourcers' management can then choose any number of useful product development practices that increase the outsourcers' productivity. Management can't change their minds and refocus the outsourced project(s) in the same way they feel free to refocus the internal projects.".

The problem is lack of focus and lack of productivity - outsourcing works because the contracts signed at least force focus (even if they can't force productivity). Since IT shops have historically lacked focus and productivity, gaining one of the two at a lower cost looks like a great deal to CEOs. Here's another thought that has resonance in our industry:

With that kind of example, the early adopters tried it and found that it solved a lot of endemic problems. In the last two decades it's become the 'heads up' thing to do in manufacturing, but only now is the effect on inventory turns becoming aparent in the US national statistics.

And that is without full participation from the industrial establishment. If manufacturing was fully Lean, you wouldn't see jobs going overseas. It's impossible to operate a pull environment when one of your processing steps takes a month to transport goods by ship. That's old Henry Ford / Frederik Taylor thinking.

Now relate that back to development. How easy is it to deal with changing requirements in an environment where the developers are 12 hours distant (timezone) and a day's plane ride away? Where the language and cultural barrier creates easy misunderstandings? If your locally based developers were actually doing the right thing, then outsourcing overseas would clearly be slower and - over time, from an ROI standpoint - more expensive. The problem is, they don't do the right things. They follow language and platform fads. They reject changing requirements. They diss users, speaking of them with contempt.

And then, they are astonished - just like the auto factory workers with their byzantine work rules - when the jobs migrate somewhere else.

I've written about this post from Joel twice now - here and here. Lots of other people (here, for instance) put up their thoughts as well. I expected a response from Joel, but I was kind of surprised at what he wrote - have a look at yesterday's post on the subject. Basically, he opts out of the issue:

There's no perfect way to write code to handle errors. Arguments about whether exception handling is "good" or "bad" quickly devolve into disjointed pros and cons which never balance each other out, the hallmark of a religious debate. There are lots of good reasons to use exceptions, and lots of good reasons not to. All design is about tradeoffs. There is no perfect design and there is certainly never any perfect code.

The above is true enough, but it's a way around the discussion, not really an entry into it. Disappointing.

The Red Sox continued to see the curse and the cubs were still haunted by the billy goat today. The Yankees put away another game this afternoon - and there were plenty of oddball infield hits and errors in this one. Meanwhile, the Cubs had a breakdown worthy of the '86 Sox. With one out, a fan grabbed a ball that would have been an easy catch (and out 2). Right after that, an error at short extended the inning. The Marlins managed to score 8 runs, in what had been looking like a cruise towards the Cubs first series since 1945. Looks like the fans who wanted a Cubs/Sox series are going to have to wait - looks to me like the Yankees will get there from the American league, and the NL side is completely in flux.

Blaine Buxton likes Seaside, and talks about how easy Smalltalk servers are to work with:

Oh well, I just wanted to write about how blown away I have been. As a side note, I've only had to stop the web server once since I started it (and it was because of a mistake I made)! Of course, to restart the web server takes 1 second (I kid you not...try that with tomcat or apache). I've been changing the code while the server is running with no special cases or what have you (try that in Java...Yeah, I know about hot swap--it only works for simple method changes).

Productivity, anyone? Stick with that Java stuff if you want to develop and deploy 3x later....

I think I'm going to get BottomFeeder 3.2 out fairly soon - and this will be the last release on the VW 7.1 engine and image. When the next development cycle starts, I'll be using VW 7.2 - which means that users are going to have to grab a new base image and VM before they can go to that (future) release. The Upgrade tool is already aware of VW versions, and won't report versions for newer VW bases as valid. I'll be updating this as things move forward

The Cincom Smalltalk distribution contains an ever increasing number of goodies - add ons submitted by various people. There are ones organized as internal - i.e., ones created by our own staff, but not considered (for a variety of reasons) to be part of the product. Keeping those in synch with new releases is (theoretically) easy. Then there are the rest of them - the goodies submitted by our users. There are a huge number of such goodies, and making sure that what we have on the CD is in synch with the release is a difficult problem. There's a stated process for it, but I've gotten complaints about that. Many authors now manage their packages in the Public Store, and have expressed the opinion that managing an additional level of versioning (via ftp) is onerous. On the other hand, asking Cincom staff to figure out what is and isn't new is a recipe for disaster as well - we just won't do a good job of it, or we won't notice.

The idea has been floated to build scripts to generate parcels from the public store, but there are problems with that -

• It's a database, not a server
• Unless the package was published binary, it's not even theoretically possible to generate a parcel from the db w/o loading the package first

That argues for one of two things - a Store server (hard, and not something that would appear overnight in any case) - or a small tool that would offer to publish a package to goodies. The idea would be that you would have ftp space on our server, and a small tool would be used as an interface to uploading the goodie to the correct place. That's simpler than the server idea, and - I hope - not too onerous for goodie authors. The tool would be able to push up either a single parcel or some specified archive file (for cases where a .pcl and a .pst aren't going to be enough).

So here's the question - would such a tool be useful? Would goodie authors use it? Would it make the whole submission process easier to deal with? Feel free to answer in comments or via email

Charles Miller has a useful retort to The Register's rant on trackbacks. Worth reading.

The Register reports that blog trackbacks are having unintended (and nasty) side effects on the google page ranking process. Here's the gist of it:

A "Trackback" is an auto-citation feature that allows solitary webloggers to feel as if they are part of a community. It's a cunning trick that allows the reader to indicate that they've read a weblog entry, or as the official description from MovableType has it: "Using TrackBack, the other weblogger can automatically send a ping to your weblog, indicating that he has written an entry referencing your original post."

The original blog then sprouts a list of "trackback" entries from other webloggers who have read, and linked to the original article. Kinda neat, huh? Except for one unforeseen technical consequence: the Trackback generates an empty page, and Google - being too dumb to tell an empty page from the context that surrounds it - gives it a very high value when it calculates its search results. So Google's search results are littered with empty pages.

Try this OS X Panther Discussion for size: it's a Google query for OS X Panther discussion. In what must be a record, Google is - at time of writing - returning empty Trackback pages as No.1, No.2, No.3 and No.4 positions. No.5 gets you to a real web page - an Apple Insider bulletin board. Then it's back to empty Trackback pages for results No.6, No.7 and No.10. In short, Google returns blog-infested blanks for seven of the top entries.

Wow. There are unintended consequences for lots of things, but that's an interesting one....

The Register reports that France is putting WiFi on their TGV trains. In the San Francisco bay area, commuter rail is getting wireless. Want to take bets on which millenium will see similar things happen on Amtrak - say in the Northeast corridor, where it would truly be useful?

The more I read this post by Joel, the worse it looks. And mind you, I wasn't impressed during the first read. I just noticed that Mark Derricutt didn't think much of the "no exceptions" idea either. So let me give a concrete example of how and why I don't agree. Here's a longish snippet from the update loop of BottomFeeder - above this is the loop, the snippet is from the feed, at the point where it's trying to update:

safelyParseAndProcess: anUrl into: anObject forceUpdate: forceUpdate 

	[| cache |
	self doFeedUpdateWithForce: forceUpdate.
	cache := HttpClientModel cacheAt: anUrl.
	(cache notNil and: [cache hasMoved]) 
		ifTrue: [self modifyMovedFeedFrom: cache.
				self doForcedFeedUpdate]]
		on: self xmlExceptions , Object errorSignal
		do: 
			[:ex | 
			HttpClientModel removeCacheFor: url.
			[self doForcedFeedUpdate]
				on: self xmlExceptions, Object errorSignal
				do: [:ex2 | ex2 return]]

Now, the relevant message send here is #doFeedUpdateWithForce:. That method (optionally) does the Http query (if it's time, based on etags, etc). The result of that query is presumably an XML document. Now, I handle the HTTP errors at the point of the HTTP query - mostly they are ignored, unless they are interesting - a 304 (not updated), a 301 (moved) - most other errors are just ignored and presumed to be transient issues (there's slightly more to it than that, but it's not important for our purposes here).

Notice that I catch the XML errors here. Why is that? Well, I've made changes to the parser itself - it ignores rafts of invalid feed issues in an attempt to be "liberal". At this level though, failure to parse means one of two things:

• Either the XML is hosed, and there's nothing we can do
• The document returned was actually not xml (likely an html error page of some sort)

Notice how I try the query a second time on XML errors? What I do is try the query again masquerading as IE, and specifically not asking for mod gzip. Testing has shown that, for whatever reason, looking like IE yields a much higher rate of success. I've also stumbled across encoding issues (both in VW and from feeds) that prevented the proper handling of gzipped feeds. Which is why it's caught at this level. Caught here, the system can make a rational choice about what to do. At the level of the http query or xml parse, those modules have no idea what the higher level application code is up to.

Say I followed Joel's advice. I'd have to make sure that every possible execution path handled and returned error objects, all the way down. That's just stupid. It would make the code very brittle, and highly resistant to change. The way I've done it, the http level or the rss parsing level just toss exceptions, and leave it to the application layer to deal with those. Intermediate levels of the call stack neither know nore care. In Joel's scheme, I end up with nasty case statements littering every single level of the call chain. In the exception scheme, there's the toss, and then the application layer with enough information to catch does so. The error passing scheme leads to baroque code that rapidly becomes brittle. Just say no

Joel Spolsky seems to simply not get it on exceptions. This is somewhat surprising; I really like most of what he writes. In this case, he's just not right, at all. Here's where he starts:

People have asked why I don't like programming with exceptions. In both Java and C++, my policy is:

• Never throw an exception of my own
• Always catch any possible exception that might be thrown by a library I'm using on the same line as it is thrown and deal with it immediately.

The reasoning is that I consider exceptions to be no better than "goto's", considered harmful since the 1960s, in that they create an abrupt jump from one point of code to another.

Hmm. Loosely coupled code, anyone? Sometimes, you have exceptions at a low level in the application that really can't be dealt with, unless you are at the UI level. Here's what Joel says:

• They are invisible in the source code. Looking at a block of code, including functions which may or may not throw exceptions, there is know way to see which exceptions might be thrown and from where. This means that even careful code inspection doesn't reveal potential bugs.
• They create too many possible exit points for a function. To write correct code, you really have to think about every possible code path through your function. Every time you call a function that can raise an exception and don't catch it on the spot, you create opportunities for surprise bugs caused by functions that terminated abruptly, leaving data in an inconsistent state, or other code paths that you didn't think about.

Hmmm. Everything he says here about exceptions is true of events as well. Are they evil? Are they to be avoided? After all, a piece of code may not know that it will get interrupted by an inbound event. So what is an exception? It's an application error event. That's what it is - nothing more, nothing less. What's Joel's answer?

A better alternative is to have your functions return error values when things go wrong, and to deal with these explicitly, no matter how verbose it might be. It is true that what should be a simple 3 line program often blossoms to 48 lines when you put in good error checking, but that's life, and papering it over with exceptions does not make your program more robust.

Bleah. I've seen code written using that theory. It very, very quickly becomes an unmaintainable nightmare, and has errors being propagated from deep in the bowels of the application up to a level where they can be handled. This is clean how? Maybe the problem is that exception handling in Java and C++ sucks - in Smalltalk I can do something like this

answer := [self doSomeThing that Calls ManyLevelsDeep]
	on: SomeException
	do: [:exception | exception isResumable
		ifTrue: [exception resume]
		ifFalse: [self reportError: exception]

So what will that do? It will resume the exception (i.e., resume as if the exception never happened in some cases, and report the error in others. It's compact, and it's easy to follow - and it has the benefit of avoiding a whole bunch of checks on whether or not I got an error throughout the call chain. In other words, it makes the code easier to read and easier to maintain. Joel's way makes the code crusty, complex, and hard to follow. It puts error handling code up and down the call chain in places it has no business residing. What Joel is advocating is writing code that misplaces responsibility - very bad form. I don't usually disagree with him, but on this, he's just wrong. A lot

Gordon Weakliem's old blog is here, and his new one is here. He wanted to redirect his rss feed, but Radio doesn't support issuing 301's. So he was trying to use this RSS level redirection, but found that few aggregators supported it. Well heck, I didn't even know that existed! So I sat down and added that support, just now. It's only in the dev stream of BottomFeeder, but it will be supported in the 3.2 release.

Gordon Weakliem states that BottomFeeder doesn't support this redirection feature. Hmm - Bf supports 301 redirects (Http level), but it seems Gordon can't do that - Radio is broken that way. I'll have a look and see how hard it would be to add support for the feed level redirect....

Eric Sink lets the world know that his company's source tool (Vault) supports the generation of RSS from checkins. I suspect that this will become a demanded feature as time goes by - the feeds on our internal source db and the VW Public Store are highly useful - it's one of the best ways to track activity!

Tim Bray has an idea for killing spam - it's the pay for email variant (each message costs some small amount of money to send). The only problem with the idea is that - as with most such ideas - it could easily be defeated by some unscrupulous set of offshore operators. All it would take to get around it is someone willing to charge less than the typical rate and willing to accept spammers. He has the whole digital signature and certification idea for working around that, but I don't buy it - I just don't see all email systems going to this model simultaneously, and that's what it would take. In the meantime, some of his anecdotes explain why I still don't use a spam killer - I'd rather hand delete the stuff than try to figure out what I've lost.

I got a complaint about BottomFeeder's handling of network errors yesterday - specifically, the way it deals with getting timeouts from all (or most) of the feeds during an update cycle. The way it handled this before was to just ignore it, treating it as (yet another) transient network issue. However, some kinds of ISP issues can make this a painful choice - say your system acts like it has network connectivity, but all http requests yield timeouts. Given the time for any particular request to timeout, this can yield a very unresponsive application.

What I've done is added a counter - if 10% or more of your feeds are failing with timeouts, then the update process will be suspended, and you'll get prompted for what to do - ignore the errors and push on, or go offline. I also added a setting that allows the old behavior (just completely ignoring the timeouts) to take place - I often get transient errors that come and go, and I'd just as soon let them pass. The way Bf is now set up, that decision is up to you as the user

Charles Miller has some hot tips for screenwriters of action flicks. Seems to me that most of them should pay attention to what he has to say.