Smalltalk Tidbits, Industry Rants

Ed Foster explains why so many email newsletters get classified as spam.

How many teams have this guy?

I just noticed that this blog is the Feedster feed of the day. That's neat. I've been a happy user of Feedster for quite some time now - BottomFeeder was one of the first aggregators to support it. Half the reason I did that was for myself, actually - I have a number of feedster search feeds that I track in Bf.

One of the really nice things about Smalltalk is the ability to replace any method in any class - there's no concept of "final" or sealed". Typically, it's better to subclass an existing class and add the new behavior there (or use delegation). However, that's not always practical. For instance, in BottomFeeder, I need to do HTTP queries (to get the XML files that make up feeds). VisualWorks has HTTP client libraries, and the libraries have an easy to use API. I soon realized that I was going to have to create a wrapper library, because it wasn't going to be sufficient to do:

response := HttpClient new get: 'http://www.somesite.com/feed.xml'.

There were a lot of things I needed to deal with:

• Exceptions (Network errors, etc)
• Permanent redirection (update the url silently)
• Cookies
• Authentication (and keeping track of username/password)
• Conditional-Get
• Compressed content

and tons of other small details. The HTTP client library in VW handles some of this, and wrapping it to provide the rest at an application level was easy enough. There came a point where I ran into a small problem though - I also use my wrapper library to handle POSTS from the blog tool that comes with BottomFeeder. Under some circumstances, an exception would be raised, and I'd have no access to the original HTTP response that was received - it had gotten nulled out by the library by the time my code got control again. Now, it's arguable that it shouldn't work that way, but hey - no library is perfect. Most developers aren't the Product Manager for the tool they end up using, so - unlike me - they can't go straight to engineering and ask questions. They have to go through support and maybe get a fix, someday. The nice thing about Smalltalk is that you can track down the issue yourself and deal with it in the meantime. I've removed numerous temporary hacks of this sort as the engineering team has dealt with them. Here's what I did in this case:

In the class HttpClient, there's a method #getResponse:. That's where the response is received from the server, before some post processing (and some of the exceptions raised here can result in the original response not being returned). Easy enough; I overrode the method, storing my version of it in an extension package that is part of my HTTP wrapper library. The part I overrode looks like this:

^[stream := self getResponseStream.  
	resp := request headerOnly
		ifTrue: [HttpResponse readHeaderFrom: stream ]
		ifFalse: 	[HttpResponse readFrom: stream.].
	 resp log. 
	self originalResponse: resp.
	(resp connection notNil and: [resp isConnectionClose ]) 
		ifTrue: [ self close]. 
	 HttpException handleResponse: resp
	]

The only addition is the #originalResponse: call. I also added an instance variable to the class so that I could store that information somewhere. My wrapper library looks there in cases where it receives a nil response, so that it can better report the actual problem that was encountered. Was this critical? Maybe not - I suppose I could have gone with just reporting a generic failure and been done with it. The point is, I didn't have to - I had the option of fixing what I perceived to be a bug. Why else is that useful? Well, it's not always the case that things you report to support get dealt with the way you would like. Some are put on the back burner, some are rejected as "not bugs". The point is, being able to muck with all the code gives you the ability to route around that issue. Support doesn't believe you? Fine, you can fix it yourself. Support says it's a low priority? See the above. Support agrees that it's important, and says that it will be fixed in the next release? See the above.

With Smalltalk, your level of control over the situation is just higher. Now, do you want to start overriding the class libraries willy nilly? Of course not. This counts as a "power tool" - it's a really good and useful thing to have, but you want to be careful when you pull it off the shelf. The nice thing is, with Smalltalk, it's on the shelf, ready to be picked up at need

Advantage, or "blowing in the wind?" - Alan Trammel, manager of the Tigers, claims that the Twins are changing the flow direction of the A/C blowers based on who's at bat:

"We almost made a comeback then and had the issue of whether or not those blowers were on," Trammell said. "It seemed like those air conditioners were blowing straight in our face in the top of the ninth. There was definitely a difference in the air conditioner in the ninth inning. There was no question that there was some air blowing in the ninth inning."

The Tigers said a flag in the left-field upper deck was flapping as though it were windy Sunday, and they saw a piece of tape stuck to a vent behind home plate that was fluttering while the Twins were hitting, implying that the Twins -- who scored all their runs on three homers -- were being aided on offense and defense.

The Twins, of course, deny it:

"It borders on the ridiculous and the absurd," Twins vice president of operations Matt Hoy said. "It's a romantic concept that we can materially affect the flight of the ball, but it's just not possible."

Ah, baseball controversies :)

You've likely heard us Smalltalkers saying how extensible Smalltalk is. Well, have a look at Travis' latest post and see what we mean by that.

Check out the BottomFeeder Review in PC World Magazine.

I'll be in planning meetings most of the next three days; I expect that blogging will be somewhat light (unless I get incredibly bored, which is always a possibility in a 3 day set of meetings). Part of this will be talking about "where we want to be" in 2-5 years - and a large part of that answer needs to come from our development community.

The MacInTouch site covers the release of Bf 3.5.

I've said this before, as have plenty of other people I know. But over at Strongly Typed, there's a great explanation of why data entry applications suffer when people are moved from text oriented screen apps to GUI apps (web or client/server):

Have you ever watched an experienced heads down data entry clerk do their job? With a green screen system they rarely look at the keyboard and in most cases ignore the screen. They are responding to audible feed back (key press clicks, console beeps, etc) with their eyes focused on the data to be input. They remember key sequences (press 1, A, down arrow 2 times, F2 to save) to navigate. A mental model of the screens become ingrained in theirs head. They SEE the application in their head and that vision is updated real-time. They know the system because it's predictable.

In the cases I mentioned above the primary source for the complaints was the mouse. It's unpredictable. It's next to impossible to remember the cursor's position at anytime - even worse, to know the exact movement. The mouse requires visual contact with the screen, an impediment for heads down data entry.

As a developer I used to think I was empowering the users by creating flexible user interfaces. Multi window applications with buttons and icons; to take advantage of a modern GUI was to deliver the best we had to offer. When the complaints rolled in about the entry screens I dismissed them "growing pains".

It took me some time to realize their dilemma. They WERE better off with the original entry screens.

Something to think about....

If you don''t think that software patents are out of control, have a look at this - Microsoft has patented the notion of an auto-generated to do list. Better not get too fancy with those grocery lists...

Cyrus wants an image:

All I am looking for here is an API that allows me to create to save the state of execution at any point and then return to that in the future. If fibers aren't the way to do that, then give me something that will. Make it a nice supported API written by people who know the runtime well and understand all the pesky issues involved so that I won't run into a situation where 3 days before shipping a product some heinous bug becomes apparent.

Saving the state of a running system... Hmmm.. Where have I seen that feature before?

Still in CicadaVille:

So I came to Cincinnati this week for a set of planning meetings - leaving Maryland, which is currently filled with 17 year Cicadas. Imagine my surprise to discover that this brood comes up here as well - and in huge numbers! There were hundreds of the little buggers buzzing around the front wall of the Cincom building (all glass).

Adele Goldberg's and Kenny Rubin's book has gotten a new plug - good review and explanation of the subject matter.

It turns out that there is a silly (but annoying) bug in BottomFeeder 3.5. On the technical side, I transitioned from change/update to trigger events (in terms of event notification). However, in the process of doing that, I dropped one event by the wayside accidentally. As it happens, that was the event that toggles menus/toobars on a change from online to offline (or vice versa). The easiest way to spot that bug is to toggle the online state and watch how the toolbar doesn't change back. I've uploaded an update to both the normal upgrade path and the dev upgrade path to fix this; you'll need to restart in order to see this take effect. Of course, if you know how to dive into the running Bf application, you can just pop up an inspector, and execute this:

	| viewer |
	viewer := RSS.RSSFeedViewer allInstances first.
	viewer feedManager when: #onlineStatusChange: send: #onlineStatusChange: to: self.
	viewer onlineStatusChange: true.

At which point, it will work properly.

I'm working on two new things for BottomFeeder:

• A synthetic GUID for all items, whether they have one or not
• A Newspaper view mode at the feed level

The first one is a little technical. Atom items have an ID, and (most) RSS 2.0 items have a guid. These items are (in theory) unique enough to compare across feeds (although they likely shouldn't be trusted outside a given feed). In any event, not all items have them, so I've taken to lazily generating them using an md5 hash. I need to do some testing on this before I roll it out; I don't want to have the current "is it new/old" code cause issues for users, as it has in the past.

The other thing is a new view mode at the feed level (possibly at the folder level as well). The idea here is simple - say you turn this on (either globally, or for a given feed). When you select a feed, all of the relevant items get displayed in the html view as a set of "Newspaper stories". The relevant items would be either

• All new
• If no new, all today's

This is a view mode that some of the other aggregators out there support, so I've decided that it's a good idea to add. It's basically done (at least for the purposes of the dev stream, even though I haven't pushed it yet) - but I'm not going to add it as a non-dev feature until the 3.6 release. Any thoughts or comments are welcome.

Brad Abrams talks about the (supposed) conflict between developer productivity and performance:

Well, today I had a heated discussion with some of the smartest people on the CLR about the balance between developer productivity and performance.  The debate goes something like this: 

Me: Feature X will make developers more productive, so we should do it!
Other Guy: Feature X will make developers apps so slow it will not matter, they will use something else...

Anyone want to take bets on the liklihood that "Other Guy" actually tested anything out and profiled it? Maybe the feature in question would be too expensive - but the all too common approach to this issue is argument by assertion

One of Sun's bloggers has an eye for the obvious:

Where this starts to fall apart is with the .NET and Mono libraries. The Java API writers have always been very careful not to introduce an API which does not make sense on all platforms. This makes Java extremely portable at the cost of not being able to do native system programming in pure Java. With .NET, Microsoft went ahead and wrote all kinds of APIs for accessing the registry, accessing COM objects, changing NTFS file permissions, and other very windows specific tasks. In my mind, this immediately eliminates .NET or Mono from ever being a purely system independent platform.

In other news, I hear that the sun rises in the east, and that politicians in opposing parties often argue....

For the development stream, I've released a newspaper view for BottomFeeder. How does this work? At the moment, there are 2 settings - one feed level, one global. If you set the global, then you'll get the newspaper view whenever you select a feed in the tree. If you set the feed level setting, you'll get newspaper view when selecting that feed. What's that do?

• An HTML summary is created for:
• All new items for that feed
• If there are no new items, all today's items for that feed

The summary will show the title, the date the item came in, the category (if any) it has set, and the conten in a page. Each item will be set into this page view, and the result shown in a zoomed HTML view. If you need to get back to the list of items, you simply toggle off zoom mode and select any item of interest. This is my first cut at this, so any and all feedback is welcome.

With some feedback from a few early testers, I've made a few adjustments:

• There are global and feed level settings for
• Having newspaper mode on
• Showing full content for items in newspaper mode, or just a short summary
• I've modified the way the items are displayed to provide a little more information
• If you have the global newspaper setting on, and you select a folder, you'll get newspaper mode for all feeds in that folder.

This stuff is only available in the dev stream; it's going to require documentation updates, and, in my opinion, more shakeout. It should be a really nice addition to the next release of BottomFeeder.

I've got a few search feeds set up in BottomFeeder, and every so often it leads me to a real gem. Consider this bit of silliness, for example:

Second, he introduces us to a cool reflection tool called Lutz Roeder's .NET Reflector. However, he goes and says, "The .NET Framework introduced the world to the concept of reflection which can be used to examine any .NET-based code, whether it is a single class or an entire assembly." Are you joking? Java has had the Reflection API for years. They were the innovators there.

It gets better:

James also talks about the .NET build tool called NAnt and the .NET based unit testing tool called NUnit and doesn't even mention the original Java-based projects these bad boys were based off, namely Ant and JUnit respectively. Quite expectedly, he does mention Microsoft's upcoming MSBuild technology (which competes on a feature basis with NAnt) which will ship with Visual Studio .NET 2005.

I hardly know where to begin with this utter lack of knowledge. I suppose I could point to this paper on reflection in Smalltalk-80 - note the 1989 date on the paper. The Java guys were the "real innovators" here, eh? I think not. And JUnit was the "original" unit testing framework? You might have a look here - page down until you see the SUnit link, and read the description. These guys remind me of some of the people I went to college with - for them, if it was north of Manhattan, it simply didn't exist (or at least, it shouldn't have :) ). An awful lot of the Java community wears the same brand of blinders, and seems to believe that everything in computer science - and in OO in particular - was invented by Sun and Gosling. Sheesh.

I'm meeting with an important customer that is visiting Cincom in Cincinnati (from France), so blogging will be light to non-existant - I'm off immediately afterwards to the airport and home. In the meantime, try and wrap your head around this post on the nature of self from Steven denBeste. It's thought provoking, to say the least...

Over on Ralph's blog, the topic of sandboxing has come up. It's a great discussion, but there's one thing I don't always agree with:

The main bad thing that could happen is that imported code could change existing code. Smalltalk makes it easy to change code. "Obect compiledMethods removeKey: #copy" will remove the copy method from class Object. It would be very bad if imported code could do this to a running application.

While blowing away methods in base classes would be bad, blowing away methods in general may not be - in BottomFeeder, an update to the application can make major changes to base application code - and sometimes to overridden methods in various libraries. It's a complex issue.

Eric Sink makes an excellent point about positioning:

If you are #2 in your category, you want to be #1, right?

Wrong.  You can't choose to be #1, but you can certainly choose to be #3 or #4.  The worst thing you can do is to try and beat the #1 player at his own game.  Instead, realize that not everyone in the market wants to play that game.  Offer those people an alternative.

Me-too-itis won't ever widen your market. You need to give people a positive reason to look at you, not a laundry list of reasons that point out how you are "just as good as" the other guy.

Matt Croydon notes that Maryland (where I live, as it happens) is rolling out WiFi at visitor centers on the highway. That's very cool.

Ted Leung spots a couple of posts (here and here) that make an interesting assertion:

science is the modern world's new superstition

This is truer than many people would like to admit, especially highly educated folks. I saw this a lot when I was an undergraduate. There were various members of the faculty at MIT that would loudly proclaim their understanding and mastery on various aspects of natural science. I always viewed this with some suspicion, and I always appreciated the professors who would get to a certain point and say "and we just don't understand how it works here"

Very much the case. Watch coverage of various "hot button" political issues that look to science for their backing arguments (from any side of the spectrum; this seems to be an equal opportunity thing) - and watch for arguments you might call "Scientific Argument by Assertion", or "My Expert is better than Your Expert".

I've been working on things related to the new (dev only) Newspaper view. I made it possible to select a folder and get a newspaper summary of all new items (or all recent ones) in a feed. That was easy. The complexity came in when I decided to add folders into the mix. I currently track 264 feeds, so selecting the top level folder could result in a huge mass of HTML - too much to display reasonably (not to mention the memory impact :) ). So, I've added paging links (next/prev) into those views. That wasn't too bad - but I still have some consistency issues with the way the keyboard navigation works with the new functionality. Even so, I've pushed it to dev - that way, I have a better shot at finding out what's still broken quickly.

I'll tell you what's louder - a herd of 5th grade girls having a sleepover. More than once, I wondered how that much noise could be being created by only 4 children....

The blog comment spam problem doesn't seemm to be subsiding - have a look at this post from last week, for instance. This site - my blog and the community blogs really haven't been affected much - there's been a handful of spam over the last year, and it was in ones and twos. There's a pretty simple reason for that - I'm not using one of the popular blogging systems. Now, it's not that any of those systems are at fault - it's that due to their popularity, it's worthwhile for spambot authors to target the comment entry systems of these blogging systems - for exactly the same reasons that there are scads of Windows based viruses and worms, and virtually none for the Mac platform. The target eco-sphere for Windows (or popular blogs) is big enough to warrant bots, while the target eco-sphere of the Mac (or off brand blog systems) is too small.

That's why I get the occasional manually entered (and easy to get rid of) spam comment, but I don't get the absolute floods of spam (like the one mentioned in the link above). Is there a solution to this? Sure, there are plenty of technically feasible solutions (the simplest being to just change the field names in your templates). The difficulty is always in the side effects (common posting tools may no longer be able to "see" your site at all). The various IP based throttles are semi-workable, but blocking that way can be anti-social in the face of proxy servers. The medium term result is most likely going to be a steady decline in the number of blogs with open comment systems...

My wife recorded all of "5 days to midnight" for me while I was in Cincinnati - and for a change, a Sci Fi channel mini-series was ok. Timothy Hutton did a decent job with the lead, and I spent the entire 5 hours of the series thinking that Drew Barrymore had been cloned 10 years ago - the girl playing the daughter looks a lot like Barrymore in her childhood flicks. I expect the suckage to return soon though - "Post Impact" is the next Sci Fi movie "event" - and ooohh, "Decoys" looks cheesy :)

If you work in the software field, it's very easy to think that everyone is computer literate and has good internet access. Then you run across an item like this one in ComputerWorld, and you start to realize that you live in something of a bubble:

Support pilot fish is trying to explain to employees who are not office workers -- and have no computer experience at all -- how to view their new online pay stubs. "The built-in browser for a large Internet provider does not work well with the 'paycheck' Web site," fish reports. "The solution is to leave the ISP logged on and start Internet Explorer. I ask one user to do this, and he reports that Explorer is not installed. I ask, 'This is a Windows machine, right?' His reply: 'I just got it and don't know what is installed.' "

Now, ComputerWorld's SharkTank is playing this for laughs - but it points to something that an awful lot of us forget - there are a lot of people who rarely (if ever) get online. Heck, there are still lots of people who don't have a PC, and have no real interest in one either. For all the talk about the rising influence of the blogosphere, we should keep in mind that a lot of people have never even heard of it. So for all the sturm and drang around the various left/right political blogs - and for all the yelling around things like RSS/Atom - it's important to realize that a significant number of people have no contact with our world... and for the most part, they don't think they are missing anything.

Sun's John Clingan questions the reliability of industry analysts:

And are analysts completely on the level? There are those analysts who get paid to make a point. Companies pay analysts to make a point in their favor. Not coming to that conclusion means they will no longer be of value. Now, of course, coming to that point may be the 100% correct, and then the competition debunks it with their own analysis from another company (analyst shopping) who makes a 100% accurate counter point (ah, the chaos of statistics). I associate a .5 probability with this point so i have plausible deniability. How did I come up with .5? SWAG, man, SWAG. But at least I disclosed how I came about it.

IMHO, questioning these people is a good idea...

jaybaz

So, when I propose No Private Methods, that's because it's the most extreme thing I could think of.  I want to know what that looks like, so that I know when I've got the right set of private methods.

He says this in a conversation about writing shorter methods. Sounds to me like he's iterating closer and closer to a Smalltalk kind of viewpoint.

Update: Now he wants everything to be an object, and notification of attribute changes. Hmm - you mean setters like this, maybe:

commentText: someText
	commentText := someText.
	self changed: #commentText.

This guy really, really needs to look at Smalltalk :)

In a general trashing of Project Manager's, we figure out why this guy has so much trouble with them - he's the sort of prima donna developer that can make a project a living hell. Take this, for instance:

on a serious note though, a lot of the problems that are plaguing software development today are a direct result of bad, no wait, pathetic project management. these s******** are driving schedules, making feature choices, spouting off time estimates, making sure your time is wasted in meetings all g****** day, and then telling you your not going to make your deliverables because your not following the plan. i don't get it...most software developers i know are loud, egotistical, and very, very strong willed...how has this state of affairs happend? and why do we continue to foster this situation??

I think it happened to this guy due to cosmic justice - the Universe simply willed that his bad attitudes be paired with poor management practices....

In a discussion on the IRC channel, one of the regulars let loose with this gem:

You know, I was just thinking how Java really is close to the worst of all possible worlds... it's closed source, but basically no "support" provided

I mean, if I find a massive problem that effects me, what recourse do I have? I can't fix it myself, but Sun won't fix it for me

Heh. See my related post here

MS apparently patented FAT

A few years ago, we put in a patio - 20' x 20', ourselves. 9 1/2 tons of sand and gravel with a wheelbarrow, plus all the blocks. That was a lot of work, but it was worth it. Every year, weeds come up between the cracks, and every year I'm out there pulling the blasted things up by hand - my wife doesn't want something like "Roundup" used in our yard. This did make her feel a little guilty though, so she went out and bought me a solution to the problem - now I have my very own home flamethrower. Now all I need is a napalm backpack, and I'll be all set attacking enemy anthills :)

Over on Rich's blog, there's been a comment thread vis-a-vis a Store. I thought I'd surface the issue here for further conversation - I'll do that by bringing up a few of Reinout's questions and answering them as best as I can:

First, I said that we would be shipping support for configurations. Reinout responds:

Too late for the shop where I work, we are already up-and-running with lineups as implemented by Cees de Groot. Why should we switch to Store config maps?

It's a fair question, and we are late to the party with this. Having said that, here's the answer - sure, you can stay with your solution - and it will probably be more specific to your needs than a general solution that we come up with. On the other hand, you end up supporting it, instead of having us do that. That's a non-zero cost that may matter someday. YMMV of course, but there it is.

Next, I brought up optimization - we haven't done a great job with indexing in all cases, and we issue a number of queries at points that don't work well over slower links. These are things we know about, and intend to fix. For that matter, we are highly motivated to fix these sorts of problems - our engineering team is highly distributed, and runs into this problem all the time. It's gotten better, and will continue to get better.

Next, Reinout lays out a problem:

Select three packages.
Imagine that these three packages' changes constitute a rename of one single class. Imagine one other person and a staging-bot simultanously publishing one of those packages, imagine three build-bots each busy building some image configuration containing these three packages.
Now pull down and select the 'publish' menu item.

I may be off here, but this seems mostly like a process issue. I consulted at a big Envy shop a few years ago, and they had this same exact problem - it's hardly unique to Store. If multiple people are fighting to publish the same package at the same time, then - IMHO - you have a process issue on your hands. What should the tools do here? At present, you'll get multiple forked versions that need to be merged - just like in any other source toolset I know of. I need to know what the proposed solution to this is...

Finally, there's this

Committing is "saving", a set of changes. I can roll back to any revision of these changes provided I remember its address (or UID, whatever). A revision carries its history. Publishing is asigning a version number and making the changes visible to other developers.

I don't get this. I rollback changes all the time in Store - loading previous versions, or even just previous versions of methods (I spent all day doing this with BottomFeeder today). I'm not following the problem here. Which isn't to say that there isn't one - I'm just not getting what is being asked yet.

Read this post and the last few in the same thread - sounds to me like jaybaz wants support for (the equivalent of) Smalltalk on the CLR...

This is fascinating. Reading journalists reporting on technology issues I know something about makes me really, really start to wonder about reporting in general. This paragraph:

Atom also allows any comments that are posted to a blogging site by readers to be syndicated, unlike RSS, which only allows the blogs themselves to be sent out as alerts.

is just awe inspiring in its cluelessness. RSS doesn't allow syndication of comments? I guess I don't really read Sam Ruby's comment feed then. And comments aren't actually embedded in my feed either. And apparently, the wfw comment module doesn't exist. I really wish reporters knew how to use Google. This reporter gets paid to be this publically uninformed?