Smalltalk Tidbits, Industry Rants

A couple of days ago I posted a dev parcel for BottomFeeder that did not read in the 2.9 format feed file - which was a problem. Turns out I messed up the schema migration block when I was doing the module support. That's fixed, so if you try the dev build now, everything should be ok.

After seeing a few bloggers report in on Matrix Reloaded, I had low expectations for it - but it was pretty good. The pace was fast, the effects good. Some of the lines fell flat, but hey - you aren't watching it for plot, right? A good movie, not a great movie. Worth a trip, not worth multiple trips, IMHO.

There will be a nicer announcement forthcoming, but Smalltalk Solutions 2003 is going forward - July 14-16. Go visit the site and register - what are you waiting for?

Later today I'll be changing the dynamic servlet - I'll have it check the http headers for the relevant 'last-modified' information, and send along just the newer stuff. That should work with decent aggregators. Like, say, BottomFeeder.....

The early reviews sound disappointing, so my expectations are low....

Here's an obvious headline for Booch: Language Once Was Key - Now It's Design. Heh, almost as much of a shock to see the UML guy pushing design to the front as it is to see me pushing Smalltalk to the front :)

Here's the link to the story. The basic claim is that .NET and web services obliterates the importance of any particular language - since all can play equally - and elevates design. Go have a look and see what you think.

Via the .NET Guy's comment feed comes this beauty;

What people should do is use aggregators that don't go polling each and every hour for new content, but that collect the feeds when you are going to read them and hit the refresh button; like Aggie does, for example.

I guess many people here will leave their computers on at night, and with the session locked out. This means a bunch of refreshes (and consequently, bandwidth) that go unused :(

What's the purpose of automatic refreshing? Do you interrupt your work when you see there is new content awaiting, or just browse blogs while taking a break? Or perhaps is me, who mostly finds new content awaiting by the morning (living in Europe and mostly reading US-based bloggers...) and thus sees no point in auto-polling each hour. I guess there are many behavioural patterns with regard to blog reading... someone should write a paper about it.

sheesh. Replace RSS Reader with email client - should we get rid of periodic polling in those as well? See, I work during the day, and like to periodically see if there's new mail or new news without having to add an extra step. My reader uses conditional-get, supports gzip encoding, and also respects the update frequency tag (if in use) in RSS - both ttl and the newer ones in one of the modules. Which means that it's pretty light on the polling side.

Via Lambda the Ultimate - Here's a reality check on static typing as security:

We present an experimental study showing that soft memory errors can lead to serious security vulnerabilities in Java and .NET virtual machines, or in any system that relies on type-checking of untrusted programs as a protection mechanism. Our attack works by sending to the JVM a Java program that is designed so that almost any memory error in its address space will allow it to take control of the JVM. All conventional Java and .NET virtual machines are vulnerable to this attack. The technique of the attack is broadly applicable against other language-based security schemes such as proof-carrying code.

So much for that theory....

With Matrix Reloaded out tomorrow, what else could it be?

I added an incremental feed. The server keeps track of when you last queried, and only answers with new content since then (if any). Some of the proposals I've seen floated for this rely on client-tool help with cookies, or something in the feed. This solution does not rely on the client at all.

I was looking at the .NET guy's blog and saw that he's a little worried about bandwidth based on RSS requests. I got pointed to this by Michael Lucas-Smith (one of the TypeLess guys), and he said I ought to write something in the way of a dynamic feed. Here's the issue:

Simon Fell remarked that he's concerned about the bandwidth from RSS on his site. His site supports HTTP caching properly, so it's probably not that big a concern.

One very prolific blogger (no need to name him, since it isn't his fault) is using a tool that does not supporting HTTP caching properly. Right now, his RSS feed is 302K. Think about that: 302K x once an hour = 7.1 megabytes per day, per subscriber. Have one thousand subscribers? 7 gigs a day. And you know what? What this person says is too valuable for me not to subscribe to his feed.

Yeah, I'd say bandwidth is an issue for some people.

This sort of implies that, sooner of later, we're going to need to leave static RSS behind and support dynamic RSS -- perhaps even a push variant for some people (although with HTTP caching, what you're saving there is a bunch of connections, not any significant bandwidth, until you're really up into the big leagues).

So, instead of bickering about XHTML in RSS, how about we hammer out a new syndication system that's a little more dynamic, so that we only get the new posts when there are some, and not get any data at all when nothing is new?

I'm less worried than the .NET guy; then again, he's likely using IIS, while I'm using Apache. I'm also using Smalltalk, so even though I don't care that much, I whipped a servlet up in between watching tv shows this evening. The servlet keeps a cache of IP addresses and the relevant (latest) item grabbed from them. On each request, it only answers things newer than the last request. I'll get around to deploying it tomorrow; it's late, and I'm tired....

Over on the .NET guy's blog, there's some interesting information on SQL Server and Stored Procedures:

Frans Bouma publishes data that some people find very surprising, namely that his testing showed there was no performance advantage to stored procs in his case. I didn't find this surprising at all.

There's a general performance improvement that went into SQL Server 2000 (I believe) w/ regards to dynamic queries: that is, if you use parameterized queries, SQL Server can plot an execution plan for the parameterized query and cache it, and then next time that query comes through, use the already stored execution plan (despite the data being different). That basically kills the performance argument, because the advantage of a pre-calculated execution plan is eliminated. (In fact, it seems that I remember reading that sprocs in SQL2K weren't even stored in compiled form; rather a new execution plan is created and cached the first time it's executed, but I could be wrong on that.)

Now that's interesting. I'll have to take a closer look at the benchmarks, but that sounds very interesting. If that holds up, it should make O/R mapping simpler. Along those lines, the next thing in that post:

Now take this to the next step, as Paul Wilson does in Frans' comment area. Why not an O/R mapper? I have a strong feeling that Yukon will prominently feature the framework for treating SQL Server as though it were an object database.

We've taken it one step farther than an O/R mapper in our product, as the database stores very little in the way of relations (only the many-to-many mappings for query purposes). Objects are stored opaquely in the database and rehydrated dynamically as needed. But that's a story for another day. :)

Fascinating. Is Oracle doing anything like this, or IBM? Does this hold up for complex databases?

Via Rob Fahrni - what's up with this? Follow that URL, and see if it opens your CD drive...

I check Feedster every so often to see what's being said about Smalltalk. Today that got me here, where I saw this and this. Here's an excerpt:

Recently I've been working on a small application which slurps a huuge dataset into memory, processes it for a bit and then writes some measurements out to disk. Now, it just so happens that I don't know exactly how to process the dataset to get the best results. Maybe I need to use an offset of 35. Or maybe it'd work better if I used 70. I don't know until I've tried a few values to see what works.

  Dataset dataset("input.dat");
  Answer answer = Process(dataset, 35);
  cout << "The answer is " << answer << "\n";

Reading the dataset into memory takes a good thirty seconds or so. So, I run it with "35" as the parameter. Ooh, that's really terrible. So, change it to "50", recompile, run the application. Wait another thirty seconds for it to reload the dataset. Zzzz, this is silly. Why do I need to reload the dataset each time?

This is the problem with C++. There are two seperate worlds. You start in the "source code world" which is inhabited by variables, type and comments. Then you pass through the Portal Of Compilation to reach the "executable world" which is inhabited by data, stack frames, and heap blocks. You're not allow back into "source code world" unless you throw all of your possesions into a big black hole, upon which they vanish forever,

Instead, I'd write a little bit of code which loads up the dataset, then switch into "executable world" and run it. Then I'd say "just wait there" to my stack frames and the heap while I nip back into "source code world" and write the next bit of code to process the dataset in some way. Then I'd return to "executable world" to run that code, see what the result is and then nip back to tweak my parameter. The "executable world" would always stick around, even though I'm changing the code.

Now, if you've only coded in C/C++ this will seem crazy. These language are a bit like a little wind-up car toy. You spend a while winding up your program, and then to run it you put it on a table and let it go .. and it goes "wheeee" and careers off in some random direction for a while until something goes wrong. So, you pick it up, wind it up some more, and try again. Wheeeeeee, it goes off in a slightly different direction to meet a different fate.

With smalltalk or lisp, you're not just a passive observer who watches as the car races off to its doom. You are actually sitting inside the toy car, keeping it on course. If the car drifts off to the left a bit, you just correct it and continue on your way. You don't have to go all the way back to the start.

That's as good a summary on the productivity benefits as you'll see. There's an RSS Feed for Andrew's blog as well

I've been mucking about some more with modules - BottomFeeder now picks up item categories - whether organized under the category tag or under the dc:subject tag. I upgraded the blog feed to included categories, since the blog already has them. Additionally, I made sure to drop accurate item dates - I was dropping the file modification time, not the time of entry for each item.

If you follow the dev versions of BottomFeeder, I have just posted an update to the main application. If you don't see it, change the upgradeURL line to read:

upgradeURL='http://www.cincomsmalltalk.com/BottomFeeder/upgrades/dev/'

After that, you should see the ongoing dev updates. I've started adding module support to the tool - look at feed properties for some of the new information (assuming that the feed you are looking at supports that information, of course). There's a module that gives error reporting information as well - so if you see a problem with a feed, and the feed's author has provided that information, you can send the information directly.

Via Dave Buck I spotted this argument in slashdot. As with most slashdot threads, this one meanders all over the place from a simple beginning:

"Perl programmers like punctuation. Python programmers like indentation. Every programming language has its own syntax, stemming from its philosophy. What I Hate About Your Programming Language examines the issues that shape languages as they grow. It's not advocacy, I promise."

So I had a look at the O'Reilly article that generated this little storm. It's an interesting piece that goes here:

Deciding a language is "good" or not is largely a matter of personal taste. I prefer dynamic typing and don't care much for the strong static typing found in C and its descendants. Part of that is personal taste - it feels like premature optimization to decide that a variable will always fit into eight or 16 bits. (Yes, I know judicious typedefs can lower some of the maintenance costs.) It's also practical; I don't write a lot of code that would benefit from static checking. (A buddy writes drivers at a semiconductor company. Static checking saves him time and effort.)

It's interesting to see just how many people are startingto show respect for dynamic typing - I think this is a large part of why there's a buzz around Python, for instance. It's no longer just us Smalltalk advocates - people who have used little (if any!) Smalltalk are coming to the same conclusions. Here's an interesting thought from that article:

Of course, choosing any one language for a project often means not choosing another language; sometimes that's the most compelling reason. I've had projects where a manager said, "We're going to use language X because that's what VCs expect us to use," where language X was, in my opinion, the wrong choice.

Going further, differences in language philosophy present a barrier to learning new languages. This doesn't just apply to existing programmers. It also applies to new programmers - learning to think like a programmer is difficult enough without bothering with syntax issues, editors, compilers, linkers, and distribution

There are way, way too many projects that went to Java for this reason, and I presume that many will go to C# for the same one. I wrote about the insanity of this kind of migration here. To sum it up, if you migrate from X to Y, what actual business value have you gained? The article gives some specific examples of the author's likes and dislikes; I like this example in particular:

It's often used inappropriately. Unless I'm writing a kernel, a device driver, a virtual machine, or an interface to a C or C++ library, writing in C is a probably premature optimization.

Now try explaining that to a long time C hacker though....

The summary is edifying as well:

Over time, my taste in languages has evolved. I used to worry about performance and variable-typing, but working with dynamic (or "agile") languages and learning to love test-driven development has biased me against strong-typing systems. Maybe learning a language like Haskell would balance my thoughts

Every language is sacred in the eyes of its zealots, but there's bound to be someone out there for whom the language just doesn't feel right. In the open source world, we're fortunate to be able to pick and choose from several high-quality and free and open languages to find what fits our minds the best

There's an interesting response to this article in the O'Reilly thread as well. The poster seems to know Squeak better than most Smalltalks, and conflates some of Squeak's issues with Smalltalk (Namespaces, for instance) - but it's got some good points:

What's not bad about Smalltalk

• Speed. It's actually not that slow. Some tasks under Squeak are faster than Java. Java is faster for others. Cincom's commercial Smalltalk is faster than Java on just about everything.
• multi-person development in an IMAGE based environment. FORTH programmers will known what I'm talking about. Classic Smalltalk stores the complete state of all objects in object memory in an IMAGE. This is basically one big database of objects. For a long time, the big deal with IMAGE based systems was that it was difficult to separate the development environment from the running application. Proper use of changesets and Projects seem to have done a great deal towards fixing this.

Of course, the commercial Smalltalk's all have good source code management support - varying from top notch stuff like Store (VW) and Envy (VA) to interfaces to more conventional (cvs style) systems. Furthermore, you can download just about any vendor Smalltalk implementation for free; check http://www.whysmalltalk.com for links.

The syndication module is now supported in BottomFeeder. I haven't pushed an update yet - I'd like to do better testing before I do that. What does this add? Any RSS 1.0 or 2.0 feed that has publish frequency information in the syndication module will now have that respected.

I've added basic module support to BottomFeeder. What are modules? Elements that live in their own namespace so that collisions are avoided. The only one that is really used at present is the content module - see here for a description. Ironically, this is the tag that Bf needs least, since the VW XML Parser detects encoded content for me; I just look for this tag because feeds that use it probably don't have a description tag. I'll be adding support for additional modules as I have time - and - more importantly - starting to use some of the newly available information.

So we get to see Angel back on the last two episodes - what I'm waiting for is the Angel/Spike dialog - that has the potential to be very amusing. The other question - since this is it for the series, it's open season - Joss could kill pretty much any character at this point. So will any of the regulars end up pushing up daisys? Hmmm....

I posted on the failure rates of J2EE and Java projects awhile ago. Now we have Jon Udell noticing the overselling of J2EE.

An oft-heard complaint, echoed recently by Annrai O'Toole, is that J2EE app servers are oversold:

The J2EE vendors have done a fantastic job of convincing the world that you can't run a line of Java unless it runs inside a J2EE container. This is just pure bunkum.

I like his formulation that "J2EE is the Java equivalent of a mainframe." We also have, of course, in COM+, the Windows version of the same idea, which in its earlier MTS incarnation predated J2EE. I also notice that the real mainframes haven't gone away. With respect to the middleware services for which the J2EE server is best known -- "TP-heavy" transaction management, connection and object pooling, role-based security, and declarative control of these aspects -- the question of when and why this stuff is or isn't overkill seems never to go away

Needless complexity.... There are times you need the heavyweight stuff. But most of the time, you don't

Heh. In the IRC today, Alan Knight let this slip:

This comes back to my notion of evolutionary IT. It's a mistake to assume that there's any guiding intelligence in IT. Decisions are made at random. Over the course of millions of years, certain decisions will result in a greater survival percentage....

I guess we settle in for a long wait....

Regardless of your policy views, this is just amusing. There's one way to "filibuster" programs you object to :)

Clarence noticed that I pointed out a new Smalltalk aggregator, and asks

where's the code?

I don't know about the Squeak code, but the VW code for BottomFeeder is at SourceForge - although for up to date access, get a public Store account and grab the BottomFeeder bundle. You should also load the latest Twoflower bundle - the system will load the 'goodie' version from the parcel directories, but it's out of date relative to the database.

Anyone interested in the code that powers this blog can grab the CST-Blog bundle from the public store as well.

I should point out (as I did here tangentially) that Douglas has a blog. He's got a fair bit of Smalltalk content (using Squeak) - check it out.

via Chris Double comes news of a Squeak based aggregator. Welcome to the fray!

Got this one in email today - along the same lines as this and this:

Things are going okay for now, but it's getting more and more difficult to convince anyone to do anything in Smalltalk. I can't understand how a company that is finally reaping benefits of a large-scale project done in Smalltalk (successfully), starts looking to throw all that investment out of the window and go with a lesser tool.

In my particular area, we deal with both Smalltalk and Java, and we (Smalltalk team)are consistently delivering a more stable product faster (which means that we usually 'wait' until the Java part is ready). My team lead sees that, my manager sees that, even my director sees that. But when it comes to the VP, it's all Java talk.

Here's a tip for IT managers everywhere: If it's not broken, don't fix it. Maybe, just maybe, the company you work for won't have to lay off half your staff if you keep that in mind....

I received more details in email from the guy who sent me this:

I forgot to tell you the really fun part. (dohh!) Yes, part of the problem was mismanagement. A bigger problem was C++ and what Rhapsody could and could not do with it. The guys ended up doing some pretty basic R&D on the Rhapsody codebase which we guilt-tripped out of the vendor. This product gives great demo, but doesn't deliver for complex tasks. And of course one could replicate it in Smalltalk in about a week and not pay the ridiculous fees.

One of the biggest problems was building a decent message passing system between CPUs. As the doors are closing, there are still rancorous arguments about CORBA vs. RPC vs. sockets, and all of them are ugly in one way or another. C++'s lack of reflective capability is simply a disaster, and our C++ jocks could barely comprehend the nature of the problem.

Dohh! again....

VisualWorks developers - there are new VW 7.1 engines available for download:

These 7.1a engines are the most up-to-date engines for VisualWorks 7.x imagesas of May 2003. Version 7.1 engines are backward-compatible with 7.0 images and are recommended.

The version 7.1a engines fix the following bugs in the 7.1 engines:

All Platforms

• The primitives for longLongAt:put: and unsignedLongLongAt:put: are broken for large integer values.

Linkable GUI Platforms (AOSF, Linux86, LinuxPPC, LinuxSPARC, Solaris)

• The linkable GUI platforms could not do international input.

These problems have been addressed; download links for all engines are available here

I got this in email from a former colleague trapped in the unproductive bowels of the Java-Verse:

Thought I'd add another entry to your probably-overflowing file of "Gee, they shoulda used Smalltalk" stories.

For the last three years, I've been working on the UI for an industrial robot as a contractor for a guy I had worked for before. I tried to sell them Smalltalk but they went for Java, which worked thanks to the use of Smalltalk techniques but still didn't meet their realtime requirements. Eventually we switched over to Qt and life got really unpleasant. But that's just a side story.

This project has been running for several years. My client originally sold this project to a Japanese company that has trouble writing software. He sold it based on a software tool called Rhapsody: draw a bunch of UML diagrams and it generates C++ for you. His client saw this and went wild.

The last three years have been an interesting roller-coaster ride, consisting mostly of the inability of a *very* strong team to create the necessary software in a reasonable amount of time. Part of the problem was mismanagement, or simply lack of management, which is odd because the guy who started this has a reputation as the ultimate software development manager.

Recently, his client sent over a Japanese OO expert to take a look at what was going on. There was a big company pizza party to introduce him. He gave a little speech in which he said there was this system that he had first used back in 1984 and it still amazed him that everyone wasn't using it.

I was told by others present at a high-level meeting that my client asked this OO guy how he would have approached this problem. He said he probably would have started off writing it in Smalltalk.

BTW my client's company is closing its doors this Thursday.

Dohh!

I made two posts this last weekend that are now pretty far down the scrolling list of posts - if MemoryPolicy and ObjectMemory interest you, have a look:

• MemoryPolicy and Large memory images
• Tweaking ObjectMemory

I've certainly never thought of OO as a silver bullet - it's a useful technique for solving software problems, nothing more. The enemies of OO certainly like to paint it that way though. There's so much unclear on the concept stuff on that page that it's hard to know where to start; here's a sample of how this guy goes off the rails:

Let me give an example of this. Suppose I want to define a ubiquitous data structure. ubiquitous data type is a data type that occurs "all over the place" in a system.

As lisp programmers have know for a long time it is better to have a smallish number of ubiquitous data types and a large number of small functions that work on them, than to have a large number of data types and a small number of functions that work on them.

A ubiquitous data structure is something like a linked list, or an array or a hash table or a more advanced object like a time or date or filename.

In an OOPL I have to choose some base object in which I will define the ubiquitous data structure, all other objects that want to use this data structure must inherit this object. Suppose now I want to create some "time" object, where does this belong and in which object...

This gets a big huh??? - even in statically typed monstrosities like Java, interfaces allow one to define objects where they belong, but create a consistent interface. Smalltalk has always allowed for this, as had CLOS. Coming from someone who seems to know Lisp, I'm just baffled. This is either a straw man argument, or a real failure to grasp the concept. There's more; go see for yourself

Microsoft seems to understand that blogs are a fine way to get information out, and encourages their people to do so. I just found this one via Lambda - which covers implementation details on the .NET CLR. RSS feed here

I posted on a CIO Insight article recently - the one thatwentinto John Parkinson's theories on who actually does most of the development work. Today, Terry Raymond points out that the ACM has noticed the artcle as well:

John Parkinson of Cap Gemini Ernst & Young concludes that, as far back as the late 1970s, the most sophisticated and reliable software applications were chiefly the work of a small portion of programmers who were labeled 10X or Power Programmers, but these individuals fell out of favor when companies started concentrating on boosting the performance of average programmers via tools and methodologies. However, Parkinson has found indications that Power Programmers are staging a comeback, as evidenced by software products such as Squeak, an advanced version of the Smalltalk object-oriented development environment, and the inSORS Grid IP Multicast collaboration and videoconferencing system.

Interesting how people are noticing high levels of productivity and Smalltalk simultaneously. Now if they could just connect the dots.....

This rundown on Startrek was just perfect. This is funny:

Yeah, I know this one is overdone, but you'd think that the first time an explosion caused the guy at the nav station to fly over the captain's head with a good 8 feet of clearance, someone would say, "You know, we might think of inventing some furutistic restraining device to prevent that from happening." So of course, they did make something like that for the second Enterprise (the first one blew up due to poor lubrication), but what was it? A hard plastic thing that's locked over your thighs. Oh, I'll bet THAT feels good in the corners. "Hey look! The leg-bars worked as advertised! There goes Kirk's torso!"

Star Trek:

Picard: "Arm photon torpedoes!" Riker: "Captain! Are you sure that's wise?" Troi: "Captain! I'm picking up conflicting feelings about this! And, it appears that you're a 'fraidy cat." Wesley: "Captain, I'm just an annoying punk, but I thought I should say something." Worf: "Captain, can I push the button? This is giving me a big Klingon warrior chubby." Giordi: "Captain, I think we should reverse the polarity on them first." Picard: "I'm so confused. I'm going to go to my stateroom and look pensive."

Firefly:

Captain: "Let's shoot them." Crewman: "Are you sure that's wise?" Captain: "Do you know what the chain of command is? It's the chain I'll BEAT YOU WITH until you realize who's in command." Crewman: "Aye Aye, sir!"

heh. Go read the whole thing

Bob Lewis has a good commentary out on IT management's tendency to want to measure everything:

Someone - probably Deming - once said that if you can't measure you can't manage. Or, if you can't measure you can't improve. Something like that.

The problem is that instead of keeping the thought in context, managers, egged on by an army of consultants, immediately applied it to every blessed thing in the business without thinking very hard about the implications.

Many have missed the most important impact of measurement, which is that once you institute a program of formal measurement, you get what you measure and only what you measure.

That last part ought to be in bold: once you institute a program of formal measurement, you get what you measure and only what you measure

Klingon???

PORTLAND, Oregon (AP) -- Position Available: Interpreter, must be fluent in Klingon.

"There are some cases where we've had mental health patients where this was all they would speak," said the county's purchasing administrator, Franna Hathaway.

County officials said that obligates them to respond with a Klingon-English interpreter, putting the language of starship Enterprise officer Worf and other Klingon characters on a par with common languages such as Russian and Vietnamese, and less common tongues including Dari and Tongan

I wrote about VW memory management here the other day. In that article, I mostly went over areas you can change by implementing a custom Memory Policy class. That's hardly the only thing you can do though; there are a variety of settings in class ObjectMemory that you should be aware of. An excellent place to start is the documentation protocol on the class side of ObjectMemory - look at spaceDescriptions and reclamationFacilities - these two methods are actually doc. After you've read that, you'll have a better handle on how the VW memory system works.

For most applications, there are two memory spaces you'll want to take note of:

• New Space (Eden)
• Old Space

In rough terms, New Space is where objects are born. Old Space is where objects that don't die immediately go to live. If you think about it, it's clear why these two areas are of interest - if New Space is too small, then objects get tenured off into old space too quickly - and depending on how you have set up growth policies (see my earlier post), you can get either rapid memory growth or thrashing - or both in sequence. On the other hand, makeit too big, and the incremental collector - the one that scans New Space on a regular basis - may well spend too much time doing so, making for slower performance. The "correct" size is a balance between processor speed and application behavior - so you'll have to test it out. The default setting is often ok as-is.

Then there's Old Spacee - objects that don't get wiped out by the incremental collector go here to live - and remember, Old Space is not going to get scanned unless:

• The result of #favorGrowthOverReclamation is false, and the system has to scavenge to create space for objects moving from New Space
• You manually invoke the garbage collector (for instance, using the launcher menu)

If you know that your application creates lots of long lived objects, it's a good idea to increase the size of Old Space up front. Recall that MemoryPolicy merely controls how thesystem behaves - not how much space is available at startup. To change the space sizes, you can do one of two things:

• Use ObjectMemory class>>sizesAtStartup: to tell the system to increase the starting sizes at system startup. If you send this message, make sure to save the image - this only happens at startup!
• Use the API for expanding memory - ObjectMemory class>>growMemoryBy:

You can also shrink memory - but you want to be careful about doing that. In general, you should let the system do that itself via the settings in MemoryPolicy.

This barely scratches the surface of things you can do with the VW memory system - the settings allow for a lot of control over how and why memory is expanded or contracted. Make sure you read the class comments in MemoryPolicy and ObjectMemory - and the documentation methods there - before you get started. Questions? Send them my way

Patrick Logan points out that with keyword messages and blocks, you get an awful lot of the goodness of macros in Lisp:

While Smalltalk doesn't have syntax macros like Lisp, Smalltalk *does* have simpler block syntax and keyword arguments.

The effect is that new keyword messages can take block closures, which essentially delay evaluation until the method behind the message decides to evaluate them, and the message send looks an awful lot like new syntax, very clean.

Many uses of macros in Lisp, frankly, are simply created to hide the appearance of the lambda keyword. Of course there are more "legitimate" uses of macros, but these are less common.

This was noticed by Ted Leung, who asked:

This use of keyword messages and closures is really interesting. In Chapter 8 of On Lisp, Graham lists 6 reasons to use macros. Delaying or altering evaluation is an aspect of the first 4 reasons. Reason 5 involves using the calling environment, which you very rarely want to do, reason 6 involves using a new lexical environment, which you also rarely want to do, and reason 7 is inlining which is a performance annotation.

I'd be curious to see some examples of this type of Smalltalk code.

Rich Demers tried to answer on Ted's blog,but ran into a small problem with BottomFeeder's implementation of the CommentAPI - I was url encoding the text. Dohh! I fixed that, but here's the link Rich was pointing to

Periodically, I get asked about using VisualWorks in large memory situations. In the sense I'm discussing it, this means more than a gigabyte. If you need to set up a VW image that is going to use more than that amount of RAM in a runtime situation, you are going to have to get into the guts of the MemoryPolicy system. The first thing you'll want to do is read the documentation - it's pretty well covered there. After that, you'll need to have a look at a few places. The place to start is class MemoryPolicy. It turns out the ObjectMemory is managed, in large part, by settings in the current memory policy. As the class comment puts it:

This class contains a class variable called CurrentMemoryPolicy which contains a reference to the currently active memory-policy object. Control is passed to the CurrentMemoryPolicy during the idle loop and during the low-space condition for the appropriate response. This class also relies on the CurrentMemoryPolicy to specify a hard and soft low-space limit. The HardLowSpaceLimit is meant to represent an emergency low-space condition, and the SoftLowSpaceLimit is a secondary threshold that the CurrentMemoryPolicy can use to gain control in advance of the hard limit. This class arranges for control to be passed to the CurrentMemoryPolicy whenever either of these two limits is exceeded. See the comments in the class MemoryPolicy for further details regarding the role that a memory policy is expected to play.

This is why you have to modify MemoryPolicy if you have an application that will use "large" amounts of RAM - the basic policies aren't going to work. For our purposes, here's the core method that will interest you in MemoryPolicy: #favorGrowthOverReclamation. This method is invoked whenever the system needs more space for new objects - it has to decide, based on the current policy, whether to ask the OS for more RAM, or whether to do a GC and hope some space clears up. Here's the basic implementation:

favorGrowthOverReclamation
	"Answer true if we want to react (at this point in time) to the low-space condition by growing memory rather than reclaiming memory."

	^growthRegimeMeasurementBlock value <= growthRegimeUpperBound

growthRegimeUpperBound is set to 32MB by default - likely not what you want! It's better than the pre-7.1 setting of 16MB, but not suitable for the situation we are discussing. The block being referenced looks like this:

[ObjectMemory dynamicallyAllocatedFootprint]

or

[ObjectMemory growthDuringCurrentSession]

What that's going to do is cap the maximum growth of memory for the Smalltalk application. You'll likely want to adjust that block based on your application and deployment situation. Another place you'll want to look is the amount of RAM that system allocates when it does allocate - which is controlled by the preferredGrowthIncrement variable. By default, this is set to 1 MB - you'll likely want it bigger. How do you do this? Here are the steps:

• Define a new subclass of MemoryPolicy. Make sure to implement the #setDefaults method, invoke the superclass version, and set the values of the appropriate variables.
• Install the policy like this: ObjectMemory installMemoryPolicy: MyMemoryPolicy new setDefaults

I've published an example implementation in the public store. The #favorGrowthOverReclamation method there looks like this:

favorGrowthOverReclamation
     ^self memoryStatus availableFreeBytes <= self
growIfFreeBytesLessThan

Where growIfFreeBytesLessThan is a new variable setting. It's the floor for how much available RAM the image ought to have. Be Careful!! with this strategy - this method puts no ceiling on image growth. If the OS allows you to limit that, or there are other application specific factors that make this "not a problem", go ahead - but you'll likely want to think long and hard about this one.

In any case, this ought to get you started. To take a look at the example Memory Policy, look at package LargeMemoryPolicy in the public store. It installs itself as the new memory policy on load, as follows:

ObjectMemory installMemoryPolicy: LargeMemoryPolicy new setDefaults

Questions? Send them my way

Jim Robertson's Axiom of Software Development:

No good deed goes unpunished, no bad deed goes unrewarded

yes, I'm feeling cynical today :)

Tim Bray attempts to denigrate dynamic languages as "big" and "bloated" when one uses them to deploy systems. Hmm. Maybe he can explain this. BottomFeeder is in the same neighborhood as the other tools out there in widespread use. So much for "fat". maybe he can explain why Netscape routinely chews more RAM on my desktop than do my running Smalltalk development environments, much less the deployed applications. Then there's the complaint that Ted comments on:

Secondly, and in the same spirit, there do remain performance issues. There are is some (small) number of people who have to write low-level webserver code, and if you've ever done this under the gun of a million-hits-a-day load, you quickly become a control freak with an insane desire to remove as many as possible of the layers which separate your code from the silicon.

Hmmm. Sure, you can get better math performance from Java or C++ than you can from Smalltalk. On the other hand, Strongtalk is comparable, and Lisp can be as fast or faster. The main reason Smalltalk environments don't optimize for math speed is that it's not very important for most business applications. Why are the C/C++/Java crowd always so obsessed with this? Get it working, get it optimized - in that order. The C language crowd invariably inverts this. Not to mention this - in my experience, the performance issues are invariably not where the developers thought they would be up front. Premature optimization is a poor development practice.

But darned if Ted Leung didn't beat me to it:

Sun has a new article describing new language features in JDK 1.5. I love this sidebar quote:

The new language features all have one thing in common: they take some common idiom and provide linguistic support for it. In other words, they shift the responsibility for writing the boilerplate code from the programmer to the compiler.

In other words, we're modifying the language because we didn't have a macro system that we could do this with -- at least for generice, enhanced for, static import, and attributes. Getting rid of boilerplate code is what macros are all about.

Also notice that 3 out of the 6 features in the article are being copied from C#.

While Smalltalk doesn't have the macro system Lisp has, generics are a non-issue in ST, as is boxing/unboxing. Ditto the "simplification" things. One could summarize this whole thing as adding complexity to a language to make up for the glaring flaws.

Torsten Bergmann has two new goodies in the public store:

• Tool Icons

This small package adds separate window icons to the Launcher and Inspectors in VisualWorks 7.x. This is very helpful for Windows users since in the default VW implementation it is hard to distinguish IDE windows since they all have the same Cincom ST icon. If you are used to use ALT+TAB to switch between VisualWorks windows or other applications this is very helpful

• More Windows Goodies

This small package adds support for the native Windows shell folder dialog in VisualWorks. It wraps the SHBrowseForFolder and the SHGetPathFromIDList() API.

Check out Dave Astel's comments

The New York City STUG is having an interesting meeting on May 28th:

NYC Smalltalk will hold its next meeting on Wednesday May 28th, 2003.

Date	May 28th, 2003
Location	Suite LLC offices
Address	440 9th Avenue, 8th Floor
Time	6:30pm to 7:00pm -- Open house
Time	7:00to 8:30 pm -- Ethnography and System Design

See below for abstract and bios.

Directions:

Take E or C train to 34th (Penn Station) walk to corner of 34th and 8th. Walk up one block to 9th.

RSVP is requested. Please send mail to charles@ocit.com with subject line of: NYC Smalltalk May 28th, 2003

Our meetings are opened to the general public. Invite a friend ! To join our mailing list simply send mail to nycsmalltalk-subscribe@yahoogroups.com.

Joining our list will give members access to all of the presentations and articles maintained on our site. Any questions send mail to charles@ocit.com

Charles, Chair - NYC Smalltalk

Abtract:

Ethnography is the output of anthropological study; literally it means "writing people" or "writing culture" and it is usually a detailed description of a particular people, organization or type of work. What, you might ask, does this have to do with computer system design?

For the past 30 years, methods from traditional anthropology have been used to study people and their work and apply what is learned to designing computer systems. This presentation describes these methods and their application. We'll tell you how an Anthropologist got started in the Smalltalk lab at Xerox PARC and how an engineer got involved with anthropologists. We'll draw the connections to certain European software communities (e.g., Kristen Nygaard and Participatory Design), tell you about a few of our projects and what we are doing now.

Our contention is that especially in an environment that emphasizes agile, iterative approaches to software, observing technology in use is an important capability that software system architects, designers, and developers can use to insure successful design, development and deployment.

Bios:

William L. Anderson

Focus

Bill is a co-founder of PRAXIS101. His consulting practice focuses on user-centered systems architecture, participatory design, software development practice innovation, and organizational learning.

Background

Prior to founding Praxis101 Bill was a Software Architect for Xerox Corporation, where he developed service discovery software as part of an agent-based, peer-to-peer, document services middleware infrastructure. He pioneered codevelopment and participatory customer collaborations in rapid prototyping and product development in projects that put end-users and engineers together, jointly developing and deploying new products. Prior to Xerox, Bill worked in the telecom, networking, and pharmaceutical industries. Bill holds a Ph.D. in Theoretical Chemistry from Rensselaer Polytechnic Institute. He is currently serving on the National Research Council's National Committee on Data for Science and Technology, and speaks internationally on issues of preservation and access to scientific and technical data.

Susan L. Anderson

Focus

Susan is a co-founder of PRAXIS101and fassforward consulting group. She uses her expertise with cultural nuance, transformation and innovation to help companies break through their tough problems, plan for the future and productively engage employees and customers.

Background

Formerly, Susan was Senior Director for Gartner.Com (covering users, technologies and competitive intelligence) where she helped build the next generation Gartner.Com. Prior to that, at Xerox, she was instrumental in developing collaborative, socio-technical approaches to the design of new technology (from knowledge solutions to remote collaboration tools and systems to document management systems). Susan's work has also emphasized change initiatives in corporations, education, health care and international development. She has been an active contributor to professional conferences and organizations. Susan holds a Ph.D. in Anthropology; she has written on topics from technology's role in health decision making to technology-mediated collaboration.

Looks different from the normal User's Group meeting - if you plan to be in NYC then, check it out!

Via Sam Ruby:

Sean McGrath: Dynamic typing is good for you. Its good for programming. Its good for data. It makes it possible to develop software without a crystal ball and without a 10x hit between "product" and services thanks to the extra levels of loose coupling it introduces into your designs. If you haven't yet tried it - get Python, get XML and do something non-trivial with the combination. Give it a couple of weeks to sink in. I promise you, you won't go back.

Although, I'd say try Smalltalk as well. Give Lisp a shot. Look at Ruby. And after all that, see what you think.

I'll be visiting a customer site this morning, so no blogging for a bit. First, a nice hot Mocha. mmmmm....

Managability is still fighting the good fight over static typing. Today, he's using a call to authority:

• Improves robustness through early error detection
• Increases performance by making required checks at the best times
• Supplements the weaknesses of unit testing

Hmm. Early error detection? I don't think his expert has done a lot of work in dynamic languages. The kinds of errors that static typing guard against just aren't that common. I can't recall the last time I made one, for instance. Increases performance - early optimization is always a bad strategy. If you are thinking of optimization at first, you made a mistake. Weakness of unit tests? You mean, testing the actual usage of the code is less useful than verifying that we passed an int to a method defined that way? Pardon me while I laugh. a lot.

After showing a demo of how nifty Eclipse is, he goes on to say:

I think given the arguments presented by Eric Allen, and the utility of Eclipse in doing TDD, the issue of Static vs Dynamic typing should be a closed issue.

Sure, just not the way you think. The Eclipse link argues about the lack of power in Ruby and Python development environments (is this true?). Here's a tip - grab a Smalltalk environment, and get back to us.

Via StronglyTyped, I see that Google now has a web log specific search page. It's not nearly as good an index as Feedster; it misses a lot

Within the context of commenting on Sun and Open Source, Ted Neward points out the irrelevance of vendor neutrality in the J2EE world:

FWIW, I agree with the large vendors that JBoss doesn't really deserve the J2EE brand name; I also think having the J2EE brand isn't worth the bytes it would take to put it on the website. By this point in time, with the market having consolidated behind just a few key players in the J2EE space, is it really all that important to be vendor-neutral?

That's about right - with how much money companies spend on J2EE server tools, migration is about as likely as db migration - i.e., not very. Given that, moving to J2EE will yield just as much vendor lock as moving to Smalltalk. It will also mean dramatically less productivity. Sure, you can find more developers with Java on their resume - but see this post for how that will work out....

Bob Martin discusses overstaffing and empire building this morning:

I once consulted for a company that had 50 developers working on a simple GUI. This GUI was a flat panel touch screen upon which several dozen dialog boxes could be made to appear. These 50 developers worked on this project for five years or more. That's 25 man-decades, 2.5 man-centuries! COME ON! Three guys could have done this in three months! My buddies and I used to joke that they had one developer per pixel and that each developer wrote the code for his pixel.

OK, so the manager was empire building. Some managers measure their worth by the number of people they manage rather by how much they can get done with how little. Still, I find this problem is not isolated. It seems to me that a large fraction (perhaps a majority) of all software projects are overstaffed by a huge factor

Another question to raise here - these problems are most rampant at the largest companies - the same ones that follow all the fads - and the same ones that are now furiously outsourcing development in order to save money. I think I see a pattern forming.