Rob Vens is talking about an interesting topic: building and distributing freeware using VW. I do that - BottomFeeder :) Rob's first language was Smalltalk, so even though he's not currently working with ST at work, he does shareware/freeware work in it.

Rob started doing these kinds of apps by making things he'd built for his own use available (personal finance applications to start with). Problems?

• Small team (just yourself, usually)
• Lack of free time
• High but inconsistent release schedules
• Anonymous user community
• As many platforms as possible
• Bug and feature driven
• Very much depending on user feedback

I identify strongly with all of those, as I face the same issues in my projects (BottomFeeder and Silt). More issues he encountered:

• Packaging
• Distribution
• Installation
• Platform Compliance
• Versions, bugs, feature requests
• User feedback
• Licensing
• Making some money?

Tools: VisualStudio (scripting), 7-Zip, Xara3D, Axialis Icon Workshop, ResHacker for Windows exe files. His update scheme involves downloading and replacing the entire deployed image/executable.

For distribution, he uses his own website - you choose a platform and download.

Platform Compliance: this is always hard, as are icons, and version/readme file issues. Tracking bugs? User feedback? Not easily tracked, mailing lists didn't work out.

Technorati Tags: smalltalk, esug

Mathieu van Echtelt of CosmoCows - here's the intro to his session. They've created a development environment on top of VisualWorks for building Business Administration systems.

Their systems are mixed, involving Smalltalk, and C, Javascript, HTML. They use Seaside to try and keep those under control - even with the Squeak VM, they use the Slang Smalltalk subset to do modifications. Their goal has been to build along the lines of the original Smalltalk design principles:

• Personal Mastery: "If a system is to serve the creative spirit, it must be entirely comprehensible to a single individual" (Dan Ingalls)
• Storage Management: "To be truly object oriented, a computer system must provide automatic storage management" (Dan Ingalls)
• Reactive Principle: "Every component accessible to the user should be able to present itself in a meaningful way for observation and manipulation" (Dan Ingalls)

The rest of the session was a live demo

Technorati Tags: smalltalk, esug

First up Friday morning - Norm Green, Smalltalk engineering director at Gemstone. The main topic: Gemstone/64. Gemstone is a privately held firm, with a very experienced development team - many have been there 10-20 years.

Brief background: Gemstone is an Object Oriented Database. Objects are stored in object format - there are no relational rows or columns, no O/R mapping. Gemstone supports ACID transactions. Gemstone is also a Smalltalk dialect - it comes with a complete set of kernel classes. The database contains the data and the behavior, which makes it a full multi-client, multi-language application server:

• Smalltalk: VW and VA
• Java
• C/C++
• Unix shell (topaz)

Why 64 bit? Customers wanted to get past the 4 gb of addressable memory limit:

• 2-3 gb page cache limit
• 1 billion object limit

For a 100 gb database, only 2% of the database can be cached, which can lead to issues - you only get 2-3 gb page cache (depends on the platform). They came up with a 2 phase project:

1. Phase 1: 64 bit address space
2. Phase 2: 64 bit object identifiers

In phase 1, they improved performance, and addressed some of their garbage collection issues. They can now support very large page (shared) cache: up to 16 TB. That gives you a "database in memory" capability.

In the old design, old objects sometimes persisted, even when they shouldn't have. That's been addressed. Online backups are now safe, and garbage collection has been parallelized and improved greatly.

The biggest thing they did in phase 1 was a major change to the VM design (which is how the stuff above was accomplished). Norm has benchmarks for the improvements in performance. The entire rewrite took 15 months. The new VM is in use by customers in production.

In phase 2, they increased the possible size of databases - up to 32 TB in size. They further improved VM performance - 30-50 percent VM speed performance. By moving SmallIntegers to 2^60 from 2^29, they were able to add additional bytecodes. Doing more in a single bytecode improves performance. Additional extensions (new class: SmallDouble) are compatible with the 64 bit work being done on VisualWorks.

How do you upgrade? You need to be on Gemstone 6.1.5 or later. When they deployed to a major shipping client, performance improvements (DB side) of 30-50 percent were seen in the field, although there was an increase in the network load (larger bytecodes). The VW client runs slower (30-50 percent) because 64 bit ids become LargePositiveInteger objects. They are working on that, but it should also improve with 64 bit VW client (on their roadmap).

They currently support Solaris and HP - adding AIX and Linux (64) shortly. Their roadmap has a number of items for future performance improvements on it.

Technorati Tags: smalltalk, gemstone, esug

Wondering what ESUG 2006 in Prague looked like this year? Check out Adriaan van Os' pictures.

Technorati Tags: esug, Prague, smalltalk

The next session is Rowan Bunning talking about Scrum - Rowan's the Product Manager for SwS, so methodology is something he'd be interested in.

Rowan started with Wizard (from which SwS was spawned). They've been a Smalltalk shop for the last decade, and he's been in the Information Services group (one of 5 there). They made a move from bespoke applications to products awhile back.

He learned about Scrum from Scott Ambler's presentation at the 2003 Smalltalk Solutions - he's become a Scrum Master and Mentor since. Scrum is used extensively at Wizard and Software with Style now. Why?

Scrum has executive backing at Wizard, and it delivers positive reinforcement to the developers. Smalltalk works really well for rapid development, which makes it even better.

Scrum uses a defined process - you try to make reality follow an upfront plan. It's also empirical - you continually adapt and adjust based on how things are going.

There are three Scrum roles:

• Product Owner - represents customer, users
• Scrum Master - guides process
• Scrum Team - 5- 10 people - multi-disciplinary, builds product - self organizing

At Wizard:

• 2 Week Sprints
• 2-4 hour reviews and meetings
• retrospectives integrated into review
• task allocation often left open
• testing on development server - bug reports from tool
• transition to QA server - formal change tracking

The rest of this session was a walk through of how they might take a task and move it through the cycle. An excellent point made: Avoid Critical Risk Tasks at the End of a Project. Words to live by in product development :)

Technorati Tags: scrum, agile, smalltalk, esug

Lukas Renggli is up again, this time talking about Seaside and how it fits with the "Web 2.0" meme. So Seaside does a lot of things out of the box: XHTML CSS RSS AJAX

Heh. He said I'd get on their case for bad RSS. Lol :)

This talk is about Seaside's use of AJAX, which gives you client level interactivity.

You want to concentrate on your web app, not on browser issues (especially browser incompatibilities). Seaside masks that by using Javascript [ed: Depending on how well your browser handles Javascript: see TinyMCE, for instance...].

The integration is via the Scriptaculous Javascript library. It handles things like drag and drop, in place editing, etc. The important thing is, you can express all of this in Smalltalk - Seaside has APIs for that.

Here's a small example of something interesting: shared state across multiple browser instances:

Example:

 
 

html effect
	id: 'hint';
	shake.

becomes:

new Effect.Shake('hint');

You integrate this the same way we saw in the short example 2 days ago.

The kind of code you can write:

 

html (updater | periodical | request | evaluator)
	[OPTIONS]; *
	[HANDLER]; *
	[TRIGGER]; *
	[CALLBACK]; *

Note: I don't guarantee that the code examples are perfect :)

 

"in place editor"
html span
	script: (html inPlaceEditor
	       triggerInPlaceEditor: [:v | anItem title];
	       callback: [:r | r render: anItem];
	with: anItem title.

"drag & drop"
html form id: 'checkboxes'; with: [
html div
	id: 'items';
	script: (html sortable
	       tag: 'div';
	       onUpdate: (html request
	               triggerSortable: 'items'
	               callback: [:v | self model item: v]));
	class: 'items';
	with: [self renderItemsOn: html]]

For debugging, you can usually debug in the Smalltalk debugger. If you have issues at the Javascript level, use Firefox and the FireBug extension that gives you some Javascript debugging capability. And see scriptaculous.seasidehosting.st.

Technorati Tags: esug, smalltalk, seaside, web2.0

Now we have Joe Armstrong - an invited speaker to talk to us about Erlang. I've always been curious about Erlang - Patrick Logan has said good things about it, for instance. He's worked in Smaltalk, and likes it. His work has been in telephony, and Smalltalk was a bit slow for their purposes (this is way back). Ordered a Tektronix Smalltalk machine (that's how far back :) ) - but there was a two month wait for it.

In the interim, he fell into Prolog by accident. By the time the Smalltalk machine arrived, he was more interested in working with Prolog. He gave the Smalltalk machine away, and Objectory came out of that donation. In further experimentation, he added concurrency to Prolog - and this is where Erlang was born - Joe is the inventor of Erlang. He's curently a senior system architect at Ericsson AB.

One thing that Erlang is very good at is creating large numbers of processes. In Java, for instance, processes map down to the OS (threads) - and that takes away predictable behavior. So what you want is:

• create large numbers of processes
• work the same on different OS's
• are garbage collected
• are location transparent
• cannot damage other processes
• are defined in the language
• creating and destroying processes should be lightweight

Niall Dalton: Erlang is Smalltalk the way Alan Kay wanted it to be - this got a nice laugh from the audience :)

Assertion: To make a fault tolerant system, you need at least two computers. That means that you are doing distributed programming. To simplify the problems, have:

• No sharing
• pure message passing
• no locks

Sharing gets in the way of fault tolerance. This gives us Concurrency Oriented Programming:

• A style where concurrency is used to structure the programming
• large number of processes
• complete isolation of processes
• no sharing of data
• location transparency
• pure message passing

So the design rules:

• Identify the concurrent operations in the problem
• identify the message channels
• write down the set of messages seen on each channel
• write down the protocols
• write down the code

Try to make the design isomorphic to the problem - i.e., a 1:1 correspondence between process/message structure in the model and the problem.

In 1998, Erlang got banned at Ericsson, due to a "we must use standard languages" wave (sounds familiar). The upshot was, Erlang got open sourced, and the Erlang folks formed Bluetail. Bluetail was later acquired.

The basic concept here: systems should be black boxes that can communicate - failures in one black box should not crash another black box. This leads to making all communication asynchronous. The problem domain:

• Highly concurrent - hundreds of thousands of parallel activities
• Real time
• Distributed
• High Availability - down times of minutes/year - never down
• Complex Software - millions of lines of code
• Continuous Operation - years
• Continuous Evolution
• In service upgrade

So to get this:

Concurrency	processes
Error encapsulation	isolation
fault detection	what failed
Fault identification	why it failed
Live code upgrade	evolving systems (multiple versions at same time)
Stable storage	crash recovery

Isolation is key here: It's why hardware components are easier to build. Erlang is designed to operate concurrently via message passing, with isolation. So:

Processes have share nothing semantics and data must be copied. Message passing is the only way to exchange data. Message passing is asynchronous. My program should not be able to crash your program.

Programming for Errors: If you can't do what you want to do, try and do something simpler :) Using these techniques, they've built a 9 nine's reliable GPRS system in Europe.

Technorati Tags: esug, smalltalk, erlang

It's Thursday morning, and Georg Heeg is up to talk about the ObjectStudio 8 project. The Heeg company was founded in 1987, and has been involved in Smalltalk since that time.

OST and VisualWorks:

• ObjectStudio and VW are Smalltalk systems
• Both owned by Cincom
• VW is descended from the same source as Squeak (ST-80, Xerox PARC)
• ObjectStudio was developed under the name Enfin, and was originally designed as an "Enterprise OO Development Environment"

Enfin

• Ease of use
• Enterprise integration

VisualWorks

• Execution speed
• Sophisticated meta-model

Customers want both, but don't often know that they need the meta-model. So - why is Cincom doing this, given the sad history of product integration (across the industry). There was an HPS2OS project that did not succeed. However, Georg points out that Smalltalk is all about modeling - so we should be able to model one system in another.

So ObjectStudio 8: Model ObjectStudio using VisualWorks meta-modeling. It uses reflection capabilities:

• Compiler
• Debugger
• Namespaces

There were significant differences, including such things as the process model. Both OST and VW live in the same image. They share the same Smalltalk kernel and most of the same base classes. Both base OST and base VW code work (mostly) unchanged from previous releases.

The idea for this was originally Georg's - he had vocal chord surgery in 2004, and couldn't speak for 2 weeks - he started this project then, and it's become a full project inside the Cincom Smalltalk team. After that time, he could load the OST codebase - didn't all work, but it did load.

After that, he hired a new team member, Jorg, who had worked on the Windows CE VM as an intern. He's been doing much of the DLL integration for ObjectStudio 8, as well as primitive migration. That started at a rate of one primitive per day (there are 450!). That's where things went to making the OST C/C++ codebase into a VW callable DLL. He started that in February of 2005. He got a basic version completed by JUne of 2005.

At that point, compatibility testing started in earnest. If a component didn't work right, they wer either fixed, or replaced. For instance, ObjectStreams have been replaced by BOSS. This took much longer than originally anticipated :)

The original strategy was to not modify the VW VM at all. Some bugs did need to be fixed, and the Event system had to be changed somewhat - but that was it. The goal for ObjectStudio 8 on the Smalltalk side was to make VW namespaces invisible. Classes live in namespace ObjectStudio, globals live in ObjectStudio.Globals. Here's Georg in front of a (not clear in the picture) architecture diagram:

And here's a capture of the image Georg sent me afterwards:

Another set of issues came up in the way code is stored. ObjectStudio uses a different chunk format, and also stores code in CLS files (one class per file) - with additional complexity in secondary files (somewhat like the .NET partial class files, actually). The answer was a new compiler for ObjectStudio, with a new parser (both written in VW, at the Smalltalk level).

There were also some syntax differences - in OST, you can use:

{ ["code here"] ["code here] }

to define an array of blocks. The VW compiler didn't do that, so they created code transformations that changed that kind of code over. As well, in ObjectStudio, you can assign to a method parameter. Again, transformations were defined to generate temp variables. Finally, there were some semantic differences - here are a few examples:

Code	VisualWorks	ObjectStudio
1=0 ifTrue: [1]	nil	false
[:n | ] value: 1	1	nil
Array new add: 1	Error	#(1)
(Array new: 1) at: 1 put: 7	7	#(7)

The team defined a set of substitutions in order to execute ObjectStudio versions of messages that are like the examples above - i.e., where OST specific behavior should be preserved.

There were also mappings for the ObjectStudio chunk formats, so that OST 8 can read and write the classic OST file formats. In ObjectStudio, there is a very string integration between Smalltalk and C: there are numerous cases where the image and the VM call back and forth. OST has always been intepreted, so optimizing this callback scheme made sense. So there are five ways to call C:

• Direct Primitives
• Numbered Primitives - <primitive: 123>
• Named Primitives - <osprim: MFC openWindow>
• Module/EnfinModule
• ExternalProcedures

To handle this, a wrapper DLL was created using DLLCC. It uses __oop as a parameter and return type. It calls ObjectStudio primitives using a simulated ObjectStudio stack.

Another issue: In OST, object pointers never move - and this assumption gets used (for instance, in Windows API calls). In VW, the garbage collector moves object pointers all the time. The solution was to make OST OPTR objects into a C++ wrapper class, which handles this. Primitives either map directly to VW primitives, or got replaced by auto-generated replacement code, which goes into the wrapped DLL.

In ObjectStudio 8, Unicode capabilities are available, period. No more variant VMs and images.

Finally, the process model differences.

VisualWorks

• Processes run at different priorities
• Windows Events are handled (almost) any time
• Shared queues move events to the target processes
• Implemented in C

ObjectStudio

• The caller of all execution is the Windows event model
• All message sends in the ASendQueue are executed when the Windows Event is empty
• Implemented in C++

This is where some modifications to VW event handling had to be made. There are specific subclasses now that communicate directly with the Windows Event system, rather than a generic cross platform wrapper.

ObjectStudio also has some interesting features missing from VW: for instance, class proxies. It has the same name as a missing class, and handles DNU by loading the missing class/application. This gives ObjectStudio lazy loading. In VW, you have parcels, but they don't do lazy loading.

A few Performance Issues:

• In OST, Smalltalk message sending is slow, while calling C is "free"
• In VW, message sends are fast, while calling C is expensive
• This has driven different optimization schemes :)

Another issue: file-in some .st code in VW:

• If a load takes 1000 seconds
• 900 are refreshing the RB
• Of 100 seconds
• 90 are repeatedly relinking the system
• Of 10 seconds
• 9 are disk flush
• 1 second compiling

This has been optimized. A pleasant side effect is, it helps both VW and OST. New stuff for ObjectStudio: Store, the RB, the debugger, the inspector, related tools (code critic, etc). Note: ObjectStudio developers do not need to migrate to Store. They can continue to use the same source code control tools they use with their current ObjectStudio code.

Where are we? We have been working closely with a customer for feedback and porting data. We intend to have a first preview (beta) ready for general customers to see in the winter. Some known incompatibilities:

• Immutability
• Declare pool dictionaries
• In the modeling tool, export in ASCII (binary format is incompatible)

Georg's parting question: Integration of Smalltalk systems is possible. So, are there other systems that should be integrated? Squak, VA, VSE? Smalltalk-X, Dolphin, Gemstone? What about other languages, like Ruby, Java?

Technorati Tags: smalltalk, esug