Making Smalltalk Easier

Two months ago I left Cincom, and this is the last post on this blog. The work on VisualWorks tools was an interesting period which taught me much. I worked with great people on a cool product, and I am happy to have had that chance. As is also often the case with interesting periods in life, I also feel happy about the change that followed. And with that change, I am starting a new blog as a more appropriate place for writing about things not necessarily related to Cincom Smalltalk.

 Farewell.

Here is another experimental package from the recent past:

VB-Sketch-Browser History

It adds a History menu to the browser, containing the most recently visited classes and packages.

I know of at least one other recent attempt (RBrowserHistory in the public repository) to add history to the browser, in the form of forward/backward buttons for sequential navigation. My Browser History use of a menu instead of the buttons is not at all a statement against the sequential approach. Rather, my focus was on playing with defining the concept of a location--something one really needs to implement any kind of history mechanism--not with the particular method of navigation. Fitting a menu into the RB simply does not require the level of voodoo that fitting navigation buttons does, this is why it was the natural first choice. Here is what the whole location business is about.

The classic Smalltalk browser defines a deeply hierarchical information space with semi-random access. The concept of a location in that space can be defined in a number of ways. Unlike in a hypertext web of pages a web browser deals with, it is not at all clear when exactly you leave one "place" and arrive at another in a Smalltalk browser. At least it is not clear if you want the history mechanism to be useful.

Useful is the key here. The obvious approach is to remember each and every selection state in the navigator and consider those the locations, and their sequence the history. As the user clicks the Back button, we just walk back through all of the remembered states. Logical enough, but consider this very common navigation scenario. I am about to write or change a method, but before I do that, I want to refresh my memory of another class's API. So I go to that class, perhaps select a likely protocol to narrow down the list of methods, look at a couple of methods that look promising, maybe select a few other protocols and look at a few other methods to make sure I understand what's going on. Now I want to get back to where I was before the exploration started. The "remember everything" approach will make sure I go back though every browser state I visited, including rather uninteresting states such as when only a protocol and no methods are selected.

In order to properly support this scenario, we need to recognize that some browser states are less important than others. And some states are so much alike that they should be considered the same. Here is how Browser History understands locations. (This definition is indeed geared towards random access--I imagine sequential access might require adjustments).

The core idea is combining two levels of granularity in defining a location. One is the package/class level, which defines what is considered a distinct location. The history keeps track of visits to classes and packages. Changes between different protocols and methods within the same class are not considered location changes. However, these visits are not ignored. When the history remembers a visit to a class, it always remembers the last exact location within that class. So, even though the granularity of what is seen as distinct locations are package and class level, the level of detail remembered about that location is the exact selection in that package and/or class.

In other words, you can choose to go back to a particular class or a particular package. Going back to a class takes you to the last method you were looking at in that class. Going back to a particular package takes you to the last class and method, if any, visited in that package.

This sketch, just as DontModeMeIn, ignores views/buffers and related things such as RB_Tabs. This is for two related reasons. One, I don't care to support something I don't use in experimental tools I make. Two, if I haven't made is sufficiently clear in the previous post, I have a strong suspicion that buffers/views come as a solution to the problem of edit modality and absence of browser history in traditional browsers. In fact, discussing these two sketches with Bob Westergaard, I was referring to them as the buffer killing experiment.

And one advice: the detailed class icons published by Travis (which I also published a few days before as VB-Icon Update-nov06) go very well with the History menu, often making it easier to scan.

This post is about an experimental piece I wrote some time ago, which for a number of reasons hasn't made it into VW proper--the main being the depth of changes it makes to the basic logic of the RB, and how unreceptive RB in general is to such changes. However, it is stable enough to be useful as an unsupported add-on. It is published to the public repository as

VB-Sketch-DontModeMeIn

"Don't mode me in" was a slogan reportedly coined by Larry Tesler in the early days of UIs, which also were the early days of Smalltalk. The switch from an application-centric culture where an editor was one application, a compiler another, and a debugger yet another again, all used one at a time--to a window-based environment where all of these were just windows all available at once must have been quite a change.

So much of a change that perhaps the modes the individual tools of the environment were imposing on the user did not seem as such a big deal. The mode I mean is the editing mode of the traditional Smalltalk browser. And by the traditional Smalltalk browser I mean not only the browser of Smalltalk-76 and -80, but also every hot browser that came into existence since then. Once you start editing a method, you have to either accept or cancel before switching elsewhere. Makes sense, doesn't it--after all, this is how it's coded in any UI tutorial. What was the name of that method in the class just above this one? Sorry, can't let you go there--must accept first!

Short of keeping two browser windows side-by-side--one for work, the other for looking things up, which really only works on nice and wide Mac screens--obviously this is why one needs multiple browser views, fondly dubbed "buffers" by some with either a very good memory or too much exposure to software antiques.

Obvious solutions often distract attention away from the best ones, while those best solutions can be hidden in plain view for years. The best solution here is simple: Just Make it Work. There is no good reason why I shouldn't be able to postpone an edit, navigate elsewhere in the same browser, and then come back to finish what I was doing.

VB-Sketch-DontModeMeIn is an attempt to fix that within the constraints of the existing RB and the UI framework.

Load it. Close any open browsers. Open a new one. Look at a method source. Start changing it. Without accepting, click on another method. Or another class. Or search for a class. The button with a red circle and white dots at the bottom of the browser is the link back to the postponed edit left behind. The same works for class comments or any other tools. For example, while writing a class comment you can have a peek at class methods to remember what that instance creation method was called.

The interesting part is that this change might not seem as such a big deal after all. I read somewhere that if you put a piece of glass across a fish tank like a wall, the fish get so much used to it that it takes a long time before they start going into the other half of the tank after the glass is removed. In some ways, we are still not that far away from the fish.

And here is another package, or a group of packages this time, just published to the public repository:

• Tools-State Machine
• Tools-State Machine-Tests
• Tools-State Machine-Pollock Example

A state machine is a useful abstraction in specifying user interface interactions. So useful in fact, that it's handy to make the abstraction concrete. The main package, Tools-State Machine, contains an implementation of a state machine integrated with Announcements. Tools-State Machine-Pollock Example contains an example of an Assistant, a Pollock pane helper providing pluggable functionality, implemented using a state machine.

In the current VisualWorks system, tooltips ("fly-by help") implementation is based on a state machine. However, the state machine transitions implemented by FlyByHelpTracker are hard to track even for myself who wrote it. Each transition from each state is a method of its own, the state shared by these methods is stored in instance variables used throughout the class, and the logic of how the thing actually works is evenly spread over a maze of little methods, all different.

OverlayAssistant in the example package also implements a state machine, but thanks to the state machine framework, its implementation is much easier to comprehend.

First, a few words about Assistants. Assistants have become part of Pollock sometime ago. They are envisioned as pluggable components that can be attached to standard panes to add non-standard behavior to them. Once attached, an assistant listens to announcements broadcast by the pane and reacts to them by doing things with panes that panes themselves are not programmed to do.

A good example of both an Assistant and the use of a state machine is an implementation of list item overlays. By an item overlay I mean a tooltip-like little window that pops up when the mouse hovers on an item which is too long to fit in the list, showing the complete text of the item. (One could argue these are also tooltips, only displayed differently. Perhaps, but I like to think of tooltips as temporary labels, and of item overlays as "real" items temporarily "pulled out" of the confines of the list that contains them. So I call them different names).

Pollock list boxes announce ZoneEntered and ZoneExited as the mouse moves over the widget, crossing over from one item ("zone") to another. These are exactly what's needed to drive overlay display, and in fact were added to Pollock to support that. (Though they are not essential--we could listen to "raw" MouseMoved announcements and track item boundary crossovers ourselves). When the mouse enters an item that could benefit from an overlay, we want to display the overlay after a short delay. But if the mouse leaves the item before the delay expires, we lose interest in that particular item. After we open a popup, we want to close it after a while or when the mouse leaves the item. Scenarios made of cases like these are a perfect job for a state machine.

OverlayAssistantExample class in Tools-State Machine-Pollock Example is a reasonably complete implementation of overlays. See the class implementation and the class-side #example method for details and an example of attaching an assistant to a pane. Here we'll show only the two key methods: #setPane:, the initialization method of any Pollock PaneHelper, and the method that actually builds the state machine.

    setPane: aPane

	super setPane: aPane.
	self buildStateMachine.
	stateMachine
		subscribeTo: ZoneEntered from: pane;
		subscribeTo: ZoneExited from: pane;
		start

    buildStateMachine

	| coldState armedState reaperState indexToOpen |
	stateMachine := StateMachine new.
	coldState := stateMachine newState.
	armedState := stateMachine newState.
	reaperState := stateMachine newState.
	coldState
		on: ZoneEntered do:
			[:announcement |
			(self itemNeedsOverlayAt: announcement index) ifTrue:
				[indexToOpen := announcement index.
				armedState enter]].
	armedState
		on: ZoneExited do:
			[coldState enter];
		afterMilliseconds: 200 do:
			[self openItemOverlayAt: indexToOpen.
			reaperState enter].
	reaperState
		on: ZoneExited do:
			[self closeItemOverlay.
			coldState enter];
		afterMilliseconds: 5000 do:
			[self closeItemOverlay.
			coldState enter]

Note how the machine is made of three states. "Armed" is the state we enter when the mouse enters an item that needs an overlay. If we linger in that state for longer than 200 ms, the overlay opens and go into the "reaper" state. The reaper state is where we are when there is an open overlay. We leave the reaper state and remove the overlay either when the mouse leaves the item or after a five second delay.

Also note how state transition actions are able to share information such as "indexToOpen" by virtue of being blocks inside the same method.

As another example, we'll have a look at implementing "regular" tooltips for a toolbar. This assistant is not included in the published example, but it should be easy enough to reproduce for those who are interested.

The difference between this assistant and the overlay one is in the announcements ToolbarTooltipAssistant listens to. The announcements are PaneEntered and PaneExited, sent by the toolbar subpanes (the individual buttons) rather than the toolbar itself. The assistant itself subscribes to SubpaneAdded announcement from the toolbar (announced each time a new component is added to it). When the announcement is broadcast, the assistant instructs its state machine to subscribe to the PaneEntered and PaneExited events from the subpane. To be neat, the assistant also listens to SubpaneRemoved announcement and unsubscribes the state machine from the announcements sent by the removed subpane.

The actual state machine resembles the one of OverlayAssistant, though it's a little more complex.

    buildStateMachine
	| coldState armedState reaperState warmState paneEntered |
	stateMachine := StateMachine new.
	coldState := stateMachine newState.
	armedState := stateMachine newState.
	reaperState := stateMachine newState.
	warmState := stateMachine newState.
	coldState 
		on: PaneEntered do: 
			[:announcement |
			paneEntered := announcement pane.
			(self hasTooltip: paneEntered)
				ifTrue: [armedState enter]].
	armedState
		on: PaneExited do: 
			[coldState enter];
		afterMilliseconds: 500 do: 
			[self openTooltipFor: paneEntered.
			reaperState enter].
	reaperState
		on: PaneExited do: 
			[:announcement | 
			self closeTooltipFor: announcement pane.
			warmState enter];
		afterMilliseconds: 5000 do: 
			[self closeTooltipFor: paneEntered.
			coldState enter].
	warmState
		on: PaneEntered do: 
			[:announcement | 
			paneEntered := announcement pane.
			(self hasTooltip: paneEntered)
				ifTrue: 
					[self openTooltipFor: paneEntered.
					reaperState enter]];
		afterMilliseconds: 300 do:
			[coldState enter]

The addition is an extra state, warmState. It is required in toolbar assistant to support easy scanning of tooltips. Once a tooltip for an item is displayed, if the user moves the mouse cursor to a different item, the proper behavior is to display the tooltip for that item immediately rather than after the usual delay. This way, it's easy to "swipe" the toolbar with the mouse looking for a particular item, without having to wait half a second at each button for its tooltip to open. To support that, when the mouse leaves a pane with an open tooltip (PaneExited in reaperState), the machine switches to warmState rather than to coldState as in the overlay assistant. In warm state, entering a different pane displays the tooltip immediately. If the machine stays in warm state for 300 milliseconds and no other panes are entered, it "cools down" and reverts to the usual policy of waiting before opening the tooltip of a visited item.

Compare readability of the method above with the same state machine implemented as methods in the 'events' protocol in FlyByHelpTracker.

This is an explanation of what the package Tools-Resource Library just published to the open repository does, and how it does that.

VisualWorks applications, at least most of those I've been working on, use images prepared outside of the VisualWorks environment. Some of the older icons I made, including those in the Launcher and the RB were created using CoolImage--but nice as it is compared to the default Image Editor, it is no Photoshop killer. And arguably, replicating Photoshop or Gimp in Smalltalk is a waste of time better used otherwise. This means that bringing images (and perhaps some other resources) prepared in external applications into the image is a task worth supporting.

Currently, the class ImageReader can import external images, creating resource methods in one of the icon libraries. Unfortunately, this workflow is hard to manage, as the association of resource methods and their source files is not preserved. It's hard to keep track of what resource methods need updating after some of the source files change. The only workaround is maintaining a master import utility method that reimports all of the files "wholesale".

Additionally, the IconLibrary facility itself is showing its age. The mechanism that maps icon names to resource methods is rather complicated, mostly because of its unused ability to support multiple icon sizes and because of caching in the library.

Tools-Resource Library addresses all of these deficiencies. As an introduction, here is how one would use it in an application.

Begin by creating a subclass of ResourceLibrary. For the sake of the example, let's assume the class name is FooResources.

For each image you want to import in the library, write a class-side resource method marked with a pragma identifying the resource type and source file name. For example:

productIcon
    <opaqueImage: 'product.gif'>

The method (for now) needs no code in the body other than the pragma. The selector and the protocol are entirely up to you. We are assuming that product.gif file is present in the current directory.

Now, in a workspace near you evaluate the expression "FooLibrary regenerate". Now look at the two methods again. If the files mentioned in the pragmas were indeed accessible, regenerating replaced the code of the methods with something like (omitting some non-essential comments):

productIcon
	<opaqueImage: 'product.gif' >
	"Generated from: product.gif
	timestamped: March 31, 2006 22:10:50.000
	on: January 15, 2007 11:08:50.161"

	| figure shape |
	figure := CachedImage on: ...stuff... .
	shape := CachedImage on: ...stuff... .
	^OpaqueImage figure: figure shape: shape

This shows the first thing that is special about ResourceLibrary and its subclasses. They can maintain methods with data imported from external files, and update them as necessary. Each time the class is sent the #regenerate message, the timestamp of the imported file stored as a comment inside a resource method is checked to see if the method needs updating.

All that's necessary to get the imported image is evaluating the expression "FooLibrary productIcon", which will return an instance of OpaqueImage, just as the method code shows.

Now, what about efficiency? It seems that every time the method is executed a new OpaqueImage is created. This certainly is a waste of both cycles and storage since these images could be cached and shared, as IconLibrary already does. Let's look at the method closer. Inspecting it shows that its bytecode in fact looks as follows:

    short AnnotatedMethod numArgs=0 numTemps=0 frameSize=0

    literals: (an OpaqueImage )

    1 <1C> push an OpaqueImage
    2 <65> return

What we have here is essentially compile-time evaluation at the method level, performed by the special compiler assigned to resource libraries. The code in the method body was evaluated by the compiler and saved as a literal returned by the method. Or, to look at it in a different perspective, we do have caching, with each resource access method doubling as a cache entry.

Only resource methods (those marked with one of the predefined resource pragmas) on the class side of ResourceLibrary and its subclasses are compiled in this fashion. All other methods are compiled and work as usual. The can also be filed out or published to a Store repository.

The following resource types (pragma keywords) are supported:

opaqueImage: -- an image file is imported and stored as an OpaqueImage. Supported image formats are those supported by ImageReader. As of VW 7.5, .gif and .png are best bets. Note that OpaqueImage does not support alpha channels, so only .png files with full transparency/opacity can be meaningfully imported.

image: -- an image file is imported and stored as a CachedImage on an Image.

xmlDocument: -- an XML file is imported, parsed and stored as an XML Document.

binaryFileContents: -- the contents of an arbitrary file are imported and saved as a ByteArray.

A library class understands the following messages of notice:

#regenerate -- regenerate the source code and recompile all resource methods that are out of date. "Out of date" means either that there is no "timestamp" line in the method, which is interpreted as a method that has not yet been generated from its file, or the timestamp of the source file remembered in the method differs from the file on the disk. If the source file of a method is missing, the method is left untouched.

#regenerateAll -- regenerates all resource methods of the class, regardless of whether they are out of date or not.

#pragmasWithMissingSourceFiles -- returns Pragma instances pointing to the methods whose original files are missing.

By default, files mentioned in resource pragmas are looked up in the current directory. ResourceLibrary defines a class instance variable rootDirectory, which can be set in each subclass to point to a different directory to look in.

After a long period of stability, there is a new version of Announcements in the public repository, preview-28.0. The change it contains is unusual in that it removes rather than adds functionality. It can therefore affect people who already use announcements.

Removed is the ability for an arbitrary object to remember subscriptions. There is a new class in System-Announcements, called Announcer (which previously was included as an example in the Systems-Announcements-Extras package). In order to be able to accept subscriptions, an object should either inherit from Announcer or implement in a sufficiently similar way the two or three methods that Announcer implements: #subscriptionRegistry:, #subscriptionRegistryOrNil and (typically) #postCopy.

In the original design, any object, including something as primitive as a SmallInteger, could accept subscription requests, remember the subscriptions, and later deliver announcements to the subscribers. This was possible because Object implemented subscription storage using one external global registry, following the tradition set by change/update (DependentsFields) and triggerEvent (EventTable). Only classes that wanted the lower cost of local subscription storage would care to store subscriptions in an instance variable instead of inheriting the global mechanism from Object.

The problem with that tradition are a number of subtle issues in the interaction between the external registry, #become:, and collection growth logic, as well as the risk of garbage accumulating in the global registry. A detailed review would easily make up a post of its own going some fifteen years of VW evolution back, but for now suffice it to say that the issues were bad enough that we decided to explore the alternative approach, more in the spirit of "worse is better".

The common case is that a relatively small number of specialized announcement sources, such as Pollock Panes, does indeed broadcast events. Such objects often replace the default storage management with local storage as a matter of optimization. In the new Announcements, this is no longer an optimization. Now it's strictly Bring Your Own SubscriptionRegistry. Any object that wants to accept subscriptions must make its own arrangements to hold onto its subscription registry.

I've been careful to say that an object has to make these arrangements "to be able to accept subscriptions" rather than "to be able to broadcast announcements". Conceptually, the framework still considers all objects as capable of broadcasting announcements. It's just that not all objects will accept subscriptions. From the framework's point of view, any newly created object starts off with zero subscriptions. It doesn't matter whether the object is an instance of Point or Pane. It is OK to send #unsubscribe: to both of those, which will be a no-op since there are no subscriptions to remove. It is even OK to send #announce: to both, which again will be a no-op as there are no interested recipients. The only difference is that one can send a #when:... message to Pane to create a new subscription, while sending it to Point will cause an error.

To meta-comment on some of the comments to the last post.

Like Pollock is nothing but a reasonable UI framework, Splash will be nothing but a reasonable UI builder. If you are hoping for the "blue plane" stuff with Splash somehow naturally blending with regular windows so that you can edit any window you work in and work in any window you edit--you will be disappointed. We don't serve this here at this restaurant.

Which is not such a bad thing, all you blue plane people out there. As an anonymous commenter observed, UI building is not rocket science. That certainly is true--whatever is meant by UI building, the building of UIs or the building of UI builder UIs. What is somewhat puzzling considering that is the abundance of crappy UIs, including UI builder UIs, out there. So what Splash is supposed to be is "simply" a UI builder that doesn't suck. Which is not little.

As a mental experiment, imagine that the current VisualWorks UI builder is beefed up so that it gets fully dynamic. You can switch any window to the edit mode and back, and any edit you make is propagated to other window that came from the same spec. Java weenies shrivel up and die at the mere sight of this powerful dynamism. This isn't that hard to do, really. Would it produce a better UI builder?

I am a very active UI builder user, and my answer is a resounding NO. Making the current builder more dynamic would address none of the issues that make working with it a pain. Here are some off the top of my list. Note that some items imply more than they say.

1. The model of VW UI builder has nothing in common with the mental model of a typical user. Everyone comes to UI building with the concept of "I'll put an input field and a button inside a window and do something simple to put a message in the field when the button is clicked". Nobody comes with concepts of "I'll define an aspect for the input field, create the aspect method and then set the aspect when the button is clicked".
2. Widget properties all neatly packed into ten different notebook pages all with an Accept button to trip over when switching to a different one is not the most usable properties representation.
3. Typing numbers into fraction/offset input fields is not how people think about laying out the widgets.
4. No undo. There is just no excuse for that.
5. Separate windows for the UI under editing, its parameters, and the tools palette were sexy in 1990 when NeXT was the hottest thing around. In the 16 years since, the world has learned the cost of unnecessary window management. (Yes, OS X Interface Builder sucks too).
6. Interaction detail, which is where the devil is. There's just too much little interaction things UI Builder got wrong. You don't initiate a drag as soon as the user presses down on the mouse button. You wait until they've dragged some initial threshold--or they will end up moving their widgets by a pixel as they click them to select. In all modern UIs, rubberband selects any objects it overlaps, not just those it encloses completely.
7. And yes, things are too closed off on the runtime side in VW. Given a window, there is no easy way to even debug it, let alone get to its definition to change it. This needs fixing too, and full dynamism is not the only solution.

So what Splash will be is simply something that won't have those issues, and others like them. And it will be really modular so that anyone can use any part of it for great fun and profit. Such as writing a whole new tool you can plug into Splash. Or writing a whole new Splash-like builder to edit live windows, instead of just blogging what a nice thing it would be.

Now for the show-and-tell. Because of sickness and boring work stuff, I only had a few good days to work on Splash since the last post. Here is a window with Splash layout editor inside.

What looks like a window without a title isn't a real window, as the selection frame around it suggests. This is to reinforce the point that Splash will not have a multiple-window setup where the UI you edit is a real window on the screen. Windows95-like widgets inside are there just because I'm using a Windows95 look. The editor takes after that, even though the fake window in the editor for now unconditionally paints itself as an OS X window.

This shot is the same window, resized and with widgets rearranged and resized, all using the editor. You can't see it in the screenshot, but dragging and rubberband selection work as they should, little details included, and with wrinkles such as holding down Shift to constrain drag to vertical and horizontal direction and to constrain resizing to preserve the original proportion.