Objology

A Mechanical Engineer's Thoughts about Unit testing

Tue, 09 Feb 2010 03:16:09 GMT

"An idea, like a ghost, must be spoken to a little before it will explain itself." -%A0Charles Dickens

Sometimes, we do things in our programming practices that just seem right. But we may struggle to figure out how to illustrate why it feels right to colleagues and peers. We have a vague idea, and some of the words involved, but a succinct explanation evades us. This post, is an attempt to shed a little light on some of the principles I follow in regards to "unit testing." Some of this fell into place, just days ago, though I've been doing this a certain way for 12 years now.

Back in the 50's, there was a sort of revolution that took place in the manufacturing world. Quality was the issue. Leading role was played by E. W. Demming. Some of the plot themes were things like process control, quality control, statistical process control, shewart charts, kanban cards, etc. It was a big hit in Japan initially, and would eventually make its way over to the States and other places. As a Mechanical Engineering student, I had this stuff pounded into my head through long years of college.

One of the things that struck me about these ideas (ideas that would eventually be caught up in and respun as "lean or agile" manufacturing practices) was a subtle shift in quality testing. Many factories convinced themselves that they had quality because they did testing. Classic manufacturing testing philosophy, said that you assembled the gadget, and at the end of the factory, you ran the assembled device thru a screen of tests. If it passed these tests you shipped it. In this era, the bolt vendors could be pretty lax in their tolerances; assembly personal at the plant would take up the slack, pairing bigger bolts with bigger washers and smaller bolts with smaller washes. As long as the machine at the end performed it's function nominally, it was good enough. Of course, when the device came under stress, this variety led to failures and subpar performance.

Demming and crew, felt that this was pointless to catch defective products that late in the cycle. It was wasteful. So they pushed testing and quality control back into the process. They advocated tolerances on the component parts, rather than the unit as a whole. The basic idea is that if you tighten up the quality on the parts that compose your device, your device ends up the better. In fact, if you had a controlled process, and high quality inputs, then you could confidently ship higher quality products while actually reducing end testing. In years to come, those that made the change to this mentality survived, those that did not, mostly failed.

It's instructive to me, that while both methods advocated and adhered to "testing", the Demming approach and application of testing was the one that produced better results.

Since the "splashdown" of XP in 1998 and the following furor of Unit Testing in Software, I've witnessed similar parallels in the application of testing software. Everyone claims they do it. When Ken Treis and I came home from OOPSLA 98, we determined we'd try this. We put together a version of a Wiki server for VisualWorks, and we used workspace scripts to make test that we got the right http responses when we sent different kinds of queries against our server. And having done that, we went right on writing software the way we always had. It didn't really matter how we architected the code as long as it passed these high level tests. Of course, we kept discovering all kinds of edge cases. Though I was well versed in the Demming school of thought about testing the components, rather than the end product, I missed the potential corollary.

It wasn't until Camp Smalltalk 2000, when Ken had the chance to work with Ron Jeffries, that we got a chance to see it done right. Ken saw, and quickly shared with me, what a real unit test looked like. The light bulb went off for us. This would change the way we developed our algorithms at a component level. I've been a happy unit tester since. I am firmly entrenched in the belief that unit testing is to software what component testing is to manufacturing. Usually, it takes the form of having a particular test class for each meaningful class in your system. There are of course adaptations, but more often than not, I find that the adaptations are rationalizations that allow people to test in the "old school" style while telling themselves they're doing it differently and making a real difference.

In manufacturing, the point is to reliable reproduce and accepted design. In software, we don't do that. The ability to reproduce our software designs, is not in question. The creative part of what we do, is to create algorithms, of various sizes and shapes, of varying complexities. When we write unit tests for the components (or objects) of our algorithms, we're embracing the same set of principles that drove the quality revolution. Try it out. Don't write so many tests for the overall features of your system. Try writing tests for your components. One to one.

Superpower Adventures in Lightweight Classing

Wed, 13 Jan 2010 17:32:36 GMT

Adrian Kuhn has been blogging a series of Smalltalk Superpowers. What a wonderful blog topic. And a great job Adrian is doing too. He's a worthy Smalltalk Superhero.

I'm having an adventure in superpowering right now. I need, for reasons as of yet undisclosed (superheros live secret lives, ya know), to have my own subclass of UILookPolicy. UILookPolicy participates with UIBuilder objects to compile (build) user interfaces from WindowSpec objects. Much of its behavior is in UILookPolicy, but it is in fact an abstract class. It has a handful of platform specific subclasses.

The problem, is that I don't want a whole new subclass. I don't want to re-implement all of UILookPolicy. What I want is a generic subclass of whatever the default subclass is, whether it's a MacOSXLookPolicy or a WinXPLookPolicy.

But the subclass card has been played already. To get that kind of structure with normal everyday living, I'll have to make a subclass of each subclass. Or...

I can step into a booth and don my Lightweight Classes costume. Here's what it looks like


UILookPolicy>>myOwnPolicy



	| lightweightClass |

	lightweightClass := MyLookPolicy copy.

	lightweightClass superclass: self class.

	^lightweightClass new

With this costume on, I can send the message myOwnPolicy to any subclass instance of UILookPolicy (e.g. a MotiffLookPolicy instance) and end up with a new instance which implements the behavior of MyLookPolicy as if it were a subclass of that class. When in reality, it's a subclass of UILookPolicy, and by default, a sibling to the likes of WinXPLookPolicy and friends.

This allows me to derive from any standard LookPolicy object, one that overrides methods like button:into:, but otherwise, inherits all of the behavior of the original instance.

Cool Cairo Machinery

Mon, 09 Nov 2009 23:20:17 GMT

Chris Thorgrimsson sent me a link to his new blog where he's talking about some of the things they're doing with Cairo and VisualWorks. The demos and stuff I show are OK, but it's always way cooler to see it in real life stuff. Thanks for sharing Chris. Chris actually prompted this post a while back.

Herling to the Top of a Micro and Soft Hill

Thu, 15 Oct 2009 01:50:56 GMT

It's the hill climbing factor. It doesn't matter if you've just climbed the smallest mole hill next to Mount Rainier, if you've worked hard to get to the top of that hill, and you've struggled to get there, you experience the biggest high having achieved a local maximum when you get to the top.

I've often theorized that this is why the masses never have taken to the higher level langauges such as Smalltalk, Lisp, ML. Building and enumerating a doubly linked list in C is a challenge! Doing it in Smalltalk. Meh. You run out of easy hills to feel good about climbing.

Tonite, I'm feeling that kind of feeling. After banging my head, googling through obscure references, I've got to the point where I can do a one-click doit to download all of the Cairo, Zlib, Libpng sources and compile Zlib and Libpng, using Microsoft's freely available Express tools.

I'm simply amazed that Microsoft took over the world, so to speak, with such an operating system/environment, where automating anything is so difficult. The flip side, is that getting an automated download and configure/compile gives me a huge rush.

On the way, I'm having continued fun with Herl. I've further fleshed out some of it's OS interface parts for Windows now. Keeping in the tradition of "making it up as I go."

Herling CairoGraphics

Sat, 10 Oct 2009 02:28:52 GMT

This last week, I've been trying to get CairoGraphics libraries built on the breadth of platforms we support VisualWorks on. Doing so makes it easy for customers to use CairoGraphics with VisualWorks in their own projects, and if we manage to get ALL the platforms, it makes it so we can begin to improve our own IDE with its capabilities. It's unknown how many, if any, we'll get included for preview for the 7.7 release, but something is better than nothing. Windows and MacOSX seems a good probability at this point. X11 is proving more difficult.

Having built CairoGraphics libraries a couple of times, over the last year or so, with long pauses in between, I often find the documentation is not quite up to date, that I end up having to ask questions of their very helpful mailing list or irc channel. I'm not GNU build guru, so they're probably naive. Which led me to want to script the download and building of these libraries. So that it was reproducible. And precisely documented.

Bash is the script environment I'm most comfortable with. But I'm no wizard. Especially when it comes to doing higher level algorithmic things. I have to ask for help for those, and I end up missing Smalltalk anyway.

So I thought I'd take the rudimentary ScriptingSupport package that we include as preview in VisualWorks, and try that. At this point, it's still pretty young. It implements stuff to make command line execution easier, and adds the & binary sugar, which is a handy way of concatenating single objects onto collections (as opposed to , which concatenates multiple elements).

And then I just decided to have fun. I followed the following design principles, which seem to be at the heart of the design of most scripting languages.

Typing sucks. It's no fun. So add lots of sugar to make things terse and conserve keyboard srokes.
Use lots of infix characters.
Make it up as you go. As you encounter new situations, add more stuff, don't try to come up with any sort of simple unifying theme. Just throw more stuff on.
Typing sucks. The other kind. Support few basic types, and abuse them. Just mash them together. You should be deluded into thinking you have don't have to worry about types, except now and then again in surprising ways. Don't make me convert from one type to another.

The only non-traditional requirement I added, was that I don't want to build a new parser or anything. So it has to strictly speaking, still be Smalltalk.

It's still a work in progress; I'm not out to start a movement. I expect derision and flames. Its all a lark to me right now. I've named my little Domain Specific Scripting Language "Herl". Ken Treis actually suggested the name; thanks Ken. It's been replicated up to the Open Repository and has a package comment that looks like any scripting language reference manual.

For those who aren't that interested (all of you likely), a snapshot of the "script" I wrote with it, is probably enough.


rootDir := '$(HOME)/BuildCairo'.

rootDir rm.

rootDir mkdir.

rootDir cd.



"Download everything"

page := #(curl   -s 'http://www.cairographics.org/releases/' ) exec.

cairo := (((page -@ $>)  detect: [:each | 'LATEST-cairo-.*' ?= each]) -@ $<) _1.

(#(curl -s) & ('http://www.cairographics.org/releases/' , cairo) , #(-o 'cairo.tgz')) exec.

pixman := (((page -@ $>)  detect: [:each | 'LATEST-pixman-.*' ?= each]) -@ $<) _1.

(#(curl -s) & ('http://www.cairographics.org/releases/' , pixman) , #(-o 'pixman.tgz')) exec.

page := #(curl  -s 'http://pkgconfig.freedesktop.org/releases/' ) exec.

pkgconfig := ((((page -@ $>)  select: [:each | each =? 'pkg-config-.*' ]) sortBy: [:each | (each -@ $.) _2 asNumber]) last -@ $<) _1.

(#(curl -s) & ('http://pkgconfig.freedesktop.org/releases/' , pkgconfig) , #(-o 'pkgconfig.tgz')) exec.

#(curl -s 'ftp://ftp.simplesystems.org/pub/libpng/png/src/libpng-1.2.40.tar.gz' -o 'libpng.tgz') exec.



"Untar and rename all 4"

#(tar -xzf 'pkgconfig.tgz') exec.

'pkgconfig.tgz' rm.

(rootDir ls: 'pkg.*') _1  mv: 'pkgconfig'.

#(tar -xzf 'libpng.tgz') exec.	

'libpng.tgz' rm.

(rootDir ls: 'libpng.*') _1 mv:  'libpng'.

#(tar -xzf 'pixman.tgz') exec.	

'pixman.tgz' rm.

(rootDir ls: 'pixman.*') _1 mv: 'pixman'.

#(tar -xzf 'cairo.tgz') exec.	

'cairo.tgz' rm.

(rootDir ls: 'cairo.*') _1 mv: 'cairo'.



"Build, install pkg-config"

(rootDir / 'pkgconfig' ) cd.

(rootDir / 'pkgconfig' / 'configure' & ('--prefix=' , (rootDir / '') forExec)) exec.

#(make) exec.

#(make install) exec.



"Environment variables for our pkg-config"

'PKG_CONFIG' setenv: rootDir / 'bin' / 'pkg-config'.

'PKG_CONFIG_PATH' setenv: rootDir / 'lib' / 'pkgconfig'.



"Setup environment variables for fat binary compilation"

'MACOSX_DEPLOYMENT_TARGET' setenv: '10.4'.

'LDFLAGS' setenv: '-arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk'.

'CFLAGS' setenv: '-Os -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk'.



"Build, install libpng"

(rootDir / 'libpng') cd.

(rootDir / 'libpng' / 'configure' & ('--prefix=' , (rootDir / '') forExec) & '--disable-dependency-tracking') exec.

#(make) exec.

#(make install) exec.



"Build, install pixman"

(rootDir / 'pixman') cd.

(rootDir / 'pixman' / 'configure' & ('--prefix=' , (rootDir / '') forExec) & '--disable-dependency-tracking' ) exec.

#(make) exec.

#(make install) exec.



"Build, install cairo"

(rootDir / 'cairo') cd.

(rootDir / 'cairo' / 'configure' & ('--prefix=' , (rootDir / '') forExec) & '--disable-xlib' & '--disable-dependency-tracking') exec.

#(make) exec.

#(make install) exec.



"Move dylibs to platform staging directories"

(rootDir / 'Frameworks') mkdir.

rootDir / 'lib' / 'libpng12.0.dylib' cp: rootDir / 'Frameworks'.

rootDir / 'lib' / 'libpixman-1.0.dylib' cp: rootDir / 'Frameworks'.

rootDir / 'lib' / 'libcairo.2.dylib' cp: rootDir / 'Frameworks'.



"Make them all relocatable"

(rootDir / 'Frameworks') cd.

#(install_name_tool -id '@executable_path/../Frameworks/libpng12.0.dylib' 'libpng12.0.dylib') exec.

#(install_name_tool -id '@executable_path/../Frameworks/libpixman-1.0.dylib' 'libpixman-1.0.dylib') exec.

#(install_name_tool -id '@executable_path/../Frameworks/libcairo.2.dylib' 'libcairo.2.dylib') exec.

(#(install_name_tool -change) & (rootDir / 'lib' / 'libpixman-1.0.dylib') , #('@executable_path/../Frameworks/libpixman-1.0.dylib' 'libcairo.2.dylib')) exec.

(#(install_name_tool -change) & (rootDir / 'lib' / 'libpng12.0.dylib') , #('@executable_path/../Frameworks/libpng12.0.dylib' 'libcairo.2.dylib')) exec.



"Clear build variables"

'MACOSX_DEPLOYMENT_TARGET' unsetenv.

'LDFLAGS' unsetenv.

'CFLAGS' unsetenv.




Feel like Herl-ing now?

Cursor consider showWhile: [Harmful]

Wed, 07 Oct 2009 03:30:15 GMT

I keep having to fix ARs that have to do with people mucking with the Cursor in non-UI processes. A couple years ago, Vassili Bykov (former VisualWorks hacker and my hero), put this excellent writeup together:

Cursors are typically used to show the current UI mode, or in other words some clue to how the application would interpret a click. For example, a paint application typically shows a magnifying glass cursor when clicking is interpreted as a zoom request, and some kind of a painting tool cursor when clicking is interpreted as painting with that tool.
The main point of this section is to point out a bad use of cursors which is unfortunately common in VisualWorks. Hopefully we'll get rid of it one day, and for now we should at least not perpetuate it.
It is the use of cursors to indicate background activity. This goes back to ST-80 use of read/write cursors to show disk activity. Back then, that was a legitimate use of cursors because reading/writing was a modal action. The system would become unresponsive, and the cursor would indicate that it's in a disk read mode.
VisualWorks does not enforce modality of input/output operations. As the result, we have situations when the system is responsive but the cursor is a twitching database disk pack because there is background database activity. That is wrong. The cursor is a local indication device showing what will happen when the user clicks where the cursor is located. It must not change to show global information, such as to indicate a background database or other activity, if that activity doesn't affect current user interaction. On the subject of hourglass cursors.
Everything said above applies to those. An hourglass cursor is an indication that the UI is in unresponsive mode. Being unresponsive is not the same as performing some activity, so applications should be careful not to show a global hourglass cursor when the activity is local and the rest of the UI is responsive.
Also, if the action that makes the UI unresponsive takes longer than a few seconds, an hourglass indication is not enough and the system should display a progress dialog with a (working!) Cancel button.

I really couldn't put it better myself. The cursor is a spatial feedback device, not a temporal one. The showWhile: pattern is one of the most deceptively attractive patterns for non-UI types, because it's quick and easy. It's a trap. Stay out of it.

It's instructive that we don't find the showWhile: pattern in other application development frameworks. It's not because they don't have blocks, one can get around that with start/end clauses. It's because this idea died a long time ago. We're playing catch up once again.

It's poor style to demand programmers eschew use of showWhile:, and yet not have a way to deal with existing call sites, or to give some feedback better than a hypnotizing whirly ball when the system is indeed unresponsive. The following describes a "back to the drawing board" design I worked up and prototyped about a year ago. The biggest problem with getting something like this integrated is that you have to change (read: remove code from) the VM, and the VM is no longer compatible with old images.

We start by delineating between two different kinds of cursors. For now, we'll call them the busy cursor and the immediate cursor.
Immediate cursors are generally associated with spatial position of the mouse. They change as the mouse moves. They include changing the cursor as the mouse enters a view. Splitter controls are a good example. Most (all?) of the Cursor show invocations are of this sort.
Busy cursors are associated with a period where the UI is unresponsive, and it becomes necessary to give the user some feedback that things aren't just hung up. They generally show up after some period of inactivity, never immediately at the beginning of a computation. Examples of this are things like read and write cursors, or GC cursors. Cursor showWhile: calls tend to be of this type.
Both cursor types, must arbitrate a shared resource, the actual cursor itself.
The busy cursor will trump the immediate cursor. But only when the system is busy and has been so for a while. By busy we mean, that it's not responding to user input. Events are not being processed. Code that might change the cursor based on it's spatial position, is not running.
We can determine system business. For a given WindowManager, it is basically when the event loop has remained running without returning to fetch more events for a longer than desired time.
Requisite in this design, is setting the Cursor of a Window, rather than a system wide global cursor. This removes a bit of overhead and impedance mismatch on the VMs part. All 3 primary display platforms have facilities for associating Cursors with Windows. I have built versions of the VM that provide this support for both X11 and OSX, basically a single primitive to "set the cursor of a window."
When a WindowManager finds itself busy (because a watchdog process notes that it hasn't return back to grab or wait for more events in a while), it will set the cursor of it's windows to be the busy cursor. When it is unbusy, it will restore them to the cursor they believe they should have (known by an instance variable added to Window). When an immediate cursor is set, it is just a matter of setting the cursor of the window. So we associate the current busy cursor with the WindowManager, and the immediate cursor with the Window. The effective cursor is the immediate cursor, unless the window manager is busy.
The prototypes done for X11 and OSX actually make cursors quite "snappy". And allows, for example, one to have two workspaces, one which has a long running computation, and the other which is available for interaction. When the cursor comes over the long running one, it shows the busy cursor (execute for this case). But is normal for the other workspace.

Selective Thought Experiment

Mon, 31 Aug 2009 14:21:19 GMT

If you think SelectionInList (and friends) is a good idea, then should there be a SelectionInText as a "value model" for text fields?

Or is the fact that TextEditor widgets don't use a special model construct an indication that SelectInList never was a good idea?

I think I have an answer I'm happy with, but am curious what readers think, and in particular, why?

How Low Can You Go?

Fri, 05 Jun 2009 04:30:22 GMT

Recently, I had a question regarding running Cairo on VisualWorks 7.4. Is it possible?

I was curious, so I tied a bunch of balloons to my computer and set out on an adventure to find out. Here's what my journey looked like:

Fire up latest version of 7.7, load NewTextFind, as something to test with.
Look at prereqs, create parcels out of CairoGraphics, Pango, Weaklings, and NewTextFind.
Install 7.4.1 on Vista.
Copy all of the dlls for Cairo/Pango in my 7.7 image directory over to the 7.4.1 one. Copy the 4 .pcl/.pst files over
Fire up 7.4.1
Load Weaklings, Cairo, and Pango parcels

Run a bare bones Cairo test:


p := Pixmap extent: 100 @ 100.

p graphicsContext newCairoContextWhile: 

		[:cr | 

		cr

			source: ColorValue purple;

			paint.

		layout := Pango.Layout text: 'Hi'.

		cr

			moveTo: 20 @ 20;

			source: ColorValue cyan.

		layout showOn: cr].

p asImage

Discover that we added the method tag based library specification in 7.6; go look at the various win* methods on LibCairo, LibGDI, and LibPango, and put those values in the class definition libraryFiles: arrays.
Rerun the test. Giggle with delight when it works.
Load NewTextFind parcel.
Hit ctrl-l in a text field to activate it; discover that we also added the message #half in 7.6 (Cairo doesn't use it, just NewTextFind)
Add ArithmeticValue:
```
half

	^self / 2
```
Chortle and Chuckle again.
Go blog about it.

The End

Based on a true story. After discovering that CairoGraphics and Pango bindings for VisualWorks worked on 7.4.1 with minimal effort, Travis was last seen wandering off to bed...