The following is not original, not in the slightest, but I thought it's worth re-iterating because sometimes these ideas come back around and talking about them can help them be realized in a context suitable for someones project.

I was having a discussion over dinner about the nature of objects and closures and how they are essentially the same thing. Patrick Logan brought up a cute old joke about a student and his master. The student excitedly exclaimed to his master, "Master! Closure are more powerful than objects!".. to which the master replied, "You fool, objects are more powerful than closures, go away and study." Many moons later, the student returns, "Master, you were right, objects are more powerful than closures," to which the master replied, "You fool, closures are more powerful than objects."

They're one in the same, but let's keep the analogy going for a moment. When I write the following code: concept I'm binding a name in to a local execution space. When I write the following code: x[concept] = "Foo" I'm binding a ma,e om to a space called x. It can be said therefore that a Dictionary or Hash object is the same as an execution space (or stack frame, if you so care). One is usually immutable or completely untouchable to the programmer (in smalltalk, you can modify the stack however you like) and the other is a common staple of programming (the key+value pairs).

Objects, too, bind names to a space, which is the object itself. In that way, a Dictionary or Hash is the same as an Object. In fact, some languages such as Javascript go so far as to make them the same thing. In Smalltalk, they are not the same thing - this is a pragmatic choice based on the technology of its era where it was possible to make Smalltalk run very fast by knowing ahead of time the shape of an Object rather than compiling shape information in to objects at runtime. Self did not even attempt to unbind this (you still had to declare your slots before you could use them), but Javascript did.

So that means that a Hash is an Object. We know that a Hash is a Closure and in this case we can also say that an Object is a Closure. In fact, if we decide performance isn't necessary then a Stack Frame is a Closure + Code Position, a Closure is an Object + Executable Code and an Object is key+value pairs. Calling a method means duplicating your closures hash, adding in the new parameters, pushing that hash on to the stack and jumping to the method. Hey now, don't jump out of your chair just yet: I did say this is if you're okay with dropping performance.

What this gains us is a uniquely simple system that contains only two data structures: Hash and Closure. The stack frame is inherit in the execution machinery plus the execution stack, however if you wish to do continuation passing style, you'd add the third structure which you'd call Continuation which includes the execution position within the executable code. The beauty of the continuation passing style is that you can do away with the stack completely, because the Continuation gives you a pointer to the closure to continue from.

The only thing we'd need to decide on is how we dispatch. In Javascript, any entry in your closure is something you can call, by referring to it symbolically followed by (), eg: myMethod(). This is a direct hashed name look up, in which case you're going to look in the current execution space's hash for something to look up and then execute; failure to find the named thing results in an exception.

In languages like clojure you can specify the matching rules for a function beyond just its name, so you can delve in to the structure of the parameters to the function you're calling and decide if the function matches. This is incredibly powerful because the predicates for a function can be bound to an entirely different execution space (the one at compile time) than the runtime, which means you never end up falling down the rabbit hole.

So given how simple this system is and the number of times I've referred to javascript, just how is it that javascript falls apart here? Well, for one, the stack frame doesn't exist to the programmer, continuations don't exist to the programmer and while functions are indeed closures, you cannot influence them that way - they have a prototype object you can access but this will apply to objects created from the function, which is sort of a cludge to allow fast dispatch by having light-weight classes called prototypes.

I think perhaps the better way is for the compiler to recognize constructs where frames are created as a result of calling a function and make it a template that can be memcpy'd easily. Of course, you'd also want the compiler to recognize function call arguments that, while technically are part of the closure, may never 'escape' that short execution path - these can be put on the stack as indexed accesses instead of named lookups. You'd have to be careful never to be caught out by the programmer of course.

The runtime environment, if smart, could also identify functions that access structures regularly and compile specialized versions of the functions that have indexed access to named things in the hash and make a light-weight variant of that hash that places some values in those same indexes. The consequence of this is that the programmer gets to think in terms of objects and closures but often those objects will never exist.

Concurrent starts to get interesting at this point too, because a closure is local and its data is local, always-copy is easy to implement especially if the closure is irregularly shaped, it is easy to copy the data to ship it off to another context guilt free of any kind of collision, erlang style. In fact, if the act of calling a function is to take a copy of the closure and add the new parameters to it and a copy of the closure means a copy of its data, then you've achieved another object-oriented tenant, which is encapsulation.I'm not sure how practical this would be as one of the advantages of a closure is to modify the state of things you can see within your scope that may not be directly local to you.

Smalltalk does not share the environment from the calling method because its environment is always bound to the object the method is installed on. You never compile functions inside functions like that in Smalltalk. You do compile blocks inside methods and in this case the behave exactly like functions/closures in javascript. That same behavior of "bound to an object" is achieved in javascript by using the prototype object of a function. When you send 'new' to a function in Javascript, you create a new object which is effectively like saying you're creating a new execution space which will have absolutely no conflicts with your current execution space.

Of course, I'm twisting everything around, back to front, trying to dispel the mystique of object oriented programming. I'm deliberately voiding the OO concepts just to see what is and is not real. When we program in the way described above, we lose polymorphism, but if we have flexible dispatch mechanics then we can achieve polymorphism by reusing common names but discriminating by types - however, we really have no types, so our discrimination has to be much more evolved and it would be easy to make a function that was too liberal in what it accepted. This is definitely an advantage of a typed system - it's easy to determine just what it is you'll be calling at compile time rather than runtime.

What about other primitive types? arrays? strings? those sorts of things.. well arrays and strings are efficient hashes, where the indexing system directly corresponds to the memory layout (actually, that's a lie, if you ever want to implement a UTF8 string you'll understand what I mean). So clearly the mechanism for looking up something must be programmable, otherwise how would you ever make structures that looked (in memory) different to a basic key+value hash.

The answer is in the closure. Since a closure has executable code already, it has to have some known structure: the hash and the executable code are two slots that must be recognizable. Two more pieces of information it must have for this entire scheme to work: getter and setter. In Javascript terms these would be [] and []= and in Smalltalk terms these would be instVarAt: and instVarAt:put:, except that these two methods will be called at runtime to do lookup within the hash. So it is clear that the hash must have the ability to modify itself, our data structures now look like this:

Hash {
void *data,
Closure *getter /*void *(*getter)(void *symbol)*/,
Closure *setter /*void *(*setter)(void *symbol, void*data)*/
}

Closure {
Hash *hash,
void*(*function)(...)
}

These are the generics of course, the compiler (or runtime optimizer) might create other more interesting structures on the fly to aid in making fast running code. Whenever you have a generic you can make something concrete. But we now have the ability for the programmer to create new kinds of Hash such that we could implement a basic ascii String as:

function AsciiString {
string = {};
data = string.data;
string.getter= function(offset) { return memory[data + offset]; }
string.setter= function(offset, value) { return memory[data + offset] = value; }
return string;
}

It's key to be able to write primitive data structures using generic data structures in such a way that they can eventually become self optimizing. In this case, we can create an AsciiString if we have functions available to use for using system memory at the data pointer. The default getter and setter functions would do a key lookup, so they would be more involved, but could also be implemented in terms of the same language (however, they are redundant). The only primitive here is the getter and setter for the memory object, which would either actually be a primitive in the virtual machine or be assembly code that has been compiled (preferably).

There are still a few other loose ends here, such as how you deal with atoms themselves - eg: the Closure, or even just a simple int32. Solving all those bits and pieces to create a full language wasn't my goal here, my goal was to unify some of the core concepts that we use in many programming languages. I hope that I have done so:

Hash, Object, Closure, Stack Frame, Continuation: they are all connected.

One difference between Smalltalk and Javascript that might surprise people is how they deal with "return" from within a closure. In Javascript, the return from a closure returns to the method that evaluated the closure. In Smalltalk, the return from a closure returns from the stack frame where the closure was created. I'll illustrate this a little better example:

function topMethod() {
    return childMethod( function() { return 1 } );
}
function childMethod(closure) {
    return closure() + 1;
}
topMethod(); /* returns 2 */

In the above example, the closure is created in topMethod, but evaluated in childMethod. The return from the closure of 1 returns to childMethod where it is evaluated.

topMethod
    ^self childMethod: [^1]

childMethod: aClosure
    ^aClosure value + 1

self topMethod "returns 1"

Where-as, in the Smalltalk example, the closure returns back to topMethod, not back to childMethod. This may be a little surprising to any Javascript programmers out there, except that there is an alternative path here in Smalltalk:

topMethod
    ^self childMethod: [1]

childMethod: aClosure
    ^aClosure value + 1

self topMethod "returns 2"

As we can see in this variant, the closure does not explicitly do a return - it has an implicit return value, which is the value of the last statement in the closure. This can be counter-intuitive to the uninitiated. In fact, it's so odd that I saw the following question roll by today in the IRC channel:

How do I return from a closure back to the method that evaluated it

Actually, it wasn't asked in those words. There was some confusion added in to the mix, because if you're in Squeak you have a way to do this already:

topMethod
    ^self childMethod: [thisContext return: 1]

childMethod: aClosure
    ^aClosure value + 1

self topMethod "returns 2"

But in VisualWorks, the only return semantics we have are to do the equivalent of ^. So I wondered, what would it take to implement a version that does the same as Squeak's #return: ?

BlockContext>>blockReturn: aValue
    self unwindUpTo: sender.
    sender == nil ifTrue: [^self cannotReturn: aValue].
    sender resumeWith: aValue.
    ^sender

This gives us the same ability as in Squeak, where we can now write the cold thisContext blockReturn: 1 and it will unwind the stack back to the sender, then resume the sender with the desired return value.

This kind of semantic control is sometimes useful where a closure has become a bit of a case statement and you desire to leave the block early rather than branch it with lots of ifTrue:ifFalse: calls.

Of course, the usual pat on Smalltalk's back is required here - we've just done some very simple stack manipulation, extended a kernel class and all without breaking a sweat to give ourselves a completely new control structure.

It takes a long time to get from "a" to "b" sometimes, especially when you have VisualWorks 7.7 on the cusp of release, some of your other high priority work suddenly has to go on hold. But last week and this week i finally get to resume my WebVelocity work.

So here is the code editor becoming integrated in to WebVelocity - the major functions are hooked up now; so I can start integrating the search and syntax highlighting. I flash up a list of my "todo" for the work, all the major pieces are in play already so i'm seeing rapid progress - take a look:

I've just finished updating the look and feel of my blog website. James kindly helped me customize it. I've cut out a lot of stuff.. no more categories, no more blog rolls, no more dorky square title area that doesn't size itself to the content, no more fancy show/hide comments. Basically, I just cut lots and lots of stuff out.

I've wanted to do a refresh of my blogs look for a long time. I'm glad that I now have enough know how to poke around James's blog server to figure out what I need to change and where.

My blog is different to James's blog. He writes lots of small posts, many times a day sometimes, at least, many times a week. Where I have 150 odd posts in a year, he may have 50 odd posts in a single month. It's a very different style of blogging. I tend to write in shallow depth about a subject of interest to me - often related to my work or health.

So the design of my new blog is simple - show the latest article up front and provide all the others on the side. This gives browsers easy access to all the content, while focusing them on my latest thoughts (which as of the time of this writing, will be this post).

James's original blog server design was more along the lines of an endless stream of small updated. Much like twitter, only with more characters. I've tried to keep the new design simple and to the point. I hope that you like the new design, leave a comment praising, blasting or ignoring it.

It's one thing to be able to edit code using a fancy new text editor in WebVelocity, but we need all the existing facilities - syntax color highlighting, contextual sensitive information for implementors/senders, etc.

Jerry Kott and I have been working hard on making that a possibility for WebVelocity 1.1 with our new text editor. I can officially say now that we broke the back of the camel and this stuff finally works at a speed that is more than acceptable and with a flavor of behavior that goes beyond our expectations.

There is more work to do, we intend to have "stylists" for Smalltalk, Javascript, PEG, CSS, XML and may be even SQL if we find there's a need for it. See the following video for an "mid-days" view of the editor in action in our parsing playground, where we've been testing how the editor and the parsers interact with each other.

The link to the video: http://dl.dropbox.com/u/20291/Cincom%20Smalltalk/WebVelocity-20091110-SourceCodeEditor.mov

We're being a little ambitious, we're trying to make our source code editor for WebVelocity 1.1 happily recognize, colourize and contextualize Smalltalk, Javascript, CSS, XML, PEG and anything else we deem useful to web development.

Today we took a big step forward with this. We are now at the point where the new text editor can parse PEG syntax, identify errors and colourize them red so that we can figure out how/why our PEG grammars for other languages isn't working.

This is a note worthy milestone - it's the point where we move from simple editing code, to interacting with the code. Things can only get better from here.

I know there are a few people out there who read my blog to hear about my observations with Type 1 Diabetes. (Not type 2, please, if you have a 'Helpful link' about type 2, keep it to yourself).

Today I have another horrifying experience to impart. On Wednesday I got the continuous blood glucose monitor. It started work well yesterday. Today was time to change my insulin site as the insulin was low and I wanted to eat.

To my horror and surprise, as I removed the insertion site, blood came pouring out of my tummy. This has never happened before. Ever. I was also getting a reasonable stream of insulin - my BGL wasn't going high, as you might expect.

There was enough blood that my shorts got stained and I had to swap to my backup shorts. After getting a new insulin site in - I went to put in the monitor site, since i want the two to be synced up on the same days.

I tried to insert the sensor, which has a big needle (that you take out). It didn't go in all the way! then the freakin' needle wouldn't come out. More blood! argh, i'm sick of blood today, seriously. The second attempt went well, but again - blood. It's like I can't escape it today. So now I get to see how well the sensor calibrates when it starts off a little bloody.

From what I've read on forums and other websites, it'll be fine after it gets "wet", which can take a few minutes.

My body is still shaking, I'm angry at the world as you might expect. But at least some of my friends from Australia are visiting.. so I'll get to relax with them soon.

I've been humming and harring over Searchlight for a while now. Version 1 searched the image directly. Version 2 indexed everything. Version 2 was such a huge improvement over Version 1 that it hasn't changed for a while now.

But still, it's a search engine for your image and it was indexing the whole image every time you made the slightest little change anywhere for anything. This felt incredibly icky - yet the rule is: Make it work (version 1), Make it right (version 2), then Make it fast. Even the 'fast' bit isn't that important. It turned out that this re-indexing as you programmed didn't impact most people.

But, indexing could take as long as 10 seconds for some people and this meant that from time to time, you'd miss a search hit you were expecting. The solution to me was obvious - incremental indexing. I knew how to do it, I just never quite got around to doing it - until today.

Searchlight version 3 is new, very new - it may have bugs. If it has bugs, please email me about them. It is an incrementally indexing search engine for your image. It also has a vastly reduced memory footprint. My image was dedicating 10.6mb to the indexes, but now it uses only 2.7mb. I could probably reduce it further if I tried, but that's good enough for now (I hope - if you disagree and have worse stats, please email me).

Searchlight 3xx is not done yet - there is one other piece to the puzzle that I really want. The ThreePaneSelectorsBrowser was created by myself and Steve Aldred when we first came to the world of VisualWorks. Ever since then, it has sat doing its job mostly, until recently when it started to break apart. There have been community contributions to keep it ticking over, but it's missing all the power that comes with Searchlight.

So my goal for Searchlight 3xx is to have a 3PSB-like UI interface. Once that's there, I can officially bury 3PSB and point everyone to Searchlight. At that point, I may even be happy enough with it to suggest it could be a candidate for VisualWorks and ObjectStudio (for those of you in the know, Searchlight is already in use in WebVelocity).

The indexes created by Searchlight have some other interesting implications for the tools in general. It's much faster to lookup the index than it is to search the image - and you can notice an image search when you try to remove a method safely that has many references. Searchlight could help there. Another place it could help is with Intellisense/Autocomplete type behavior, since the indexes are now up to date as soon as you change things.

Happy image searching!