• Support for proxy actions. You can give a proxy a block that it will execute when it's instantiated.
• Some APIs are simplified. For example, read: instead of returningManyOf:. Or you no longer need to implement constructAllClasses, it'll figure out from the descriptor methods if they're there.
• GlorpActiveRecord - the ability to have automatic mappings for things that are relatively simple, and to gracefully scale up when they get more complicated. I talked about this in my user conference presentation, and the code is in the public repository.
• The ability to have expressions in mappings. This means that you could make a direct mapping that didn't go to a field, but to a function of a field. Or to a subselect. Such mappings would likely have to be read-only, but this is still very valuable.
• Support for a variety of trickier mappings in general, including a number of improvements for situations when the primary keys aren't themselves mapped, but are used as part of a relationship mapping.
• Better support for generated values of different types, generalizing from just generated primary keys or optimistic locking versions to allowing things like timestamps for last modification and more complex generated values.
• Support for reading database metdata using INFORMATION SCHEMA. This lets us avoid putting in the tables ourselves for the GlorpActiveRecord above. Currently works only for Oracle and Postgresql, though it should be fairly easy to adapt. Done in a nicely recursive way. We have descriptors that map from Glorp's objects to the INFORMATION SCHEMA tables.
• Glorp now properly handles the case where you delete and re-insert the same object in the same unit of work.
• Added the ability to have exclusive relationships. These will delete the target of the relationship when the source is deleted or the reference to the target goes away.
• Added some tools (in the GlorpAnalysis package) for performance analysis, keeping track of which sorts of queries were executed, and how long they took.
• A lot of improvements to the dictionary mapping. It still doesn't do everything desirable, but, and here is the essential point, it is better than it was.
• A DirectToManyMapping for collections of simple types.
• A variety of additional functions and things you can do in expressions. This includes the ability to use includes:, select:, intersect:, count:where:, copyFrom:to: to generate substring expressions,.
• The ability to specify an encoding and have it used for reading/writing string data.
• Efficient queries for horizontal inheritance, reading all of the subclasses in one (really messy) query. See here
• Basic support for groupBy:
• A lot of fixups related to types, validation and better error messages, generation of values, performance, and others, too small to mention.

One of the things that I've been background working on for a while is improvements to horizontal inheritance. OK, mostly that's because a couple of people have been pestering me about it until I did something just to make them stop <grin>. The old mechanism is that if you used horizontal inheritance, and wanted to do a query on the superclass, you'd end up doing a separate query for each subclass. That's not particularly efficient, but the question is, how to do it better?

What I came up with is essentially UNIONing all those separate queries together. But you can't just union them, because they're bringing back different data (if they weren't, there'd be no point not just putting them all in the same table). So what you end up doing is making a query that has columns for every column the result can contain, knows how to build the appropriate object of each subclass type using some subset of those columns, and has nulls for the ones it doesn't want. That probably works better with an example

(SELECT t1.ID, t1.NAME, NULL, NULL FROM t1 TABLE1) UNION ALL (SELECT NULL, NULL, t2.FOO, t2.BAR FROM t2 TABLE2).

Of course it's more complicated because on some databases you'll actually have to write (CAST NULL AS theTypeForThisColumn) instead of just NULL. This seems to work pretty well. There are a number of other complications as well, and I had to create a special class VariableJoin that knows it's one of several different joins, depending on the circumstances. And relationships are interesting. But so far it's working significantly better than before, and people have (mostly) stopped bugging me about it...

One of the things I've resisted doing in Glorp is support for "group by" clauses. Partly out of laziness, but partly from wanting to get the concepts clear in my head for how that works with object-level expressions. It hasn't come clear yet, but the issue has come up often enough that I decided maybe I better just do a basic version without it making sense. Maarten Mostert was asking in the Glorp list about how to do a query that retrieves customers by some criteria along with a count of their transactions, and that's exactly the sort of thing you want groupBy for.

So, because it's first thing in the morning and I haven't gotten intimidated by my to-do list yet, I did up a completely minimal version of support for it. The API is, using the Store schema as an example

q := Query read: StoreBundle where: [:each |
   (each name asUppercase like: 'G%')].
q retrieve: [:each | each primaryKey].
q retrieve: [:each | each packages countStar].
q groupBy: [:each | each primaryKey]. 
session execute: q.
 

So what this will do is read back, for all the bundles in the Store database whose name starts with the letter "g", the primary key of the bundle and the number of packages it contains. Note that there are lots of limitations on what you can retrieve in conjunction with the aggregate count. For example, you can't retrieve the actual bundle objects, because SQL doesn't want to bring back anything in conjunction with the aggregate except things that the aggregate is grouped by. Also, you have to group by something that maps to a field, so you have to specify the primaryKey field, not just the object. There is only one level of groupBy supported. But it enables a bunch of useful queries. This is in the 0.4.121 version of Glorp published in the public store repository.

A couple of nicely complimentary posts about Glorp from Todd Blanchard's blog. Here and here.

Also some very cool stuff from Ramon Leon on connection pooling, and using Magritte to provide the metadata for the class models and to generate the schema

Marco Qualizza has graciously done the legwork and set up a wiki for Glorp on the sourceforge site, using their built-in facilities (which is why it's in PHP, not Smalltalk :-). The main Sourceforge page now points to that.

In addition, Martin Laubach has started a Wiki for Glorp documentation at http://www2.laubach.at/kwiki/

Many thanks to both of them.

And as long as I'm actually blogging, I'll mention an item of Glorp news, that on December 6th, at 4:00 pm, I'll be speaking at the Cincom Smalltalk User Conference in Frankfurt Germany, on "Advanced Object-Relational Mapping with Glorp". Here's the abstract:

Glorp is an open-source library for mapping Smalltalk objects to and from relational databases. It includes a number of powerful features. This talk will give a brief overview of Glorp's architecture and then explore some of these advanced features. Topics will include the use of functions and subselects in queries and in mappings, mappings to collections of simple types and to dictionaries, query optimizations, dynamically reading and manipulating database schema and descriptor information, and mapping to schemas that are not well-matched to the object model.

I've been playing around with doing an ActiveRecord-like persistence mechanism using Glorp. What I mean by that is using convention-based mechanisms to automatically create descriptors and mappings when the classes and tables match up nicely. At the same time, I'd like to be able to gracefully move towards more complex mappings. That is, if you start having more complex mappings, you shouldn't have to abandon the automatic piece, but should be able to specify only the parts that can't be built automatically. A lot of the talk will be about that, and the technical underpinnings of that, including the GlorpMetadataDescriptorSystem for reading schema from the database. What I've done so far is published in GlorpActiveRecord and GlorpActiveRecordTest packages in the public Store

I'll also be doing a talk on Upcoming Store Developments.

Hope to see you there!

I recently added an interesting new capability for doing subselects. A user was trying to do some reasonably complicated subselects. The general syntax for subselects is to use a query object. So

  ...where: [:each | each something in: aQuery]

This is very general, but can be quite verbose to write out. Also, it does have limitations. Consider the case of a library application where we want to find all the borrowers who have at least 3 overdue books. We could write something like

  ...where: [:each | 
      q := Query count: Book where: [:book | 
         book borrower = each AND: (book dueDate < Date today)].
      (each getConstant: 3) <= q].

Aside: The getConstant: there is one limitation, because of the way expressions are built. They're expected to be based on the block argument, because that's how we build up the tree. So if we had 3=q, or q=3, neither side is actually based on the argument, so we need a way to introduce it.

But the big limitation there is that we have to ask the book who its borrower is. That does a join that we might otherwise be able to avoid, and it does it across a relationship that might not exist. Do books really need to know who their borrowers are?

There are a number of places in Glorp where we construct these kind of backwards relationships automatically, even if they don't exist in the model. For example, in filtered reads. So we can apply the same trick here. The obvious syntax to use is the Smalltalk-like select:. So,

   ...where: [:each |
      (each books select: [:book |
          book dueDate < Date today]) sqlCount >= 3].

It's shorter, it's more expressive, it's more like the normal Smalltalk, and it avoids the annoying getConstant:. I'm even considering whether I might be able to replace #sqlCount with the more natural #size, but I'm a little bit worried about pathological polymorphism with the system method size, which is used quite pervasively.

By the way, the SQL that it generates will be of the form

SELECT ...
 FROM Borrower t1
 WHERE ((SELECT  COUNT(*)
 FROM Book s1t1, Borrower s1t2, Book_Borrower_Link s1t3
 WHERE ((s1t2.primaryKey = t1.primaryKey) AND
    ((s1t3.borrower_id = s1t2.primaryKey) AND 
       dueDate < (someDateValue)) AND 
          (s1t1.primaryKey = s1t3.Book_id))))) >= 3)

Bruce Badger talks about finding Smalltalkers , and I like the idea of using OpenSkills for that.

It also made me think about the issue of skills. Perhaps it would be helpful to focus on generalized skills than on the narrower issue of technologies. Smalltalkers tend to have been around a while - a lot of them learned Smalltalk during its peak period of use in the early to mid 90's. As such, they tend to be at a more advanced point in their careers, commanding higher salaries. Employers are often looking for younger (i.e. cheaper) developers, and there are comparatively fewer of those.

But there are lots of developers out there nowadays who have used dynamic languages, and experience with any dynamic language is probably a close second to actual Smalltalk experience. The hard part to learning Smalltalk is not the language syntax or the specific class libraries, but the common patterns, ways of working and the aesthetics. Other dynamic languages have many of these things in common with Smalltalk, and many developers, particularly younger ones have experience with dynamic languages like Ruby and Python. They aren't quite the same as Smalltalk, but there are enough similarities that I would expect a Ruby developer to pick up Smalltalk much more quickly than, say, a VB or Java developer who has not been exposed to dynamic languages. LISP users, of course, are easy.

So if you're someone looking for Smalltalk developers, consider broadening the search to those with related experience.

Unfortunately, I'm afraid that the fatal flaw in this may be HR departments, with their tendency to require 5 years experience with exactly the technology in question (even if the technology in question has only existed for 18 months). The idea of looking for people with related skills might be beyond them.