It's Thursday morning, and Georg Heeg is up to talk about the ObjectStudio 8 project. The Heeg company was founded in 1987, and has been involved in Smalltalk since that time.
OST and VisualWorks:
- ObjectStudio and VW are Smalltalk systems
- Both owned by Cincom
- VW is descended from the same source as Squeak (ST-80, Xerox PARC)
- ObjectStudio was developed under the name Enfin, and was originally designed as an "Enterprise OO Development Environment"
Enfin
- Ease of use
- Enterprise integration
VisualWorks
- Execution speed
- Sophisticated meta-model
Customers want both, but don't often know that they need the meta-model. So - why is Cincom doing this, given the sad history of product integration (across the industry). There was an HPS2OS project that did not succeed. However, Georg points out that Smalltalk is all about modeling - so we should be able to model one system in another.
So ObjectStudio 8: Model ObjectStudio using VisualWorks meta-modeling. It uses reflection capabilities:
- Compiler
- Debugger
- Namespaces
There were significant differences, including such things as the process model. Both OST and VW live in the same image. They share the same Smalltalk kernel and most of the same base classes. Both base OST and base VW code work (mostly) unchanged from previous releases.
The idea for this was originally Georg's - he had vocal chord surgery in 2004, and couldn't speak for 2 weeks - he started this project then, and it's become a full project inside the Cincom Smalltalk team. After that time, he could load the OST codebase - didn't all work, but it did load.
After that, he hired a new team member, Jorg, who had worked on the Windows CE VM as an intern. He's been doing much of the DLL integration for ObjectStudio 8, as well as primitive migration. That started at a rate of one primitive per day (there are 450!). That's where things went to making the OST C/C++ codebase into a VW callable DLL. He started that in February of 2005. He got a basic version completed by JUne of 2005.
At that point, compatibility testing started in earnest. If a component didn't work right, they wer either fixed, or replaced. For instance, ObjectStreams have been replaced by BOSS. This took much longer than originally anticipated :)
The original strategy was to not modify the VW VM at all. Some bugs did need to be fixed, and the Event system had to be changed somewhat - but that was it. The goal for ObjectStudio 8 on the Smalltalk side was to make VW namespaces invisible. Classes live in namespace ObjectStudio, globals live in ObjectStudio.Globals. Here's Georg in front of a (not clear in the picture) architecture diagram:
And here's a capture of the image Georg sent me afterwards:
Another set of issues came up in the way code is stored. ObjectStudio uses a different chunk format, and also stores code in CLS files (one class per file) - with additional complexity in secondary files (somewhat like the .NET partial class files, actually). The answer was a new compiler for ObjectStudio, with a new parser (both written in VW, at the Smalltalk level).
There were also some syntax differences - in OST, you can use:
{ ["code here"] ["code here] }
to define an array of blocks. The VW compiler didn't do that, so they created code transformations that changed that kind of code over. As well, in ObjectStudio, you can assign to a method parameter. Again, transformations were defined to generate temp variables. Finally, there were some semantic differences - here are a few examples:
| Code | VisualWorks | ObjectStudio |
| 1=0 ifTrue: [1] | nil | false |
| [:n | ] value: 1 | 1 | nil |
| Array new add: 1 | Error | #(1) |
| (Array new: 1) at: 1 put: 7 | 7 | #(7) |
The team defined a set of substitutions in order to execute ObjectStudio versions of messages that are like the examples above - i.e., where OST specific behavior should be preserved.
There were also mappings for the ObjectStudio chunk formats, so that OST 8 can read and write the classic OST file formats. In ObjectStudio, there is a very string integration between Smalltalk and C: there are numerous cases where the image and the VM call back and forth. OST has always been intepreted, so optimizing this callback scheme made sense. So there are five ways to call C:
- Direct Primitives
- Numbered Primitives - <primitive: 123>
- Named Primitives - <osprim: MFC openWindow>
- Module/EnfinModule
- ExternalProcedures
To handle this, a wrapper DLL was created using DLLCC. It uses __oop as a parameter and return type. It calls ObjectStudio primitives using a simulated ObjectStudio stack.
Another issue: In OST, object pointers never move - and this assumption gets used (for instance, in Windows API calls). In VW, the garbage collector moves object pointers all the time. The solution was to make OST OPTR objects into a C++ wrapper class, which handles this. Primitives either map directly to VW primitives, or got replaced by auto-generated replacement code, which goes into the wrapped DLL.
In ObjectStudio 8, Unicode capabilities are available, period. No more variant VMs and images.
Finally, the process model differences.
VisualWorks
- Processes run at different priorities
- Windows Events are handled (almost) any time
- Shared queues move events to the target processes
- Implemented in C
ObjectStudio
- The caller of all execution is the Windows event model
- All message sends in the ASendQueue are executed when the Windows Event is empty
- Implemented in C++
This is where some modifications to VW event handling had to be made. There are specific subclasses now that communicate directly with the Windows Event system, rather than a generic cross platform wrapper.
ObjectStudio also has some interesting features missing from VW: for instance, class proxies. It has the same name as a missing class, and handles DNU by loading the missing class/application. This gives ObjectStudio lazy loading. In VW, you have parcels, but they don't do lazy loading.
A few Performance Issues:
- In OST, Smalltalk message sending is slow, while calling C is "free"
- In VW, message sends are fast, while calling C is expensive
- This has driven different optimization schemes :)
Another issue: file-in some .st code in VW:
- If a load takes 1000 seconds
- 900 are refreshing the RB
- Of 100 seconds
- 90 are repeatedly relinking the system
- Of 10 seconds
- 9 are disk flush
- 1 second compiling
This has been optimized. A pleasant side effect is, it helps both VW and OST. New stuff for ObjectStudio: Store, the RB, the debugger, the inspector, related tools (code critic, etc). Note: ObjectStudio developers do not need to migrate to Store. They can continue to use the same source code control tools they use with their current ObjectStudio code.
Where are we? We have been working closely with a customer for feedback and porting data. We intend to have a first preview (beta) ready for general customers to see in the winter. Some known incompatibilities:
- Immutability
- Declare pool dictionaries
- In the modeling tool, export in ASCII (binary format is incompatible)
Georg's parting question: Integration of Smalltalk systems is possible. So, are there other systems that should be integrated? Squak, VA, VSE? Smalltalk-X, Dolphin, Gemstone? What about other languages, like Ruby, Java?
Technorati Tags:
smalltalk, esug
