Smalltalk Tidbits, Industry Rants

James,

You say that Store does distributed version control well, but VW does not distribute versioned code to its customers. Any sense of version control for Base VisualWorks is completely gone now. I know the arguments Cincom has posited in the past about what database? what delivery mechanism? Now, I don't care. Pick one. But do something. You just released VW 7.5 and still your chosen delivery mechanism is by parcels? Please, this is just getting embarassing.

So, in answer to your question "Should Store move forward?" I think Cincom should move forward and actually use the version control system it developed. If there are improvements to be made by studying other approaches, then fine, go for it. But first deliver versioned code to your customers.

Charles Adams

Adventa

Charlie's right regarding the distribution of VW.  Cincom needs to put its money where it's mouth is, so to speak.

 Chris  Winemiller (also from Adventa)

I really like the idea that store is backed by a database. I think with a bit of performance tuning and a couple of new features (dependencies on specific prerequisite versions, improved merge tools) it will be a top-notch version control system.

Personally and IMHO, I think that Store is a mistake and should be dropped.

First, of course, is the fact that Store maintenance and development redirects resources that Cincom could be using on its other projects.  This would be ok if it weren't for the fact that there are orgs out there that do SCCS as their core competency, and Cincom's core *isn't* SCCS.

Second is the fact that, frankly, Store just *sucks*.  My biggest beef is that it allows you to check in code that it can't correctly load.  (I don't know how I've managed to get classes with duplicate instance variables defined, but I have, and Store let me check the code in.  Load it again?  That was an adventure in surprises.) But even with that fixed, its usability is still exceptionally primitive compared to what, for example, CVS offers.

Honestly, I think that Cincom would do much better to take an existing SCCS and just write a really good client for it.  Like (forgive the comparison) Eclipse does for CVS.

And to those who would argue that you can't really do reasonable versioning when your code is in chunk format, the differences are just to great even with minor changes, I reply "Use XML format.  Sort by protocol, then method (both alphabetically).  And then do intelligent things."  So it won't work as well for non-Smalltalk text-based browsers trying to do diffs.  Right now non-Smalltalk text-based browsers don't really have the option (with Store).

Bazaar is a very good version control system for projects up to a certain size (and being optimised all the time). It has best-of-breed support for branching, renaming, moving and copying.  It used by large, distributed teams (Ubuntu).

Mark Shuttleworth has recently written some very good blog posts about version control systems and how they effect development teams.

Not a current customer, but I was a big Envy fan back in the day. All my Smalltalk is on Squeak. I just started using Monticello 1 and I'm surprised by how much I like it. Works well with distributed groups and is well suited to Smalltalk version control (fine grained, unlike CVS front ends). I'm looking forward to MC2 and I think this would be a good opportunity (as with Seaside) to provide the upgrade path for Squeakers to get the pro support Cincom offers.

Don't go file based with stuff like bazaar or git. File based sucks, they have no idea about Smalltalk, they are not aware of classes, methods and the like. All they see is lines. You
get conflicts on lines, which could pretty much be everything, instead of the where the
acutal conflict was (class or method). Squeak had the once, it was called DVS/MonticelloCVS,
it sucked, nobody is looking back to it.

Didn't JPMorgan handle over a big check to IBM for the sole reason that they don't have to
use Store? ;)

I think Cincom should stick to Store and improve their plans. See “Store Roadmap with Alan Knight (December 6, 2006)”.I would also look at the commercial available products Visual Studio Team System and AccuRev. Some of the features found in these products are highly relevant for Store.

Basically Store should support “open editions” of bundles, like Envy has open editions of config maps. AccuRev provides the same functionality, but uses different terminology. In addition, Store should have some kind of Stream support.

I second the proposal to go with Bazar (or possibly Mercurial) - it looks a lot like Monticello however it will support more artifacts than just smalltalk code.

The flaw in Envy (and Monticello follows in the same path from what I can see) is that it simply was not easy to store no-code artifacts with your projects. While a lot of my stuff lives in my image - there are documents, drawings, graphics that all need to be versioned along side my work. Having two vcs's is a pain. I also want something that other team members (not coders) can use to store their stuff in as well.

Bazar does that - its definitely somethng to keep an eye on.

Tim

I see several audiences:

1. Tiny teams (1 or 2 persons) working on a project. Often version control degenerates to 'having a backup available'.
   
2. Anarchistic teams (open source projects) with an emphasis on using many repositories with dissimilar content without breaking merge tools . Monticello has this down with a simple trick: every file that is transferred (a Parcel) has a complete list of it's lineage (a parcel property). So if you receive a Parcel with version 403 it will have a property containing 402 [repositoryname, versionstring] tuples. This allows the merge tools to find a common ancestor between two package versions and do its work correctly, even if the two version came from different repositories.
3. Enterprise teams. (10 or more developers distributed over a couple of locations. Hundreds of Pundles per project. Multiple projects (with shared evolving frameworks) active concurrently. Multiple branches per project 'active '.
   

Having noted the trick Monticello applies for 'no central repository' development I'll concentrate on 3) here, I don't know how to structure this into a coherent narrative so I'll just give a list of issues:

• Put the bar higher. In past discussions about Store deficiencies I have too often been given the reply "but Envy, CVS, X, Y, Z don't support that either' by Store developers. This expresses a desire to *not* be innovative and has driven me up the wall several times. Please - we are Smalltalkers and expect from ourselves to lead, not follow.
• Give us a server. IMO it was a bad idea to make the database comms the transport protocol for Store: Policies cannot be enforced (is simulated by the client now..), backward compatibility maintenance is hard (for instance since the format for package properties changed, they are now stored twice - in different formats) and internet comms is too slow (due to gazillions of round-trips the performance is dominated by network latency, not available bandwidth).
• Mirror repositories. When our developers are situated in multiple locations we want multiple Store repositories in order to overcome the latency issues. However keeping repositories synced is now a manual job using the replicator. What we need is multiple repositories that stay synced automagically (in both/all directions) - skip the recurring manual labor.
• Support branches, at the moment we have resort to using different repositories for different branches. This works but is very painful.
• 'Open editions' ('running maps' or whatever your team calls them), we use Configs/Lineups to get this functionality but it is only a partial solution because the tools don't know about them. It is important to have these in larger teams that share all the code, otherwise Bundles degenerate into a *very expensive* versioning-and-needles-remerging bottleneck.
• Support replication, Store sorts versions by database key number, this fails when older packages get replicated into a repo, they erroneously show up as newer versions -> use the date column for sorting, not the primary key (see package "Store-Replication support patches" in the pub repo).
• Faster replication. Replication works fine for single packages, but syncing a couple of hundred of packages takes forever.
  
• Finish the replicator. I've noted before that its UI is deficient (shows only one repository, not both). Another conspicuous absentee is 'replicate with prerequisites' which is almost impossible to emulate manually in a reliable fashion (not to mention that it is needless labor).
• Remember the Store-Envy bridge. If Cincom were ever to change to another repository we will need a bridge that actually works, the migration from Envy to Store was very painful (due to a less than useful bridge). I don't want to relive that experience...
  
• Keep the granularity. If Cincom moves us to CVS, GIT or similar beast we will still need full history at the class/method/namespace/static level, not just at the package level.
  
• Historical senders of/references to. If we need to change the behavior of some code in one of our frameworks we need to know which projects will be affected, so we need Store to tell us the historical references to a method/call/namespace/static. Currently we have this kludge where our nightly build creates an 'AllInOne' image containing all our projects, but this is clumsy. I'd rather ask Store from within my current image rather than having to start another (huge) image.
• Support versioning external files. In know we can attach files to bundles but that is kludgey and lacks tool support. Please make files first-class citizens in Store and its tools.
• Support activity based versioning. Currently Store only versions the state of a project and interspersed in the blessing comments you can see what activity the new version related to. We need the inverse: 'which version increments relate to activity X' so we can isolate them from a branch and merge them into another branch. (roughly: having versioned changesets). A good VC tool is neither state based (like Store is) nor change based but somewhere in between. See here for a discussion.
• Support pair programming. Versions ins store are associated with a single developer even if two worked on that version. I want to see both developer's names in the blessings and the various queries Store supports. If I have a question obout a bit of code I want to know who I can talk to - particularly when one of those developers is not available I need to know the name of the other.
• Integrate the Store tools. Make the menus consistent. Make the gestures consistent (when comparing two versions some tools have a menu item 'compare with...' while others require you to first select two versions first and then pull down a menu 'compare...'. Integrate with the latest tools (some Store menu items will bring up an old-fashioned CHB instead of the RB). This stuff breaks my flow 'all the time' and seems relatively easy to fix - I'm stumped that this still hasn't been fixed after all these years.
  
• Better repository cleanup tools. The current tool allows very little control over which versions can be removed (currently it is time-based where we need to be able to select particular versions for each project. The tool is now modeled as if the repository hold only one project).

Enough for now, I have several modeling features I would want in my VC system but that seems over the top here (briefly: decoule versioning from the shape of the versioned artifacts, introduce a MOP so I can introduce new artifact shapes that I want versioned (workspaces come to mind) etc, etc.

R

-

I forgot:

• Support renames and moves. When I rename a class or move it to another namespace I don't want to loose the version history. Renaming is one of the cheapest refactorings while having a huge ROI. Currently Store diminishes that value, renaming a namespace for instance will mark the contained classes and methods as having no history.

In my opinion, STORE is a great VCS and Cincom should enhance it, not replace it. I do not want to have to buy additional software to manage our code. I think STORE is a very powerful system. Granted I am not a shop with tons of developers, but we have had a number of developers and we run multiple simultaneous project all of the time here.

 My suggestion would be to review input from users and enhance the product. From my experience, you may also want to review your blessing levels with regard to the steps people take to release code. In my shop the code moves from independant development (Work in Progress), to shared development (Development), then along the path of Alpha, Beta and Production releases. At any point there may be a merge of code in order to integrate the latest from other branches and a release into any blessing level. This varies a bit from Cincom's approach (as I understand it) in which merging appears at one step in the process to release. While that may be appropriate for your shop, that doesn't work here where I may be bringing multiple projects along the path simultaneously.

Chip 

Thanks for bringing up this topic. I am very particular about version control systems, and especially particular about which version control system to use for Smalltalk. The biggest problem, I think, with STORE is that its needlessly complex for that it does, and its versioning model is also rather crude. I think that the version control system that comes with Dolphin, STS, is perfect for Smalltalk. In STS, each revision can be identified arbitarily by a symbol; it need not be assigned by a version number such as 1.2. Moreover, after checking in the code, one can rename the version number, from say 1.2 to 2.1 or RELEASE-2.1. This type of flexibility dramatically eases the ability to manage and maintain code storage for a variety of projects, and simplifies notions such as forking or merging versions together. Dave Goricek gave away Omnibase and STS to Dolphin for a song. With very little effort, I'm certain that STS can be ported to VW as well.

Comment by David Buck

I like Store, but it falls short of what I'd like to use as a professional development tool. Here are the issues I see:

First, you need some entity higher than a Bundle. You need a LineUp which contains everything you need to manage an entire project (the term Project would work as well).

Second, the prerequisites are a mess. If I create a release of my application that I ship to the field, I need to be able to rebuild the exact environment at any point in the future. With Store, when you load a bundle, it nicely asks you to pick which versions of the prerequisites to load. No, picking "Latest Development" doesn't solve the problem. You really do need the version that was included in the release. More recent ones may not work or may unexpectedly fix or hide problems. I do realize that it's possible to set prerequisite version numbers in the properties tab of Bundles, but have you really tried using it? It's awful.

Third, method overrides shouldn't disappear if the package being overridden just happens to be part of the same bundle. Enough said on that one.

Fourth, I'd really like to see the concept of an atomic update. An update is a set of method changes, class changes, etc. that are made together for a particular purpose. Updates are applied in order. You should be able to load, say, version Iteration 5 of a bundle, find out what updates were applied between Iteration 5 and Iteration 6, apply them in that order, and have the same result as loading Iteration 6. With any update, you load all of the changes for that update. Otherwise, people will load bits and pieces of changes and things won't work. You'll get bugs or inconsistencies and have problems figuring out how to fix them.

Fifth, you really need good support for non-Smalltalk files like HTML, CSS, RTP files, workspaces, etc. This includes tool support and the ability to perform DIFFs on text files. You should be able to save and restore binary files without modification. I should be able to load a Line-up and get ALL of the files I need put into the proper directories as well as getting all my code loaded into my image.