[|] Less is More

Tetris?  Yes Tetris!

It had been awhile since I had actually picked up my own product and really immersed myself in a coding project.  Recently my Tivo had recorded a short history of the game of Tetris (quite interesting), and this piqued my interest ..... why not develop a game of Tetris? (for fun and personal use of course)  Oh, I know, its' been done;  it has even been used as an example for design in Smalltalk for years, and there is even an example that ships with the product  .... So I set a few ground rules.

·       Don't look at existing implementations (no cheating, eyes on your own monitor! ;-) )

·       Make it simple, less is more is a theme here

·       Use a recent VisualWorks 7.7 beta to develop in, and experience our new changes to the environment. No enhancements, just the stock, out of the box environment.

The result?  Wow!  The fast time to get something working reaffirmed to me the power of Smalltalk, and the continued refinement in our environments.  Some notes:

·       Within half an hour I had a grid, a colored tetris piece (stick) displayed on the grid, and the stick dropping down the grid. Then I worked on:

o   moving the active piece left and right

o   recognizing when the activePiece could drop no further, and making its cells static in the grid

o   pause and resume when pressing the escape key

o    rotation of the object when pressing the up arrow

o   Dropping the activePiece when pressing the down arrow

Within a relaxed two hours I had the basic functionality complete, and continued to add remaining pieces, experiment with the nature and direction of rotation, and play the game.

I had a total of a dozen classes; eight of them were the seven pieces (shapes) and an abstract superclass. The remaining four: TetrisApp, TetrisModel, TetrisView, GridCell. 

Some observations:

There is VERY little code.  There are half a dozen to a dozen methods in the major classes, and most of those are one-liners.  The more complex methods were ones for reporting which cells a piece occupied for its current orientation, and even most of those were a half dozen or less lines of code.

The environment, Wow!  The improvements our tools team have continued to put in to our environment pleasantly surprised me.  How could I be surprised?  I frequently load and run our beta versions of the product, and some changes are very obvious. But until you sit down and really use it, some changes may be hard to appreciate.  Lots of little refinements that make the experience more pleasant and productive abound.  The browser just conveys more useful information than ever before. Many of the changes are new for 7.7, and some are refinements from recent versions.  Either way, the development experience has made big strides in recent years, and recent months.

There are still some minor errors in the behavior of the application, which I may leave in there for instructional reasons.  There are many, many improvements and refinement that can be made.  I will publish this at some time, if you want to try it out, shoot me an email at: athomas@cincom.com

I mentioned in an earlier post that we would be taking our message on the road, and visiting some cities in a seminar series.  The first will be in Minneapolis, Minnesota, on April 29^th. 

See more information here.  Hope to see you there!

Why learn Smalltalk?

First it is a great language to work in, but you can make that determination yourself.

Second, even if you dont plan to work in Smalltalk, learning it will make you a better OO programmer in ANY language.

One of the best ways to learn Smalltalk is to take a class.

I want to reiterate some good information from James post.

The Georg Heeg company is running a public Smalltalk class in Koethen, Germany, May 4-8. Interested? Head to their website and contact them.

Leveraging Multi-Core Computers redux

A while back you may remember my series of experiments and observations with an experimental framework that allows you to take advantage of multi-core computers with your Smalltalk applications.  It really comes down to approaches to concurrency.  The goals I want us to attain in an approach are:

·         Keep it Smalltalk simple

·         Maximum gain with minimum pain

The framework offered allows easy implementation of large grained concurrent processes, which can give big advantages (where it fits of course), can be done fairly simply, and usually steers clear of the worst issues that can occur when using concurrency.

Threading is a alternative approach, which can be effective, but can also bring out the worst and most difficult issues in concurrency.

A friend sent me a links to some articles of interest, and that support this direction, one from the Intel developer network, and another from IEEE Computer.

If you would like to learn more about our approach to leveraging multi-core in Smalltalk, please contact us.

Do you know someone looking to learn Smalltalk, or who wants to lean the right way to do Object-Oriented?

Here are some links you can send them, and a three step program to a successful start!

Smalltalk - VisualWorks

VisualWorks is a modern descendent of the revolutionary work done at PARC, and a fantastic way to start learning, or building, professional applications.

First go to  Cincom Smalltalk,  click on downloads, and request a VisualWorks non-commercial CD be sent to you.  Or fill out the forms and download it.

Videos

Next, click on the "Developer's Gallery"  icon, choose the "Cincom Smalltalk Videos"  link followed by "Introduction to the Product Videos"

Books

Here are some links to free PDF versions of published books.  They can be bought on Amazon if a hardcopy is desired.

The nice thing about these books is that they use VisualWorks as the example Smalltalk, so it makes it much easier to get started. A thank you to Stephane Ducasse, for making these available.

Smalltalk by Example: the Developer's GuideAlex Sharp, McGraw Hill Text; ISBN: 0079130364

The Art and Science of Smalltalk, by Simon Lewis, Prentice-Hall 

Smalltalk An Introduction to Application Development using VisualWorks, Trevor Hopkins and Bernard Horan, Pearson Education


There are many more resources and lots of information available, from numerous blogs, weekly podcasts (avail at iTunes and other sources), a very active Smalltalk community, and other Smalltalk environments!  (ObjectStudio8, Web Velocity).

Visit Cincom Smalltalk

 

Musings;  then and now ...

First, a Happy New Year to all!

A couple of years after I had started with ParcPlace Systems (back in the early 1990's), I had entertained the notion of an investment strategy comprised of investing in the public companies who were using VisualWorks (or even any flavor of Smalltalk, but I was an SE advocating VisualWorks at the time).  It would have done really, really well.  It's just as fun to ponder why this was;  was it because the companies were astute and competent (noted by their choice in language) and therefore did well?  Or was it because of the productivity of the language itself that let them rise above their competitors?  Random luck?  There is no way of knowing for sure; probably some combination of factors, but it makes for an interesting discussion.

Fast forward to recent times ... with all the chaos in the credit and financial markets impacting the economy and companies directly, is there any correlation with companies using Smalltalk?  Although somewhat anecdotal, the companies in the banking and financial sector who are using VisualWorks and ObjectStudio (Cincom's major Smalltalk products) seemed to have fared far better than those who don't.   A major issue for many firms was there risk & exposure to certain financial instruments.  One company, fairly well known to the Smalltalk community, uses our products specifically to manage risk, and appears to be doing rather well compared to their peers.

I like this as an example of why folks should consider using our Smalltalk products.  In this case the advantage realized is probably a combination of its productivity, but especially that it lets you get your arms around more difficult problems, more effectively than about anything else.

Smalltalk Solutions

You may have seen the announcement by STIC (Smalltalk Industry Council) president, Georg Heeg, that Smalltalk Solutions will be postponed this year, largely due to company travel restrictions due to the economy.

Our Cincom Smalltalk Program Director has a response to this:

  "If you can't come to us, we'll come to you!"

We will be travelling in the USA and Europe to bring Smalltalk news, information, upcoming product changes, and more, to a city near you.

See initial information about Cincom's Smalltalk events here.

More to come, stay tuned.

   - Arden

Program Quality Rule

A post on a Quality Rule for programming was brought to my attention (thanks Michael).

It gives a metric based on the skill of the coders and the size.

No surprise here:

"The bottom line is: big programs suck. Beauty lies in minimal complexity."

Well, with my blog titled "Less is More" that song is a familiar tune  :-)

Multi Core Playground - Wrap up and Observations

Adding back in the work from the prior two posts reduced the overall load time to 35 seconds, utilizing concurrency.  Going from 114 to 35 seconds without that much work or difficulty, I view as a big win.

Some observations:

-          Measure the elapsed time of work - it helps to know where the problems are in order to address them

-          There are multiple way to do concurrency.  In one experiment, I backed off using the new framework, because a simpler approach worked better.  This was a pleasant surprise - the green thread model was actually very well suited for that problem.  I ended up combining both for a superior solution.

-          Strive for simplicity.   Remember if it gets out of hand, concurrency can get ugly and difficult very quickly.  Follow the KISS principle (Keep it Simple, Smalltalk  ;-) ). 

-          The launch of drone images was faster than I expected

-          As mentioned in the commentary, my measurements included startup and shutdown of images, a (overly?) conservative approach that may be atypical to how many use this technology.  Results would be even better had I not included that.

-          The experimental work has some pieces that I think show a lot of foresight.  Mechanisms for handling some common issues.  Like what?  If I had a dozen drone images running and lost my references and the ability to close them, closing the main image shuts everything down.  Things like that save a lot of time, avoid aggravation, and make it fun.

-          My expectations on the simplicity and robustness of the experimental framework were surpassed!  (Thank you  Michael and Martin!)  All the work and experiments were done in half a day.  One of my goals as product manager is to make this kind of concurrency easy for our customers.  An overly complex or difficult solution would be antithetical to the benefits of Smalltalk.  I'll steal one of James favorite lines (back when he had P.M. title - "If the product manager can do it, how hard can it be?" J

I know I will get the question:  "When will this code be available?"

When the initial tasks are completed and the responsible engineers are happy with the stability of it, we will look at making it available.   For what I used it for, I found it rock stable. 

Earlier versions may be made available to those who can commit to trying it and giving us feedback. 

If you are interested in getting the code when it does become available,  let me know and I can send an email when available.

-          Arden

Multi Core playground - Experiment 2b

Yesterday we improved the load time of Mutual Funds, making it load more than twice as fast (80 seconds ->  18 seconds), removing this code from the bottleneck of the overall loading.

Stepping back, the reason the sequential code took so long, was that, for each letter:

-          An http call was made
-          A couple seconds passed for the data to come back (wait)
-          The data was quickly processed.

So the problem is essentially the wait in the second step.  We could never get to the next letter, until we completed waiting for, then processing the first.

But it was pointed out to me, that the vm is not blocked on the wait for the data, and was suggested I should attempt to do the calls concurrently.

So I did - Wow!  I was able to fire up a machine, and have it fire up 13 processes (using the simple message #fork) to process my letter queue, and it resulted in bringing the time down to a nominal 15 seconds.  This solution also more effectively utilized the cpu for the image it was running in (less wait, more work).

I was very pleased - this was an area where the green threads model (that Cincom Smalltalks use)  was of great benefit at reducing the time and increasing the throughput.  Combine this with the model that allowed me to outsource that work from the current image to another (which ran headlessly in the background) and I had a solution far, far superior to the original.  Outsourcing the loading work to other images (which have no UI) offloads the work, and allows the current image to remain responsive to the user - a big benefit.

- Arden

Multi Core playground - Experiment 2

In the last installment, we had improved load time of a large amount of data from various source from 114 seconds to 80.  But we noted that the overall time was almost all due to one item taking about that long - a bottleneck that we should try to improve.  The item is the loading of mutual funds, over 24,000 of them.   What it does is make 26 http calls: one for all funds beginning with ‘A' and so on.  This was simply the easiest and obvious way to extract the data from a finance site.   So we conveniently have the problem already broken down into 26 sub-pieces.  My first attempt was to fire up three "machines" (images) and dole out a letter whenever they were ready for more.  This quickly dropped the time for this part of the load to only 31 seconds, a huge improvement!  But I had another question now; with 26 pieces, what was the optimal number of machines (drones I often call them) to "fire up" at the problem?

Hey that's what loops are for!  After a chat with an engineer (thanks Michael!),  I ended up with more versatile code which you can see at the bottom, where I can pass in "n" for n number of machines to try.

You can see in the tests below, 8 seemed to be the sweet spot. 
The upshot?  This section of the load no longer became the bottleneck.
Tomorrow:  More than one way too solve with concurrency.

-          Arden

My workspace looked something like this:

3 to: 20 do:[:n |
                ObjectMemory globalGarbageCollect.
                ms := Time millisecondsToRun: [MutualFund loadMulti: n ].
                Transcript cr; show: 'Time with: ', n printString , ' drones: ' , ms printString ].

Results?

Time with: 3 drones: 30244
Time with: 4 drones: 22373
Time with: 5 drones: 20674
Time with: 6 drones: 18271
Time with: 7 drones: 18570
Time with: 8 drones: 17619
Time with: 9 drones: 17802
Time with: 10 drones: 17691
Time with: 11 drones: 19558
Time with: 12 drones: 18905
Time with: 13 drones: 17658
Time with: 14 drones: 19696
Time with: 15 drones: 21028
Time with: 16 drones: 19704
Time with: 17 drones: 21899
Time with: 18 drones: 19400
Time with: 19 drones: 19884
Time with: 20 drones: 20698

 
                queue := self aToZQueue.
                done := Semaphore new.
                results := SharedQueue new.
                machines := VirtualMachine new: n.
                machines do: [:machine |
                [ | next |  [
                    next := queue nextAvailable.
                    next isNil] whileFalse: [results nextPut: (machine doit: 'MutualFund loadForChar: char' with: (Dictionary new at: #char put: next ; yourself))].
                machine release.
                done signal] fork].
                machines size timesRepeat: [done wait].

Multi Core Playground Experiment 1 elaboration

Here is some elaboration and clarification of the code changes I needed to do in order to change my code from sequentially loading data, to loading concurrently for my testing.

Original sequential load code:

nyse := Security loadNYSE.
amex := Security loadAMEX.
nasd := Security loadNasdaq.
funds := MutualFund load.
etfs := ETF load.

Concurrent load code:

nyseLoad := self  promise: ‘Security loadNYSE'.
amexLoad := self promise: ‘Security loadAMEX'.
nasdLoad  := self promise: ‘Security loadNasdaq'.
fundsLoad  := self promise: ‘MutualFund load'.
etfsLoad := self promise: ‘ETF load'.

nyse := nyseLoad value.
amex := amexLoad value.
nasd := nasdLoad  value.
funds := fundsLoad  value.
etfs := etfsLoad value.

Note that in the first 5 lines of the concurrent load code,  each line results in a headless (no ui) image being started and given the instruction in quotes, these lines execute immediately, so the 5 images are started and are executing concurrently.  The second set of five lines say #value to a promise object, which will wait until the data (from the "drone" image) is available.  The only other change was the #promise:  method which was in the last post.

I hope this more complete example is clearer.  - Arden

Multi Core Commentary

I had an interesting discussion about my timing tests.

The question was asked "do the times for the concurrent solutions include drone image startup & shutdown?"

My answer "Yes"

The person I had this discussion with felt this was wrong, and I understand the point.

For doing a code benchmark, we don't include the time needed to start the image. Likewise for a long, or always running application, we would not include the few seconds needed to fire up the drone images, since it would only be done once, and the resources would simply be there.  And in my tests, startup times may be a significant percent of the overall time.

So please note that my times are very conservative.

Why did I include them?  For this particular test, my perspective was that I could start and release additional resources, on the fly, and still get a very significant increase in performance - 3 fold.  It might be very likely that I would keep and use the resources, in which case I would not include the startup times and the throughput and percentage improvement would measure significantly higher.

I may do those measurements for added perspective.

Regards

                Arden

Multi Core playground - Experiment 1

So I had 5 large grained tasks to perform - I felt this was a good match for the tool I was about to experiment with. 

Recall the code I am using is some experimental work done by our engineering folks, aiming to make it easy to leverage the use of multiple cpu cores (which are becoming ubiquitous).

What I was pleased with was the relative ease of divvying up the work.  Where I had :

nyse := Security loadNYSE.

I could now use:

nyseLoad := self  promise: ‘Security loadNYSE'.
nyse := nyseLoad value.

The #promise method gave me a way to asynchronously fire up all the loading.

Results?  Good, but not great (yet ;-) )

Serial load : 114 seconds total time

Concurrent load:  80 seconds total time (includes added resource startup and shutdown)

Which is an improvement, but this fails to tell the full story.  Much of the work finished in under 20 seconds, but one, the bottleneck piece, took almost 80!  We will address that in the next post....

-          Arden

For those who are curious about the #promise method I added, here it is:

promise:  aString
| machine |
                machine := VirtualMachine new.
                ^[ | val |
                                val := machine doit:  aString.
                                machine release.
                                val] promise