Polycephaly: What Is It and How Do I Use It?
Arden Thomas, the Product Manager for Cincom Smalltalk, recently spoke about Polycephaly at Smalltalk Solutions 2011 in Las Vegas, Nevada. On his blog, Arden posted:
This month I will be giving a presentation at Smalltalk Solutions about Polycephaly, the Cincom Smalltalk framework for leveraging multi-core processors. Come see the presentation, if you can, for more information and results of experiments with Polycephaly.
I’d like to see more customers benefitting from the advantages of Polycephaly, so I thought an example would be useful.
Suppose your company processes records for all of its customers, daily. (This could be doing any kind of processing, as this is a generic example).
First, we need a way to segment our customers, as a basis for processing groups of customers concurrently. A simple way to divide customers would be by the first letter of their last name. So we will process all customers whose last name begins with “A” as one group, “B” as another etc, … “Z”.
Let’s start with a method that gives us a collection of “A” to “Z”:aToZ“Letters A to Z”"This is a SharedQueue for managing concurrent access"| queue |queue := SharedQueue new: 26.(10 to: 35) do:[:ea | queue nextPut: (String with: (Character digitValue: ea)) ].^queue
Now let’s create a method to control and distribute the processing, and call it processAToZ:processAToZ"self processAToZ""This is a generalized example of how (one way) to use Polycephaly for using multiple cores to solve a problem.This is an example of breaking our universe into pieces to process by letters - for example it could process all customers with last names beginning with 'A', all with 'B', etc, separately."| letterQueue done results vms |letterQueue := self aToZ. "A collection (SharedQueue) of letters A .. Z "done := Semaphore new. "To make sure we wait until all processing is complete before proceeding"results := SharedQueue new. "Use SharedQueues for safe concurrent access"vms := Polycephaly.VirtualMachines new: 3. "Create three headless processing images - benchmark different sizes to see what works best for you""below we fork as many processes as we have machines (headless processing images).Each process grabs a letter, then sends it to the machine for processingResults are returned and added, and then another letter is grabbed, until there are no more.Only after there are no more letters to process, it releases the machine (closes the image) and signals its completion"vms machines do: [:machine |[ | letter |[letter := letterQueue nextAvailable.letter isNil] whileFalse: [results nextPut: (machine do: [:ch | ProcessAtoZ processLetter: ch ] with:letter)].machine release. "Shut down the vm"done signal] fork]. "Signal that the machine has completed its work"vms machines size timesRepeat: [done wait]. "Wait until each machine has reported in as completed"^ results
That’s it! OK lets implement a trivial #processLetter: so this does something we can see:processLetter: letter| letters |letters := OrderedCollection new: 100.1 to: 100 do:[:n | letters add: letter ].^letters
If you want a parcel with this code, send me an email firstname.lastname@example.org
To use this code, start an image, load Polycephaly, load this parcel, then save the image!!! (since the headless images started are copies of this main image, they will not work with Polycephaly, nor be able to do #processLetter: unless you first save the image), run my #test method (in my pcl) to make sure poly is working properly, then inspect the results of #processAToZ.
Congratulations! You now have a means of boosting performance on some of your applications!
I hope you find this interesting and valuable.
Here is Arden’s presentation from Smalltalk Solutions 2011:
[scribd id=51270583 key=key-18qc8ua9n7w44g5c1qu mode=slideshow]