Edit Rename Changes History Upload Download Back to Top

VW Benchmarks

Cincom has been telling customers that 5i is faster - this page proves it! The following benchmarks were run on a 500 Mhz pentium III with 256MB of RAM, running Windows ME. The second set below is for Linux (RedHat 7.0 on a PII 400 Mhz system).

The code for these tests is in the Public PostgreSQL Repository. Just load VRAllGoodies, and then pull down the tools menu on the launcher - the interface to the tests is there. To run the Opentalk, DST, or Socket tests, you'll need to set up a second image properly, and get the DST server configured. You can set up the socket and Opentalk information from the tool itself. Send questions to Arden Thomas. Feel more than free to port these tests to other dialects of Smalltalk; I would appreciate hearing about the efforts though.

Web Toolkit Benchmarks

  1. Towers Of Hanoi - 22
  2. Proxy Test - 100,000 repetitions of a #become: on a 100 element collection into a 1000 element collection
  3. Socket Test - 1000 sends of a string to a server, with a simple string return from the server
  4. Proxy Message Test (DNU) - 100,000 messages sent to an object that handles them via a #doesNotUnderstand: handler which then constructs and sends a message via #perform:
  5. DST Message Test - 1000 sends of an Employee Object from the client to the server
  6. Opentalk Message Test - 1000 sends of an Employee Object from the client to the server.
  7. Polymorphic send test - uses 5 unrelated classes with a common interface. Stick 500 of them in a collection, and iterate, sending the common message to each. Repeated 10,000 times.
  8. Database Query Test - repeated 100 times - get all entries in the EMP table (from the sample Oracle Database)
  9. Database Insert Test - repeated 100 times - do an insert into the EMP table of a newly created EMP object (sample Oracle Database)
  10. Stream Read Test - 100,000 repetitions - reading a stream using #upTo: and #upToAll:
  11. Stream Write Test - 100,000 repetitions - writing a long string and ashort string to a stream, followed by cr's.
  12. Allocation Speed Test - 100,000 repetitions of the creation of 10 new Array objects, followed by an incremental GC at each step.
  13. Array Write Test - 1,000,000 iterations of writes into the first 10 positions of an Array object
  14. Dictionary Write Test - 10,000 iterations of writing 10 keys and values into an IdentityDictionary
  15. Floating Math Test - Using Floating point numbers, 300,000 repetitons (10 times each time through) of this calculation: e := (e * a + b) * c + d.
  16. Fixed Point Math Test - Using Fixed Point numbers, 300,000 repetitons (10 times each time through) of this calculation: e := (e * a + b) * c + d.
  17. Integer Math Test - Using integral numbers, 300,000 repetitons (10 times each time through) of this calculation: e := (e * a + b) * c + d.
  18. OrderedCollection Iteration Test - Iterating over a 10 element collection, repeated 100,000 times
  19. OrderedCollection Write Test - Using a 10 element collection, write an object into it 10 times, repeat 100,000 times
  20. String Compare Test - Compares 'this is a test of a string compare of two long strings' to 'this is a test of a string compare of two long strings' 100,000 times.
Note - On the Database Tests, I could not execute under 2.5 or 2.5.1 - I could not get either version to communicate with Oracle under Windows ME, due to their age.

Note - Opentalk is faster than DST, even though the tests below use IDL. I3 is a slower. I did not run Opentalk tests in 5i.2 - the protocol was fairly different at that early stage.

The results:

VW Benchmark Results - all times are milliseconds
 Test Name  2.5  2.5.1  2.5.2  3.1  5i.2  5i.3  5i.4  7.0  2.5.2 to 7.0  3.1 to 7.0
 Towers Of Hanoi  310  303  290  290  255  254  260  261  7%  6%
 Proxy Speed Test  8045  8106  6762  4739  3806  5049  3749  3951  42%  17%
 Socket Speed Test  56055  56396  57583  55248  10880  8685  8857  7688  89%  86%
 DNU Proxy Test  1402  1448  1364  867  1001  750  814  642  53%  26%
 DST Test  4756  4713  4706  5190  4663  4236  4233  4096  13%  21%
 Opentalk Test  N/A  N/A  N/A  N/A  N/A  3700  4094  4072  N/A  N/A
 Polymorphic Interface Test  1341  1350  1366  1260  999  892  664  658  52%  48%
 DB Query Test  N/A  N/A  2283  2348  694  2226  2141  2023  11%  6%
 DB Insert Test  N/A  N/A  1385  1097  3300  1980  1098  1099  21%  0%
 Stream Read Test  1779  1745  1772  1674  1239  1254  1441  1408  21%  16%
 Stream Write Test  431  453  459  431  369  404  398  370  19%  14%
 Alloc Speed Test  269  268  270  270  270  272  300  275  -1%  -1%
 Array Write Speed Test  743  744  742  754  745  788  752  737  1%  2%
 Dictionary Write Speed Test  152  146  148  133  115  110  112  109  26%  18%
 Floating Math Speed Test  3080  3117  3386  3266  2108  3811  1794  1727  49%  47%
 Fixed Point Math Test  18776  20700  20079  19520  12272  13566  11747  11743  42%  40%
 Integer Math Speed Test  482  479  481  483  248  296  248  243  49%  50%
 Ordered Collection Iterate Speed Test  2184  2228  2039  2050  1264  1065  1065  1060  48%  48%
 Ordered Collection Write Speed Test  359  455  363  355  268  258  246  228  37%  36%
 String Compare Speed Test  695  800  1143  715  647  665  740  737  35%  -3%

The Results (Linux):

VW Benchmark Results - all times are milliseconds
 Test Name  3.1  5i.2  5i.3  5i.4  7.0  3.1 to 7.0
 Towers Of Hanoi  323  365  374  349  333  -3%
 Alloc Speed Test  315  287  344  329  315  0%
 Array Write Speed Test  944  944  909  906  946  0%
 Dictionary Write Speed Test  150  138  125  120  127  15%
 Floating Math Speed Test  2024  1999  2021  2024  1949  4%
 Fixed Point Math Test  14317  14094  13943  13700  13523  6%
 Integer Math Speed Test  578  299  325  324  299  48%
 Ordered Collection Iterate Speed Test  1695  1531  1262  1259  1240  27%
 Ordered Collection Write Speed Test  357  290  302  255  267  25%
 String Compare Speed Test  1346  935  1331  933  928  32%
 Proxy Speed Test  5282  5252  5328  5300  5222  1%
 Socket Speed Test  5950  5908  5439  5371  5008  16%
 DNU Test  1184  1464  929  1079  968  18%
 DST Test  5047  5161  4642  4598  4607  1%
 Opentalk Test  N/A  N/A  3003  3227  3194  N/A
 Polymorphic Interface Test  1146  1090  798  786  781  32%
 Read Stream  2135  2140  2263  1772  1601  25%
 Write Stream  462  498  471  527  513  -10%
 DB Get Test  1184  909  884  891  894  24%
 DB Insert Test  2761  1636  1474  1650  1187  57%

Web Toolkit Benchmarks

Edit Rename Changes History Upload Download Back to Top