Why both? The stripped engine is significantly smaller and some customers think its neat to be able to fit a VM on to a floppy, so Cincom has to provide it. But in this age of cheap gigabyte drives the saving in space is insignificant. You need to use the production engine *with* debug symbols to be able to call debugging functions, or to get a meaningful stack trace. So we recommend and urge you to use the unstripped production engine, as when crashes happen you'll be in a much better position to debug.

All platforms provide a "debug" engine. This is an unoptimized engine that also includes many assertion checks. This is the vwdbg engine, e.g. vwsun5dbg or vwntconsoledbg.exe. Depending on what the engine is doing, this engine is *much* slower than the production engine. The generated Smalltalk machine code is the same, but the C code is peppered with assertion checks, and is not at all optimized for full debuggability. So unless the application is doing pure Smalltalk it should be much slower. Further, the debug engine has two levels of asserts, one that are always on and another that are so time consuming they have to be turned on explicitly. To turn on these so-called dasserts use the -o11s argument. e.g.

vwlinuxdbg -o11s image.im

As of vw7 all platforms provide an "assert" engine. This is an optimized engine that also includes many assertion checks. This is the vwast engine, e.g. vwsun5ast or vwntconsoleast.exe. Depending on what the engine is doing, this engine is only moderately slower than the production engine, e.g. around 50%. The generated Smalltalk machine code is the same, but the C code is peppered with assertion checks, so unless the application is doing pure Smalltalk it should be slower, but not nearly as slow as the debug engine because the VM code is optimized. This doesn't allow full debuggab9ility, but does make it practicable to use in place of the proiduction engine for testing the engine using asserts. Like the debug engine the assert engine has two levels of asserts, one that are always on and another that are so time consuming they have to be turned on explicitly. To turn on these so-called dasserts use the -o11s argument. e.g.

vwlinuxdbg -o11s image.im

Oh yes, those other three Windows engines, vwntconsole.exe et al, are engines that stay connected to their command prompt, so they can read and write to standard input. This distinction isn't necessary on Unix because there are decent shells which allow you to start things in the background.

Debugging on Unixes

The main gotcher on Unix debuggers is the need to turn-off signal catching. By default a Unix debugger will halt execution when a signal is delivered. Given that e.g. mouse movements are communicated to the engine via signals this makes the system unusable unless one turns this off. One can do this with a command, typically the ignore command. But its much less tedious to put the commands in the debugger's initialization file. This file typically resides in a user's home directory, and can contain all sorts of useful stuff, like setting the debugger's prompt to ugh>, which makes it seem so much cleverer.

- with dbx (e.g. Solaris)

the init file is called .dbxrc. Here's a sample

setenv PS1 "ugh> "
ignore io usr1 alrm 36
dalias l list
dalias n next
dalias ni nexti
dalias s step
dalias si stepi
dalias c cont
dalias b1 file assertFail.c\;stop at 54\;assign once=1
dalias sd 'call printStack(currentStack)'
dalias tf 'call printTopFrame(currentStack)'

To debug make sure VISUALWORKS is set and then launch dbx with the engine executable. Finally run the image using the run command, along with any other arguments. e.g.

$ VISUALWORKS=/usr/local/vw5i.3
dbx /usr/local/vw5i.3/bin/solaris/vwsun5
ugh> run /usr/local/vw5i.3/image/visual.im

You can get a C stack trace using the where command, e.g.

ugh> where
=>[[1] updatePtrs() (optimized), at 0x5ce80 (line ~2853) in "/pps/build/hpsNext/build/sun5/fast/src/mman/mmInit.c"
  [[2] initMmAfter() (optimized), at 0x5a1d8 (line ~548) in "/pps/build/hpsNext/build/sun5/fast/src/mman/mmInit.c"
  [[3] startKernel() (optimized), at 0x59ef8 (line ~358) in "/pps/build/hpsNext/build/sun5/fast/src/mman/mmInit.c"
  [[4] startSupervisor() (optimized), at 0x31f48 (line ~216) in "/pps/build/hpsNext/build/sun5/fast/src/ext/exInit.c"
  [[5] oeMain(0x3, 0xeffffa44, 0xeffffa54, 0x0, 0x0, 0x0), at 0xb22ec
  [[6] main(0x3, 0xeffffa44, 0xeffffa54, 0xf9000, 0x1, 0x0), at 0xde8fc

You can call functions in the VM using the call command, e.g.

ugh> call dumpAll(1)

- with gdb (e.g. linux)

the init file is called .gdbinit. Here's a sample

handle SIGUSR1 nostop noprint noignore
handle SIGUSR2 nostop noprint noignore
handle SIGALRM nostop noprint noignore
handle SIGPOLL nostop noprint noignore

To debug make sure VISUALWORKS is set and then launch dbx with the engine executable. Finally run the image using the run command, along with any other arguments. e.g.

bash$ VISUALWORKS=/usr/local/vw5i.3
gdb /usr/local/vw5i.3/bin/solaris/vwsun5
(gdb) run -=/usr/local/vw5i.3/image/visual.im some other arguments

Just like dbx you can get a C stack trace using the where command, and can call functions in the VM using the call command. See above for examples.

Debugging on Windows: VW 7.7 and later

As of VW 7.7, the Windows virtual machines are now compiled to store debug symbols in PDB files. Cincom operates a Windows symbol server as the primary method to access these files. Formerly, COFF debugging symbols were embedded inside the VM binary files. The PDB format is more useful as it includes more debugging information and by using a symbol server it is possible to ensure that the correct symbols are being used for the binaries being debugged.

The Cincom symbol server is located at http://www.cincomsmalltalk.com/downloads/symbols. If you are unfamiliar with using symbol servers, Microsoft provides a good overview at msdn.microsoft.com/en-us/library/ee416588(VS.85).aspx.

- with WinDBG

WinDBG is a free debugger from Microsoft. You only need to install the debugger, not any of the other SDK components that are offered. You give commands in the line at the bottom of the window, and they and the results are echoed to the terminal display above. You can have all that output logged to a file too.

If you have VM sources, you can set the source path from the File menu to the binsrc directory. WinDBG supports getting debug symbols from Cincom's symbol server. The path for symbols can include multiple servers and a local directory for caching: the following path worked, but there's probably a better way:

.sympath http://www.cincomsmalltalk.com/downloads/symbols;SRV*c:\pdbs*http://msdl.microsoft.com/download/symbols

The VM process has several threads, and if you attach the debugger to a running VW it seems to pick the last thread, which generally isn't interesting. To list all threads, use ~. To switch to thread 0, use ~0s. To list the call stack of the current thread, use kp.

To use the VM debugging functions, switch to a thread that is not doing anything, and call the function with .call, e.g. to dump stacks of all Smalltalk Processes in the image to stack.txt:

~4s
.call dumpAllToFile(0, 1)

If you have a VM using 100% CPU, you can find which thread is running. To list all threads in order of how much CPU time they have used, use !runaway 3. You can probably guess from the output, which lists the thread with the most time first. To be sure, choose Debug | Go (which sets VW running again), wait a couple of seconds, choose Debug | Break, reissue the !runaway 3 command, and compare the times for the threads.

Debugging Facilities in the VM

The VM is filled with a veritable hoard of debugging functions for

• dumping the stacks of all Smalltalk processes (as of vw7.1a)
• examining the Smalltalk stack
• examining Smalltalk objects
• examining native code

miscellaneous but useful

• vmExit() - called by the VM to exit; a good place to put a break-point to understand a VM abort
• pdDebugLog(int) - returns the engine's notion of standard out, standard error or the debug file "debuglog.txt", depending on the argument being 0, 1 or 2 respectively.
• pdCloseDebugLog() - closes "debuglog.txt" if it was opened via pdDebugLog(2)

dumping the stacks of all Smalltalk processes

For those with the VM sources these functions live in stack/checks.c. These are useful for findign out the state of the system when it hangs or runs out of memory..

• dumpAllProcesses(int brief) - prints the stacks of all Process instances in the image. Use brief to get a Smalltalk-debugger-style context list. i.e. dumpAllprocesses(1). Using brief as 0 will deluge you with information.
• dumpAllProcessesToFile(int brief) - prints the stacks of all Process instances in the image to the file processes.txt.

examining the Smalltalk stack

For those with the VM sources these functions live in stack/checks.c

• dumpAll(int brief) - prints the stack; if brief is zero includes the gory details o fteh contents of each stack frame
• dumpAllToFile(1) - prints the stack to a file called stack.txt in the current directory
• briefPrintFrames() - prints the frames in the current stack segment in brief format
• printAllFramesAndContexts() - prints the entire stack
• printAllFramesAndContextsFrom(frame *sfp, oopSized *esp, bool brief) - prints the entire stack from a given frame
• printAllFramesAndContextsFrom(oop ctx, bool brief) - prints the entire stack from a given context. oop is the C type of Smalltalk objects

examining Smalltalk objects

For those with the VM sources these functions live in mman/checks.c

• dbObjResidence(oop o) - prints where an address lies relative to the VMs various spaces
• dbObjHeader(oop o) - prints the header of an object. This is similar to an object table entry, but the VM doesn't really have an object table, so "header" is preferred.
• dbObjClassName(oop o) - prints the class name of an object
• dbObjData(oop o) - prints an objects' contents

examining native code

For those with the VM sources these functions live in tran/tmain.c and elsewhere

• nMethodForPC(void *pc) - returns the native method for a given PC, if PC is within an nMethod
• dbPrintNMethod(nMethod *nmeth) - prints the header of an nmethod including its selector
• printEntireNMethod(nMethod *nmeth, memIndex *pcMap, FILE *out, bool regNames) - disassembles an nmethod

The Debug Engine

...is primarily for VM debugging as opposed to application debugging. It is slow but its assertion checks tend to catch errors early. It also includes more debugging functions which are compiled in only in an engine with ASSERTs. The assertFail routine is in stack/assertFail.c, is called whenever an assert fails, and prints the expression that failed, and the file and line number where the expression failed. A common approach to debugging is to put a breakpoint in assertFail, run until a crash causes an assertFail and then start looking around.

Why and How to Create Reproducible Crashes

In our book a reproducible crash is one which can be provoked without user interaction. Anything from moving the mouse to covering a window with another can send events via the engine up into Smalltalk and causes the system to "do stuff". Such "stuff" inevitably results in objects being created. So the slightest change in user interaction will change the number and order of object allocations and can make keeping track of things from run to run a real headache.

Life is much easier if one can cause the system to crash simply by starting up an image and letting it run to destruction. One can then much more easily observe it from run to run building up a complete picture. An easy way to do this is to try to develop a test case as a doit in a workspace and then add a "saveAs:..." expression to the doit, save the image and hope it crashes on start-up without one having to intervene. e.g.

| a b c |
ObjectMemory saveAs: 'bug' thenQuit: true.
b := (1 to: 100) collect: [:i| Array new: 1].
a := (1 to: 1000000) collect: [:ign| Object new].
a size.
c := (1 to: 100) collect: [:i| Array new: 1].
1 to: b size by: 2 do: [:i| b at: i put: nil].
1 to: c size by: 2 do: [:i| c at: i put: nil].
a := nil.
ObjectMemory garbageCollect.
ScheduledControllers restore.
ObjectMemory verboseGlobalCompactingGC

or

ObjectMemory saveAs: 'crash' thenQuit: true.
'load.st' asFilename fileIn

Copied from the wayback machine web.archive.org/web/*/http://wiki.cs.uiuc.edu/VisualWorks/debugging+at+the+virtual+machine+level