VM Info

Occasionally it may be necessary to debug at the virtual machine level. If your virtual machine suddenly crashes finding out why either requires figuring it out or using a lower-level debugger. The reasons your VM might crash include holes in the VM, bugs in external C code and VM bugs. This page exists to tell you

• what the different VMs are
• how to run the VM on a lower-level debugger on each platform
• what commands you can use to debug at the engine level
• a bit about the debug engine
• why and how to create reproducible crashes

What are the various engines?

7.x VMs have three build variants: fast (production), debug, and assert. The fast variant is compiled using maximal optimization. The debug variant is compiled using minimal optimization and includes significant debugging assertion checking (thus, this variant is the easiest to debug but also exhibits the slowest performance). The assert variant is a compromise between fast and debug: it includes typically useful assertion checks and is compiled with moderate to full optimization.

Furthermore, non-fast engines have two levels of asserts, one that are always on and another that are so time consuming they have to be turned on explicitly. To turn on these so-called dasserts use the -o11s argument. e.g.

vwlinuxdbg -o11s image.im

The VMs report relevant debugging output through the console. On Windows, you will need to use the *console.exe virtual machines to see this output.

The assertFail routine is in stack/assertFail.c, is called whenever an assert fails, and prints the expression that failed, and the file and line number where the expression failed. A common approach to debugging is to put a breakpoint in assertFail, run until a crash causes an assertFail and then start looking around.

Debugging on Unixes

The main gotcher on Unix debuggers is the need to turn-off signal catching. By default a Unix debugger will halt execution when a signal is delivered. Given that e.g. mouse movements are communicated to the engine via signals this makes the system unusable unless one turns this off. One can do this with a command, typically the ignore command. But its much less tedious to put the commands in the debugger's initialization file. This file typically resides in a user's home directory, and can contain all sorts of useful stuff.

- with dbx (e.g. Solaris)

the init file is called .dbxrc. Here's a sample

ignore io usr1 alrm 36
dalias l list
dalias n next
dalias ni nexti
dalias s step
dalias si stepi
dalias c cont
dalias b1 file assertFail.c\;stop at 54\;assign once=1
dalias sd 'call printStack(currentStack)'
dalias tf 'call printTopFrame(currentStack)'

To debug make sure VISUALWORKS is set and then launch dbx with the engine executable. Finally run the image using the run command, along with any other arguments. e.g.

$ VISUALWORKS=/usr/local/vw5i.3
dbx /usr/local/vw5i.3/bin/solaris/vwsun5
> run /usr/local/vw5i.3/image/visual.im

You can get a C stack trace using the where command, e.g.

> where
=>[[1] updatePtrs() (optimized), at 0x5ce80 (line ~2853) in "/pps/build/hpsNext/build/sun5/fast/src/mman/mmInit.c"
  [[2] initMmAfter() (optimized), at 0x5a1d8 (line ~548) in "/pps/build/hpsNext/build/sun5/fast/src/mman/mmInit.c"
  [[3] startKernel() (optimized), at 0x59ef8 (line ~358) in "/pps/build/hpsNext/build/sun5/fast/src/mman/mmInit.c"
  [[4] startSupervisor() (optimized), at 0x31f48 (line ~216) in "/pps/build/hpsNext/build/sun5/fast/src/ext/exInit.c"
  [[5] oeMain(0x3, 0xeffffa44, 0xeffffa54, 0x0, 0x0, 0x0), at 0xb22ec
  [[6] main(0x3, 0xeffffa44, 0xeffffa54, 0xf9000, 0x1, 0x0), at 0xde8fc

You can call functions in the VM using the call command, e.g.

> call dumpAll(1)

- with gdb (e.g. linux)

the init file is called .gdbinit. Here's a sample

handle SIGUSR1 nostop noprint noignore
handle SIGUSR2 nostop noprint noignore
handle SIGALRM nostop noprint noignore
handle SIGPOLL nostop noprint noignore

To debug make sure VISUALWORKS is set and then launch dbx with the engine executable. Finally run the image using the run command, along with any other arguments. e.g.

bash$ VISUALWORKS=/usr/local/vw5i.3
gdb /usr/local/vw5i.3/bin/solaris/vwsun5
(gdb) run -=/usr/local/vw5i.3/image/visual.im some other arguments

Just like dbx you can get a C stack trace using the where command, and can call functions in the VM using the call command. See above for examples.

Debugging on Windows: VW 7.7 and later

As of VW 7.7, the Windows virtual machines are now compiled to store debug symbols in PDB files. Cincom operates a Windows symbol server as the primary method to access these files. Formerly, COFF debugging symbols were embedded inside the VM binary files. The PDB format is more useful as it includes more debugging information and by using a symbol server it is possible to ensure that the correct symbols are being used for the binaries being debugged.

The Cincom symbol server is located at http://www.cincomsmalltalk.com/downloads/symbols. If you are unfamiliar with using symbol servers, Microsoft provides a good overview at msdn.microsoft.com/en-us/library/ee416588(VS.85).aspx.

- with WinDBG

WinDBG is a free debugger from Microsoft. You only need to install the debugger, not any of the other SDK components that are offered. You give commands in the line at the bottom of the window, and they and the results are echoed to the terminal display above. You can have all that output logged to a file too.

If you have VM sources, you can set the source path from the File menu to the binsrc directory. WinDBG supports getting debug symbols from Cincom's symbol server. The path for symbols can include multiple servers and a local directory for caching: the following path worked, but there's probably a better way:

.sympath http://www.cincomsmalltalk.com/downloads/symbols;SRV*c:\pdbs*http://msdl.microsoft.com/download/symbols

The VM process has several threads, and if you attach the debugger to a running VW it seems to pick the last thread, which generally isn't interesting. To list all threads, use ~. To switch to thread 0, use ~0s. To list the call stack of the current thread, use kp.

To use the VM debugging functions, switch to a thread that is not doing anything, and call the function with .call, e.g. to dump stacks of all Smalltalk Processes in the image to stack.txt:

~4s
.call dumpAllToFile(0, 1)

If you have a VM using 100% CPU, you can find which thread is running. To list all threads in order of how much CPU time they have used, use !runaway 3. You can probably guess from the output, which lists the thread with the most time first. To be sure, choose Debug | Go (which sets VW running again), wait a couple of seconds, choose Debug | Break, reissue the !runaway 3 command, and compare the times for the threads.

- with Visual Studio 2010

If the Threads pane is not visible, choose Choose Debug | Windows | Threads to show it. Generally you want to select the Main Thread, by double-clicking it. The selected thread's stack is shown in the Call Stack pane at the bottom.

You can set the path to the symbol library with Debug | Options and Settings | Debugging | Symbols. Press the yellow 'new folder' button to add http://www.cincomsmalltalk.com/downloads/symbols

You can set the path to the VM source from the Solution Explorer: select the solution and choose Properties from its pop-up menu. Add your VW bin directory to the Debug Source Files.

You can run the VM debugging commands from the Immediate pane (when you have selected a suitable thread):

? dumpAllToFile(0, 1)

Debugging Facilities in the VM

The VM is filled with a veritable hoard of debugging functions for

• dumping the stacks of all Smalltalk processes (as of vw7.1a)
• examining the Smalltalk stack
• examining Smalltalk objects
• examining native code

miscellaneous but useful

• vmExit() - called by the VM to exit; a good place to put a break-point to understand a VM abort
• pdDebugLog(int) - returns the engine's notion of standard out, standard error or the debug file "debuglog.txt", depending on the argument being 0, 1 or 2 respectively.
• pdCloseDebugLog() - closes "debuglog.txt" if it was opened via pdDebugLog(2)

dumping the stacks of all Smalltalk processes

For those with the VM sources these functions live in stack/checks.c. These are useful for findign out the state of the system when it hangs or runs out of memory..

• dumpAllProcesses(int brief) - prints the stacks of all Process instances in the image. Use brief to get a Smalltalk-debugger-style context list. i.e. dumpAllProcesses(1). Using brief as 0 will deluge you with information.
• dumpAllProcessesToFile(int brief) - prints the stacks of all Process instances in the image to the file processes.txt.

examining the Smalltalk stack

For those with the VM sources these functions live in stack/checks.c

• dumpAll(int brief) - prints the stack; if brief is zero includes the gory details o fteh contents of each stack frame
• dumpAllToFile(1) - prints the stack to a file called stack.txt in the current directory
• briefPrintFrames() - prints the frames in the current stack segment in brief format
• printAllFramesAndContexts() - prints the entire stack
• printAllFramesAndContextsFrom(frame *sfp, oopSized *esp, bool brief) - prints the entire stack from a given frame
• printAllFramesAndContextsFrom(oop ctx, bool brief) - prints the entire stack from a given context. oop is the C type of Smalltalk objects

examining Smalltalk objects

For those with the VM sources these functions live in mman/checks.c

• dbObjResidence(oop o) - prints where an address lies relative to the VMs various spaces
• dbObjHeader(oop o) - prints the header of an object. This is similar to an object table entry, but the VM doesn't really have an object table, so "header" is preferred.
• dbObjClassName(oop o) - prints the class name of an object
• dbObjData(oop o) - prints an objects' contents

examining native code

For those with the VM sources these functions live in tran/tmain.c and elsewhere

• nMethodForPC(void *pc) - returns the native method for a given PC, if PC is within an nMethod
• dbPrintNMethod(nMethod *nmeth) - prints the header of an nmethod including its selector
• printEntireNMethod(nMethod *nmeth, memIndex *pcMap, FILE *out, bool regNames) - disassembles an nmethod

Why and How to Create Reproducible Crashes

In our book a reproducible crash is one which can be provoked without user interaction. Anything from moving the mouse to covering a window with another can send events via the engine up into Smalltalk and causes the system to "do stuff". Such "stuff" inevitably results in objects being created. So the slightest change in user interaction will change the number and order of object allocations and can make keeping track of things from run to run a real headache.

Life is much easier if one can cause the system to crash simply by starting up an image and letting it run to destruction. One can then much more easily observe it from run to run building up a complete picture. An easy way to do this is to try to develop a test case as a doit in a workspace and then add a "saveAs:..." expression to the doit, save the image and hope it crashes on start-up without one having to intervene. e.g.

| a b c |
ObjectMemory saveAs: 'bug' thenQuit: true.
b := (1 to: 100) collect: [:i| Array new: 1].
a := (1 to: 1000000) collect: [:ign| Object new].
a size.
c := (1 to: 100) collect: [:i| Array new: 1].
1 to: b size by: 2 do: [:i| b at: i put: nil].
1 to: c size by: 2 do: [:i| c at: i put: nil].
a := nil.
ObjectMemory garbageCollect.
ScheduledControllers restore.
ObjectMemory verboseGlobalCompactingGC

or

ObjectMemory saveAs: 'crash' thenQuit: true.
'load.st' asFilename fileIn

Copied from the wayback machine web.archive.org/web/*/http://wiki.cs.uiuc.edu/VisualWorks/debugging+at+the+virtual+machine+level