In the thread that followed this post, I was asked whether I had read this OOPSLA 04 paper. I had a brief look last night, and more this morning. Here's what jumped out at me:
What is new here is the application of this technology "outside the box" of a debugger.
A debugger is used in a different mode than the editor - first you edit your code, then you switch to the debugger and manually run the code with some inputs. The debugger presents an entirely different UI and mode of interaction than the editor. The goal here is to eliminate this mode-switching by unifying the debugger and editor into a single tool with a consistent UI. This can be described as an example-enlightened editor.
In addition to sidelining the debugger, this approach supplants the need for a Read-Eval-Print-Loop: the canonical exploratory UI to an interpreter. Expressions typed into a REPL are instead now just example snippets in a source file, with their results appearing in the example view rather than inserted into the transcript. Results are automatically refreshed whenever the code changes, which avoids the hidden pitfalls of anachronistic definitions
Here's the part where the Smalltalkers realize that yes, in fact, there's nothing that fascinating here: A debugger is used in a different mode than the editor - first you edit your code, then you switch to the debugger and manually run the code with some inputs. The Smalltalk debugger is both a debugger and code browser - it's not the separate tool that the author discusses. I'm sure that this kind of tool looks very interesting to people using the mainstream languages - while to those of us using Smalltalk it elicits mostly "I've had equivalent capabilities for years now". Not identical mind you, but very much akin.
After that section, the author discusses unit tests, and how they are useful as examples. Nothing to argue with there - many developers view unit tests as something close to a documentation replacement. The unit testing assistance that the tool offers looks interesting, but - in theory, you write the test first. Thus, picking out a code snippet that qualifies as an assertion puts the cart somewhat before the horse. On the other hand, I hardly qualify as a testing purist, so this kind of support likely would be useful - if it helped generate a unit test. As outlined, it suggests that the tool is obviating the need for a separate test. That's likely a bad idea - once the initial developer leaves, this artifact goes with him. A separate unit test survives as a marker.
Ultimately, I'm not convinced that it's a good idea to encourage people to not write separate tests - which is what this paper argues for. Tests, like code, are primarily communication - a link between the original author and the future maintainer. Code maintenance always goes on longer than initial development, and any increase in the communication between the original author and the future maintainer is a good thing. The path suggested by this paper would reduce communication... not a good thing at all. I realize that the author posits a new set of IDE tools where all of this is integrated - and if it was all tied together, it would be more impressive. I still don't see it as much of an advance over the Smalltalk debugger I currently have though.