Well, it's been a great 3 days, even if I spent most of them jetlagged after traveling to London from Maryland by way of Los Angeles :) For people who like the notion of an unconference - you want to attend SPA 2007. These folks were doing a "conference for particpants" before anyone else had the idea. I'll be back next year, and I intend to arrive earlier.

If you ever have a chance to hear Dave Thomas give this talk, run, don't walk. He gave a great talk on the "received wisdom" that too many of us in the software industry follow. Chief amongst those, of course, is declarative typing. I can't summarize the whole talk - he was funny, and his slides were amusing. If Dave is speaking at a conference near you, go.

Update: Bernard has posted some of the preliminary outputs from the session. First, Laura Hill's preliminary notes are here. The audio of the session has been posted here.

I participated in a fascinating discussion forum this morning - Laura Hill and Bernard Horan (both of Sun) organized a "fishbowl" on the topic of distributed workplaces. That works like this - there were 7 chairs arranged in a circle in the center of the room, with a large ring of chairs around them. There were 6 of us in that center ring, and discussion kicked off with our opening statements on distributed working (which we had submitted previously). From there, it was an open discussion under the following ground rules:

• Only people in the bowl could talk (fish)
• People on the outside could get up and enter the bowl at any time
• There always had to be at least one open chair in the center

What that means is that people are entering and leaving the bowl regularly. We got participation from nearly the entire audience, and the conversation ranged over a lot of stuff: whether remote working is desirable, whether it works for anyone, how you can manage agile development, how you do communication.

The talking really focused on communication - both on tools (IM, NetMeeting, IRC, phone), and on the practice - how often, ad-hoc or planned, how often you need face to face meetings, that sort of thing. It was a good time, and a good discussion. In know that Laura and Bernard were taking notes and recording the session, so if any of that comes online, I'll link to it.

Here's a talk I'm seeing at a lot of conferences - something to address the widespread worry about offshoring of IT sector jobs. Personally, I'm skeptical of governmental programs to "address" the "problem" - this is nothing new. Take textiles, for instance. Between the 18th and 21st century, that industry moved from France to the UK, on to New England, then off to the US south, on to Latin America, and now to Asia. In all that time, there have always been high value textile jobs in all the "losing" areas.

I also wonder about the supposed gap between comp sci graduates and the job market; in general, the market tends to solve those problems by itself. Put another way, I'm wary of solutions that chase ill defined problems. The bottom line is, costs in the IT sector are dropping inexorably, as they have in other industries that have globalized. There's no way to get around the existance of highly trained, rapidly industrializing populations in places like India, China (etc).

Also, I wonder if one of the problems isn't the uptick in demand for credentials before hiring into a development position. When I got into the business in the late 80's, it was quite common for people to get into software development who did not have software related degrees. I certainly didn't; most of the people who worked where I first worked didn't have software degrees either. Good comment from a professor in the room on this: no one has a shared understanding of the term "software engineering", if you go across university departments, industry, etc.

Here's a good consensus - the difficulty arises in attempting to have developers over there (wherever the offshoring location is) and the managers here (US, UK, wherever). The difficulty is that there are communication difficulties that add huge overhead to such projects. Project management in general is bad in the software field; adding a large communication disruption into that already poor discipline just makes things worse. IMHO, the companies that ought to worry are software development firms (like Microsoft), who will, over time, find that new firms in India (etc) will be at least as effective as they are, but with lower costs. The company that isn't actually in IT, but has IT needs won't be nearly as impacted.

This a is a case study of a mobile phone operator - the idea being to specify behavior and run tests all by filling in a few forms. A look at the issues of defining and testing a web application - present some approaches, a 2005 case study, some of the tools used. Requirements:

• Underlying business logic needs to be explicit
• Marketing oriented requirements tend to be imprecise - concentrating on user stories, providing few usable test cases. They had to feed the requirements back to marketing for verification

For testing, you cannot rely entirely on vendor testing - you need to test the configuration. The testing in question here is functional testing. To create the test specification:

• specify system behavior, end to end
• highlight boundary conditions and end points
• model the tests while system functionality is still fluid
• tests should be specified before detailed APIs are specified
• maintain traceability between tests and requirements

Some of the tools considered: WebRunner, HttpUnit, HTMLUnit, Homebrew XML based requirements management tools. Various toolsets were considered afterwards - FIT, WATIR (Ruby), Selenium (Javascript), Agitator, Canoo Web Test. These latter tools were unknown to the team when the project was ongoing. They ended up using the homebrew system.

The Problem: A Mobile Internet Content Filter. There was no software available at the time to filter inapproriate content from mobile systems. A definition: "on-net" means on the operator's portal.

The Solution: Interpose a proxy between mobiles and "off net" (i.e., general internet) content. Apply rules to determine whether the content is appropriate. Redirect on failure of that test, allow user to change the settings.

There are a few possible outcomes:

• Access Denied (can change settings to allow)
• Illegal Page (cannot view under any circumstances - some 2000 sites are so classified in the UK)
• Allowed (normal viewing)

The "Denied" category has rules that involve the operator and their classification scheme - these become policy rules. Testing types?

• Automated tests for basic stuff
• User acceptance testing (manual)

The first is the one they needed to deal with. Test cases were based on use cases, requirements, exception conditions. Did not attempt to achieve complete decision coverage, instead using representative test cases. The tools that were developed:

• XMLSpy/Authentic - Schema per data type, stylesheets per data type
• Java - Documentation Generator, batch file to execute from files, FilterTestSuite based on HttpUnit
• Directory Load Scripts - Templates for the data files

And then a demo. The tools they created allow them to store their test cases as XML docs. The test cases are created via an XSchema driven form.

After lunch (which I skipped to exercise) and a nap (ad-hoc, in front my laptop in the common room), we're on to XML database applications with Chris Wallace. He's starting off with some XQuery examples. The backing data is in an XML database (eXist).

Heh. He says that XQuery and eXist are the most fun he's had in software since Smalltalk, which he's used since 1983. The focus with these tools is on data more than functionality. He's doing all this to explore the design space (XML Databases and Documents). In terms of information systems, the focus here is on semi-structured data (RSS, anyone :) ). The problem space includes spreadsheets, documents, ad-hoc databases, and web integrated data.

The database he's using supports XQuery, XUpdate, XSLT, XQuery extensions, and free text searching. It supports a RESTful interface (Java servlets), SOAP, and XML-RPC. One of the example applications he's working on is a Faculty Online Database - currently the data exists across Access, SQL Databases, flat text files, spreadsheets, etc. The plan is to simplify all that and still support distributed data ownership. Code:

• 3000 lines of XQuery
• 3000 lines of XSLT
• 300 of XSD (One schema)
• 10 lines of PHP (not much web work done yet)
• 25 pages online thus far

When storing data, trying to use "real world" identifiers as much as possible (names, room numbers, etc). Reduces the gap between the real world domain and the system, but it does have issues - you can easily hit duplicates (example: if I mention "Dave Thomas", which one do I mean? pragDave, or Bedarra Dave?).

In terms of data, decided against using attributes - just went with more elements. Integrity? Schema validation is too weak and too restrictive. NXD stores any well formed XML. Referential Integrity? RDBMS' are "eager, integrity failures have to be repaired outside the db. NXD - stores data on demand, but integrity failures can be persisted. repair is inside the db. XML ids only checked within a document, NXD stores all nodes with internal ids.

For information systems, veracity of the model is what's important.

Functionality delivered via:

• XQuery generating HTML
• code moving to lunction libraries and XSLT as it matures
• XQuery for request input, sessions, selection of nodes, computation of views
• XSLT to generate the interface
• CSS for presentation style

Today's invited talk is on project retrospectives, and a retrospective on the book "Project Retrospectives" by the author, Norman Kerth. The idea: after a project (any sort), it's a good idea to sit back and discuss what lessons were learned from it.

It's not enough to just gather and identify "what went wrong" and point fingers - the idea is to have the retrospective become a post project ritual that focuses on learning, not blame. For purposes of the retrospective, assume that everyone did the best job that they could at the time with the skills/tools/resources available to them at the time. Example: The campfire discussion of the buffalo hunt in "Dances With Wolves". The upshot: No one person has a complete picture of what happened - you need the perspectives of all the participants, so you can see the things you missed.

Important - you want an "atmosphere of safety" in order to get honest (and complete) feedback. The key things to learn in a retrospective:

• What worked well?
• What have we learned?
• What would we do differently next time?
• What still puzzles us?
• What needs furher work?

The way corporate culture changes is by changing the stories they tell about themselves. Constant learning implies a continual change in the way work gets done. Many organizations simply don't want to change. Some teams do want change:

• Agile teams
• Software Process Groups
• Teams at Wits end
• Consulting firms
• Highly Dynamic firms by design
• Disaster Response teams

Retrospectices can also be useful after milestones are reached, or after a merger, or after a manager/lead has been replaced. in other words, after an important event.

Good question: What do you do about people who lie, or spend their time working to undermine someone else? The idea is to focus on the events that happened, not on the people themselves. Seems to me that this might require a highly focused facilitator with many teams.

Michael Feathers, Bruce Anderson, Jason Gorman, Dave Thomas (pragDave), John Daniels, James Dobson are on the panel. It's clearly not a completely serious effort :)

Web 2.0 - what is it, and where can I buy it?

Dave Thomas - a huge success in marketing, and he's not sure where you can buy it. Web 2.0 is all about making browsers suck less (AJAX, asynch updates, etc). There's no worse place for an application than a 3270 that also gets porn - are we putting more and more lipstick on a pig? He's saying that the browser should be dead, and we should be moving to smarter clients. For a lesson, look at how the next generation plays video games - why can't we produce applications that are as useful as that? We need a more immersive, useful environment.

Michael Feathers - Yes, the browser model is somewhat broken, but it's non-proprietary. It's lousy, but it works.

Audience question - what about Flash? Michael says yes, it's definitely part of Web 2.0. Jason says it's unclear what Web 2.0 even is. It feels like something I already have, but with a cooler name. Bruce - keeps thinking about Google Earth. A large part is that IT shops like the browser because of the low maintenance aspect of installation.

Audience followup - what about mashups? Browsers enable it, it looks cool, but how maintainable/scalable is it? Dave: XML should have been shot at birth. DJ's (music) are an example of mashups. HTTP's advantages: dynamic typing and port 80.

Automatic vs. Manual Tests: What do you recommend?

Jason - manual tests are run once as a sanity check, so that you know what's being done. Then move to automation.

John - don't do manual testing - automate it. You'll find that you won't need manual tests. James: what about more ad-hoc testing that still needs to be done? What about UI tests? John: Talking about system/acceptance testing more than unit testing. Michael: as many automated tests as possible is good. Manual testing is useful for "exploratory testing", when you are trying things out. James: User Interface testing, when we are talking about subjective judgements, cannot be automated.

Audience feedback - Manual testing is error prone and cannot be as predictable as automated testing. Dave: testing is a design/analytic tool as much - or more - than a bug finding tool. In this case, the question becomes meaningless. All the testing is manual, with some of it becoming automated over time as we move through development. Michael: Testing tells us about the design, and provides a frame for "holding it up". Michael: We've moved from BDUF to BTUF (Big Test Up Front). We've gone too far.

What are the best motivational techniques for developers?

Bruce - I like to see developers taken seriously. It's about being genuinely involved in the task at hand.

Jason - Need to be able to "put my stamp" on things. Being able to walk away and say "I was there".You lose motivation when you aren't taken seriously

John - I only hire developers who care about what they are doing. There are the 9 to 5 types, and the people who attend conferences like this. If you have the latter, and you give them the space to do what they are good at, then you'll get results.

Dave - No way to motivate, but tons of ways to de-motivate. It comes from a lack of respect.

Michael - Different people are motivated differently. A lot of self selection happens.

Bruce - At IBM, we get involved in large projects, where you might fear death marches. We find it's easy to attract people to the challenge.

Audience question - What about the tension between what product management wants/needs, and what developers want/need? Need more and better communication to bridge that gap. Feedback - it can be very demotivating to end up doing what you are sure is not the right thing.

Is Management the root of all evil?

John - I'm pretty sure it's no :)

Dave - Listening to managers is the root of all evil. You have a responsibility to say no if you think you are being told to do the wrong thing.

Jason - Money is the root of all evil. Managers are just the weed growing around the mony

Bruce - A good manager shields developers from the day to day "crap" that crops up.

Michael - Bruce just described good management. Too many organizations have people who cannot understand the ramifications of bad decisions.

John - it's about having a good relationship between managers and developers. It's about mutual respect toward common goals.

Audience question - Should managers come up from technical staff? Jospeh: would prefer someone who understand what's going on. We are in one of the few professions where we have managers who do not understand technology. Michael: Do leadership and management have to be the same? Dave: Some teams need a strong tech lead, some teams need a non-technical manager. It depends on the team. Michael: You want the soft skills, regardless.

I'm in Dilbert Hell. What do I do?

Consensus: Get another job, or if not possible, find a hobby.