How Class Promise can help

Update: Fixed the code snippet

One of the slower operations on the blog server right now is a text search - I don't cache all blog entries in memory, and I have the same (OS) process that gets the request fetch all the items in order to index and do the search. That's fairly expensive, and it makes the image unresponsive if it happens at the same priority as everything else.

The solution in VW terms is to have that expensive search take place in another process. The problem? Well, there's the browser on the far end that made the request, and can't deal with having the server fork the request and hand back nothing. Enter class Promise. From the comment:

A Promise represents a value that is being computed by a concurrently executing process. An attempt to read the value of a Promise will wait until the process has finished computing it. If the process terminates with an exception, an attempt to read the value of the Promise will raise the same exception.

In short, what that means is that you can push an expensive calculation off to another process, and have the one that is waiting for the result just wait for it. In server terms, said process goes into a non-runnable state, which lets other things happen.

At this point, understanding the VW process model is useful. Processes at the same priority level do not time slice. So, as requests come in from browsers, the server forks off processes at the same priority - and they all compete for the CPU. In general, if they complete quickly or end up waiting on I/O, it all works out ok. The latter is what I needed in this case - I needed to have the process wait until results were ready. Here's the old code:

	allResults := self 
		actuallySearchFor: searchText 
		inTitle: searchInTitle 
		inText: searchInText.
	^allResults asSortedCollection: [:a :b | a timestamp > b timestamp].

Inside that method, the actual search happens, which can be slow. So, I modified it like so:

	promise := [self 
		actuallySearchFor: searchText 
		inTitle: searchInTitle 
		inText: searchInText] promiseAt: Processor userBackgroundPriority.
	allResults := promise value.
	^allResults asSortedCollection: [:a :b | a timestamp > b timestamp].

What that does is set up a process that runs at the given priority, with a Semaphore signalling when results are ready. As a consequence, the value allResults isn't produced until that process finishes - and the process waiting for that result goes into a non-runnable state until that happens. Which means that other processes at that priority level don't get starved for CPU as that one munches through the backlog of posts searching.

Realistically, I need to add a cache here, and I'm in the process of doing that. In the meantime, this pushes an expensive task to the background, and optimizes server responses for everyone else.