Fortunately installing Visual Studio 2003, .NET 1.1 and all of the associated service packs gave me plenty of time to consider my options. By the time all the installers had completed I had decided to use node.js, implemented and had running my first node.js server using node's express web development framework. As it states in the excellent express guide and using the command line node package manager (npm), this is as simple as:

// Install express
$ npm install express
// Create server
$ express apistub && cd apistub
// Install dependence
$ npm install -d
// Run the server
$ node app.js

To add the API handlers to match the specification required a little extra effort:

// Login API
app.post('/apistub/controllers/ApiLogin/', function(req, res) {
  req.session.identifier = uuid.v4();
  console.log("LoginAPI [%s]", req.session.identifier);
  res.writeHead(404, authenticatedCookie(req.session.identifier));
  res.end();
})

// Logout API
app.post('/home/common/jsp/smlogout.jsp', function(req, res) {
  console.log("LogoutAPI [%s]", req.session.identifier);
  req.session.destroy();
  res.writeHead(302);
  res.end();
});

// Entry API
app.post('/apistub/controllers/:api/', function(req,res) {
  var xmlBaseFolder = 'api\\entry\\';
  var xsdBaseFolder = 'schema\\entry\\';
  handleApiCall(req,res,xmlBaseFolder,xsdBaseFolder);
});

// Exit API
app.post('/exit/controllers/:api/', function(req,res) {
  var xmlBaseFolder = 'api\\exit\\';
  var xsdBaseFolder = 'schema\\exit\\';
  handleApiCall(req,res,xmlBaseFolder,xsdBaseFolder);
});

The functional APIs send an XML document in the body of the request and return another XML document in the body of the response. Ideally I would like to validate the request XML against the schemas published in the specification. This is where things got tricky. As a .NET developer I am used to having libraries readily to hand to solve such problems. The node runtime is very minimal and XML schema validation is not something you get out-of-the-box.

Eventually I decided to fire-up my newly installed Visual Studio 2003 and create a .NET 1.1 console application that accepted an XML file and XML schema file as arguments and validated the XML against the schema using the usual (circa 2003) .NET libraries. I could then execute the XML validation application from within node like so:

var handleApiCall = function(req,res,xmlBaseFolder,xsdBaseFolder) {
  console.log("%s [%s]", req.params.api, req.session.identifier);
  // Validate session cookie.
  if (req.cookies.smsession != req.session.identifier) {
    res.writeHead(401, notAuthenticatedCookie(req.session.identifier));    
    res.end();
  } else { 
    // Validate request xml against request schema.
    var xmlFile = xmlBaseFolder + req.params.api + 'Request.xml';
    var schemaFile = xsdBaseFolder + req.params.api + 'Request.xsd';
    fs.writeFileSync(xmlFile, req.body.INPUT);
    var child = spawn(validateXml, [xmlFile, schemaFile]);
    child.stdout.on('data', function(data) {
      console.log('stdout: ' + data);
    });
    child.on('exit', function(code){
      if (code == 0) {
        // Request is valid so send response xml.
        var responseXmlFile = 
            xmlBaseFolder + req.params.api + 'Response.xml';
        var responseXml = fs.readFileSync(responseXmlFile);
        res.writeHead(200, authenticatedCookie(req.session.identifier));
        res.end(responseXml);
      } else {
        // Request is invalid so send error xml.
        res.writeHead(500, authenticatedCookie(req.session.identifier));
        res.end(errorInvalidXml);
      }
    });
  }
}

The response cookies were set using:

var authenticatedCookie = function(identifier) {
  return [['Set-Cookie', 
    'APIAUTHENTICATION=xxxx; APIAUTHORIZATION=xxxx; SMSESSION=' 
    + identifier + 
    '; TEST_STUB']];  
}

var notAuthenticatedCookie = function(identifier) {
  return [['Set-Cookie',
    'APIAUTHENTICATION=xxxx; APIAUTHORIZATION=xxxx; SMSESSION='
    + identifier +
    '; TEST_STUB']];
}

Job done! Integration testing can proceed using a simple, lightweight, self-contained HTTP server to mimic the behaviour of the third party API. I liked the results and can see it being used for stubbing other HTTP APIs in my landscape.

An alternative .NET only approach would have been to use the shiny new ASP.NET Web API. However, this would have introduced .NET 4 dependencies into the legacy application and, at the time of writing, the support for cookies is not good - although it is on its way.

The graph above (you will not see anything in IE8 and below) represents a domain model for a power market comprising settled (past), trading (present) and forecast (future) data.

Each node in the graph represents something of interest in the market, e.g. energy production, energy consumption, temperature, wind, energy price, capacity, flow, dynamic forecast, foreign exchange etc. modelled as a collection of time series data over a rolling time window.

Some of the nodes subscribe to market datastreams and update their state on receipt of market data messages. Once their state is updated they notify connected nodes which in turn update their state based on the newly arrived data. For example, the dynamic forecast nodes would trigger a recalculation of their market forecast.

NoOO (Not Only Object Oriented) + NoSQL

OO is supposed by some to be good for domain modelling and a possible implementation for the domain model would be as an object graph persisted in a relational database. However, inspired by Joe Armstrong's assertion that Erlang is the most objected oriented language, I opted to implement the domain model as a collection of message passing actors - this has some very useful consequences.

The domain model is concurrent. Updates to a node asynchronously and concurrently trigger updates to other nodes.

Each node in the network is the guardian of its own state and is responsible for its persistence in a document database. Gone are the complications of ORMs and managing units of work. The persisted state is eventually consistent, ideal for a domain model being continually updated in real-time where consistency is a slippery concept.

The implementation used F# mailbox processors for the actors and RavenDB for the document store (a previous post presented the concept for this, but now the implementation is complete). The visualisation uses d3 (hover the nodes to see the labels, click and drag to rearrange nodes).

I had some requests to publish the code for the AV Voting System that I presented at the start of my last talk at F#unctional Londoners. This is the code I wrote when I got bored writing slides for the talk. It is representative of the kind of code I write when I am exploring a problem for the first time using F#. Note that the model is used more for me to think about the shape of the data involved; it is not used explicitly by the code itself. This low-ceremony, low committment approach to modelling is ideal for exploratory programming. It allows me to cut-to-the-chase and explore the algorithm without delay. Combine this with the REPL (F# interative) and I have a very productive enviroment in which to play.

This code has bugs but has served its purpose by revealing a rather nice recursive approach to implementing voting systems. Hopefully more illuminating than the rather disappointing level of debate we had on the same subject leading up to the recent referendum in the UK.

 1: // The Model
 2: 
 3: type Candidate = string
 4: type Preference = int
 5: type Vote = (Candidate * Preference) list
 6: type Election = Vote list
 7: 
 8: // The Code
 9: 
10: let partitionCandidates result =
11:     let _,losingPercentage = List.minBy (fun (c,p) -> p) result
12:     List.partition (fun (c,p) -> p = losingPercentage) result
13:     |> (fun (ls, ws) -> List.unzip ls |> fst, List.unzip ws |> fst)
14: 
15: let remove losers election =
16:     let isLosing candidate = List.exists (fun loser -> candidate = loser) losers
17:     List.map (List.filter (fun (c,_) -> not <| isLosing c)) election
18: 
19: let firstPreference vote =
20:     let min cp1 cp2 = if (snd cp2) < (snd cp1) then cp2 else cp1
21:     List.fold min ("", System.Int32.MaxValue) vote
22:     |> fst
23: 
24: let add candidate candidateTotals =
25:     let incr ct = if candidate = fst ct then candidate, snd ct + 1 else ct
26:     List.map incr candidateTotals
27: 
28: let isThereAWinner result =
29:     List.exists (fun (_, percentage) -> percentage >= 50.) result
30: 
31: let firstPreferenceResult candidates election =
32:     let totalNumberOfVotes = float <| List.length election
33:     let initialTotals = List.map (fun c -> c,0) candidates
34:     let toPercentage (candidate,total) = 
35:         candidate,(float total)/totalNumberOfVotes*100.0
36:     let addVote candidateTotals vote = add (firstPreference vote) candidateTotals
37:     List.fold addVote initialTotals election
38:     |> List.map toPercentage
39: 
40: let rec electionResult candidates election =
41:     let result = firstPreferenceResult candidates election
42: 
43:     if isThereAWinner result then
44:         printfn "final %A" result
45:     else
46:         let losers, winners = partitionCandidates result
47:         printfn "-- %A\n    winners %A losers %A" result winners losers
48:         electionResult winners (remove losers election)
49: 
50: // The Election Results
51: 
52: let castVote candidates preferences =
53:     List.zip candidates preferences
54: 
55: let candidates = ["a"; "b"; "c"; "d"]
56: let election = [
57:     castVote candidates [1;2;3;4]
58:     castVote candidates [1;2;3;4]
59:     castVote candidates [1;2;3;4]
60:     castVote candidates [1;2;3;4]
61:     castVote candidates [4;3;2;1]
62:     castVote candidates [4;3;2;1]
63:     castVote candidates [3;4;1;2]
64:     castVote candidates [3;4;1;2]
65:     castVote candidates [4;1;3;2]
66:     castVote candidates [3;4;2;1]
67: ]
68: 
69: electionResult candidates election

Thanks to Tomas P for the awesome F# web snippet generator over at fssnip.net

Thank you to Philip Trelford for the invite and the organisers at Skills Matter and to everyone that attended. It was a pleasure talking with you all.