Links
Twitter
Search

Thursday
Mar212013

JSON Converter for F# Map Type

Newtonsoft's Json.NET library provides an extension mechanism to add support for the serialization of additional types not natively supported by the library. This is handy for the F# developer wanting to work with JSON. Here I add support for F#'s Map type by defining a MapTypeConverter. This code will be made available, along with other converters, in the FSharp.Enterprise library on github in Json.fs.

type MapTypeConverter() =
    inherit JsonConverter()

    let doRead (reader:JsonReader) = 
        reader.Read() |> ignore

    let readKeyValuePair (serializer:JsonSerializer) (argTypes:Type []) (reader:JsonReader) =
        doRead reader // "key"
        doRead reader // value
        let key = serializer.Deserialize(reader, argTypes.[0])
        doRead reader // "value"
        doRead reader // value
        let value = serializer.Deserialize(reader, argTypes.[1])
        doRead reader // }
        FSharpValue.MakeTuple([|key;value|], FSharpType.MakeTupleType(argTypes))

    let readArray elementReaderF (reader:JsonReader) =
        doRead reader // [
        if reader.TokenType = JsonToken.StartArray then
            [|
                while reader.TokenType <> JsonToken.EndArray do
                    doRead reader // {
                    if reader.TokenType = JsonToken.StartObject then
                        let element = elementReaderF reader
                        yield element
            |]
        else
            Array.empty

    let writeKeyValuePair (serializer:JsonSerializer) (writer:JsonWriter) kv =
        let kvType = kv.GetType()
        let k = kvType.GetProperty("Key").GetValue(kv, null)
        let v = kvType.GetProperty("Value").GetValue(kv, null)
        writer.WriteStartObject()
        writer.WritePropertyName("key")
        serializer.Serialize(writer,k)
        writer.WritePropertyName("value")
        serializer.Serialize(writer,v)
        writer.WriteEndObject()

    let writeArray elementWriterF (writer:JsonWriter) (kvs:System.Collections.IEnumerable) =
        writer.WriteStartArray()
        for kv in kvs do
            elementWriterF writer kv
        writer.WriteEndArray()

    override x.CanConvert(typ:Type) =
        typ.IsGenericType && typ.GetGenericTypeDefinition() = typedefof<Map<_,_>>

    override x.WriteJson(writer:JsonWriter, value:obj, serializer:JsonSerializer) =
        if value <> null then
            let valueType = value.GetType()
            if valueType.IsGenericType then
                let baseType = valueType.GetGenericTypeDefinition()
                if baseType = typedefof<Map<_,_>> then
                    let kvs = value :?> System.Collections.IEnumerable
                    writer.WriteStartObject()
                    writer.WritePropertyName("pairs")
                    writeArray (writeKeyValuePair serializer) writer kvs
                    writer.WriteEndObject()   

    override x.ReadJson(reader:JsonReader, objectType:Type, existingValue:obj, serializer:JsonSerializer) =
        let argTypes = objectType.GetGenericArguments()
        let tupleType = FSharpType.MakeTupleType(argTypes)
        let constructedIEnumerableType = typedefof<IEnumerable<_>>.GetGenericTypeDefinition().MakeGenericType(tupleType)
        if reader.TokenType <> JsonToken.Null then
            doRead reader // "pairs"
            let kvs = readArray (readKeyValuePair serializer argTypes) reader
            doRead reader // }
            let kvs' = System.Array.CreateInstance(tupleType, kvs.Length)
            System.Array.Copy(kvs, kvs', kvs.Length)
            let methodInfo = objectType.GetMethod("Create", BindingFlags.Static ||| BindingFlags.NonPublic, null, [|constructedIEnumerableType|], null)
            methodInfo.Invoke(null, [|kvs'|])
        else
            box Map.empty

The MapTypeConverter can be added to the list of converters used by the Newtonsoft JSON library:

let settings = 
    let jss = new JsonSerializerSettings()
    jss.Converters.Add(new MapTypeConverter())
    ...
    jss

Add a couple of helper functions:

let ofObject payload = 
    JsonConvert.SerializeObject(payload,Formatting.None, settings)

let toObject<'a> js = 
    JsonConvert.DeserializeObject<'a>(js, settings)

Then F# Maps can be converted to and from JSON like so:

let m = List.zip ["a";"b";"c"] [1..3] |> Map.ofList
let j = Json.ofObject m
let o =Json.toObject<Map<string,int>> j
m = o

Resulting in the following:

val m : Map<string,int> = map [("a", 1); ("b", 2); ("c", 3)]
val j : string =
  "{"pairs":[{"key":"a","value":1},{"key":"b","value":2},{"key":"+[16 chars]
val o : Map<string,int> = map [("a", 1); ("b", 2); ("c", 3)]
val it : bool = true
Thursday
Mar072013

Why bugs don't like F#

I have delivered a number of projects in F# that all share in common a very low defect rate (zero since go live as-of the publication date of this article). I was recently asked what I put this down to. Here is my list:

  1. I use the functional programming style as much as possible (notonlyoo.org)
  2. In the few places I can't I am very very very careful.

That's it. Everything else follows from these guiding principles. This is not specific to F# but F#, and other languages like it, strongly encourage and support this approach.

Whole classes of errors disappear because of the following:

  1. No nulls
    • there are no nulls, there are no null reference exceptions
      • F# has types that express absence more safely, Options, Choices ....
  2. Immutable data
    • no complex state interaction errors
      • F# is immutable by default
  3. Very strong type system
    • the compiler works hard to stop me writing nonsense
      • F# type inference and repl gives immediate feedback on nonsense
  4. Composition of small functions

    • a small function is easy to reason about, easy to test, and it will continue to work when composed with other small functions, if there is an error it will be in a single small function. Reuse is pervasive.
      • F# supports first class functions
      • F# has a syntax friendly to function definition and composition
  5. Asynchronous programming abstractions

    • no tedious error prone chaining, no asynchronous heisenbug errors
      • F# asynchronous workflows make composing asynchronous computation easy and error free
      • F# agent abstraction makes dealing with state and concurrency easier to reason about
  6. Higher-order functions over collections

    • no indexing errors, boundary errors
      • F# has a rich library of functions over collections
  7. Units of measure
    • no calculation errors caused by unit misunderstandings
      • F# supports typing of numeric types

There is no magic to this. The functional programming style is a simple and safe paradigm in which to express solutions. F# provides excellent support for this style.

Friday
Feb222013

Does the language you choose make a difference?

Let's compare the code from two projects using Kit Eason's code analyser on fssnip. One project is written in C# and the other in F#. Both projects consume data feeds, perform some calculations and report the results to the users. The major difference between the projects is that the F# project consumes many more feeds and performs all the calculations in real-time.

Comparison of Lines of Code


lines of code piecharts

Summary

Here are the numbers:

Metric C# Project F# Project The F# Difference
Useful lines 183856 7112 26x
Comment lines 57624 182 32x
Null check lines 3036 27 112x
Blank lines 33678 1416 24x
Brace lines 62282 265 235x
Team Size 7 2 3.5x
Project Duration 5 years 4 months 15x
Defects since go live too many zero tending to infinity

Conclusion

WTF C#!!, OMG F#!!

... more seriously

It is hard to draw any conclusions comparing different projects written by different teams using only various line counts as a metrics.

I am both a C# dev and an F# dev. I can only offer subjective anecdotal evidence based on my experience of delivering projects in both languages (I am too busy delivering software to do anything else). I will leave rigour to others (btw let me know if you write a code analyser that can produce metrics for both C# and F#, I would love to run it on the projects I have and report the results).

That said, the one stat in the summary that I find most compelling is the defect rate. I have now delivered three business critical projects written in F#. I am still waiting for the first bug to come in. This is not the case with the C# projects I have delivered. I will continue to monitor and report on this. It might be that I am just on a lucky streak, but I suspect that the clarity and concision of F# code contributes greatly to its correctness.

Updates:

More details on the F# project here.

More details on the scarcity of bugs in F# code here.

Saxon Matt performs another C#/F# code comparison here.

Tuesday
Jan152013

Function Composition in R

I am learning to program in R with 40,000 other students by taking the excellent (and free!) Computing for Data Analysis course run by Coursera.

R has many useful functions for slicing and dicing data, particularly if you work in an industry built on CSV files. What is missing though are some of the function operators that I am used to using from my work with F#. This means I keep having to type brackets to terminate long expressions:

(a(b(c(d(2)))))

Fortunately functions are first class citizens in R and you can define infix operators. So the operators I am used to using in F# (>>, <<, |> and <|) are only a function definition away:

"%>>%" <- function(a,b) {
   function(x) {
     b(a(x))
   }
}

"%<<%" <- function(a,b) {
  function(x) {
    a(b(x))
  }
}

"%|>%" <- function(x,f) {
  f(x)
}

"%<|%" <- function(f,x) {
  f(x)
}

Now I can write function compositions without the hefty bracket tax:

(a %>>% b %>>% c %>>% d)(2)

and pipeline function applications too

2 %|>% function(x) { 2 * x } %|>% function(x) { 2 + x }

Now my R code can look a bit more like my F# code.

Wednesday
Oct312012

Progressive F# Tutorials 2012

It's that time of year again and I will be presenting a session at the Progressive F# Tutorials 2012 with Tomas Petricek on asynchronous programming in F#. See you there.

The introductory slide pack is here.

The code samples I will be covering are available on BitBucket here.

The FSharpEnt library is here.