Some notes on .NET performance

Rico Mariani gave a talk based on a paper on .NET performance. It’s mostly what you’d expect from a managed runtime. Over here someone took the time to detail the performance of most IL instructions for .NET 1. The exact perf numbers don’t matter so much as the magnitude. I wonder if lambdas in C# 3.0 are still as expensive as delegates (8X more than an instance call). I do wish someone would explain when the JIT will inline code. This could go a long way towards writing more efficient libraries. The thing about performance is that it’s more than merely learning to use a language/runtime. It’s also about conforming to the style and expectations of the language/runtime implementors.

F# relies on slow .NET features

I reported earlier that the N-Queens problem on F# was too slow. I translated the benchmark in C# and it ran 2X faster than F#. It was essentially the same program… lots of recursion and list processing. So how can F# be 2X slower than C# for the same code? One simple instruction: isinst. I learned long ago that type checking on the CLR is too slow for dynamic languages. Though F# is statically typed, it still performs type checking on discriminated union types. Specifically, operations on lists must always check if the list is null. F# does this by checking if the object is a Nil class: “obj is Nil” in C#. Type checking is extremely slow and should be avoided whenever possible in hotspots. List processing is definitely a hot spot.

When I rewrote the N-Queens problem I used my own list class which used null to represent Nil. Checking “obj == null” is extremely fast. I then modified my list to use a Nil class and performance dropped down to F# speed. The solution I used with my Scheme compiler was to use null to represent the end of the list. An alternative is skip the nil check and wrap the list access in a try-catch. It only fails in the exceptional situation anyway, since properly written code will check for nil before it calls hd. This should speed up list performance dramatically.

I’m using this service…

I’m using this service called Jott. They have a very good voice recognition system. They use these computers and human transcribers. They can post directly from the phone to a blog or other services. It’s working quite well so far. listen

Powered by Jott

Architecture lessons from Google Talk

An engineer working on Google Talk gave a talk describing the lessons learned from building a very large, scalable system. (incidentally, implemented in Java)

• Measure the right thing: the difficult part for IM is presence (who’s online now), not messaging.
• Real life load tests: when they added GTalk to Gmail and Orkut, they didn’t reveal it to the user for a few weeks. Instead, they simulated IM connections to test against huge loads.
• Dynamic resharding: Prepare to add/subtract machines from your data center and rebalance data across those machines.
• Add abstractions to hide system complexity: make GTalk a “service”; hide all complexity from other systems like Gmail.
• Understand semantics of lower level library: Choose the right low level library to match the characteristics of your application.
• Protect against operational problems: Everything breaks, so prepare and recover for inevitable failures.
• Any scalable system is a distributed system: must have fault tolerance; collect metrics; trace transactions; etc.
• Software development strategies: binaries are backward compatible; features can be rolled out incrementally for experimentation; engineers work on production machines.

Amazon SimpleDB probably uses Erlang

This guy claims that Amazon’s new SimpleDB web service uses the Erlang language. Erlang is a concurrent/functional programming language designed for reliable, distributed systems. I haven’t figured out what Erlang means by “reliable” yet.

What is REST?

I’ve tried to ignore tech fashions, but the REST buzzword keeps popping up everywhere. From what little I’ve been able to understand, the REST architectural concept is a generalization of the web. All objects are restricted to a simple API: GET, PUT, POST and DELETE methods. Since you can’t get more methods, you are forced to define lots of objects (they call them “resources”). The messages passed to and from objects could be anything: text, XML, JSON, etc. They are application specific. The main benefit of REST is that it builds on top of the Web. Since the Web scales well, then your application can scale well by reusing those same techniques and technologies.

I don’t think Web Services (WS-*) contradict REST architecture, but it does require a splash of self-discipline to keep your service fairly stateless. In fact, WS-* subsumes REST. If I write a service that only has those 4 methods, isn’t that a REST architecture? That REST prefers URLs to SOAP messages is an insignificant implementation detail. The only reason I can see to prefer REST is that it doesn’t require the libraries required to process the WS-* standards. Is that all?

F# performance on N-Queens problem

I’m writing a bunch of Scheme benchmarks in F#. The N-Queens problem is determining how to place 8 queens on an 8×8 chess board so all are safe. Computing all 92 possibilities does a fair amount of recursive function calls and list processing. To do this 2000 times with F# takes about 17 seconds. For comparison, it takes 26 seconds with Scheme48 (a Scheme interpreter). Compiled implementations of Scheme can solve this problem in 2 to 3 seconds. Why is F# only 2X faster than Scheme48? I’ll report the rest of the benchmarks later.