I’ve always thought that there was a strange gap in load testing tools. For very simple tests I tended to use Apache Benchmark (ab) which can generate a moderate concurrent load but lacks a way to script more complicated tests which may be required for load testing web flows. When I needed more advanced scripts my only choice for open source tools was JMeter.
What is there to say about JMeter that hasn’t already been said? It is a fairly stable tool but one that has not kept up with the needs of modern web applications. Aside from quibbles about the UI or specific features the real problem is that JMeter itself tends to choke at high load. As you increase concurrent load for a test the performance will start to degrade. The result is that it’s hard to tell if the test itself is choking or the application being tested. The solution was always to rope together some number of servers into some kind of load testing cluster capable of generating moderate (by today’s standards) load.
Luckily there is a new player in the load testing game. Gatling is an open source load testing framework built on top of an excellent non-blocking stack using Scala, Akka, Netty, and other great open source tools. It uses Akka actors to scale load which means that a single Gatling node can generate enormous concurrent load without needing huge resources behind it. A single node running on a development laptop can easily generate thousands of concurrent connections. The bottleneck is more likely to be OS socket limitations than memory or CPU resources.
Gatling scripts can be implemented as Scala code which means you can use all the normal Scala structures like traits and objects to structure you tests in a readable way. Not only that but Gatling already provides a Recorder which can act as a proxy to record interactions in a web browser and output a complete load test based on those interactions. Even things like headers and cookies are automatically handled by Gatling, making load testing session-based applications a trivial affair.
The reports that are generated from a test execution include several graphs breaking down performance across different metrics, including by global sessions and per-request. I have recently been using Gatling to tune JVM garbage collection for one of our projects and the graphs of performance over time let me see exactly where GC spikes and performance degrades. In this case simple thoughput averages would be no help since I’m tuning for acute GC spikes.
I’ve been having a lot of fun with Gatling over the past couple weeks. The project is very new but it promises new approaches for load testing in a world where thousands or millions of concurrent users is not unheard of. The Gatling team is very responsive on the Gatling user list and I suspect the community will grow quickly.