Friday, July 17, 2009

Simple Load Testing in JMeter

The following is not the be-all end-all tutorial for JMeter load testing, but hopefully it might help.

Designing a load test in JMeter:

* Download it and extract it
* In OS X, you'd run jmeter.sh in its bin directory
* First right-click on the test plan and create a new thread group
** Number of threads should be enough to produce load without time waiting on each request to get in the way. In other words, could be 5, could be 50, but too many will not be better (if you need that many, you need more boxes running JMeter).
** Ramp up period - I've read ways to calculate this, but I think that 30 sec or 60 sec (depending) is good. I did a Run - Clear All Data during the test after each ramp up (you can watch the ramp-up in the upper right during the test).
** Loop count - I did 5000. Forever is just as good. I manually stopped each test when I got the amount of data I wanted anyway.
* Right click to add an HTTP Request under the thread group. Set the request up to do what you want. If you want multiple requests, there are ways to record that in JMeter or BadBoy (in Windows only. Used to have to use that if you wanted HTTPS request recording and export to JMeter). In this case, I was testing an API, so I just had a single request per service I was testing (and I only did one at a time).
* Right click to add a Constant Throughput Timer (thanks to Richard for pointing me to this one). Other than #users/threads, this is the thing you will manipulate most often. Use the all active threads (shared) option.
* Right click to add an Aggregate Report.

Running the test:

* Should test against an environment where you can clear logs after each test.
* Change the throughput in the timer to 50, 100, 1000, 2000, 3000, 4000, 5000, 10000, 20000, 30000, 40000, ... repeat and record results.
* Go to Run - Clear All after the ramp up is complete, so the ramp up won't interfere with test data, unless you really need it.
* Stop the test after it reaches a decent number of samples. For a recent round of testing, I aimed for 20000 for higher load test, 5000 for medium load, and 500 for low load. However, the best is to do it as long as it takes to get steady looking data. Edit: actually stopping the test prematurely can cause valid socket close exceptions. You might rather have the test stop on its own (after a number of loops), but be aware that this will drop throughput by a bit while the threads are stopped. If you want to measure throughput and disregard the socket close errors, then stopping the test manually should be ok.
* Periodically JMeter won't be able to stop a thread. When this happens, save it, and kill JMeter and start over. Otherwise it might not be able to completely ramp up.

No comments: