rephrase… “do not use jMeter to test very fast response time applications”

about httpclient: But why isn’t the problem deterministic? Shouldn’t it never recover once the problem starts to happen? The magic is Java garbage collection. I reproduced the effect by forcing garbage collection. It will clean CLOSE_WAIT connections. But, to be accurate, JVM garbage collection does not handle socket closing by itself. It only frees memory. It is Socket object who closes sockets in finanize method as discussed here.

– CLOSE_WAIT socket state, Socket object, finalize & garbage collection

you end up with having garbage collection slowdowns on client side (one more link), and that will effect your metrics collection as well… no matter how high you tune your ulimit(this limits how many sockets your process can concurrently open)… you simply can’t test a such an application with a jMeter… it is “too slow” …grinder is a bad choice as well (everything Java based!)

if you want to do so, good will be “ab” aka Apache Benchmark, with the limitations to the scenario you can describe… it won’t allow you do to complex load and performance testing scenarios… it is basically hitting ONLY one url… but it can help when you need to stress a specific call, or your apache settings

other good choice might be tSung, written in Erlang… the system creates queues of tasks, and you can simulate a pretty heavy amount of “browser like scenarios”, having multiple parallel request just after the html has been retrieved… to get js, css, and images…

(4 italians only: “ditelo a Brunetta! – il Fatto Quotidiano: Certificati medici online, sistemi in tilt“)

webservices? well… in 2 years and a half in Vodafone we came to the conclusion that grinder with our self made HTTP-QAT toolkit was the best choice… (create your request from the WSDL with SoapUI and then put it in as a template inside HTTP-QAT, and fire!!!! :p )

I’d like to spend time to integrate tools like cucumber and tSung…
performance and easy way to describe/document the test scenario

Also Hadoop, with JFreeChart would be a good rewrite of Ground Report

what’s “ab”?
(read below, 95% of my requests were pretty fast…
we have some long tails on 5% of the overall sent http calls… )

grinder:~ zeph$ ab -c 5 -n 100 http://localhost/
This is ApacheBench, Version 2.3 
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done


Server Software:        Apache/2.2.15
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        44 bytes

Concurrency Level:      5
Time taken for tests:   0.570 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      42500 bytes
HTML transferred:       4400 bytes
Requests per second:    175.45 [#/sec] (mean)
Time per request:       28.498 [ms] (mean)
Time per request:       5.700 [ms] (mean, across all concurrent requests)
Transfer rate:          72.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       1
Processing:     0   15  80.6      1     569
Waiting:        0   15  80.6      1     569
Total:          1   15  80.7      1     570

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      2
  90%      2
  95%     78
  98%    569
  99%    570
 100%    570 (longest request)