Eventually, a constant result, even increasing the benchmark, parameters may tell you where the limitation is: Normally, a good approach is to play with the benchmark options until reaching the application limitation (e.g increasing the connections). All the benchmark tools have their own limitations, some of them describe their limitations on the main page. This is a fairly common issue when benchmarking on a small machine or using a high number of connections.
The load tool is like any software, it can and eventually will be affected by hardware limitations, for instance, autocannon (an HTTP/1.1 benchmarking tool) can saturate the CPU, where the autocannon process reaches 100% affecting the load generated, and consequently the final result.
HTTP BENCHMARK TOOL SOFTWARE
Benchmark Tool LimitationĪ common problem is one of benchmarking-benchmark, where the result reported is limited by the benchmark software itself. Therefore, the next subsections include some behaviours you may face during the benchmarking process. This article’s objective is to transform Unknown-Unknowns into Known-Unknowns. The Known-Unknowns is a term used to express the risks that come from situations that are so unexpected that they would not be considered - I wrote an article on Performance Methodologies that expands on this subject performance-wise.
The simple task of measuring latency can hide one of the biggest pitfalls of the benchmarking industry - We’ll take a look at it in the following sections. Metrics can be confusing, complicated, unreliable, inaccurate, and even plain wrong (due to bugs). Metrics are hard and evaluating them is a crucial part of a benchmark report. Have you ever considered that the metrics reported by your benchmark tool might be inaccurate?Īs with any work based on analysis and measurements, a common mistake is to think you know everything about your software/architecture. Accurate Metrics are Essential when Benchmarking
0 kommentar(er)
