I wrote my first benchmark using Google Benchmark to check what is faster between the use of std::string::length
and strlen
to compute the length of a std::string
.
I would like to know what I can improve in my code to use it as a base for future benchmarks. My main concerns are about:
- the creation of the
std::string
at the beginning of my functions: is it the right place to do it? Should I use fixtures? Or create a globalstd::vector<std::string>
that contains every strings I'll use for my benchmark? - the use of
benchmark::DoNotOptimize
andbenchmark::ClobberMemory
: I'm not sure I use these functions as I should. The idea is the following: I want to be sure thestd::string s
is created and stored into memory before entering in my loop and I want to be sure the functionsstd::string::length
andstrlen
are really called and not optimized. Is it the good way to do it? - the use of
DenseRange
to create a benchmark with multiple length of string. At first, I wanted to pass anstd::string
directly to my function but, apparently, onlyint
can be returned bystate.range
. Is there another way to pass a string to my functions?
Here is my code:
#include <benchmark/benchmark.h>
#include <string>
#include <cstring>
static void BM_InneficientBasicStringLength(benchmark::State& state)
{
std::string s;
for (int i = 0; i < state.range(0); i++)
s += 'a';
benchmark::DoNotOptimize(s.data());
benchmark::ClobberMemory();
size_t size;
for (auto _ : state)
{
benchmark::DoNotOptimize(size = strlen(s.c_str()));
benchmark::ClobberMemory();
}
}
BENCHMARK(BM_InneficientBasicStringLength)->DenseRange(0, 100, 10);
static void BM_InneficientBasicStringLength_Fix(benchmark::State& state)
{
std::string s;
for (int i = 0; i < state.range(0); i++)
s += 'a';
benchmark::DoNotOptimize(s.data());
benchmark::ClobberMemory();
size_t size;
for (auto _ : state)
{
benchmark::DoNotOptimize(size = s.length());
benchmark::ClobberMemory();
}
}
BENCHMARK(BM_InneficientBasicStringLength_Fix)->DenseRange(0, 100, 10);
BENCHMARK_MAIN();
It compiles with: g++ -o example example.cpp -lbenchmark -lpthread
.
EDIT: For your information, the results are better with std::string::length
which have a constant complexity while strlen
have
a linear complexity:
> ./example
2020-08-14T14:48:17+02:00
Running ./example
Run on (12 X 3900 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x6)
L1 Instruction 32 KiB (x6)
L2 Unified 1024 KiB (x6)
L3 Unified 8448 KiB (x1)
Load Average: 5.18, 6.05, 6.32
----------------------------------------------------------------------------------
Benchmark Time CPU Iterations
----------------------------------------------------------------------------------
BM_InneficientBasicStringLength/0 4.79 ns 4.79 ns 150060775
BM_InneficientBasicStringLength/10 4.66 ns 4.66 ns 151148682
BM_InneficientBasicStringLength/20 5.39 ns 5.39 ns 136288403
BM_InneficientBasicStringLength/30 5.36 ns 5.35 ns 135353692
BM_InneficientBasicStringLength/40 5.19 ns 5.18 ns 137059957
BM_InneficientBasicStringLength/50 5.22 ns 5.22 ns 134737400
BM_InneficientBasicStringLength/60 5.32 ns 5.32 ns 134098029
BM_InneficientBasicStringLength/70 6.65 ns 6.65 ns 107061965
BM_InneficientBasicStringLength/80 6.85 ns 6.85 ns 109856173
BM_InneficientBasicStringLength/90 6.78 ns 6.78 ns 106762788
BM_InneficientBasicStringLength/100 6.66 ns 6.66 ns 98081332
BM_InneficientBasicStringLength_Fix/0 3.01 ns 3.01 ns 231875548
BM_InneficientBasicStringLength_Fix/10 3.06 ns 3.06 ns 221447369
BM_InneficientBasicStringLength_Fix/20 2.96 ns 2.96 ns 237989703
BM_InneficientBasicStringLength_Fix/30 2.98 ns 2.98 ns 234755616
BM_InneficientBasicStringLength_Fix/40 2.99 ns 2.98 ns 231015140
BM_InneficientBasicStringLength_Fix/50 2.94 ns 2.94 ns 223906062
BM_InneficientBasicStringLength_Fix/60 3.04 ns 3.04 ns 225199556
BM_InneficientBasicStringLength_Fix/70 3.07 ns 3.07 ns 230208201
BM_InneficientBasicStringLength_Fix/80 3.02 ns 3.02 ns 219373040
BM_InneficientBasicStringLength_Fix/90 3.10 ns 3.10 ns 219921248
BM_InneficientBasicStringLength_Fix/100 3.10 ns 3.10 ns 238076969
for (auto _ : state)
. Don't be obtuse in your code it is supposed to be human readable. \$\endgroup\$benchmark::DoNotOptimize
andbenchmark::ClobberMemory
. However, this is one of my questions: am I using this functions correctly in order to have a correct measure ofstrlen
andstd::string::length
. About the duration of my benchmark, that's all the problem with microbenchmarking. The idea is to take a very little piece of code outside of its context to measure its efficiency. To prevent any issue about the reproducibility, we can see each measures are performed at least 98081332 times. \$\endgroup\$