std::string::length vs strlen using Google Benchmark

Question

I wrote my first benchmark using Google Benchmark to check what is faster between the use of std::string::length and strlen to compute the length of a std::string.

I would like to know what I can improve in my code to use it as a base for future benchmarks. My main concerns are about:

• the creation of the std::string at the beginning of my functions: is it the right place to do it? Should I use fixtures? Or create a global std::vector<std::string> that contains every strings I'll use for my benchmark?
• the use of benchmark::DoNotOptimize and benchmark::ClobberMemory: I'm not sure I use these functions as I should. The idea is the following: I want to be sure the std::string s is created and stored into memory before entering in my loop and I want to be sure the functions std::string::length and strlen are really called and not optimized. Is it the good way to do it?
• the use of DenseRange to create a benchmark with multiple length of string. At first, I wanted to pass an std::string directly to my function but, apparently, only int can be returned by state.range. Is there another way to pass a string to my functions?

Here is my code:

#include <benchmark/benchmark.h>

#include <string>
#include <cstring>

static void BM_InneficientBasicStringLength(benchmark::State& state)
{
    std::string s;

    for (int i = 0; i < state.range(0); i++)
        s += 'a';

    benchmark::DoNotOptimize(s.data());
    benchmark::ClobberMemory();
    size_t size;

    for (auto _ : state)
    {
        benchmark::DoNotOptimize(size = strlen(s.c_str()));
        benchmark::ClobberMemory();
    }
}

BENCHMARK(BM_InneficientBasicStringLength)->DenseRange(0, 100, 10);

static void BM_InneficientBasicStringLength_Fix(benchmark::State& state)
{
    std::string s;

    for (int i = 0; i < state.range(0); i++)
        s += 'a';

    benchmark::DoNotOptimize(s.data());
    benchmark::ClobberMemory();
    size_t size;

    for (auto _ : state)
    {
        benchmark::DoNotOptimize(size = s.length());
        benchmark::ClobberMemory();
    }
}
BENCHMARK(BM_InneficientBasicStringLength_Fix)->DenseRange(0, 100, 10);

BENCHMARK_MAIN();

It compiles with: g++ -o example example.cpp -lbenchmark -lpthread.

---

EDIT: For your information, the results are better with std::string::length which have a constant complexity while strlen have a linear complexity:

> ./example 
2020-08-14T14:48:17+02:00
Running ./example
Run on (12 X 3900 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x6)
  L1 Instruction 32 KiB (x6)
  L2 Unified 1024 KiB (x6)
  L3 Unified 8448 KiB (x1)
Load Average: 5.18, 6.05, 6.32
----------------------------------------------------------------------------------
Benchmark                                        Time             CPU   Iterations
----------------------------------------------------------------------------------
BM_InneficientBasicStringLength/0             4.79 ns         4.79 ns    150060775
BM_InneficientBasicStringLength/10            4.66 ns         4.66 ns    151148682
BM_InneficientBasicStringLength/20            5.39 ns         5.39 ns    136288403
BM_InneficientBasicStringLength/30            5.36 ns         5.35 ns    135353692
BM_InneficientBasicStringLength/40            5.19 ns         5.18 ns    137059957
BM_InneficientBasicStringLength/50            5.22 ns         5.22 ns    134737400
BM_InneficientBasicStringLength/60            5.32 ns         5.32 ns    134098029
BM_InneficientBasicStringLength/70            6.65 ns         6.65 ns    107061965
BM_InneficientBasicStringLength/80            6.85 ns         6.85 ns    109856173
BM_InneficientBasicStringLength/90            6.78 ns         6.78 ns    106762788
BM_InneficientBasicStringLength/100           6.66 ns         6.66 ns     98081332
BM_InneficientBasicStringLength_Fix/0         3.01 ns         3.01 ns    231875548
BM_InneficientBasicStringLength_Fix/10        3.06 ns         3.06 ns    221447369
BM_InneficientBasicStringLength_Fix/20        2.96 ns         2.96 ns    237989703
BM_InneficientBasicStringLength_Fix/30        2.98 ns         2.98 ns    234755616
BM_InneficientBasicStringLength_Fix/40        2.99 ns         2.98 ns    231015140
BM_InneficientBasicStringLength_Fix/50        2.94 ns         2.94 ns    223906062
BM_InneficientBasicStringLength_Fix/60        3.04 ns         3.04 ns    225199556
BM_InneficientBasicStringLength_Fix/70        3.07 ns         3.07 ns    230208201
BM_InneficientBasicStringLength_Fix/80        3.02 ns         3.02 ns    219373040
BM_InneficientBasicStringLength_Fix/90        3.10 ns         3.10 ns    219921248
BM_InneficientBasicStringLength_Fix/100       3.10 ns         3.10 ns    238076969