5
\$\begingroup\$

The code below implements a class, CountingOStream, whose job is to count the number of characters written to it while also discarding those characters and performing no memory allocation. It would be used to measure the size of some data that is being considered for writing before actually writing it.

It is based on examples at:

None of them do precisely what I want, but the first is quite close, so I chose that as the basis of my attempt.

It appears to work properly based on the included testing.

Specific areas of concern:

  • Have I missed some important technical requirement when inheriting std::streambuf and std::ostream?

  • Should there be additional constructors? My class is closest in spirit to std::ostringstream, which has a lot of constructors, but I've only ever used its default constructor, so that is all I defined in my class. Is there some important idiom for which only having the default will be a problem?

  • Are there unforeseen issues, including potential performance problems, in overriding overflow this way? It almost seems too easy...

  • My class is a streambuf, albeit privately, in addition to being an ostream. That raises the potential for ambiguities that would not be encountered when using some other ostream derived class.

    • I already hit one resulting ambiguity, since int_type is declared on both sides. That one is easy to deal with inside the class, but if someone asks for CountingOStream::int_type they will hit the ambiguity. I could add a declaration of CountingOStream::int_type to cover that, but do not expect it to be an issue in practice since I'm unaware of stream clients needing to use int_type (or other types potentially similarly affected).

    • There do not seem to be any operators (especially operator<<) overloaded for streambuf, but perhaps I've overlooked something that could cause a CountingOStream to not be usable where a std::ostringstream would.

Known issues I am not concerned about

  • It is my understanding that a std::ofstream opened in text mode on Windows will turn LF into CRLF and hence would produce a longer file than this counter reports. For now at least, that's fine.
// counting-ostream.cc
// ostream that just counts the characters written to it.

// Loosely based on answers found at:
//
//   https://stackoverflow.com/questions/772355/how-to-inherit-from-stdostream
//   https://stackoverflow.com/questions/41377045/is-there-a-simple-way-to-get-the-number-of-characters-printed-in-c
//   https://stackoverflow.com/questions/27534955/c-get-number-of-characters-printed-when-using-ofstream
//
// with the first one providing the actual code starting point.


// -------------------------- Implementation ---------------------------
#include <cstddef>                     // size_t
#include <iostream>                    // std::streambuf, std::ostream


// Present the 'std::ostream' interface and count the number of
// characters written to it.  That is, the final count equals what one
// would get by instead writing to a 'std::ostringstream' and then
// asking for the length of the string it contains at the end.  But this
// class avoids all of the memory allocation that that would entail.
//
// 'streambuf' is inherited privately because we need to override one of
// its virtual methods (so it cannot be a member) but 'CountingOStream'
// is not meant to be treated as a 'streambuf' by clients.
//
class CountingOStream : private std::streambuf, public std::ostream {
public:      // data
  // Count of characters seen.
  size_t m_count;

public:      // methods
  CountingOStream()
    : std::streambuf(),
      std::ostream(this /*as a streambuf*/),
      m_count(0)
  {}

private:     // methods
  // Override the method to count and discard written characters.
  //
  // There is an 'int_type' coming from both superclasses, so pick the
  // one relevant to the overridden method.
  virtual std::streambuf::int_type overflow(
    std::streambuf::int_type c) override
  {
    ++m_count;

    // Anything other than Traits::eof() (usually -1) means success.
    return 0;
  }
};


// ------------------------------- Test --------------------------------
#include <cassert>                     // assert
#include <sstream>                     // std::ostringstream


// Run a single test.  'writeOperands' is a chain of things to be
// written using 'operator<<'.
#define TEST_WITH(writeOperands)                        \
{                                                       \
  CountingOStream cos;                                  \
  cos << writeOperands;                                 \
                                                        \
  std::ostringstream oss;                               \
  oss << writeOperands;                                 \
                                                        \
  size_t actual = cos.m_count;                          \
  size_t expect = oss.str().size();                     \
                                                        \
  /* Print each test case and flush before checking. */ \
  std::cout                                             \
    << "actual=" << actual                              \
    << " expect=" << expect                             \
    << " string: " << oss.str()                         \
    << std::endl;                                       \
  assert(actual == expect);                             \
}


int main()
{
  TEST_WITH("Hello, world!\n");
  TEST_WITH("Look a number: " << std::hex << 29 << std::endl);
  TEST_WITH('\0');
  TEST_WITH('\377');

  // Larger test to exercise any buffering, resizing, etc.
  {
    CountingOStream cos;
    std::ostringstream oss;

    for (int i=0; i < 10000; ++i) {
      cos << "string" << 'x' << 123;
      oss << "string" << 'x' << 123;
    }

    size_t actual = cos.m_count;
    size_t expect = oss.str().size();

    std::cout << "larger: actual=" << actual
              << " expect=" << expect
              << std::endl;
    assert(actual == expect);
  }

  return 0;
}


// EOF

Output from running the test program:

actual=14 expect=14 string: Hello, world!

actual=18 expect=18 string: Look a number: 1d

actual=1 expect=1 string: 
actual=1 expect=1 string: <a replacement character>
larger: actual=100000 expect=100000
\$\endgroup\$

2 Answers 2

6
\$\begingroup\$

Areas of concern

Have I missed some important technical requirement when inheriting std::streambuf and std::ostream?

This is not the correct way to go about creating a custom stream.

I can’t even begin to guess what kind of problems you’ll run into dual-inheriting from both std::streambuf and std::ostream, because it’s just such a bizarre idea. Certainly, a lot of things will be very weird.

The correct way to “do something custom” with streamed data is to create a custom stream buffer (and then, optionally, a new stream type that uses that stream buffer by default). Streams are really just thin wrappers around stream buffers. Streams are the formatting layer; stream buffers are where the business happens (even if the business, in your case, is just counting). You are not changing the way formatting is done (you want the formatting to be done normally, and then you want to count the characters that result), therefore you shouldn’t be doing anything in the stream layer.

You don’t even really need to create a new stream type at all, because you can just use the buffer in an extant stream. But making a custom stream makes things more ergonomic.

So you should have something like this:

class counting_streambuf final :
    public std::streambuf
{
    auto overflow(int_type ci = traits_type::eof()) -> int_type override
    {
        ++count;
        return traits_type::not_eof(ci);
    }

public:
    // For practical purposes, you're going to want streams and stream buffers
    // to be movable and swappable. So that means you need the move
    // constructor, move assignment operator, and swap.

    counting_streambuf() = default;
    counting_streambuf(counting_streambuf&&) = default;
    auto operator=(counting_streambuf&&) -> counting_streambuf& = default;

    auto swap(counting_streambuf& rhs) -> void
    {
        std::streambuf::swap(rhs);
        std::ranges::swap(count, rhs.count);
    }

    std::size_t count = 0;
};

// You don't *need* this stream class. You could just do:
//  auto buf = counting_streambuf{};
//  auto os = std::ostream{&buf};
//  os << "foo";
//  std::cout << buf.count; // should print 3
//
// But the stream class is much nicer:
//  auto os = counting_ostream{};
//  os << "foo";
//  std::cout << os.count(); // should print 3

class counting_ostream :
    public std::ostream
{
    counting_streambuf _buf;

public:
    // As with the stream buffer, you want the stream to be movable and
    // swappable.
    //
    // Unfortunately, we can't default the constructors, because we need to
    // make sure the address of the buffer is passed to the base class.

    counting_ostream() :
        std::ostream{&_buf}
    {}

    counting_ostream(counting_ostream&& rhs) :
        std::ostream{std::move(rhs)},
        _buf{std::move(rhs._buf)}
    {
        set_rdbuf(&_buf);
    }

    auto operator=(counting_ostream&& rhs) -> counting_ostream& = default;

    auto swap(counting_ostream& rhs) -> void
    {
        std::ostream::swap(rhs);
        _buf.swap(rhs._buf);
    }

    // Defining this is a tactical choice to block access to
    // std::ios_base::rdbuf()... both overloads.
    // Note we can't use "override" because we're changing the return type.
    //
    // We don't *need* to do this, but by doing it we make it harder to
    // replace the buffer, and we have it return a correctly-typed pointer.
    auto rdbuf() const -> counting_streambuf*
    {
        return const_cast<counting_streambuf*>(&_buf);
    }

    auto count() const noexcept { return _buf.count; }
};

Should there be additional constructors?

I don’t see why.

std::ostringstream has constructors for:

  1. Setting the mode. But that’s irrelevant for your use case. Your only “mode” is output. Appending versus truncating is meaningless. Seeking is meaningless. Binary versus text is also meaningless.
  2. Setting the initial text content. Again, irrelevant for your use case.

Are there unforeseen issues, including potential performance problems, in overriding overflow this way?

This works (as your tests show), but it is going to be wildly inefficient in practice.

Suppose I have a few thousand characters of data that I want to write to this stream. The natural way would be to do os.write(data, size). This will work just fine… but consider how it works. Each single character of those thousands of characters will be counted… one… at… a… time.

And, each character will trigger multiple virtual calls to be counted. You can take some steps to make this less excruciating. For example, you could make the stream buffer final, so that the calls should be de-virtualized in the stream. Still, you are counting each character one-by-one.

But… you were literally told how many characters are in the data. It was right there in the call: os.write(data, size).

Consider also overriding .xsputn(). The standard allows you to not call .overflow() for each character, so you could just add the new count to the existing count and be done with it. That way, instead of thousands upon thousands of calls to .overflow(), each incrementing the count by one, you can do it all in one fell swoop.

My class is a streambuf, albeit privately, in addition to being an ostream. That raises the potential for ambiguities that would not be encountered when using some other ostream derived class.

That’s an understatement.

Keeping the stream buffer base private probably protects against most problems… but whenever you have the choice composition is much better than private inheritance. In this case, there is literally no benefit to private inheritance.

It is my understanding that a std::ofstream opened in text mode on Windows will turn LF into CRLF and hence would produce a longer file than this counter reports. For now at least, that's fine.

Okay, but for the record, you are massively underestimating the complexity here.

For starters, if your ultimate goal is to estimate the size of a file after a bunch of stuff is streamed into it… this ain’t gonna work. You’ve already identified that the way Windows transforms newlines is one problem… but you know, that’s not the only translation Windows does in text mode. So even if you checked for newlines and added one for each… your count could still be wrong. And that’s just on Windows. Other platforms may have other nasty surprises up that alley.

But there’s more.

Because you’re also not taking the locale into account, nor the character traits. It is quite possible for someone to imbue a locale with a character transformation. In that situation, simply counting the characters you’re given won’t suffice. You’d have to do a codecvt transformation first, and count the result.

My machine’s locale is en_CA.utf8; Canadian English, UTF-8. But someone else’s might be ja_JP.SJIS. When writing the same string to files in each locale, you might end up with very different results, including different sizes.

There is no way to solve all of these issues. The only portable solution is to straight-up write the file, then query its size (but even that is fraught, for various complex reasons).

So if that’s your end goal… you ain’t gonna get there.

If all you want is to just count the number of characters conceptually put into a buffer after formatting… then sure, this will work. If you want the count after the transformations done by a particular buffer (like std::filebuf)… then no, that’s impossible (unless the buffer helpfully gives you that information, or puts the result somewhere where you can then count after the fact… like std::filebuf does by putting it in a file that you can then inspect).

Comments on code

#include <cstddef>                     // size_t

That’s std::size_t. The std:: is not optional.

Naked size_t “works” on most platforms for historical reasons. But that will very likely change in the near future, as modules become the norm.

  virtual std::streambuf::int_type overflow(
    std::streambuf::int_type c) override

Use either virtual or override… not both.

I’m not going to bother critiquing the test code because… well, it’s test code. But I will say that you should use a proper testing library, rather than trying to roll your own.

\$\endgroup\$
5
  • \$\begingroup\$ Many good points! (1) In counting_ostream, isn't it technically undefined behavior to initialize a base class subobject using a pointer to a member since the member has not been constructed? (2) Regarding xsputn, that is exactly the sort of thing I thought must exist--perfect, thanks! \$\endgroup\$ Commented May 18 at 1:29
  • \$\begingroup\$ The member object does not exist yet, but the memory has been allocated, so as long as the base constructor doesn’t dereference the pointer (or do any other pointer UB stuff), there is no UB. And the standard promises (indirectly) that the std::ostream constructor only compares the pointer to nullptr, and keeps a copy, so it’s fine. I know it’s skeevy-looking, but IOStreams is literally the oldest standard library, with a ton of other skeevy practices (two-phase construction!), so… 🤷🏼 \$\endgroup\$
    – indi
    Commented May 18 at 20:43
  • \$\begingroup\$ If it really bothers you aesthetically, you can always pass nullptr to the base constructor then call init() in the derived constructor body with the actual pointer. Though, do bear in mind this has its own dangers. I vaguely recall the Microsoft standard library used to crash or something like that when you did this. But I just realized that was a long, long time ago, and I’m getting really old. I believe doing that is fine these days, and was eventually explicitly blessed by later standards. \$\endgroup\$
    – indi
    Commented May 18 at 20:44
  • \$\begingroup\$ On a whim, I decided to peek at how the most recent standard libraries on my machine do things in ofstream. Clang 18 does the same thing I did (file <fstream>, lines 1404–1410); has a member filebuf object, and passes its address to the ostream constructor in the initializer list. GCC 13 is a little weirder (as usual). It uses a non-standard default-constructor for ostream, then calls init() in the constructor body (file <fstream>, lines 786–787). The non-standard ostream constructor just does init(nullptr) (file <ostream>, lines 431–432). \$\endgroup\$
    – indi
    Commented May 18 at 23:13
  • \$\begingroup\$ I asked a follow-up question about the constructor call validity: Is it undefined behavior to pass a pointer to an unconstructed streambuf object to the ostream constructor? \$\endgroup\$ Commented May 22 at 4:54
4
\$\begingroup\$

Avoid macros

There are lots of problems with macros, and unless you are careful you will fall into one of its traps. Consider for example:

int i = 1;
TEST_WITH(i++);

This will cause the assertion to trigger. But more importantly, your macro didn't manage to avoid all of the code duplication, which was the whole point. Luckily, you almost never have to use macros in C++. Instead, make it a regular function, and to make it handle complex tests, have it take another function as a parameter:

static void test_with(std::function<void(std::ostream&)> test_function) {
    CountingOStream cos;
    std::ostringstream oss;

    test_function(cos);
    test_function(oss);
    …
}

int main() {
    test_with([](std::ostream& os){ os << "Hello, world!\n"; });
    …
    test_with([](std::ostream& os){
        for (int i = 0; i < 10000; ++i) {
            os << "string" << 'x' << 123;
        }
    });
}

Make m_count private

I would make m_count private and instead provide a get_count() member function. This prevents writes to m_count, and also allows you to change the implementation of how you actually store the count without having to modify any of the users of your class.

What about other character types?

Note that std::ostream is actually a std::basic_ostream<char>. However, there are more character types than just char. It would be nice to mirror what the standard library is doing, and provide a BasicCountingOStream template, and make CountingOStream an alias for BasicCountingOStream<char>.

\$\endgroup\$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.