3
\$\begingroup\$

This code creates a text file and writes a given number of characters to it. Is this a good method, or is there a faster method?

#include <fstream>
#include <iostream>

std::ofstream file;

int main() {
    int size = 10000000; //~10MB
    file.open("text_file.txt");
    for (int i = 0; i < size; i++) {
        file << "a";
    }
    return 0;
}
\$\endgroup\$
  • \$\begingroup\$ This is what the dd command is for. dd if=/dev/zero of=output.dat bs=1M count=10 Will fill a file with zero. If you specifically want a then yes 'a' | dd of=output.dat bs=1M count=10 \$\endgroup\$ – Martin York Jan 13 '17 at 17:04
  • \$\begingroup\$ Note: 10000000 is not 10MB \$\endgroup\$ – Martin York Jan 13 '17 at 17:06
  • \$\begingroup\$ @LokiAstari It is 10 MB, but not 10 MiB. \$\endgroup\$ – 200_success Jan 13 '17 at 17:48
  • 1
    \$\begingroup\$ @200_success: en.wikipedia.org/wiki/Megabyte The megabyte is commonly used to measure either 1000^2 bytes or 1024^2 bytes I think yuo will also find that all tools like dd use the 1024^2 definition. \$\endgroup\$ – Martin York Jan 13 '17 at 18:08
5
\$\begingroup\$

A great deal here depends on what you want to accomplish.

If you want to ensure that the file occupies N bytes of storage space, then your current method may easily fail. Some file systems (e.g., NTFS) support file compression. Writing the same character many times gives highly compressible data, so as you've written it, the file is likely to use only a small amount of storage ("small" is relatively, but certainly a lot less than the ~10 megabytes you intended to write).

If you're all right with that sort of result, then you can probably do quite a bit better with something like this:

int main() {
    std::ofstream file("text_file.txt");
    file.seekp(10000000);
    file << 'a';
}

This will often be much faster than what you've written. It'll also typically occupy very little space on disk. A directory listing will still normally show it as being the same size though.

If you want to ensure that your file really uses ~10 megabytes of disk space, you'll probably need to fill it with semi-random data instead of just repeating a single value. One way to do this would be to use a random number generator:

std::mt19937 gen{ std::random_device()() };

std::uniform_int_distribution<> dis(0, 255);

std::ofstream file("text_file.txt");
std::generate_n(std::ostream_iterator<char>(file, ""), size, [&]{ return dis(gen); });

Code Review

Globals vs. locals

There seems to be no real reason for making file a global variable as you did. Even if you're going to write to all the bytes of the file, it's almost certainly better to make it local to main (and initialize it upon creation):

int main() {
    int size = 10000000; //~10MB
    std::ofstream file("text_file.txt");
    [...]

Standard algorithms

Given that you're trying fill a container with a specified value, you might want to also consider using a standard algorithm designed for precisely that purpose:

    std::ofstream file("text_file.txt");
    std::fill_n(std::ostream_iterator<char>(file), size, 'n');

Use of headers

As it stands right now, you've included fstream.h, even though you don't seem to be using anything that's declared in that header. It's generally preferred to include only the headers you actually use.

\$\endgroup\$
1
\$\begingroup\$

In contrast with Jerry, I'll provide alternative solution that is supported by upcoming (C++17) filesystem library (modern compilers should already support most of it).

There is a function called std::filesystem::resize_file(), which takes a path to the file and the new size. The problem is that file needs to be created first. It can probably be done by simply creation an std::ofstream object and closing it immediately.

std::ofstream file(filename);
if (!file.is_open())
{
    throw std::runtime_error("File opening failed");
}

file.close();

Then using the function:

std::filesystem::resize_file(filename, newsize); //will throw if fails

Note that std::filesystem::path has a constructor that is not explicit. It is the reason for passing bare filename, without creating temporary variables and whatnot.

With all of this, it is possible now to accept arguments right from the command line arguments to make it easy to use:

int main(int argc, char* argv[])
{
    if (argc != 3) //first argument is always program's name on Linux and Windows
    {
        std::cout << "usage: filename newsize";
        return 0;
    }

    //parse the arguments and pass into the file resizing function.
}
\$\endgroup\$
0
\$\begingroup\$

I'm going to assume that what you want is really a file with 10 M as. If this is not the case, the other two answers already discuss some alternatives.

Separate different concerns into different functions

Coding everything int main is not very maintainable and not very testable. You should identify boundaries at which to separate you program's concerns into different functions. An obvious boundary in this case is the decision how many characters to write to what file and the logic that actually does this. So the refactoring would be to extract most of your code into a new function that takes a file name and a size as parameters and have your main call that function with the appropriate arguments. (Which probably shouldn't be hard-coded but read from the user as shown in Incomputable's answer.)

Do I/O in chunks, if all possible

You've asked about a “faster method”. Your current code is calling file << 'a'; 10 M times. And unfortunately, C++ streams are slow. (Like really slow.) You can speed up your code enormously by doing I/O in chunks. This is true for just about any I/O library I'm aware of but it is especially true for C++ streams.

Since I'm assuming that you want all as, std::strings constructor that takes a count and a character comes in handy. std::string(1024, 'a') will give you a string filled with 1024 as. People often use buffer sizes that are a power of 2 because the operating system might handle them a bit more efficiently. You have to benchmark it if you really care to find the optimum size and be prepared for this optimum to vary from machine to machine. Personally, I often use a buffer size of 4 KiB (4 Ki = 212 = 4096) which also is an often-used page size for virtual memory.

Of course, we now have a problem if the desired size of our output is not a multiple of the chosen chunk size. We have to take care of this by first looping as often as we can write full chunks and then eventually write a smaller chunk. Doing this with the << operators is not very useful. Better use the write member function of std::ostream.

Here's what we might come up with.

void
fill_file_with_as(const std::string& filename, const std::size_t size)
{
  const auto chunk = std::string(4096, 'a');
  auto file = std::ofstream{filename};
  //file.exceptions(std::ios_base::failbit);  /* see below */
  auto remaining = size;
  while (remaining > 0)
    {
      const auto n = std::min(chunk.size(), remaining);
      file.write(chunk.data(), n);
      remaining -= n;
    }
}

On my computer, it gives me a 10 × speedup (25 ms versus 250 ms) over your original code.

Always handle I/O errors

I/O can fail for various reasons that are outside the control of your program so you should always check their status and respond to errors appropriately. Often times, there isn't much your application can do about the problem but at least, it should inform the user by printing a helpful error message and exiting with a status reporting failure.

A seemingly simple approach to do this is setting the failbit in the exception mask of the stream. You do this by calling the exceptions member function as shown in the commented-out line in the code snippet above. Now, if any I/O operation fails, an exception will be thrown. This alone might be enough to not silently ignore the error but it would be nicer if your application printed a nice error message, too.

On a POSIX system, you can try writing to /dev/full (which will always fail) to test your program's error handling.

int
main()
{
  try {
    fill_file_with_as("/dev/full", 10000000);
    return EXIT_SUCCESS;
  } catch (const std::exception& e) {
    std::cerr << "error: " << e.what() << std::endl;
    return EXIT_FAILURE;
  }
}

Unfortunately, this gives the not-so-helpful error message on my system.

error: basic_ios::clear

I thought that C++11 should have improved this but apparently, my standard library didn't implement it.

The only reliable way I know to get useful error messages is to go back to good old C-style I/O. (Which many prefer over C++ streams anyway.)

void
fill_file_with_as(const std::string& filename, const std::size_t size)
{
  const auto chunk = std::string(4096, 'a');
  auto file = std::unique_ptr<std::FILE, decltype(&std::fclose)>{
    std::fopen(filename.c_str(), "w"),
    &std::fclose
  };
  if (!file)
    throw std::system_error{errno, std::system_category(), filename};
  auto remaining = size;
  while (remaining > 0)
    {
      const auto n = std::min(chunk.size(), remaining);
      const auto count = std::fwrite(chunk.data(), 1, n, file.get());
      if (count == 0)
        throw std::system_error{errno, std::system_category(), filename};
      remaining -= count;
    }
  if (std::fclose(file.release()) < 0)
    throw std::system_error{errno, std::system_category(), filename};
}

This gives me the following error message that might actually help a user diagnose the problem.

error: /dev/full: No space left on device

The down-side is that you have to check manually after each function call. Whether the code got easier to understand is also open to debate. If you do this a lot, writing small wrappers might be worthwhile.

Note that std::fwrite may write fewer than the requested number of bytes so we have to check its return value.

Also note that I'm calling std::fclose explicitly at the end of the function because closing a file can also fail and in this case, not all data might be written. If I'm leaving the function because of an exception, I don't care about additional errors but only want the std::unique_ptr to close the file at all so the resources are released again.

For a production-quality application, I recommend that you go through the extra effort to give your users the most helpful error messages possible. For less important code, just enabling exceptions in C++ streams might be acceptable. Knowing that something went wrong but not exactly what is still much better than pretending that everything went fine when in fact it didn't.

For completeness, here is the set of headers I've #included in the final version of the code.

#include <algorithm>     // std::min
#include <cerrno>        // errno
#include <cstddef>       // std::size_t
#include <cstdio>        // std::FILE, std::fopen, std::fclose, std::fwrite
#include <cstdlib>       // EXIT_SUCCESS, EXIT_FAILURE
#include <iostream>      // std::cerr, std::endl
#include <memory>        // std::unique_ptr
#include <string>        // std::string
#include <system_error>  // std::system_error, std::system_category
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.