15 line Rust program, which writes the numbers 1 to N to a file

Question

I have this simple Rust program, which creates a file called numbers.txt and writes the numbers 1 to N (N is here 10000000) to it, separated with newlines:

use std::fs::File;
use std::io::Write;

const N: i32 = 1e7 as i32;

fn main() {
    let mut data = String::new();

    for i in 1..=N  {
        data.push_str(&format!("{0}\n", i));
    }

    let mut f = File::create("numbers.txt").expect("Unable to create file");
    f.write_all(data.as_bytes()).expect("Unable to write data");
}

I've been running and timing the program with:

$ time cargo run --release

numbers.txt after running:

1
2
3
4
...
9999999
10000000

I've timed the execution of the program, and it takes around 4.8 seconds to run on my machine. I tried timing just the string building, by commenting out the last two lines, and the time taken stayed pretty much the same. We can conclude that the bottleneck is the data string building and not writing to the file.

How could I make building data faster? Should I use something else than a String?

Answer 1

I improved the performances (and simplicity) like this:

use std::fs::File;
use std::io::Write;
use std::io::BufWriter;

const N: i32 = 1e7 as i32;

fn main() {
    let mut f = BufWriter::new(File::create("numbers.txt").expect("Unable to create file"));
    for i in 1..=N  {
        write!(f, "{0}\n", i);
    }

}

I just don't store the strings in a gigantic buffer but asks the BufWriter to manage the buffering. As a side effect, it's now possible to write bigger files without holding too much RAM.

Answer 2

This cuts the runtime by about a third (4.8s -> 3.2s):

use std::fs::File;
use std::io::Write;

const N: i32 = 1e7 as i32;

fn main() {
    let data = (1..=N)
        .map(|n| n.to_string())
        .collect::<Vec<String>>()
        .join("\n");

    let mut f = File::create("numbers.txt").expect("Unable to create file");
    f.write_all(data.as_bytes()).expect("Unable to write data");
}

Answer 3

How about some threads?

use rayon::prelude::*;
use std::fs::File;
use std::io::Write;

const N: i32 = 1e7 as i32;

fn main() {
    let mut output = File::create("numbers.txt").unwrap();

    let data: Vec<String> = (1..=N)
        .into_par_iter()
        .fold(String::new, |mut lhs, rhs| {
            use std::fmt::Write;
            write!(&mut lhs, "{}\n", rhs).unwrap();
            lhs
        })
        .collect();

    for part in data {
        output.write_all(&part.into_bytes()).unwrap();
    }
}