# Java splitting too large text file enhance performance

I have a too large data.txt file containing data as follows:

a = line0
line1
line2
€
b = line0
line1
line2
€
...
z = line0
line1


I am still beginner with java, I wrote this code to split the data.txt into multiple .txt files named a.txt , b.txt etc... it splits the data.txt at each €\n character.

It works perfectly, but it is so slow, I noticed, as if it reads everything at once and makes its mission, then at once the result .txt files are created and showed at once in the destination folder.

How could I make my code faster, for example when it is finished with the first part then it should create the result a.txt file immediatly then the second, third and so on... Thank in advance

import java.util.*;
import java.io.File;
import java.io.IOException;
import java.io.*;

public class Main
{
public static void main(String[] args)
{
File file = new File("/path/path2/file.txt");

try{
Scanner scanner = new Scanner(file);
String str = "";

while(scanner.hasNext()){

str+=scanner.nextLine()+"\n";

}

String charac = "€";
String end = "end of file";

for(int i = 0; i < (str.split(charac).length)-1; i++){

String name = str.split(charac)[i].split(" = ")[0];

String out= "/path/path2/path3/Folder/"+name+".txt";
FileWriter fw = new FileWriter(out);
BufferedWriter bw = new BufferedWriter(fw);

bw.write(str.split(charac)[i]+"\n"+end);
bw.close();
}

System.out.println("Splitting is finished");

}
catch(Exception e){}
}
}

• Your program is slow because it load the full file in memory before splitting it (str+=scanner.nextLine()+"\n";). You can do the split inside the first loop (where your are reading the file) – gervais.b Dec 11 '18 at 8:38
• What do you mean exactly, should I move the while block with its statement str+=scanner.nextLine()+"\n"; into the for-loop as first statement? – Khaled Dec 11 '18 at 10:38
• you should move the whole splitting logic inside the first loop while(scanner.hasNext()) – gervais.b Dec 11 '18 at 11:18
• and use a StringBuilder object rather than string concatenation – pcoates Dec 19 '18 at 23:06

while(scanner.hasNext()){