0
\$\begingroup\$

I have the following .txt file structure that i want to parse in Java(8):

string1
string2
B|C|D
R|S|T
100 200 300 400
1 2 3 4
H|A
B|C|D
R|S|T
200 300 400 500
2 3 4 5
H|A

If there was string3 then there would be another related set of lines such as:

B|C|D
R|S|T
200 300 400 500
2 3 4 5
H|A

The parsed values will then be eventually printed out or played with somehow. Here's the code:

BufferedReader readerMain = new BufferedReader(new FileReader(file));

        int nbNonEmptyLine = 1;
        

     
        long count = Files.lines(Paths.get(file)).filter(line -> line.length() > 0).count();
        String currentLineMain = new String();
        String line0Value = new String();
        String line1Value = new String();
        ArrayList<String> line2Value = new ArrayList<String>();
        ArrayList<String> line3Value = new ArrayList<String>();
        String line4Value = new String();
        
       
        ArrayList<String> firstStrings = new ArrayList<String>();
        
        String firstString = new String();
        

        while ( (currentLineMain = readerMain.readLine())  != null  ){
            if (currentLineMain.length() == 0) continue;
            if ( currentLineMain.equals("B") || currentLineMain.equals("C") || currentLineMain.equals("D") ) break;
            StringTokenizer st = new StringTokenizer(currentLineMain);
            firstString = st.nextToken();

            System.out.println(firstString);
            firstStrings.add(firstString);

        }
         
        do  {
            if (currentLineMain.length() == 0) continue;
            switch (nbNonEmptyLine) {
                case 1:
                    StringTokenizer st2 = new StringTokenizer(currentLineMain);
                    line0Value = st2.nextToken();
                    if (line0Value.equals("B") || line0Value.equals("C") || line0Value.equals("D"))
                        System.out.println(line0Value);
                    break;

                case 2:
                    StringTokenizer st3 = new StringTokenizer(currentLineMain);
                    line1Value = st3.nextToken();
                    if (line1Value.equals("R") || line1Value.equals("S") || line1Value.equals("T") 
                        System.out.println(line1Value);
                    break;

                case 3:
                    StringTokenizer st4 = new StringTokenizer(currentLineMain);

                    while (st4.hasMoreTokens()) {

                        line2Value.add(st4.nextToken());
                    }
                    System.out.println(line2Value);
                    break;

                case 4:
                    StringTokenizer st5 = new StringTokenizer(currentLineMain);
                    while (st5.hasMoreTokens()) {

                        line3Value.add(st5.nextToken());
                    }
                    System.out.println(line3Value);
                    break;

                case 5:
                    StringTokenizer st6 = new StringTokenizer(currentLineMain);
                    line4Value = st6.nextToken();
                    if (line4Value.equals("H") || line4Value.equals("A") )
                        System.out.println(line4Value);
                    nbNonEmptyLine = 0;
                    break;




            }

            line0Value = new String();
            line1Value = new String();
            line2Value = new ArrayList<String>();
            line3Value = new ArrayList<String>();
            line4Value = new String();
            
            
            
            firstString = new String();

            nbNonEmptyLine++;
         }
        while ((currentLineMain = readerMain.readLine()) != null && nbNonEmptyLine <= count);

The first while loop is consuming string1, string2... then the second while loop deals with the rest:

My question is: is there a solution here with only one while loop parsing the whole file?

PS : the code works as expected.

\$\endgroup\$
2
  • \$\begingroup\$ I haven't tried this, but maybe a lexical parer like jflex is what your looking for? \$\endgroup\$
    – Neil
    Jan 1, 2022 at 19:20
  • 4
    \$\begingroup\$ Where did these data come from? What are the field meanings? \$\endgroup\$
    – Reinderien
    Jan 1, 2022 at 19:38

2 Answers 2

2
\$\begingroup\$

Your code had a compiler error and was missing some context. I used the following input as random.txt.'

string1
string2
B|C|D
R|S|T
100 200 300 400
1 2 3 4
H|A
B|C|D
R|S|T
200 300 400 500
2 3 4 5
H|A

After fixing your code and running it, I generated the following output.

string1
string2
B|C|D
R|S|T
100
1
H|A
B|C|D
R|S|T
200
2
H|A

You're missing some numbers. If you're just saving the lines as Strings, what's the point of all your code? I'm confused.

You basically have three different line formats.

  1. string
  2. B|C
  3. 100 200

You have no good way of telling these lines apart, so what you have to do is check for line three format first, then line two format, then line one format.

Here's the output from one of my test runs.

Group [title=string1]
Group [title=string2
    letters=[B, C, D, R, S, T, H, A, B, C, D, R, S, T, H, A]
    numbers=[100, 200, 300, 400, 1, 2, 3, 4, 200, 300, 400, 500, 2, 3, 4, 5]]

I created a Group class to group letters and numbers together. If there's any distinction between letter groups and number groups, you didn't make the distinction clear in your question. So, I put the letters together and the numbers together.

I processed the line as a number line first, then a letter line. If neither parse succeeded, I assumed it was the beginning of a new Group instance.

Parsing the letter lines turned out to be a little tricky.

Here's the complete runnable code. I made the Group class an inner class so I could post this code as one block.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

public class ParseTextFile {
    
    public static void main(String[] args) {
        new ParseTextFile().processTextFile();
    }
    
    public void processTextFile() {
        try {
            InputStream is = ParseTextFile.class.getResourceAsStream("/random.txt");
            BufferedReader reader = new BufferedReader(new InputStreamReader(is));
            String line = reader.readLine();
            Group group = null;
            
            while (line != null) {
                int[] numbers = processNumberLine(line);
                if (numbers[0] == Integer.MIN_VALUE) {
                    String[] letters = processLetterLine(line);
                    
                    if (letters.length <= 1) {
                        if (group != null) {
                            System.out.println(group.toString());
                        }
                        group = new Group(line);
                    } else {
                        group.addLetters(letters);
                    }
                } else {
                    group.addNumbers(numbers);
                }
                
                line = reader.readLine();
            }; 
            
            System.out.println(group.toString());
            reader.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    
    private int[] processNumberLine(String line) {
        String[] parts = line.split(" ");
        int[] output = new int[parts.length];
        
        for (int index = 0; index < parts.length; index++) {
            try {
                output[index] = Integer.valueOf(parts[index]);
            } catch (NumberFormatException e) {
                output[0] = Integer.MIN_VALUE;
                break;
            }
        }
        
        return output;
    }
    
    private String[] processLetterLine(String line) {
        return line.split("\\|");
    }
    
    public class Group {
        
        private List<Integer> numbers;
        
        private List<String> letters;
        
        private final String title;
        
        public Group(String title) {
            this.title = title;
            this.numbers = new ArrayList<>();
            this.letters = new ArrayList<>();
        }
        
        public void addLetters(String... letter) {
            for (String value : letter) {
                letters.add(value);
            }
        }
        
        public void addNumbers(int... number) {
            for (int value : number) {
                numbers.add(value);
            }
        }

        @Override
        public String toString() {
            StringBuilder builder = new StringBuilder();
            builder.append("Group [title=");
            builder.append(title);
            
            if (letters.size() > 0) {
                builder.append(System.lineSeparator());
                builder.append("    ");
                builder.append("letters=");
                builder.append(Arrays.toString(letters.toArray(new String[letters.size()])));
            }
            
            if (numbers.size() > 0) {
                builder.append(System.lineSeparator());
                builder.append("    ");
                builder.append("numbers=");
                builder.append(Arrays.toString(numbers.toArray(new Integer[numbers.size()])));
            }
            
            builder.append("]");
            return builder.toString();
        }
        
    }

}
\$\endgroup\$
9
  • \$\begingroup\$ for the output your first mention, what was the input ? \$\endgroup\$
    – singh raj
    Jan 1, 2022 at 16:51
  • 2
    \$\begingroup\$ @singh raj: When writing a question, it's a good idea to show the desired output. Your question lacks information and is too abstract to be answered correctly. Garbage in, garbage out. \$\endgroup\$ Jan 1, 2022 at 16:51
  • \$\begingroup\$ @singh raj: I created an input file from the random lines you posted in your question. \$\endgroup\$ Jan 1, 2022 at 16:55
  • \$\begingroup\$ the code should parse that file and output the same values contained in the file that's it \$\endgroup\$
    – singh raj
    Jan 1, 2022 at 16:55
  • 1
    \$\begingroup\$ @singhraj Unfortunately, played with somehow is the opposite of clear. \$\endgroup\$
    – Reinderien
    Jan 1, 2022 at 22:56
2
\$\begingroup\$

Parsing your file is effectively meaningless unless you give us some constraints, background information, context and realistic scenarios. I'm going to make some sweeping assumptions and show you one way to parse the data that's moderately advanced, but first, on your existing code:

It's not only all in one class, it's all in one function (that you don't show). This is Java-unidiomatic. At the absolute least, you should define an object that represents each record in your file; and your parsing code shouldn't be baked into your main entry point.

Don't embed println in your parsing code. Consumption and printing should happen outside of it.

Rather than running a StringTokenizer over each line, it's probably lighter-weight to just split(). Java documents this as accepting a regex, but that's at least partially a lie; the implementation includes a fast path for splitting on single characters.

Generally, while you should be proud to have made a working solution, I find it to be fragile and repetitive.

in Java(8):

Unless you're a time traveller from 2014, target something newer. My sample code uses Java 17.

is there a solution here with only one while loop parsing the whole file?

That depends on a raft of information that you haven't stated. Are the initial title strings guaranteed not to contain pipe characters; and are the two "header" lines in each record guaranteed to have at least two columns? If so, yes, you can do this in one pass. There are many other solutions that imply inner loops executed by the runtime and zero loops of your own.

One solution involves

  • Reading all lines into memory
  • Creating a stream-compatible Spliterator that iterates over those lines in a potentially parallel manner
  • From that stream, producing instances of a record.

It's difficult to say whether parallelism is called for in your parsing, or even whether performance is at all a concern. The following succeeds in parsing your sample data:

Parser.java

package com.stackexchange.MysteryFormat;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Arrays;
import java.util.List;
import java.util.ListIterator;
import java.util.Spliterator;
import java.util.function.Consumer;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;

public class Parser implements Spliterator<Item> {
    private static final int RECORD_LINES = 6;

    private final List<String> lines;
    private final long itemCount;
    private long index, end;
    private final ListIterator<String> bodyLines;

    public Parser(Path path) throws IOException {
        this(Files.readAllLines(path));
    }

    public Parser(List<String> lines) {
        // This will discard any trailing lines when the line count is not a multiple of RECORD_LINES
        this(lines, lines.size() / RECORD_LINES);
    }

    private Parser(List<String> lines, long itemCount) {
        this(lines, itemCount, 0, itemCount);
    }

    private Parser(List<String> lines, long itemCount, long start, long end) {
        this.lines = lines;
        this.itemCount = itemCount;
        this.index = start;
        this.end = end;
        long bodyIndex = itemCount + index*(RECORD_LINES - 1);
        bodyLines = lines.listIterator((int)bodyIndex);
    }

    @Override
    public long estimateSize() { return end - index; }

    @Override
    public int characteristics() {
        return ORDERED | SIZED | SUBSIZED | NONNULL | IMMUTABLE;
    }

    @Override
    public Spliterator<Item> trySplit() {
        long size = estimateSize();
        if (size < 2) return null;
        long nextStart = index + size/2;
        var next = new Parser(lines, itemCount, nextStart, end);
        // System.out.printf("Splitting %d-%d to %d-%d, %d-%d%n",
        //        index, end, index, nextStart, nextStart, end);
        end = nextStart;
        return next;
    }

    private String[] splitPipe() {
        return bodyLines.next().split("\\|");
    }

    private int[] splitSpace() {
        return Arrays.stream(
                bodyLines.next().split(" ")
            )
            .mapToInt(Integer::parseInt)
            .toArray();
    }

    @Override
    public boolean tryAdvance(Consumer<? super Item> action) {
        if (index >= end)
            return false;

        var item = new Item(
            lines.get((int)index++),
            splitPipe(), splitPipe(),
            splitSpace(), splitSpace(),
            splitPipe()
        );
        action.accept(item);
        return true;
    }

    public Stream<Item> stream() {
        // Change this depending on scaling characteristics
        boolean parallel = true;
        return StreamSupport.stream(this, parallel);
    }
}

LoadTest.java

import com.stackexchange.MysteryFormat.Item;
import com.stackexchange.MysteryFormat.Parser;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

import java.io.IOException;
import java.net.URISyntaxException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

public class LoadTest {
    @Test
    public void testLoading() throws URISyntaxException, IOException {
        Path path = Paths.get(
            getClass().getResource("/op-example.txt").toURI()
        );
        var parser = new Parser(path);
        List<Item> items = parser.stream().toList();

        // .. add your asserts here
    }

    @Test
    public void bigTest() {

        int n = 100_000;
        List<String> lines =
            IntStream.range(0, n)
            .mapToObj(Integer::toString)
            .collect(Collectors.toList());
        for (int i = 0; i < n; i++) {
            lines.add("B|C|D");
            lines.add("R|S|T");
            lines.add("100 200 300 400");
            lines.add("1 2 3 4");
            lines.add("H|A");
        }

        var parser = new Parser(lines);
        List<Item> items = parser.stream().toList();

        Set<String> set = items.stream()
            .map(Item::title)
            .collect(Collectors.toUnmodifiableSet());

        assertEquals(n, set.size());

        // .. add your asserts here
    }
}

Item.java

package com.stackexchange.MysteryFormat;

public record Item(
    String title,
    String[] header1,
    String[] header2,
    int[] values1,
    int[] values2,
    String[] footer
) {
}

A simpler, iterator-based, single-pass implementation could look like

package com.stackexchange.MysteryFormat;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.nio.file.Path;
import java.util.*;

public class SimpleParser implements Iterable<Item>, Iterator<Item>, AutoCloseable {
    private final BufferedReader reader;
    private final FileReader file;
    private final ListIterator<String> titles;
    private String nextLine;

    public SimpleParser(Path path) throws IOException {
        file = new FileReader(path.toFile());
        reader = new BufferedReader(file);
        List<String> titleList = new ArrayList<>();

        while (true) {
            String line = reader.readLine();
            if (line == null)
                throw new IOException("End of file before record body found");
            if (line.contains("|")) {
                nextLine = line;
                break;
            }
            titleList.add(line);
        }

        titles = titleList.listIterator();
    }

    @Override
    public void close() throws Exception {
        reader.close();
        file.close();
    }

    @Override
    public Iterator<Item> iterator() { return this; }

    @Override
    public boolean hasNext() { return nextLine != null; }

    private String[] splitPipe() throws IOException {
        return splitPipe(reader.readLine());
    }

    private static String[] splitPipe(String line) {
        return line.split("\\|");
    }

    private int[] splitSpace() throws IOException {
        return Arrays.stream(
                reader.readLine().split(" ")
            )
            .mapToInt(Integer::parseInt)
            .toArray();
    }

    @Override
    public Item next() {
        Item item;
        try {
            item = new Item(
                titles.next(),
                splitPipe(nextLine), splitPipe(),
                splitSpace(), splitSpace(),
                splitPipe()
            );
            nextLine = reader.readLine();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        return item;
    }
}
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.