Closing InputStream of a running process

Question

Let's say you want to extract the output of a console application till you've found a certain keyword. If you have found the keyword the started console process still should be running, however the function which has found the keyword should return.

Here is a minimal application which simply creates an output of n lines (n is the first argument).

LineOutput.java

public class LineOutput {
    public static void main(String[] args) {
        int iterations = 10;
        if (args.length > 0) {
            iterations = Integer.valueOf(args[0]);
        }
        for (int i=0; i < iterations; i++) {
            System.out.println("Line " + i);
        }
    }
}

And here is the main application with the readStreamUntilFound method, which returns as soon as the keyword was found:

ReadTest.java

package org.test;

import java.io.BufferedInputStream;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.Arrays;
import java.util.List;

public class ReadTest {
    public static void main(String[] args) {
        System.out.println("Result " + readStreamUntilFound(Arrays.asList("java", "-jar", "test.jar", "10000"), "Line 99"));
    }

    public static boolean readStreamUntilFound(List<String> cmd, String keyword) {
        boolean keywordFound = false;
        ProcessBuilder pb = new ProcessBuilder();
        pb.command(cmd);
        pb.redirectErrorStream(true);
        Process p;
        try {
            p = pb.start();

            try {
                BufferedInputStream bis = new BufferedInputStream(p.getInputStream());
                BufferedReader r = new BufferedReader(
                        new InputStreamReader(bis));

                String line;
                while ((line = r.readLine()) != null) {
                    System.out.println(line);
                    if (line.equals(keyword)) {
                        keywordFound = true;
                        break;
                    }
                }
            } finally {
                p.getInputStream().close();
            } 
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return keywordFound;
    }
}

I know that the InputStream of a started process must be read, otherwise the process might be blocked. Therefore I am wondering if it is problematic to close the InputStream although the console application still is running (especially in the case where a Java Process object is kept as reference).

Answer 1

To add to @ChrisWue's answer I would pull the Process away from the actual processing of it's data. IE: Make a separate class that takes a stream and does processing. Think about it this way: what is your code doing? It is taking data off of some InputStream and checking for a specific keyword. Does that processing code need a Process instance to do that? Nope! You could pass your code any InputStream and it would be able to look for a keyword. Making sure your methods are only given exactly what it needs dramatically increases testability, flexibility, and re-usability.

For the second part actually using the Process' Input/Output/Error streams. If you read the JavaDocs you will find this:

By default, the created subprocess does not have its own terminal or console. All its standard I/O (i.e. stdin, stdout, stderr) operations will be redirected to the parent process, where they can be accessed via the streams obtained using the methods getOutputStream(), getInputStream(), and getErrorStream(). The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, or even deadlock.

Basically, what this is telling me is that you need to make sure you are managing all of these streams appropriately. If you close one of the processes streams prematurely you have to be sure that the Process can handle that case. Most applications will crash or exit with an error code. Unless you know you won't break the Process you're trying to run be sure to process all the data from these streams until that process is done. You could use a simple thread which just reads and discards the data if you don't need it.

Just a thought but if you are controlling the application you are running as a Process you could possibly tell the application to close gracefully by sending it something using the OutputStream.

Answer 2

I'm not a Java coder so just some general things:

1. In your implementation as it stands you do not need the keywordFound variable. You can simply return when you have found the line:

           while ((line = r.readLine()) != null) {
               System.out.println(line);
               if (line.equals(keyword)) {
                   return true;
               }
           }
   

   And return false; at the end of the method. Returning out of a try block will execute the finally clause.

2. Using more descriptive variable names instead of p, bis or r would make the code nicer to read. It's just a good habit to get into even for short programs. There are exceptions for single letter variable names like loop counters (i, j, k) and coordinates (x, y, z) but they do not apply in your case.

3. This might just be an exercise but command line based grep implementations for Windows do exist (and are pretty much part of the standard toolset for *nix systems) and already solve this problem (looking for a pattern in the output of a program).

4. The Java documentation states:

   failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, or even deadlock

   From this I would suspect that the running process may eventually block after you have closed the input stream. There are a few options:

   1. Kill the process after you have found the line. This may have unintended consequences depending and what exactly the process is doing.
   2. Perform the reading of the process output on a background thread raising an event when the desired line has been found but continue reading until the process has terminated afterwards. This will tie the lifetime of your program to that of the called program but that might be unavoidable.