7
\$\begingroup\$

I'm trying to make a backend for QuizUp like application: user connects to a server, sends credentials and gets paired up with another user. After that server handles each pair, periodicaly sending server messages to each user in a pair and also redirecting user's mesages between them.

Server class:

private static class Server{

    private static final int NUM_THREADS = 2400;

    private ExecutorService executorService;

    private ServerSocket serverSocket; 

    private int listeningPort;

    public volatile boolean isRunning;  

    private Thread mainThread;

    private volatile Map<String, Conn> playRequests;

    public Server(int port){

        try {
            executorService = Executors.newFixedThreadPool(NUM_THREADS);
            listeningPort = port;
            serverSocket = new ServerSocket(listeningPort);
            isRunning = true;
            playRequests = new ConcurrentHashMap<String, Conn>();
            mainThread = new Thread(new Runnable(){

                @Override
                public void run() {
                    handleIncomingConnections();
                }
            });

        } catch (IOException e) {
            System.out.println(e.toString());
        }
    }

    public void run(){
        mainThread.start();
    }

    private void handleIncomingConnections(){
        while(isRunning){               
            try {
                final Socket client = serverSocket.accept();
                Runnable gameRunnable = new Runnable(){

                    @Override
                    public void run() {
                        try{
                            BufferedReader reader = new BufferedReader(new InputStreamReader(client.getInputStream()));
                            PrintWriter writer = new PrintWriter(new BufferedWriter(new OutputStreamWriter(client.getOutputStream())), true);  

                            String read = null;
                            String id = null;
                            boolean isRequesting = false;
                            String rid = null;

                            while(!(read = reader.readLine()).equals("FIN_1")){
                                String[] str = read.split("#");
                                if(str[0].equals("id")){
                                    id = str[1];
                                }else if(str[0].equals("isRequesting")){
                                    isRequesting = (str[1].equals("1"));
                                }else if(str[0].equals("rid")){
                                    rid = str[1];
                                }
                            }
                            Conn connection = new Conn(client, isRequesting, id, writer, reader);

                            if(isRequesting){
                                playRequests.put(rid, connection);
                            }else{
                                if(playRequests.containsKey(id)){
                                    Conn conn = playRequests.get(id);
                                    playRequests.remove(id);
                                    handleGame(conn, connection);
                                }
                            }                               
                        }catch(Exception e){
                            System.out.println(e.toString());
                        }
                    }                       
                };

                executorService.execute(gameRunnable);

            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }           
        }
    }

    private void handleGame(Conn a, Conn b){

        new GameHandler(a, b).execute();

    }
}

GameHandler class:

private class GameHandler{

        private volatile Conn a;
        private volatile Conn b;    

        private Thread aReadThread;
        private Thread bReadThread;
        private Thread messageThread;

        private Runnable aReadRunnable;
        private Runnable bReadRunnable;
        private Runnable messageRunnable;

        private volatile PrintWriter aWriter;
        private volatile PrintWriter bWriter;

        private volatile BufferedReader aReader;
        private volatile BufferedReader bReader;

        private volatile boolean aIsReady;
        private volatile boolean bIsReady;

        private volatile boolean isGameRunning;
        public GameHandler(final Conn s1, final Conn s2){
            this.a = s1;            
            this.b = s2;
            isGameRunning = true;
            try {
                aWriter = a.writer;
                bWriter = b.writer;
                aReader = a.reader;
                bReader = b.reader;
            } catch (Exception e) {
                try {
                    isGameRunning = false;
                    a.close();
                    b.close();
                } catch (IOException e1) {
                    // TODO Auto-generated catch block
                    e1.printStackTrace();
                }
                System.out.println(e.toString());
            }

            messageRunnable = new Runnable(){

                @Override
                public void run() {
                    System.out.println(a.id + " " + b.id);

                    messageThread = Thread.currentThread();
                    for(int i = 0; i < 6; i++){
                        if(isGameRunning){
                            try{
                                Thread.sleep(4000);
                            }catch(InterruptedException e){

                            }
                        }
                    }

                    //end game
                    isGameRunning = false;
                    try {
                        a.close();
                        b.close();
                    } catch (IOException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                    }
                }

            };

            aReadRunnable = new Runnable(){

                @Override
                public void run() {     
                    aReadThread = Thread.currentThread();
                    String line = null;
                    try {
                        while (isGameRunning && (line = aReader.readLine()) != null &&  !(line = aReader.readLine()).equals("FIN")){
                            bWriter.println(line);
                        }
                        a.close();
                        System.out.println(a.id + " done");
                    } catch (Exception e) {
                        try {
                            isGameRunning = false;
                            a.close();
                            b.close();
                        } catch (IOException e1) {
                            // TODO Auto-generated catch block
                            e1.printStackTrace();
                        }
                        System.out.println(e.toString());
                    }
                }

            };

            bReadRunnable = new Runnable(){

                @Override
                public void run() {
                    bReadThread = Thread.currentThread();
                    String line = null;
                    try {
                        while (isGameRunning && (line = bReader.readLine()) != null &&  !(line = bReader.readLine()).equals("FIN")){
                            aWriter.println(line);
                        }
                        b.close();
                        System.out.println(b.id + " done");
                    } catch (Exception e) {
                        try {
                            isGameRunning = false;
                            a.close();
                            b.close();
                        } catch (IOException e1) {
                            // TODO Auto-generated catch block
                            e1.printStackTrace();
                        }
                        System.out.println(e.toString());
                    }
                }                   
            };
        }           

        public void execute() {
            executorService.execute(messageRunnable);
            executorService.execute(aReadRunnable);
            executorService.execute(bReadRunnable);         
        }           
    } 

And a container class for each users to hold open socket, in/out streams, credentials, etc:

private class Conn{
        public Socket s;
        public boolean isRequesting;
        public PrintWriter writer;
        public String id;
        public BufferedReader reader;
        Conn(Socket s, boolean isRequesting, String id, PrintWriter writer, BufferedReader reader){
            this.s = s;
            this.isRequesting = isRequesting;
            this.id = id;   
            this.writer = writer;
            this.reader = reader;
        }

        public void close() throws IOException{
            s.close();
        }
    }

The logic is following:

Server has a mainThread, where it accepts incoming connections and creates client sockets. For each new socket it creates a gameRunnable, where it listens for client's credentials (whether this client is the one requesting connection, id of the user it wants to connect to, id of itself). After receiving credentials, server creates a new Conn object, storing all the info(id, and also socket and in/out streams, so it doesn't have to open it again after) there, and than places it in the Map (playRequests) with requested user id as a key. If there is a matching pair in a map, server creates a new GameHandler for these two Conn objects (all this still goes inside the gameRunnable). Each GameHandler contains three Runnables: messageRunnable to send messages from server to both users, and two Runnables (aReadRunnable and bReadRunnable) to read incoming data from both sockets. So basically, each communication session (game) requires 4 threads (1 to get credentials and start a game, and three to maintain the game before the end). Here are the questions I have:

  1. Are there any design/implementation issues you see here? Please be as picky as possible because I'd really not want it to crash under high load. If you see smth, you are more than welcome to give your solutions

  2. I know that having large and uncontrolled number of threads is a bad practice, so I'm using an executer with fixed thread pool to execute all the Runnables. However, due to the game features, I can't make users who are requesting connections wait for empty threads in a pool, what is obviously going to happen if I have a lot of incoming connections. So is usage of thread pool reasonable here? If yes, what number of threads should I use, given that I need 4 threads per game, and each game lasts approximately 2 minutes.

  3. Am I closing all the sockets correctly? Are there any memory leaks?

Other questions regard server deploying:

  1. I'm planning to run it on Amazon EC2. Should I use Tomcat server for this, or can I just run it as a plain java program on JVM?

  2. I tested it on my laptop, and having many simultaneous connections, heap size is not enough to handle all of them. Should I increase heap size before lunch as much as possible, or it may affect performance?

\$\endgroup\$
8
\$\begingroup\$

Blocking I/O doesn't scale well

Blocking I/O usually requires a 1:1 coupling between threads and streams. Thin clients can get away with using blocking I/O, because they're not going to have 100+ connections open. For servers with long-lasting connections, it's not workable.

You've already noticed this with your memory requirements spiking. Consider that, if a thread uses just 256KB (or even 1MB) in stack space, and you run 2400 threads, you're running 600MB (or 2400MB) just for your executor.

Enter non-blocking I/O

Java 1.4 introduced non-blocking I/O (NIO) to get around the 1:1 thread-to-stream coupling. NIO works pretty much like event handling in a GUI: you attach streams to a selector, you poll that selector for interesting events (e.g. stream has unread data, connection is ready), and then you do something with that event. Just like a GUI doesn't need a separate thread per component, NIO allows you to run a server with basically a single worker thread on your end.

Here's a quick & dirty example of something running under NIO, minus the executor and clean-up for brevity:

import java.io.*;
import java.nio.*;
import java.nio.channels.*;
import java.util.*;

public class Repeater {
  public static void main(String[] args) throws IOException {
    Selector selector = Selector.open();
    ServerSocketChannel server = ServerSocketChannel.open();
    server.configureBlocking(false);
    server.socket().bind(null);
    server.register(selector, SelectionKey.OP_ACCEPT, new AcceptHandler());
    System.out.println("Listening on " + server.socket().getInetAddress() + " @ " + server.socket().getLocalPort());

    while ( selector.select() > 0 ) {
      Iterator<SelectionKey> keys = selector.selectedKeys().iterator();
      while ( keys.hasNext() ) {
        SelectionKey key = keys.next();
        try {
          ((Handler) key.attachment()).handle(selector, key);
        } catch (Exception ex) {
          ex.printStackTrace();
          continue;
        } finally {
          keys.remove(); // [!]
        }
      }
    }
  }

  static interface Handler {
    void handle(Selector selector, SelectionKey key) throws IOException;
  }

  static final int BUFFER_SIZE_IN_BYTES = 140;
  static class AcceptHandler implements Handler {
    public void handle(Selector selector, SelectionKey key) throws IOException {
      if ( key.isAcceptable() ) {
        ServerSocketChannel server = (ServerSocketChannel) key.channel();
        SocketChannel client = server.accept();
        client.configureBlocking(false);
        client.write(ByteBuffer.wrap(("Please leave a message no longer than " + BUFFER_SIZE_IN_BYTES + " bytes.\r\n").getBytes()));

        client.register(selector, SelectionKey.OP_READ, new ReadHandler());
      }
    }
  }

  static class ReadHandler implements Handler { 
    private final ByteBuffer myStorage = ByteBuffer.allocate(BUFFER_SIZE_IN_BYTES); 
    public void handle(Selector selector, SelectionKey key) throws IOException {
      if ( key.isReadable() ) {
        SocketChannel client = (SocketChannel) key.channel();
        client.read(myStorage);
        if ( !myStorage.hasRemaining() || new String(myStorage.array(), 0, myStorage.position()).endsWith("\n") ) {
          myStorage.flip();
          client.write(myStorage);
          myStorage.clear();
        } else {
          client.register(selector, SelectionKey.OP_READ, this);
        }
      }
    }
  }
}

A quick test on a 64-bit JVM had process size on about 22MB for 4000 connections, and 36MB for 10000 connections. Actual use will be a bit more because this example is simplistic, but you get a sense of how it scales.

If you need to do serious processing before sending an answer, you can still delegate the channel handling to an executor.

...or consider asynchronous I/O

Java 7 introduced asynchronous I/O under the name NIO.2, which looks like it can do away with manually controlling the selector. I haven't used it in any serious manner yet, so I can't comment on how well it handles things or its ease of use, but it's worth taking a look to see which approach better floats your boat.

Networking libraries

I'm assuming you're also doing this as an exercise to yourself. If not, consider using Apache MINA to shield you from some of the complexity.

\$\endgroup\$
  • \$\begingroup\$ thank you for mentioning asynchronous i/o, that gives me a good starting point. \$\endgroup\$ – Thufir Jun 30 '14 at 8:38
  • \$\begingroup\$ @JvR I am also using socket to send data but in my case I need to make sure that I am not sharing same socket between two threads. I have a question here and wanted to see if you can help me out if possible. \$\endgroup\$ – david Dec 22 '17 at 7:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.