6
\$\begingroup\$

I wrote a Java NIO Echo Server and want to maximize the number of connections to the server.

The problem is that when I try to connect more than 10k clients the clients get their connections refused:

error Connection Refused : no further information available.

The client program that I use generates a batch of 50 clients per 5 seconds and each client sends data to the server every 5 seconds.

I have added a backlog queue parameter to the server and want to review he server code and find any flaws which may impact scaling.

Server

public class Server implements Runnable  {

public final static String ADDRESS = "192.168.1.3";
public final static int PORT = 8511;
public final static long TIMEOUT = 10000;
public int clients;
ByteBuffer readBuffer = ByteBuffer.allocate(1024);
private ServerSocketChannel serverChannel;
private Selector selector;

private Map<SocketChannel,byte[]> dataTracking = new HashMap<SocketChannel, byte[]>();

public Server(){
    init();
}

private void init(){
    System.out.println("initializing server");

    if (selector != null) return;
    if (serverChannel != null) return;

    try {

        selector = Selector.open();
        serverChannel = ServerSocketChannel.open();
        serverChannel.configureBlocking(false);
        serverChannel.socket().bind(new InetSocketAddress(ADDRESS, PORT),5000);
        serverChannel.register(selector, SelectionKey.OP_ACCEPT);

    } catch (IOException e) {
        e.printStackTrace();
    }
}

@Override
public void run() {
    System.out.println("Now accepting connections...");
    try{
        while (!Thread.currentThread().isInterrupted()){

            int ready = selector.select();
            if(ready==0)
                continue;
            Iterator<SelectionKey> keys = selector.selectedKeys().iterator();

            while (keys.hasNext()){
                SelectionKey key = keys.next();
                keys.remove();
                if (!key.isValid()){
                    continue;
                }

                if (key.isAcceptable()){
                    System.out.println("Accepting connection");

                    accept(key);
                }

                if (key.isWritable()){
                    System.out.println("Writing...");

                    write(key);
                }

                if (key.isReadable()){
                    System.out.println("Reading connection");

                    read(key);
                }
            }
        }
    } catch (IOException e){
        e.printStackTrace();
    } finally{
        closeConnection();
    }

}

private void write(SelectionKey key) throws IOException{

    SocketChannel channel = (SocketChannel) key.channel();
    byte[] data = dataTracking.get(channel);
    dataTracking.remove(channel);
    int count = channel.write(ByteBuffer.wrap(data));
    if(count == 0)
    {
        key.interestOps(SelectionKey.OP_WRITE);
        return;
    }
    else if(count > 0)
    {
        key.interestOps(0);
        key.interestOps(SelectionKey.OP_READ);  
    }

}

private void closeConnection(){

    System.out.println("Closing server down");
    if (selector != null){
        try {
            selector.close();
            serverChannel.socket().close();
            serverChannel.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

private void accept(SelectionKey key) throws IOException
{
    ServerSocketChannel serverSocketChannel = (ServerSocketChannel) key.channel();
    SocketChannel socketChannel = serverSocketChannel.accept();
    if(socketChannel == null)
    {
        throw new IOException();
    }
    socketChannel.configureBlocking(false);
     clients++;
    //socketChannel.register(selector, SelectionKey.OP_WRITE|SelectionKey.OP_READ);
    SelectionKey skey = socketChannel.register(selector, SelectionKey.OP_READ);

    byte[] hello = new String("Hello from server").getBytes();
    dataTracking.put(socketChannel, hello);

    try
    {
        write(skey);
    }
    catch(IOException e)
    {
        System.out.println("Problem in initial hello from Server  "+e);
    }
}

private void read(SelectionKey key) throws IOException
{
    SocketChannel channel = (SocketChannel) key.channel();
    readBuffer.clear();
    int length;
    try {
        length = channel.read(readBuffer);
    } catch (IOException e) {
        System.out.println("Reading problem, closing connection");
        System.out.println("No of clients :"+clients);
        key.cancel();
        channel.close();
        return;
    }
    if (length == -1){
        System.out.println("Nothing was there to be read, closing connection");
        channel.close();
        key.cancel();
        return;
    }

    readBuffer.flip();
    byte[] data = new byte[1000];
    readBuffer.get(data, 0, length);
    String fromclient = new String(data,0,length,"UTF-8");
    System.out.println("Received: "+fromclient);
    String dat = fromclient+channel.getRemoteAddress();
    data= dat.getBytes();
    echo(key,data);
}

private void echo(SelectionKey key, byte[] data) throws IOException
{
    SocketChannel socketChannel = (SocketChannel) key.channel();
    dataTracking.put(socketChannel, data);
    //key.interestOps(SelectionKey.OP_WRITE);
    try
    {
        write(key);
    }
    catch(IOException e)
    {
        System.out.println("Problem in echo "+e);
        e.printStackTrace();
    }
}

public static void main(String [] args)
{
    Thread serv = new Thread(new Server());
    serv.start();
}
}
\$\endgroup\$
  • \$\begingroup\$ This looks like a "why isn't my programming working?" Not a proper code review. Isn't it a requirement that code should be working? You should ask this on StackOverflow. \$\endgroup\$ – Stephen C Jun 8 '15 at 7:49
  • 1
    \$\begingroup\$ @StephenC It's an edge-case, but I don't think this is off-topic for Code Review. Technically this code works just fine, but he wants to find scalability flaws. This isn't much different from people asking for improved speed. The code seems to work, so it's not broken code. It's not about adding features either, since it's already capable of handling many clients. Just not enough to the OP's liking. \$\endgroup\$ – Mast Jun 8 '15 at 8:13
  • \$\begingroup\$ @StephenC : The code is working but I am not able to scale above 10 k clients. \$\endgroup\$ – cruxion effux Jun 8 '15 at 9:00
  • \$\begingroup\$ That may be a OS enforced limitation to not allow more than 10k connections to a single program. \$\endgroup\$ – ratchet freak Jun 8 '15 at 10:40
  • \$\begingroup\$ @ratchetfreak : I don't think there is such a limitation. Also when I try to connect lets say 12k clients , it is that 11k gets connected and rest 1k fails.So its just not a fixed number. Its like upto 10k I am able to service all clients successfully. \$\endgroup\$ – cruxion effux Jun 8 '15 at 11:11
9
\$\begingroup\$

There are a number of things in here which could be a problem, and a number of other things which are a problem....

Ephemerals

First up, you may just be running in to resource limitations. A TCP based computer has just 64K ports available for sockets. The first 1K are reserved for root, and the remainder are available for programs. You may think you have > 60K ports you can connect from, but, if your client program is all running from one computer, then each client instance will have its own source port. These source ports are allocated from the ephemeral range. You may be limited to just 15K ports or so.... This is close enough to your listed limit to be significant.

Can you push things further if you run multiple client programs from multiple computers?

Performance

Next, as I understand it, every 5 seconds you get an additional 50 connections, so, it takes > 15 minutes to get to 10,000 connections.

At that point, every connection is sending a message every 5 seconds. So, 15 minutes in, you are getting 2000 messages per second.

That's not particularly large, but, let's consider what your code does:

  • it does a System.out.println(...) !!!!

That's a real problem. Println's are slow, and will likely be a bottleneck for you. Remove them.

You are also dedicating just one thread to service the entire socket. This should be enough for a fairly high volume of echo's, but it's not huge. Your echo process does in fact do a fair amount of work. A lot of byte[] buffer creation, etc. which generates a fair amount of garbage.

I would consider monitoring your garbage collection to see if you have long GC cycles. A long cycle may cause a backlog of connections to accumulate.

Speaking of that, why do you do this for every connection?

byte[] hello = new String("Hello from server").getBytes();

That should be stored as a static-final constant. No need to rebuild it (and collect it) each time.

NIO

The NIO component, at face value, looks OK. I worry that you have only one thread. I know that this is often listed as being OK for NIO, but I would recommend a small thread pool for handling the active sockets. Each active socket is handled in a new thread. One thread is plenty for handling the scheduling of the sockets, but adding the latency that the computation involves will likely limit your throughput (especially the printlns.).

\$\endgroup\$
0
\$\begingroup\$

For your main query, your OS limits including file handles, max open, etc, ulimit -a for Unix/Mac, see details in this link (search fs.file-max) about that & more settings for Linux, https://mrotaru.wordpress.com/2015/05/20/how-migratorydata-solved-the-c10m-problem-10-million-concurrent-connections-on-a-single-commodity-server/

Additionally, there are couple of bug in the write path of code,

  1. if write fails due to channel being full (written bytes = 0), next OP_WRITE will not find the bytebuffer in map
byte[] data = dataTracking.get(channel);
dataTracking.remove(channel);
int count = channel.write(ByteBuffer.wrap(data));
if(count == 0)
{
    key.interestOps(SelectionKey.OP_WRITE);
    return;
}

It needs to be reread

  1. write can be successful but not all data has been written, the check should be for size of buffer.
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.