1
\$\begingroup\$

I write an app server that uses TCP socket on Linux. When there is no traffic (no data is sent by client, no client connect() or close()), the process sleeps on epoll_wait() while waiting for events hit the socket file descriptors.

What is a good thing to do while the process is sleeping?

So my initiative is to make the sleep time short and force the process to read from memory again and again until the events come.

Reason for doing this is that to keep critical (for performance) data hot in the cache.

  • Is that a worthwhile thing to do?
  • Or better I let the process sleeping until events come?

My understanding

While the process is sleeping on epoll_wait() for too long, the kernel will schedule to run other processes.

If my app is scheduled away for too long, then its data on memory will be evicted from the cache, as cache is shared across multiple processes, then other processes' data will take place in the cache.

epoll_wait documentation

Relevant Part of Code (event_loop and exec_epoll_wait)

struct srv_tcp_state {
    int                 epoll_fd;
    int                 tcp_fd;
    int                 tun_fd;

    bool                stop;
    struct_pad(0, 3);

    struct cl_slot_stk  client_stack;
    struct srv_cfg      *cfg;
    struct client_slot  *clients;
    uint16_t            *epoll_map;

    /*
     * We only support maximum of CIDR /16 number of clients.
     * So this will be `uint16_t [256][256]`
     */
    uint16_t            (*ip_map)[256];

    /* Counters */
    uint32_t            read_tun_c;
    uint32_t            write_tun_c;

    struct bc_arr       bc_arr_ct;
    utsrv_pkt_t         send_buf;
    struct iface_cfg    siff;
    bool                need_iface_down;
    bool                aff_ok;
    struct_pad(1, 4);
    cpu_set_t           aff;
};


static int exec_epoll_wait(int epoll_fd, struct epoll_event *events,
                           int maxevents, struct srv_tcp_state *state)
{
    int err;
    int retval;
    int timeout = 50; /* in milliseconds */

    retval = epoll_wait(epoll_fd, events, maxevents, timeout);
    if (unlikely(retval == 0)) {
        /*
         * epoll_wait() reaches timeout
         *
         * TODO: Do something meaningful here.
         */

        /*
         * Force the process to read critical data so
         * it is always hot (at least in L2 or L3?)
         */
        memcmp_explicit(state, state, sizeof(*state));
        return 0;
    }

    if (unlikely(retval < 0)) {
        err = errno;
        if (err == EINTR) {
            retval = 0;
            prl_notice(0, "Interrupted!");
            return 0;
        }

        pr_err("epoll_wait(): " PRERF, PREAR(err));
        return -err;
    }

    return retval;
}


static int event_loop(struct srv_tcp_state *state)
{
    int retval = 0;
    int maxevents = 64;
    int epoll_fd = state->epoll_fd;
    struct epoll_event events[64];

    /* Shut the valgrind up! */
    memset(events, 0, sizeof(events));

    while (likely(!state->stop)) {
        retval = exec_epoll_wait(epoll_fd, events, maxevents, state);

        if (unlikely(retval == 0))
            continue;

        if (unlikely(retval < 0))
            goto out;

        retval = handle_events(state, events, retval);
        if (unlikely(retval < 0))
            goto out;
    }

out:
    return retval;
}

memcmp_explicit

This code is located in different file, this prevents compiler to inline or optimize or remove the memcmp call (which happens when epoll_wait reaches its timeout).

int memcmp_explicit(const void *s1, const void *s2, size_t n)
{
    return memcmp(s1, s2, n);
}

likely and unlikely macros

#define likely(EXPR)   __builtin_expect(!!(EXPR), 1)
#define unlikely(EXPR) __builtin_expect(!!(EXPR), 0)

About __builtin_expect

\$\endgroup\$
4
  • 1
    \$\begingroup\$ At least 2 functions are missing from the code, likely() and unlikely(). In addition the function exec_epoll_wait() contains TODO in the comments. In the Code Review Community we review finished code and need all referenced functions to be able to do a good review. Please read How do I ask a good question. I look forward to you updating the question so that I can do a review. \$\endgroup\$
    – pacmaninbw
    Mar 15, 2021 at 17:57
  • \$\begingroup\$ [memcmp_explicit()] prevents compiler to inline or optimize or remove the memcmp - does it, without separate compilation and/or cheating on, say const correctness? \$\endgroup\$
    – greybeard
    Mar 15, 2021 at 21:24
  • \$\begingroup\$ @pacmaninbw oh I missed that, they are actually a macro, I edited the question. \$\endgroup\$ Mar 15, 2021 at 22:13
  • \$\begingroup\$ @greybeard I don't understand, could you elaborate? (The compilation of memcmp_explicit is separated from that event loop, anyway). \$\endgroup\$ Mar 15, 2021 at 22:20

1 Answer 1

1
\$\begingroup\$

Keeping the cache hot

So my initiative is to make the sleep time short and force the process to read from memory again and again until the events come.

Reason for doing this is that to keep critical (for performance) data hot in the cache.

If nothing else running on the same CPU, there is no reason for the cache to go cold. Things in the cache don't time out, they just stay there until they are evicted when necessary.

If there are other processes running, then your strategy might help your socket server, but there are several issues with this:

  • It is selfish; you might prevent other processes from keeping their data in the cache, thus lowering their performance.
  • It might not work at all, since once another process gets a time slice, their memory access patterns may evict your data from the cache again.
  • By not staying idle, your CPU might not get a chance to go into a low power state, thus keeping your cache literally hot.

My recommendation is to just use an infinite timeout for epoll_wait().

About memcmp_explicit()

Be aware that compilers are getting better and better at optimizing things. Even if you put memcmp_explicit() in a different translation unit than where it is called, the compiler might optimize it out if link time optimization is enabled.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.