5
\$\begingroup\$

I'm trying to understand where my ruby implementation of this fails and hoping for an expert to shed some light on this situation. I've tried to make this a very simple example with the idea that setup is minimal.

The Problem:

I need to be able to log very quickly, ideally over UDP. Alternatively I can log to a file but I'd prefer not to for a variety of reasons but mainly logfile rotations, another dependency like a "forwarding" app, etc.

With the code below I am able to benchmark 100_000 [<= 16byte] log messages:

                            Logfile  UDP
 Time elapsed (in seconds)  1.36    24.48

Specifics on that code include: ruby 2.0.0-p353 MRI

The Logfile Setup

require 'logger'
logger = Logger.new("logfile.log")

start_time = Time.now
100_000.times do |i|
  logger.info "#{i}"
end

puts "Elapsed time was #{Time.now - start_time}"

UDP

The server (run this first):

require 'socket'
s = UDPSocket.new
s.bind(nil, 1234)
loop do
  text, sender = s.recvfrom_nonblock(16)
  # puts text
end

The client:

require 'socket'
s = UDPSocket.new
before_time = Time.now
100_000.times do |i|
  s.send("#{i}", 0, 'localhost', 1234)
end
puts "Time elapsed was #{Time.now - before_time}"

Why UDP? Well I was under the impression that UDP was a "fire and forget" protocol where the client would not wait for the server to say they received it. Fire and forget is fine for me in this scenario, I care more about performance.

Issue 1

As outlined in "The Problem" area - why is UDP so much slower than writing to a file?

Issue 2

You'll notice I have a commented out puts statement in the server. If you uncomment that, the client's benchmark approaches 40+ seconds which really makes me think the client is waiting for the server or the server is communicating with the client. Either way I'm probably not understanding UDP correctly.

\$\endgroup\$
1
  • \$\begingroup\$ TCP can also be used as fire-and-forget if you don't need/want to wait for the server reply (or if the server doesn't generate any reply) \$\endgroup\$
    – slebetman
    Apr 16 '15 at 1:50
7
\$\begingroup\$

Well I learned a lot today, specifically that writing to UDP is on average faster than writing to a file on my local machine and on my servers. My implementation on the UDP client was off.

If you look at this code:

require 'socket'
s = UDPSocket.new
before_time = Time.now
100_000.times do |i|
  s.send("#{i}", 0, 'localhost', 1234)
end
puts "Time elapsed was #{Time.now - before_time}"

There's something sneaky in there that I didn't catch with this line:

s.send("#{i}", 0, 'localhost', 1234)

When you pass in host and port into the send method it DNS resolves before each send, which has major overhead. If instead I change the code to connect ahead of time like so:

require 'socket'
s = UDPSocket.new
before_time = Time.now
s.connect('127.0.0.1', 1234)
100_000.times do |i|
  s.send("#{i}", 0)
end
puts "Time elapsed was #{Time.now - before_time}"

I get benchmarks that beat File IO. Boom.

\$\endgroup\$
1
  • 1
    \$\begingroup\$ Please note that UDP in unreliable ! This is likely very bad for most application. You may lose log entries, or receive them in any order. This can be troublesome when trying to make sense of your logs. (I know you stated that in your question, but still.) \$\endgroup\$
    – Xaqq
    Apr 15 '15 at 23:02
3
\$\begingroup\$

Regarding the first issue, all network interaction will be slower than file interaction. To write to a file, you dont need to create packets with addresses, and you dont have to have another process setup to handle this data. Network interactions are slow, and go through the OSI model http://en.wikipedia.org/wiki/OSI_model

Regarding the second issue: You're doing a nonblocking receive, and your asking for 16 bytes of data. Because this is nonblocking when you ask it for data, it will just give you that data which flushes the read. (If I understand it correctly) When you dont use the data, it will wait until it can read 16 bytes of data OR until an EOF is sent. You don't send any EOF's across from the client so the server doesn't know when to stop reading.

I may be incorrect on that last point, my socket io is mega rusty.

Its worth noting that UDP is an unreliable protocal, TCP more represents a stream and acts kind of similar to file IO, much more than UDP because there is a persistent connection. For production essential stuffs, I would recommend using TCP. See http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/

\$\endgroup\$
3
  • \$\begingroup\$ Welcome to CodeReview, Senjai. Informative answer you have here, +1. I hope you enjoy the site. \$\endgroup\$
    – Legato
    Apr 15 '15 at 17:33
  • \$\begingroup\$ Thanks again for the comment on Freenode Senja - but I was able to beat File IO writes with my new UDP implementation which was my goal. \$\endgroup\$
    – Anthony
    Apr 15 '15 at 19:06
  • \$\begingroup\$ I am not surprised that writing on the disk can be slower that writing to a network interface. UDP CPU cost is probably equivalent to the cost of file system interaction (if not less), and a disk is way slower than most network interface. Since this was over localhost, it didn't even leave memory. \$\endgroup\$
    – Xaqq
    Apr 15 '15 at 23:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.