I need to implement a Ring Buffer/FIFO for data coming from a TCP socket.
It must support the following operations:
- Append a the recv()'ed chunk of bytes.
- Allow me to peek at the beginning of the buffer, since i get differently-sized packets, and I must decode a small fixed-size header to know how many bytes to process.
- Remove a chunk of bytes from the beginning of the buffer for processing.
I guess my needs are pretty standard for a TCP-based streaming protocol, but surprisingly I have never found a "best practice" method for doing this.
There are similar questions on SO already, most suggest to use
collections.deque, which is fine but has some shortcomings that must be worked around for my needs:
- It doesn't easily allow peeking.
- It doesn't allow removal of chunks of bytes.
Mixing various suggestions I came up with the following implementation, which seems working but I wonder: can I do any better, performance-wise? Removing a single byte at a time in
get() doesn't look optimal at all.
import collections import itertools class RingBuffer (object): """Ring buffer""" def __init__ (self, size = 4096): self._buf = collections.deque (maxlen = size) def put (self, data): """Adds data to the end of the buffer""" self._buf.extend (data) def get (self, size): """Retrieves data from the beginning of the buffer""" data = str () for i in xrange (size): data += self._buf.popleft () return data def peek (self, size): """\"Peeks\" at the beginning of the buffer (i.e.: retrieves data without removing them from the buffer)""" return str (bytearray (itertools.islice (self._buf, size))) def len (self): """Returns the length of the buffer""" return len (self._buf)
If you are wondering why I am returning a string from
peek(), that is because I need to process its return value with