2
\$\begingroup\$

Summary: How can I best write tests that override socket-based communication for a library, capturing what is sent over the socket, and simulating responses based on what is sent?

Details

I'm adding a new feature to ruby-mpd, and it's extensive enough that I'd like to add tests covering many scenarios.

The library communicates with MPD using a socket connection, pushing and receiving streams of information, parsing that information and presenting it helpfully.

Although MPD is stateful, I'm not interesting in testing the stateful nature. (That's up to the MPD team.) Instead, I want to test:

  1. when I invoke certain code, specific strings are sent over the socket
  2. when I invoke that code—and my test returns a reasonable response over the socket—that the results of parsing that response are what I expect

I do not want to have to instantiate an MPD server and mock song library to run these tests.

Example

For example, I need to test that this code…

ids = @mpd.command_list(:values) do
  clear
  mysongs.each{ |song| addid(song) }
end

…causes this to be sent over the socket:

command_list_begin
clear
addid foo
addid bar
addid jim
command_list_end

And then I need to simulate that after sending that it receives this over the socket…

Id: 1060
Id: 1061
Id: 1062
OK

…and that, as a result, the return value ids is [1060,1061,1062].

What I've Done

My current plan (which may not be the best one) is to replace the real socket with a delegator that intercepts puts and gets. There's a recording socket to associate streams of commands and associate them with responses. There's a playback socket that consumes this association and returns lines of responses from the previous stream of commands.

This setup does not require that sets of commands be issued in the same order, but does require that each set of commands is issued exactly the same.

Current Code

Recorder

class RecordingSocket
  def initialize(real_socket)
    @socket = real_socket
    @current = @command = []
    @responses = { @command=>[] }
  end
  def puts(*a)
    @socket.puts(*a).tap{ @command.concat(a.empty? ? [nil] : a) }
  end
  def gets
    @socket.gets.tap do |result|
      unless @command.empty?
        @current,@command=@command,[]
        @responses[@current] = []
      end
      @responses[@current] << result
    end
  end
  def save_recording(filename=nil)
    File.open(filename || "socket_recording-#{Time.now.strftime '%Y%m%d_%H%M%S'}.marshal",'wb'){ |f| f << Marshal.dump(@responses) }
  end
  def method_missing(*a)
    @socket.send(*a)
  end
end

class RecordingMPD < MPD
  def socket
    @recording_socket ||= RecordingSocket.new(super)
  end
  def save_recording(filename=nil)
    @recording_socket.save_recording(filename)
  end
end

Recording Responses

m = RecordingMPD.new('music.local').tap(&:connect)
begin
  songs = %w[test1.mp3 test2.mp3 test3.mp3]
  m.command_list do
    clear
    s.each{ |f| addid(f) }
  end
  m.queue
  m.playlists.find{ |pl| pl.name=='user-gkistner' }.songs
ensure
  m.save_recording 'mpd-responses.marshal'
end

Playback Setup

require_relative '../lib/ruby-mpd'

class PlaybackSocket
  def initialize(filename=nil)
    @responses={}
    @command = []
    load_recording(filename) if filename
  end
  def load_recording(filename)
    @responses = Marshal.load(File.open(filename,'rb',&:read))
    self
  end
  def last_messages
    @current
  end
  def puts(*a)
    @command.concat(a.empty? ? [nil] : a)
  end
  def gets
    @current,@command=@command,[] unless @command.empty?
    if @responses[@current]
      @responses[@current].shift
    else
      raise "PlaybackSocket has no (further) recording for #{@current}"
    end
  end
  def method_missing(*a)
    raise "PlaybackSocket has no support for #{a.shift}(#{a.map(&:inspect).join(', ')})"
  end
end

class PlaybackMPD < MPD
  attr_reader :socket
  def initialize( socket_recording_file=nil )
    super()
    load_recording(socket_recording_file) if socket_recording_file
  end
  def load_recording(filename)
    @socket = PlaybackSocket.new.load_recording(filename)
    self
  end
  def last_messages
    @socket.last_messages
  end
end

Running the Tests

require 'minitest/autorun'
class TestQueue < MiniTest::Unit::TestCase
  def setup
    @mpd = PlaybackMPD.new 'mpd-responses.marshal'
  end

  def test_songs
    songs = @mpd.queue
    assert_equal ["playlistinfo"], @mpd.last_messages
    assert_equal 5, songs.length
    assert songs.all?{ |value| value.is_a? MPD::Song }
    assert_equal [273, 289, 129, 258, 347], songs.map(&:track_length)
  end

  def test_command_lists
    ids = @mpd.command_list(:values) do
      clear
      %w[test1.mp3 test2.mp3 test3.mp3].each{ |f| addid(f) }
    end
    assert_equal(
      ["command_list_begin", "clear", "addid test1.mp3", "addid test2.mp3",
       "addid test3.mp3", "command_list_end"],
      @mpd.last_messages
    )
    assert_equal [101,102,103], ids
  end
end

Specific Questions

  • Is there a better way to simulate sample data from the socket?
  • Is there a better way to record the series-of-puts and associate them with the responses I get? I feel like there might be a bug lurking with that unless @command.empty? logic.
\$\endgroup\$
1
\$\begingroup\$

I've found that instead of a monolithic recording hash and inscrutable .marshal file, it's far easier to have the library write each puts/gets set as a file on disk in plain text. Binary file formats are like premature optimization.

The library uses a directory to store files in. I've made the name of the file not matter (so that you can name them sensibly), and put the commands into the file itself. While I still have a recorder library, this also makes it easy to hand-edit test data files, if you know exactly the strings that should be sent.

For example, here is the result of telnetting to the server and issuing one command (note that the server spits out one response before any commands are sent):

$ telnet 0 6600
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
OK MPD 0.19.0
status                    # here's the command I send
volume: -1
repeat: 0
random: 0
single: 0
consume: 1
playlist: 2090
playlistlength: 0
mixrampdb: 0.000000
state: stop
OK

...and here are the test files that mock this behavior:

hello.recording

--putsabove--getsbelow--
OK MPD 0.19.0

status.recording

status
--putsabove--getsbelow--
volume: -1
repeat: 0
random: 0
single: 0
consume: 1
playlist: 2090
playlistlength: 0
mixrampdb: 0.000000
state: stop
OK

Here's the library code so far:

require 'fileutils' # used for mkdir_p
require 'digest'    # used to generate unique file names

# A library for testing socket-based applications.
#
# Allows you to create a socket that records +puts+ commands
# and uses those to decide the (pre-recorded) responses to
# yield for subsequent calls to +gets+.
module SocketSpoof
  # The line in each recording file separating commands and responses
  SPLITTER = "--putsabove--getsbelow--\n"

  # Socket wrapper that generates 'recording' files consumed
  # by SocketSpoof::Player.
  #
  # To use, replace your own socket with a call to:
  #
  #     @socket = SocketSpoof::Recorder.new( real_socket )
  #
  # This will (by default) create a directory named "socket_recordings"
  # and create files within there for each sequence of +puts+ followed
  # by one or more gets.
  class Recorder
    # @param socket [Socket] The real socket to use for communication.
    # @param directory [String] The directory to store recording files in.
    def initialize(socket,directory:"socket_recordings")
      @socket   = socket
      @commands = []
      FileUtils.mkdir_p( @directory=directory )
    end
    def puts(*a)
      @socket.puts(*a).tap{ @commands.concat(a.empty? ? [nil] : a) }
    end
    def gets
      @socket.gets.tap do |response|
        unless @file && @commands.empty?
          @file = File.join( @directory, Digest::SHA256.hexdigest(@commands.inspect) )
          File.open(@file,'w'){ |f| f.puts(@commands); f<<SPLITTER }
          @commands=[]
        end
        File.open(@file,'a'){ |f| f.puts response }
      end
    end
    def method_missing(*a)
      @socket.send(*a)
    end
  end

  # Socket stand-in using files on disk to send responses.
  #
  # A SocketSpoot::Player uses a sequence of calls to +puts+ along
  # with playback files to decide what to send back when +gets+
  # is called.
  #
  # Simply replace your normal socket instance with a Player, and
  # point that player to a directory where recording files are stored.
  #
  #    @socket = SocketSpoof::Player.new( directory:'test_data' )
  #
  # The name of each recording file in the directory does not matter;
  # name them as you like to make them easier to find.
  # The format of the files must have zero or more lines of command
  # strings, followed by the +SPLITTER+ string, followed by zero or
  # more lines of response strings. For example:
  #
  #    prepare
  #    listplaylists
  #    --putsabove--getsbelow--
  #    playlist: Mix Rock Alternative Electric
  #    Last-Modified: 2015-11-23T15:58:51Z
  #    playlist: Enya-esque
  #    Last-Modified: 2015-11-18T16:19:12Z
  #    playlist: RecentNice
  #    Last-Modified: 2015-12-01T15:52:38Z
  #    playlist: Dancetown
  #    Last-Modified: 2015-11-18T16:19:26Z
  #    playlist: Piano
  #    Last-Modified: 2015-11-18T16:17:13Z
  #    OK
  #
  # With the above file in place in the directory:
  #
  #    @socket = SocketSpoof::Player.new
  #    @socket.puts "prepare"
  #    @socket.puts "listplaylists"
  #    loop do
  #      case response=@socket.gets
  #      when "OK\n",nil then puts "all done!"
  #      else                 puts response
  #    end
  #
  # ...will output all lines from the file. As with a normal
  # socket, the call to +gets+ will include a newline at the end
  # of the response.
  #
  # If your code calls +gets+ before it ever calls +puts+, you
  # will need a file with no content above the +SPLITTER+ line.
  #
  # To verify that your library sent the commands that you expected,
  # the +last_messages+ method returns an array of strings sent to
  # +puts+ since the last call to +gets+.
  class Player
    # @param directory [String] the name of the directory to find recordings in; defaults to "socket_recordings".
    # @param auto_update [Boolean] whether the directory should be rescanned (slow!) before each call to +gets+; defaults to +false+.
    def initialize(directory:"socket_recordings",auto_update:false)
      @commands = []
      FileUtils.mkdir_p( @directory=directory )
      @auto_update = auto_update
      @response_line = -1
      rescan
    end

    # Find out what messages were last sent to the socket.
    #
    # Returns an array of strings sent to +puts+ since the
    # last time +gets+ was called on the socket.
    # @return [Array<String>] messages previously sent through +puts+
    def last_messages
      @current
    end
    def puts(*a)
      @commands.concat(a.empty? ? [nil] : a)
      @response_line = -1
      nil # match the return value of IO#puts, just in case
    end
    def gets
      rescan if @auto_update
      @current,@commands=@commands,[] unless @commands.empty?
      if @responses[@current]
        @responses[@current][@response_line+=1]
      else
        raise "#{self.class} has no recording for #{@current}"
      end
    end
    def method_missing(*a)
      raise "#{self.class} has no support for #{a.shift}(#{a.map(&:inspect).join(', ')})"
    end

    private
    def rescan
      @responses = {}
      Dir[File.join(@directory,'*')].each do |file|
        commands,responses = File.open(file,'r:utf-8',&:read).split(SPLITTER,2)
        if responses
          @responses[commands.split("\n")] = responses.lines.to_a
        else
          warn "#{self.class} ignoring #{file} because it does not appear to have #{SPLITTER.inspect}."
        end
      end
    end
  end
end
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.