I have created an open source project that uses Rspec to validate the formatting of AWS::CloudFormation::Init YAML blocks in CloudFormation templates.1

My project is here.

Some general concerns:

  • Is this a valid use-case for using Rspec?

  • Is the documentation in the code okay?

I have some specific concerns about the code. Concerning the recursive compare() method:

# Method: compare
# Recursively compare real blocks in a CloudFormation template
# with the spec defined in $types.
# @param data [Hash] The real data in the template.
# @param spec [Hash] The reference data corresponding to this
#                    key in $types.
def compare(data, spec)
  if spec.is_a?(Hash)
    check_keys(data.keys, spec.keys)
  data.each do |k,v|
    context k do
      if spec == Array
        it "#{k} should match Array" do
          expect(v.class).to eq Array

      elsif spec[k] == Hash
        it "#{k} should match Hash" do
          expect(v.class).to eq Hash

      elsif spec == {String => Array} or
            spec == {String => String}
        it "#{k}=>#{v} should match #{spec}" do
          expect(k.class).to eq spec.keys.first
          expect(v.class).to eq spec[spec.keys.first]

      elsif spec.has_key?(k)
        if v.is_a?(Hash)
          compare(v, spec[k]) # recurse.
        elsif spec[k].is_a?(Regexp)
          it "#{k} should match #{spec[k]}" do
            expect(v).to match spec[k]
        elsif spec[k] == String

          # Actually, an Array of Strings joined by CloudFormation
          # is also okay.
          it "#{k} should match #{spec[k]}" do
            expect([String, Array].include?(v.class)).to be true
        elsif spec[k] == Fixnum
          it "#{k} should match (or be cast to) #{spec[k]}" do
            expect { v.to_i }.to_not raise_error
        elsif [Array, TrueClass, FalseClass]
          it "#{v} should be a #{spec[k]}" do
            expect(v).to be_a spec[k]

      elsif v.is_a?(Hash)
        spec_key = spec.keys.first
        compare(v, spec[spec_key]) # recurse.

        raise "Something went wrong"
  • Can this conditional logic be simplified in any way?

The compare() method compares input data against this reference spec:

module Boolean; end
class TrueClass; include Boolean; end
class FalseClass; include Boolean; end

# Supporting Ruby < 2.4.
if not defined?(Fixnum)
  class Fixnum < Integer; end

# This hash defines the expected format of
# config sets in AWS::CloudFormation::Init blocks.
# Data types and regular expressions used as
# placeholders for real values.
# Based on documentation here:
# https://docs.aws.amazon.com/AWSCloudFormation/
#   latest/UserGuide/aws-resource-init.html
$types = {
  'packages' => {
    'apt'      => {String => Array},
    'msi'      => {String => String},
    'python'   => {String => Array},
    'rpm'      => {String => String},
    'rubygems' => {String => Array},
    'yum'      => {String => Array},
  'groups' => {
    String => {'gid' => Fixnum},
  'users' => {
    String => {
      'groups'  => Array,
      'uid'     => Fixnum,
      'homeDir' => String,
  'sources' => {String => String},
  'files' => {
    String => {
      'content'   => String,
      'source'    => /^http/,
      'encoding'  => /plain|base64/,
      'group'     => String,
      'owner'     => String,
      'mode'      => Fixnum,
      'authentication' => String,
      'context'        => String,
  'commands' => {
    String => {
      'command'  => String, # TODO. Apparently the only mandatory
      'env'      => Hash,   #       attribute.
      'cwd'      => String,
      'test'     => String,
      'ignoreErrors'   => Boolean,
      'waitAfterCompletion' => Boolean,
  'services' => {
    /sysvinit|windows/ => {
      String => {
        'ensureRunning' => Boolean,
        'enabled'   => Boolean,
        'files'     => Array,
        'sources'   => Array,
        'packages'  => Array,
        'commands'  => Array,
  • Is the use of a global variable okay there? I chose that just to make it stand out.

And any general feedback most welcome.

1 There is some confusion in the comments about what I actually "implemented" - i.e. where the code-under-test is. I haven't implemented anything. I am using RSpec purely to make assertions about the formatting of YAML files that could come from anywhere. I hope that clarifies this.

  • \$\begingroup\$ I feel weird using RSpec for the final solution. What are the benefits of using RSpec to do the comparison over putting the comparison logic into classes and using RSpec to test those classes? \$\endgroup\$ – Peter Toth - Toma May 2 '19 at 10:14
  • \$\begingroup\$ @PeterToth-Toma, I'm not sure what you mean. If I understand you, then that would be "testing the tests" wouldn't it? CloudFormation is an Amazon product and I'm not the author of that. The intention here is to provide a layer of automated testing that goes beyond what CloudFormation itself can do. \$\endgroup\$ – Alex Harvey May 2 '19 at 10:17
  • \$\begingroup\$ I mean, you provide a tool that does something. But how do you make sure it works? Usually with automated tests by using a test framework like RSpec. If you create this tool only for yourself to make your life easier, then it's perfectly alright. But if you are thinking to provide it to a wider audience, then I would suggest to use RSpec to test your implementation, which is to validate config files. So I wouldn't use RSpec for implementing the validation. \$\endgroup\$ – Peter Toth - Toma May 2 '19 at 10:32
  • \$\begingroup\$ @PeterToth-Toma, I think you are misunderstanding what I have done. Did you look in my test runner? \$\endgroup\$ – Alex Harvey May 2 '19 at 11:30
  • \$\begingroup\$ Ahhha, so the test_runner.sh represents the tests for the implementation, which is in the folder for testing. Well, when I come across a project and want to know its behaviour I like to take a look at the tests. They are usually in the tests or in the rspec folder. But in our case we find the implementation there instead of the real tests. \$\endgroup\$ – Peter Toth - Toma May 2 '19 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.