First time BDD: Testing the same things in both acceptance and unit tests

Question

I just tried BDD for the first time and implemented a simple Semantic Versioning Bumper in Python.

The class takes a version string in the format of major.minor.patch (i.e. 3.2.2, where major=3, minor=2, patch=2). Then you can bump a level, say patch, and you'd get 3.2.3. If you bump minor, you'd get 3.3.0 and if you bump major, you'd get 4.0.0.

For the BDD, I followed this workflow:

I wrote a high-level features file, then wrote the unit tests and then refactored until everything was working.

My problem is that both the unit tests and the acceptance tests are testing the same things, which doesn't feel right to me, but I don't know where to draw the line.

The code is not too complex so someone who is not too familiar with Python should be able to understand what's going on as well.

I would love to hear feedback regarding the implementation of my tests (or the actual code too, but mainly the tests).

This is the feature file:

Feature: Bump version strings

  Scenario: Increase patch version
    Given the version 0.1.2
      When we bump patch
      Then version is 0.1.3

  Scenario: Increase minor version
    Given the version 0.1.2
      When we bump minor
      Then version is 0.2.0

  Scenario: Increase major version
    Given the version 0.1.2
      When we bump major
      Then version is 1.0.0

This is the steps file:

@given('the version {version}')
def step_impl(context, version):
    context.versionbump = VersionBump(version)

@when('we bump {level}')
def step_impl(context, level):
    context.versionbump.bump(level)

@then('version is {version}')
def step_impl(context, version):
    context.versionbump.get() == version

@then('{level} is {number:d}')
def step_impl(context, level, number):
    assert context.versionbump.get(level) == number

These are the unit tests:

def test_same_version_after_parsing(vb):
    assert vb.get_version() == '2.0.1'

def test_level_access(vb):
    assert vb.get_level('major') == 2
    assert vb.get_level('minor') == 0
    assert vb.get_level('patch') == 1

def test_bump():
    vb = VersionBump('2.0.1')
    vb.bump()
    assert vb.get('patch') == 2

def test_zeroize():
    vb = VersionBump('1.2.3')
    vb.zeroize_after_level('major')
    assert vb.get_version() == '1.0.0'

def test_invalid_version():
    with pytest.raises(ValueError):
        VersionBump('2.0.1.0')

def test_get_invalid_level(vb):
    with pytest.raises(KeyError):
        vb.get_level('foo')

def test_bump_invalid_level():
    vb = VersionBump('2.0.1')
    with pytest.raises(KeyError):
        vb.bump('foo')

And here is the code:

_LEVELS = ['major', 'minor', 'patch']
_REGEX = re.compile('^(?P<major>[0-9]+)'
                    '\.(?P<minor>[0-9]+)'
                    '\.(?P<patch>[0-9]+)$')

class VersionBump(object):
    def __init__(self, version):
        self.version_info = self.parse(version)

    def bump(self, level='patch'):
        """ Bump version following semantic versioning rules. """
        self.version_info[level] += 1
        self.zeroize_after_level(level)

    def zeroize_after_level(self, base_level):
        """ Set all levels after ``base_level`` to zero. """
        index = _LEVELS.index(base_level) + 1
        for level in _LEVELS[index:]:
            self.version_info[level] = 0

    def get(self, level=None):
        """ Return complete version string if called with no parameters.
            Return value of given level if ``level`` is given.
        """
        if level:
            return self.get_level(level)
        else:
            return self.get_version()

    def get_version(self):
        """ Return complete version string. """
        version = '{major}.{minor}.{patch}'.format(**self.version_info)
        return version

    def get_level(self, level):
        """ Return value of given level. """
        return self.version_info[level]

    def parse(self, version):
        match = _REGEX.match(version)
        if match is None:
            raise ValueError('Invalid version: {}'.format(version))

        version_info = {}
        for level, number in match.groupdict().iteritems():
            if number:
                version_info[level] = int(number)
            else:
                version_info[level] = None
        return version_info

Answer 1

Your problem seems to arise due to the fact, that your project is build from only one class (= unit).

With acceptance tests you test a non-programmers view of your project, that is whether it fullfills given requirements. Acceptance tests are formulated in a style of someone who actually uses features of your program and has no insights on what is going on behind the scenes.

With unit tests you test parts inside your program, which helps you (as a programmer) to verify, that single parts of your program work as expected. You could of course test functionality which the user of your program never gets to see directly.

Since your program has only one unit, your acceptance and unit tests are similar, but formulated differently. If your program was build from different units, the tests would indeed look more different.

Answer 2

Unit Tests were created for the benefit of programmers, programmer were not created for the benefit of for unit tests.

So if writing unit tests don’t help you writing the code then don’t write unit tests for code that is tested by acceptance tests.

However unit tests help you developer the API to your class and provides examples of how to use the class, so may be of value even when you already have 100% test coverage.

On a normal sized code base, it may also be quicker to track down way a unit test if failing.