3
\$\begingroup\$

I am writing a series of unit tests in Python 3.5 unittest, which run the exact same test methods on different datasets. The purpose is to validate proper behavior of each tested function over a range of inputs at different extremes of the likely numerical range of use (large values, small values, badly scaled values, etc.). I would like to construct the TestCase so that it dynamically generates all relevant test_xxx methods for the datasets I've entered. This way it leaves me less room for typos, and avoids the need to write a raft of new functions every time I add a dataset.

I first wrote the test code with all of my dataset dict objects housed in a simple list, and a single test_the_thing function within my unittest.TestCase subclass. test_the_thing would iterate over the list of dict datasets, running the test code on each. Two primary problems arose with this approach:

  1. unittest considers the entire execution of test_the_thing to be a single test, and thus I have to search within my datasets to figure out which one failed when the test fails/errors.
  2. When any given test fails/errors in the middle of the iteration, the remainder of the tests are not run.

What I've now got in its place is the following (the code for the entire class can be perused on GitHub):

class TestOpanUtilsVectorProjRejAngle(unittest.TestCase):
    import numpy as np
    from opan.const import OpanEnum

    class DType(OpanEnum):
        V1 = 'V1'
        V2 = 'V2'
        PROJ = 'PROJ'
        REJ = 'REJ'
        ANG = 'ANG'

    class VecType(OpanEnum): # Types of vectors
        O1 = 'O1'   # Both order-one
        LOL = 'LOL' # Both large (large on large)
        SOS = 'SOS' # Both small (small on small)
        LOS = 'LOS' # Large onto small
        SOL = 'SOL' # Small onto large
        BS = 'BS'   # Badly-scaled

    class RelType(OpanEnum): # Type of vector relationship
        NS = 'NS'      # Nonspecific
        PAR = 'PAR'    # Nearly parallel
        NORM = 'NORM'  # Nearly normal
        AP = 'AP'      # Nearly anti-parallel

    namestr = "{0}_{1}"

    # Dict of dicts of data
    data = {
            # Unremarkable vectors with ~order-one components
            namestr.format(RelType.NS, VecType.O1) :
                {DType.V1: np.array([1, 2, 3]),
                 DType.V2: np.array([-1, 3, 8]),
                 DType.PROJ: np.array([-0.391892, 1.175676, 3.135135]),
                 DType.REJ: np.array([1.391892, 0.824324, -0.135135]),
                 DType.ANG: np.float_(25.712002)},
            # ... more data sub-dictionaries are included
            }

    # Template functions
    # Vector projection template
    def template_proj(self, name, data):
        from opan.utils.vector import proj

        v1 = data[self.DType.V1]
        v2 = data[self.DType.V2]
        p = proj(v1, v2)
        for i, t in enumerate(zip(p, data[self.DType.PROJ])):
            self.assertAlmostEqual(*t, delta=1e-6,
                       msg="Test {0}: Index {1}; V1 = {2}; V2 = {3}"
                            .format(name, i, v1, v2))

    # Two more template functions ...

    # Populate the local namespace with the auto-generated 
    #  test methods
    for k, d in data.items():
        # Vector projection
        fxnname = "test_Vector_Proj_Good_{0}".format(k)
        fxn = lambda self, k=k, d=d: self.template_proj(k, d)
        locals().update({fxnname: fxn})

    # Populate for the other two template methods by the same approach ...

The test suite runs as expected: if I insert artificial errors in my 'known-good' values in TestOpanUtilsVectorProjRejAngle.data, the respective tests all fail as desired.

Some questions I have include:

  • Is there a way to achieve this that is better/more secure than locals().update(...)?

  • Is the lambda the best (or only?) way to encapsulate each function for packaging into the TestOpanUtilsVectorProjRejAngle local namespace?

General recommendations on making the code cleaner, more Pythonic, etc. are also appreciated.

\$\endgroup\$
6
  • \$\begingroup\$ Can you post your project on GitHub? I am interested in reading more of this. \$\endgroup\$ Apr 25, 2016 at 16:44
  • \$\begingroup\$ I find this question very interesting but i also think it should be edited. \$\endgroup\$ Apr 25, 2016 at 16:47
  • \$\begingroup\$ @Richard_Grant The project is already on GitHub: github.com/bskinn/opan. This particular code is not yet committed to the project, however. Once I get my current tests fully converted to this paradigm, I will be committing to the v.tests branch there. I will think about how to edit my question to use the live code as cleanly as possible. \$\endgroup\$
    – hBy2Py
    Apr 25, 2016 at 17:07
  • 1
    \$\begingroup\$ I had a feeling this was a much larger project. :) I will take some time and review the entire project (it might take some time). \$\endgroup\$ Apr 25, 2016 at 17:10
  • \$\begingroup\$ @Richard_Grant You can find an alternative approach I used to streamline test population in opan/opan/test/opan_utils_inertia.py, where I created a superclass inheriting from object that defines all of the test_xxx methods, and then subclasses that multiple-inherit from the superclass and from unittest.TestCase that are actually parsed and run by unittest. \$\endgroup\$
    – hBy2Py
    Apr 25, 2016 at 17:13

1 Answer 1

2
\$\begingroup\$

I think your code is pretty good and you are following good practices with the exception to one (common mistake)

for name, datadict in data.items():
     fxnname = "test_data_{0}".format(name)
     fxn = lambda self, n=name, d=datadict: self.template_test_fxn(n, d)
     locals().update({fxnname: fxn})

Should be:

locals_ = locals()
for name, datadict in data.items():
     fxnname = "test_data_{0}".format(name)
     fxn = lambda self, n=name, d=datadict: self.template_test_fxn(n, d)
     locals_.update({fxnname: fxn})

This will prevent locals() from being evaluated when not necessary, but that's not even that big of a deal.

check out for another users use of locals https://stackoverflow.com/questions/8028708/dynamically-set-local-variable-in-python

i encourage other developers to review this, as it is a very interesting project.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.