Method injection into TestClass local namespace for automatic generation of Python unittest "test_xxx" methods

Question

I am writing a series of unit tests in Python 3.5 unittest, which run the exact same test methods on different datasets. The purpose is to validate proper behavior of each tested function over a range of inputs at different extremes of the likely numerical range of use (large values, small values, badly scaled values, etc.). I would like to construct the TestCase so that it dynamically generates all relevant test_xxx methods for the datasets I've entered. This way it leaves me less room for typos, and avoids the need to write a raft of new functions every time I add a dataset.

I first wrote the test code with all of my dataset dict objects housed in a simple list, and a single test_the_thing function within my unittest.TestCase subclass. test_the_thing would iterate over the list of dict datasets, running the test code on each. Two primary problems arose with this approach:

1. unittest considers the entire execution of test_the_thing to be a single test, and thus I have to search within my datasets to figure out which one failed when the test fails/errors.
2. When any given test fails/errors in the middle of the iteration, the remainder of the tests are not run.

What I've now got in its place is the following (the code for the entire class can be perused on GitHub):

class TestOpanUtilsVectorProjRejAngle(unittest.TestCase):
    import numpy as np
    from opan.const import OpanEnum

    class DType(OpanEnum):
        V1 = 'V1'
        V2 = 'V2'
        PROJ = 'PROJ'
        REJ = 'REJ'
        ANG = 'ANG'

    class VecType(OpanEnum): # Types of vectors
        O1 = 'O1'   # Both order-one
        LOL = 'LOL' # Both large (large on large)
        SOS = 'SOS' # Both small (small on small)
        LOS = 'LOS' # Large onto small
        SOL = 'SOL' # Small onto large
        BS = 'BS'   # Badly-scaled

    class RelType(OpanEnum): # Type of vector relationship
        NS = 'NS'      # Nonspecific
        PAR = 'PAR'    # Nearly parallel
        NORM = 'NORM'  # Nearly normal
        AP = 'AP'      # Nearly anti-parallel

    namestr = "{0}_{1}"

    # Dict of dicts of data
    data = {
            # Unremarkable vectors with ~order-one components
            namestr.format(RelType.NS, VecType.O1) :
                {DType.V1: np.array([1, 2, 3]),
                 DType.V2: np.array([-1, 3, 8]),
                 DType.PROJ: np.array([-0.391892, 1.175676, 3.135135]),
                 DType.REJ: np.array([1.391892, 0.824324, -0.135135]),
                 DType.ANG: np.float_(25.712002)},
            # ... more data sub-dictionaries are included
            }

    # Template functions
    # Vector projection template
    def template_proj(self, name, data):
        from opan.utils.vector import proj

        v1 = data[self.DType.V1]
        v2 = data[self.DType.V2]
        p = proj(v1, v2)
        for i, t in enumerate(zip(p, data[self.DType.PROJ])):
            self.assertAlmostEqual(*t, delta=1e-6,
                       msg="Test {0}: Index {1}; V1 = {2}; V2 = {3}"
                            .format(name, i, v1, v2))

    # Two more template functions ...

    # Populate the local namespace with the auto-generated 
    #  test methods
    for k, d in data.items():
        # Vector projection
        fxnname = "test_Vector_Proj_Good_{0}".format(k)
        fxn = lambda self, k=k, d=d: self.template_proj(k, d)
        locals().update({fxnname: fxn})

    # Populate for the other two template methods by the same approach ...

The test suite runs as expected: if I insert artificial errors in my 'known-good' values in TestOpanUtilsVectorProjRejAngle.data, the respective tests all fail as desired.

Some questions I have include:

• Is there a way to achieve this that is better/more secure than locals().update(...)?

• Is the lambda the best (or only?) way to encapsulate each function for packaging into the TestOpanUtilsVectorProjRejAngle local namespace?

General recommendations on making the code cleaner, more Pythonic, etc. are also appreciated.

Answer 1

I think your code is pretty good and you are following good practices with the exception to one (common mistake)

for name, datadict in data.items():
     fxnname = "test_data_{0}".format(name)
     fxn = lambda self, n=name, d=datadict: self.template_test_fxn(n, d)
     locals().update({fxnname: fxn})

Should be:

locals_ = locals()
for name, datadict in data.items():
     fxnname = "test_data_{0}".format(name)
     fxn = lambda self, n=name, d=datadict: self.template_test_fxn(n, d)
     locals_.update({fxnname: fxn})

This will prevent locals() from being evaluated when not necessary, but that's not even that big of a deal.

check out for another users use of locals https://stackoverflow.com/questions/8028708/dynamically-set-local-variable-in-python

i encourage other developers to review this, as it is a very interesting project.