groupsoli.blogg.se - Python runner technique

Python runner technique code#

Coupled with third-party tools they can be even more powerful. Python's built-in test discovery, reporting and running facilities are very powerful.

Python runner technique code#

I've used this technique in a number of projects over the past couple of years and found it very useful more than once, I replaced a whole complex test runner program with about 20-30 lines of code using this technique, and gained access to many more capabilities for free. Moreover, you may have a huge amount of tests and want to use tools that shard your tests for parallel execution - in this case you almost certainly need separate test cases. In general, test cases are better isolated and share less, than tests within one test case. Why would you want to generate whole test cases dynamically rather than just single tests? It all depends on your specific needs, really. FAILįile "dynamic_test_classes.py", line 8, in test Here's a sample invocation to see this in action:

In the code above, we generate a bunch of tests into a single test case. unittest defines any number of "test cases" (classes), each with any number of "tests" (methods). One interesting variation on this theme is aiming the dynamic generation at a different testing "layer". Moreover, you can build on top of any number of third-party tools for working with unittest results - HTML/XML reporting, logging, automatic CI integration, and so on. You can now invoke tests from the command line, control verbosity, control "fast fail" behavior, easily filter which tests to run and which not to run, use all kinds of assertion methods for readability and reporting (why write your own smart list comparison assertions?). unittest is powerful - armed to its teeth with useful tools for testing. So what do we gain from this, you may ask? Quite a lot. The main idea here is the dynamic test method creation. I hope it's obvious that testmap could really be test files found on disk, or whatever else.

Keep in mind that this is a very basic example.

The loop creates test functions from the data description in testmap and attaches them to the test class.

This is just a trivial template - it could do anything, or there can be multiple such "makers" fur multiple purposes.

make_test_function creates a test function (a method, to be precise) that compares its inputs.

The test class TestsContainer will contain dynamically generated test methods.

Setattr(TestsContainer, 'test_'.format(name), test_func) Test_func = make_test_function(name, params, params) LongMessage = True def make_test_function(description, a, b):įor name, params in eritems(): Import unittest class TestsContainer(unittest.TestCase): Here's a very short code snippet that can serve as a template to achieve this: Why not employ Python's existing "test runner" capabilities to do the same? I'm sure all these test runners share a lot of common infrastructure - I know that mine do. A script that looks at some directory tree, finds all the "test files" there, runs each through the transformation, compares, reports, etc. It commonly comes up when the program under test is a data-transformation mechanism of some sort (compiler, encryptor, encoder, compressor, translator etc.) The details of this are immaterial, but seasoned programmers usually encounter such testing rigs very frequently. The output of the program is compared to some "expected results" file, or maybe is encoded in the data file itself in some way. Let's assume my tests are some sort of data files which have to be fed to a program. In short, I'm actually using Python's unittest, combined with the dynamic nature of the language, to run all kinds of tests. Having written my share of such custom test runners, I've recently gravitated towards a very convenient approach which I want to share here. This is where we usually end up with a custom "test runner" script. However, unit testing does not cover all kinds of testing we may want to do - for example, all kinds of whole program tests and integration tests.

While many different kinds and levels of testing exist, there's good library support only for unit tests (the Python unittest package and its moral equivalents in other languages).