python - Separate test cases per input files? -


most test frameworks assume "1 test = 1 python method/function", , consider test passed when function executes without raising assertions.

i'm testing compiler-like program (a program reads *.foo files , process contents), want execute same test on many input (*.foo) files. iow, test looks like:

class test(unittest.testcase):     def one_file(self, filename):         # actual test      def list_testcases(self):         # os.listdir('tests/') , filter *.foo files.      def test_all(self):         f in self.list_testcases():             one_file(f)

my current code uses unittest python's standard library, i.e. one_file uses self.assert...(...) statements check whether test passes.

this works, in sense program succeeds/fails when code ok/buggy, i'm loosing lot of advantages of testing framework:

  • i don't relevant reporting "x failures out of y tests" nor list of passed/failed tests. (i'm planning use such system not test own development grade student's code teacher, reporting important me)

  • i don't test independence. second test runs on environment left first, , on. first failure stops testsuite: testcases coming after failure not ran @ all.

  • i feeling i'm abusing test framework: there's 1 test function automatic test discovery of unittest sounds overkill example. same code (should?) written in plain python basic assert.

an obvious alternative change code like

class test(unittest.testcase):     def one_file(self, filename):         # actual test      def test_file1(self):         one_file("first-testcase.foo")      def test_file2(self):         one_file("second-testcase.foo")

then advantages of unittest back, but:

  • it's lot more code write.

  • it's easy "forget" testcase, i.e. create test file in tests/ , forget add python test.

i can imagine solution generate 1 method per testcase dynamically (along lines of setattr(self, 'test_file' + str(n), ...)), generate code second solution without having write hand. sounds overkill use-case doesn't seem complex.

how best of both, i.e. automatic testcase discovery (list tests/*.foo files), test independence , proper reporting?

if can use pytest test runner, pretty straightforward using parametrize decorator:

import pytest, glob  all_files = glob.glob('some/path/*.foo')  @pytest.mark.parametrize('filename', all_files) def test_one_file(filename):     # actual test

this automatically name tests in useful way, can see files have failed:

$ py.test ================================== test session starts =================================== platform darwin -- python 3.6.1, pytest-3.1.3, py-1.4.34, pluggy-0.4.0 [...] ======================================== failures ======================================== _____________________________ test_one_file[some/path/a.foo] _____________________________  filename = 'some/path/a.foo'      @pytest.mark.parametrize('filename', all_files)     def test_one_file(filename): >      assert false e      assert false  test_it.py:7: assertionerror _____________________________ test_one_file[some/path/b.foo] _____________________________  filename = 'some/path/b.foo'      @pytest.mark.parametrize('filename', all_files)     def test_one_file(filename): [...]

Comments

Post a Comment

Popular posts from this blog

Is there a better way to structure post methods in Class Based Views -

i'm using class based views in django , never specify url in ajax calls , omit action parameter in form because know these requests go through post method of class based view. notice view - profile page make many requests depending on user updating on page - post method become convoluted , if/else mess example: def post(self, request, *args, **kwargs): # handle user changing available date if request.post.get('availabledate') != none: self.updateavailabledate(request) return jsonresponse({'result':'success'}) if request.post.get('newprojectname') != none: creative_user = creativeuserprofile.objects.get(id = request.user.id) project = project.create_project(creative_user, request.post['newprojectname']) project.save() return jsonresponse({'projectid': project.id }) if request.post.get('projectimage') != none , request.files['file']: proj...
Read more

What is happening when Matlab is starting a "parallel pool"? -

running parallel cpu processes in matlab starts command parpool() according documentation , function: [creates] special job on pool of workers, , [connects] matlab client parallel pool. this function takes bit of time execute, on order of 30 seconds. in other multi-cpu paradigms openmp, parallel execution seems totally transparent -- i've never noticed behavior analogous matlab (granted i'm not experienced parallel programming). so, happening between time parpool() called , when finishes executing? takes long? parallel computing toolbox enables run matlab code in parallel using several different paradigms (e.g. jobs , tasks, parfor , spmd , parfeval , batch processing), , run either locally (parallelised across cores in local machine) or remotely (parallelised across machines in cluster - either 1 own, or 1 in cloud). in of these cases, code run on matlab workers , copies of matlab without interactive desktop. if you're intending run on remo...
Read more

php - Cannot override Laravel Spark authentication with own implementation -

during login process of laravel spark need request token of remote api. api stores exact same users , passwords laravel spark application. therefore need username , non hashed password of user during authentication process. i thought overriding authenticated method solution problem. in routes/web.php override post /login endpoint , pointing endpoint own logincontroller: <?php namespace app\http\controllers\auth; use illuminate\support\facades\log; use illuminate\http\request; use laravel\spark\http\controllers\auth\logincontroller sparklogincontroller; class logincontroller extends sparklogincontroller { /** * create new login controller instance. * * @return void */ public function __construct() { parent::__construct(); } /** * handle successful authentication attempt. * * @param request $request * @param \illuminate\contracts\auth\authenticatable $user * @return response */ public fu...
Read more