python - Separate test cases per input files? -


most test frameworks assume "1 test = 1 python method/function", , consider test passed when function executes without raising assertions.

i'm testing compiler-like program (a program reads *.foo files , process contents), want execute same test on many input (*.foo) files. iow, test looks like:

class test(unittest.testcase):     def one_file(self, filename):         # actual test      def list_testcases(self):         # os.listdir('tests/') , filter *.foo files.      def test_all(self):         f in self.list_testcases():             one_file(f)

my current code uses unittest python's standard library, i.e. one_file uses self.assert...(...) statements check whether test passes.

this works, in sense program succeeds/fails when code ok/buggy, i'm loosing lot of advantages of testing framework:

  • i don't relevant reporting "x failures out of y tests" nor list of passed/failed tests. (i'm planning use such system not test own development grade student's code teacher, reporting important me)

  • i don't test independence. second test runs on environment left first, , on. first failure stops testsuite: testcases coming after failure not ran @ all.

  • i feeling i'm abusing test framework: there's 1 test function automatic test discovery of unittest sounds overkill example. same code (should?) written in plain python basic assert.

an obvious alternative change code like

class test(unittest.testcase):     def one_file(self, filename):         # actual test      def test_file1(self):         one_file("first-testcase.foo")      def test_file2(self):         one_file("second-testcase.foo")

then advantages of unittest back, but:

  • it's lot more code write.

  • it's easy "forget" testcase, i.e. create test file in tests/ , forget add python test.

i can imagine solution generate 1 method per testcase dynamically (along lines of setattr(self, 'test_file' + str(n), ...)), generate code second solution without having write hand. sounds overkill use-case doesn't seem complex.

how best of both, i.e. automatic testcase discovery (list tests/*.foo files), test independence , proper reporting?

if can use pytest test runner, pretty straightforward using parametrize decorator:

import pytest, glob  all_files = glob.glob('some/path/*.foo')  @pytest.mark.parametrize('filename', all_files) def test_one_file(filename):     # actual test

this automatically name tests in useful way, can see files have failed:

$ py.test ================================== test session starts =================================== platform darwin -- python 3.6.1, pytest-3.1.3, py-1.4.34, pluggy-0.4.0 [...] ======================================== failures ======================================== _____________________________ test_one_file[some/path/a.foo] _____________________________  filename = 'some/path/a.foo'      @pytest.mark.parametrize('filename', all_files)     def test_one_file(filename): >      assert false e      assert false  test_it.py:7: assertionerror _____________________________ test_one_file[some/path/b.foo] _____________________________  filename = 'some/path/b.foo'      @pytest.mark.parametrize('filename', all_files)     def test_one_file(filename): [...]

Comments

Post a Comment

Popular posts from this blog

Is there a better way to structure post methods in Class Based Views -

i'm using class based views in django , never specify url in ajax calls , omit action parameter in form because know these requests go through post method of class based view. notice view - profile page make many requests depending on user updating on page - post method become convoluted , if/else mess example: def post(self, request, *args, **kwargs): # handle user changing available date if request.post.get('availabledate') != none: self.updateavailabledate(request) return jsonresponse({'result':'success'}) if request.post.get('newprojectname') != none: creative_user = creativeuserprofile.objects.get(id = request.user.id) project = project.create_project(creative_user, request.post['newprojectname']) project.save() return jsonresponse({'projectid': project.id }) if request.post.get('projectimage') != none , request.files['file']: proj...
Read more

performance - Why is XCHG reg, reg a 3 micro-op instruction on modern Intel architectures? -

i'm doing micro-optimization on performance critical part of code , came across sequence of instructions (in at&t syntax): add %rax, %rbx mov %rdx, %rax mov %rbx, %rdx i thought had use case xchg allow me shave instruction , write: add %rbx, %rax xchg %rax, %rdx however, dimay found agner fog's instruction tables , xchg 3 micro-op instruction 2 cycle latency on sandy bridge, ivy bridge, broadwell, haswell , skylake. 3 whole micro-ops , 2 cycles of latency! 3 micro-ops throws off 4-1-1-1 cadence , 2 cycle latency makes worse original in best case since last 2 instructions in original might execute in parallel. now... cpu might breaking instruction micro-ops equivalent to: mov %rax, %tmp mov %rdx, %rax mov %tmp, %rdx where tmp anonymous internal register , suppose last 2 micro-ops run in parallel latency 2 cycles. given register renaming occurs on these micro-architectures, though, doesn't make sense me done way. why wouldn't register renamer ...
Read more

c# - Asp.net web api : redirect unauthorized requst to forbidden page -

im trying redirect unauthorized request forbidden page instead i'm getting forbidden page in response body , how can fix ? here's startup class : app.createperowincontext(storecontext.create); app.createperowincontext<applicationusermanager>(applicationusermanager.create); app.createperowincontext<applicationsigninmanager>(applicationsigninmanager.create); app.usecookieauthentication(new cookieauthenticationoptions { authenticationtype = defaultauthenticationtypes.applicationcookie, expiretimespan = timespan.fromdays(30), }); app.useexternalsignincookie(defaultauthenticationtypes.externalcookie); app.usetwofactorsignincookie(defaultauthenticationtypes.twofactorcookie, timespan.fromminutes(5)); app.usetwofactorrememberbrowsercookie(defaultauthenticationtypes.twofactorrememberbrowsercookie); the method im trying reach : [httpget] [authorize(roles = "admin")] public string getcurrentusername() { return userman...
Read more