From d02709446da07d9c571282d6a37c46b6a5bf1e83 Mon Sep 17 00:00:00 2001 From: AutomatedTester Date: Fri, 1 Dec 2017 13:25:14 +0000 Subject: [PATCH] Squashed 'tools/pytest/' changes from 168daaa71f..369c711f14 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 369c711f14 Merge pull request #2971 from blueyed/fix-ZeroDivisionError a9dd37f429 Merge pull request #2980 from nicoddemus/immutable-fix-parameters 4de433e280 Merge pull request #2983 from nicoddemus/improve-assert-2979 70f1e3b4b0 Improve getscopeitem assertion message fdfc1946da Add CHANGELOG entry about pytest.fixture "params" being now immutable 88ed1ab648 Merge pull request #2964 from rpuntaie/master 191e8c6d9b Merge pull request #2969 from nicoddemus/null-bytes-2957 6bbd741039 Add test for #2971 0f5fb7ed05 Fix ZeroDivisionError with 0 collected tests 5f1a7330b2 Small fixes in changelog items 2a75ae46c3 Improve test that blocks setuptools plugins from being loaded 89cf943e04 Always escape null bytes when setting PYTEST_CURRENT_TEST 833f33fa0c removed comments 3dbac17d75 Merge branch 'master' of https://github.com/rpuntaie/pytest 6843d45c51 added test for #2920 fix 4a840a7c09 Fix formatting in CHANGELOG 9b7e4ab0c6 prepare pull request for #2920 fix 4ea7bbc197 fix issue #2920 4d2f05e4b9 Merge pull request #2962 from pytest-dev/fix-nbsp-in-changelog 454b60b6c5 Merge pull request #2963 from redtoad/patch-1 f6be23b68b Add changelog entry. 644fdc5237 Fix broken link to project 4b5f0d5ffa replace non-breaking space with space 9f7ba00611 Merge pull request #2958 from nicoddemus/issue-2956 796db80ca4 Only escape str-like arguments passed to warnings d95c8a2204 Fix link to #2636 in CHANGELOG 4678cbeb91 Merge remote-tracking branch 'upstream/features' 67ad0fa364 Merge pull request #2945 from nicoddemus/release-3.3.0 6cdd851227 CHANGELOG and docs changes requested during review c58715371c Merge pull request #2954 from blueyed/rewritten d5f038e29a Fix spelling: s/re-writ/rewrit/g 6eeacaba3e Fix typos in CHANGELOG 6b90ad4d4b Merge pull request #2949 from eprikazc/master e273f5399d Update github "bugs" link 0de1a65644 Update pluggy pin to pluggy>=0.5,<0.7 after pluggy-0.6 release 95de11a44e Block pytest-catchlog and issue a warning 05cfdcc8cb Revert making TerminalWriter public in TerminalReporter plugin 0ddd3e2839 Fix linting in CHANGELOG aa9a02ec44 Preparing release version 3.3.0 e97c774f8e Remove vendoring import from tasks/__init__ f50ace7c0a Merge remote-tracking branch 'upstream/master' into release-3.3.0 49c0c599b0 Merge pull request #2944 from nicoddemus/regendoc-norm e0d236c031 Remove gendoc normalization for '=' and '_' headers b533c2600a Merge pull request #2858 from nicoddemus/console-progress-2657 dc574c60ef Use regex match-count syntax to improve test readability 1d26f3730f Fix docstrings in pytester 27935ebec9 Merge pull request #2943 from Perlence/fix-marks-without-description 378eb5d67b Minor formatting change in CHANGELOG 5e71ffab87 Handle marks without description 8df7ed12c1 Merge pull request #2940 from nicoddemus/rewrite-bug-2939 f05333ab75 Fix rewrite to not write past the edge of the screen c8d52b633b Fix assertion rewrite to match module names correctly 2455f8670e Add changelog 3a5dbabf60 Add tests for progress output and docs for ``console_output_style`` 3441084bd2 Add progress output for verbose mode with xdist 8b92527d7d Add re_match_lines and re_match_lines_random to pytester fixture dab889304e Implement progress percentage reporting while running tests 7a7cb8c8c5 Require py>=1.5.0 77bd0aa02f Merge pull request #2931 from nicoddemus/deprecation-roadmap-docs b0f558da44 Add Future/Past Releases section ca1f4bc537 Merge pull request #2935 from asottile/capsysbinary 219b758949 Add capsysbinary fixture 6161bcff6e Merge pull request #2925 from asottile/capfdbinary 99a4a93dbc Merge pull request #2932 from nicoddemus/deprecate-add-call 99ba3c9700 Add CHANGELOG entry for #2876 c19708b193 Merge pull request #2930 from nicoddemus/pin-pluggy-0.5 1f08d990d5 Deprecate metafunc.addcall e2c59d3282 Change pluggy._CallOutcome to pluggy._Result after update to pluggy-0.5 f9029f11af Add Deprecation Roadmap to backward compatibility document c7be83ac47 Update pluggy pin to 0.5.* 74aaf91653 Merge pull request #2927 from nicoddemus/release-3.2.5 1aeb58b531 Merge pull request #2928 from nicoddemus/python-requires 7b3febd314 Merge pull request #2924 from nicoddemus/merge-master-into-features e87ff07370 Prevent pip from installing pytest in unsupported Python versions a220a40350 Preparing release version 3.2.5 dd6c534468 Remove py<1.5 restriction 4a0aea2deb Add missing entry to CHANGELOG for 3.2.4 8f90812481 Add capfdbinary fixture 3b3bf9f53d Merge remote-tracking branch 'upstream/master' into merge-master-into-features 54cea3d178 Small formatting changes in CHANGELOG 9628c71210 Merge pull request #2921 from nicoddemus/release-3.2.4 a0ad9e31da Preparing release version 3.2.4 22cff038f8 Merge pull request #2916 from nicoddemus/bin-py-1.5 685387a43e Merge pull request #2127 from malinoff/fix-2124 d26c1e3ad9 Pin py<1.5 as 1.5 drops py26 and py33 support a6f2d2d2c9 Rename FixtureDef.finalizer to FixtureDef.finalizers 6d3fe0b826 Explicitly clear finalizers list in finalize to ensure cleanup bdad345f99 Fix passing request to finish() in FixtureDef 063335a715 Add changelog entries for #2124 f074fd9ac6 Merge remote-tracking branch 'upstream/features' into malinoff/fix-2124 6550b9911b pytest_fixture_post_finalizer now receives a request argument 258031afe5 Merge remote-tracking branch 'upstream/master' into malinoff/fix-2124 259b86b6ab Merge pull request #2776 from cryporchild/fix-missing-nodeid-with-pyargs f0f2d2b861 Merge branch 'master' into fix-missing-nodeid-with-pyargs d1af369800 Merge pull request #2913 from nicoddemus/merge-master-into-features b671c5a8bf Merge pull request #2914 from nicoddemus/addfinalizer-refactor f320686fe0 Make SubRequest.addfinalizer an explicit method 742f9cb825 Merge pull request #2911 from RonnyPfannschmidt/remove-nodeinfo 99496d9e5b Merge pull request #2910 from RonnyPfannschmidt/scopenode-sanitize-specials 983a09a2d4 Merge remote-tracking branch 'upstream/master' into merge-master-into-features 66fbebfc26 Merge pull request #2894 from nicoddemus/fix-linting-errors 0108f262b1 Fix typo in CHANGELOG 76f3be452a remove unused _pytest.runner.NodeInfo class c47dcaa713 switch a special case in scope node lookup to a general one b2b1eb262f Merge pull request #2906 from nicoddemus/larger-parametrize-section 9fce430c89 Merge pull request #2907 from nicoddemus/lazy-fixture e114feb458 Mention pytest-lazy-fixture plugin in the proposal for parametrize_with_fixtures c09f69df2a Make the "examples" section more prominent in the docs 3900879a5c Mark test_py2_unicode as xfail in PyPy2 on Windows 7b1cc55add Merge pull request #2903 from pagles/docs-3.6 d904981bf3 Rename 2903.trivial to 2903.doc f13333afce Create changelog entry fad1fbe381 List python 3.6 in the documented supported versions c33074c8b9 Merge pull request #2641 from RonnyPfannschmidt/introduce-attrs b11640c1eb Fix linting E722: do not use bare except 03829fde8a Fix linting E741: ambiguous variable name e351976ef4 Merge remote-tracking branch 'upstream/features' into RonnyPfannschmidt/introduce-attrs b18a9deb4c Merge pull request #2892 from nicoddemus/merge-master-into-features 2e2f72156a Merge pull request #2893 from fmichea/patch-1 22e9b006da Add fragment per PR's guidelines. 802585cb66 Clarify language of proposal for parametrized fixtures d7e8eeef56 Merge pull request #2878 from RonnyPfannschmidt/collector-makeitem-deprecate e58e8faf47 Add CHANGELOG entry for attrs module dependency 7d43225c36 Merge pull request #2877 from RonnyPfannschmidt/extract-extract-parameterset 460cae02b0 Small formatting fix in CHANGELOG f3a119c06a Merge upstream/master into features d1aa553f73 add mocked integrationtest for the deprecationwarning of makeitem cd747c48a4 Merge pull request #2874 from nicoddemus/fix-py27-xdist-envs-2843 07b2b18a01 introduce attrs as dependency and use it 766de67392 Fix linting error in deprecated.py 821f9a94d8 deprecate the public internal PyCollector.makeitem method 26019b33f8 Merge pull request #2882 from thisch/doctest_lineno cb30848e5a Merge pull request #2880 from samueldg/capture-result-namedtuple d00e2da6e9 Merge pull request #2881 from dawran6/2658 2f993af54a Fix context output handling for doctests af5e9238c8 Document pytest.param 8e178e9f9b Add myself to AUTHORS list 8e28815d44 Add changelog entry for issue #2879 b27dde24d6 Use a nametuple for `readouterr()` results 4a436f2255 move responsibility for parameterset extraction into parameterset class 27cea340f3 Remove trailing whitespace c3ba9225ef Change directory for py27 xdist-related envs 111d640bdb Merge pull request #2873 from stephenfin/doc/skipping-entire-files 734c435d00 Merge pull request #2870 from Perlence/rewrite-python-37-docstring 27bb2eceb4 Add comment about why we remove docstrings on test_assertrewrite 383239cafc doc: Include collector config in the skip doc fd7bfa30d0 Put imports on the last line unless there are other exprs 3427d27d5a Try to get docstring from module node def471b975 Merge pull request #2869 from nicoddemus/merge-master-into-features f743e95cfc Merge pull request #2791 from OfirOshir/features 4e581b637f Use zip and map from six 6b86b0dbfe Fix additional linting issues 0b540f98b1 Merge pull request #2864 from bilderbuchi/fix-1505 bdab29fa3d Merge pull request #2867 from Perlence/ini-markers-whitespace 6821d36ca5 Merge remote-tracking branch 'upstream/master' into merge-master-into-features 5631a86296 Merge pull request #2862 from tom-dalton-fanduel/issue-2836-fixture-collection-bug 52aadcd7c1 Strip whitespace from markers in INI config f5e72d2f5f Unused import / lint a5ac19cc5e Merge branch 'issue-2836-fixture-collection-bug' of github.com:tom-dalton-fanduel/pytest into issue-2836-fixture-collection-bug 14e3a5fcb9 Move the generic separator to a constant 7b608f976d Merge pull request #2865 from nicoddemus/contributing-docs fe560b7192 Make CONTRIBUTING and PR template more consistent regarding doc contributions b61cbc4fba Merge pull request #2859 from OwenTuz/issue-2692-document-setup-teardown-mismatch-in-unittest-integration a3ec3df0c8 Add E722 and E741 flake errors to the ignore list e23af009f9 Introduce a dedicated section about conftest.py 531e0dcaa3 Merge pull request #2863 from lancelote/patch-1 dc5f33ba5c Remove typo @ in assignment 655ab0bf8b Address more review comments, fix massive bug I reintroduced in the node-splitting code :-/ a7199fa8ab Docstring typo d714c196a5 Shorter code, longer docstring ee7e1c94d2 Remove redundant if, tidy if-body de9d116a49 Added Tom Dalton to AUTHORS f003914d4b Add changelog entry for #2836 1e6dc6f8e5 Working (I think) fix for #2836 c03612f729 Test now looks for real expected output 29fa9d5bff Add failing test 3cdbb1854f #2692: Document setup/teardown behaviour when using unittest-based suites 083084fcbc Merge pull request #2842 from ceridwen/features f7387e45ea Fix linting 3da28067f3 Replace introspection in compat.getfuncargnames() with inspect/funcsigs.signature 5c71151967 Add more text to the 2.6 and 3.3 announcement 3f9f4be070 Merge pull request #2845 from jespino/fix/2832 4cb60dac3d Merge pull request #2850 from bilderbuchi/docs-911 8c7974af01 Merge pull request #2848 from bilderbuchi/fix-538 e81b275eda Update formatting in CHANGELOG 537fc3c315 Merge pull request #2824 from dirk-thomas/pytest_addopts_before_initini 46cc9ab77c Add documentation about python -m pytest invocation. 2d08005039 Merge pull request #2847 from meawoppl/patch-1 baadd569e8 Clarify the documentation of fixture scopes. Closes #538. 11b391ff49 Update mark.py 00d3abe6dc Adding Failed exception to manage maxfail behavior 71c76d96d3 Merge pull request #2834 from aysonje/ignore-setup 3676da594c Merge pull request #2841 from pgiraud/patch-1 843872b501 Improve formatting in 502.feature file a4fd5cdcb5 Fix auto-use fixture doc ae4e596b31 Merge pull request #2840 from MarSoft/patch-2 cfdebb3ba4 Fix typo in parametrization doc eaf38c7239 call path.read(), add tests, add news fragment b29a9711c4 ignore valid setup.py during --doctest-modules c750a5beec Merge pull request #2794 from thisch/catchlog df37cdf51f Merge pull request #2808 from georgeyk/allow-module-level-skip af75ca435b Fix some coding-style issues in the logging plugin 8aed5fecd9 Remove test_logging_initialized_in_test f3261d9418 Move logging docu into own rst file 775f4a6f2f Fix flake8 issue 502652ff02 Add preliminary documentation for logging-plugin 0e83511d6d Rename name of registered logging plugin 815dd19fb4 Remove unicode literal compat code 1f3ab118fa Remove usage of get_logger_obj 0ec72d0745 Improve get_option_ini and get_actual_log_level 69f3bd8336 Add changelog entry for catchlog plugin 10a3b9118b Use a relative cache_dir in test because of how arguments are parsed on Windows ce8c829945 add test for #2824 ed7aa074aa add changelog file for #2824 66e9a79472 get PYTEST_ADDOPTS before calling _initini 1480aed781 Merge pull request #2823 from hugovk/features-rm-2.6 be0e2132b7 Update authors [CI skip] 7113c76f0d Remove unused import ef732fc51d Remove code for unsupported Python versions dd45f8ba6c Merge pull request #2822 from RonnyPfannschmidt/nomore-py26 c486598440 remove some support code for old python versions 059455b45d Merge pull request #2773 from RonnyPfannschmidt/fix-markeval-2767 73ff53c742 remove eol python from the ci config 88366b393c start the removal of python 2.6/3.3 support 9b0ce535c9 Merge pull request #2801 from nicoddemus/capture-fixture 8a6bdb282f fix changelog entry 46e30435eb Merge pull request #2819 from leezu/fix_kwargs_fixtures e86ba41a32 Add testcase for #2819 e89abe6a40 Defensive fallback in case of kwargs not being present 48b5c13f73 Add changelog for #2819 c24ffa3b4c Fix pytest.parametrize when argnames are specified as kwarg 459cc40192 skipping: cleanup e3b73682b2 flake8 fix 8480075f01 resuffle markevaluator internal structure 9ad2b75038 skipping: replace _evalskip with a more consistent _skipped_by_mark a33650953a remove unused import 667e70f555 switch out the placeholder MarkEvaluator in unittest plugin 761d552814 Merge pull request #2815 from xuanluong/issue-1997-document-xpass 4bc6ecb8a5 Add mention of xpass in skip/xfail documentation 0668a6c6d3 Add myself to authors file 03ce0adb79 Fix: handle CollectReport in folded_skips function e7a4d3d8cf Merge remote-tracking branch 'upstream/master' into features 9ee0a1f5c3 Merge pull request #2813 from nicoddemus/release-3.2.3 6b91bc88de Preparing release version 3.2.3 61eb20df71 Merge pull request #2810 from nicoddemus/issue-2809 df6d5cd4e7 Use ascii_escaped to escape unicode warnings fbb9e9328b Fix warning about non-ascii warnings even when they are ascii 9824499396 Add 2808.feature changelog entry 59f66933cd Update documentation example of pytest.skip(allow_module_level=True) c1aa63c0bb Fix docstring alignment and typos e4a6e52b81 Update skipping documentation to include usage of allow_module_level kwarg 06307be15d Add initial tests using skip with allow_module_level kwarg 79d3353081 Add allow_module_level kwarg to skip helper 6690b8a444 Merge pull request #2807 from xuanluong/issue-1442-mention-not--k-help 7093d8f65e Add example of -k 'not test' in help text f9589f7b64 Resume output capturing after capsys/capfd.disabled() context manager 794d4585d3 Remove unnecessary complexity in _check_initialpaths_for_relpath(). d132c502e6 Merge pull request #2804 from nicoddemus/terminal-reporter-tw-2803 3b30c93f73 Deprecate TerminalReporter._tw c0c859ce99 Merge pull request #2799 from blueyed/cleanup-tox.ini 22f338d74d Refactor some names for better understanding and consistency 9919269ed0 Allow to use capsys and capfd in other fixtures 87596714bf minor: cleanup tox.ini 296ac5c476 Add thisch to AUTHORS ad21d5cac4 Remove pytest-capturelog backward compat code 2559ec8bdb use 'formatter' kwarg of catching_logs 207f153ec1 Remove logging_at_level ctx manager 3a4011585f catching_logs: Remove usage of 'closing' ctx manager 57f66a455a catching_logs: Remove unused 'filter' kwarg e41fd52e8c Introduce live_logs context manager 08f6b5f4ea Use pytest.hookimpl instead of pytest.mark.hookwrapper d13e17cf51 Don't modify the 'config' object in __init__ f1f6109255 Remove _catchlog_ prefix 87b8dc5afb Move 'config' handling from pytest_configure to __init__ fc965c1dc5 Remove outdated docstring a1bd54e4ea Clean-up LogCaptureHandler 36cceeb10e Set type of log_print ini-variable to 'bool' 3e71a50403 Remove unneeded sys import from unittest 98209e92ee Remove superfluous whitespace in docstring 1bea7e6985 Cleanup pytest_addoption 1ba219e0da Adapt (logging) unittest a8e3effb6c Upgrade py ca46f4fe2a Remove conftest 5130f5707f Fix name clash 6607478b23 Add unittests for LoggingPlugin (excluding perf tests) 8eafbd05ca Merge the pytest-catchlog plugin de0d19ca09 Merge pull request #2790 from nicoddemus/merge-master-into-features d96869ff66 fixing cr 966391c77e Merge pull request #2789 from Avira/master 9c8847a0cb Merge pull request #2792 from Avira/fix-tox-docs-link 58aaabbb10 fix tox documentation link b57a84d065 updating bugfix changelog c89827b9f2 updating import plugin error test in order to make sure it also checks that the original traceback has been shown to the users 062a0e3e68 If an exception happens while loading a plugin, PyTest no longer hides the original traceback. In python2 it will show the original traceback with a new message that explains in which plugin. In python3 it will show 2 canonized exceptions, the original exception while loading the plugin in addition to an exception that PyTest throws about loading a plugin. 2802135741 fix 'DoctestItem' object has no attribute '_fixtureinfo' a2da5a691a Update tox and appveyor environments to use py36 by default afe7966683 Fix call to outcome.get_result now that outcome.result is deprecated 3ebfb881c9 Merge remote-tracking branch 'upstream/master' into features bf77daa2ee Merge pull request #2785 from nicoddemus/py36 9933635cf7 Change to py36 as main environment for Python 3 environments in tox ac5c5cc1ef Merge pull request #2750 from evanunderscore/fix-filescompleter 810320f591 Small fixes to development_guide: title and label names 25d2acbdb2 Merge pull request #2760 from nicoddemus/labels 52c134aed3 Add development guide to docs 14b6380e5f Fix #2775 - running pytest with "--pyargs" will result in Items with empty "parent.nodeid" if run from a different root directory 70cdfaf661 Merge pull request #2754 from nicoddemus/xfail_strict abfd9774ef Remove xfail mark from passing test in py26 e57cc55719 Merge pull request #2766 from xuanluong/issue-1548-docs-multiple-calls-metafunc-parametrize 696c702da7 Update documentation on multiple calls of metafunc.parametrize bee2c864d8 Merge pull request #2765 from xuanluong/pass-callable-to-marker-typo e27a0d69aa Rename changelog file to PR id number 15222ceca2 Fix typo in example of passing a callable to markers 3c1ca03b9c Merge pull request #2763 from jhamman/docs/skipif_class_warning 25ed4edbc7 Merge pull request #2764 from xuanluong/issue-2758-fix-mark-decorator-equality 1e93089165 [bugfix] Checking MarkDecorator equality returns False for non-MarkDecorator object b2a8e06e4f add warning to skipping docs re marker inheritance 9273e11f21 Merge branch 'master' into features 09349c344e Merge pull request #2757 from nicoddemus/release-3.2.2 6cf515b164 Fix crash in FastFilesCompleter with no prefix 6967f3070e Merge pull request #2711 from massich/mimic_raises_signature_in_warns a0c6758202 Add changelog 80d165475b Add documentation aa6a67044f Add match_regex functionality to warns c52f87ede3 Preparing release version 3.2.2 549f5c1a47 Merge pull request #2736 from xuanluong/issue-2604-documents-mark.with_args 3f8ff7f090 [DOC] Add examples for mark.with_args b55a4f805f Merge pull request #2744 from nicoddemus/pluggy-master d01f08e96f Merge branch 'features' into pluggy-master ad36407747 Merge pull request #2700 from nicoddemus/staticmethods-fixtures e1f2254fc2 Merge pull request #2734 from RonnyPfannschmidt/simplify-string-safening 10d43bd3bf Set xfail_strict=True in pytest's own test suite f825b4979b Merge remote-tracking branch 'upstream/master' into features 3d70727021 Improve wording in changelog entry 1fc185b640 Add comment about possible future refactoring in the fixture mechanism 7d59b2e350 Fix call to outcome.force_result d9992558fc Refactor tox.ini so pluggymaster envs share definitions d1f71b0575 Merge pull request #2752 from tarcisiofischer/issue-2751-fix-flaky-testdir de6b41e318 Update changelog and AUTHORS files, following the CONTRIBUTING guidelines 8d1903fed3 Avoid creating arbitrary filenames for tmpdir on Testdir's constructor 13eac944ae restore ascii escaping for python 3.3/3.4 d8ecca5ebd Add test to design warns signature in TDD mimicking raises signature 9bbf14d0f6 Create explicit 'pluggymaster' env definitions c42d966a40 Change all pytest report options to the more concise '-ra' in tox.ini 3dc0da9339 Remove __multicall__ warning and usages in testing 11ec6aeafb Add test environment using pluggy from master branch 9d373d83ac Merge pull request #2741 from nicoddemus/pytester-makepyfile 181bd60bf9 Merge pull request #2742 from nicoddemus/resultlog-deprecation 3288c9a110 Improve user guidance regarding ``--resultlog`` deprecation 221797c609 Encode utf-8 byte strings in pytester's makefile 5e00549ecc Merge pull request #2735 from fgmacedo/fgm-fix-reprfuncargs-toterminal b770a32dc8 Merge pull request #2707 from cybergrind/fix_baseexception f9157b1b6b Improve CHANGELOG entry to be more user-friendly f4e811afc0 Merge remote-tracking branch 'upstream/master' into cybergrind/fix_baseexception 59cdef92be fixes #2731 ReprFuncArgs with mixed unicode and utf-8 args 78a027e128 simplyfy ascii escaping by using backslashreplace error handling 709b8b65a4 Merge pull request #2721 from josepht/patch-1 0824076e11 Merge pull request #2710 from massich/raises_match_doc 488bbd2aeb Merge pull request #2719 from tgoodlet/stop_vendoring_pluggy 312891daa6 Add a trivial changelog entry fe415e3ff8 Use latest patch release ff35c17ecf Drop wrapper module; import directly 67161ee9f8 Add changelog item for PR #2721. 1c891d7d97 Fix typo in goodpractices.rst 9ab83083d1 Update docs 756db2131f Drop vendoring from packaging cb700208e8 Drop vendoring task 333a9ad7fa Stop vendoring pluggy 12b1bff6c5 `compat.safe_getattr` now catches OutcomeExceptions too 657976e98a update raises documentation regarding regex match a993add783 Allow tests declared as @staticmethod to use fixtures 539523cfee Merge pull request #2697 from nicoddemus/match-kw-version f18780ed8a Update docs: ``match`` keyword was introduced in 3.1 806d47b4d4 Merge pull request #2691 from anhiga/trivial_error bfc9f61482 Update the number of plugins in index.rst 2a99d82c3b Fixed error in 'Good Practices' code snippet 5c0feb2877 Merge pull request #2680 from prokaktus/skipping-same-module 9b2753b302 Merge pull request #2687 from nicoddemus/use-py36-ci 5f17caa156 Merge pull request #2675 from RonnyPfannschmidt/mark-callspeck e9bfccdf2d Merge pull request #2678 from jespino/fix/2676 7b5d26c1a8 Use py36 as preferred Python 3 interpreter for CI testing 98bf5fc9be Fold skipped tests with global pytestmark variable 362b1b3c4f Use tox release candidates in CI 5c0c1977e3 Merge pull request #2682 from pelme/getfuncargvalue-fix 39331856ed Use the correct stacklevel for getfuncargvalue() deprecation warning. dc9154e8ff Add default values documentation for python_files, python_classes and python_functions 021fba4e84 Update number of plugins on README and poiint to plugincompat link eb462582af fix #2675 - store marks correctly in callspecs fd84c886ee Merge pull request #2671 from nicoddemus/release-3.2.1 e6020781f6 Merge pull request #2653 from felipedau/slow-sharing-note acd3c4fbc4 Update changelog for #2653 c847b83d56 Use `pytest_collection_modifyitems()` in the run/skip option example 45d2962e97 Preparing release version 3.2.1 8b322afcdb Make generated doc in simple.rst more reliable 523bfa6151 Merge pull request #2667 from nicoddemus/py36-windows-workaround-error cc0f2473eb Fix windows console workaround error with non-standard io-streams 76c55b31c6 Merge pull request #2630 from srinivasreddy/2591 a0101f024e remove os.sep as it behaves differently linux and windows. d5f4496bdf Merge pull request #2656 from nicoddemus/unittest-features 37353a854e Implement suggestions by Raphael 12e60956de Merge pull request #2655 from nicoddemus/terminal-collecting-glitch 15cdf137d5 Document which pytest features work with `unittest` 9e62a31b63 Merge pull request #2650 from srinivasreddy/2642 2e33d9b35e Add changelog entry for using six for portability dc563e4954 convert py module references to six module ad52f714a9 Fix small terminal glitch when collecting a single test item 8969bd43c9 Merge pull request #2646 from nicoddemus/issue-2644 7703dc921c Only skip null bytes before setting the environment variable 1deac2e210 Properly escape test names when setting PYTEST_CURRENT_TEST environment variable 02da156351 Merge pull request #2645 from alex/patch-1 84061233ef Tiny rst syntax fix 40254b64e5 Merge branch 'master' into features 0a15edd573 Merge branch 'release-3.2.0' 51ebad76f2 Fix merge instruction after a minor/major release d2bca93109 Update grammar in changelog as requested 6e7547244b Merge pull request #2636 from RonnyPfannschmidt/remove-preinit 333ec8ba5a Merge pull request #2638 from RonnyPfannschmidt/function-definition dcaeef7c10 take review comments into account 8a2e6a8d51 Fix linting 74d536314f pytester: make pytest fullpath a constant ceb016514b remove dead code - Node._memoizedcall e90f876b34 remove the last own implementation of pytest_namespace c68a89b4a7 remove preinit, its no longer needed 07dd1ca7b8 Preparing release version 3.2.0 f1467f8f03 Merge remote-tracking branch 'upstream/master' into features 763c580a2a Merge pull request #2576 from maiksensi/feat/raise-not-implemented-for-lt-gt-in-approx e1aed8cb17 Merge pull request #2490 from RonnyPfannschmidt/fix-580 713f7636e1 Merge pull request #2632 from jmoldow/pep_0415_suppress_exception_context 4cd8727379 Merge pull request #2617 from wence-/fix/nondeterministic-fixtures 5e0e038fec Merge pull request #2625 from nicoddemus/historical-notes 2e61f702c0 Support PEP-415's Exception.__suppress_context__ 8c2319168a Rephrase the bit about unittest migration to pytest 768edde899 Merge pull request #2624 from nicoddemus/lf-report-after-collection be401bc2f8 fix linting issues 06a49338b2 make Test Outcomes inherit from BaseException instead of exception 7a12acb6a1 Fix linting 5acb64be90 Add versionadded tag to pytest_report_collectionfinish hook 75e6f7717c Use new hook to report accurate tests skipped in --lf and --ff 17121960b4 Merge pull request #2621 from nicoddemus/cumulative-cache 7082320f3f Apply modifications requested in review 6fe7069cbb Move historical notes to their own doc d46006f791 Fixes in "contact" doc f770f16294 Replace deprecated "config.option." usages from docs eb1bd3449e xfail and skipped tests are removed from the "last-failed" cache 22212c4d61 Add xfail specific tests 62810f61b2 Make cache plugin always remember failed tests e97fd5ec55 Merge pull request #2623 from nicoddemus/post-collection-report-hook 17c544e793 Introduce new pytest_report_collectionfinish hook ddf1751e6d Merge pull request #2613 from nicoddemus/features 3d89905114 Merge remote-tracking branch 'upstream/master' into features 1a9bc141a5 Merge pull request #2620 from mihaic/multiple-issues-in-changelog 10ded399d8 Show multiple issue links in CHANGELOG entries f047e078e2 Mention new (py26) ordereddict dependency in changelog and docs f8bd693f83 Add ordereddict to install_requires for py26 a546a612bd Fix nondeterminism in fixture collection order dd294aafb3 Merge pull request #2557 from blueyed/EncodedFile-name b39f957b88 Add test of issue #920 2c2cf81d0a Merge pull request #2580 from andras-tim/fix-runpytest-subprocess 80f4699572 approx raises TypeError in Python 2 for comparison operators other than != and == 57a232fc5a Remove out of scope change 1851f36beb Add PR requirements changelog and authors f0936d42fb Fix linter errors d3ab1b9df4 Add user documentation 0603d1d500 capture: ensure name of EncodedFile being a string 79097e84e2 Merge pull request #2615 from blueyed/revisit_contributing.rst 1a42e26586 Improve changelog wording to be more user-oriented 595ecd23fd Merge pull request #2548 from blueyed/skip-fix-lineno 949a1406f0 Revisit CONTRIBUTING.rst 71947cb4f0 Merge pull request #2546 from blueyed/better-skip-not-allowed-error 869eed9898 Fix lineno offset in show_skipped 72531f30c0 Improve error message for CollectError with skip/skipif 73c6122f35 Merge remote-tracking branch 'upstream/master' into features 70d9f8638f Merge pull request #2610 from AgriConnect/doctest-lineno d40d77432c Add test case for DoctestItem.reportinfo() e44284c125 Merge pull request #2611 from segevfiner/patch-1 dea671f8ba Add changelog for #2610 cdaa720bc4 Merge remote-tracking branch 'upstream/master' into doctest-lineno d0ecfdf00f Delete trailing whitespace 81ad185f0d Merge pull request #2595 from nicoddemus/docs-rootdir-pythonpath d90bef44cc Update changelog file for #2510 df12500661 Create 2611.bugfix 43544a431c Early import colorama so that it get's the correct terminal 0aa2480e6a Fix travis build after change from "precise" to "trusty" 6473a81b8b Merge pull request #2608 from nicoddemus/contributing-update af2c153324 Report lineno from doctest 309152d9fd Merge pull request #2599 from nicoddemus/turn-warnings-into-errors d5bb2004f9 Fix travis build after change from "precise" to "trusty" bda07d8b27 Ignore socket warnings on windows for trial tests 0726d9a09f Turn warnings into errors in pytest's own test suite 61219da0e2 Update PR guide and add a "short" version 1b732fe361 Merge pull request #2606 from kalekundert/simplify-numpy b35554ca2b Merge pull request #2607 from kalekundert/remove-code-dup 7e0553267d Remove unused import. ebc7346be4 Raise TypeError for types that can't be compared to arrays. a3b35e1c4b Remove `raises` and `approx` from `python.py`. 4c45bc9971 Add the numpy tests back into tox.ini 495f731760 Simplify how comparisons with numpy arrays work. 50764d9ebb Avoid interactive pdb when pytest tests itself - fix #2023 6461dc9fc6 Merge pull request #2600 from RonnyPfannschmidt/mark_explicit_params 1cf826624e Merge pull request #2602 from blueyed/doc-fix-filterwarnings 97e5a3c889 Fix help for filterwarnings ini option 65b2de13a3 fix #2540, introduce mark.with_args 3d24485cae Clarify PYTHONPATH changes and ``rootdir`` roles ccc4b3a501 Merge pull request #2596 from nicoddemus/autopep-tox cbceef2008 Merge pull request #2598 from nicoddemus/filterwarnings-mark 7341da1bc1 Introduce pytest.mark.filterwarnings 3c28a8ec1a Improve formatting/grammar of changelog entries 22f54784c2 Add "fix-lint" tox environment to fix linting errors abb5d20841 Merge branch 'master' into features da12c52347 Fix: do not load hypothesis during test_logging_initialized_in_test 9e3e58af60 Merge pull request #2594 from nicoddemus/fix-flake8-errors 56e6b4b501 Merge pull request #2578 from Llandy3d/2375 d44565f385 Merge remote-tracking branch 'upstream/master' into fix-flake8-errors 24da938321 Fix additional flake8 errors 26ee2355d9 Merge remote-tracking branch 'upstream/features' into fix-flake8-errors c92760dca8 Merge branch 'fix-flake8-issues' 61d4345ea4 Merge pull request #2593 from hackebrot/extend-pytester-docs-for-testing-plugin-code 1ac02b8a3b Add plugin code d06d97a7ac Remove unnecessary comma from docs e73a2f7ad9 Add changelog entry changelog/971.doc 91b4b229aa Update documentation for testing plugin code 2840634c2c Fix typo and improve comment about cookiecutter-template eb79fa7825 Merge pull request #2590 from nicoddemus/current-test-var d7f182ac4f Remove SETUPTOOLS_SCM_PRETEND_VERSION during linting 2d4f1f022e Introduce PYTEST_CURRENT_TEST environment variable 2c03000b96 Merge pull request #2585 from RonnyPfannschmidt/fix-2573 62556bada6 remove the MARK_INFO_ATTRIBUTE warning until we can fix internal usage 637e566d05 Separate all options for running/selecting tests into sections 3a1c9c0e45 Clarify in the docs how PYTEST_ADDOPTS and addopts ini option work together 0e559c978f Merge pull request #2587 from nicoddemus/remove-impl-file bd96b0aabc Remove _pytest/impl file 7b1870a94e Fix flake8 in features branch 4fd92ef9ba Merge branch 'fix-flake8-issues' into features ac3f2207bb Merge pull request #2575 from MartinAltmayer/master b2a5ec3b94 updated meta b49e8baab3 Fixed E731 flake8 errors 15610289ac Fixed E712 flake8 errors 5ae59279f4 Fixed E704 flake8 errors bf259d3c93 Fixed E702 flake8 errors 85141a419f Fixed E701 flake8 errors 7d2ceb7872 Fixed E501 flake8 errors b9e318866e Fixed E402 flake8 errors 45ac863069 Fixed E401 flake8 errors 7248b759e8 Fixed E303 flake8 errors b840622819 Fixed E302 flake8 errors 17a21d540b Fixed E301 flake8 errors 9bad9b53d8 Fixed E293 flake8 errors 4730c6d99d Fixed E272 flake8 errors c9a081d1a3 Fixed E271 flake8 errors 195a816522 Fixed E265 flake8 errors eae8b41b07 Fixed E262 flake8 errors 8f3eb6dfc7 Fixed E261 flake8 errors b226454582 Fixed E251 flake8 errors 4c24947785 Fixed E241 flake8 errors 617e510b6e Fixed E231 flake8 errors 4b22f270a3 Fixed E226 flake8 errors 2e8caefcab Fixed E225 flake8 errors 3fabc4d219 Fixed E222 flake8 errors f640e0cb04 Fixed E221 flake8 errors ebb6d0650b Fixed E203 flake8 errors ba0a4d0b2e Fixed E202 flake8 errors 1ff54ba205 Fixed E201 flake8 errors df54bf0db5 Fixed E131 flake8 errors 1c935db571 Fixed E129 flake8 errors cf97159009 Fixed E128 flake8 errors 57438f3efe Fixed E127 flake8 errors e855a79dd4 Fixed E126 flake8 errors 92e2cd9c68 Fixed E125 flake8 errors 051d76a63f Fixed E124 flake8 errors 4b20b9d8d9 Fixed E123 flake8 errors 425665cf25 Fixed E122 flake8 errors 0be97624b7 Fixed E121 flake8 errors 64a4b9058c Fixed E113 flake8 errors 8de49e8742 Fixed E111 flake8 errors 6146ac97d9 Fixed E101 flake8 errors 6af2abdb53 Fixed flake8 warnings 796ffa5123 reformatted tox.ini ba9a76fdb3 Provides encoding attribute on CaptureIO cc39f41c53 Add myself to AUTHORS as required by the PR help text. 2a979797ef Add a changelog entry. e5169a026a #2574: --fixtures, --fixtures-per-test keep indentation of docstring 3578f4e405 Merge pull request #2571 from ahartoto/master 97fdc9a7fe Ensure final collected line doesn't include artifacts 771cedd3da Merge pull request #2567 from nicoddemus/add-report-section-docs 81cec9f5e3 Merge pull request #2563 from pv/yield-warn-spam 1485a3a902 Merge pull request #2566 from jmsdvl/iss2518 f16c3b9568 Merge pull request #2569 from nicoddemus/backwards-sidebar e6b9a81ccf Add a link to our backwards compatibility policy to our side-bar 67fca04050 update docs and note; add virtualenv collection tests 73b07e1439 Add docs for Item.add_report_section in the docs b32cfc88da use presence of activate script rather than sys.prefix to determine if a dir is a virtualenv 676c4f970d trim trailing ws c2d49e39a2 add news item 89c73582ca ignore the active python installation unless told otherwise d9aaab7ab2 Merge remote-tracking branch 'upstream/master' into features 9e0b19cce2 Merge pull request #2561 from jmsdvl/iss2533 a87f6f84cc Merge pull request #2559 from RockBomber/features 8a7d98fed9 Make YIELD_TEST warning less spammy cdd788085d add news fragment to changelog folder 80595115b0 replace all _escape_strings to _ascii_escaped bd52eebab4 changelog for ini option cache_dir 91418eda3b docs for ini option cache_dir 7a9fc69435 tests for ini option cache_dir f471eef661 ini option cache_dir ef62b86335 Merge pull request #2492 from kalekundert/features 7cd03d7611 Merge pull request #2554 from nicoddemus/pytest-configure-order 3667086acc Clarify pytest_configure hook call order db24a3b0fb Merge pull request #2552 from nicoddemus/strict-help-message 221f42c5ce Change --strict help message to clarify it deals with unregistered markers 7a1a439049 Use `cls` instead of `ApproxNumpyBase`. b62aef3372 Merge branch 'master' into features c111e9dac3 Avoid making multiple ApproxNumpy types. 8524a57075 Add "approx" to all the repr-strings. b63f6770a1 Preparing release version 3.1.3 8a8687122d Add wheel to tasks/requirements.txt: required for package creation 7277fbdb20 Fix SMTP port in fixture docs 6908d93ba1 Merge pull request #2475 from ant31/master c578418791 Add changelog for triple leading '/' problem. 0303d95a53 Merge pull request #2531 from waisbrot/staticmethods 9b9fede5be allow staticmethods to be detected as test functions 9b51fc646c Merge pull request #2526 from nicoddemus/merge-master-into-features 6eeab45a8f Merge pull request #2525 from nicoddemus/deprecate-old-style 16df4da1f7 Fix exclude_path check 3de93657bd Merge remote-tracking branch 'upstream/master' into merge-master-into-features 1906f8c565 Merge pull request #2524 from RonnyPfannschmidt/fix-2427 655d44b413 Add changelog entry explicitly deprecating old-style classes from pytest API 0d0b01bded introduce deprecation warnings for legacy parametersets, fixes #2427 6e2b5a3f1b Merge pull request #2523 from RonnyPfannschmidt/vendoring-tasks b3bf7fc496 add tasks for updating vendored libs bb659fcffe Merge pull request #2521 from nicoddemus/show-trivial-changelog 6de19ab7ba Show "trivial" category in CHANGELOG bab18e10eb Merge pull request #2517 from RonnyPfannschmidt/mark-expose-nontransfered 8d5f2872d3 minor code style fix b0b6c355f7 fixup changelog, thanks Bruno 23d016f114 address review comments 22b7701431 Merge pull request #2480 from nicoddemus/issue-2469-deprecated-call-ctx 1d926011a4 add deprecation warnings for using markinfo attributes ff8dbd0ad8 Add tracebackhide to function call form of deprecated_call 5e832017d5 Merge pull request #2487 from nicoddemus/recursion-error-2486 c791895c93 changelog addition 19b12b22e7 store pristine marks on function.pytestmark 64ae6ae25d extract application of marks and legacy markinfos bdec2c8f9e move marker transfer to _pytest.mark 4a62102b57 Merge pull request #2511 from nicoddemus/addfinalizer-docs f2ba8d70b9 Fix typo and add suggestion from review afe847ecdc fixture docs: highlight difference between yield and addfinalizer methods 9597e674d9 Use sets to compare dictionary keys. d6000e5ab1 Remove py36 from .travis.yml 4d02863b16 Remove a dict-comprehension. 5d2496862a Only test numpy with py27 and py35. 50769557e8 Skip the numpy doctests. b41852c93b Use `autofunction` to document approx. 8badb47db6 Implement suggestions from code review. 8c3c4307db Merge pull request #2503 from nicoddemus/remove-manifest-check 731c35fcab Remove MANIFEST.in and related lint check 31b971d79d Merge pull request #2488 from nicoddemus/issue-links-changelog 4e57a39067 Merge pull request #2468 from nicoddemus/collection-report-2464 af0344e940 Merge pull request #2500 from nicoddemus/issue-2434-doctest-modules 97367cf773 Remove obsolete comment from rewrite.py 336cf3e1f5 Merge pull request #2496 from rmfitzpatrick/pytest2440_handle_subrequest_finalizer_exceptions 4e4ebbef5a Improve test to ensure the expected function is re-raised b09d60c60a Fix decode error in Python 2 for doctests in docstrings 0908f40e43 Merge pull request #2499 from hackebrot/update-license-dates 0e73724e58 Add changelog/2499.trivial 9970dea8c1 Update copyright date in doc pages 218af42325 Update copyright date in LICENSE and README.rst 6fa7b16482 Merge pull request #2497 from pkch/firstresult 4a992bafdb Changelog 21137cf8c5 Add firstresult=True to the hook docs 5a856b6e29 handle and reraise subrequest finalizer exceptions 89292f08dc Add a changelog entry. 8c22aee256 Resolve merge conflict due to approx being moved. 9f3122fec6 Add support for numpy arrays (and dicts) to approx. 9bd8907716 Merge pull request #2489 from RonnyPfannschmidt/move-python-api f8b2277413 changelog fragment 6be57a3711 move python api helpers out of the python module 36251e0db4 move approx to own file f0541b685b Improve CHANGELOG formatting a bit 536f1723ac Add issue links in the CHANGELOG entries 8bb589fc5d Fix internal error when trying to detect the start of a recursive traceback. 467c526307 Merge remote-tracking branch 'upstream/master' into features b2d7c26d80 Merge pull request #2483 from nicoddemus/release-3.1.2 7cbf265bb5 Preparing release version 3.1.2 917b9a8352 Merge pull request #2476 from nicoddemus/fix-2459-numpy-comparison 2127a2378a Fix internal error with recursive tracebacks with that frames contain objects that can't be compared d2db6626cf Merge pull request #2466 from nicoddemus/remove-unicode-warning 620ba5971f deprecated_call context manager captures warnings already raised c67bf9d82a Merge remote-tracking branch 'upstream/master' into features 57e2ced969 Merge pull request #2473 from ApaDoctor/docs-fixes 80944e32ad Add CHANGELOG entry 54a90e9555 docs: Create links for objects to show the api 9d41eaedbf Issue UnicodeWarning only for non-ascii unicode 46d157fe07 Fix collection report when collecting a single test item 87e4a28351 Fix incorrect collected items report when specifying tests on the command-line 5ee9793c99 Fix CHANGELOG issue id 1863b7c7b2 Merge pull request #2462 from segevfiner/py36-windowsconsoleio-workaround 01ed6dfc3b Added a changelog entry for the WindowsConsoleIO workaround 59b3693988 Fixed wrong if in the WindowsConsoleIO workaround 05796be21a A workaround for Python 3.6 WindowsConsoleIO breaking with FDCapture f826b23f58 Merge pull request #2458 from segevfiner/fix-required-options-help 9abff7f72f Add a docstring to HelpAction f74f14f038 Fix --help with required options bcbad5b1af Merge pull request #2140 from pelme/issue2121 5d785e415e Merge pull request #2454 from nicoddemus/xfail-docs 409d2f1d54 Make it clear that pytest.xfail stops the test 9adf513c4b Merge remote-tracking branch 'upstream/master' into features cca4de20cf Merge pull request #2450 from nicoddemus/release-3.1.1 c98ad2a0a0 Install py 1.4.33 that contains the fnmatch py.std import fix. 5de203195c Add changelog for #2121 021e843427 Fixed #2121 Use `py.path`s fnmatch. This fixes an issue where python_files handled properly when rewriting assertions. ac9c8fcdab Failing test for issue #2121 3871810d1c Merge pull request #2451 from nicoddemus/update-release-howto 281fcd5a58 Update HOWTORELEASE 2fd7626046 Preparing release version 3.1.1 0540d72c87 Add extra space between changelog items 1dee443c2b Merge pull request #2445 from nicoddemus/warnings-remove-filter 32e2642233 No longer override existing warning filters during warnings capture 454426cba5 Merge pull request #2446 from nicoddemus/issue-2441 f96a1d89c5 pytest.deprecated_call now captures PendingDeprecationWarning in context manager form ee0844dbd8 Merge pull request #2431 from RonnyPfannschmidt/towncrier b74c626026 switch changelog management to towncrier 4e6e29dbee Merge pull request #2442 from nicoddemus/merge-master-into-features 6117930642 Merge pull request #2438 from nicoddemus/issue-2434 7bb06b6dad Merge pull request #2439 from nicoddemus/warnings-docs 7950c26a8e Add Hui Wang to AUTHORS list 836dc451f4 Fix unicode issue while running doctests in Python 2 8df3e55a31 Merge pull request #2437 from coldnight/master 53add4435f Add ChangeLog d7a5c5716f Add UnicodeWarning for unicode warnings in Python2 313a884459 Warn that warning-capture can break existing suites in the docs and CHANGELOG c39689da41 Correct warnings with unicode message. 17f64704c2 Merge remote-tracking branch 'upstream/features' f9953fbe7c Merge pull request #2425 from nicoddemus/publish-task 0ea80eb63c Merge pull request #2428 from The-Compiler/param-id-docs 38ebf8dd10 Merge pull request #2429 from The-Compiler/regenschauer 04b1583d10 Merge pull request #2426 from The-Compiler/fix-changelog d9b93674c3 Make --cache-show output deterministic 7d6bde2496 Add docs for id= with pytest.param bd065a12bb Fix up 3.1 changelog f9df750025 Update 3.1.0 release date d343f9497c Merge branch 'release-3.1' 5192191c38 Create task for publishing a release 69343310c6 Merge pull request #2422 from pytest-dev/refactor-config c9c2c34b44 Remove unused parameter 9beeef970e Parse the filename in the generator expression 43aa037ebd Reindent 2abf2070f2 Collapse nested for loops into a generator expression ce0ff0040f Reindent and add docstring 6d2e11b7d1 Extract method for _mark_plugins_for_rewrite 9b48613baa Preparing release version 3.1.0 0ff7f5d0c6 Prepare CHANGELOG for version 3.1.0 36cf89a2de Merge remote-tracking branch 'upstream/master' into features 3a4d37248d Merge pull request #2414 from nicoddemus/revert-new-style-classes 637550b249 Merge pull request #2418 from axil/master 598aefc686 small fix in an example from the docs 7af0e6bda1 Merge pull request #2415 from flub/training f7247dc99d Remove past training d86c89e193 Revert refactor of old-style to new-style classes e484f4760f Merge pull request #2412 from pytest-dev/pytest-book-ref 70bcd1fb7b Add a reference to Python testing book to talks docs 6f407ef308 Merge pull request #2411 from nicoddemus/automate-pre-release feab3ba70f Implement tasks to improve release automation 00e7ee532e Fix minor regendoc issues fe49c78f32 Merge remote-tracking branch 'upstream/master' into release-3.1 3c41349fe1 Merge pull request #2406 from RonnyPfannschmidt/regendoc-reduce-version-noise bd708068ab Merge branch 'features' into regendoc-reduce-version-noise 783670b84e Merge pull request #2274 from dmand/feat/junitxml/suite-name-option 03753ca201 intermediate state after attempt with the plain env, DONT MERGE 456925b604 Merge pull request #2395 from RonnyPfannschmidt/consider-all-modules f39f416c5d Improve tests a bit d1e44d16e7 regenerate docs from the pytest env 2ab8d12fe3 Update changelog and add usage info c9282f9e94 Transition to using ini option for suite name bcfa6264f1 Update AUTHORS list 204db4d1e2 Update Changelog fe7d89f033 Add '--junit-suite-name' CLI option c765fa6d04 add regendoc normaliz for pytest --version f1c4e2c032 regendoc: reduce version noise by replacing minor/patch with placeholders a92e397011 add changelog for fixing #2391 b6125d9a13 Merge pull request #2397 from nicoddemus/announce-task 66ba3c3aa4 Introduce a task to generate the announcement file for releases 7ee2db23df Merge pull request #2396 from johndgiese/patch-1 52c67af63c Clarify opening paragraph of parameterization docs 8bcf88ec12 try to consider all modules after registration as plugin daca618012 Merge pull request #2389 from nicoddemus/merge-master-into-features f3b359f5b8 Merge remote-tracking branch 'upstream/master' into merge-master-into-features 3fc917a261 Merge pull request #2385 from nicoddemus/anaconda-badge 814ea9d62c Merge pull request #2387 from nicoddemus/trial-errors 630cca2fba Fix py35-trial environment bfd2563b3a Merge pull request #2386 from robin0371/show-correct-msg 60b8339166 Issue #2383 - Show the correct error message when collect "parametrize" func with wrong args and add test for this case. 34f488757f Add badge for anaconda package version cccb2cc92b Merge pull request #1834 from RonnyPfannschmidt/setuptools-scm-take-2 d7d2249d99 Merge pull request #2378 from szuliq/patch-1 a280e43949 fix import error f0533194ed Update fixture.rst a9b44c4529 Update docs e02cb6d7ce restore setuptools_scm write_to usage 314d4afa57 Merge pull request #2367 from nicoddemus/py36-official-travis 25371ddbfd Merge pull request #2315 from RonnyPfannschmidt/namespace-hook 80225ce72c Merge pull request #2374 from Kodiologist/getmodpath-file-ext 4242bf6262 use unknown to specify unknown versions dcefb287fc Try not to assume a module's file extension is .py 2cf422733c restore linting, drop _pytest._version for check-manifest c0a51f5662 restore check-manifst functionality 31e6fe8f52 HOWTORELEASE.tst: use restructuredtext autonumbering c3aee4b1e6 second take at setuptools_scm 581b463b60 Merge pull request #2372 from nicoddemus/pytest-dont-rewrite-docs 90be44c812 Document PYTEST_DONT_REWRITE 80cabca21a Merge pull request #2292 from nicoddemus/defer-hook-checking cac82e71d8 Improve item.warn handling of fslocation parameter 6e2bbe88b1 Test against py36 official release for consistency with other python versions d9a2e70155 Change LsofFdLeakChecker to emit a warning instead of failing when detecting leaked FDs 7dfdfa5813 Merge pull request #2359 from pytest-dev/fix-2343 7d4ac14a31 Fix #2343: Replace version checks by constants. 731776702d Fix hook name in LsofFdLeakChecker 0baf5e1499 Fix test that expected "unknown hook" error on stderr 83c508eea3 Verify hooks after collection completes 78ac1bf5d1 Merge pull request #2350 from nicoddemus/future-imports-rewrite 02da278894 Merge pull request #2357 from ojii/lastfailed-failedfirst 1125786e78 Improve --lf/--ff test as commented during review 08d83a5c6a updated changelog and authors files 0ab85e7a9c Changed behavior if --lf and --ff are both used. 47a2a77cb4 Merge pull request #2354 from shobute/patch-1 21f1c2b03f Update fixtures.py 8c69d5c939 Merge pull request #1940 from skylarjhdownes/master f2300fbab2 Fix links in docs 45852386e5 Fix small typo in docs 5462697924 Small formatting fixes to nose.rst 639c592f31 Add missing link in CHANGELOG f7caa56a6b moved documentation of conftest.py hack to nose.rst 3aa4fb62d6 Merge branch 'master' into master c734a2d8d5 Merge pull request #2352 from pytest-dev/fix-search-docs 44a3db3dc6 Pin sphinx to 1.4 when generating docs to workaround search issues on RTD 1b5f898dc5 Ensure rewritten modules don't inherit __future__ flags from pytest 83b241b449 Merge pull request #2347 from reutsharabani/features 24ac923938 Add CHANGELOG entry 333ce9849d added acceptance test for unicode plugin names 417b54abed added option to unicode plugin name 144d90932e Merge pull request #2337 from nicoddemus/2336-unicode-tb a542ed48a2 Convert using utf-8 instead of ascii in safe_str() 58ac4faf0c Fix exception formatting while importing test modules afb1778294 put in a singular namespace hook to work around the strange issue ebeba79be3 remove the namespace hook from mark after the param feature merge 6165939b0d fix rebase mistakes efe03400d8 fixup nose/pytest plugins c9ab421398 fix python2 only import loop failure 147bb8aea5 correct setting pytest.config 7cdefce656 fix up oversights 4d31ea8316 add a comment explaining the modimport tests bb750a7945 add missed file 92f6ab1881 fix all singular internal module imports and add a test for them 809c36e1f6 add a changelog note for pytest_namespace 23bc9815c4 remove pytest_namespace from _pytest.fixtures ae234786ea remove pytest_namespace from _pytest.python 99c8f2d403 remove pytest_namespace from _pytest.main 61f418a267 hollow out pytest_namespace in _pytest.fixtures 9b58d6eaca prepare a own pytest.collect fake module in oder to remove the nested builtin namespaces 839c936153 _pytest.mark: fix unconfigure after bad configure, still potential bug 7d797b7dbf add a note about the deprecation of the pytest_namespace hook 9b755f6ec6 remove pytest_namespace from _pytest.skipping 90788defb2 remove pytest_namespace from _pytest.mark and fix latent pytest nesting bug 6a02cdbb35 remove pytest_namespace from _pytest/runner.py c74103f395 remove pytest_namespace from recwarn and fixture decorators 794fd5658c remove pytest_namespace from _pytest/debugging.py fab9b993f8 remove pytest_namespace from _pytest.freeze_support 5818e65cf3 remove pytest_namespace from _pytest/assertion 2a130daae6 Merge pull request #2072 from nicoddemus/integrate-pytest-warnings 0c1c2580d0 Add CHANGELOG entry 74b54ac0ec Fix errors related to warnings raised on pypy test environment 2c730743f1 Fix errors related to warnings raised by xdist 916d272c44 Fix test on linux eabe3eed6b Add docs for the warnings functionality fa56114115 Clean up warnings generated by pytest's own suite d027f760c0 Avoid displaying the same warning multiple times for an item 3373e02eae Add __future__ imports to warnings module 9f85584656 Merge remote-tracking branch 'upstream/features' into integrate-pytest-warnings de8607deb2 Merge pull request #1921 from RonnyPfannschmidt/marked-value e8a1b36c82 add changelog 6cfe087261 Merge pull request #2320 from pawelad/2239/exit-code-docs 8b57aaf944 Added 'Paweł Adamczak' to AUTHORS d58bc14645 Added 'Possible exit codes' section to docs (#2239) e368fb4b29 implement pytest.param a122ae85e9 Merge pull request #2316 from nicoddemus/add-future-imports 4d947077bb Fix test in py26 that expected a floor division error message e5021dc9dc Replace py.builtin.print_() calls by builtin print() function 42a5d6bdfa Add __future__ imports to all pytest modules 7684b3af7b Recommend using py36 for testing on CONTRIBUTING 78194093af Improve warning representation in terminal plugin and fix tests be5db6fa22 Capture warnings around the entire runtestprotocol 0baed781fe Merge remote-tracking branch 'upstream/features' into integrate-pytest-warnings 337f891d78 Fixed tests 5482dfe0f3 Merge pull request #2303 from nicoddemus/recwarn-refactor 75ec893d75 Merge pull request #2297 from nicoddemus/init-files-docs 76df77418d Merge pull request #2313 from nicoddemus/DontReadFromInput-exceptions 55b891ddd0 Merge pull request #2312 from nicoddemus/merge-master-into-features-post-3.0.7 aad4946fb6 Move CHANGELOG entry for #2276 to 3.0.8 9062fbb9cc Add AUTHORS & CHANGELOG dc6890709e Change ValueError to io.UnsupportedOperation in capture.py. Resolves issue #2276 272aba98e2 Mention the src layout as recommended practice 6c9011c12f Merge branch 'master' into merge-master-into-features-post-3.0.7 fa15ae7545 Post 3.0.7 release handling 5056d8cbe8 Merge pull request #2304 from nicoddemus/release-3.0.7 4a9348324d Add more information to test-layout docs as discussed during PR 5e52a4dda4 Merge pull request #2307 from nicoddemus/clarify-record-xml-property 92b49d246e Clarify that record_xml_property is experimental, not junitxml 90c934e25e Include release 3.0.7 announce in index.rst 3c07072bfd Fix test_recwarn in Python 3.6 d58780f9a6 Update regendoc b1ab2ca963 Bump to version 3.0.7 and update CHANGELOG 22864b75ee Refactor recwarn to use warnings.catch_warnings instead of custom code d1ea7c8cc8 Merge pull request #2301 from nicoddemus/merge-master-into-features 1e0cf5ce4d Merge remote-tracking branch 'upstream/master' into merge-master-into-features 581857aab6 Fix typo 841f731707 Attempt to clarify the confusion regarding __init__ files and unique test names 906b40fbb2 Merge pull request #2289 from fbjorn/fix-trailing-whitespace-in-terminal cee578e327 Fix trailing whitespace in terminal output 29383d477d Merge pull request #2288 from nodakai/patch-1 e05ff0338a assert.rst: typographical correction 272afa9422 Display node ids and the warnings generated by it bddb922f7b Rename internal option to disable_warnings de09023e45 Also capture warnings during setup/teardown e24081bf76 Change warning output b28749eb92 Merge pull request #2284 from omerhadari/bugfix-unprintable-assertion-errors 07623e78ce Merge pull request #2286 from pytest-dev/disable-py37-travis dd25ae7f33 added in the correct alphabitcal order 02dc545311 added in the correct alphabitcal order b61dcded37 Allow py37-nightly to fail on Travis f71467f5b1 added link to changelog 6aaf7ae18b added to authors and changelog 6a52fe1650 fixed internal error on unprintable raised AssertionErrors 0c94f517a1 Merge pull request #2236 from KKoukiou/junitxml-change-schema 26e50f1162 junitxml: adjust junitxml output file to comply with JUnit xsd 5721d8aed1 Merge pull request #2249 from pfhayes/anydbmfix 3aac3d0a00 Merge branch 'master' into anydbmfix 3e3f20380e Merge pull request #2277 from nicoddemus/yield-fixture-docs-2262 bb5f200ed7 Improve docs for yield-fixture and with statement a bit 0f3d7acdc4 Merge pull request #2266 from asottile/capture_v2 8b598f00e9 Make pytester use pytest's capture implementation 6ba3475448 Make capsys more like stdio streams in python3. Resolves #1407. 0a89db2739 Merge pull request #2271 from KKoukiou/double-tag d3a6be4130 junitxml: Fix double system-out tags per testcase 6680cb9100 Merge pull request #2264 from asottile/simplify_travis 44ad369c17 Merge pull request #2263 from nicoddemus/revert-pluggy 5fd010c4c3 Simplify travis.yml with tox environment variables 82785fcd40 Use warnings.catch_warnings instead of WarningsRecorder a7643a5fbe Merge branch 'features' into integrate-pytest-warnings f1900bbea6 Revert subclassing explicitly from object introduced by accident in #2260 21a09f0895 Merge pull request #2261 from vmuriart/doc-report_header a88017cf26 Add note documenting #2257 58d7f4e048 Correct typo abd6ad3751 Merge pull request #2260 from MichalTHEDUDE/feature/NewStyleClasses-2147 fb0b90646e New-style classes implemented for python 2.7 - #2147 9c809f5ad0 Merge pull request #2255 from scop/spelling 27f12ed0c3 Merge pull request #2254 from scop/py36-escseq 0a26132232 Merge pull request #2241 from nicoddemus/override-python-files da828aac05 Merge pull request #2253 from The-Compiler/norecursedirs 8f98ac5ae8 Fix typo in docs "textures" -> "fixtures" ede4e9171f Spelling fixes eeb6603d71 Python 3.6 invalid escape sequence deprecation fixes 231e2f9a90 Merge pull request #2252 from nicoddemus/fixture-visibility-docs c4d974460c Improve pytest_plugins docs 91c6bef77a Add venv to the default norecursedirs 6b5566db66 Update changelog 49289fed52 Fix docs 00ec30353b Update docs as requested 58ce3a9e8c Safer sys.modules delete 427bf42a52 Merge pull request #2247 from flub/flub/training b536fb7ace Mention next training event. 9eb1d73951 --override-ini now correctly overrides some fundamental options like "python_files" 3d9c5cf19f Merge pull request #2225 from mbyt/allow_skipping_unittests_with_pdb_active 6a097aa0f1 Merge branch 'master' into allow_skipping_unittests_with_pdb_active a4fb971c1f Merge pull request #2235 from bluetech/dont-execute-properties 3a0a0c2df9 Ignore errors raised from descriptors when collecting fixtures 87fb689ab1 Remove an unneeded `except KeyboardInterrupt` ccf9877447 Merge pull request #2232 from vidartf/patch-1 a4d2a5785b Merge pull request #2142 from barneygale/xfail_without_condition_getglobals 832c89dd5f Test for `pytest.mark.xfail` with non-Python Item 1a88a91c7a Update authors/history bad261279c Do not asssume `Item.obj` in 'skipping' plugin 208fae5bf0 Merge pull request #2227 from Kriechi/raises-info abbff681ba Fix '{0}' format for py26 43662ce789 allow error message matching in pytest.raises ad56cd8027 extract a _handle_skip method, secure PY2 branch 176c680e19 Merge branch 'master' into allow_skipping_unittests_with_pdb_active da5a3dba87 Merge pull request #2226 from nicoddemus/raise-stop-iteration e1c5314d80 Replace 'raise StopIteration' usages in the code by 'return's in accordance to PEP-479 36b6f17727 fixing code-style, keep flake8 happy d1c725078a Allow to skip unittests if --pdb active 3b47cb45e6 Merge pull request #2222 from RonnyPfannschmidt/features 3f30c22894 fix changelog merge mistake 713bdc1f9f merge master into features 0931fe2c89 Merge pull request #2221 from pytest-dev/ionelmc-patch-1 34e98bce0a Merge pull request #2198 from unsignedint/make-parametrize-enhancement c8032a9bbb Fix reference. d98d122e81 Discourage users from using this all the time. beb77c1a38 Fix release date for 3.0.6 d076e4158f Merge pull request #2216 from vmuriart/patch-1 902fd2ff6a Add py37-nightly to travis 839aa963a1 Add py36 identifier 400b0779f9 Merge pull request #2213 from RonnyPfannschmidt/release-3.0.6 c9f327dc87 bump version to next dev 0e64cd045c take off author_email after pypi rejects 22da561ae5 fix copy+paste error, its supposed to be 3.0.6 449b88c640 rerun regendoc with correct install 34b898b47e generate the release announcement 01eaf9db51 fix the xfail docstring typo at the actual docstring + regendoc 4d0c635252 regendoc 55f21bd2b9 bump version c39d846c1b Merge pull request #2215 from RonnyPfannschmidt/fix-doctesting/devpi-bug 403122281a fix devpi test for doctesting env 0e58c3fa80 updates for PR review #2198 c848d0a771 Pass parameter name to `make_parametrize_id` hook function 15a3b57ec7 Merge pull request #2120 from RonnyPfannschmidt/fix-2118 6a96b464ab update changelog as suggested 7b4afd8946 remove unused import 1a2d6388ac Merge pull request #2211 from nicoddemus/trial-envs 3766060893 Merge branch 'master' into trial-envs 4082f4024a comment out compatproperty deprecations e0c48b4fe7 Merge pull request #2212 from nicoddemus/pytester-rewrite 7b4368f3f4 Merge pull request #2184 from eli-b/parseoutcomes-explicit-failure 88f7befabb Merge pull request #2209 from RonnyPfannschmidt/bugfix-2208/get_real_func_loop_limit c477f09177 Assert statements of the pytester plugin again benefit from assertion rewriting 2574da8d32 Fix pytester internal plugin to work correctly with latest versions of zope.interface 250597d468 get_real_func: use saferepr when formatting the error message 123289a88e fixes #2208 by introducing a iteration limit d15724f74f Merge pull request #2204 from nicoddemus/linux-marker-doc 61fa91f3d0 Fix marker example on "linux" platform 125e89b7f8 Merge pull request #2194 from rjprins/remove-reinterp-from-docs 46a9861d29 Remove mention of --assert=reinterp in documentation 3dfdbaf490 Merge pull request #2186 from nicoddemus/pytest-plugins-env-rewrite 7cd7c283dd Refactor plugin specs handling into an isolated function 043aadeaf2 Consider plugins loaded by PYTEST_PLUGINS for assertion rewrite e18b2a427a Fail assert_outcomes() on missing terminal report Currently if the terminal report of testdir.runpytest() is missing, assert_outcomes() on its output fails because parseoutcomes() returns an unexpected value (None). It's better to fail parseoutcomes() directly. ff309b3584 Merge pull request #2182 from pombredanne/patch-1 aa82db9fe2 Ensure the LICENSE is included in built wheels 6c011f43e9 Merge pull request #2179 from mandeep/new-style-classes e412ea1d5a Added name to AUTHORS and change to CHANGELOG d4afa1554b Refactored old style classes to new style classes 64cb67b703 Merge pull request #2174 from nicoddemus/appveyor-py36 7559400183 Add py36 to test on AppVeyor 9477f598d8 Merge pull request #2173 from jeffwidman/patch-1 6d81c684cc Switch monkeypatch fixture to yield syntax 3494dd06fe Remove duplicate target in rst 9e9547a9e4 Simplify condition 7930a8373d Newline for flake8 0bd8159b60 Add a test when multiple classes are specified in warns 56d1858ea2 Remove duplicate '@lesteve' link from CHANGELOG 6fd0394c63 pytest.warns checks for subclass relationship 8f1450114f Merge pull request #2167 from fogo/parametrize-ids-silent-failure b769e41d8f Merge pull request #2170 from gogoengie/broken-links ef903460b1 Fix broken links df409a0c0e Fix CHANGELOG.rst 8db9915374 Update AUTHORS, CHANGELOG 3d18c9c1c6 'xfail' markers without a condition no longer rely on the underlying `Item` deriving from `PyobjMixin` a9193a1531 No longer silently ignore errors in parametrize callable ids 78f03888f4 Merge pull request #2168 from jwilk/spelling 03a7a2cd3e Fix typos 964ccb93bb Merge pull request #2163 from nicoddemus/merge-master-into-features 402fbe503a Merge branch 'master' into merge-master-into-features 7592c5b491 Sort issues and user references in CHANGELOG 091148f843 Merge pull request #2136 from hroncok/i2132 718f0b0255 Merge pull request #2130 from malinoff/fix-2129 b4295aa19e Merge pull request #2144 from sscherfke/patch-1 7d259401cd Merge pull request #2161 from nicoddemus/silence-trial-on-ci 515fb09995 Move module error compatibility code to _pytest.compat 088b742d40 Merge pull request #2149 from pelme/issue2148 6b24ce2a9d Test Python 3.6 on Travis CI 1680eeb3a3 Tests: Check for ModuleNotFoundError on Python 3.6+ 0bb8a4a36d Fixed #2148 - parse directory names properly when args contains ::. f7a1d369c3 Allow trial environments to fail on CI for now 316406291d Merge pull request #2150 from lesteve/add-caught-warnings-info-when-warns-fail a27c824fd0 Merge pull request #2146 from lwm/minor-docs-fixup fc74eb332b Merge pull request #2128 from nicoddemus/pytest-m bfada968d3 Update AUTHORS and CHANGELOG.rst c5f0b751f4 Improve error message when pytest.warns fail a63b34c685 Switch to item fspath f94189b48b Fix wrong fixture name. Closes #2143. 3f5edc705a Explicitly add setuptools to install_requires caee5ce489 Avoid importing asyncio directly because that in turn initializes logging (#8) 1312b83866 Add CHANGELOG entry for #2129 45eb9b566c Move compat tests to a single file using testdir 3a59acf69f Use inspect to properly detect generators. Fixes #2129 81c9bdcd0b Mention that Python also adds CWD to sys.path using python -m 522d59e844 Use session.config.hook instead of ihook. Fixes #2124 da40bcf97f Merge pull request #2119 from nicoddemus/changelog-dates a4a30ae4a2 Merge pull request #2123 from oscarh/documentation-issue-687 f42a954cb3 Also update yield teardown for Issue #687 9c285dfc1d fix #2118 - rework Node._getcustomclass and Node compat properties 8afca5d0fa Add release dates to CHANGELOG entry 3a0a1d2de3 Bump version to 3.0.6.dev0 6a52afc8c9 Merge pull request #2116 from nicoddemus/release-3.0.5 f592c7746a Regendocs for 3.0.5 31f114e51f Add release announcement for 3.0.5 833acb9d3c Finalize CHANGELOG for 3.0.5 0febd855e1 Bump version to 3.0.5 3c81f83602 Merge pull request #2113 from nicoddemus/approx-repr-unicode 57c4489916 Use a simple ``+-`` ASCII string in the string representation of pytest.approx In Python 2 5365f7c9ca Merge pull request #2112 from ismail-s/patch-1 1f0401ab62 Fix minor typo 7480342710 Fix typo in docstring of register_assert_rewrite db62f160e1 Merge pull request #2073 from nicoddemus/fix-hookproxy-cache 81528ea81f Remove hook proxy cache 64193add91 Merge pull request #2110 from nicoddemus/rewrite-warning-pytest-plugins bc0f7e6243 Fix false-positive assert rewrite warnings when using 'pytest_plugins' 9ed3d76b51 Merge pull request #2108 from lwm/exp-2105 c856537e71 Add warning for incorrect passing args to `-o`. e612619aea Merge pull request #2106 from nicoddemus/sys-modules-none 30f0152ae6 Remove unused import f8d195253e Remove support code for earlier Python 3 version in Source.compile 669332b7e0 Merge pull request #2101 from wheerd/doctest-encoding 9c224c94f0 Merge pull request #2099 from lwm/fixup-2007 2edfc805af Added "versionadded" for doctest_encoding ini option to docs. f5afd8cb54 Add missing `__test__` check for test discovery. f8fef07b4c Fixed the tests for python 2.6 b7fb9fac91 Fixed the documentation for doctest_encoding. 1f62e5b5a0 Added CHANGELOG entry and myself to AUTHORS. c043bbb854 Changed the doctest_encoding option to an ini option. Parametrized the tests for it. 929912de29 Changed the tests to pass on python 2 as well. d254c6b0ae Added some tests for --docstring-encoding option. Added option to specify encoding for internal testdir._makefile() for the tests. ed977513ec Added a console option to specify the encoding to use for doctest files. Defaults to UTF-8. 8208a77a3e Merge pull request #2098 from DuncanBetts/master 8b4da9d955 Merge pull request #2100 from blueyed/fix-help-grammar 454d288138 Merge pull request #2097 from lwm/junitxml-warn 40cffacadc minor: fix grammar with help for --setup-{only,show} 6473c3d87e Improved description of functionality for Issue #687 4e1609b12e Add `type` validation. 36eb5b36d1 Merge pull request #2096 from nicoddemus/merge-master-into-features b30a6d22c5 Merge branch 'master' into merge-master-into-features 0735d4549d Merge pull request #2088 from nmundar/issue-2034 f25ba4dd0b Merge pull request #2095 from nicoddemus/readme-example 788e394c93 Use "inc" instead of "func" in the snipped on README and doc index 2d7197926a Improve CHANGELOG entry for #2034 0a30f072e6 Show name argment in compatproperty deprecation message 0e6ad8e59f update CHANGELOG and AUTHORS b38fad4b82 Add compatproperty deprecation warning. 483754216f Merge pull request #2094 from nicoddemus/remove-setuptools-pin 1e97ea60f7 Merge pull request #2091 from lwm/move-595-along 58f28bf049 Merge pull request #2093 from lwm/add-docs-notes 5566b3ccb6 Remove setuptools pin now that upstream has been fixed 8138d88da2 Merge pull request #2087 from lwm/ref-recwarns 0aa891543d Add documentation building note. 6c5475660a Add test case for #595. 1aa5bfea11 Add `:ref:` targets to `recwarn.rst`. a6084ed797 Merge pull request #2090 from avojnovicDk/issue-1808 d05d19da78 Merge pull request #2085 from DuncanBetts/master fa4d5da4ca Merge pull request #2092 from nicoddemus/pin-setuptools 6120570198 Pin setuptools to < 29 because of AppVeyor failures 2e6a58ab69 Clarify test discovery docs. c1b83cdeea Add hint of Issue #478 to error text 33796c8a13 Merge pull request #2084 from nicoddemus/fix-coveralls-appveyor 8763590eef Only execute "coveralls" toxenv on master once b3efd9aa59 Merge pull request #2083 from nicoddemus/approx-check-float 33c0b06fdf Fix error in approx's repr with complex numbers 38f7562c7c Merge pull request #2080 from nicoddemus/confcutdir-check 629d8e9fd6 Show an error if --confcutdir is not a valid directory a5b5090c72 Merge pull request #2070 from nedbat/bug2038 bd343ef757 Merge remote-tracking branch 'upstream/features' into integrate-pytest-warnings 5ce551e469 Merge pull request #2075 from pytest-dev/master a3319ffe80 Merge pull request #2071 from nicoddemus/fix-flake8 1e2b2af296 Merge pull request #2074 from nedbat/fix-double-spaces 632c4d5daf Remove an accidental double space 26ca5a702e Add tests and integrated the original code into the core 1da1906483 Rename code to _pytest.warnings and delete old files from the repository 9db32aea48 Merge c:\pytest-warnings\ into integrate-pytest-warnings e31421a5d2 Moving all stuff to a subdirectory to try to retain history 984d4ce5ec Fix test_excinfo_getstatement that broke because of whitespace changes 1eb5a690d4 Fix flake8 E305 and E306 errors 06bb61bbe3 Don't fail if imp can't find the source for a .pyc file. #2038 cbf261c74e Merge pull request #2065 from sebastinas/spelling2 0ba930a11d Fix spelling mistakes 75740337d1 Merge pull request #2060 from pytest-dev/master 6876ba9ba6 Merge pull request #1995 from mattduck/feat/restructure-assert-truncation 73d481552d Merge pull request #2058 from idlesign/patch-1 50328f47db Docs: Added pytest promotional talk in Russian f0e0250cd5 Merge pull request #2056 from nicoddemus/appveyor-list ec69514eb2 Only install pypy on AppVeyor for "pypy" tox-env c169c883d3 Use one job for each tox env on AppVeyor 5185f2a6ae Merge pull request #2053 from nicoddemus/check-manifest-script 351395b7ea Use a wrapper script to bypass check-manifest if not under git 0fab78ee8f Merge pull request #2054 from nicoddemus/merge-master-features 71b68334e2 Merge pull request #2055 from nicoddemus/error-matrix-for-failing-jobs 98caeedd9e Allow failure of pypy on AppVeyor 1519b38af0 Allow failure of py35-trial on Travis efc54b2e56 Merge branch 'master' into merge-master-features 3e01e83390 Bump version to 3.0.5.dev ad4ef4f583 Merge pull request #2050 from nicoddemus/release-3.0.4 5717c71179 Merge pull request #2052 from nicoddemus/lint-readme 6c8c1da428 add pygments dependency because of rst-lint b8c6f13b37 Check README.rst with rst-lint 8e0f7d3793 Merge pull request #2051 from nicoddemus/check-manifest aaa547e763 Add some recursive-exclude related to hypothesis and freeze 26b1519534 Add keywords to setup.py as suggested by pyroma 84d7068723 Add "check-manifest" to linting and remove unused scripts from root ab274299fe Regen doc for 3.0.4 ff72db2f1a Version bump to 3.0.4, CHANGELOG, announcement fc304b8b44 Merge pull request #2006 from MSeifert04/fix-1965 1130b9f742 Fix the stubborn test about cyclic references left by pytest.raises 552c7d4286 added test (thanks @nicoddemus) and added links in Changelog 1e5b21cd61 Fix memory leak with pytest.raises by using weakref 0b94c43bac Merge pull request #2046 from d-b-w/clean-up-unittest-issue1649 e46e653794 Clean up unittest TestCase objects after tests are complete (#1649). 07af307e4a Merge pull request #2045 from manueljacob/normalized-version a190ad27f2 Change version to be in normal form according to PEP 440. f331e8f576 Merge pull request #2028 from nicoddemus/empty-tracebacks 006a901b86 Properly handle exceptions in multiprocessing tasks 45b21fa9b0 Merge pull request #2041 from gdyuldin/fix_xuint_teardown e2bb4f893b Fix teardown error message in generated xUnit XML d49e9e5562 Merge pull request #2014 from RonnyPfannschmidt/warning-record-namedtuple e3544553b7 Merge pull request #2030 from matclab/fix/442 1e6ed2a25a Merge pull request #2033 from The-Compiler/workshop 382fa231a1 Update "Next Open Trainings" 6f93ffb5d4 Report teardown output on test failure b3c337db00 add changelog entry and documentation note about RecordedWarning e9668d75b8 turn RecordedWarning into a namedtuple 377e649e61 local merge of #1967 - Change exception raised by capture.DontReadFromInput.fileno() 35d154f580 Merge pull request #2011 from nicoddemus/false-rewrite-warnings 4e9c633185 Merge pull request #2021 from nicoddemus/doctest-modules-ci 6ec0c3f369 Bump. ce138060ac Prepare pytest-warnings 0.2.0. f229b573fa Bump version, add changelog entry and move stuff around for added coverage reporting. 28621b0510 Merge pull request #2 from Carreau/filter-regex bc94a51a96 Add a warning option which does not escape its arguments. 7f95ea31d5 Merge pull request #2024 from jaraco/issue-2022 f2c01c5407 Restore pexpect tests and bypass isalive/wait on macOS. Ref #2022. 60a347aeb5 Also use flush where wait was called unconditionally 11ec96a927 Extract child flush as a staticmethod 37dcdfbc58 Re-enable docstring testing of _pytest modules on CI 9d00615bbf Merge remote-tracking branch 'upstream/master' into merge-master-into-features 2a2b8cee09 Fix false-positive warnings from assertion rewrite hook 82fb63ca2d Merge pull request #2019 from nicoddemus/fix-metavar-pair-help 620b384b69 Fix cmdline help message for custom options with two or more metavars 5cbfefbba0 Merge pull request #2018 from nicoddemus/disable-py35-trial 95007ddeca Disable py35-trial while #1989 is not fixed 995e60efbf Merge pull request #2017 from nicoddemus/pytest-main-args-docs 918af99a2a Remove example of obsolete pytest.main usage with string c0719a5b4c Merge pull request #2009 from pytest-dev/RonnyPfannschmidt-patch-docs-remove-main-string afc1e2b0e1 docs: remove mention of string args to main de1614923f Merge pull request #2000 from nicoddemus/issue-1998 acee88a118 Fix truncation tests: pyruntest() match bc1f8666aa Fix link in CHANGELOG for #1853 78eec0d7f8 Handle import errors with non-ascii messages when importing plugins 0061d9bd3d Fix flake8 (unused import, trailng whitespace) b629da424e Restructure truncation of assertion messages 3301a1c173 Merge pull request #1987 from Budulianin/master 65ebc75ee8 Update fixture.rst cf13355d3f Merge pull request #1979 from nicoddemus/show-traceback-during-collection 1289cbb9a5 Merge pull request #1988 from nicoddemus/pr-template-small-fixes 10433db225 Mention small doc fixes don't need tests/changelog entries in PR template 50b960c1f0 Add note about not monkey-patching builtins (#1986) d47ae799a7 Merge pull request #1983 from pytest-dev/fix-1981-improve-ini-options-help-text c93a9e3361 Fix #1981, improve ini-options help text a1d446b8e8 Display traceback from Import errors using pytest's short representation 7d66e4eae1 Display full traceback from Import errors when collecting test modules fc02003220 Merge pull request #1975 from nicoddemus/pytest-skip-message 336d7900c5 Fix test about pytest.skip message being used at global level 57bb3c6922 Improve error message when using pytest.skip at module level 4667b4decc Merge branch 'master' into features a87b1c79c1 post 3.0.3 release changes 30f3d95aeb Merge pull request #1973 from nicoddemus/release-3.0.3 ba6ecc14c8 Include release-3.0.3 into the announce toctree 41d3b3f4f9 Regendoc for version 3.0.3 dda17994ec Prepare for 3.0.3 release b0c78c867d Update CHANGELOG.rst 3d211da9bd add existing test suite page to table of contents 1ab1962eb1 make issue #1934 reference a link 427cee109a Merge pull request #1964 from nicoddemus/pluggy-0.4.0 12ac3c7338 remove existing tests stuff, add link to new page 7e2f66adc3 Create existingtestsuite.rst 654af0ba25 Merge remote-tracking branch 'upstream/master' into features acac78adc0 Add link to profile. 30d459e2e3 Merge pull request #1968 from nicoddemus/use-latest-hypothesis f31447b82b Use hypothesis >= 3.5.2 3444796f3e Fix formating error. ff492ca73f Thanked my self in the change log :) 8985c0be3e Change exception raised by DontReadFromInput.fileno() from ValueError to io.UnsupportedOperation b071fdc633 Merge pull request #1960 from nicoddemus/skip-module-level-doc 19766ef0bc Add a summary on how to skip all tests in a module in different situations 94155ee62a Add a note about pytest.skip not being allowed at module level 835328d862 Vendor pluggy 0.4.0 8dc497b54b Merge pull request #1966 from nicoddemus/disable-pypy-windows da201c7d29 Disable pypy on AppVeyor until #1963 gets fixed 01068e5571 Merge pull request #1962 from nicoddemus/fix-ci 3fce78498f Mention #1952 on the CHANGELOG 5e96edd435 Merge pull request #1952 from davidszotten/pdbcls_without_pdb_on_fail 73cab77249 Pin hypothesis to 3.5.0 because 3.5.1 breaks the test suite 09bcf7f170 Merge pull request #1958 from rowillia/master f1c4cfea2c Remove implementation of `__getslice__` e5deb8a927 Merge pull request #1955 from rowillia/fix_python3_deprecation_warnings 24db3c123d Explicitly set to None to have consistent behavior in Python 2 and Python 3 940ed7e943 Fix `DeprecationWarnings` found when running py.test in Python 2.7 with the -3 flag. f279bd2bd5 Merge pull request #1918 from tgoodlet/explain_bad_scope 6db2f315fa Explain a bad scope value to the user d75748ef6f test for calling set_trace with custom pdb cls c7b4b8cf6f test 0ac85218d1 allow pdbcls without implying usepdb 7660a19d0a Merge pull request #1945 from axil/master 8cba033bbb added a test 6b56c36ae7 added to changelog and authors 65be1231b1 AttributeError chaining bug #1944 fix 887c097f8e Merge pull request #1951 from mattduck/feat/1512-dict-compare-output 01db0f1cd1 Merge pull request #1923 from RonnyPfannschmidt/mark-internal-value 4df74a5cfb Add AUTHORS and CHANGELOG for #1512 999e7c6541 Tidy formatting of assertion truncation dd64d823b9 Don't display dict common items if verbosity=1 dc16fe2bb9 Merge junit-xml url attribute branch 5b260d80f9 Merge pull request #1883 from RonnyPfannschmidt/kill-memoized-call 8639bf7554 fixup! Merge pkg_resources workaround f484e7c9ca Merge pkg_resources workaround 2bafa74765 Merge pull request #1942 from mbyt/better_doc_for_1890 77d842ceb2 better doc for #1890 based on #1932 34117be98b Update goodpractices.rst 330a2f6784 Update getting-started.rst f1faaea3fd Update CHANGELOG.rst d781b76627 Update AUTHORS 81a733f2dc add how-to for getting started on existing project 07ad71e851 clarified purpose of `pip install -e` command b4fd74c6ff add mention of setup.py develop 69f72c6f4b fix typo 383fc02ba6 fix spacing d217984129 documenting how to point pytest at local code 863b7d0c50 Merge pull request #1933 from pytest-dev/metafunc-members-docs 40ec35767f Merge pull request #1935 from nicoddemus/assert-rewrite-dev-plugins 04cf5e1df4 Fixed assertion rewriting for plugins in development mode 138e255631 Remove duplicated for Metafunc members on docs 45524241a5 mark: fix introduced linting error 1812387bf0 Mark: fix python 3 compatibility 10094a3f09 use consistent inner repressentation for marks 51378aba01 Merge pull request #1919 from The-Compiler/pdf 8e67dd13e7 Fix link to PDF docs 99efc281ee Merge pull request #1914 from RonnyPfannschmidt/docs-remove-getdoctarget a2b8981b50 docs: remove unused helper script 1c9bd9278e Merge pull request #1913 from flub/builtin-assertion e21ae3991d docs: remove the need for special scripts 28b1896e9a Remove BuiltinAssertionError 8a41b26f56 Merge pull request #1908 from nicoddemus/parametrize-unicode-id e1674f60e7 remove memocollect anmd memoized_call e572c16d3f remove memoizedcall in Module b6f766ae87 Merge pull request #1912 from flub/reg-assert-rewrite-changelog ceeb5149f3 Mention register_assert_rewrite explicitly b38cf77562 Merge pull request #1911 from irachex/patch-1 c60854474a Fix keyword docs c69fbd79a9 Merge pull request #1910 from nicoddemus/monkeypatch-members-docs 183f3737d4 Fix reference docs for monkeypatch 1e10de574d The "ids" argument to "parametrize" again accepts unicode strings in Python 2 722f9eadcd Merge pull request #1906 from pytest-dev/adjustment-how-to-release 7a4fe7582c Send only minor/major release announces to TIP mailing list 98ac1dd34b Merge pull request #1902 from nicoddemus/features f5d900d972 Merge remote-tracking branch 'upstream/master' into features 96a1d2941b Merge pull request #1900 from nicoddemus/finalize-3.0.2-release ee284ec587 Set version to 3.0.3.dev0 927f411ee2 Fix release 3.0.2 release announcement list of authors 919f50a3bd Merge pull request #1896 from nicoddemus/release-3.0.2 4e58c9a7d0 Fix use of deprecated getfuncargvalue method in the internal doctest plugin a9f3053f72 Fix version typo in announce for 3.0.2 d512e7f26b Run regendoc for 3.0.2 release f985f47a02 Fix reportingdemo call to pytest 4c45b93007 Changes for 3.0.2 release a094fb3aa6 Merge pull request #1890 from mbyt/disable_tearDown_and_cleanups_for_post_mortem_debugging e43d1174f7 spelling 696a9112be integrating review commets of @nicoddemus be08223d5a Merge branch 'master' into disable_tearDown_and_cleanups_for_post_mortem_debugging 10b3274924 adding corresponding test, authors and changelog 67ba8aaaa2 Merge pull request #1891 from nicoddemus/find-spec-rewrite-hook edf8283bd8 Add CHANGELOG entry for #1888 c8a366e551 Fix issue where pytest_plugins as string was marking wrong modules for rewrite 9d2149d9c0 Merge pull request #1884 from pytest-dev/master 4eeb475138 avoid tearDown and cleanup for unittest debugging 82218e4ee1 Merge pull request #1887 from mbyt/pdbcls_example_with_tab_completion 8593bb12ee pdbcls ipython example with tab completion e75078eae2 Merge pull request #1848 from pytest-dev/doc-warnings 519f02b014 docs warnings as errors and initial page similar to README cb7c472e34 Merge pull request #1876 from nicoddemus/remove-monkeypatch-invocation-example a947e83be9 Merge pull request #1870 from AiOO/bugfix/assertion-with-unicode e92d373460 Remove example of "monkeypatch" used in session scope 86b8801470 Update AUTHORS and CHANGELOG 856ad719d3 Fix UnicodeEncodeError when string comparison with unicode has failed. 1b259f70f3 Testcase reports with a url attribute will now properly write this to junitxml 9c45d6cd83 Merge pull request #1866 from joguSD/bugfix-stdin-stub-buffer a152ea2dbb Add buffer attribute to stdin stub 8f516d27fa Moved import pkg_resources to else clause. 3345ac9216 Merge pull request #1861 from nicoddemus/ids-invalid-type-msg c5675d3efc Remove merge-conflict marker merged by accident 6359894fe4 Merge pull request #1859 from nicoddemus/merge-master-into-features 972a5fb5d5 Improve error message when passing non-string ids to pytest.mark.parametrize 7704f73db9 Merge branch 'master' into merge-master-into-features ea0febad28 Updates for new patch version 3.0.2 ecb20b96fc Merge pull request #1856 from nicoddemus/release-3.0.1 49fc4e5e4c Changes for 3.0.1 release 3c866e7080 Merge pull request #1855 from nicoddemus/fix-empty-parametrize-ids df200297e2 Fix internal error when parametrizing using and empty list of ids() 7538680c98 Merge pull request #1847 from nicoddemus/parametrize-session-scopes d3f4b3d14a mend a1597aca89 Added #1853 to changelog as requested on PR. 7d498fdc82 Added myself as AUTHOR as requested on PR. 2f11a85698 Import pkg_resources only when necessary. 847067fb02 Merge pull request #1851 from adamchainz/deprecated_setup.cfg 1673667232 Delete unreferenced setup.cfg from docs 53a0e2b118 Fix code which guesses parametrized scope based on arguments d99ceb1218 Merge pull request #1844 from nicoddemus/importer-error b54ea74d4d Update AUTHORS and CHANGELOG for #1837 6a8160b318 Add acceptance test for module loader c3148d1d6b Merge pull request #1842 from Stranger6667/doc-fix b0ede044ac Merge pull request #1835 from RonnyPfannschmidt/mark-extract-param 3fbf2e7a80 Fix documentation af0ec120fe Merge pull request #1836 from asvetlov/patch-1 678750c8f8 Fix importer call 4e9c1fbe96 Remove duplicated name 406777d104 mark plugin: move the unclean marked parameter extraction abe8f5e23f Merge pull request #1824 from nicoddemus/update-howtorelease 6a0e849067 Update HOWTORELEASE based on the 3.0.0 release a20c3f9c44 Merge pull request #1827 from blueyed/Fix-spelling-s-outside-a-outside-of-a- 783aff17dd Merge pull request #1828 from blueyed/minor-s-no-pkg_resources-pkg_resources- 86ec3f37af minor: s/no pkg_resources/pkg_resources/ e306a53999 Fix spelling: s/outside a/outside of a/ 90fb8cb08b Merge pull request #1823 from nicoddemus/importorskip-regression 5129c2f867 Merge pull request #1826 from blueyed/Remove-duplicate-entry-for-#717-in-CHANGELOG 87d2d1d838 Remove duplicate entry for #717 in CHANGELOG 9aec8d9a47 py.test team -> Pytest team :) c8f53d6690 Fix typo in 3.0.0 release announcement 875bcd4224 Add 3.0.0 release announcement to index.rst 63dc71c57e Fix regression when using importorskip at module level db922403cc Bump version to 3.1.0.dev 3a200b75c9 Bump version to 3.0.1.dev 745c8c17f1 Merge remote-tracking branch 'upstream/master' 5ecf3d78f1 Merge pull request #1819 from matthiasha/patch-1 b85a3b0d71 Merge pull request #1820 from The-Compiler/changelog-duplicates 8777f61e7b Merge pull request #1818 from nicoddemus/release-3.0 31ede2432c Remove duplicate CHANGELOG messages c8fbf3ae34 remove dot in pytest 3455dfc804 add missing @matthiasha link cd39cc1eec Mention doc update in CHANGELOG ccfa8d15bf Update AUTHORS 799dab9dba Documentation update for rootdir discovery c74ce371ab Add release announcement 7f18367582 Finalize CHANGELOG a3e6c14da3 Bump version to 3.0.0 be9356aeba Merge pull request #1817 from nicoddemus/regendoc-run 9ce30e0085 Run regendoc for 3.0 release 3748112a94 Merge pull request #1816 from nicoddemus/merge-master-into-features 0334e75c30 Use "pytest" instead of "py.test" on trial environments 56df9fcc72 Tweak CHANGELOG: move "change" entry to the proper place 66673c0dd3 Remove obsolete docstring 030c42203d Fix conflicts in CHANGELOG 3ba475c0f2 Move internal _is_unittest_unexpected_success_a_failure to "compat" module 463e6572c5 Merge branch 'master' into merge-master-into-features 3e685d6a8d Merge pull request #1795 from hackebrot/fix-report-outcome-for-xpass 68ebf552a1 Merge remote-tracking branch 'upstream/master' into fix-report-outcome-for-xpass a01cbce662 Merge pull request #1814 from nicoddemus/py35-trial-windows 0fb34cd2a1 Update CHANGELOG entries 224ef67374 Quick fix for tests in config depended on the current directory 4ed412eb59 unittest's unexpectedSuccess should work as non-strict xpass 92498109e4 Enable py35-trial testenv on Windows dfc659f781 Fix sys.version_info expression in xfail marker 0173952961 Fix py3 xfail expression evaluation and parametrize strict 767c28d422 Fix broken test in test_junitxml d1f2f779ee Use a better xfail reason bb3d6d87b6 Merge branch 'master' into fix-report-outcome-for-xpass 018197d72a Fix broken test in test_skipping and add one for strict xfail ea379e0e4f Fix test in test_junitxml and add one for strict 789e4670e7 Merge pull request #1813 from nicoddemus/pytest-setup.cfg c8ab79402c Merge pull request #1811 from nicoddemus/revert-invocation-fixtures ab86dea529 Support [tool:pytest] in setup.cfg files 707b6b5e3f Revert all invocation-fixtures code 09e647c7d9 Merge pull request #1812 from nicoddemus/deprecate-resultlog f25771a101 Deprecate --resultlog cmdline option 55ec1d7f56 Update test_junitxml.py to interpret XPASS as passed 497152505e Add CHANGELOG entry for #1809 ca5957932b Merge pull request #1806 from blueyed/fix-off-by-one-error-with-warnings d58a8e36d7 Merge pull request #1807 from blueyed/improve-multiline-error-format d3b855104c Merge pull request #1809 from blueyed/exitstatus-with-pytest_terminal_summary c163cc7937 Improve display of continuation lines with multiline errors 16cb5d01b1 Fix off-by-one error with lines from request.node.warn 5b95ee3c19 Pass exitstatus to pytest_terminal_summary hook 225341cf2c Set wasxfail only for xpass w/o strict and else set longrepr 14a4dd0697 Extend test to verify longrepr in stdout 296f42a2c9 Treat unittest.expectedFailure pass as a failure 10a6ed1707 Update unittest test to expect failure for an unexpected success 34925a31a9 Merge pull request #1799 from cryporchild/junitxml-tests-tally-fix c4d9c7ea55 Add thanks as requested. e4028b4505 Fix #1798 to include errors in total tests in junit xml output. 99a4a1a784 Merge pull request #1791 from nicoddemus/ide-integration-1790 4ab2e57ebd Merge pull request #1797 from nicoddemus/merge-master-into-features 802755ceed Merge remote-tracking branch 'upstream/master' into merge-master-into-features ac5c39e534 Merge pull request #1796 from nicoddemus/fix-rootdir-common-ancestor 21230aa017 Add CHANGELOG entry eb08135280 Update documentation to describe expected rootdir behaviour a2891420de Fix determining rootdir from common_ancestor 4fc20d09fe Change outcome to 'passed' for xfail unless it's strict 1a79137d04 Add originalname attribute to Function 530b0050b4 Improve TestReport.sections docs a bit ff296fd541 Add capstdout and capstderr attrs to TestReport 08002ab75a Add longreprtext property to TestReport objects 6759b042b5 Merge pull request #1789 from nicoddemus/regendoc-take-2 8b8c698f1a Add more interpreter versions to multipython example d28801d794 Make parametrize example deterministic 72df32f1fd Fix missing print() 701d5fc727 Regendoc after more fixes on features branch 6e3105dc8f Merge pull request #1787 from nicoddemus/fix-rewrite-conftest 6711b1d6ab Rewrite asserts in test-modules loaded very early in the startup d5be6cba13 Merge pull request #1788 from nicoddemus/available-fixtures-sorted 277b6d3974 Sort fixture names when a fixture lookup error occurs ea6191a0cd Merge pull request #1779 from RonnyPfannschmidt/deselect-no-reason 9540106fe7 Merge pull request #1780 from nicoddemus/regen-pytest30 1c8fe962f3 add changelog entry 48f4e18280 fix deselect tests to match reason removal 21a90c8c50 Run regendoc again eed21e06db Sort yml items to get same results for regendoc runs a6b2732507 Pass list of params to pytest.main() in docs 946466abf4 Run regen-docs for pytest 3.0 3cd2e37c55 Merge pull request #1783 from nicoddemus/inv-scoped-fixture-msg 553dc2600f Strip invocation-scope suffix when displaying fixture lookup error 226f2795ba Merge pull request #1782 from nicoddemus/invocation-fixtures-help b380eb5b34 Merge pull request #1781 from nicoddemus/highlight-file-loc 44ecf2ab2f invocation-scoped fixtures show up once with --fixtures 31c5194562 Fix broken test in py3k 3c8222f1db Highlight the path of file location in error report ac215e9cff terminal: dont pretend to know the deselection reason aa145fa83e Add 'invocation' scope option to fixture docstring 76fbc6379f Fix deprecated directive in docstring 80508203b0 Merge pull request #1776 from nicoddemus/pytest-30-version eaf8d9ce19 Set version to 3.0.0.dev1 510a6083ba Merge pull request #1758 from nicoddemus/deprecated-module ffb583ae91 Merge pull request #1773 from nicoddemus/fix-freeze ae9d3bf886 Freeze docs: PyInstaller hook and wording 7b1520687e Merge pull request #1771 from nicoddemus/sort-links-changelog fd765f854c Merge pull request #1772 from nicoddemus/pdf-link ed36d627e4 Use PyInstaller for freeze test env 9a68681719 Point doc links in README to docs.pytest.org 0b8a91b858 Fix pdf links in the documentation 5565c1f4ae Sort link refs in CHANGELOG c40dcb3c18 Merge pull request #1768 from ioggstream/1609-fix-description 05728d1317 fix keep-duplicates help line. b4ad4cc46e Merge pull request #1767 from nicoddemus/merge-master c2864aba3d Merge branch 'master' into merge-master 9cf09cda7b Remove some comments and improved changelog eb5b163698 Merge branch '1609-features' of https://github.com/ioggstream/pytest into features d911bfcb8a Merge branch 'issue634-scopes' of https://github.com/Stranger6667/pytest 8f29ce26e9 Merge branch 'mark_missing_fixture_error' of https://github.com/eolo999/pytest 6c8b0a28e1 Add deprecation module to centralize deprecation messages and bits of code d72afe7e08 Fixed scope override inside metafunc.parametrize. Fixes #634 ab6aef1d1f feature: default behavior now is to ignore duplicate paths specified from the command line. Use --keep-duplicates to retain duplicate paths. a2b04d02c2 Merge pull request #1759 from Stranger6667/issue1579-invalid-class f7ad173fee Fixed collection of classes with custom ``__new__`` method d37af20527 Merge pull request #1765 from The-Compiler/1763-cleanup a309a571d9 Cleanups for #1763 e9d729bd46 drop parenthesis around GH issue number c9a2e611ce Merge pull request #1763 from tomviner/issue1210-exit-msg a24146dd3c Merge pull request #1757 from tramwaj29/improved-message-when-not-using-parametrized-variable 7862517c28 Merge pull request #1760 from blueyed/followup-pr1718-remove-newline 42adaf5a61 Fix #1210 display msg for early calls to exit 4aede6faa6 fixed conflicts d000f2536a added a test for when the indirect is just a string 0ae77be9f0 Add new target links in CHANGELOG 5c5d7e05f7 Followup to #1718: style/formatting f450e0a1db Thanks to Tom Viner for his guidance during EuroPython2016 sprint fabe8cda2f Thanking myself in CHANGELOG 3c4158ac35 Add changelog entry for this bugfix branch e00199212c Add myself to the AUTHORS e9a67e6702 Adjust test involving FixtureLookupErrorRepr 6799a47c78 Start FixtureLookupErrorRepr with an 'E' 655df7f839 Merge pull request #1755 from diegorusso/master 9891593413 Merge pull request #1754 from hartym/1749_doctest_report_format bbc7f3a631 updated changelog, resolve #1539 1704b7d265 Test case for overriding autouse fixtures 3631719050 added myself to authors f26fa5a441 changed error message for unused parametrize name 94731fc2a1 Changes variable name so it better describes what it does now. 51fa244650 Cleaner implementation of #1749. d5a70acd48 Simplify test in test_doctest_report_none_or_only_first_failure. 018acc6bae Explain why thanks yourself in the CHANGELOG f8f690de64 adds the versionadded flag in docs (#1749) e229a27e8b using @pytest.mark.parametrize instead of calling one test many times from another as suggested by @nicoddemus in pr #1754 ec7695e15d adds a bit of doctest hint on why the key and value getters are separate functions. 1a33025a76 Refactors test to be in its own class, as discussed with @nicoddemus. 922a295729 fixes changelog, I think I got it 014ebc9202 Removed bug reference in changelog because it makes the linting fail. 87ca4b95fb Separate the option keys and value to avoid importing "doctest" (and tested things like "logging") for argument parsing (fixes #1749) fd8e019cc1 Choose the doctest output format in case of failure, still work in progress as a few checks fail (related to #1749) 625b603f1f Implements an option to choose the doctest output format in case of failure. (fixes #1749) 38e0e08074 Merge pull request #1751 from javiromero/docstrings 1aab6e3bc2 Add changes to changelog. 7e37497d5a Uppercase first word in docstrings. Change to an imperative form. Add name to authors. ae0798522f Merge pull request #1711 from nicoddemus/invocation-scoped-fixtures 832ada1b44 Merge pull request #1747 from nicoddemus/line-match-stringio 4c112401c5 Log LineMatcher output in a stream instead of stderr 05f3422d7c Make monkeypatch invocation-scoped 4f2bf965cb Merge remote-tracking branch 'upstream/features' into invocation-scoped-fixtures eaa4ee3fdf Merge pull request #1746 from pytest-dev/conftest-exception-printing 6aea164b6d Add more tests for invocation scoped fixtures 20f97c3041 Small documentation improvements e0f08a73ab Merge branch 'features' into conftest-exception-printing 93aae987a2 Merge pull request #1744 from RonnyPfannschmidt/existfirst-override 1204cbade4 Merge pull request #1745 from RonnyPfannschmidt/skip-compat 9dadaa8a41 skipping plugin: remove python2.5 compat code 0403266bf0 record --exitfirst change in changelog 3fd8257c17 add test for --maxfail=NUM overiding -x 2a05c311e9 implement exitfirst as store_const option bcc58ec916 Merge pull request #1740 from RonnyPfannschmidt/float-argument 9af872a230 update changelog 61cc5c4d4e argument parsing: always warn for string types 317b3f257d optparse compatibility - add float and complex 2b9973846e Merge pull request #1736 from Avira/features 58a8150bc5 add backwards compatibility policy 0ac3eaa1db Merge pull request #1735 from flub/reinterpret-docs 0a53797fa3 Document the re-writing of plugins 8a73a2ad60 Merge pull request #1734 from nicoddemus/issue-1728-inconsistent-setup-teardown fb21493856 Merge pull request #1733 from flub/remove-reinterpret ff8fb4950e setup_* and teardown_* functions argument now optional d1852a48b7 Remove assertion reinterpretation ee374e3b80 Merge pull request #1731 from nicoddemus/improve-test-args-deprecated 3328cd2620 Make assert in test_str_args_deprecated more resilient 350ebc9167 Merge pull request #1730 from RedBeardCode/pytest-1536 24fbbbef1f Merge pull request #1641 from flub/rewrite-plugins 22bb43413f Added confcutdir in testing/test_conftest.py::test_conftest_import_order and testing/python/fixture.py::TestAutouseManagement::():: test_class_function_parametrization_finalization to avoid problems with abandoned conftest.py files in /tmp dir. Fixes #1536 691dc8bc68 Merge pull request #1727 from nicoddemus/deprecate-str-pytest-main 14af12cb7b Merge pull request #1717 from nicoddemus/nose-yield-tests-docs 51ee7f8734 Fixup things after rebase 9007e16cdf Document limitations for yield-tests in nose 02dd7d612a Remove duplicated changelog entry and formatting fix ab0b6faa5f Deprecate support for passing command-line as string to pytest.main() 1fb09d9dd5 Merge pull request #1726 from nicoddemus/warnings-displayed-by-default 1266ebec83 Merge remote-tracking branch 'upstream/features' into warnings-displayed-by-default 6e9ee2b766 Merge pull request #1724 from blueyed/followup-pr1718-remove-newline 3cfebdd7c5 funcarg_prefix_warning: remove newline 743f59afb2 Introduce pytest.register_assert_rewrite() 944da5b98a Avoid rewrite warning for inline runs a98e3cefc5 Enable re-writing of setuptools-installed plugins dd5ce96cd7 Merge pull request #1718 from blueyed/fix-funcarg_prefix_warning aeccd6b4a2 Merge pull request #1720 from nicoddemus/changelog-formatting 44c3055e79 Merge pull request #1721 from bronsen/patch-1 4a763accc5 Improve overall CHANGELOG formatting and consistency for 3.0 dfe1209d2c De-404 links in changelog 54ea27c283 Merge pull request #1719 from nicoddemus/fix-2.10-versions-in-docs f827810fa8 Fix 2.10 -> 3.0 versions in docs 15e97a7c78 Add punctuation to funcarg_prefix_warning c4f20a1834 Merge pull request #1712 from anntzer/custom-debugger 4c56c95eb8 Merge pull request #1714 from nicoddemus/deprecate-yield-tests-funcarg-prefix 7ee3dd1cb5 Add tests for custom pdb class. 458ecae1df Replace all usages of "pytest_funcarg__" for @pytest.fixture ad4125dc0d Deprecate "pytest_funcarg__" prefix to declare fixtures 5506dc700c Deprecate yield tests 0dd1c8bf14 Add test to ensure capsys and capfd error out when using "getfixturevalue" inside a test 0ca06962e9 Improve docs 2ffe354f21 Add CHANGELOG for invocation-scoped fixtures fb4da00a32 Merge remote-tracking branch 'upstream/features' into invocation-scoped-fixtures 6f68dfcc47 Merge pull request #1710 from RonnyPfannschmidt/fixture-split 6383b53ad9 Allow passing a custom Pdb subclass via --pdbcls. 8ed055efd8 Add acceptance test for invocation-scoped fixtures 775100881a Implement invocation-scoped fixtures 29289b472f Add documentation for "invocation" scoped fixture 8c49561470 split most fixture related code into own plugin 7a2058e3db Merge pull request #1709 from The-Compiler/changelog 668ebb102c Clean up changelog 293351cfd0 Merge pull request #1705 from RonnyPfannschmidt/merge-master dad6aa8a16 fix duplicate target in changelog b9a91dc112 merge from master to features f31c31a73c Merge pull request #1695 from sallner/feature-setup-show 94e4a2dd67 * implemented changes recommended by nicoddemus 0171cfa30f Fixing link to issue and creating testcase that shows that it finds the line in the stderr lines 61e605f60b Making conftest import failures easier to debug cc0920ceb1 Merge pull request #1699 from nicoddemus/404-links-on-talks-docs 067e044f97 Merge pull request #1700 from nicoddemus/split-appveyor 10c5e6fd9c Split AppVeyor test runs in multiple jobs to avoid timeout issues 8d39ce17da Fix links and removed 404 links from talks.rst 6438895a23 Fix PEP-8. b650c3c118 Implement --setup-show cli flag f7b5bb2f97 Merged branch features into features f74dd8550f Merge pull request #1692 from pytest-dev/changelog e3c43a1462 Add changelog to requirements for pytest-dev plugins 7927dff8a1 Merge pull request #1678 from RonnyPfannschmidt/drop-python30-32 1451a1ab00 remove unsupported python versions from code/source xfail 75ecd94294 Merge pull request #1689 from quodlibetor/autouse-docs 771c4539fa Document the interaction of autouse scopes 2a43237527 docs: no longer include python 3.0-3.2 in the index page 7dc8d1ab60 fix typo and remove python3.2 from readme 1e60294188 Merge pull request #1679 from eli-b/patch-1 e877e25587 drop python 3.0-3.2 support code from setup.py 21d27784eb catched -> caught ccd395ffe0 Merge pull request #1659 from RonnyPfannschmidt/failtest-586 76756c0c0b mark tests: use better name of the test for #568 bc5a8c761e Fix version in readme. 3feee0c483 Prepare 0.1 release. b9c4ecf5a8 Add MANIFEST.in. 44695ae46c Fix CHANGELOG: obestwalter appearing twice due to mergig separate PRs 7e78965c79 Merge branch 'logic_brackets' 58e558141d Remove commented out code 22c2d87633 Fix bad merge in CHANGELOG 5891061ac1 Merge pull request #1675 from kvas-it/issue-1562 48ac1a0986 Merge branch 'remove-old-entry-points' into features db9b3e9522 Merge pull request #1677 from nicoddemus/remove_cmd_options b9536608c5 Add issue and obestwalter to CHANGELOG f7a2048e96 Merge branch 'features' of https://github.com/Avira/pytest into remove-old-entry-points 7239f36466 Improve formatting in CHANGELOG 1b0dbd8c40 Move the freezing function from genscript.py to a new module freeze_support.py 0e2ebc96ff Remove deprecated cmd options b4e0fabf93 * added missing link to the referenced issue 1ce4670062 * removed tailing whitespaces 6d4cee2159 Add a test for indirect use of tuple in the assert that should not cause a warning 0db4ae15a9 Update changelog c17027e576 Warn about asserts on tuples (fixes #1562) e04d9ff80b * now showing pytest warnings summary by default. * added ``--disable-pytest-warnings` flag to let users disable the warnings summary. * extended/changed unit tests for the changes in the pytest core. e2f550156e Improve of the test output for logical expression with brackets. Fixes #925 68bed00d5b Merge pull request #1667 from fengxx/feature/override_ini_option 856e6cab75 add --override-ini option to overrides ini values 13a188fe37 Merge pull request #1647 from sallner/features 3a9e9fdf82 rephrase changes in CHANGELOG.rst 72450408ed add changes in CHANGELOG.rst 95b83958f4 add xfailing test for issue #568 2af3a7a9d7 Merge pull request #1519 from omarkohl/pytest_skip_decorator 35cd12e4de Merge pull request #1660 from hackebrot/parametrize-with-fixtures 1b6bc4d606 Add feature proposal to changelog c519b9517a Merge pull request #1663 from aostr/master eb98a8eefb Change version in issues section to pytest 3.0 only acfdd85dff Move document to proposals subfolder e0242146ec Merge pull request #1666 from pytest-dev/1564-changelog df17f862fa Merge pull request #1648 from blueyed/simplify-Argument-__repr__ 7eb1318db2 Merge pull request #1656 from pytest-dev/rm-indiegogo-links ce603dc0ea Add changelog entry for #1564 70ea3ce7f7 Merge pull request #1564 from The-Compiler/issue1479 9a5224e2f8 Renamed the pdb module and changed unit tests accordingly 32ca5cdb09 Update changelog for new fixture hooks. 891e1677b6 Remove all py.test-X* entry points. 9877bf47e3 Improve commenting for setupplan unittest. 7a3daac85b Add docs for setuponly and setupplan options. da5c579d82 Move setupplan and setuponly options to their respective modules. 032ce8baf6 Switch setuponly and setupplan options to a hook-based implementation. 05b5554cac Renamed pytest pdb to debugging which conflicts with python pdb. Combining multiple imports the "import pdb" imports the pytest module as opposed to the python debugger. 5860c609ae Remove note on scoping 526c564576 Fix rst bullet point lists 693859210a Add yielded values to function example 4f8b8c8d31 Add alternative approach that uses wrappers c6a711c2fc Add proposed solution using a module function 84f0dcecf8 Add issues section to proposal doc 757f37f445 Don't ignore ImportError with setuptools plugins 939407ef63 Simplify Argument.__repr__ b3615ac29d Update CHANGELOG with #607 and add version 3.0.0.dev1 d81f23009b Raise CollectError if pytest.skip() is called during collection 0c63762d9c Merge pull request #1654 from tomviner/issue1503/remove_collapse_false 7612b650a0 update sprint page to be past tense 62255d8000 rm global header ea5bda0898 remove links to funding campaign 77689eb486 Fixes #1503 no longer collapse false explanations 3d263c64c3 Merge pull request #1626 from tomviner/issue1625/rename-getfuncargvalue dc55551213 Start proposal for parametrize with fixtures df9918eda3 issue1625, name getfuncargvalue to getfixturevalue c6af737d4e Fix fixture parameter display when ids is a function 1a5e530b98 Fix capturing with --setup-only/--setup-plan f2bb3df310 Merge pull request #1645 from userzimmermann/sprint/addoption-check 6359e75ff8 Trivial spelling fix in runtest_setup.py 69bab4ab04 added check for already existing option names to OptionGroup.addoption() ecc97aa3b9 Use correct links in changelog. dd97a2e7c8 Merge from upstream 0dbf77e08e Add to changelog and authors. f7d50dfa91 Add a test for handling of dynamic requests for fixtures from other fixtures 7d60fcc098 Merge pull request #1643 from RonnyPfannschmidt/merge-master c8c32fd9c0 Merge with upstream 61992b4e22 Implement --setup-plan option f59d8f7720 Fix the tests (#3) 18ef7de96b merge from master again e96cd51a2a Merge remote-tracking branch 'upstream/features' into merge-master f7585c7549 Merge pull request #1635 from Avira/master 9cb851716d Merge pull request #1644 from KangarooCreativeTeam/patch-1 a2420ce051 Merge pull request #1640 from Avira/features be1dabd6a9 cache.rst: Fix wrong command used 5e0d78f4f1 Fix .format string failures on python 2.6 083f64100d merge master into features db79ed5c4d Add tests to make sure expected entry points exist (#1629) fb79fa711b Merge branch 'issue1553/diff-final-newline' of https://github.com/tomviner/pytest 1a75139f72 Fix the tests ee311e1eae Merge pull request #2 from kvas-it/features 2c5c4f3f78 Add printing of fixture dependencies 6b135c83be Initial commit. 2c6cfa42fa Disable capturing for setuponly output de9ed5e3f4 Merge pull request #1 from kvas-it/features 92bcc36266 Refactor logging of fixtures 7d19f83982 Add test for setuponly option 7d87a1b127 Add test for failing assertion 6874c3a3cc Factor setuponly code out of runtestprotocol(). 24179dc99f Merge pull request #1639 from tomviner/fix-changelog-links ec25d398a5 fix changelog links 98adf204b2 issue 1553: Include terminal newlines in diffs 499c9551c8 Implement working version of --setuponly 144dc12e55 Merge pull request #1638 from blueyed/fix-CONTRIBUTING 09389d2b20 CONTRIBUTING.rst: spelling fixes c3ee1c17bc Merge pull request #1620 from tomviner/issue460/parameterized-subrequest 4350f499b2 Merge branch 'issue1629' of https://github.com/davehunt/pytest into features 61ede096a3 Merge pull request #1631 from Avira/master 2f9d5c2586 Merge pull request #1623 from blueyed/ignore-hidden-files-in-test_pytest_collect_file f05d65dc42 Merge pull request #1622 from nicoddemus/issue-1619-conftest-assert-rewrite 573866bfad Merge remote-tracking branch 'upstream/features' into issue-1619-conftest-assert-rewrite 2305d3271d Merge pull request #1628 from omarkohl/exit_on_collection_error 406355fd10 Merge pull request #1624 from blueyed/getexecutable-skip-non-zero-returncode 393167b94f Update CHANGELOG and add Oliver Bestwalter to AUTHORS ef9dd14963 Introduce pytest command as recommended entry point Fixes #1629 2b5c2f3ed5 help the user in the rare case this assertion actually fails ede7478dcc Exit pytest on collection error (without executing tests) 819942e964 Return explicit None from rewrite hook's find_module 8b0fb47c79 Remove print left by accident 5854a71ece Issue 460: getfuncargvalue fixture w. params err 5d8d1db4df Update repository transfer instructions 9118c0222f Merge .set_config() into constructor e53e45b55c tests: getexecutable: call `--version` on all Pythons e0cb046885 Update AUTHORS and CHANGELOG 3a81d2e012 conftest files now use assertion rewriting e9d7989140 Ignore hidden files (.testmondata) in test_pytest_collect_file 54872e94b4 Fix test name typo 4f2db6c08d Merge pull request #1616 from palaviv/pytest.raises-message f3aead8e49 Merge pull request #1600 from nicoddemus/issue-1599-disable-cap-fixtures f8d4cadc3d Added versionchanged directives c29130d400 Updated documentation ca093673fb pytest.raises accept cutom message only when used as context manager d81ee9acfb Merge pull request #1597 from taschini/pyargs-fix 72bf11cbe9 Add disabled() method to capsys and capfd e6ff01ada3 CR fixes 8ddbca36c9 Add CHANGLOG entry d21886c005 pytest.raises accpets custom message 7f8e315285 Merge pull request #1615 from gnprice/deadcode c5424643f0 Merge pull request #1614 from gnprice/loop ab8b2e75a3 Simplify default pytest_runtestloop 2a3cbdf4d1 Cut a dead test helper function 308396ae3c Merge pull request #1606 from hackebrot/show-fixtures-per-test feeee2803e Merge pull request #1586 from nicoddemus/issue-1461-merge-yield-fixture 1218392413 Added taschini to AUTHORS and #1597 to CHANGELOG.rst. 4d9e293b4d Incorporated feedback (#1597). Fixed problem caused in a test on Windows by file left open by PyPy and not immediately garbage collected. e2e6e31711 Ensure that a module within a namespace package can be found by --pyargs. adc50ac72f Change format for test function locations 66e66f61e8 Merge pull request #1605 from guyzmo/issue/1604 accd962c9f Fixed issue shadowing error when missing argument on teardown_method b99aace8a9 Fix py26 by using indices in str.format bbc6c18448 Add changelog entry for new cli flag 7eea168106 Implement show_fixtures_per_test and add cli flag b47f155d74 Implement tests for --fixtures-per-test 577cce2554 Merge pull request #1593 from marscher/fix_cwd_explosion bdc29968b8 Remove dead code and simplify code in call_fixture_func ed69424917 Remove most content from yieldfixture as it is now deprecated 371fbe4388 Update CHANGELOG (yield statements in normal fixtures) fe4f23c1bf Update docs in prol of using yield statements 98acda426f Remove yieldctx variable being passed around d712428d33 Fix custom name for yield_fixtures 366879db27 Make normal fixtures work with "yield" 92323895c9 Use same python which invoked tox for "doctesting" env 09d163aa3a [exception handling] Fix case the current working directory (CWD) gets deleted during testing. 70fdab4cfa Merge pull request #1584 from mgedmin/patch-1 3ad5b9de86 Docs: config.option is deprecated 9b6dc93496 Merge pull request #1583 from nicoddemus/fix-tox-doctesting 2c4b76b754 Use same python which invoked tox for "doctesting" env 63ced4d486 List contributors alphabetically in AUTHORS 97c89e6dc3 Merge branch 'fix-all-skipped-but-none-exist' dd9c81ca26 Mention #1580 in the CHANGELOG and add Thomas Grainger to AUTHORS 74862b8f2f Don't mark empty doctest files as skipped, fixes #1578 057007fb52 Merge pull request #1582 from graingert/patch-1 2898dffb9e Fix rst-lint CHANGELOG error 7305adfdba fix another copy&paste error in the announcement list fad6266a47 finish up the version bump in __init__ i forgot b5bd4d959d merge master to features f423f08b45 prepare cangelog for the next bugfix release 5c8b0fb523 fix minor typos 978bb190a1 add release announcement a2a904466c alter changelog header 77c28825df regendoc and comment out a python2.7 example as per #1573 d3dcc2b8f1 bump regendoc python version to 3.5 85541113eb update version 7ef06822cb ignore hypothesis temp folder 289e0091de Merge pull request #1577 from pytest-dev/docs-hookwrapper-experimental 28efdebfcd Remove "experimental" status from hookwrapper fb2e7cc727 Merge pull request #1575 from hackebrot/fix-showfixtures-for-multiple-fixturedefs fb8ad714b1 Implement a test for showfixtures to show fixtures with same name 945072b89a Update changelog with --fixtures fix 0d80a9c729 Change _showfixtures_main to loop over fixturedefs 357a7c79ef Merge branch 'tomviner-issue-1496-xfail-cond-kw' 158f3cfaea merge master 82c74fe7e6 Merge pull request #1530 from RonnyPfannschmidt/fix-510 afc5c7e4f6 better message for empty argument skip 03eb9203fd remove the old empty argument triggers unlike the marker based one its not composable ae4dff0e0a add missing link to changelog d217b52508 fix #510 by adding a describing skip marker on empty parameterize 8c1be624a6 Merge pull request #1554 from RonnyPfannschmidt/merge-master 16794feaf6 Merge branch 'master' into merge-master 436e13ac25 Merge pull request #1566 from nicoddemus/fix-win32-path 9fb5ddf778 Fix shell argument split in win32 5ab5a11544 Merge pull request #1565 from tomviner/issue1544/ignore-build-dist-dirs 26b526967e merge from master again d6dfb1a393 issue 1544: norecursedirs build & dist dirs 85393d34b6 Merge pull request #1561 from AbdealiJK/ajk/1558_feature be7a86270c Merge pull request #1541 from RonnyPfannschmidt/document-1540 d4c9fa9f1a unittest.UnitTestCase: Allow __test__ for methods ec5e05834f fix typo 6f98cd6faa Merge pull request #1560 from nicoddemus/fix-py26-tox-comment 561a5fb558 Move comment in tox.ini due to recent bug in pip 47739291cf Merge pull request #1557 from adamchainz/readthedocs.io 8a39869347 Convert readthedocs link for their .org -> .io migration for hosted projects eab762ea99 Merge branch 'master' into merge-master c49863aa63 merge next chunk from master and fix changelog linting issue 01d2ff804b Merge commit '56156bb119194014129ac08c4a2c370f0b893104' into merge-master 68f658b6cc Merge commit '890c2fa555314a67a8d97a1b8ea4881a14be69c4' into merge-master 4bde70d060 Merge commit 'ec62a3c9e47c3b5b07aa1656815145ffa2882a09' into merge-master 8a94c66e68 Merge pull request #1551 from fushi/master 0d07b64571 Fixes Issue 1549 98dd2ce75c document reason for #1540 308e76e19c add xfailing test for #1540 60212e8831 Merge pull request #1537 from ben4ever/patch-1 75abfbe8d4 Fix typo in doc 6cc56b4a1b Merge pull request #1535 from palaviv/parametrize-test-ids-hook 9733127951 pytest_make_parametrize_id receive config object fdee88f086 Merge pull request #1520 from omarkohl/invalid_test_module_name 53429ed8b8 Added hook to plugin docs and new CHANGELOG record b9faf78d51 Added test_pytest_make_parametrize_id 79927428d1 Added pytest_make_parametrize_id hook 56855893ca Raise CollectError if import test module fails 52babba33e Update sprint 2016 8552860a9b Merge pull request #1532 from RibeiroAna/features f02f72e651 Update sprint2016.rst 6d661ace0a Merge pull request #1523 from MengJueM/master f51c34ef31 Add changelog b220c96bf8 Merge pull request #1526 from The-Compiler/tracebackhide aa87395c39 Use py.builtin.callable 75160547f2 Use a callable __tracebackhide__ for filtering 4c552d4ef7 Fix tests for python 2.6 b607f6728f Filter selectively with __tracebackhide__ a986b8f945 Merge pull request #1525 from The-Compiler/traceback-item c24e8e01b4 Fix TracebackItem documentation in pytest.code 1a37035d71 Add test test_absolute_win32_path 99c4b6fdc3 issue 1496 - xfail with condition keyword dd2425675b Fix a small issue about shlex.split not working well with win32 path 6a3c943ce2 Merge pull request #1514 from The-Compiler/docfix 98430a17f2 doc: Use ascii chars for file tree fe6e1b2059 Merge pull request #1506 from prusse-martin/fix-repr-tb-with-unicode 7ce5873da2 Perform a "unicode aware" check for maximum recursion depth error 0eb80bcb5a Merge pull request #1504 from guettli/patch-1 fb45f82840 Hudson -> Jenkins 0f7aeafe7c Merge pull request #1486 from roolebo/fix-issue-138 e3bc6faa2b Merge pull request #1470 from ceridwen/features 909d72b474 Merge pull request #1502 from omarkohl/excinfo_match 0c38aacc33 Add self to AUTHORS c578226d43 Implement ExceptionInfo.match() to match regexp on str(exception) 23a8e2b469 Add .hypothesis to .gitignore and try an older version of Hypothesis for 2.6 08671fcf4a Fix the changelog and dependencies for tox 491b30c5d9 Add Hypothesis test for _idval and fix bug it found b631fc0bc1 Fix test_escaped_parametrized_names_xml 5af5ba11d3 Merge pull request #1500 from tgoodlet/lstrip-keywordexpr 053c052190 Always lstrip() keyword expression 9b438d56e8 Fix a test_unicode_idval_python2 (now test_unicode_idval) and associated string handling on Python 3 56156bb119 Merge pull request #1490 from omarkohl/get_issues e048315d9b Adapt get_issues.py script for GitHub (instead of Bitbucket) 89df701ae9 Comment a workaround for #1485. fed89ef549 Merge pull request #1474 from palaviv/improve-idmaker-duplicate-names 3ffce6ae4a Added thanks and PR links in changlog c66aedfa65 checking first there are duplciates ids before changing to unique names 3155d0ca9c Merge pull request #1475 from nicoddemus/classifier-docs 53d319144d Mention Pytest::Framework PyPI classifier on docs 412042d987 added entry to CHANGELOG b8c15a0215 improved idmaker name selection in case of duplicate ids 1f46015de5 Merge remote-tracking branch 'upstream/features' into features 6ddfd60ce7 Merge pull request #1472 from pquentin/master 653a53226a Add strict parameter to xfail marker doc 1fbd19b8cb Fix pytest.mark.skip mark when used in strict mode 4405dd0ffe Escape both bytes and unicode strings for "ids" in Metafunc.parametrize 890c2fa555 Merge pull request #1469 from omarkohl/doc-fixes 725290a8ab Add 'Minor doc fixes' to CHANGELOG da1045151f Merge pull request #1468 from palaviv/allow-none-as-parametrized-test-id 98c707561c Document --full-trace option and KeyboardInterrupt a341dddc74 Replace --fulltrace with --full-trace in doc dd384f7f78 Added changlog entries 7885e43b78 Merge remote-tracking branch 'upstream/features' into allow-none-as-parametrized-test-id 32f44ce2fd updated parametrize documentation 16e49d96d1 Replace --traceconfig with --trace-config in doc ec62a3c9e4 Fix minor typo in 'writing plugins' doc f70ed83479 Fix 'test grouping by fixture instances' doc example dff914cadd Add short explanation and link to yield fixtures in fixture doc 3135463573 No longer refer to the 'yield fixture mechanism' as experimental (doc) 266b53dfc2 Add Jenkins xUnit Plugin link to doc bdb3581a52 Fix minor mistake in test discovery doc 27b62740e3 Fix minor mistake in usage doc (pkg instead of pypkg) 52bc2f8616 Update authors & changelog a736e26734 Merge remote-tracking branch 'pytest-dev/master' into fix-issue-138 fbc5ba08d9 Fix issue #138 4b0237c8ee added test for unique names when recievieng identical ids in parametrize 877ca5a0bf added test for None in paramtrized ids list a8cfd54871 added test for None in idmaker be1954afbc allow None to be passed in ids list for specific test and recieve idmaker name for test 2cfcf12d09 Merge pull request #1466 from nicoddemus/merge-master-into-features 5fcce8a7d6 Merge branch 'master' into merge-master-into-features a70e92777f Bump version to 2.9.2.dev1 3c011c05db Merge pull request #1460 from nicoddemus/release-2.9.1 ceacc12b52 Merge pull request #1454 from tareqalayan/add-global-properties-node fa6acdcfd4 junitxml: add properties node in testsuite level 5fd82078ad Merge pull request #1441 from kalekundert/features 5ceee08590 Fix CHANGELOG merge conflicts. 0dcc862a56 Fix some typos in the documentation. 9e7206a1cf Fix a few stylistic issues. 0cacdef6c5 Merge pull request #1444 from novas0x2a/fixture-custom-name 861265411f Add "thanks" line to the CHANGELOG. 916c0a8b36 Fix Decimal() and __ne__() errors. 078448008c Discuss alternative float comparison algorithms. 42a7e0488d Properly handle inf, nan, and built-in numeric types. 9577120592 Allow custom fixture names for fixtures 7d155bd3cf Fix sys.version_info errors. 6a902924f8 Fix trailing whitespace errors. c9c73b8d8e Fix zero-length field name error in python2.6 4d0f066db7 Add approx() to the CHANGELOG. 5dab0954a0 Add approx() to the Sphinx docs. b8a8382c2c Reduce the default absolute error threshold to 1e-12. bf97d5b817 Use the plus/minus unicode symbol in the repr string. dd28e28b34 Make a few stylistic improvements. 6f5e1e386a Add a convenient and correct way to compare floats. c2b9196a7c Merge remote-tracking branch 'upstream/master' into features 4a76b2e9b6 Merge branch 'doctest_namespace_inject' into features 28937a5cd9 Add versionadded directive to doctest_namespace section 2e02a1c370 Give proper credit for PR #1428 6dd2ff5332 Correct indentation in documentation 891e029518 Add a new doctest_namespace fixture 5a2500800d Add 2.10.0.dev1 to CHANGELOG c8c5a416ef Bump version to 2.10.0.dev1 git-subtree-dir: tools/pytest git-subtree-split: 369c711f140fc5fac051a8ea1749a18874de268f --- .coveragerc | 5 +- .github/ISSUE_TEMPLATE.md | 2 +- .github/PULL_REQUEST_TEMPLATE.md | 15 +- .gitignore | 5 + .travis.yml | 56 +- AUTHORS | 118 +- CHANGELOG.rst | 1910 +++++++++++++-- CONTRIBUTING.rst | 149 +- HOWTORELEASE.rst | 105 +- ISSUES.txt | 365 --- LICENSE | 2 +- MANIFEST.in | 34 - README.rst | 63 +- _pytest/__init__.py | 10 +- _pytest/_argcomplete.py | 18 +- _pytest/_code/__init__.py | 4 +- _pytest/_code/_py2traceback.py | 6 +- _pytest/_code/code.py | 329 ++- _pytest/_code/source.py | 92 +- _pytest/_pluggy.py | 11 - _pytest/assertion/__init__.py | 166 +- _pytest/assertion/reinterpret.py | 407 ---- _pytest/assertion/rewrite.py | 199 +- _pytest/assertion/truncate.py | 102 + _pytest/assertion/util.py | 86 +- _pytest/cacheprovider.py | 71 +- _pytest/capture.py | 383 ++- _pytest/compat.py | 322 +++ _pytest/config.py | 554 +++-- _pytest/{pdb.py => debugging.py} | 52 +- _pytest/deprecated.py | 52 + _pytest/doctest.py | 139 +- _pytest/fixtures.py | 1147 +++++++++ _pytest/freeze_support.py | 43 + _pytest/genscript.py | 132 -- _pytest/helpconfig.py | 97 +- _pytest/hookspec.py | 176 +- _pytest/impl | 254 -- _pytest/junitxml.py | 110 +- _pytest/logging.py | 337 +++ _pytest/main.py | 393 ++-- _pytest/mark.py | 329 ++- _pytest/monkeypatch.py | 42 +- _pytest/nodes.py | 37 + _pytest/nose.py | 27 +- _pytest/outcomes.py | 147 ++ _pytest/pastebin.py | 20 +- _pytest/pytester.py | 373 +-- _pytest/python.py | 2071 ++++------------- _pytest/python_api.py | 619 +++++ _pytest/recwarn.py | 177 +- _pytest/resultlog.py | 21 +- _pytest/runner.py | 257 +- _pytest/setuponly.py | 74 + _pytest/setupplan.py | 25 + _pytest/skipping.py | 273 ++- _pytest/standalonetemplate.py | 89 - _pytest/terminal.py | 314 ++- _pytest/tmpdir.py | 15 +- _pytest/unittest.py | 96 +- _pytest/vendored_packages/README.md | 13 - _pytest/vendored_packages/__init__.py | 0 .../pluggy-0.3.1.dist-info/DESCRIPTION.rst | 10 - .../pluggy-0.3.1.dist-info/METADATA | 39 - .../pluggy-0.3.1.dist-info/RECORD | 8 - .../pluggy-0.3.1.dist-info/WHEEL | 6 - .../pluggy-0.3.1.dist-info/metadata.json | 1 - .../pluggy-0.3.1.dist-info/pbr.json | 1 - .../pluggy-0.3.1.dist-info/top_level.txt | 1 - _pytest/vendored_packages/pluggy.py | 777 ------- _pytest/warnings.py | 96 + appveyor.yml | 40 +- changelog/2920.bugfix | 1 + changelog/2949.trivial | 1 + changelog/2956.bugfix | 1 + changelog/2957.bugfix | 1 + changelog/2963.doc | 1 + changelog/2971.bugfix | 1 + changelog/_template.rst | 40 + doc/en/Makefile | 30 +- doc/en/_getdoctarget.py | 16 - doc/en/_templates/globaltoc.html | 1 + doc/en/_templates/layout.html | 14 - doc/en/_templates/links.html | 7 +- doc/en/adopt.rst | 4 + doc/en/announce/index.rst | 24 +- doc/en/announce/release-2.0.2.rst | 4 +- doc/en/announce/release-2.0.3.rst | 4 +- doc/en/announce/release-2.2.1.rst | 2 +- doc/en/announce/release-2.2.4.rst | 2 +- doc/en/announce/release-2.3.0.rst | 4 +- doc/en/announce/release-2.3.2.rst | 2 +- doc/en/announce/release-2.3.3.rst | 6 +- doc/en/announce/release-2.3.5.rst | 8 +- doc/en/announce/release-2.4.0.rst | 4 +- doc/en/announce/release-2.5.0.rst | 4 +- doc/en/announce/release-2.5.2.rst | 2 +- doc/en/announce/release-2.6.3.rst | 4 +- doc/en/announce/release-2.7.0.rst | 2 +- doc/en/announce/release-2.7.1.rst | 2 +- doc/en/announce/release-2.9.0.rst | 4 +- doc/en/announce/release-2.9.1.rst | 2 + doc/en/announce/release-2.9.2.rst | 78 + doc/en/announce/release-3.0.0.rst | 82 + doc/en/announce/release-3.0.1.rst | 26 + doc/en/announce/release-3.0.2.rst | 24 + doc/en/announce/release-3.0.3.rst | 27 + doc/en/announce/release-3.0.4.rst | 29 + doc/en/announce/release-3.0.5.rst | 27 + doc/en/announce/release-3.0.6.rst | 33 + doc/en/announce/release-3.0.7.rst | 33 + doc/en/announce/release-3.1.0.rst | 61 + doc/en/announce/release-3.1.1.rst | 23 + doc/en/announce/release-3.1.2.rst | 23 + doc/en/announce/release-3.1.3.rst | 23 + doc/en/announce/release-3.2.0.rst | 48 + doc/en/announce/release-3.2.1.rst | 22 + doc/en/announce/release-3.2.2.rst | 28 + doc/en/announce/release-3.2.3.rst | 23 + doc/en/announce/release-3.2.4.rst | 36 + doc/en/announce/release-3.2.5.rst | 18 + doc/en/announce/release-3.3.0.rst | 50 + doc/en/announce/sprint2016.rst | 75 +- doc/en/assert.rst | 132 +- doc/en/backwards-compatibility.rst | 105 + doc/en/bash-completion.rst | 6 +- doc/en/builtin.rst | 83 +- doc/en/cache.rst | 120 +- doc/en/capture.rst | 62 +- doc/en/conf.py | 13 +- doc/en/contact.rst | 5 +- doc/en/contents.rst | 38 +- doc/en/customize.rst | 185 +- doc/en/development_guide.rst | 108 + doc/en/doctest.rst | 82 +- doc/en/example/assertion/failure_demo.py | 12 +- .../assertion/test_setup_flow_example.py | 2 +- doc/en/example/attic.rst | 6 +- doc/en/example/costlysetup/conftest.py | 6 +- doc/en/example/index.rst | 4 +- doc/en/example/layout1/setup.cfg | 4 - doc/en/example/markers.rst | 330 +-- doc/en/example/multipython.py | 4 +- doc/en/example/nonpython.rst | 44 +- doc/en/example/nonpython/conftest.py | 4 +- doc/en/example/parametrize.rst | 187 +- doc/en/example/pythoncollection.py | 4 +- doc/en/example/pythoncollection.rst | 141 +- doc/en/example/reportingdemo.rst | 190 +- doc/en/example/simple.rst | 468 ++-- doc/en/example/special.rst | 8 +- doc/en/existingtestsuite.rst | 34 + doc/en/faq.rst | 29 +- doc/en/feedback.rst | 8 - doc/en/fixture.rst | 445 ++-- doc/en/funcarg_compare.rst | 15 +- doc/en/genapi.py | 4 +- doc/en/getting-started.rst | 101 +- doc/en/goodpractices.rst | 199 +- doc/en/historical-notes.rst | 177 ++ doc/en/index.rst | 115 +- doc/en/license.rst | 2 +- doc/en/links.inc | 4 +- doc/en/logging.rst | 192 ++ doc/en/mark.rst | 1 + doc/en/monkeypatch.rst | 33 +- doc/en/nose.rst | 30 +- doc/en/overview.rst | 13 - doc/en/parametrize.rst | 96 +- doc/en/plugins.rst | 17 +- doc/en/projects.rst | 4 +- .../proposals/parametrize_with_fixtures.rst | 160 ++ doc/en/pythonpath.rst | 76 + doc/en/recwarn.rst | 131 +- doc/en/requirements.txt | 3 + doc/en/setup.rst | 10 - doc/en/skipping.rst | 323 +-- doc/en/status.rst | 5 - doc/en/talks.rst | 55 +- doc/en/test/attic.rst | 4 +- doc/en/test/mission.rst | 2 +- doc/en/test/plugin/cov.rst | 14 +- doc/en/test/plugin/coverage.rst | 2 +- doc/en/test/plugin/figleaf.rst | 2 +- doc/en/test/plugin/genscript.rst | 28 - doc/en/test/plugin/helpconfig.rst | 4 +- doc/en/test/plugin/links.rst | 2 - doc/en/test/plugin/nose.rst | 4 +- doc/en/test/plugin/terminal.rst | 4 +- doc/en/test/plugin/xdist.rst | 12 +- doc/en/tmpdir.rst | 20 +- doc/en/unittest.rst | 165 +- doc/en/usage.rst | 253 +- doc/en/warnings.rst | 299 +++ doc/en/writing_plugins.rst | 252 +- doc/en/xdist.rst | 197 -- doc/en/xunit_setup.rst | 37 +- doc/en/yieldfixture.rst | 104 +- extra/get_issues.py | 70 +- plugin-test.sh | 20 - pyproject.toml | 35 + pytest.py | 71 +- requirements-docs.txt | 3 - runtox.py | 8 - scripts/call-tox.bat | 8 + scripts/check-rst.py | 11 + scripts/install-pypy.bat | 6 + setup.cfg | 7 + setup.py | 98 +- tasks/__init__.py | 12 + tasks/generate.py | 162 ++ tasks/release.minor.rst | 27 + tasks/release.patch.rst | 17 + tasks/requirements.txt | 5 + testing/acceptance_test.py | 233 +- testing/code/test_code.py | 76 +- testing/code/test_excinfo.py | 608 +++-- testing/code/test_source.py | 131 +- testing/code/test_source_multiline_block.py | 26 + testing/cx_freeze/install_cx_freeze.py | 64 - testing/cx_freeze/runtests_setup.py | 15 - testing/cx_freeze/tox_run.py | 15 - testing/deprecated_test.py | 114 + testing/freeze/.gitignore | 3 + testing/freeze/create_executable.py | 12 + .../{cx_freeze => freeze}/runtests_script.py | 18 +- .../tests/test_doctest.txt | 0 .../tests/test_trivial.py | 13 +- testing/freeze/tox_run.py | 12 + testing/logging/test_fixture.py | 70 + testing/logging/test_reporting.py | 398 ++++ testing/python/approx.py | 392 ++++ testing/python/collect.py | 324 ++- testing/python/fixture.py | 1294 ++++++---- testing/python/integration.py | 57 +- testing/python/metafunc.py | 755 ++++-- testing/python/raises.py | 76 +- testing/python/setup_only.py | 243 ++ testing/python/setup_plan.py | 19 + testing/python/show_fixtures_per_test.py | 158 ++ testing/python/test_deprecations.py | 22 + testing/test_argcomplete.py | 35 +- testing/test_assertinterpret.py | 274 --- testing/test_assertion.py | 614 ++++- testing/test_assertrewrite.py | 378 ++- testing/test_cache.py | 235 +- testing/test_capture.py | 348 ++- testing/test_collection.py | 328 ++- testing/test_compat.py | 101 + testing/test_config.py | 476 +++- testing/test_conftest.py | 171 +- testing/test_doctest.py | 330 ++- testing/test_entry_points.py | 14 + testing/test_genscript.py | 51 - testing/test_helpconfig.py | 16 +- testing/test_junitxml.py | 328 ++- testing/test_mark.py | 308 ++- testing/test_modimport.py | 25 + testing/test_monkeypatch.py | 31 +- testing/test_nodes.py | 18 + testing/test_nose.py | 95 +- testing/test_parseopt.py | 92 +- testing/test_pastebin.py | 16 +- testing/test_pdb.py | 159 +- testing/test_pluginmanager.py | 129 +- testing/test_pytester.py | 33 +- testing/test_recwarn.py | 183 +- testing/test_resultlog.py | 34 +- testing/test_runner.py | 311 ++- testing/test_runner_xunit.py | 99 +- testing/test_session.py | 79 +- testing/test_skipping.py | 220 +- testing/test_terminal.py | 309 ++- testing/test_tmpdir.py | 11 +- testing/test_unittest.py | 287 ++- testing/test_warnings.py | 258 ++ tox.ini | 258 +- 277 files changed, 23069 insertions(+), 10839 deletions(-) delete mode 100644 ISSUES.txt delete mode 100644 MANIFEST.in delete mode 100644 _pytest/_pluggy.py delete mode 100644 _pytest/assertion/reinterpret.py create mode 100644 _pytest/assertion/truncate.py create mode 100644 _pytest/compat.py rename _pytest/{pdb.py => debugging.py} (66%) create mode 100644 _pytest/deprecated.py create mode 100644 _pytest/fixtures.py create mode 100644 _pytest/freeze_support.py delete mode 100755 _pytest/genscript.py delete mode 100644 _pytest/impl create mode 100644 _pytest/logging.py create mode 100644 _pytest/nodes.py create mode 100644 _pytest/outcomes.py create mode 100644 _pytest/python_api.py create mode 100644 _pytest/setuponly.py create mode 100644 _pytest/setupplan.py delete mode 100755 _pytest/standalonetemplate.py delete mode 100644 _pytest/vendored_packages/README.md delete mode 100644 _pytest/vendored_packages/__init__.py delete mode 100644 _pytest/vendored_packages/pluggy-0.3.1.dist-info/DESCRIPTION.rst delete mode 100644 _pytest/vendored_packages/pluggy-0.3.1.dist-info/METADATA delete mode 100644 _pytest/vendored_packages/pluggy-0.3.1.dist-info/RECORD delete mode 100644 _pytest/vendored_packages/pluggy-0.3.1.dist-info/WHEEL delete mode 100644 _pytest/vendored_packages/pluggy-0.3.1.dist-info/metadata.json delete mode 100644 _pytest/vendored_packages/pluggy-0.3.1.dist-info/pbr.json delete mode 100644 _pytest/vendored_packages/pluggy-0.3.1.dist-info/top_level.txt delete mode 100644 _pytest/vendored_packages/pluggy.py create mode 100644 _pytest/warnings.py create mode 100644 changelog/2920.bugfix create mode 100644 changelog/2949.trivial create mode 100644 changelog/2956.bugfix create mode 100644 changelog/2957.bugfix create mode 100644 changelog/2963.doc create mode 100644 changelog/2971.bugfix create mode 100644 changelog/_template.rst delete mode 100755 doc/en/_getdoctarget.py create mode 100644 doc/en/announce/release-2.9.2.rst create mode 100644 doc/en/announce/release-3.0.0.rst create mode 100644 doc/en/announce/release-3.0.1.rst create mode 100644 doc/en/announce/release-3.0.2.rst create mode 100644 doc/en/announce/release-3.0.3.rst create mode 100644 doc/en/announce/release-3.0.4.rst create mode 100644 doc/en/announce/release-3.0.5.rst create mode 100644 doc/en/announce/release-3.0.6.rst create mode 100644 doc/en/announce/release-3.0.7.rst create mode 100644 doc/en/announce/release-3.1.0.rst create mode 100644 doc/en/announce/release-3.1.1.rst create mode 100644 doc/en/announce/release-3.1.2.rst create mode 100644 doc/en/announce/release-3.1.3.rst create mode 100644 doc/en/announce/release-3.2.0.rst create mode 100644 doc/en/announce/release-3.2.1.rst create mode 100644 doc/en/announce/release-3.2.2.rst create mode 100644 doc/en/announce/release-3.2.3.rst create mode 100644 doc/en/announce/release-3.2.4.rst create mode 100644 doc/en/announce/release-3.2.5.rst create mode 100644 doc/en/announce/release-3.3.0.rst create mode 100644 doc/en/backwards-compatibility.rst create mode 100644 doc/en/development_guide.rst delete mode 100644 doc/en/example/layout1/setup.cfg create mode 100644 doc/en/existingtestsuite.rst delete mode 100644 doc/en/feedback.rst create mode 100644 doc/en/historical-notes.rst create mode 100644 doc/en/logging.rst delete mode 100644 doc/en/overview.rst create mode 100644 doc/en/proposals/parametrize_with_fixtures.rst create mode 100644 doc/en/pythonpath.rst create mode 100644 doc/en/requirements.txt delete mode 100644 doc/en/setup.rst delete mode 100644 doc/en/status.rst delete mode 100644 doc/en/test/plugin/genscript.rst create mode 100644 doc/en/warnings.rst delete mode 100644 doc/en/xdist.rst delete mode 100644 plugin-test.sh create mode 100644 pyproject.toml delete mode 100644 requirements-docs.txt delete mode 100644 runtox.py create mode 100644 scripts/call-tox.bat create mode 100644 scripts/check-rst.py create mode 100644 scripts/install-pypy.bat create mode 100644 tasks/__init__.py create mode 100644 tasks/generate.py create mode 100644 tasks/release.minor.rst create mode 100644 tasks/release.patch.rst create mode 100644 tasks/requirements.txt create mode 100644 testing/code/test_source_multiline_block.py delete mode 100644 testing/cx_freeze/install_cx_freeze.py delete mode 100644 testing/cx_freeze/runtests_setup.py delete mode 100644 testing/cx_freeze/tox_run.py create mode 100644 testing/deprecated_test.py create mode 100644 testing/freeze/.gitignore create mode 100644 testing/freeze/create_executable.py rename testing/{cx_freeze => freeze}/runtests_script.py (82%) rename testing/{cx_freeze => freeze}/tests/test_doctest.txt (100%) rename testing/{cx_freeze => freeze}/tests/test_trivial.py (65%) create mode 100644 testing/freeze/tox_run.py create mode 100644 testing/logging/test_fixture.py create mode 100644 testing/logging/test_reporting.py create mode 100644 testing/python/approx.py create mode 100644 testing/python/setup_only.py create mode 100644 testing/python/setup_plan.py create mode 100644 testing/python/show_fixtures_per_test.py create mode 100644 testing/python/test_deprecations.py delete mode 100644 testing/test_assertinterpret.py create mode 100644 testing/test_compat.py create mode 100644 testing/test_entry_points.py delete mode 100644 testing/test_genscript.py create mode 100644 testing/test_modimport.py create mode 100644 testing/test_nodes.py create mode 100644 testing/test_warnings.py diff --git a/.coveragerc b/.coveragerc index 27db64e09c125b7..61ff66749dc805d 100644 --- a/.coveragerc +++ b/.coveragerc @@ -1,7 +1,4 @@ [run] -omit = +omit = # standlonetemplate is read dynamically and tested by test_genscript *standalonetemplate.py - # oldinterpret could be removed, as it is no longer used in py26+ - *oldinterpret.py - vendored_packages diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index bc62e8a3f7d197f..fbcbb16fc351de7 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -4,5 +4,5 @@ Here's a quick checklist in what to include: - [ ] Include a detailed description of the bug or suggestion - [ ] `pip list` of the virtual environment you are using -- [ ] py.test and operating system versions +- [ ] pytest and operating system versions - [ ] Minimal example if possible diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index d09edce43d3ecde..bf9fc199f59fde8 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -2,7 +2,14 @@ Thanks for submitting a PR, your contribution is really appreciated! Here's a quick checklist that should be present in PRs: -- [ ] Target: for bug or doc fixes, target `master`; for new features, target `features` -- [ ] Make sure to include one or more tests for your change -- [ ] Add yourself to `AUTHORS` -- [ ] Add a new entry to the `CHANGELOG` (choose any open position to avoid merge conflicts with other PRs) +- [ ] Add a new news fragment into the changelog folder + * name it `$issue_id.$type` for example (588.bug) + * if you don't have an issue_id change it to the pr id after creating the pr + * ensure type is one of `removal`, `feature`, `bugfix`, `vendor`, `doc` or `trivial` + * Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files." +- [ ] Target: for `bugfix`, `vendor`, `doc` or `trivial` fixes, target `master`; for removals or features target `features`; +- [ ] Make sure to include reasonable tests for your change if necessary + +Unless your change is a trivial or a documentation fix (e.g., a typo or reword of a small section) please: + +- [ ] Add yourself to `AUTHORS`, in alphabetical order; diff --git a/.gitignore b/.gitignore index e4355b859ccc7d1..3b7ec9facf235eb 100644 --- a/.gitignore +++ b/.gitignore @@ -16,7 +16,11 @@ include/ *.class *.orig *~ +.hypothesis/ +# autogenerated +_pytest/_version.py +# setuptools .eggs/ doc/*/_build @@ -32,3 +36,4 @@ env/ .coverage .ropeproject .idea +.hypothesis diff --git a/.travis.yml b/.travis.yml index 3a8f36e95f8f5bb..938391cde09d8e8 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,33 +1,49 @@ sudo: false language: python python: - - '3.5' + - '3.6' # command to install dependencies -install: "pip install -U tox" +install: + - pip install --upgrade --pre tox # # command to run tests env: matrix: # coveralls is not listed in tox's envlist, but should run in travis - - TESTENV=coveralls + - TOXENV=coveralls # note: please use "tox --listenvs" to populate the build matrix below - - TESTENV=linting - - TESTENV=py26 - - TESTENV=py27 - - TESTENV=py33 - - TESTENV=py34 - - TESTENV=py35 - - TESTENV=pypy - - TESTENV=py27-pexpect - - TESTENV=py27-xdist - - TESTENV=py27-trial - - TESTENV=py35-pexpect - - TESTENV=py35-xdist - - TESTENV=py35-trial - - TESTENV=py27-nobyte - - TESTENV=doctesting - - TESTENV=py27-cxfreeze + - TOXENV=linting + - TOXENV=py27 + - TOXENV=py34 + - TOXENV=py36 + - TOXENV=py27-pexpect + - TOXENV=py27-xdist + - TOXENV=py27-trial + - TOXENV=py27-numpy + - TOXENV=py27-pluggymaster + - TOXENV=py36-pexpect + - TOXENV=py36-xdist + - TOXENV=py36-trial + - TOXENV=py36-numpy + - TOXENV=py36-pluggymaster + - TOXENV=py27-nobyte + - TOXENV=doctesting + - TOXENV=docs -script: tox --recreate -e $TESTENV +matrix: + include: + - env: TOXENV=pypy + python: 'pypy-5.4' + - env: TOXENV=py35 + python: '3.5' + - env: TOXENV=py35-freeze + python: '3.5' + - env: TOXENV=py37 + python: 'nightly' + allow_failures: + - env: TOXENV=py37 + python: 'nightly' + +script: tox --recreate notifications: irc: diff --git a/AUTHORS b/AUTHORS index dfc0a542ec52e19..44ae6aa43ab073d 100644 --- a/AUTHORS +++ b/AUTHORS @@ -3,85 +3,187 @@ merlinux GmbH, Germany, office at merlinux eu Contributors include:: +Abdeali JK Abhijeet Kasurde +Ahn Ki-Wook +Alexander Johnson +Alexei Kozlenok Anatoly Bubenkoff +Andras Tim Andreas Zeidler +Andrzej Ostrowski Andy Freeland Anthon van der Neut +Anthony Sottile +Antony Lee Armin Rigo Aron Curzon Aviv Palivoda +Barney Gale +Ben Webb Benjamin Peterson +Bernard Pratz Bob Ippolito Brian Dorsey Brian Okken Brianna Laugher Bruno Oliveira +Cal Leeming Carl Friedrich Bolz +Ceridwen Charles Cloud +Charnjit SiNGH (CCSJ) Chris Lamb +Christian Boelsen Christian Theunert Christian Tismer Christopher Gilling Daniel Grana Daniel Hahler Daniel Nuri +Daniel Wandschneider +Danielle Jenkins Dave Hunt +David Díaz-Barquero David Mohr David Vierra +Daw-Ran Liou +Denis Kirisov +Diego Russo +Dmitry Dygalo +Dmitry Pribysh +Duncan Betts Edison Gustavo Muenz +Edoardo Batini Eduardo Schettino -Endre Galaczi +Eli Boyarski Elizaveta Shashkova +Endre Galaczi Eric Hunsberger Eric Siegerman Erik M. Bray +Feng Ma Florian Bruhin Floris Bruynooghe Gabriel Reis +George Kussumoto Georgy Dyuldin Graham Horler +Greg Price Grig Gheorghiu +Grigorii Eremeev (budulianin) Guido Wesdorp Harald Armin Massa +Hugo van Kemenade +Hui Wang (coldnight) Ian Bicking Jaap Broekhuizen Jan Balster Janne Vanhala Jason R. Coombs +Javier Domingo Cansino +Javier Romero +Jeff Widman +John Eddie Ayson +John Towler +Jon Sonesen +Jonas Obrist +Jordan Guymon +Jordan Moldow Joshua Bronson Jurko Gospodnetić +Justyna Janczyszyn +Kale Kundert Katarzyna Jachim Kevin Cox +Kodi B. Arfer +Lawrence Mitchell Lee Kamentsky +Lev Maximov +Llandy Riveron Del Risco +Loic Esteve Lukas Bednar +Luke Murphy Maciek Fijalkowski Maho +Maik Figura +Mandeep Bhutani +Manuel Krebber Marc Schlaich +Marcin Bachry Mark Abramowitz Markus Unterwaditzer Martijn Faassen +Martin Altmayer +Martin K. Scherer +Martin Prusse +Mathieu Clabaut Matt Bachmann +Matt Duck +Matt Williams +Matthias Hafner +Maxim Filipenko +mbyt Michael Aquilina Michael Birtwell Michael Droettboom +Michael Seifert +Michal Wajszczuk +Mihai Capotă +Mike Lundy +Nathaniel Waisbrot +Ned Batchelder +Neven Mundar Nicolas Delaby +Oleg Pidsadnyi +Oliver Bestwalter +Omar Kohl +Omer Hadari +Patrick Hayes +Paweł Adamczak Pieter Mulder Piotr Banaszkiewicz Punyashloka Biswal +Quentin Pradet Ralf Schmitt +Ran Benita Raphael Pierzina +Raquel Alegre +Ravi Chandra +Roberto Polli +Romain Dorgueil +Roman Bolshakov Ronny Pfannschmidt Ross Lawley +Russel Winder Ryan Wooden +Samuel Dion-Girardeau Samuele Pedroni +Segev Finer +Simon Gomizelj +Skylar Downes +Srinivas Reddy Thatiparthy +Stefan Farmbauer +Stefan Zimmermann +Stefano Taschini +Steffen Allner +Stephan Obermann +Tarcisio Fischer +Tareq Alayan +Ted Xiao +Thomas Grainger +Thomas Hisch +Tom Dalton Tom Viner Trevor Bekolay +Tyler Goodlet +Vasily Kuznetsov +Victor Uriarte +Vidar T. Fauske +Vitaly Lashmanov +Vlad Dragos Wouter van Ackooy -David Díaz-Barquero -Eric Hunsberger -Simon Gomizelj -Russel Winder -Ben Webb -Alexei Kozlenok -Cal Leeming +Xuan Luong +Xuecong Liao +Zoltán Máté +Roland Puntaier diff --git a/CHANGELOG.rst b/CHANGELOG.rst index f18f646f00849b9..7e7bfaf04415d83 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -1,5 +1,1608 @@ -2.9.1 -===== +.. + You should *NOT* be adding new change log entries to this file, this + file is managed by towncrier. You *may* edit previous change logs to + fix problems like typo corrections or such. + To add a new change log entry, please see + https://pip.pypa.io/en/latest/development/#adding-a-news-entry + we named the news folder changelog + +.. towncrier release notes start + +Pytest 3.3.0 (2017-11-23) +========================= + +Deprecations and Removals +------------------------- + +- Pytest no longer supports Python **2.6** and **3.3**. Those Python versions + are EOL for some time now and incur maintenance and compatibility costs on + the pytest core team, and following up with the rest of the community we + decided that they will no longer be supported starting on this version. Users + which still require those versions should pin pytest to ``<3.3``. (`#2812 + `_) + +- Remove internal ``_preloadplugins()`` function. This removal is part of the + ``pytest_namespace()`` hook deprecation. (`#2636 + `_) + +- Internally change ``CallSpec2`` to have a list of marks instead of a broken + mapping of keywords. This removes the keywords attribute of the internal + ``CallSpec2`` class. (`#2672 + `_) + +- Remove ParameterSet.deprecated_arg_dict - its not a public api and the lack + of the underscore was a naming error. (`#2675 + `_) + +- Remove the internal multi-typed attribute ``Node._evalskip`` and replace it + with the boolean ``Node._skipped_by_mark``. (`#2767 + `_) + +- The ``params`` list passed to ``pytest.fixture`` is now for + all effects considered immutable and frozen at the moment of the ``pytest.fixture`` + call. Previously the list could be changed before the first invocation of the fixture + allowing for a form of dynamic parametrization (for example, updated from command-line options), + but this was an unwanted implementation detail which complicated the internals and prevented + some internal cleanup. See issue `#2959 `_ + for details and a recommended workaround. + +Features +-------- + +- ``pytest_fixture_post_finalizer`` hook can now receive a ``request`` + argument. (`#2124 `_) + +- Replace the old introspection code in compat.py that determines the available + arguments of fixtures with inspect.signature on Python 3 and + funcsigs.signature on Python 2. This should respect ``__signature__`` + declarations on functions. (`#2267 + `_) + +- Report tests with global ``pytestmark`` variable only once. (`#2549 + `_) + +- Now pytest displays the total progress percentage while running tests. The + previous output style can be set by configuring the ``console_output_style`` + setting to ``classic``. (`#2657 `_) + +- Match ``warns`` signature to ``raises`` by adding ``match`` keyword. (`#2708 + `_) + +- Pytest now captures and displays output from the standard `logging` module. + The user can control the logging level to be captured by specifying options + in ``pytest.ini``, the command line and also during individual tests using + markers. Also, a ``caplog`` fixture is available that enables users to test + the captured log during specific tests (similar to ``capsys`` for example). + For more information, please see the `logging docs + `_. This feature was + introduced by merging the popular `pytest-catchlog + `_ plugin, thanks to `Thomas Hisch + `_. Be advised that during the merging the + backward compatibility interface with the defunct ``pytest-capturelog`` has + been dropped. (`#2794 `_) + +- Add ``allow_module_level`` kwarg to ``pytest.skip()``, enabling to skip the + whole module. (`#2808 `_) + +- Allow setting ``file_or_dir``, ``-c``, and ``-o`` in PYTEST_ADDOPTS. (`#2824 + `_) + +- Return stdout/stderr capture results as a ``namedtuple``, so ``out`` and + ``err`` can be accessed by attribute. (`#2879 + `_) + +- Add ``capfdbinary``, a version of ``capfd`` which returns bytes from + ``readouterr()``. (`#2923 + `_) + +- Add ``capsysbinary`` a version of ``capsys`` which returns bytes from + ``readouterr()``. (`#2934 + `_) + +- Implement feature to skip ``setup.py`` files when run with + ``--doctest-modules``. (`#502 + `_) + + +Bug Fixes +--------- + +- Resume output capturing after ``capsys/capfd.disabled()`` context manager. + (`#1993 `_) + +- ``pytest_fixture_setup`` and ``pytest_fixture_post_finalizer`` hooks are now + called for all ``conftest.py`` files. (`#2124 + `_) + +- If an exception happens while loading a plugin, pytest no longer hides the + original traceback. In python2 it will show the original traceback with a new + message that explains in which plugin. In python3 it will show 2 canonized + exceptions, the original exception while loading the plugin in addition to an + exception that PyTest throws about loading a plugin. (`#2491 + `_) + +- ``capsys`` and ``capfd`` can now be used by other fixtures. (`#2709 + `_) + +- Internal ``pytester`` plugin properly encodes ``bytes`` arguments to + ``utf-8``. (`#2738 `_) + +- ``testdir`` now uses use the same method used by ``tmpdir`` to create its + temporary directory. This changes the final structure of the ``testdir`` + directory slightly, but should not affect usage in normal scenarios and + avoids a number of potential problems. (`#2751 + `_) + +- Pytest no longer complains about warnings with unicode messages being + non-ascii compatible even for ascii-compatible messages. As a result of this, + warnings with unicode messages are converted first to an ascii representation + for safety. (`#2809 `_) + +- Change return value of pytest command when ``--maxfail`` is reached from + ``2`` (interrupted) to ``1`` (failed). (`#2845 + `_) + +- Fix issue in assertion rewriting which could lead it to rewrite modules which + should not be rewritten. (`#2939 + `_) + +- Handle marks without description in ``pytest.ini``. (`#2942 + `_) + + +Trivial/Internal Changes +------------------------ + +- pytest now depends on `attrs `_ for internal + structures to ease code maintainability. (`#2641 + `_) + +- Refactored internal Python 2/3 compatibility code to use ``six``. (`#2642 + `_) + +- Stop vendoring ``pluggy`` - we're missing out on its latest changes for not + much benefit (`#2719 `_) + +- Internal refactor: simplify ascii string escaping by using the + backslashreplace error handler in newer Python 3 versions. (`#2734 + `_) + +- Remove unnecessary mark evaluator in unittest plugin (`#2767 + `_) + +- Calls to ``Metafunc.addcall`` now emit a deprecation warning. This function + is scheduled to be removed in ``pytest-4.0``. (`#2876 + `_) + +- Internal move of the parameterset extraction to a more maintainable place. + (`#2877 `_) + +- Internal refactoring to simplify scope node lookup. (`#2910 + `_) + +- Configure ``pytest`` to prevent pip from installing pytest in unsupported + Python versions. (`#2922 + `_) + + +Pytest 3.2.5 (2017-11-15) +========================= + +Bug Fixes +--------- + +- Remove ``py<1.5`` restriction from ``pytest`` as this can cause version + conflicts in some installations. (`#2926 + `_) + + +Pytest 3.2.4 (2017-11-13) +========================= + +Bug Fixes +--------- + +- Fix the bug where running with ``--pyargs`` will result in items with + empty ``parent.nodeid`` if run from a different root directory. (`#2775 + `_) + +- Fix issue with ``@pytest.parametrize`` if argnames was specified as keyword arguments. + (`#2819 `_) + +- Strip whitespace from marker names when reading them from INI config. (`#2856 + `_) + +- Show full context of doctest source in the pytest output, if the line number of + failed example in the docstring is < 9. (`#2882 + `_) + +- Match fixture paths against actual path segments in order to avoid matching folders which share a prefix. + (`#2836 `_) + +Improved Documentation +---------------------- + +- Introduce a dedicated section about conftest.py. (`#1505 + `_) + +- Explicitly mention ``xpass`` in the documentation of ``xfail``. (`#1997 + `_) + +- Append example for pytest.param in the example/parametrize document. (`#2658 + `_) + +- Clarify language of proposal for fixtures parameters (`#2893 + `_) + +- List python 3.6 in the documented supported versions in the getting started + document. (`#2903 `_) + +- Clarify the documentation of available fixture scopes. (`#538 + `_) + +- Add documentation about the ``python -m pytest`` invocation adding the + current directory to sys.path. (`#911 + `_) + + +Pytest 3.2.3 (2017-10-03) +========================= + +Bug Fixes +--------- + +- Fix crash in tab completion when no prefix is given. (`#2748 + `_) + +- The equality checking function (``__eq__``) of ``MarkDecorator`` returns + ``False`` if one object is not an instance of ``MarkDecorator``. (`#2758 + `_) + +- When running ``pytest --fixtures-per-test``: don't crash if an item has no + _fixtureinfo attribute (e.g. doctests) (`#2788 + `_) + + +Improved Documentation +---------------------- + +- In help text of ``-k`` option, add example of using ``not`` to not select + certain tests whose names match the provided expression. (`#1442 + `_) + +- Add note in ``parametrize.rst`` about calling ``metafunc.parametrize`` + multiple times. (`#1548 `_) + + +Trivial/Internal Changes +------------------------ + +- Set ``xfail_strict=True`` in pytest's own test suite to catch expected + failures as soon as they start to pass. (`#2722 + `_) + +- Fix typo in example of passing a callable to markers (in example/markers.rst) + (`#2765 `_) + + +Pytest 3.2.2 (2017-09-06) +========================= + +Bug Fixes +--------- + +- Calling the deprecated `request.getfuncargvalue()` now shows the source of + the call. (`#2681 `_) + +- Allow tests declared as ``@staticmethod`` to use fixtures. (`#2699 + `_) + +- Fixed edge-case during collection: attributes which raised ``pytest.fail`` + when accessed would abort the entire collection. (`#2707 + `_) + +- Fix ``ReprFuncArgs`` with mixed unicode and UTF-8 args. (`#2731 + `_) + + +Improved Documentation +---------------------- + +- In examples on working with custom markers, add examples demonstrating the + usage of ``pytest.mark.MARKER_NAME.with_args`` in comparison with + ``pytest.mark.MARKER_NAME.__call__`` (`#2604 + `_) + +- In one of the simple examples, use `pytest_collection_modifyitems()` to skip + tests based on a command-line option, allowing its sharing while preventing a + user error when acessing `pytest.config` before the argument parsing. (`#2653 + `_) + + +Trivial/Internal Changes +------------------------ + +- Fixed minor error in 'Good Practices/Manual Integration' code snippet. + (`#2691 `_) + +- Fixed typo in goodpractices.rst. (`#2721 + `_) + +- Improve user guidance regarding ``--resultlog`` deprecation. (`#2739 + `_) + + +Pytest 3.2.1 (2017-08-08) +========================= + +Bug Fixes +--------- + +- Fixed small terminal glitch when collecting a single test item. (`#2579 + `_) + +- Correctly consider ``/`` as the file separator to automatically mark plugin + files for rewrite on Windows. (`#2591 `_) + +- Properly escape test names when setting ``PYTEST_CURRENT_TEST`` environment + variable. (`#2644 `_) + +- Fix error on Windows and Python 3.6+ when ``sys.stdout`` has been replaced + with a stream-like object which does not implement the full ``io`` module + buffer protocol. In particular this affects ``pytest-xdist`` users on the + aforementioned platform. (`#2666 `_) + + +Improved Documentation +---------------------- + +- Explicitly document which pytest features work with ``unittest``. (`#2626 + `_) + + +Pytest 3.2.0 (2017-07-30) +========================= + +Deprecations and Removals +------------------------- + +- ``pytest.approx`` no longer supports ``>``, ``>=``, ``<`` and ``<=`` + operators to avoid surprising/inconsistent behavior. See `the approx docs + `_ for more + information. (`#2003 `_) + +- All old-style specific behavior in current classes in the pytest's API is + considered deprecated at this point and will be removed in a future release. + This affects Python 2 users only and in rare situations. (`#2147 + `_) + +- A deprecation warning is now raised when using marks for parameters + in ``pytest.mark.parametrize``. Use ``pytest.param`` to apply marks to + parameters instead. (`#2427 `_) + + +Features +-------- + +- Add support for numpy arrays (and dicts) to approx. (`#1994 + `_) + +- Now test function objects have a ``pytestmark`` attribute containing a list + of marks applied directly to the test function, as opposed to marks inherited + from parent classes or modules. (`#2516 `_) + +- Collection ignores local virtualenvs by default; `--collect-in-virtualenv` + overrides this behavior. (`#2518 `_) + +- Allow class methods decorated as ``@staticmethod`` to be candidates for + collection as a test function. (Only for Python 2.7 and above. Python 2.6 + will still ignore static methods.) (`#2528 `_) + +- Introduce ``mark.with_args`` in order to allow passing functions/classes as + sole argument to marks. (`#2540 `_) + +- New ``cache_dir`` ini option: sets the directory where the contents of the + cache plugin are stored. Directory may be relative or absolute path: if relative path, then + directory is created relative to ``rootdir``, otherwise it is used as is. + Additionally path may contain environment variables which are expanded during + runtime. (`#2543 `_) + +- Introduce the ``PYTEST_CURRENT_TEST`` environment variable that is set with + the ``nodeid`` and stage (``setup``, ``call`` and ``teardown``) of the test + being currently executed. See the `documentation + `_ for more info. (`#2583 `_) + +- Introduced ``@pytest.mark.filterwarnings`` mark which allows overwriting the + warnings filter on a per test, class or module level. See the `docs + `_ for more information. (`#2598 `_) + +- ``--last-failed`` now remembers forever when a test has failed and only + forgets it if it passes again. This makes it easy to fix a test suite by + selectively running files and fixing tests incrementally. (`#2621 + `_) + +- New ``pytest_report_collectionfinish`` hook which allows plugins to add + messages to the terminal reporting after collection has been finished + successfully. (`#2622 `_) + +- Added support for `PEP-415's `_ + ``Exception.__suppress_context__``. Now if a ``raise exception from None`` is + caught by pytest, pytest will no longer chain the context in the test report. + The behavior now matches Python's traceback behavior. (`#2631 + `_) + +- Exceptions raised by ``pytest.fail``, ``pytest.skip`` and ``pytest.xfail`` + now subclass BaseException, making them harder to be caught unintentionally + by normal code. (`#580 `_) + + +Bug Fixes +--------- + +- Set ``stdin`` to a closed ``PIPE`` in ``pytester.py.Testdir.popen()`` for + avoid unwanted interactive ``pdb`` (`#2023 `_) + +- Add missing ``encoding`` attribute to ``sys.std*`` streams when using + ``capsys`` capture mode. (`#2375 `_) + +- Fix terminal color changing to black on Windows if ``colorama`` is imported + in a ``conftest.py`` file. (`#2510 `_) + +- Fix line number when reporting summary of skipped tests. (`#2548 + `_) + +- capture: ensure that EncodedFile.name is a string. (`#2555 + `_) + +- The options ``--fixtures`` and ``--fixtures-per-test`` will now keep + indentation within docstrings. (`#2574 `_) + +- doctests line numbers are now reported correctly, fixing `pytest-sugar#122 + `_. (`#2610 + `_) + +- Fix non-determinism in order of fixture collection. Adds new dependency + (ordereddict) for Python 2.6. (`#920 `_) + + +Improved Documentation +---------------------- + +- Clarify ``pytest_configure`` hook call order. (`#2539 + `_) + +- Extend documentation for testing plugin code with the ``pytester`` plugin. + (`#971 `_) + + +Trivial/Internal Changes +------------------------ + +- Update help message for ``--strict`` to make it clear it only deals with + unregistered markers, not warnings. (`#2444 `_) + +- Internal code move: move code for pytest.approx/pytest.raises to own files in + order to cut down the size of python.py (`#2489 `_) + +- Renamed the utility function ``_pytest.compat._escape_strings`` to + ``_ascii_escaped`` to better communicate the function's purpose. (`#2533 + `_) + +- Improve error message for CollectError with skip/skipif. (`#2546 + `_) + +- Emit warning about ``yield`` tests being deprecated only once per generator. + (`#2562 `_) + +- Ensure final collected line doesn't include artifacts of previous write. + (`#2571 `_) + +- Fixed all flake8 errors and warnings. (`#2581 `_) + +- Added ``fix-lint`` tox environment to run automatic pep8 fixes on the code. + (`#2582 `_) + +- Turn warnings into errors in pytest's own test suite in order to catch + regressions due to deprecations more promptly. (`#2588 + `_) + +- Show multiple issue links in CHANGELOG entries. (`#2620 + `_) + + +Pytest 3.1.3 (2017-07-03) +========================= + +Bug Fixes +--------- + +- Fix decode error in Python 2 for doctests in docstrings. (`#2434 + `_) + +- Exceptions raised during teardown by finalizers are now suppressed until all + finalizers are called, with the initial exception reraised. (`#2440 + `_) + +- Fix incorrect "collected items" report when specifying tests on the command- + line. (`#2464 `_) + +- ``deprecated_call`` in context-manager form now captures deprecation warnings + even if the same warning has already been raised. Also, ``deprecated_call`` + will always produce the same error message (previously it would produce + different messages in context-manager vs. function-call mode). (`#2469 + `_) + +- Fix issue where paths collected by pytest could have triple leading ``/`` + characters. (`#2475 `_) + +- Fix internal error when trying to detect the start of a recursive traceback. + (`#2486 `_) + + +Improved Documentation +---------------------- + +- Explicitly state for which hooks the calls stop after the first non-None + result. (`#2493 `_) + + +Trivial/Internal Changes +------------------------ + +- Create invoke tasks for updating the vendored packages. (`#2474 + `_) + +- Update copyright dates in LICENSE, README.rst and in the documentation. + (`#2499 `_) + + +Pytest 3.1.2 (2017-06-08) +========================= + +Bug Fixes +--------- + +- Required options added via ``pytest_addoption`` will no longer prevent using + --help without passing them. (#1999) + +- Respect ``python_files`` in assertion rewriting. (#2121) + +- Fix recursion error detection when frames in the traceback contain objects + that can't be compared (like ``numpy`` arrays). (#2459) + +- ``UnicodeWarning`` is issued from the internal pytest warnings plugin only + when the message contains non-ascii unicode (Python 2 only). (#2463) + +- Added a workaround for Python 3.6 ``WindowsConsoleIO`` breaking due to Pytests's + ``FDCapture``. Other code using console handles might still be affected by the + very same issue and might require further workarounds/fixes, i.e. ``colorama``. + (#2467) + + +Improved Documentation +---------------------- + +- Fix internal API links to ``pluggy`` objects. (#2331) + +- Make it clear that ``pytest.xfail`` stops test execution at the calling point + and improve overall flow of the ``skipping`` docs. (#810) + + +Pytest 3.1.1 (2017-05-30) +========================= + +Bug Fixes +--------- + +- pytest warning capture no longer overrides existing warning filters. The + previous behaviour would override all filters and caused regressions in test + suites which configure warning filters to match their needs. Note that as a + side-effect of this is that ``DeprecationWarning`` and + ``PendingDeprecationWarning`` are no longer shown by default. (#2430) + +- Fix issue with non-ascii contents in doctest text files. (#2434) + +- Fix encoding errors for unicode warnings in Python 2. (#2436) + +- ``pytest.deprecated_call`` now captures ``PendingDeprecationWarning`` in + context manager form. (#2441) + + +Improved Documentation +---------------------- + +- Addition of towncrier for changelog management. (#2390) + + +3.1.0 (2017-05-22) +================== + + +New Features +------------ + +* The ``pytest-warnings`` plugin has been integrated into the core and now ``pytest`` automatically + captures and displays warnings at the end of the test session. + + .. warning:: + + This feature may disrupt test suites which apply and treat warnings themselves, and can be + disabled in your ``pytest.ini``: + + .. code-block:: ini + + [pytest] + addopts = -p no:warnings + + See the `warnings documentation page `_ for more + information. + + Thanks `@nicoddemus`_ for the PR. + +* Added ``junit_suite_name`` ini option to specify root ```` name for JUnit XML reports (`#533`_). + +* Added an ini option ``doctest_encoding`` to specify which encoding to use for doctest files. + Thanks `@wheerd`_ for the PR (`#2101`_). + +* ``pytest.warns`` now checks for subclass relationship rather than + class equality. Thanks `@lesteve`_ for the PR (`#2166`_) + +* ``pytest.raises`` now asserts that the error message matches a text or regex + with the ``match`` keyword argument. Thanks `@Kriechi`_ for the PR. + +* ``pytest.param`` can be used to declare test parameter sets with marks and test ids. + Thanks `@RonnyPfannschmidt`_ for the PR. + + +Changes +------- + +* remove all internal uses of pytest_namespace hooks, + this is to prepare the removal of preloadconfig in pytest 4.0 + Thanks to `@RonnyPfannschmidt`_ for the PR. + +* pytest now warns when a callable ids raises in a parametrized test. Thanks `@fogo`_ for the PR. + +* It is now possible to skip test classes from being collected by setting a + ``__test__`` attribute to ``False`` in the class body (`#2007`_). Thanks + to `@syre`_ for the report and `@lwm`_ for the PR. + +* Change junitxml.py to produce reports that comply with Junitxml schema. + If the same test fails with failure in call and then errors in teardown + we split testcase element into two, one containing the error and the other + the failure. (`#2228`_) Thanks to `@kkoukiou`_ for the PR. + +* Testcase reports with a ``url`` attribute will now properly write this to junitxml. + Thanks `@fushi`_ for the PR (`#1874`_). + +* Remove common items from dict comparision output when verbosity=1. Also update + the truncation message to make it clearer that pytest truncates all + assertion messages if verbosity < 2 (`#1512`_). + Thanks `@mattduck`_ for the PR + +* ``--pdbcls`` no longer implies ``--pdb``. This makes it possible to use + ``addopts=--pdbcls=module.SomeClass`` on ``pytest.ini``. Thanks `@davidszotten`_ for + the PR (`#1952`_). + +* fix `#2013`_: turn RecordedWarning into ``namedtuple``, + to give it a comprehensible repr while preventing unwarranted modification. + +* fix `#2208`_: ensure a iteration limit for _pytest.compat.get_real_func. + Thanks `@RonnyPfannschmidt`_ for the report and PR. + +* Hooks are now verified after collection is complete, rather than right after loading installed plugins. This + makes it easy to write hooks for plugins which will be loaded during collection, for example using the + ``pytest_plugins`` special variable (`#1821`_). + Thanks `@nicoddemus`_ for the PR. + +* Modify ``pytest_make_parametrize_id()`` hook to accept ``argname`` as an + additional parameter. + Thanks `@unsignedint`_ for the PR. + +* Add ``venv`` to the default ``norecursedirs`` setting. + Thanks `@The-Compiler`_ for the PR. + +* ``PluginManager.import_plugin`` now accepts unicode plugin names in Python 2. + Thanks `@reutsharabani`_ for the PR. + +* fix `#2308`_: When using both ``--lf`` and ``--ff``, only the last failed tests are run. + Thanks `@ojii`_ for the PR. + +* Replace minor/patch level version numbers in the documentation with placeholders. + This significantly reduces change-noise as different contributors regnerate + the documentation on different platforms. + Thanks `@RonnyPfannschmidt`_ for the PR. + +* fix `#2391`_: consider pytest_plugins on all plugin modules + Thanks `@RonnyPfannschmidt`_ for the PR. + + +Bug Fixes +--------- + +* Fix ``AttributeError`` on ``sys.stdout.buffer`` / ``sys.stderr.buffer`` + while using ``capsys`` fixture in python 3. (`#1407`_). + Thanks to `@asottile`_. + +* Change capture.py's ``DontReadFromInput`` class to throw ``io.UnsupportedOperation`` errors rather + than ValueErrors in the ``fileno`` method (`#2276`_). + Thanks `@metasyn`_ and `@vlad-dragos`_ for the PR. + +* Fix exception formatting while importing modules when the exception message + contains non-ascii characters (`#2336`_). + Thanks `@fabioz`_ for the report and `@nicoddemus`_ for the PR. + +* Added documentation related to issue (`#1937`_) + Thanks `@skylarjhdownes`_ for the PR. + +* Allow collecting files with any file extension as Python modules (`#2369`_). + Thanks `@Kodiologist`_ for the PR. + +* Show the correct error message when collect "parametrize" func with wrong args (`#2383`_). + Thanks `@The-Compiler`_ for the report and `@robin0371`_ for the PR. + + +.. _@davidszotten: https://github.com/davidszotten +.. _@fabioz: https://github.com/fabioz +.. _@fogo: https://github.com/fogo +.. _@fushi: https://github.com/fushi +.. _@Kodiologist: https://github.com/Kodiologist +.. _@Kriechi: https://github.com/Kriechi +.. _@mandeep: https://github.com/mandeep +.. _@mattduck: https://github.com/mattduck +.. _@metasyn: https://github.com/metasyn +.. _@MichalTHEDUDE: https://github.com/MichalTHEDUDE +.. _@ojii: https://github.com/ojii +.. _@reutsharabani: https://github.com/reutsharabani +.. _@robin0371: https://github.com/robin0371 +.. _@skylarjhdownes: https://github.com/skylarjhdownes +.. _@unsignedint: https://github.com/unsignedint +.. _@wheerd: https://github.com/wheerd + + +.. _#1407: https://github.com/pytest-dev/pytest/issues/1407 +.. _#1512: https://github.com/pytest-dev/pytest/issues/1512 +.. _#1821: https://github.com/pytest-dev/pytest/issues/1821 +.. _#1874: https://github.com/pytest-dev/pytest/pull/1874 +.. _#1937: https://github.com/pytest-dev/pytest/issues/1937 +.. _#1952: https://github.com/pytest-dev/pytest/pull/1952 +.. _#2007: https://github.com/pytest-dev/pytest/issues/2007 +.. _#2013: https://github.com/pytest-dev/pytest/issues/2013 +.. _#2101: https://github.com/pytest-dev/pytest/pull/2101 +.. _#2166: https://github.com/pytest-dev/pytest/pull/2166 +.. _#2208: https://github.com/pytest-dev/pytest/issues/2208 +.. _#2228: https://github.com/pytest-dev/pytest/issues/2228 +.. _#2276: https://github.com/pytest-dev/pytest/issues/2276 +.. _#2308: https://github.com/pytest-dev/pytest/issues/2308 +.. _#2336: https://github.com/pytest-dev/pytest/issues/2336 +.. _#2369: https://github.com/pytest-dev/pytest/issues/2369 +.. _#2383: https://github.com/pytest-dev/pytest/issues/2383 +.. _#2391: https://github.com/pytest-dev/pytest/issues/2391 +.. _#533: https://github.com/pytest-dev/pytest/issues/533 + + + +3.0.7 (2017-03-14) +================== + + +* Fix issue in assertion rewriting breaking due to modules silently discarding + other modules when importing fails + Notably, importing the ``anydbm`` module is fixed. (`#2248`_). + Thanks `@pfhayes`_ for the PR. + +* junitxml: Fix problematic case where system-out tag occured twice per testcase + element in the XML report. Thanks `@kkoukiou`_ for the PR. + +* Fix regression, pytest now skips unittest correctly if run with ``--pdb`` + (`#2137`_). Thanks to `@gst`_ for the report and `@mbyt`_ for the PR. + +* Ignore exceptions raised from descriptors (e.g. properties) during Python test collection (`#2234`_). + Thanks to `@bluetech`_. + +* ``--override-ini`` now correctly overrides some fundamental options like ``python_files`` (`#2238`_). + Thanks `@sirex`_ for the report and `@nicoddemus`_ for the PR. + +* Replace ``raise StopIteration`` usages in the code by simple ``returns`` to finish generators, in accordance to `PEP-479`_ (`#2160`_). + Thanks `@tgoodlet`_ for the report and `@nicoddemus`_ for the PR. + +* Fix internal errors when an unprintable ``AssertionError`` is raised inside a test. + Thanks `@omerhadari`_ for the PR. + +* Skipping plugin now also works with test items generated by custom collectors (`#2231`_). + Thanks to `@vidartf`_. + +* Fix trailing whitespace in console output if no .ini file presented (`#2281`_). Thanks `@fbjorn`_ for the PR. + +* Conditionless ``xfail`` markers no longer rely on the underlying test item + being an instance of ``PyobjMixin``, and can therefore apply to tests not + collected by the built-in python test collector. Thanks `@barneygale`_ for the + PR. + + +.. _@pfhayes: https://github.com/pfhayes +.. _@bluetech: https://github.com/bluetech +.. _@gst: https://github.com/gst +.. _@sirex: https://github.com/sirex +.. _@vidartf: https://github.com/vidartf +.. _@kkoukiou: https://github.com/KKoukiou +.. _@omerhadari: https://github.com/omerhadari +.. _@fbjorn: https://github.com/fbjorn + +.. _#2248: https://github.com/pytest-dev/pytest/issues/2248 +.. _#2137: https://github.com/pytest-dev/pytest/issues/2137 +.. _#2160: https://github.com/pytest-dev/pytest/issues/2160 +.. _#2231: https://github.com/pytest-dev/pytest/issues/2231 +.. _#2234: https://github.com/pytest-dev/pytest/issues/2234 +.. _#2238: https://github.com/pytest-dev/pytest/issues/2238 +.. _#2281: https://github.com/pytest-dev/pytest/issues/2281 + +.. _PEP-479: https://www.python.org/dev/peps/pep-0479/ + + +3.0.6 (2017-01-22) +================== + +* pytest no longer generates ``PendingDeprecationWarning`` from its own operations, which was introduced by mistake in version ``3.0.5`` (`#2118`_). + Thanks to `@nicoddemus`_ for the report and `@RonnyPfannschmidt`_ for the PR. + + +* pytest no longer recognizes coroutine functions as yield tests (`#2129`_). + Thanks to `@malinoff`_ for the PR. + +* Plugins loaded by the ``PYTEST_PLUGINS`` environment variable are now automatically + considered for assertion rewriting (`#2185`_). + Thanks `@nicoddemus`_ for the PR. + +* Improve error message when pytest.warns fails (`#2150`_). The type(s) of the + expected warnings and the list of caught warnings is added to the + error message. Thanks `@lesteve`_ for the PR. + +* Fix ``pytester`` internal plugin to work correctly with latest versions of + ``zope.interface`` (`#1989`_). Thanks `@nicoddemus`_ for the PR. + +* Assert statements of the ``pytester`` plugin again benefit from assertion rewriting (`#1920`_). + Thanks `@RonnyPfannschmidt`_ for the report and `@nicoddemus`_ for the PR. + +* Specifying tests with colons like ``test_foo.py::test_bar`` for tests in + subdirectories with ini configuration files now uses the correct ini file + (`#2148`_). Thanks `@pelme`_. + +* Fail ``testdir.runpytest().assert_outcomes()`` explicitly if the pytest + terminal output it relies on is missing. Thanks to `@eli-b`_ for the PR. + + +.. _@barneygale: https://github.com/barneygale +.. _@lesteve: https://github.com/lesteve +.. _@malinoff: https://github.com/malinoff +.. _@pelme: https://github.com/pelme +.. _@eli-b: https://github.com/eli-b + +.. _#2118: https://github.com/pytest-dev/pytest/issues/2118 + +.. _#1989: https://github.com/pytest-dev/pytest/issues/1989 +.. _#1920: https://github.com/pytest-dev/pytest/issues/1920 +.. _#2129: https://github.com/pytest-dev/pytest/issues/2129 +.. _#2148: https://github.com/pytest-dev/pytest/issues/2148 +.. _#2150: https://github.com/pytest-dev/pytest/issues/2150 +.. _#2185: https://github.com/pytest-dev/pytest/issues/2185 + + +3.0.5 (2016-12-05) +================== + +* Add warning when not passing ``option=value`` correctly to ``-o/--override-ini`` (`#2105`_). + Also improved the help documentation. Thanks to `@mbukatov`_ for the report and + `@lwm`_ for the PR. + +* Now ``--confcutdir`` and ``--junit-xml`` are properly validated if they are directories + and filenames, respectively (`#2089`_ and `#2078`_). Thanks to `@lwm`_ for the PR. + +* Add hint to error message hinting possible missing ``__init__.py`` (`#478`_). Thanks `@DuncanBetts`_. + +* More accurately describe when fixture finalization occurs in documentation (`#687`_). Thanks `@DuncanBetts`_. + +* Provide ``:ref:`` targets for ``recwarn.rst`` so we can use intersphinx referencing. + Thanks to `@dupuy`_ for the report and `@lwm`_ for the PR. + +* In Python 2, use a simple ``+-`` ASCII string in the string representation of ``pytest.approx`` (for example ``"4 +- 4.0e-06"``) + because it is brittle to handle that in different contexts and representations internally in pytest + which can result in bugs such as `#2111`_. In Python 3, the representation still uses ``±`` (for example ``4 ± 4.0e-06``). + Thanks `@kerrick-lyft`_ for the report and `@nicoddemus`_ for the PR. + +* Using ``item.Function``, ``item.Module``, etc., is now issuing deprecation warnings, prefer + ``pytest.Function``, ``pytest.Module``, etc., instead (`#2034`_). + Thanks `@nmundar`_ for the PR. + +* Fix error message using ``approx`` with complex numbers (`#2082`_). + Thanks `@adler-j`_ for the report and `@nicoddemus`_ for the PR. + +* Fixed false-positives warnings from assertion rewrite hook for modules imported more than + once by the ``pytest_plugins`` mechanism. + Thanks `@nicoddemus`_ for the PR. + +* Remove an internal cache which could cause hooks from ``conftest.py`` files in + sub-directories to be called in other directories incorrectly (`#2016`_). + Thanks `@d-b-w`_ for the report and `@nicoddemus`_ for the PR. + +* Remove internal code meant to support earlier Python 3 versions that produced the side effect + of leaving ``None`` in ``sys.modules`` when expressions were evaluated by pytest (for example passing a condition + as a string to ``pytest.mark.skipif``)(`#2103`_). + Thanks `@jaraco`_ for the report and `@nicoddemus`_ for the PR. + +* Cope gracefully with a .pyc file with no matching .py file (`#2038`_). Thanks + `@nedbat`_. + +.. _@syre: https://github.com/syre +.. _@adler-j: https://github.com/adler-j +.. _@d-b-w: https://bitbucket.org/d-b-w/ +.. _@DuncanBetts: https://github.com/DuncanBetts +.. _@dupuy: https://bitbucket.org/dupuy/ +.. _@kerrick-lyft: https://github.com/kerrick-lyft +.. _@lwm: https://github.com/lwm +.. _@mbukatov: https://github.com/mbukatov +.. _@nedbat: https://github.com/nedbat +.. _@nmundar: https://github.com/nmundar + +.. _#2016: https://github.com/pytest-dev/pytest/issues/2016 +.. _#2034: https://github.com/pytest-dev/pytest/issues/2034 +.. _#2038: https://github.com/pytest-dev/pytest/issues/2038 +.. _#2078: https://github.com/pytest-dev/pytest/issues/2078 +.. _#2082: https://github.com/pytest-dev/pytest/issues/2082 +.. _#2089: https://github.com/pytest-dev/pytest/issues/2089 +.. _#2103: https://github.com/pytest-dev/pytest/issues/2103 +.. _#2105: https://github.com/pytest-dev/pytest/issues/2105 +.. _#2111: https://github.com/pytest-dev/pytest/issues/2111 +.. _#478: https://github.com/pytest-dev/pytest/issues/478 +.. _#687: https://github.com/pytest-dev/pytest/issues/687 + + +3.0.4 (2016-11-09) +================== + +* Import errors when collecting test modules now display the full traceback (`#1976`_). + Thanks `@cwitty`_ for the report and `@nicoddemus`_ for the PR. + +* Fix confusing command-line help message for custom options with two or more ``metavar`` properties (`#2004`_). + Thanks `@okulynyak`_ and `@davehunt`_ for the report and `@nicoddemus`_ for the PR. + +* When loading plugins, import errors which contain non-ascii messages are now properly handled in Python 2 (`#1998`_). + Thanks `@nicoddemus`_ for the PR. + +* Fixed cyclic reference when ``pytest.raises`` is used in context-manager form (`#1965`_). Also as a + result of this fix, ``sys.exc_info()`` is left empty in both context-manager and function call usages. + Previously, ``sys.exc_info`` would contain the exception caught by the context manager, + even when the expected exception occurred. + Thanks `@MSeifert04`_ for the report and the PR. + +* Fixed false-positives warnings from assertion rewrite hook for modules that were rewritten but + were later marked explicitly by ``pytest.register_assert_rewrite`` + or implicitly as a plugin (`#2005`_). + Thanks `@RonnyPfannschmidt`_ for the report and `@nicoddemus`_ for the PR. + +* Report teardown output on test failure (`#442`_). + Thanks `@matclab`_ for the PR. + +* Fix teardown error message in generated xUnit XML. + Thanks `@gdyuldin`_ for the PR. + +* Properly handle exceptions in ``multiprocessing`` tasks (`#1984`_). + Thanks `@adborden`_ for the report and `@nicoddemus`_ for the PR. + +* Clean up unittest TestCase objects after tests are complete (`#1649`_). + Thanks `@d_b_w`_ for the report and PR. + + +.. _@adborden: https://github.com/adborden +.. _@cwitty: https://github.com/cwitty +.. _@d_b_w: https://github.com/d_b_w +.. _@gdyuldin: https://github.com/gdyuldin +.. _@matclab: https://github.com/matclab +.. _@MSeifert04: https://github.com/MSeifert04 +.. _@okulynyak: https://github.com/okulynyak + +.. _#442: https://github.com/pytest-dev/pytest/issues/442 +.. _#1965: https://github.com/pytest-dev/pytest/issues/1965 +.. _#1976: https://github.com/pytest-dev/pytest/issues/1976 +.. _#1984: https://github.com/pytest-dev/pytest/issues/1984 +.. _#1998: https://github.com/pytest-dev/pytest/issues/1998 +.. _#2004: https://github.com/pytest-dev/pytest/issues/2004 +.. _#2005: https://github.com/pytest-dev/pytest/issues/2005 +.. _#1649: https://github.com/pytest-dev/pytest/issues/1649 + + +3.0.3 (2016-09-28) +================== + +* The ``ids`` argument to ``parametrize`` again accepts ``unicode`` strings + in Python 2 (`#1905`_). + Thanks `@philpep`_ for the report and `@nicoddemus`_ for the PR. + +* Assertions are now being rewritten for plugins in development mode + (``pip install -e``) (`#1934`_). + Thanks `@nicoddemus`_ for the PR. + +* Fix pkg_resources import error in Jython projects (`#1853`_). + Thanks `@raquel-ucl`_ for the PR. + +* Got rid of ``AttributeError: 'Module' object has no attribute '_obj'`` exception + in Python 3 (`#1944`_). + Thanks `@axil`_ for the PR. + +* Explain a bad scope value passed to ``@fixture`` declarations or + a ``MetaFunc.parametrize()`` call. Thanks `@tgoodlet`_ for the PR. + +* This version includes ``pluggy-0.4.0``, which correctly handles + ``VersionConflict`` errors in plugins (`#704`_). + Thanks `@nicoddemus`_ for the PR. + + +.. _@philpep: https://github.com/philpep +.. _@raquel-ucl: https://github.com/raquel-ucl +.. _@axil: https://github.com/axil +.. _@tgoodlet: https://github.com/tgoodlet +.. _@vlad-dragos: https://github.com/vlad-dragos + +.. _#1853: https://github.com/pytest-dev/pytest/issues/1853 +.. _#1905: https://github.com/pytest-dev/pytest/issues/1905 +.. _#1934: https://github.com/pytest-dev/pytest/issues/1934 +.. _#1944: https://github.com/pytest-dev/pytest/issues/1944 +.. _#704: https://github.com/pytest-dev/pytest/issues/704 + + + + +3.0.2 (2016-09-01) +================== + +* Improve error message when passing non-string ids to ``pytest.mark.parametrize`` (`#1857`_). + Thanks `@okken`_ for the report and `@nicoddemus`_ for the PR. + +* Add ``buffer`` attribute to stdin stub class ``pytest.capture.DontReadFromInput`` + Thanks `@joguSD`_ for the PR. + +* Fix ``UnicodeEncodeError`` when string comparison with unicode has failed. (`#1864`_) + Thanks `@AiOO`_ for the PR. + +* ``pytest_plugins`` is now handled correctly if defined as a string (as opposed as + a sequence of strings) when modules are considered for assertion rewriting. + Due to this bug, much more modules were being rewritten than necessary + if a test suite uses ``pytest_plugins`` to load internal plugins (`#1888`_). + Thanks `@jaraco`_ for the report and `@nicoddemus`_ for the PR (`#1891`_). + +* Do not call tearDown and cleanups when running tests from + ``unittest.TestCase`` subclasses with ``--pdb`` + enabled. This allows proper post mortem debugging for all applications + which have significant logic in their tearDown machinery (`#1890`_). Thanks + `@mbyt`_ for the PR. + +* Fix use of deprecated ``getfuncargvalue`` method in the internal doctest plugin. + Thanks `@ViviCoder`_ for the report (`#1898`_). + +.. _@joguSD: https://github.com/joguSD +.. _@AiOO: https://github.com/AiOO +.. _@mbyt: https://github.com/mbyt +.. _@ViviCoder: https://github.com/ViviCoder + +.. _#1857: https://github.com/pytest-dev/pytest/issues/1857 +.. _#1864: https://github.com/pytest-dev/pytest/issues/1864 +.. _#1888: https://github.com/pytest-dev/pytest/issues/1888 +.. _#1891: https://github.com/pytest-dev/pytest/pull/1891 +.. _#1890: https://github.com/pytest-dev/pytest/issues/1890 +.. _#1898: https://github.com/pytest-dev/pytest/issues/1898 + + +3.0.1 (2016-08-23) +================== + +* Fix regression when ``importorskip`` is used at module level (`#1822`_). + Thanks `@jaraco`_ and `@The-Compiler`_ for the report and `@nicoddemus`_ for the PR. + +* Fix parametrization scope when session fixtures are used in conjunction + with normal parameters in the same call (`#1832`_). + Thanks `@The-Compiler`_ for the report, `@Kingdread`_ and `@nicoddemus`_ for the PR. + +* Fix internal error when parametrizing tests or fixtures using an empty ``ids`` argument (`#1849`_). + Thanks `@OPpuolitaival`_ for the report and `@nicoddemus`_ for the PR. + +* Fix loader error when running ``pytest`` embedded in a zipfile. + Thanks `@mbachry`_ for the PR. + + +.. _@Kingdread: https://github.com/Kingdread +.. _@mbachry: https://github.com/mbachry +.. _@OPpuolitaival: https://github.com/OPpuolitaival + +.. _#1822: https://github.com/pytest-dev/pytest/issues/1822 +.. _#1832: https://github.com/pytest-dev/pytest/issues/1832 +.. _#1849: https://github.com/pytest-dev/pytest/issues/1849 + + +3.0.0 (2016-08-18) +================== + +**Incompatible changes** + + +A number of incompatible changes were made in this release, with the intent of removing features deprecated for a long +time or change existing behaviors in order to make them less surprising/more useful. + +* Reinterpretation mode has now been removed. Only plain and rewrite + mode are available, consequently the ``--assert=reinterp`` option is + no longer available. This also means files imported from plugins or + ``conftest.py`` will not benefit from improved assertions by + default, you should use ``pytest.register_assert_rewrite()`` to + explicitly turn on assertion rewriting for those files. Thanks + `@flub`_ for the PR. + +* The following deprecated commandline options were removed: + + * ``--genscript``: no longer supported; + * ``--no-assert``: use ``--assert=plain`` instead; + * ``--nomagic``: use ``--assert=plain`` instead; + * ``--report``: use ``-r`` instead; + + Thanks to `@RedBeardCode`_ for the PR (`#1664`_). + +* ImportErrors in plugins now are a fatal error instead of issuing a + pytest warning (`#1479`_). Thanks to `@The-Compiler`_ for the PR. + +* Removed support code for Python 3 versions < 3.3 (`#1627`_). + +* Removed all ``py.test-X*`` entry points. The versioned, suffixed entry points + were never documented and a leftover from a pre-virtualenv era. These entry + points also created broken entry points in wheels, so removing them also + removes a source of confusion for users (`#1632`_). + Thanks `@obestwalter`_ for the PR. + +* ``pytest.skip()`` now raises an error when used to decorate a test function, + as opposed to its original intent (to imperatively skip a test inside a test function). Previously + this usage would cause the entire module to be skipped (`#607`_). + Thanks `@omarkohl`_ for the complete PR (`#1519`_). + +* Exit tests if a collection error occurs. A poll indicated most users will hit CTRL-C + anyway as soon as they see collection errors, so pytest might as well make that the default behavior (`#1421`_). + A ``--continue-on-collection-errors`` option has been added to restore the previous behaviour. + Thanks `@olegpidsadnyi`_ and `@omarkohl`_ for the complete PR (`#1628`_). + +* Renamed the pytest ``pdb`` module (plugin) into ``debugging`` to avoid clashes with the builtin ``pdb`` module. + +* Raise a helpful failure message when requesting a parametrized fixture at runtime, + e.g. with ``request.getfixturevalue``. Previously these parameters were simply + never defined, so a fixture decorated like ``@pytest.fixture(params=[0, 1, 2])`` + only ran once (`#460`_). + Thanks to `@nikratio`_ for the bug report, `@RedBeardCode`_ and `@tomviner`_ for the PR. + +* ``_pytest.monkeypatch.monkeypatch`` class has been renamed to ``_pytest.monkeypatch.MonkeyPatch`` + so it doesn't conflict with the ``monkeypatch`` fixture. + +* ``--exitfirst / -x`` can now be overridden by a following ``--maxfail=N`` + and is just a synonym for ``--maxfail=1``. + + +**New Features** + +* Support nose-style ``__test__`` attribute on methods of classes, + including unittest-style Classes. If set to ``False``, the test will not be + collected. + +* New ``doctest_namespace`` fixture for injecting names into the + namespace in which doctests run. + Thanks `@milliams`_ for the complete PR (`#1428`_). + +* New ``--doctest-report`` option available to change the output format of diffs + when running (failing) doctests (implements `#1749`_). + Thanks `@hartym`_ for the PR. + +* New ``name`` argument to ``pytest.fixture`` decorator which allows a custom name + for a fixture (to solve the funcarg-shadowing-fixture problem). + Thanks `@novas0x2a`_ for the complete PR (`#1444`_). + +* New ``approx()`` function for easily comparing floating-point numbers in + tests. + Thanks `@kalekundert`_ for the complete PR (`#1441`_). + +* Ability to add global properties in the final xunit output file by accessing + the internal ``junitxml`` plugin (experimental). + Thanks `@tareqalayan`_ for the complete PR `#1454`_). + +* New ``ExceptionInfo.match()`` method to match a regular expression on the + string representation of an exception (`#372`_). + Thanks `@omarkohl`_ for the complete PR (`#1502`_). + +* ``__tracebackhide__`` can now also be set to a callable which then can decide + whether to filter the traceback based on the ``ExceptionInfo`` object passed + to it. Thanks `@The-Compiler`_ for the complete PR (`#1526`_). + +* New ``pytest_make_parametrize_id(config, val)`` hook which can be used by plugins to provide + friendly strings for custom types. + Thanks `@palaviv`_ for the PR. + +* ``capsys`` and ``capfd`` now have a ``disabled()`` context-manager method, which + can be used to temporarily disable capture within a test. + Thanks `@nicoddemus`_ for the PR. + +* New cli flag ``--fixtures-per-test``: shows which fixtures are being used + for each selected test item. Features doc strings of fixtures by default. + Can also show where fixtures are defined if combined with ``-v``. + Thanks `@hackebrot`_ for the PR. + +* Introduce ``pytest`` command as recommended entry point. Note that ``py.test`` + still works and is not scheduled for removal. Closes proposal + `#1629`_. Thanks `@obestwalter`_ and `@davehunt`_ for the complete PR + (`#1633`_). + +* New cli flags: + + + ``--setup-plan``: performs normal collection and reports + the potential setup and teardown and does not execute any fixtures and tests; + + ``--setup-only``: performs normal collection, executes setup and teardown of + fixtures and reports them; + + ``--setup-show``: performs normal test execution and additionally shows + setup and teardown of fixtures; + + ``--keep-duplicates``: py.test now ignores duplicated paths given in the command + line. To retain the previous behavior where the same test could be run multiple + times by specifying it in the command-line multiple times, pass the ``--keep-duplicates`` + argument (`#1609`_); + + Thanks `@d6e`_, `@kvas-it`_, `@sallner`_, `@ioggstream`_ and `@omarkohl`_ for the PRs. + +* New CLI flag ``--override-ini``/``-o``: overrides values from the ini file. + For example: ``"-o xfail_strict=True"``'. + Thanks `@blueyed`_ and `@fengxx`_ for the PR. + +* New hooks: + + + ``pytest_fixture_setup(fixturedef, request)``: executes fixture setup; + + ``pytest_fixture_post_finalizer(fixturedef)``: called after the fixture's + finalizer and has access to the fixture's result cache. + + Thanks `@d6e`_, `@sallner`_. + +* Issue warnings for asserts whose test is a tuple literal. Such asserts will + never fail because tuples are always truthy and are usually a mistake + (see `#1562`_). Thanks `@kvas-it`_, for the PR. + +* Allow passing a custom debugger class (e.g. ``--pdbcls=IPython.core.debugger:Pdb``). + Thanks to `@anntzer`_ for the PR. + + +**Changes** + +* Plugins now benefit from assertion rewriting. Thanks + `@sober7`_, `@nicoddemus`_ and `@flub`_ for the PR. + +* Change ``report.outcome`` for ``xpassed`` tests to ``"passed"`` in non-strict + mode and ``"failed"`` in strict mode. Thanks to `@hackebrot`_ for the PR + (`#1795`_) and `@gprasad84`_ for report (`#1546`_). + +* Tests marked with ``xfail(strict=False)`` (the default) now appear in + JUnitXML reports as passing tests instead of skipped. + Thanks to `@hackebrot`_ for the PR (`#1795`_). + +* Highlight path of the file location in the error report to make it easier to copy/paste. + Thanks `@suzaku`_ for the PR (`#1778`_). + +* Fixtures marked with ``@pytest.fixture`` can now use ``yield`` statements exactly like + those marked with the ``@pytest.yield_fixture`` decorator. This change renders + ``@pytest.yield_fixture`` deprecated and makes ``@pytest.fixture`` with ``yield`` statements + the preferred way to write teardown code (`#1461`_). + Thanks `@csaftoiu`_ for bringing this to attention and `@nicoddemus`_ for the PR. + +* Explicitly passed parametrize ids do not get escaped to ascii (`#1351`_). + Thanks `@ceridwen`_ for the PR. + +* Fixtures are now sorted in the error message displayed when an unknown + fixture is declared in a test function. + Thanks `@nicoddemus`_ for the PR. + +* ``pytest_terminal_summary`` hook now receives the ``exitstatus`` + of the test session as argument. Thanks `@blueyed`_ for the PR (`#1809`_). + +* Parametrize ids can accept ``None`` as specific test id, in which case the + automatically generated id for that argument will be used. + Thanks `@palaviv`_ for the complete PR (`#1468`_). + +* The parameter to xunit-style setup/teardown methods (``setup_method``, + ``setup_module``, etc.) is now optional and may be omitted. + Thanks `@okken`_ for bringing this to attention and `@nicoddemus`_ for the PR. + +* Improved automatic id generation selection in case of duplicate ids in + parametrize. + Thanks `@palaviv`_ for the complete PR (`#1474`_). + +* Now pytest warnings summary is shown up by default. Added a new flag + ``--disable-pytest-warnings`` to explicitly disable the warnings summary (`#1668`_). + +* Make ImportError during collection more explicit by reminding + the user to check the name of the test module/package(s) (`#1426`_). + Thanks `@omarkohl`_ for the complete PR (`#1520`_). + +* Add ``build/`` and ``dist/`` to the default ``--norecursedirs`` list. Thanks + `@mikofski`_ for the report and `@tomviner`_ for the PR (`#1544`_). + +* ``pytest.raises`` in the context manager form accepts a custom + ``message`` to raise when no exception occurred. + Thanks `@palaviv`_ for the complete PR (`#1616`_). + +* ``conftest.py`` files now benefit from assertion rewriting; previously it + was only available for test modules. Thanks `@flub`_, `@sober7`_ and + `@nicoddemus`_ for the PR (`#1619`_). + +* Text documents without any doctests no longer appear as "skipped". + Thanks `@graingert`_ for reporting and providing a full PR (`#1580`_). + +* Ensure that a module within a namespace package can be found when it + is specified on the command line together with the ``--pyargs`` + option. Thanks to `@taschini`_ for the PR (`#1597`_). + +* Always include full assertion explanation during assertion rewriting. The previous behaviour was hiding + sub-expressions that happened to be ``False``, assuming this was redundant information. + Thanks `@bagerard`_ for reporting (`#1503`_). Thanks to `@davehunt`_ and + `@tomviner`_ for the PR. + +* ``OptionGroup.addoption()`` now checks if option names were already + added before, to make it easier to track down issues like `#1618`_. + Before, you only got exceptions later from ``argparse`` library, + giving no clue about the actual reason for double-added options. + +* ``yield``-based tests are considered deprecated and will be removed in pytest-4.0. + Thanks `@nicoddemus`_ for the PR. + +* ``[pytest]`` sections in ``setup.cfg`` files should now be named ``[tool:pytest]`` + to avoid conflicts with other distutils commands (see `#567`_). ``[pytest]`` sections in + ``pytest.ini`` or ``tox.ini`` files are supported and unchanged. + Thanks `@nicoddemus`_ for the PR. + +* Using ``pytest_funcarg__`` prefix to declare fixtures is considered deprecated and will be + removed in pytest-4.0 (`#1684`_). + Thanks `@nicoddemus`_ for the PR. + +* Passing a command-line string to ``pytest.main()`` is considered deprecated and scheduled + for removal in pytest-4.0. It is recommended to pass a list of arguments instead (`#1723`_). + +* Rename ``getfuncargvalue`` to ``getfixturevalue``. ``getfuncargvalue`` is + still present but is now considered deprecated. Thanks to `@RedBeardCode`_ and `@tomviner`_ + for the PR (`#1626`_). + +* ``optparse`` type usage now triggers DeprecationWarnings (`#1740`_). + + +* ``optparse`` backward compatibility supports float/complex types (`#457`_). + +* Refined logic for determining the ``rootdir``, considering only valid + paths which fixes a number of issues: `#1594`_, `#1435`_ and `#1471`_. + Updated the documentation according to current behavior. Thanks to + `@blueyed`_, `@davehunt`_ and `@matthiasha`_ for the PR. + +* Always include full assertion explanation. The previous behaviour was hiding + sub-expressions that happened to be False, assuming this was redundant information. + Thanks `@bagerard`_ for reporting (`#1503`_). Thanks to `@davehunt`_ and + `@tomviner`_ for PR. + +* Better message in case of not using parametrized variable (see `#1539`_). + Thanks to `@tramwaj29`_ for the PR. + +* Updated docstrings with a more uniform style. + +* Add stderr write for ``pytest.exit(msg)`` during startup. Previously the message was never shown. + Thanks `@BeyondEvil`_ for reporting `#1210`_. Thanks to `@JonathonSonesen`_ and + `@tomviner`_ for the PR. + +* No longer display the incorrect test deselection reason (`#1372`_). + Thanks `@ronnypfannschmidt`_ for the PR. + +* The ``--resultlog`` command line option has been deprecated: it is little used + and there are more modern and better alternatives (see `#830`_). + Thanks `@nicoddemus`_ for the PR. + +* Improve error message with fixture lookup errors: add an 'E' to the first + line and '>' to the rest. Fixes `#717`_. Thanks `@blueyed`_ for reporting and + a PR, `@eolo999`_ for the initial PR and `@tomviner`_ for his guidance during + EuroPython2016 sprint. + + +**Bug Fixes** + +* Parametrize now correctly handles duplicated test ids. + +* Fix internal error issue when the ``method`` argument is missing for + ``teardown_method()`` (`#1605`_). + +* Fix exception visualization in case the current working directory (CWD) gets + deleted during testing (`#1235`_). Thanks `@bukzor`_ for reporting. PR by + `@marscher`_. + +* Improve test output for logical expression with brackets (`#925`_). + Thanks `@DRMacIver`_ for reporting and `@RedBeardCode`_ for the PR. + +* Create correct diff for strings ending with newlines (`#1553`_). + Thanks `@Vogtinator`_ for reporting and `@RedBeardCode`_ and + `@tomviner`_ for the PR. + +* ``ConftestImportFailure`` now shows the traceback making it easier to + identify bugs in ``conftest.py`` files (`#1516`_). Thanks `@txomon`_ for + the PR. + +* Text documents without any doctests no longer appear as "skipped". + Thanks `@graingert`_ for reporting and providing a full PR (`#1580`_). + +* Fixed collection of classes with custom ``__new__`` method. + Fixes `#1579`_. Thanks to `@Stranger6667`_ for the PR. + +* Fixed scope overriding inside metafunc.parametrize (`#634`_). + Thanks to `@Stranger6667`_ for the PR. + +* Fixed the total tests tally in junit xml output (`#1798`_). + Thanks to `@cryporchild`_ for the PR. + +* Fixed off-by-one error with lines from ``request.node.warn``. + Thanks to `@blueyed`_ for the PR. + + +.. _#1210: https://github.com/pytest-dev/pytest/issues/1210 +.. _#1235: https://github.com/pytest-dev/pytest/issues/1235 +.. _#1351: https://github.com/pytest-dev/pytest/issues/1351 +.. _#1372: https://github.com/pytest-dev/pytest/issues/1372 +.. _#1421: https://github.com/pytest-dev/pytest/issues/1421 +.. _#1426: https://github.com/pytest-dev/pytest/issues/1426 +.. _#1428: https://github.com/pytest-dev/pytest/pull/1428 +.. _#1435: https://github.com/pytest-dev/pytest/issues/1435 +.. _#1441: https://github.com/pytest-dev/pytest/pull/1441 +.. _#1444: https://github.com/pytest-dev/pytest/pull/1444 +.. _#1454: https://github.com/pytest-dev/pytest/pull/1454 +.. _#1461: https://github.com/pytest-dev/pytest/pull/1461 +.. _#1468: https://github.com/pytest-dev/pytest/pull/1468 +.. _#1471: https://github.com/pytest-dev/pytest/issues/1471 +.. _#1474: https://github.com/pytest-dev/pytest/pull/1474 +.. _#1479: https://github.com/pytest-dev/pytest/issues/1479 +.. _#1502: https://github.com/pytest-dev/pytest/pull/1502 +.. _#1503: https://github.com/pytest-dev/pytest/issues/1503 +.. _#1516: https://github.com/pytest-dev/pytest/pull/1516 +.. _#1519: https://github.com/pytest-dev/pytest/pull/1519 +.. _#1520: https://github.com/pytest-dev/pytest/pull/1520 +.. _#1526: https://github.com/pytest-dev/pytest/pull/1526 +.. _#1539: https://github.com/pytest-dev/pytest/issues/1539 +.. _#1544: https://github.com/pytest-dev/pytest/issues/1544 +.. _#1546: https://github.com/pytest-dev/pytest/issues/1546 +.. _#1553: https://github.com/pytest-dev/pytest/issues/1553 +.. _#1562: https://github.com/pytest-dev/pytest/issues/1562 +.. _#1579: https://github.com/pytest-dev/pytest/issues/1579 +.. _#1580: https://github.com/pytest-dev/pytest/pull/1580 +.. _#1594: https://github.com/pytest-dev/pytest/issues/1594 +.. _#1597: https://github.com/pytest-dev/pytest/pull/1597 +.. _#1605: https://github.com/pytest-dev/pytest/issues/1605 +.. _#1616: https://github.com/pytest-dev/pytest/pull/1616 +.. _#1618: https://github.com/pytest-dev/pytest/issues/1618 +.. _#1619: https://github.com/pytest-dev/pytest/issues/1619 +.. _#1626: https://github.com/pytest-dev/pytest/pull/1626 +.. _#1627: https://github.com/pytest-dev/pytest/pull/1627 +.. _#1628: https://github.com/pytest-dev/pytest/pull/1628 +.. _#1629: https://github.com/pytest-dev/pytest/issues/1629 +.. _#1632: https://github.com/pytest-dev/pytest/issues/1632 +.. _#1633: https://github.com/pytest-dev/pytest/pull/1633 +.. _#1664: https://github.com/pytest-dev/pytest/pull/1664 +.. _#1668: https://github.com/pytest-dev/pytest/issues/1668 +.. _#1684: https://github.com/pytest-dev/pytest/pull/1684 +.. _#1723: https://github.com/pytest-dev/pytest/pull/1723 +.. _#1740: https://github.com/pytest-dev/pytest/issues/1740 +.. _#1749: https://github.com/pytest-dev/pytest/issues/1749 +.. _#1778: https://github.com/pytest-dev/pytest/pull/1778 +.. _#1795: https://github.com/pytest-dev/pytest/pull/1795 +.. _#1798: https://github.com/pytest-dev/pytest/pull/1798 +.. _#1809: https://github.com/pytest-dev/pytest/pull/1809 +.. _#372: https://github.com/pytest-dev/pytest/issues/372 +.. _#457: https://github.com/pytest-dev/pytest/issues/457 +.. _#460: https://github.com/pytest-dev/pytest/pull/460 +.. _#567: https://github.com/pytest-dev/pytest/pull/567 +.. _#607: https://github.com/pytest-dev/pytest/issues/607 +.. _#634: https://github.com/pytest-dev/pytest/issues/634 +.. _#717: https://github.com/pytest-dev/pytest/issues/717 +.. _#830: https://github.com/pytest-dev/pytest/issues/830 +.. _#925: https://github.com/pytest-dev/pytest/issues/925 + + +.. _@anntzer: https://github.com/anntzer +.. _@bagerard: https://github.com/bagerard +.. _@BeyondEvil: https://github.com/BeyondEvil +.. _@blueyed: https://github.com/blueyed +.. _@ceridwen: https://github.com/ceridwen +.. _@cryporchild: https://github.com/cryporchild +.. _@csaftoiu: https://github.com/csaftoiu +.. _@d6e: https://github.com/d6e +.. _@davehunt: https://github.com/davehunt +.. _@DRMacIver: https://github.com/DRMacIver +.. _@eolo999: https://github.com/eolo999 +.. _@fengxx: https://github.com/fengxx +.. _@flub: https://github.com/flub +.. _@gprasad84: https://github.com/gprasad84 +.. _@graingert: https://github.com/graingert +.. _@hartym: https://github.com/hartym +.. _@JonathonSonesen: https://github.com/JonathonSonesen +.. _@kalekundert: https://github.com/kalekundert +.. _@kvas-it: https://github.com/kvas-it +.. _@marscher: https://github.com/marscher +.. _@mikofski: https://github.com/mikofski +.. _@milliams: https://github.com/milliams +.. _@nikratio: https://github.com/nikratio +.. _@novas0x2a: https://github.com/novas0x2a +.. _@obestwalter: https://github.com/obestwalter +.. _@okken: https://github.com/okken +.. _@olegpidsadnyi: https://github.com/olegpidsadnyi +.. _@omarkohl: https://github.com/omarkohl +.. _@palaviv: https://github.com/palaviv +.. _@RedBeardCode: https://github.com/RedBeardCode +.. _@sallner: https://github.com/sallner +.. _@sober7: https://github.com/sober7 +.. _@Stranger6667: https://github.com/Stranger6667 +.. _@suzaku: https://github.com/suzaku +.. _@tareqalayan: https://github.com/tareqalayan +.. _@taschini: https://github.com/taschini +.. _@tramwaj29: https://github.com/tramwaj29 +.. _@txomon: https://github.com/txomon +.. _@Vogtinator: https://github.com/Vogtinator +.. _@matthiasha: https://github.com/matthiasha + + +2.9.2 (2016-05-31) +================== + +**Bug Fixes** + +* fix `#510`_: skip tests where one parameterize dimension was empty + thanks Alex Stapleton for the Report and `@RonnyPfannschmidt`_ for the PR + +* Fix Xfail does not work with condition keyword argument. + Thanks `@astraw38`_ for reporting the issue (`#1496`_) and `@tomviner`_ + for PR the (`#1524`_). + +* Fix win32 path issue when putting custom config file with absolute path + in ``pytest.main("-c your_absolute_path")``. + +* Fix maximum recursion depth detection when raised error class is not aware + of unicode/encoded bytes. + Thanks `@prusse-martin`_ for the PR (`#1506`_). + +* Fix ``pytest.mark.skip`` mark when used in strict mode. + Thanks `@pquentin`_ for the PR and `@RonnyPfannschmidt`_ for + showing how to fix the bug. + +* Minor improvements and fixes to the documentation. + Thanks `@omarkohl`_ for the PR. + +* Fix ``--fixtures`` to show all fixture definitions as opposed to just + one per fixture name. + Thanks to `@hackebrot`_ for the PR. + +.. _#510: https://github.com/pytest-dev/pytest/issues/510 +.. _#1506: https://github.com/pytest-dev/pytest/pull/1506 +.. _#1496: https://github.com/pytest-dev/pytest/issues/1496 +.. _#1524: https://github.com/pytest-dev/pytest/pull/1524 + +.. _@prusse-martin: https://github.com/prusse-martin +.. _@astraw38: https://github.com/astraw38 + + +2.9.1 (2016-03-17) +================== **Bug Fixes** @@ -23,17 +1626,19 @@ * Fix (`#649`_): parametrized test nodes cannot be specified to run on the command line. +* Fix (`#138`_): better reporting for python 3.3+ chained exceptions .. _#1437: https://github.com/pytest-dev/pytest/issues/1437 .. _#469: https://github.com/pytest-dev/pytest/issues/469 .. _#1431: https://github.com/pytest-dev/pytest/pull/1431 .. _#649: https://github.com/pytest-dev/pytest/issues/649 +.. _#138: https://github.com/pytest-dev/pytest/issues/138 .. _@asottile: https://github.com/asottile -2.9.0 -===== +2.9.0 (2016-02-29) +================== **New Features** @@ -51,7 +1656,7 @@ ``xfail_strict`` ini option that can be used to configure it project-wise. Thanks `@rabbbit`_ for the request and `@nicoddemus`_ for the PR (`#1355`_). -* ``Parser.addini`` now supports options of type ``bool``. +* ``Parser.addini`` now supports options of type ``bool``. Thanks `@nicoddemus`_ for the PR. * New ``ALLOW_BYTES`` doctest option. This strips ``b`` prefixes from byte strings @@ -62,26 +1667,26 @@ Fixes `#1366`_. Thanks to `@hpk42`_ for the report and `@RonnyPfannschmidt`_ for the PR. -* Catch ``IndexError`` exceptions when getting exception source location. +* Catch ``IndexError`` exceptions when getting exception source location. Fixes a pytest internal error for dynamically generated code (fixtures and tests) where source lines are fake by intention. **Changes** -* **Important**: `py.code `_ has been - merged into the ``pytest`` repository as ``pytest._code``. This decision - was made because ``py.code`` had very few uses outside ``pytest`` and the - fact that it was in a different repository made it difficult to fix bugs on +* **Important**: `py.code `_ has been + merged into the ``pytest`` repository as ``pytest._code``. This decision + was made because ``py.code`` had very few uses outside ``pytest`` and the + fact that it was in a different repository made it difficult to fix bugs on its code in a timely manner. The team hopes with this to be able to better refactor out and improve that code. This change shouldn't affect users, but it is useful to let users aware if they encounter any strange behavior. - - Keep in mind that the code for ``pytest._code`` is **private** and + + Keep in mind that the code for ``pytest._code`` is **private** and **experimental**, so you definitely should not import it explicitly! - Please note that the original ``py.code`` is still available in - `pylib `_. + Please note that the original ``py.code`` is still available in + `pylib `_. * ``pytest_enter_pdb`` now optionally receives the pytest config object. Thanks `@nicoddemus`_ for the PR. @@ -117,7 +1722,7 @@ Thanks `@biern`_ for the PR. * Fix `traceback style docs`_ to describe all of the available options - (auto/long/short/line/native/no), with `auto` being the default since v2.6. + (auto/long/short/line/native/no), with ``auto`` being the default since v2.6. Thanks `@hackebrot`_ for the PR. * Fix (`#1422`_): junit record_xml_property doesn't allow multiple records @@ -125,6 +1730,7 @@ .. _`traceback style docs`: https://pytest.org/latest/usage.html#modifying-python-traceback-printing +.. _#1609: https://github.com/pytest-dev/pytest/issues/1609 .. _#1422: https://github.com/pytest-dev/pytest/issues/1422 .. _#1379: https://github.com/pytest-dev/pytest/issues/1379 .. _#1366: https://github.com/pytest-dev/pytest/issues/1366 @@ -149,14 +1755,16 @@ .. _@RonnyPfannschmidt: https://github.com/RonnyPfannschmidt .. _@rabbbit: https://github.com/rabbbit .. _@hackebrot: https://github.com/hackebrot +.. _@pquentin: https://github.com/pquentin +.. _@ioggstream: https://github.com/ioggstream -2.8.7 -===== +2.8.7 (2016-01-24) +================== - fix #1338: use predictable object resolution for monkeypatch -2.8.6 -===== +2.8.6 (2016-01-21) +================== - fix #1259: allow for double nodeids in junitxml, this was a regression failing plugins combinations @@ -172,7 +1780,7 @@ - fix #1292: monkeypatch calls (setattr, setenv, etc.) are now O(1). Thanks David R. MacIver for the report and Bruno Oliveira for the PR. -- fix #1223: captured stdout and stderr are now properly displayed before +- fix #1223: captured stdout and stderr are now properly displayed before entering pdb when ``--pdb`` is used instead of being thrown away. Thanks Cal Leeming for the PR. @@ -187,8 +1795,8 @@ Thanks Georgy Dyuldin for the PR. -2.8.5 -===== +2.8.5 (2015-12-11) +================== - fix #1243: fixed issue where class attributes injected during collection could break pytest. PR by Alexei Kozlenok, thanks Ronny Pfannschmidt and Bruno Oliveira for the review and help. @@ -201,8 +1809,8 @@ Bruno Oliveira for the PR. -2.8.4 -===== +2.8.4 (2015-12-06) +================== - fix #1190: ``deprecated_call()`` now works when the deprecated function has been already called by another test in the same @@ -225,8 +1833,8 @@ - a number of documentation modernizations wrt good practices. Thanks Bruno Oliveira for the PR. -2.8.3 -===== +2.8.3 (2015-11-18) +================== - fix #1169: add __name__ attribute to testcases in TestCaseFunction to support the @unittest.skip decorator on functions and methods. @@ -247,14 +1855,14 @@ Thanks Gabriel Reis for the PR. - add more talks to the documentation -- extend documentation on the --ignore cli option -- use pytest-runner for setuptools integration +- extend documentation on the --ignore cli option +- use pytest-runner for setuptools integration - minor fixes for interaction with OS X El Capitan system integrity protection (thanks Florian) -2.8.2 -===== +2.8.2 (2015-10-07) +================== - fix #1085: proper handling of encoding errors when passing encoded byte strings to pytest.parametrize in Python 2. @@ -273,8 +1881,8 @@ Thanks Sergey B Kirpichev and Vital Kudzelka for contributing and Bruno Oliveira for the PR. -2.8.1 -===== +2.8.1 (2015-09-29) +================== - fix #1034: Add missing nodeid on pytest_logwarning call in addhook. Thanks Simon Gomizelj for the PR. @@ -298,7 +1906,7 @@ - (experimental) adapt more SEMVER style versioning and change meaning of master branch in git repo: "master" branch now keeps the bugfixes, changes - aimed for micro releases. "features" branch will only be be released + aimed for micro releases. "features" branch will only be released with minor or major pytest releases. - Fix issue #766 by removing documentation references to distutils. @@ -320,8 +1928,8 @@ - fix issue 1029: transform errors when writing cache values into pytest-warnings -2.8.0 -===== +2.8.0 (2015-09-18) +================== - new ``--lf`` and ``-ff`` options to run only the last failing tests or "failing tests first" from the last run. This functionality is provided @@ -432,7 +2040,7 @@ - new option ``--import-mode`` to allow to change test module importing behaviour to append to sys.path instead of prepending. This better allows - to run test modules against installated versions of a package even if the + to run test modules against installed versions of a package even if the package under test has the same import root. In this example:: testing/__init__.py @@ -510,8 +2118,8 @@ properly used to discover ``rootdir`` and ``ini`` files. Thanks Peter Lauri for the report and Bruno Oliveira for the PR. -2.7.3 (compared to 2.7.2) -============================= +2.7.3 (2015-09-15) +================== - Allow 'dev', 'rc', or other non-integer version strings in ``importorskip``. Thanks to Eric Hunsberger for the PR. @@ -553,8 +2161,8 @@ directories created by this fixture (defaults to $TEMP/pytest-$USER). Thanks Bruno Oliveira for the PR. -2.7.2 (compared to 2.7.1) -============================= +2.7.2 (2015-06-23) +================== - fix issue767: pytest.raises value attribute does not contain the exception instance on Python 2.6. Thanks Eric Siegerman for providing the test @@ -582,15 +2190,15 @@ which has a refined algorithm for traceback generation. -2.7.1 (compared to 2.7.0) -============================= +2.7.1 (2015-05-19) +================== - fix issue731: do not get confused by the braces which may be present and unbalanced in an object's repr while collapsing False explanations. Thanks Carl Meyer for the report and test case. - fix issue553: properly handling inspect.getsourcelines failures in - FixtureLookupError which would lead to to an internal error, + FixtureLookupError which would lead to an internal error, obfuscating the original problem. Thanks talljosh for initial diagnose/patch and Bruno Oliveira for final patch. @@ -615,8 +2223,8 @@ - reintroduced _pytest fixture of the pytester plugin which is used at least by pytest-xdist. -2.7.0 (compared to 2.6.4) -============================= +2.7.0 (2015-03-26) +================== - fix issue435: make reload() work when assert rewriting is active. Thanks Daniel Hahler. @@ -648,7 +2256,7 @@ - fix issue655: work around different ways that cause python2/3 to leak sys.exc_info into fixtures/tests causing failures in 3rd party code -- fix issue615: assertion re-writing did not correctly escape % signs +- fix issue615: assertion rewriting did not correctly escape % signs when formatting boolean operations, which tripped over mixing booleans with modulo operators. Thanks to Tom Viner for the report, triaging and fix. @@ -685,8 +2293,8 @@ ``sys.last_traceback`` are set, so that a user can inspect the error via postmortem debugging (almarklein). -2.6.4 -===== +2.6.4 (2014-10-24) +================== - Improve assertion failure reporting on iterables, by using ndiff and pprint. @@ -714,8 +2322,8 @@ - fix issue614: fixed pastebin support. -2.6.3 -===== +2.6.3 (2014-09-24) +================== - fix issue575: xunit-xml was reporting collection errors as failures instead of errors, thanks Oleg Sinyavskiy. @@ -732,8 +2340,8 @@ dep). Thanks Charles Cloud for analysing the issue. - fix conftest related fixture visibility issue: when running with a - CWD outside a test package pytest would get fixture discovery wrong. - Thanks to Wolfgang Schnerring for figuring out a reproducable example. + CWD outside of a test package pytest would get fixture discovery wrong. + Thanks to Wolfgang Schnerring for figuring out a reproducible example. - Introduce pytest_enter_pdb hook (needed e.g. by pytest_timeout to cancel the timeout when interactively entering pdb). Thanks Wolfgang Schnerring. @@ -741,8 +2349,8 @@ - check xfail/skip also with non-python function test items. Thanks Floris Bruynooghe. -2.6.2 -===== +2.6.2 (2014-09-05) +================== - Added function pytest.freeze_includes(), which makes it easy to embed pytest into executables using tools like cx_freeze. @@ -770,8 +2378,8 @@ replace the py.test introspection message but are shown in addition to them. -2.6.1 -===== +2.6.1 (2014-08-07) +================== - No longer show line numbers in the --verbose output, the output is now purely the nodeid. The line number is still shown in failure reports. @@ -907,8 +2515,8 @@ in monkeypatch plugin. Improves output in documentation. -2.5.2 -===== +2.5.2 (2014-01-29) +================== - fix issue409 -- better interoperate with cx_freeze by not trying to import from collections.abc which causes problems @@ -932,11 +2540,11 @@ - fix issue429: comparing byte strings with non-ascii chars in assert expressions now work better. Thanks Floris Bruynooghe. -- make capfd/capsys.capture private, its unused and shouldnt be exposed +- make capfd/capsys.capture private, its unused and shouldn't be exposed -2.5.1 -===== +2.5.1 (2013-12-17) +================== - merge new documentation styling PR from Tobias Bieniek. @@ -956,8 +2564,8 @@ -2.5.0 -===== +2.5.0 (2013-12-12) +================== - dropped python2.5 from automated release testing of pytest itself which means it's probably going to break soon (but still works @@ -989,7 +2597,7 @@ to problems for more than >966 non-function scoped parameters). - fix issue290 - there is preliminary support now for parametrizing - with repeated same values (sometimes useful to to test if calling + with repeated same values (sometimes useful to test if calling a second time works as with the first time). - close issue240 - document precisely how pytest module importing @@ -1092,8 +2700,8 @@ - fix verbose reporting for @mock'd test functions -2.4.2 -===== +2.4.2 (2013-10-04) +================== - on Windows require colorama and a newer py lib so that py.io.TerminalWriter() now uses colorama instead of its own ctypes hacks. (fixes issue365) @@ -1123,8 +2731,8 @@ - add pluginmanager.do_configure(config) as a link to config.do_configure() for plugin-compatibility -2.4.1 -===== +2.4.1 (2013-10-02) +================== - When using parser.addoption() unicode arguments to the "type" keyword should also be converted to the respective types. @@ -1203,7 +2811,7 @@ new features: - fix issue322: tearDownClass is not run if setUpClass failed. Thanks Mathieu Agopian for the initial fix. Also make all of pytest/nose - finalizer mimick the same generic behaviour: if a setupX exists and + finalizer mimic the same generic behaviour: if a setupX exists and fails, don't run teardownX. This internally introduces a new method "node.addfinalizer()" helper which can only be called during the setup phase of a node. @@ -1308,8 +2916,8 @@ Bug fixes: ".section(title)" and ".line(msg)" methods to print extra information at the end of a test run. -2.3.5 -===== +2.3.5 (2013-04-30) +================== - fix issue169: respect --tb=style with setup/teardown errors as well. @@ -1322,11 +2930,11 @@ Bug fixes: (thanks Adam Goucher) - Issue 265 - integrate nose setup/teardown with setupstate - so it doesnt try to teardown if it did not setup + so it doesn't try to teardown if it did not setup -- issue 271 - dont write junitxml on slave nodes +- issue 271 - don't write junitxml on slave nodes -- Issue 274 - dont try to show full doctest example +- Issue 274 - don't try to show full doctest example when doctest does not know the example location - issue 280 - disable assertion rewriting on buggy CPython 2.6.0 @@ -1362,7 +2970,7 @@ Bug fixes: - allow to specify prefixes starting with "_" when customizing python_functions test discovery. (thanks Graham Horler) -- improve PYTEST_DEBUG tracing output by puting +- improve PYTEST_DEBUG tracing output by putting extra data on a new lines with additional indent - ensure OutcomeExceptions like skip/fail have initialized exception attributes @@ -1373,8 +2981,8 @@ Bug fixes: - fix issue266 - accept unicode in MarkEvaluator expressions -2.3.4 -===== +2.3.4 (2012-11-20) +================== - yielded test functions will now have autouse-fixtures active but cannot accept fixtures as funcargs - it's anyway recommended to @@ -1393,8 +3001,8 @@ Bug fixes: need to write as -k "TestClass and test_method" to match a certain method in a certain test class. -2.3.3 -===== +2.3.3 (2012-11-06) +================== - fix issue214 - parse modules that contain special objects like e. g. flask's request object which blows up on getattr access if no request @@ -1411,7 +3019,7 @@ Bug fixes: - fix issue209 - reintroduce python2.4 support by depending on newer pylib which re-introduced statement-finding for pre-AST interpreters -- nose support: only call setup if its a callable, thanks Andrew +- nose support: only call setup if it's a callable, thanks Andrew Taumoefolau - fix issue219 - add py2.4-3.3 classifiers to TROVE list @@ -1425,8 +3033,8 @@ Bug fixes: - fix issue127 - improve documentation for pytest_addoption() and add a ``config.getoption(name)`` helper function for consistency. -2.3.2 -===== +2.3.2 (2012-10-25) +================== - fix issue208 and fix issue29 use new py version to avoid long pauses when printing tracebacks in long modules @@ -1458,8 +3066,8 @@ Bug fixes: - add tox.ini to pytest distribution so that ignore-dirs and others config bits are properly distributed for maintainers who run pytest-own tests -2.3.1 -===== +2.3.1 (2012-10-20) +================== - fix issue202 - fix regression: using "self" from fixture functions now works as expected (it's the same "self" instance that a test method @@ -1471,8 +3079,8 @@ Bug fixes: - link to web pages from --markers output which provides help for pytest.mark.* usage. -2.3.0 -===== +2.3.0 (2012-10-19) +================== - fix issue202 - better automatic names for parametrized test functions - fix issue139 - introduce @pytest.fixture which allows direct scoping @@ -1507,7 +3115,7 @@ Bug fixes: - fix issue128: show captured output when capsys/capfd are used -- fix issue179: propperly show the dependency chain of factories +- fix issue179: properly show the dependency chain of factories - pluginmanager.register(...) now raises ValueError if the plugin has been already registered or the name is taken @@ -1548,10 +3156,10 @@ Bug fixes: - don't show deselected reason line if there is none - - py.test -vv will show all of assert comparisations instead of truncating + - py.test -vv will show all of assert comparisons instead of truncating -2.2.4 -===== +2.2.4 (2012-05-22) +================== - fix error message for rewritten assertions involving the % operator - fix issue 126: correctly match all invalid xml characters for junitxml @@ -1559,7 +3167,7 @@ Bug fixes: - fix issue with unittest: now @unittest.expectedFailure markers should be processed correctly (you can also use @pytest.mark markers) - document integration with the extended distribute/setuptools test commands -- fix issue 140: propperly get the real functions +- fix issue 140: properly get the real functions of bound classmethods for setup/teardown_class - fix issue #141: switch from the deceased paste.pocoo.org to bpaste.net - fix issue #143: call unconfigure/sessionfinish always when @@ -1567,13 +3175,13 @@ Bug fixes: - fix issue #144: better mangle test ids to junitxml classnames - upgrade distribute_setup.py to 0.6.27 -2.2.3 -===== +2.2.3 (2012-02-05) +================== -- fix uploaded package to only include neccesary files +- fix uploaded package to only include necessary files -2.2.2 -===== +2.2.2 (2012-02-05) +================== - fix issue101: wrong args to unittest.TestCase test function now produce better output @@ -1592,8 +3200,8 @@ Bug fixes: - allow adding of attributes to test reports such that it also works with distributed testing (no upgrade of pytest-xdist needed) -2.2.1 -===== +2.2.1 (2011-12-16) +================== - fix issue99 (in pytest and py) internallerrors with resultlog now produce better output - fixed by normalizing pytest_internalerror @@ -1609,14 +3217,14 @@ Bug fixes: - fix collection crash due to unknown-source collected items, thanks to Ralf Schmitt (fixed by depending on a more recent pylib) -2.2.0 -===== +2.2.0 (2011-11-18) +================== - fix issue90: introduce eager tearing down of test items so that teardown function are called earlier. - add an all-powerful metafunc.parametrize function which allows to parametrize test function arguments in multiple steps and therefore - from indepdenent plugins and palces. + from independent plugins and places. - add a @pytest.mark.parametrize helper which allows to easily call a test function with different argument values - Add examples to the "parametrize" example page, including a quick port @@ -1644,8 +3252,8 @@ Bug fixes: - simplify junitxml output code by relying on py.xml - add support for skip properties on unittest classes and functions -2.1.3 -===== +2.1.3 (2011-10-18) +================== - fix issue79: assertion rewriting failed on some comparisons in boolops - correctly handle zero length arguments (a la pytest '') @@ -1653,8 +3261,8 @@ Bug fixes: - fix issue75 / skipping test failure on jython - fix issue77 / Allow assertrepr_compare hook to apply to a subset of tests -2.1.2 -===== +2.1.2 (2011-09-24) +================== - fix assertion rewriting on files with windows newlines on some Python versions - refine test discovery by package/module name (--pyargs), thanks Florian Mayer @@ -1676,8 +3284,8 @@ Bug fixes: - fix issue61: assertion rewriting on boolean operations with 3 or more operands - you can now build a man page with "cd doc ; make man" -2.1.0 -===== +2.1.0 (2011-07-09) +================== - fix issue53 call nosestyle setup functions with correct ordering - fix issue58 and issue59: new assertion code fixes @@ -1696,8 +3304,8 @@ Bug fixes: - report KeyboardInterrupt even if interrupted during session startup - fix issue 35 - provide PDF doc version and download link from index page -2.0.3 -===== +2.0.3 (2011-05-11) +================== - fix issue38: nicer tracebacks on calls to hooks, particularly early configure/sessionstart ones @@ -1711,13 +3319,13 @@ Bug fixes: - don't require zlib (and other libs) for genscript plugin without --genscript actually being used. -- speed up skips (by not doing a full traceback represenation +- speed up skips (by not doing a full traceback representation internally) - fix issue37: avoid invalid characters in junitxml's output -2.0.2 -===== +2.0.2 (2011-03-09) +================== - tackle issue32 - speed up test runs of very quick test functions by reducing the relative overhead @@ -1759,17 +3367,17 @@ Bug fixes: this. - fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular - thanks to Laura Creighton who also revieved parts of the documentation. + thanks to Laura Creighton who also reviewed parts of the documentation. -- fix slighly wrong output of verbose progress reporting for classes +- fix slightly wrong output of verbose progress reporting for classes (thanks Amaury) - more precise (avoiding of) deprecation warnings for node.Class|Function accesses - avoid std unittest assertion helper code in tracebacks (thanks Ronny) -2.0.1 -===== +2.0.1 (2011-02-07) +================== - refine and unify initial capturing so that it works nicely even if the logging module is used on an early-loaded conftest.py @@ -1817,12 +3425,12 @@ Bug fixes: parametraization remains the "pytest_generate_tests" mechanism, see the docs. -2.0.0 -===== +2.0.0 (2010-11-25) +================== - pytest-2.0 is now its own package and depends on pylib-2.0 - new ability: python -m pytest / python -m pytest.main ability -- new python invcation: pytest.main(args, plugins) to load +- new python invocation: pytest.main(args, plugins) to load some custom plugins early. - try harder to run unittest test suites in a more compatible manner by deferring setup/teardown semantics to the unittest package. @@ -1862,8 +3470,8 @@ Bug fixes: - add ability to use "class" level for cached_setup helper - fix strangeness: mark.* objects are now immutable, create new instances -1.3.4 -===== +1.3.4 (2010-09-14) +================== - fix issue111: improve install documentation for windows - fix issue119: fix custom collectability of __init__.py as a module @@ -1871,8 +3479,8 @@ Bug fixes: - fix issue115: unify internal exception passthrough/catching/GeneratorExit - fix issue118: new --tb=native for presenting cpython-standard exceptions -1.3.3 -===== +1.3.3 (2010-07-30) +================== - fix issue113: assertion representation problem with triple-quoted strings (and possibly other cases) @@ -1886,8 +3494,8 @@ Bug fixes: (thanks Armin Ronacher for reporting) - remove trailing whitespace in all py/text distribution files -1.3.2 -===== +1.3.2 (2010-07-08) +================== **New features** @@ -1959,8 +3567,8 @@ Bug fixes: - fix homedir detection on Windows - ship distribute_setup.py version 0.6.13 -1.3.1 -===== +1.3.1 (2010-05-25) +================== **New features** @@ -2029,8 +3637,8 @@ Bug fixes: (and internally be more careful when presenting unexpected byte sequences) -1.3.0 -===== +1.3.0 (2010-05-05) +================== - deprecate --report option in favour of a new shorter and easier to remember -r option: it takes a string argument consisting of any @@ -2061,7 +3669,7 @@ Bug fixes: - extend and refine xfail mechanism: ``@py.test.mark.xfail(run=False)`` do not run the decorated test ``@py.test.mark.xfail(reason="...")`` prints the reason string in xfail summaries - specifiying ``--runxfail`` on command line virtually ignores xfail markers + specifying ``--runxfail`` on command line virtually ignores xfail markers - expose (previously internal) commonly useful methods: py.io.get_terminal_with() -> return terminal width @@ -2094,8 +3702,8 @@ Bug fixes: - added links to the new capturelog and coverage plugins -1.2.0 -===== +1.2.0 (2010-01-18) +================== - refined usage and options for "py.cleanup":: @@ -2133,8 +3741,8 @@ Bug fixes: - fix plugin links -1.1.1 -===== +1.1.1 (2009-11-24) +================== - moved dist/looponfailing from py.test core into a new separately released pytest-xdist plugin. @@ -2217,8 +3825,8 @@ Bug fixes: - fix docs, fix internal bin/ script generation -1.1.0 -===== +1.1.0 (2009-11-05) +================== - introduce automatic plugin registration via 'pytest11' entrypoints via setuptools' pkg_resources.iter_entry_points @@ -2286,7 +3894,7 @@ Bug fixes: * add the ability to specify a path for py.lookup to search in -* fix a funcarg cached_setup bug probably only occuring +* fix a funcarg cached_setup bug probably only occurring in distributed testing and "module" scope with teardown. * many fixes and changes for making the code base python3 compatible, @@ -2321,16 +3929,16 @@ Bug fixes: * simplified internal localpath implementation -1.0.2 -===== +1.0.2 (2009-08-27) +================== * fixing packaging issues, triggered by fedora redhat packaging, also added doc, examples and contrib dirs to the tarball. * added a documentation link to the new django plugin. -1.0.1 -===== +1.0.1 (2009-08-19) +================== * added a 'pytest_nose' plugin which handles nose.SkipTest, nose-style function/method/generator setup/teardown and @@ -2363,14 +3971,14 @@ Bug fixes: * simplified multicall mechanism and plugin architecture, renamed some internal methods and argnames -1.0.0 -===== +1.0.0 (2009-08-04) +================== * more terse reporting try to show filesystem path relatively to current dir * improve xfail output a bit -1.0.0b9 -======= +1.0.0b9 (2009-07-31) +==================== * cleanly handle and report final teardown of test setup @@ -2403,8 +4011,8 @@ Bug fixes: * item.repr_failure(excinfo) instead of item.repr_failure(excinfo, outerr) -1.0.0b8 -======= +1.0.0b8 (2009-07-22) +==================== * pytest_unittest-plugin is now enabled by default @@ -2457,8 +4065,8 @@ Bug fixes: * make __name__ == "__channelexec__" for remote_exec code -1.0.0b3 -======= +1.0.0b3 (2009-06-19) +==================== * plugin classes are removed: one now defines hooks directly in conftest.py or global pytest_*.py @@ -2552,10 +4160,10 @@ serve as a reference for developers. * fixed issue with 2.5 type representations in py.test [45483, 45484] * made that internal reporting issues displaying is done atomically in py.test [45518] -* made that non-existing files are igored by the py.lookup script [45519] +* made that non-existing files are ignored by the py.lookup script [45519] * improved exception name creation in py.test [45535] * made that less threads are used in execnet [merge in 45539] -* removed lock required for atomical reporting issue displaying in py.test +* removed lock required for atomic reporting issue displaying in py.test [45545] * removed globals from execnet [45541, 45547] * refactored cleanup mechanics, made that setDaemon is set to 1 to make atexit diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 75ee3ec32d3ad16..d85a894b910777e 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -34,13 +34,13 @@ If you are reporting a bug, please include: * Your operating system name and version. * Any details about your local setup that might be helpful in troubleshooting, - specifically Python interpreter version, - installed libraries and pytest version. + specifically the Python interpreter version, installed libraries, and pytest + version. * Detailed steps to reproduce the bug. -If you can write a demonstration test that currently fails but should pass (xfail), -that is a very useful commit to make as well, even if you can't find how -to fix the bug yet. +If you can write a demonstration test that currently fails but should pass +(xfail), that is a very useful commit to make as well, even if you cannot +fix the bug itself. .. _fixbugs: @@ -48,8 +48,8 @@ to fix the bug yet. Fix bugs -------- -Look through the GitHub issues for bugs. Here is sample filter you can use: -https://github.com/pytest-dev/pytest/labels/bug +Look through the GitHub issues for bugs. Here is a filter you can use: +https://github.com/pytest-dev/pytest/labels/type%3A%20bug :ref:`Talk ` to developers to find out how you can fix specific bugs. @@ -60,8 +60,7 @@ Don't forget to check the issue trackers of your favourite plugins, too! Implement features ------------------ -Look through the GitHub issues for enhancements. Here is sample filter you -can use: +Look through the GitHub issues for enhancements. Here is a filter you can use: https://github.com/pytest-dev/pytest/labels/enhancement :ref:`Talk ` to developers to find out how you can implement specific @@ -70,17 +69,26 @@ features. Write documentation ------------------- -pytest could always use more documentation. What exactly is needed? +Pytest could always use more documentation. What exactly is needed? * More complementary documentation. Have you perhaps found something unclear? * Documentation translations. We currently have only English. * Docstrings. There can never be too many of them. * Blog posts, articles and such -- they're all very appreciated. -You can also edit documentation files directly in the Github web interface -without needing to make a fork and local copy. This can be convenient for -small fixes. +You can also edit documentation files directly in the GitHub web interface, +without using a local copy. This can be convenient for small fixes. +.. note:: + Build the documentation locally with the following command: + + .. code:: bash + + $ tox -e docs + + The built documentation should be available in the ``doc/en/_build/``. + + Where 'en' refers to the documentation language. .. _submitplugin: @@ -95,13 +103,14 @@ in repositories living under the ``pytest-dev`` organisations: - `pytest-dev on Bitbucket `_ All pytest-dev Contributors team members have write access to all contained -repositories. pytest core and plugins are generally developed +repositories. Pytest core and plugins are generally developed using `pull requests`_ to respective repositories. The objectives of the ``pytest-dev`` organisation are: * Having a central location for popular pytest plugins -* Sharing some of the maintenance responsibility (in case a maintainer no longer whishes to maintain a plugin) +* Sharing some of the maintenance responsibility (in case a maintainer no + longer wishes to maintain a plugin) You can submit your plugin by subscribing to the `pytest-dev mail list `_ and writing a @@ -111,7 +120,7 @@ the following: - PyPI presence with a ``setup.py`` that contains a license, ``pytest-`` prefixed name, version number, authors, short and long description. -- a ``tox.ini`` for running tests using `tox `_. +- a ``tox.ini`` for running tests using `tox `_. - a ``README.txt`` describing how to use the plugin and on which platforms it runs. @@ -121,33 +130,26 @@ the following: - an issue tracker for bug reports and enhancement requests. +- a `changelog `_ + If no contributor strongly objects and two agree, the repository can then be transferred to the ``pytest-dev`` organisation. Here's a rundown of how a repository transfer usually proceeds (using a repository named ``joedoe/pytest-xyz`` as example): -* One of the ``pytest-dev`` administrators creates: - - - ``pytest-xyz-admin`` team, with full administration rights to - ``pytest-dev/pytest-xyz``. - - ``pytest-xyz-developers`` team, with write access to - ``pytest-dev/pytest-xyz``. - -* ``joedoe`` is invited to the ``pytest-xyz-admin`` team; - -* After accepting the invitation, ``joedoe`` transfers the repository from its - original location to ``pytest-dev/pytest-xyz`` (A nice feature is that GitHub handles URL redirection from - the old to the new location automatically). - -* ``joedoe`` is free to add any other collaborators to the - ``pytest-xyz-admin`` or ``pytest-xyz-developers`` team as desired. +* ``joedoe`` transfers repository ownership to ``pytest-dev`` administrator ``calvin``. +* ``calvin`` creates ``pytest-xyz-admin`` and ``pytest-xyz-developers`` teams, inviting ``joedoe`` to both as **maintainer**. +* ``calvin`` transfers repository to ``pytest-dev`` and configures team access: + + - ``pytest-xyz-admin`` **admin** access; + - ``pytest-xyz-developers`` **write** access; The ``pytest-dev/Contributors`` team has write access to all projects, and every project administrator is in it. We recommend that each plugin has at least three people who have the right to release to PyPI. -Repository owners can be assured that no ``pytest-dev`` administrator will ever make +Repository owners can rest assured that no ``pytest-dev`` administrator will ever make releases of your repository or take ownership in any way, except in rare cases where someone becomes unresponsive after months of contact attempts. As stated, the objective is to share maintenance and avoid "plugin-abandon". @@ -156,23 +158,41 @@ As stated, the objective is to share maintenance and avoid "plugin-abandon". .. _`pull requests`: .. _pull-requests: -Preparing Pull Requests on GitHub ---------------------------------- +Preparing Pull Requests +----------------------- -There's an excellent tutorial on how Pull Requests work in the -`GitHub Help Center `_ +Short version +~~~~~~~~~~~~~ +#. Fork the repository; +#. Target ``master`` for bugfixes and doc changes; +#. Target ``features`` for new features or functionality changes. +#. Follow **PEP-8**. There's a ``tox`` command to help fixing it: ``tox -e fix-lint``. +#. Tests are run using ``tox``:: -.. note:: - What is a "pull request"? It informs project's core developers about the - changes you want to review and merge. Pull requests are stored on - `GitHub servers `_. - Once you send pull request, we can discuss it's potential modifications and - even add more commits to it later on. + tox -e linting,py27,py36 + + The test environments above are usually enough to cover most cases locally. + +#. Write a ``changelog`` entry: ``changelog/2574.bugfix``, use issue id number + and one of ``bugfix``, ``removal``, ``feature``, ``vendor``, ``doc`` or + ``trivial`` for the issue type. +#. Unless your change is a trivial or a documentation fix (e.g., a typo or reword of a small section) please + add yourself to the ``AUTHORS`` file, in alphabetical order; + + +Long version +~~~~~~~~~~~~ -There's an excellent tutorial on how Pull Requests work in the -`GitHub Help Center `_, -but here is a simple overview: +What is a "pull request"? It informs the project's core developers about the +changes you want to review and merge. Pull requests are stored on +`GitHub servers `_. +Once you send a pull request, we can discuss its potential modifications and +even add more commits to it later on. There's an excellent tutorial on how Pull +Requests work in the +`GitHub Help Center `_. + +Here is a simple overview, with pytest-specific bits: #. Fork the `pytest GitHub repository `__. It's @@ -208,38 +228,43 @@ but here is a simple overview: #. Run all the tests - You need to have Python 2.7 and 3.5 available in your system. Now + You need to have Python 2.7 and 3.6 available in your system. Now running tests is as simple as issuing this command:: - $ python runtox.py -e linting,py27,py35 + $ tox -e linting,py27,py36 - This command will run tests via the "tox" tool against Python 2.7 and 3.5 - and also perform "lint" coding-style checks. ``runtox.py`` is - a thin wrapper around ``tox`` which installs from a development package - index where newer (not yet released to pypi) versions of dependencies - (especially ``py``) might be present. + This command will run tests via the "tox" tool against Python 2.7 and 3.6 + and also perform "lint" coding-style checks. -#. You can now edit your local working copy. +#. You can now edit your local working copy. Please follow PEP-8. You can now make the changes you want and run the tests again as necessary. - To run tests on py27 and pass options to pytest (e.g. enter pdb on failure) - to pytest you can do:: + If you have too much linting errors, try running:: + + $ tox -e fix-lint + + To fix pep8 related errors. - $ python runtox.py -e py27 -- --pdb + You can pass different options to ``tox``. For example, to run tests on Python 2.7 and pass options to pytest + (e.g. enter pdb on failure) to pytest you can do:: - or to only run tests in a particular test module on py35:: + $ tox -e py27 -- --pdb - $ python runtox.py -e py35 -- testing/test_config.py + Or to only run tests in a particular test module on Python 3.6:: + + $ tox -e py36 -- testing/test_config.py #. Commit and push once your tests pass and you are happy with your change(s):: $ git commit -a -m "" $ git push -u - Make sure you add a CHANGELOG message, and add yourself to AUTHORS. If you - are unsure about either of these steps, submit your pull request and we'll - help you fix it up. +#. Create a new changelog entry in ``changelog``. The file should be named ``.``, + where *issueid* is the number of the issue related to the change and *type* is one of + ``bugfix``, ``removal``, ``feature``, ``vendor``, ``doc`` or ``trivial``. + +#. Add yourself to ``AUTHORS`` file if not there yet, in alphabetical order. #. Finally, submit a pull request through the GitHub website using this data:: @@ -248,6 +273,6 @@ but here is a simple overview: base-fork: pytest-dev/pytest base: master # if it's a bugfix - base: feature # if it's a feature + base: features # if it's a feature diff --git a/HOWTORELEASE.rst b/HOWTORELEASE.rst index 3ebfa28b1ed97dd..48a3461d4bc9206 100644 --- a/HOWTORELEASE.rst +++ b/HOWTORELEASE.rst @@ -1,92 +1,65 @@ -How to release pytest --------------------------------------------- +Release Procedure +----------------- -Note: this assumes you have already registered on pypi. +Our current policy for releasing is to aim for a bugfix every few weeks and a minor release every 2-3 months. The idea +is to get fixes and new features out instead of trying to cram a ton of features into a release and by consequence +taking a lot of time to make a new one. -0. create the branch release-VERSION - use features as base for minor/major releases - and master as base for bugfix releases +.. important:: -1. Bump version numbers in _pytest/__init__.py (setup.py reads it) + pytest releases must be prepared on **Linux** because the docs and examples expect + to be executed in that platform. -2. Check and finalize CHANGELOG +#. Install development dependencies in a virtual environment with:: -3. Write doc/en/announce/release-VERSION.txt and include - it in doc/en/announce/index.txt:: + pip3 install -r tasks/requirements.txt - git log 2.8.2..HEAD --format='%aN' | sort -u # lists the names of authors involved +#. Create a branch ``release-X.Y.Z`` with the version for the release. -4. Use devpi for uploading a release tarball to a staging area:: + * **patch releases**: from the latest ``master``; - devpi use https://devpi.net/USER/dev - devpi upload --formats sdist,bdist_wheel + * **minor releases**: from the latest ``features``; then merge with the latest ``master``; -5. Run from multiple machines:: + Ensure your are in a clean work tree. - devpi use https://devpi.net/USER/dev - devpi test pytest==VERSION +#. Generate docs, changelog, announcements and upload a package to + your ``devpi`` staging server:: -6. Check that tests pass for relevant combinations with:: + invoke generate.pre-release --password - devpi list pytest - - or look at failures with "devpi list -f pytest". - -7. Regenerate the docs examples using tox, and check for regressions:: - - tox -e regen - git diff - - -8. Build the docs, you need a virtualenv with py and sphinx - installed:: + If ``--password`` is not given, it is assumed the user is already logged in ``devpi``. + If you don't have an account, please ask for one. - cd doc/en - make html +#. Open a PR for this branch targeting ``master``. - Commit any changes before tagging the release. +#. Test the package -9. Tag the release:: + * **Manual method** - git tag VERSION - git push + Run from multiple machines:: -10. Upload the docs using doc/en/Makefile:: + devpi use https://devpi.net/USER/dev + devpi test pytest==VERSION - cd doc/en - make install # or "installall" if you have LaTeX installed for PDF + Check that tests pass for relevant combinations with:: - This requires ssh-login permission on pytest.org because it uses - rsync. - Note that the ``install`` target of ``doc/en/Makefile`` defines where the - rsync goes to, typically to the "latest" section of pytest.org. - - If you are making a minor release (e.g. 5.4), you also need to manually - create a symlink for "latest":: - - ssh pytest-dev@pytest.org - ln -s 5.4 latest - - Browse to pytest.org to verify. - -11. Publish to pypi:: - - devpi push pytest-VERSION pypi:NAME + devpi list pytest - where NAME is the name of pypi.python.org as configured in your ``~/.pypirc`` - file `for devpi `_. + * **CI servers** + Configure a repository as per-instructions on + devpi-cloud-test_ to test the package on Travis_ and AppVeyor_. + All test environments should pass. -12. Send release announcement to mailing lists: +#. Publish to PyPI:: - - pytest-dev - - testing-in-python - - python-announce-list@python.org + invoke generate.publish-release + where PYPI_NAME is the name of pypi.python.org as configured in your ``~/.pypirc`` + file `for devpi `_. -13. **after the release** Bump the version number in ``_pytest/__init__.py``, - to the next Minor release version (i.e. if you released ``pytest-2.8.0``, - set it to ``pytest-2.9.0.dev1``). +#. After a minor/major release, merge ``release-X.Y.Z`` into ``master`` and push (or open a PR). -14. merge the actual release into the master branch and do a pull request against it -15. merge from master to features +.. _devpi-cloud-test: https://github.com/obestwalter/devpi-cloud-test +.. _AppVeyor: https://www.appveyor.com/ +.. _Travis: https://travis-ci.org diff --git a/ISSUES.txt b/ISSUES.txt deleted file mode 100644 index 081d727e86a071a..000000000000000 --- a/ISSUES.txt +++ /dev/null @@ -1,365 +0,0 @@ - - -recorder = monkeypatch.function(".......") -------------------------------------------------------------- -tags: nice feature - -Like monkeypatch.replace but sets a mock-like call recorder: - - recorder = monkeypatch.function("os.path.abspath") - recorder.set_return("/hello") - os.path.abspath("hello") - call, = recorder.calls - assert call.args.path == "hello" - assert call.returned == "/hello" - ... - -Unlike mock, "args.path" acts on the parsed auto-spec'ed ``os.path.abspath`` -so it's independent from if the client side called "os.path.abspath(path=...)" -or "os.path.abspath('positional')". - - -refine parametrize API -------------------------------------------------------------- -tags: critical feature - -extend metafunc.parametrize to directly support indirection, example: - - def setupdb(request, config): - # setup "resource" based on test request and the values passed - # in to parametrize. setupfunc is called for each such value. - # you may use request.addfinalizer() or request.cached_setup ... - return dynamic_setup_database(val) - - @pytest.mark.parametrize("db", ["pg", "mysql"], setupfunc=setupdb) - def test_heavy_functional_test(db): - ... - -There would be no need to write or explain funcarg factories and -their special __ syntax. - -The examples and improvements should also show how to put the parametrize -decorator to a class, to a module or even to a directory. For the directory -part a conftest.py content like this:: - - pytestmark = [ - @pytest.mark.parametrize_setup("db", ...), - ] - -probably makes sense in order to keep the declarative nature. This mirrors -the marker-mechanism with respect to a test module but puts it to a directory -scale. - -When doing larger scoped parametrization it probably becomes necessary -to allow parametrization to be ignored if the according parameter is not -used (currently any parametrized argument that is not present in a function will cause a ValueError). Example: - - @pytest.mark.parametrize("db", ..., mustmatch=False) - -means to not raise an error but simply ignore the parametrization -if the signature of a decorated function does not match. XXX is it -not sufficient to always allow non-matches? - - -allow parametrized attributes on classes --------------------------------------------------- - -tags: wish 2.4 - -example: - - @pytest.mark.parametrize_attr("db", setupfunc, [1,2,3], scope="class") - @pytest.mark.parametrize_attr("tmp", setupfunc, scope="...") - class TestMe: - def test_hello(self): - access self.db ... - -this would run the test_hello() function three times with three -different values for self.db. This could also work with unittest/nose -style tests, i.e. it leverages existing test suites without needing -to rewrite them. Together with the previously mentioned setup_test() -maybe the setupfunc could be omitted? - -optimizations ---------------------------------------------------------------- -tags: 2.4 core - -- look at ihook optimization such that all lookups for - hooks relating to the same fspath are cached. - -fix start/finish partial finailization problem ---------------------------------------------------------------- -tags: bug core - -if a configure/runtest_setup/sessionstart/... hook invocation partially -fails the sessionfinishes is not called. Each hook implementation -should better be repsonsible for registering a cleanup/finalizer -appropriately to avoid this issue. Moreover/Alternatively, we could -record which implementations of a hook succeeded and only call their -teardown. - - -relax requirement to have tests/testing contain an __init__ ----------------------------------------------------------------- -tags: feature -bb: http://bitbucket.org/hpk42/py-trunk/issue/64 - -A local test run of a "tests" directory may work -but a remote one fail because the tests directory -does not contain an "__init__.py". Either give -an error or make it work without the __init__.py -i.e. port the nose-logic of unloading a test module. - -customize test function collection -------------------------------------------------------- -tags: feature - -- introduce pytest.mark.nocollect for not considering a function for - test collection at all. maybe also introduce a pytest.mark.test to - explicitly mark a function to become a tested one. Lookup JUnit ways - of tagging tests. - -introduce pytest.mark.importorskip -------------------------------------------------------- -tags: feature - -in addition to the imperative pytest.importorskip also introduce -a pytest.mark.importorskip so that the test count is more correct. - - -introduce pytest.mark.platform -------------------------------------------------------- -tags: feature - -Introduce nice-to-spell platform-skipping, examples: - - @pytest.mark.platform("python3") - @pytest.mark.platform("not python3") - @pytest.mark.platform("win32 and not python3") - @pytest.mark.platform("darwin") - @pytest.mark.platform("not (jython and win32)") - @pytest.mark.platform("not (jython and win32)", xfail=True) - -etc. Idea is to allow Python expressions which can operate -on common spellings for operating systems and python -interpreter versions. - -pytest.mark.xfail signature change -------------------------------------------------------- -tags: feature - -change to pytest.mark.xfail(reason, (optional)condition) -to better implement the word meaning. It also signals -better that we always have some kind of an implementation -reason that can be formualated. -Compatibility? how to introduce a new name/keep compat? - -allow to non-intrusively apply skipfs/xfail/marks ---------------------------------------------------- -tags: feature - -use case: mark a module or directory structures -to be skipped on certain platforms (i.e. no import -attempt will be made). - -consider introducing a hook/mechanism that allows to apply marks -from conftests or plugins. (See extended parametrization) - - -explicit referencing of conftest.py files ------------------------------------------ -tags: feature - -allow to name conftest.py files (in sub directories) that should -be imported early, as to include command line options. - -improve central pytest ini file -------------------------------- -tags: feature - -introduce more declarative configuration options: -- (to-be-collected test directories) -- required plugins -- test func/class/file matching patterns -- skip/xfail (non-intrusive) -- pytest.ini and tox.ini and setup.cfg configuration in the same file - -new documentation ----------------------------------- -tags: feature - -- logo pytest -- examples for unittest or functional testing -- resource management for functional testing -- patterns: page object - -have imported module mismatch honour relative paths --------------------------------------------------------- -tags: bug - -With 1.1.1 pytest fails at least on windows if an import -is relative and compared against an absolute conftest.py -path. Normalize. - -consider globals: pytest.ensuretemp and config --------------------------------------------------------------- -tags: experimental-wish - -consider deprecating pytest.ensuretemp and pytest.config -to further reduce pytest globality. Also consider -having pytest.config and ensuretemp coming from -a plugin rather than being there from the start. - - -consider pytest_addsyspath hook ------------------------------------------ -tags: wish - -pytest could call a new pytest_addsyspath() in order to systematically -allow manipulation of sys.path and to inhibit it via --no-addsyspath -in order to more easily run against installed packages. - -Alternatively it could also be done via the config object -and pytest_configure. - - - -deprecate global pytest.config usage ----------------------------------------------------------------- -tags: feature - -pytest.ensuretemp and pytest.config are probably the last -objects containing global state. Often using them is not -necessary. This is about trying to get rid of them, i.e. -deprecating them and checking with PyPy's usages as well -as others. - -remove deprecated bits in collect.py -------------------------------------------------------------------- -tags: feature - -In an effort to further simplify code, review and remove deprecated bits -in collect.py. Probably good: -- inline consider_file/dir methods, no need to have them - subclass-overridable because of hooks - -implement fslayout decorator ---------------------------------- -tags: feature - -Improve the way how tests can work with pre-made examples, -keeping the layout close to the test function: - -@pytest.mark.fslayout(""" - conftest.py: - # empty - tests/ - test_%(NAME)s: # becomes test_run1.py - def test_function(self): - pass -""") -def test_run(pytester, fslayout): - p = fslayout.findone("test_*.py") - result = pytester.runpytest(p) - assert result.ret == 0 - assert result.passed == 1 - -Another idea is to allow to define a full scenario including the run -in one content string:: - - runscenario(""" - test_{TESTNAME}.py: - import pytest - @pytest.mark.xfail - def test_that_fails(): - assert 0 - - @pytest.mark.skipif("True") - def test_hello(): - pass - - conftest.py: - import pytest - def pytest_runsetup_setup(item): - pytest.skip("abc") - - runpytest -rsxX - *SKIP*{TESTNAME}* - *1 skipped* - """) - -This could be run with at least three different ways to invoke pytest: -through the shell, through "python -m pytest" and inlined. As inlined -would be the fastest it could be run first (or "--fast" mode). - - -Create isolate plugin ---------------------- -tags: feature - -The idea is that you can e.g. import modules in a test and afterwards -sys.modules, sys.meta_path etc would be reverted. It can go further -then just importing however, e.g. current working directory, file -descriptors, ... - -This would probably be done by marking:: - - @pytest.mark.isolate(importing=True, cwd=True, fds=False) - def test_foo(): - ... - -With the possibility of doing this globally in an ini-file. - - -fnmatch for test names ----------------------- -tags: feature-wish - -various testsuites use suffixes instead of prefixes for test classes -also it lends itself to bdd style test names:: - - class UserBehaviour: - def anonymous_should_not_have_inbox(user): - ... - def registred_should_have_inbox(user): - .. - -using the following in pytest.ini:: - - [pytest] - python_classes = Test *Behaviour *Test - python_functions = test *_should_* - - -mechanism for running named parts of tests with different reporting behaviour ------------------------------------------------------------------------------- -tags: feature-wish-incomplete - -a few use-cases come to mind: - -* fail assertions and record that without stopping a complete test - - * this is in particular hepfull if a small bit of a test is known to fail/xfail:: - - def test_fun(): - with pytest.section('fdcheck, marks=pytest.mark.xfail_if(...)): - breaks_on_windows() - -* divide functional/acceptance tests into sections -* provide a different mechanism for generators, maybe something like:: - - def pytest_runtest_call(item) - if not generator: - ... - prepare_check = GeneratorCheckprepare() - - gen = item.obj(**fixtures) - for check in gen - id, call = prepare_check(check) - # bubble should only prevent exception propagation after a failure - # the whole test should still fail - # there might be need for a lower level api and taking custom markers into account - with pytest.section(id, bubble=False): - call() - - diff --git a/LICENSE b/LICENSE index 9e27bd7841951b8..629df45ac405532 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ The MIT License (MIT) -Copyright (c) 2004-2016 Holger Krekel and others +Copyright (c) 2004-2017 Holger Krekel and others Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in diff --git a/MANIFEST.in b/MANIFEST.in deleted file mode 100644 index 266a9184dc369b4..000000000000000 --- a/MANIFEST.in +++ /dev/null @@ -1,34 +0,0 @@ -include CHANGELOG.rst -include LICENSE -include AUTHORS - -include README.rst -include CONTRIBUTING.rst - -include tox.ini -include setup.py - -include .coveragerc - -include plugin-test.sh -include requirements-docs.txt -include runtox.py - -recursive-include bench *.py -recursive-include extra *.py - -graft testing -graft doc - -exclude _pytest/impl - -graft _pytest/vendored_packages - -recursive-exclude * *.pyc *.pyo - -exclude appveyor/install.ps1 -exclude appveyor.yml -exclude appveyor - -exclude ISSUES.txt -exclude HOWTORELEASE.rst diff --git a/README.rst b/README.rst index 68fc92211d8be80..3630dd4c62aa228 100644 --- a/README.rst +++ b/README.rst @@ -1,83 +1,90 @@ -.. image:: http://pytest.org/latest/_static/pytest1.png - :target: http://pytest.org +.. image:: http://docs.pytest.org/en/latest/_static/pytest1.png + :target: http://docs.pytest.org :align: center :alt: pytest ------ .. image:: https://img.shields.io/pypi/v/pytest.svg - :target: https://pypi.python.org/pypi/pytest + :target: https://pypi.python.org/pypi/pytest + +.. image:: https://anaconda.org/conda-forge/pytest/badges/version.svg + :target: https://anaconda.org/conda-forge/pytest + .. image:: https://img.shields.io/pypi/pyversions/pytest.svg - :target: https://pypi.python.org/pypi/pytest + :target: https://pypi.python.org/pypi/pytest + .. image:: https://img.shields.io/coveralls/pytest-dev/pytest/master.svg - :target: https://coveralls.io/r/pytest-dev/pytest + :target: https://coveralls.io/r/pytest-dev/pytest + .. image:: https://travis-ci.org/pytest-dev/pytest.svg?branch=master :target: https://travis-ci.org/pytest-dev/pytest + .. image:: https://ci.appveyor.com/api/projects/status/mrgbjaua7t33pg6b?svg=true :target: https://ci.appveyor.com/project/pytestbot/pytest The ``pytest`` framework makes it easy to write small tests, yet -scales to support complex functional testing for applications and libraries. +scales to support complex functional testing for applications and libraries. An example of a simple test: .. code-block:: python # content of test_sample.py - def func(x): + def inc(x): return x + 1 def test_answer(): - assert func(3) == 5 + assert inc(3) == 5 To execute it:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.3, pytest-2.8.5, py-1.4.31, pluggy-0.3.1 + $ pytest + ============================= test session starts ============================= collected 1 items test_sample.py F - ======= FAILURES ======== - _______ test_answer ________ + ================================== FAILURES =================================== + _________________________________ test_answer _________________________________ def test_answer(): - > assert func(3) == 5 + > assert inc(3) == 5 E assert 4 == 5 - E + where 4 = func(3) + E + where 4 = inc(3) test_sample.py:5: AssertionError - ======= 1 failed in 0.12 seconds ======== + ========================== 1 failed in 0.04 seconds =========================== + + +Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. See `getting-started `_ for more examples. -Due to ``py.test``'s detailed assertion introspection, only plain ``assert`` statements are used. See `getting-started `_ for more examples. - Features -------- -- Detailed info on failing `assert statements `_ (no need to remember ``self.assert*`` names); +- Detailed info on failing `assert statements `_ (no need to remember ``self.assert*`` names); - `Auto-discovery - `_ + `_ of test modules and functions; -- `Modular fixtures `_ for +- `Modular fixtures `_ for managing small or parametrized long-lived test resources; -- Can run `unittest `_ (or trial), - `nose `_ test suites out of the box; +- Can run `unittest `_ (or trial), + `nose `_ test suites out of the box; -- Python2.6+, Python3.2+, PyPy-2.3, Jython-2.5 (untested); +- Python 2.7, Python 3.4+, PyPy 2.3, Jython 2.5 (untested); -- Rich plugin architecture, with over 150+ `external plugins `_ and thriving community; +- Rich plugin architecture, with over 315+ `external plugins `_ and thriving community; Documentation ------------- -For full documentation, including installation, tutorials and PDF documents, please see http://pytest.org. +For full documentation, including installation, tutorials and PDF documents, please see http://docs.pytest.org. Bugs/Requests @@ -89,13 +96,13 @@ Please use the `GitHub issue tracker `_ page for fixes and enhancements of each version. +Consult the `Changelog `__ page for fixes and enhancements of each version. License ------- -Copyright Holger Krekel and others, 2004-2016. +Copyright Holger Krekel and others, 2004-2017. Distributed under the terms of the `MIT`_ license, pytest is free and open source software. diff --git a/_pytest/__init__.py b/_pytest/__init__.py index 723fb9e855a1a86..6e41f0504e47064 100644 --- a/_pytest/__init__.py +++ b/_pytest/__init__.py @@ -1,2 +1,8 @@ -# -__version__ = '2.9.1' +__all__ = ['__version__'] + +try: + from ._version import version as __version__ +except ImportError: + # broken installation, we don't even try + # unknown only works because we do poor mans version compare + __version__ = 'unknown' diff --git a/_pytest/_argcomplete.py b/_pytest/_argcomplete.py index 955855a964819c3..0625a75f9f1b705 100644 --- a/_pytest/_argcomplete.py +++ b/_pytest/_argcomplete.py @@ -4,9 +4,6 @@ to find the magic string, so _ARGCOMPLETE env. var is never set, and this does not need special code. -argcomplete does not support python 2.5 (although the changes for that -are minor). - Function try_argcomplete(parser) should be called directly before the call to ArgumentParser.parse_args(). @@ -57,26 +54,29 @@ which should throw a KeyError: 'COMPLINE' (which is properly set by the global argcomplete script). """ - +from __future__ import absolute_import, division, print_function import sys import os from glob import glob + class FastFilesCompleter: 'Fast file completer class' + def __init__(self, directories=True): self.directories = directories def __call__(self, prefix, **kwargs): """only called on non option completions""" - if os.path.sep in prefix[1:]: # + if os.path.sep in prefix[1:]: prefix_dir = len(os.path.dirname(prefix) + os.path.sep) else: prefix_dir = 0 completion = [] globbed = [] if '*' not in prefix and '?' not in prefix: - if prefix[-1] == os.path.sep: # we are on unix, otherwise no bash + # we are on unix, otherwise no bash + if not prefix or prefix[-1] == os.path.sep: globbed.extend(glob(prefix + '.*')) prefix += '*' globbed.extend(glob(prefix)) @@ -87,6 +87,7 @@ def __call__(self, prefix, **kwargs): completion.append(x[prefix_dir:]) return completion + if os.environ.get('_ARGCOMPLETE'): try: import argcomplete.completers @@ -95,7 +96,8 @@ def __call__(self, prefix, **kwargs): filescompleter = FastFilesCompleter() def try_argcomplete(parser): - argcomplete.autocomplete(parser) + argcomplete.autocomplete(parser, always_complete_options=False) else: - def try_argcomplete(parser): pass + def try_argcomplete(parser): + pass filescompleter = None diff --git a/_pytest/_code/__init__.py b/_pytest/_code/__init__.py index c046b9716caed43..815c13b42c25bd3 100644 --- a/_pytest/_code/__init__.py +++ b/_pytest/_code/__init__.py @@ -1,12 +1,10 @@ """ python inspection/code generation API """ +from __future__ import absolute_import, division, print_function from .code import Code # noqa from .code import ExceptionInfo # noqa from .code import Frame # noqa from .code import Traceback # noqa from .code import getrawcode # noqa -from .code import patch_builtins # noqa -from .code import unpatch_builtins # noqa from .source import Source # noqa from .source import compile_ as compile # noqa from .source import getfslineno # noqa - diff --git a/_pytest/_code/_py2traceback.py b/_pytest/_code/_py2traceback.py index a830d9899ae0cd0..5aacf0a428d8806 100644 --- a/_pytest/_code/_py2traceback.py +++ b/_pytest/_code/_py2traceback.py @@ -2,8 +2,10 @@ # CHANGES: # - some_str is replaced, trying to create unicode strings # +from __future__ import absolute_import, division, print_function import types + def format_exception_only(etype, value): """Format the exception part of a traceback. @@ -29,7 +31,7 @@ def format_exception_only(etype, value): # would throw another exception and mask the original problem. if (isinstance(etype, BaseException) or isinstance(etype, types.InstanceType) or - etype is None or type(etype) is str): + etype is None or type(etype) is str): return [_format_final_exc_line(etype, value)] stype = etype.__name__ @@ -61,6 +63,7 @@ def format_exception_only(etype, value): lines.append(_format_final_exc_line(stype, value)) return lines + def _format_final_exc_line(etype, value): """Return a list of a single line -- normal case for format_exception_only""" valuestr = _some_str(value) @@ -70,6 +73,7 @@ def _format_final_exc_line(etype, value): line = "%s: %s\n" % (etype, valuestr) return line + def _some_str(value): try: return unicode(value) diff --git a/_pytest/_code/code.py b/_pytest/_code/code.py index bc68aac55484586..3fb232bd430e9b4 100644 --- a/_pytest/_code/code.py +++ b/_pytest/_code/code.py @@ -1,19 +1,22 @@ +from __future__ import absolute_import, division, print_function import sys from inspect import CO_VARARGS, CO_VARKEYWORDS +import re +from weakref import ref +from _pytest.compat import _PY2, _PY3, PY35, safe_str import py - builtin_repr = repr -reprlib = py.builtin._tryimport('repr', 'reprlib') - -if sys.version_info[0] >= 3: +if _PY3: from traceback import format_exception_only else: from ._py2traceback import format_exception_only + class Code(object): """ wrapper around Python code objects """ + def __init__(self, rawcode): if not hasattr(rawcode, "co_filename"): rawcode = getrawcode(rawcode) @@ -22,12 +25,14 @@ def __init__(self, rawcode): self.firstlineno = rawcode.co_firstlineno - 1 self.name = rawcode.co_name except AttributeError: - raise TypeError("not a code object: %r" %(rawcode,)) + raise TypeError("not a code object: %r" % (rawcode,)) self.raw = rawcode def __eq__(self, other): return self.raw == other.raw + __hash__ = None + def __ne__(self, other): return not self == other @@ -35,12 +40,16 @@ def __ne__(self, other): def path(self): """ return a path object pointing to source code (note that it might not point to an actually existing file). """ - p = py.path.local(self.raw.co_filename) - # maybe don't try this checking - if not p.check(): + try: + p = py.path.local(self.raw.co_filename) + # maybe don't try this checking + if not p.check(): + raise OSError("py.path check failed.") + except OSError: # XXX maybe try harder like the weird logic # in the standard lib [linecache.updatecache] does? p = self.raw.co_filename + return p @property @@ -72,6 +81,7 @@ def getargs(self, var=False): argcount += raw.co_flags & CO_VARKEYWORDS return raw.co_varnames[:argcount] + class Frame(object): """Wrapper around a Python frame holding f_locals and f_globals in which expressions can be evaluated.""" @@ -109,7 +119,7 @@ def exec_(self, code, **vars): """ f_locals = self.f_locals.copy() f_locals.update(vars) - py.builtin.exec_(code, self.f_globals, f_locals ) + py.builtin.exec_(code, self.f_globals, f_locals) def repr(self, object): """ return a 'safe' (non-recursive, one-line) string repr for 'object' @@ -133,13 +143,15 @@ def getargs(self, var=False): pass # this can occur when using Psyco return retval + class TracebackEntry(object): """ a single entry in a traceback """ _repr_style = None exprinfo = None - def __init__(self, rawentry): + def __init__(self, rawentry, excinfo=None): + self._excinfo = excinfo self._rawentry = rawentry self.lineno = rawentry.tb_lineno - 1 @@ -157,7 +169,7 @@ def relline(self): return self.lineno - self.frame.code.firstlineno def __repr__(self): - return "" %(self.frame.code.path, self.lineno+1) + return "" % (self.frame.code.path, self.lineno + 1) @property def statement(self): @@ -174,18 +186,6 @@ def getlocals(self): return self.frame.f_locals locals = property(getlocals, None, None, "locals of underlaying frame") - def reinterpret(self): - """Reinterpret the failing statement and returns a detailed information - about what operations are performed.""" - from _pytest.assertion.reinterpret import reinterpret - if self.exprinfo is None: - source = py.builtin._totext(self.statement).strip() - x = reinterpret(source, self.frame, should_fail=True) - if not py.builtin._istext(x): - raise TypeError("interpret returned non-string %r" % (x,)) - self.exprinfo = x - return self.exprinfo - def getfirstlinesource(self): # on Jython this firstlineno can be -1 apparently return max(self.frame.code.firstlineno, 0) @@ -220,16 +220,24 @@ def ishidden(self): """ return True if the current frame has a var __tracebackhide__ resolving to True + If __tracebackhide__ is a callable, it gets called with the + ExceptionInfo instance and can decide whether to hide the traceback. + mostly for internal use """ try: - return self.frame.f_locals['__tracebackhide__'] + tbh = self.frame.f_locals['__tracebackhide__'] except KeyError: try: - return self.frame.f_globals['__tracebackhide__'] + tbh = self.frame.f_globals['__tracebackhide__'] except KeyError: return False + if callable(tbh): + return tbh(None if self._excinfo is None else self._excinfo()) + else: + return tbh + def __str__(self): try: fn = str(self.path) @@ -240,25 +248,28 @@ def __str__(self): line = str(self.statement).lstrip() except KeyboardInterrupt: raise - except: + except: # noqa line = "???" - return " File %r:%d in %s\n %s\n" %(fn, self.lineno+1, name, line) + return " File %r:%d in %s\n %s\n" % (fn, self.lineno + 1, name, line) def name(self): return self.frame.code.raw.co_name name = property(name, None, None, "co_name of underlaying code") + class Traceback(list): """ Traceback objects encapsulate and offer higher level access to Traceback entries. """ Entry = TracebackEntry - def __init__(self, tb): - """ initialize from given python traceback object. """ + + def __init__(self, tb, excinfo=None): + """ initialize from given python traceback object and ExceptionInfo """ + self._excinfo = excinfo if hasattr(tb, 'tb_next'): def f(cur): while cur is not None: - yield self.Entry(cur) + yield self.Entry(cur, excinfo=excinfo) cur = cur.tb_next list.__init__(self, f(tb)) else: @@ -281,8 +292,8 @@ def cut(self, path=None, lineno=None, firstlineno=None, excludepath=None): (excludepath is None or not hasattr(codepath, 'relto') or not codepath.relto(excludepath)) and (lineno is None or x.lineno == lineno) and - (firstlineno is None or x.frame.code.firstlineno == firstlineno)): - return Traceback(x._rawentry) + (firstlineno is None or x.frame.code.firstlineno == firstlineno)): + return Traceback(x._rawentry, self._excinfo) return self def __getitem__(self, key): @@ -294,27 +305,27 @@ def __getitem__(self, key): def filter(self, fn=lambda x: not x.ishidden()): """ return a Traceback instance with certain items removed - fn is a function that gets a single argument, a TracebackItem + fn is a function that gets a single argument, a TracebackEntry instance, and should return True when the item should be added to the Traceback, False when not - by default this removes all the TracebackItems which are hidden + by default this removes all the TracebackEntries which are hidden (see ishidden() above) """ - return Traceback(filter(fn, self)) + return Traceback(filter(fn, self), self._excinfo) def getcrashentry(self): """ return last non-hidden traceback entry that lead to the exception of a traceback. """ - for i in range(-1, -len(self)-1, -1): + for i in range(-1, -len(self) - 1, -1): entry = self[i] if not entry.ishidden(): return entry return self[-1] def recursionindex(self): - """ return the index of the frame/TracebackItem where recursion + """ return the index of the frame/TracebackEntry where recursion originates if appropriate, None if no recursion occurred """ cache = {} @@ -322,29 +333,33 @@ def recursionindex(self): # id for the code.raw is needed to work around # the strange metaprogramming in the decorator lib from pypi # which generates code objects that have hash/value equality - #XXX needs a test + # XXX needs a test key = entry.frame.code.path, id(entry.frame.code.raw), entry.lineno - #print "checking for recursion at", key - l = cache.setdefault(key, []) - if l: + # print "checking for recursion at", key + values = cache.setdefault(key, []) + if values: f = entry.frame loc = f.f_locals - for otherloc in l: + for otherloc in values: if f.is_true(f.eval(co_equal, - __recursioncache_locals_1=loc, - __recursioncache_locals_2=otherloc)): + __recursioncache_locals_1=loc, + __recursioncache_locals_2=otherloc)): return i - l.append(entry.frame.f_locals) + values.append(entry.frame.f_locals) return None + co_equal = compile('__recursioncache_locals_1 == __recursioncache_locals_2', '?', 'eval') + class ExceptionInfo(object): """ wraps sys.exc_info() objects and offers help for navigating the traceback. """ _striptext = '' + _assert_start_repr = "AssertionError(u\'assert " if _PY2 else "AssertionError(\'assert " + def __init__(self, tup=None, exprinfo=None): import _pytest._code if tup is None: @@ -352,8 +367,8 @@ def __init__(self, tup=None, exprinfo=None): if exprinfo is None and isinstance(tup[1], AssertionError): exprinfo = getattr(tup[1], 'msg', None) if exprinfo is None: - exprinfo = str(tup[1]) - if exprinfo and exprinfo.startswith('assert '): + exprinfo = py.io.saferepr(tup[1]) + if exprinfo and exprinfo.startswith(self._assert_start_repr): self._striptext = 'AssertionError: ' self._excinfo = tup #: the exception class @@ -365,7 +380,7 @@ def __init__(self, tup=None, exprinfo=None): #: the exception type name self.typename = self.type.__name__ #: the exception traceback (_pytest._code.Traceback instance) - self.traceback = _pytest._code.Traceback(self.tb) + self.traceback = _pytest._code.Traceback(self.tb, excinfo=ref(self)) def __repr__(self): return "" % (self.typename, len(self.traceback)) @@ -394,10 +409,10 @@ def _getreprcrash(self): exconly = self.exconly(tryshort=True) entry = self.traceback.getcrashentry() path, lineno = entry.frame.code.raw.co_filename, entry.lineno - return ReprFileLocation(path, lineno+1, exconly) + return ReprFileLocation(path, lineno + 1, exconly) def getrepr(self, showlocals=False, style="long", - abspath=False, tbfilter=True, funcargs=False): + abspath=False, tbfilter=True, funcargs=False): """ return str()able representation of this exception info. showlocals: show locals per traceback entry style: long|short|no|native traceback style @@ -414,7 +429,7 @@ def getrepr(self, showlocals=False, style="long", )), self._getreprcrash()) fmt = FormattedExcinfo(showlocals=showlocals, style=style, - abspath=abspath, tbfilter=tbfilter, funcargs=funcargs) + abspath=abspath, tbfilter=tbfilter, funcargs=funcargs) return fmt.repr_excinfo(self) def __str__(self): @@ -427,6 +442,19 @@ def __unicode__(self): loc = ReprFileLocation(entry.path, entry.lineno + 1, self.exconly()) return unicode(loc) + def match(self, regexp): + """ + Match the regular expression 'regexp' on the string representation of + the exception. If it matches then True is returned (so that it is + possible to write 'assert excinfo.match()'). If it doesn't match an + AssertionError is raised. + """ + __tracebackhide__ = True + if not re.search(regexp, str(self.value)): + assert 0, "Pattern '{0!s}' not found in '{1!s}'".format( + regexp, self.value) + return True + class FormattedExcinfo(object): """ presenting information about failing Functions and Generators. """ @@ -445,15 +473,15 @@ def __init__(self, showlocals=False, style="long", abspath=True, tbfilter=True, def _getindent(self, source): # figure out indent for given source try: - s = str(source.getstatement(len(source)-1)) + s = str(source.getstatement(len(source) - 1)) except KeyboardInterrupt: raise - except: + except: # noqa try: s = str(source[-1]) except KeyboardInterrupt: raise - except: + except: # noqa return 0 return 4 + (len(s) - len(s.lstrip())) @@ -489,7 +517,7 @@ def get_source(self, source, line_index=-1, excinfo=None, short=False): for line in source.lines[:line_index]: lines.append(space_prefix + line) lines.append(self.flow_marker + " " + source.lines[line_index]) - for line in source.lines[line_index+1:]: + for line in source.lines[line_index + 1:]: lines.append(space_prefix + line) if excinfo is not None: indent = 4 if short else self._getindent(source) @@ -522,10 +550,10 @@ def repr_locals(self, locals): # _repr() function, which is only reprlib.Repr in # disguise, so is very configurable. str_repr = self._saferepr(value) - #if len(str_repr) < 70 or not isinstance(value, + # if len(str_repr) < 70 or not isinstance(value, # (list, tuple, dict)): - lines.append("%-10s = %s" %(name, str_repr)) - #else: + lines.append("%-10s = %s" % (name, str_repr)) + # else: # self._line("%-10s =\\" % (name,)) # # XXX # py.std.pprint.pprint(value, stream=self.excinfowriter) @@ -551,14 +579,14 @@ def repr_traceback_entry(self, entry, excinfo=None): s = self.get_source(source, line_index, excinfo, short=short) lines.extend(s) if short: - message = "in %s" %(entry.name) + message = "in %s" % (entry.name) else: message = excinfo and excinfo.typename or "" path = self._makepath(entry.path) - filelocrepr = ReprFileLocation(path, entry.lineno+1, message) + filelocrepr = ReprFileLocation(path, entry.lineno + 1, message) localsrepr = None if not short: - localsrepr = self.repr_locals(entry.locals) + localsrepr = self.repr_locals(entry.locals) return ReprEntry(lines, reprargs, localsrepr, filelocrepr, style) if excinfo: lines.extend(self.get_exconly(excinfo, indent=4)) @@ -578,31 +606,91 @@ def repr_traceback(self, excinfo): traceback = excinfo.traceback if self.tbfilter: traceback = traceback.filter() - recursionindex = None - if excinfo.errisinstance(RuntimeError): - if "maximum recursion depth exceeded" in str(excinfo.value): - recursionindex = traceback.recursionindex() + + if is_recursion_error(excinfo): + traceback, extraline = self._truncate_recursive_traceback(traceback) + else: + extraline = None + last = traceback[-1] entries = [] - extraline = None for index, entry in enumerate(traceback): einfo = (last == entry) and excinfo or None reprentry = self.repr_traceback_entry(entry, einfo) entries.append(reprentry) - if index == recursionindex: - extraline = "!!! Recursion detected (same locals & position)" - break return ReprTraceback(entries, extraline, style=self.style) + def _truncate_recursive_traceback(self, traceback): + """ + Truncate the given recursive traceback trying to find the starting point + of the recursion. + + The detection is done by going through each traceback entry and finding the + point in which the locals of the frame are equal to the locals of a previous frame (see ``recursionindex()``. + + Handle the situation where the recursion process might raise an exception (for example + comparing numpy arrays using equality raises a TypeError), in which case we do our best to + warn the user of the error and show a limited traceback. + """ + try: + recursionindex = traceback.recursionindex() + except Exception as e: + max_frames = 10 + extraline = ( + '!!! Recursion error detected, but an error occurred locating the origin of recursion.\n' + ' The following exception happened when comparing locals in the stack frame:\n' + ' {exc_type}: {exc_msg}\n' + ' Displaying first and last {max_frames} stack frames out of {total}.' + ).format(exc_type=type(e).__name__, exc_msg=safe_str(e), max_frames=max_frames, total=len(traceback)) + traceback = traceback[:max_frames] + traceback[-max_frames:] + else: + if recursionindex is not None: + extraline = "!!! Recursion detected (same locals & position)" + traceback = traceback[:recursionindex + 1] + else: + extraline = None + + return traceback, extraline + def repr_excinfo(self, excinfo): - reprtraceback = self.repr_traceback(excinfo) - reprcrash = excinfo._getreprcrash() - return ReprExceptionInfo(reprtraceback, reprcrash) + if _PY2: + reprtraceback = self.repr_traceback(excinfo) + reprcrash = excinfo._getreprcrash() + + return ReprExceptionInfo(reprtraceback, reprcrash) + else: + repr_chain = [] + e = excinfo.value + descr = None + while e is not None: + if excinfo: + reprtraceback = self.repr_traceback(excinfo) + reprcrash = excinfo._getreprcrash() + else: + # fallback to native repr if the exception doesn't have a traceback: + # ExceptionInfo objects require a full traceback to work + reprtraceback = ReprTracebackNative(py.std.traceback.format_exception(type(e), e, None)) + reprcrash = None + + repr_chain += [(reprtraceback, reprcrash, descr)] + if e.__cause__ is not None: + e = e.__cause__ + excinfo = ExceptionInfo((type(e), e, e.__traceback__)) if e.__traceback__ else None + descr = 'The above exception was the direct cause of the following exception:' + elif (e.__context__ is not None and not e.__suppress_context__): + e = e.__context__ + excinfo = ExceptionInfo((type(e), e, e.__traceback__)) if e.__traceback__ else None + descr = 'During handling of the above exception, another exception occurred:' + else: + e = None + repr_chain.reverse() + return ExceptionChainRepr(repr_chain) -class TerminalRepr: + +class TerminalRepr(object): def __str__(self): s = self.__unicode__() - if sys.version_info[0] < 3: + if _PY2: s = s.encode('utf-8') return s @@ -615,24 +703,51 @@ def __unicode__(self): return io.getvalue().strip() def __repr__(self): - return "<%s instance at %0x>" %(self.__class__, id(self)) + return "<%s instance at %0x>" % (self.__class__, id(self)) -class ReprExceptionInfo(TerminalRepr): - def __init__(self, reprtraceback, reprcrash): - self.reprtraceback = reprtraceback - self.reprcrash = reprcrash +class ExceptionRepr(TerminalRepr): + def __init__(self): self.sections = [] def addsection(self, name, content, sep="-"): self.sections.append((name, content, sep)) def toterminal(self, tw): - self.reprtraceback.toterminal(tw) for name, content, sep in self.sections: tw.sep(sep, name) tw.line(content) + +class ExceptionChainRepr(ExceptionRepr): + def __init__(self, chain): + super(ExceptionChainRepr, self).__init__() + self.chain = chain + # reprcrash and reprtraceback of the outermost (the newest) exception + # in the chain + self.reprtraceback = chain[-1][0] + self.reprcrash = chain[-1][1] + + def toterminal(self, tw): + for element in self.chain: + element[0].toterminal(tw) + if element[2] is not None: + tw.line("") + tw.line(element[2], yellow=True) + super(ExceptionChainRepr, self).toterminal(tw) + + +class ReprExceptionInfo(ExceptionRepr): + def __init__(self, reprtraceback, reprcrash): + super(ReprExceptionInfo, self).__init__() + self.reprtraceback = reprtraceback + self.reprcrash = reprcrash + + def toterminal(self, tw): + self.reprtraceback.toterminal(tw) + super(ReprExceptionInfo, self).toterminal(tw) + + class ReprTraceback(TerminalRepr): entrysep = "_ " @@ -648,7 +763,7 @@ def toterminal(self, tw): tw.line("") entry.toterminal(tw) if i < len(self.reprentries) - 1: - next_entry = self.reprentries[i+1] + next_entry = self.reprentries[i + 1] if entry.style == "long" or \ entry.style == "short" and next_entry.style == "long": tw.sep(self.entrysep) @@ -656,12 +771,14 @@ def toterminal(self, tw): if self.extraline: tw.line(self.extraline) + class ReprTracebackNative(ReprTraceback): def __init__(self, tblines): self.style = "native" self.reprentries = [ReprEntryNative(tblines)] self.extraline = None + class ReprEntryNative(TerminalRepr): style = "native" @@ -671,6 +788,7 @@ def __init__(self, tblines): def toterminal(self, tw): tw.write("".join(self.lines)) + class ReprEntry(TerminalRepr): localssep = "_ " @@ -687,7 +805,7 @@ def toterminal(self, tw): for line in self.lines: red = line.startswith("E ") tw.line(line, bold=True, red=red) - #tw.line("") + # tw.line("") return if self.reprfuncargs: self.reprfuncargs.toterminal(tw) @@ -695,7 +813,7 @@ def toterminal(self, tw): red = line.startswith("E ") tw.line(line, bold=True, red=red) if self.reprlocals: - #tw.sep(self.localssep, "Locals") + # tw.sep(self.localssep, "Locals") tw.line("") self.reprlocals.toterminal(tw) if self.reprfileloc: @@ -708,6 +826,7 @@ def __str__(self): self.reprlocals, self.reprfileloc) + class ReprFileLocation(TerminalRepr): def __init__(self, path, lineno, message): self.path = str(path) @@ -721,7 +840,9 @@ def toterminal(self, tw): i = msg.find("\n") if i != -1: msg = msg[:i] - tw.line("%s:%s: %s" %(self.path, self.lineno, msg)) + tw.write(self.path, bold=True, red=True) + tw.line(":%s: %s" % (self.lineno, msg)) + class ReprLocals(TerminalRepr): def __init__(self, lines): @@ -731,6 +852,7 @@ def toterminal(self, tw): for line in self.lines: tw.line(line) + class ReprFuncArgs(TerminalRepr): def __init__(self, args): self.args = args @@ -739,11 +861,11 @@ def toterminal(self, tw): if self.args: linesofar = "" for name, value in self.args: - ns = "%s = %s" %(name, value) + ns = "%s = %s" % (safe_str(name), safe_str(value)) if len(ns) + len(linesofar) + 2 > tw.fullwidth: if linesofar: tw.line(linesofar) - linesofar = ns + linesofar = ns else: if linesofar: linesofar += ", " + ns @@ -754,29 +876,6 @@ def toterminal(self, tw): tw.line("") - -oldbuiltins = {} - -def patch_builtins(assertion=True, compile=True): - """ put compile and AssertionError builtins to Python's builtins. """ - if assertion: - from _pytest.assertion import reinterpret - l = oldbuiltins.setdefault('AssertionError', []) - l.append(py.builtin.builtins.AssertionError) - py.builtin.builtins.AssertionError = reinterpret.AssertionError - if compile: - import _pytest._code - l = oldbuiltins.setdefault('compile', []) - l.append(py.builtin.builtins.compile) - py.builtin.builtins.compile = _pytest._code.compile - -def unpatch_builtins(assertion=True, compile=True): - """ remove compile and AssertionError builtins from Python builtins. """ - if assertion: - py.builtin.builtins.AssertionError = oldbuiltins['AssertionError'].pop() - if compile: - py.builtin.builtins.compile = oldbuiltins['compile'].pop() - def getrawcode(obj, trycall=True): """ return code object for given function. """ try: @@ -793,3 +892,15 @@ def getrawcode(obj, trycall=True): return x return obj + +if PY35: # RecursionError introduced in 3.5 + def is_recursion_error(excinfo): + return excinfo.errisinstance(RecursionError) # noqa +else: + def is_recursion_error(excinfo): + if not excinfo.errisinstance(RuntimeError): + return False + try: + return "maximum recursion depth exceeded" in str(excinfo.value) + except UnicodeError: + return False diff --git a/_pytest/_code/source.py b/_pytest/_code/source.py index a1521f8a212a6b6..2638c598b749db2 100644 --- a/_pytest/_code/source.py +++ b/_pytest/_code/source.py @@ -1,10 +1,11 @@ -from __future__ import generators +from __future__ import absolute_import, division, generators, print_function from bisect import bisect_right import sys -import inspect, tokenize +import six +import inspect +import tokenize import py -from types import ModuleType cpy_compile = compile try: @@ -20,6 +21,7 @@ class Source(object): possibly deindenting it. """ _compilecounter = 0 + def __init__(self, *parts, **kwargs): self.lines = lines = [] de = kwargs.get('deindent', True) @@ -31,7 +33,7 @@ def __init__(self, *parts, **kwargs): partlines = part.lines elif isinstance(part, (tuple, list)): partlines = [x.rstrip("\n") for x in part] - elif isinstance(part, py.builtin._basestring): + elif isinstance(part, six.string_types): partlines = part.split('\n') if rstrip: while partlines: @@ -52,22 +54,21 @@ def __eq__(self, other): return str(self) == other return False + __hash__ = None + def __getitem__(self, key): if isinstance(key, int): return self.lines[key] else: if key.step not in (None, 1): raise IndexError("cannot slice a Source with a step") - return self.__getslice__(key.start, key.stop) + newsource = Source() + newsource.lines = self.lines[key.start:key.stop] + return newsource def __len__(self): return len(self.lines) - def __getslice__(self, start, end): - newsource = Source() - newsource.lines = self.lines[start:end] - return newsource - def strip(self): """ return new source object with trailing and leading blank lines removed. @@ -75,7 +76,7 @@ def strip(self): start, end = 0, len(self) while start < end and not self.lines[start].strip(): start += 1 - while end > start and not self.lines[end-1].strip(): + while end > start and not self.lines[end - 1].strip(): end -= 1 source = Source() source.lines[:] = self.lines[start:end] @@ -88,8 +89,8 @@ def putaround(self, before='', after='', indent=' ' * 4): before = Source(before) after = Source(after) newsource = Source() - lines = [ (indent + line) for line in self.lines] - newsource.lines = before.lines + lines + after.lines + lines = [(indent + line) for line in self.lines] + newsource.lines = before.lines + lines + after.lines return newsource def indent(self, indent=' ' * 4): @@ -97,7 +98,7 @@ def indent(self, indent=' ' * 4): all lines indented by the given indent-string. """ newsource = Source() - newsource.lines = [(indent+line) for line in self.lines] + newsource.lines = [(indent + line) for line in self.lines] return newsource def getstatement(self, lineno, assertion=False): @@ -136,7 +137,8 @@ def isparseable(self, deindent=True): try: import parser except ImportError: - syntax_checker = lambda x: compile(x, 'asd', 'exec') + def syntax_checker(x): + return compile(x, 'asd', 'exec') else: syntax_checker = parser.suite @@ -145,8 +147,8 @@ def isparseable(self, deindent=True): else: source = str(self) try: - #compile(source+'\n', "x", "exec") - syntax_checker(source+'\n') + # compile(source+'\n', "x", "exec") + syntax_checker(source + '\n') except KeyboardInterrupt: raise except Exception: @@ -166,8 +168,8 @@ def compile(self, filename=None, mode='exec', """ if not filename or py.path.local(filename).check(file=0): if _genframe is None: - _genframe = sys._getframe(1) # the caller - fn,lineno = _genframe.f_code.co_filename, _genframe.f_lineno + _genframe = sys._getframe(1) # the caller + fn, lineno = _genframe.f_code.co_filename, _genframe.f_lineno base = "<%d-codegen " % self._compilecounter self.__class__._compilecounter += 1 if not filename: @@ -182,7 +184,7 @@ def compile(self, filename=None, mode='exec', # re-represent syntax errors from parsing python strings msglines = self.lines[:ex.lineno] if ex.offset: - msglines.append(" "*ex.offset + '^') + msglines.append(" " * ex.offset + '^') msglines.append("(code was compiled probably from here: %s)" % filename) newex = SyntaxError('\n'.join(msglines)) newex.offset = ex.offset @@ -193,14 +195,6 @@ def compile(self, filename=None, mode='exec', if flag & _AST_FLAG: return co lines = [(x + "\n") for x in self.lines] - if sys.version_info[0] >= 3: - # XXX py3's inspect.getsourcefile() checks for a module - # and a pep302 __loader__ ... we don't have a module - # at code compile-time so we need to fake it here - m = ModuleType("_pycodecompile_pseudo_module") - py.std.inspect.modulesbyfile[filename] = None - py.std.sys.modules[None] = m - m.__loader__ = 1 py.std.linecache.cache[filename] = (1, None, lines, filename) return co @@ -208,8 +202,8 @@ def compile(self, filename=None, mode='exec', # public API shortcut functions # -def compile_(source, filename=None, mode='exec', flags= - generators.compiler_flag, dont_inherit=0): + +def compile_(source, filename=None, mode='exec', flags=generators.compiler_flag, dont_inherit=0): """ compile the given source to a raw code object, and maintain an internal cache which allows later retrieval of the source code for the code object @@ -218,7 +212,7 @@ def compile_(source, filename=None, mode='exec', flags= if _ast is not None and isinstance(source, _ast.AST): # XXX should Source support having AST? return cpy_compile(source, filename, mode, flags, dont_inherit) - _genframe = sys._getframe(1) # the caller + _genframe = sys._getframe(1) # the caller s = Source(source) co = s.compile(filename, mode, flags, _genframe=_genframe) return co @@ -255,17 +249,19 @@ def getfslineno(obj): # helper functions # + def findsource(obj): try: sourcelines, lineno = py.std.inspect.findsource(obj) except py.builtin._sysex: raise - except: + except: # noqa return None, -1 source = Source() source.lines = [line.rstrip() for line in sourcelines] return source, lineno + def getsource(obj, **kwargs): import _pytest._code obj = _pytest._code.getrawcode(obj) @@ -276,19 +272,21 @@ def getsource(obj, **kwargs): assert isinstance(strsrc, str) return Source(strsrc, **kwargs) + def deindent(lines, offset=None): if offset is None: for line in lines: line = line.expandtabs() s = line.lstrip() if s: - offset = len(line)-len(s) + offset = len(line) - len(s) break else: offset = 0 if offset == 0: return list(lines) newlines = [] + def readline_generator(lines): for line in lines: yield line + '\n' @@ -300,11 +298,11 @@ def readline_generator(lines): try: for _, _, (sline, _), (eline, _), _ in tokenize.generate_tokens(lambda: next(it)): if sline > len(lines): - break # End of input reached + break # End of input reached if sline > len(newlines): line = lines[sline - 1].expandtabs() if line.lstrip() and line[:offset].isspace(): - line = line[offset:] # Deindent + line = line[offset:] # Deindent newlines.append(line) for i in range(sline, eline): @@ -322,30 +320,28 @@ def get_statement_startend2(lineno, node): import ast # flatten all statements and except handlers into one lineno-list # AST's line numbers start indexing at 1 - l = [] + values = [] for x in ast.walk(node): if isinstance(x, _ast.stmt) or isinstance(x, _ast.ExceptHandler): - l.append(x.lineno - 1) + values.append(x.lineno - 1) for name in "finalbody", "orelse": val = getattr(x, name, None) if val: # treat the finally/orelse part as its own statement - l.append(val[0].lineno - 1 - 1) - l.sort() - insert_index = bisect_right(l, lineno) - start = l[insert_index - 1] - if insert_index >= len(l): + values.append(val[0].lineno - 1 - 1) + values.sort() + insert_index = bisect_right(values, lineno) + start = values[insert_index - 1] + if insert_index >= len(values): end = None else: - end = l[insert_index] + end = values[insert_index] return start, end def getstatementrange_ast(lineno, source, assertion=False, astnode=None): if astnode is None: content = str(source) - if sys.version_info < (2,7): - content += "\n" try: astnode = compile(content, "source", "exec", 1024) # 1024 for AST except ValueError: @@ -400,7 +396,7 @@ def getstatementrange_old(lineno, source, assertion=False): raise IndexError("likely a subclass") if "assert" not in line and "raise" not in line: continue - trylines = source.lines[start:lineno+1] + trylines = source.lines[start:lineno + 1] # quick hack to prepare parsing an indented line with # compile_command() (which errors on "return" outside defs) trylines.insert(0, 'def xxx():') @@ -412,10 +408,8 @@ def getstatementrange_old(lineno, source, assertion=False): continue # 2. find the end of the statement - for end in range(lineno+1, len(source)+1): + for end in range(lineno + 1, len(source) + 1): trysource = source[start:end] if trysource.isparseable(): return start, end raise SyntaxError("no valid source range around line %d " % (lineno,)) - - diff --git a/_pytest/_pluggy.py b/_pytest/_pluggy.py deleted file mode 100644 index 87d32cf8dd12afa..000000000000000 --- a/_pytest/_pluggy.py +++ /dev/null @@ -1,11 +0,0 @@ -""" -imports symbols from vendored "pluggy" if available, otherwise -falls back to importing "pluggy" from the default namespace. -""" - -try: - from _pytest.vendored_packages.pluggy import * # noqa - from _pytest.vendored_packages.pluggy import __version__ # noqa -except ImportError: - from pluggy import * # noqa - from pluggy import __version__ # noqa diff --git a/_pytest/assertion/__init__.py b/_pytest/assertion/__init__.py index 6921deb2a606880..a48e98c85aa2bbd 100644 --- a/_pytest/assertion/__init__.py +++ b/_pytest/assertion/__init__.py @@ -1,11 +1,13 @@ """ support for presenting detailed information in failing assertions. """ -import py -import os +from __future__ import absolute_import, division, print_function import sys -from _pytest.monkeypatch import monkeypatch +import six + from _pytest.assertion import util +from _pytest.assertion import rewrite +from _pytest.assertion import truncate def pytest_addoption(parser): @@ -13,25 +15,45 @@ def pytest_addoption(parser): group.addoption('--assert', action="store", dest="assertmode", - choices=("rewrite", "reinterp", "plain",), + choices=("rewrite", "plain",), default="rewrite", metavar="MODE", - help="""control assertion debugging tools. 'plain' - performs no assertion debugging. 'reinterp' - reinterprets assert statements after they failed - to provide assertion expression information. - 'rewrite' (the default) rewrites assert - statements in test modules on import to - provide assert expression information. """) - group.addoption('--no-assert', - action="store_true", - default=False, - dest="noassert", - help="DEPRECATED equivalent to --assert=plain") - group.addoption('--nomagic', '--no-magic', - action="store_true", - default=False, - help="DEPRECATED equivalent to --assert=plain") + help="""Control assertion debugging tools. 'plain' + performs no assertion debugging. 'rewrite' + (the default) rewrites assert statements in + test modules on import to provide assert + expression information.""") + + +def register_assert_rewrite(*names): + """Register one or more module names to be rewritten on import. + + This function will make sure that this module or all modules inside + the package will get their assert statements rewritten. + Thus you should make sure to call this before the module is + actually imported, usually in your __init__.py if you are a plugin + using a package. + + :raise TypeError: if the given module names are not strings. + """ + for name in names: + if not isinstance(name, str): + msg = 'expected module names as *args, got {0} instead' + raise TypeError(msg.format(repr(names))) + for hook in sys.meta_path: + if isinstance(hook, rewrite.AssertionRewritingHook): + importhook = hook + break + else: + importhook = DummyRewriteHook() + importhook.mark_rewrite(*names) + + +class DummyRewriteHook(object): + """A no-op import hook for when rewriting is disabled.""" + + def mark_rewrite(self, *names): + pass class AssertionState: @@ -40,57 +62,37 @@ class AssertionState: def __init__(self, config, mode): self.mode = mode self.trace = config.trace.root.get("assertion") + self.hook = None -def pytest_configure(config): - mode = config.getvalue("assertmode") - if config.getvalue("noassert") or config.getvalue("nomagic"): - mode = "plain" - if mode == "rewrite": - try: - import ast # noqa - except ImportError: - mode = "reinterp" - else: - # Both Jython and CPython 2.6.0 have AST bugs that make the - # assertion rewriting hook malfunction. - if (sys.platform.startswith('java') or - sys.version_info[:3] == (2, 6, 0)): - mode = "reinterp" - if mode != "plain": - _load_modules(mode) - m = monkeypatch() - config._cleanup.append(m.undo) - m.setattr(py.builtin.builtins, 'AssertionError', - reinterpret.AssertionError) # noqa - hook = None - if mode == "rewrite": - hook = rewrite.AssertionRewritingHook() # noqa - sys.meta_path.insert(0, hook) - warn_about_missing_assertion(mode) - config._assertstate = AssertionState(config, mode) - config._assertstate.hook = hook - config._assertstate.trace("configured with mode set to %r" % (mode,)) +def install_importhook(config): + """Try to install the rewrite hook, raise SystemError if it fails.""" + # Jython has an AST bug that make the assertion rewriting hook malfunction. + if (sys.platform.startswith('java')): + raise SystemError('rewrite not supported') + + config._assertstate = AssertionState(config, 'rewrite') + config._assertstate.hook = hook = rewrite.AssertionRewritingHook(config) + sys.meta_path.insert(0, hook) + config._assertstate.trace('installed rewrite import hook') + def undo(): hook = config._assertstate.hook if hook is not None and hook in sys.meta_path: sys.meta_path.remove(hook) + config.add_cleanup(undo) + return hook def pytest_collection(session): # this hook is only called when test modules are collected # so for example not in the master process of pytest-xdist # (which does not collect test modules) - hook = session.config._assertstate.hook - if hook is not None: - hook.set_session(session) - - -def _running_on_ci(): - """Check if we're currently running on a CI system.""" - env_vars = ['CI', 'BUILD_NUMBER'] - return any(var in os.environ for var in env_vars) + assertstate = getattr(session.config, '_assertstate', None) + if assertstate: + if assertstate.hook is not None: + assertstate.hook.set_session(session) def pytest_runtest_setup(item): @@ -106,8 +108,8 @@ def callbinrepr(op, left, right): This uses the first result from the hook and then ensures the following: - * Overly verbose explanations are dropped unless -vv was used or - running on a CI. + * Overly verbose explanations are truncated unless configured otherwise + (eg. if running in verbose mode). * Embedded newlines are escaped to help util.format_explanation() later. * If the rewrite mode is used embedded %-characters are replaced @@ -120,16 +122,9 @@ def callbinrepr(op, left, right): config=item.config, op=op, left=left, right=right) for new_expl in hook_result: if new_expl: - if (sum(len(p) for p in new_expl[1:]) > 80*8 and - item.config.option.verbose < 2 and - not _running_on_ci()): - show_max = 10 - truncated_lines = len(new_expl) - show_max - new_expl[show_max:] = [py.builtin._totext( - 'Detailed information truncated (%d more lines)' - ', use "-vv" to show' % truncated_lines)] + new_expl = truncate.truncate_if_required(new_expl, item) new_expl = [line.replace("\n", "\\n") for line in new_expl] - res = py.builtin._totext("\n~").join(new_expl) + res = six.text_type("\n~").join(new_expl) if item.config.getvalue("assertmode") == "rewrite": res = res.replace("%", "%%") return res @@ -141,35 +136,10 @@ def pytest_runtest_teardown(item): def pytest_sessionfinish(session): - hook = session.config._assertstate.hook - if hook is not None: - hook.session = None - - -def _load_modules(mode): - """Lazily import assertion related code.""" - global rewrite, reinterpret - from _pytest.assertion import reinterpret # noqa - if mode == "rewrite": - from _pytest.assertion import rewrite # noqa - - -def warn_about_missing_assertion(mode): - try: - assert False - except AssertionError: - pass - else: - if mode == "rewrite": - specifically = ("assertions which are not in test modules " - "will be ignored") - else: - specifically = "failing tests may report as passing" - - sys.stderr.write("WARNING: " + specifically + - " because assert statements are not executed " - "by the underlying Python interpreter " - "(are you using python -O?)\n") + assertstate = getattr(session.config, '_assertstate', None) + if assertstate: + if assertstate.hook is not None: + assertstate.hook.set_session(None) # Expose this plugin's implementation for the pytest_assertrepr_compare hook diff --git a/_pytest/assertion/reinterpret.py b/_pytest/assertion/reinterpret.py deleted file mode 100644 index f4262c3aced2d1d..000000000000000 --- a/_pytest/assertion/reinterpret.py +++ /dev/null @@ -1,407 +0,0 @@ -""" -Find intermediate evalutation results in assert statements through builtin AST. -""" -import ast -import sys - -import _pytest._code -import py -from _pytest.assertion import util -u = py.builtin._totext - - -class AssertionError(util.BuiltinAssertionError): - def __init__(self, *args): - util.BuiltinAssertionError.__init__(self, *args) - if args: - # on Python2.6 we get len(args)==2 for: assert 0, (x,y) - # on Python2.7 and above we always get len(args) == 1 - # with args[0] being the (x,y) tuple. - if len(args) > 1: - toprint = args - else: - toprint = args[0] - try: - self.msg = u(toprint) - except Exception: - self.msg = u( - "<[broken __repr__] %s at %0xd>" - % (toprint.__class__, id(toprint))) - else: - f = _pytest._code.Frame(sys._getframe(1)) - try: - source = f.code.fullsource - if source is not None: - try: - source = source.getstatement(f.lineno, assertion=True) - except IndexError: - source = None - else: - source = str(source.deindent()).strip() - except py.error.ENOENT: - source = None - # this can also occur during reinterpretation, when the - # co_filename is set to "". - if source: - self.msg = reinterpret(source, f, should_fail=True) - else: - self.msg = "" - if not self.args: - self.args = (self.msg,) - -if sys.version_info > (3, 0): - AssertionError.__module__ = "builtins" - -if sys.platform.startswith("java"): - # See http://bugs.jython.org/issue1497 - _exprs = ("BoolOp", "BinOp", "UnaryOp", "Lambda", "IfExp", "Dict", - "ListComp", "GeneratorExp", "Yield", "Compare", "Call", - "Repr", "Num", "Str", "Attribute", "Subscript", "Name", - "List", "Tuple") - _stmts = ("FunctionDef", "ClassDef", "Return", "Delete", "Assign", - "AugAssign", "Print", "For", "While", "If", "With", "Raise", - "TryExcept", "TryFinally", "Assert", "Import", "ImportFrom", - "Exec", "Global", "Expr", "Pass", "Break", "Continue") - _expr_nodes = set(getattr(ast, name) for name in _exprs) - _stmt_nodes = set(getattr(ast, name) for name in _stmts) - def _is_ast_expr(node): - return node.__class__ in _expr_nodes - def _is_ast_stmt(node): - return node.__class__ in _stmt_nodes -else: - def _is_ast_expr(node): - return isinstance(node, ast.expr) - def _is_ast_stmt(node): - return isinstance(node, ast.stmt) - -try: - _Starred = ast.Starred -except AttributeError: - # Python 2. Define a dummy class so isinstance() will always be False. - class _Starred(object): pass - - -class Failure(Exception): - """Error found while interpreting AST.""" - - def __init__(self, explanation=""): - self.cause = sys.exc_info() - self.explanation = explanation - - -def reinterpret(source, frame, should_fail=False): - mod = ast.parse(source) - visitor = DebugInterpreter(frame) - try: - visitor.visit(mod) - except Failure: - failure = sys.exc_info()[1] - return getfailure(failure) - if should_fail: - return ("(assertion failed, but when it was re-run for " - "printing intermediate values, it did not fail. Suggestions: " - "compute assert expression before the assert or use --assert=plain)") - -def run(offending_line, frame=None): - if frame is None: - frame = _pytest._code.Frame(sys._getframe(1)) - return reinterpret(offending_line, frame) - -def getfailure(e): - explanation = util.format_explanation(e.explanation) - value = e.cause[1] - if str(value): - lines = explanation.split('\n') - lines[0] += " << %s" % (value,) - explanation = '\n'.join(lines) - text = "%s: %s" % (e.cause[0].__name__, explanation) - if text.startswith('AssertionError: assert '): - text = text[16:] - return text - -operator_map = { - ast.BitOr : "|", - ast.BitXor : "^", - ast.BitAnd : "&", - ast.LShift : "<<", - ast.RShift : ">>", - ast.Add : "+", - ast.Sub : "-", - ast.Mult : "*", - ast.Div : "/", - ast.FloorDiv : "//", - ast.Mod : "%", - ast.Eq : "==", - ast.NotEq : "!=", - ast.Lt : "<", - ast.LtE : "<=", - ast.Gt : ">", - ast.GtE : ">=", - ast.Pow : "**", - ast.Is : "is", - ast.IsNot : "is not", - ast.In : "in", - ast.NotIn : "not in" -} - -unary_map = { - ast.Not : "not %s", - ast.Invert : "~%s", - ast.USub : "-%s", - ast.UAdd : "+%s" -} - - -class DebugInterpreter(ast.NodeVisitor): - """Interpret AST nodes to gleam useful debugging information. """ - - def __init__(self, frame): - self.frame = frame - - def generic_visit(self, node): - # Fallback when we don't have a special implementation. - if _is_ast_expr(node): - mod = ast.Expression(node) - co = self._compile(mod) - try: - result = self.frame.eval(co) - except Exception: - raise Failure() - explanation = self.frame.repr(result) - return explanation, result - elif _is_ast_stmt(node): - mod = ast.Module([node]) - co = self._compile(mod, "exec") - try: - self.frame.exec_(co) - except Exception: - raise Failure() - return None, None - else: - raise AssertionError("can't handle %s" %(node,)) - - def _compile(self, source, mode="eval"): - return compile(source, "", mode) - - def visit_Expr(self, expr): - return self.visit(expr.value) - - def visit_Module(self, mod): - for stmt in mod.body: - self.visit(stmt) - - def visit_Name(self, name): - explanation, result = self.generic_visit(name) - # See if the name is local. - source = "%r in locals() is not globals()" % (name.id,) - co = self._compile(source) - try: - local = self.frame.eval(co) - except Exception: - # have to assume it isn't - local = None - if local is None or not self.frame.is_true(local): - return name.id, result - return explanation, result - - def visit_Compare(self, comp): - left = comp.left - left_explanation, left_result = self.visit(left) - for op, next_op in zip(comp.ops, comp.comparators): - next_explanation, next_result = self.visit(next_op) - op_symbol = operator_map[op.__class__] - explanation = "%s %s %s" % (left_explanation, op_symbol, - next_explanation) - source = "__exprinfo_left %s __exprinfo_right" % (op_symbol,) - co = self._compile(source) - try: - result = self.frame.eval(co, __exprinfo_left=left_result, - __exprinfo_right=next_result) - except Exception: - raise Failure(explanation) - try: - if not self.frame.is_true(result): - break - except KeyboardInterrupt: - raise - except: - break - left_explanation, left_result = next_explanation, next_result - - if util._reprcompare is not None: - res = util._reprcompare(op_symbol, left_result, next_result) - if res: - explanation = res - return explanation, result - - def visit_BoolOp(self, boolop): - is_or = isinstance(boolop.op, ast.Or) - explanations = [] - for operand in boolop.values: - explanation, result = self.visit(operand) - explanations.append(explanation) - if result == is_or: - break - name = is_or and " or " or " and " - explanation = "(" + name.join(explanations) + ")" - return explanation, result - - def visit_UnaryOp(self, unary): - pattern = unary_map[unary.op.__class__] - operand_explanation, operand_result = self.visit(unary.operand) - explanation = pattern % (operand_explanation,) - co = self._compile(pattern % ("__exprinfo_expr",)) - try: - result = self.frame.eval(co, __exprinfo_expr=operand_result) - except Exception: - raise Failure(explanation) - return explanation, result - - def visit_BinOp(self, binop): - left_explanation, left_result = self.visit(binop.left) - right_explanation, right_result = self.visit(binop.right) - symbol = operator_map[binop.op.__class__] - explanation = "(%s %s %s)" % (left_explanation, symbol, - right_explanation) - source = "__exprinfo_left %s __exprinfo_right" % (symbol,) - co = self._compile(source) - try: - result = self.frame.eval(co, __exprinfo_left=left_result, - __exprinfo_right=right_result) - except Exception: - raise Failure(explanation) - return explanation, result - - def visit_Call(self, call): - func_explanation, func = self.visit(call.func) - arg_explanations = [] - ns = {"__exprinfo_func" : func} - arguments = [] - for arg in call.args: - arg_explanation, arg_result = self.visit(arg) - if isinstance(arg, _Starred): - arg_name = "__exprinfo_star" - ns[arg_name] = arg_result - arguments.append("*%s" % (arg_name,)) - arg_explanations.append("*%s" % (arg_explanation,)) - else: - arg_name = "__exprinfo_%s" % (len(ns),) - ns[arg_name] = arg_result - arguments.append(arg_name) - arg_explanations.append(arg_explanation) - for keyword in call.keywords: - arg_explanation, arg_result = self.visit(keyword.value) - if keyword.arg: - arg_name = "__exprinfo_%s" % (len(ns),) - keyword_source = "%s=%%s" % (keyword.arg) - arguments.append(keyword_source % (arg_name,)) - arg_explanations.append(keyword_source % (arg_explanation,)) - else: - arg_name = "__exprinfo_kwds" - arguments.append("**%s" % (arg_name,)) - arg_explanations.append("**%s" % (arg_explanation,)) - - ns[arg_name] = arg_result - - if getattr(call, 'starargs', None): - arg_explanation, arg_result = self.visit(call.starargs) - arg_name = "__exprinfo_star" - ns[arg_name] = arg_result - arguments.append("*%s" % (arg_name,)) - arg_explanations.append("*%s" % (arg_explanation,)) - - if getattr(call, 'kwargs', None): - arg_explanation, arg_result = self.visit(call.kwargs) - arg_name = "__exprinfo_kwds" - ns[arg_name] = arg_result - arguments.append("**%s" % (arg_name,)) - arg_explanations.append("**%s" % (arg_explanation,)) - args_explained = ", ".join(arg_explanations) - explanation = "%s(%s)" % (func_explanation, args_explained) - args = ", ".join(arguments) - source = "__exprinfo_func(%s)" % (args,) - co = self._compile(source) - try: - result = self.frame.eval(co, **ns) - except Exception: - raise Failure(explanation) - pattern = "%s\n{%s = %s\n}" - rep = self.frame.repr(result) - explanation = pattern % (rep, rep, explanation) - return explanation, result - - def _is_builtin_name(self, name): - pattern = "%r not in globals() and %r not in locals()" - source = pattern % (name.id, name.id) - co = self._compile(source) - try: - return self.frame.eval(co) - except Exception: - return False - - def visit_Attribute(self, attr): - if not isinstance(attr.ctx, ast.Load): - return self.generic_visit(attr) - source_explanation, source_result = self.visit(attr.value) - explanation = "%s.%s" % (source_explanation, attr.attr) - source = "__exprinfo_expr.%s" % (attr.attr,) - co = self._compile(source) - try: - try: - result = self.frame.eval(co, __exprinfo_expr=source_result) - except AttributeError: - # Maybe the attribute name needs to be mangled? - if not attr.attr.startswith("__") or attr.attr.endswith("__"): - raise - source = "getattr(__exprinfo_expr.__class__, '__name__', '')" - co = self._compile(source) - class_name = self.frame.eval(co, __exprinfo_expr=source_result) - mangled_attr = "_" + class_name + attr.attr - source = "__exprinfo_expr.%s" % (mangled_attr,) - co = self._compile(source) - result = self.frame.eval(co, __exprinfo_expr=source_result) - except Exception: - raise Failure(explanation) - explanation = "%s\n{%s = %s.%s\n}" % (self.frame.repr(result), - self.frame.repr(result), - source_explanation, attr.attr) - # Check if the attr is from an instance. - source = "%r in getattr(__exprinfo_expr, '__dict__', {})" - source = source % (attr.attr,) - co = self._compile(source) - try: - from_instance = self.frame.eval(co, __exprinfo_expr=source_result) - except Exception: - from_instance = None - if from_instance is None or self.frame.is_true(from_instance): - rep = self.frame.repr(result) - pattern = "%s\n{%s = %s\n}" - explanation = pattern % (rep, rep, explanation) - return explanation, result - - def visit_Assert(self, assrt): - test_explanation, test_result = self.visit(assrt.test) - explanation = "assert %s" % (test_explanation,) - if not self.frame.is_true(test_result): - try: - raise util.BuiltinAssertionError - except Exception: - raise Failure(explanation) - return explanation, test_result - - def visit_Assign(self, assign): - value_explanation, value_result = self.visit(assign.value) - explanation = "... = %s" % (value_explanation,) - name = ast.Name("__exprinfo_expr", ast.Load(), - lineno=assign.value.lineno, - col_offset=assign.value.col_offset) - new_assign = ast.Assign(assign.targets, name, lineno=assign.lineno, - col_offset=assign.col_offset) - mod = ast.Module([new_assign]) - co = self._compile(mod, "exec") - try: - self.frame.exec_(co, __exprinfo_expr=value_result) - except Exception: - raise Failure(explanation) - return explanation, value_result - diff --git a/_pytest/assertion/rewrite.py b/_pytest/assertion/rewrite.py index 14b8e49db2b9281..f64358f490bc91a 100644 --- a/_pytest/assertion/rewrite.py +++ b/_pytest/assertion/rewrite.py @@ -1,12 +1,14 @@ """Rewrite assertion AST to produce nice error messages""" - +from __future__ import absolute_import, division, print_function import ast +import _ast import errno import itertools import imp import marshal import os import re +import six import struct import sys import types @@ -32,32 +34,32 @@ PYC_EXT = ".py" + (__debug__ and "c" or "o") PYC_TAIL = "." + PYTEST_TAG + PYC_EXT -REWRITE_NEWLINES = sys.version_info[:2] != (2, 7) and sys.version_info < (3, 2) ASCII_IS_DEFAULT_ENCODING = sys.version_info[0] < 3 -if sys.version_info >= (3,5): +if sys.version_info >= (3, 5): ast_Call = ast.Call else: - ast_Call = lambda a,b,c: ast.Call(a, b, c, None, None) + def ast_Call(a, b, c): + return ast.Call(a, b, c, None, None) class AssertionRewritingHook(object): """PEP302 Import hook which rewrites asserts.""" - def __init__(self): + def __init__(self, config): + self.config = config + self.fnpats = config.getini("python_files") self.session = None self.modules = {} + self._rewritten_names = set() self._register_with_pkg_resources() + self._must_rewrite = set() def set_session(self, session): - self.fnpats = session.config.getini("python_files") self.session = session def find_module(self, name, path=None): - if self.session is None: - return None - sess = self.session - state = sess.config._assertstate + state = self.config._assertstate state.trace("find_module called for: %s" % name) names = name.rsplit(".", 1) lastname = names[-1] @@ -78,7 +80,12 @@ def find_module(self, name, path=None): tp = desc[2] if tp == imp.PY_COMPILED: if hasattr(imp, "source_from_cache"): - fn = imp.source_from_cache(fn) + try: + fn = imp.source_from_cache(fn) + except ValueError: + # Python 3 doesn't like orphaned but still-importable + # .pyc files. + fn = fn[:-1] else: fn = fn[:-1] elif tp != imp.PY_SOURCE: @@ -86,24 +93,13 @@ def find_module(self, name, path=None): return None else: fn = os.path.join(pth, name.rpartition(".")[2] + ".py") + fn_pypath = py.path.local(fn) - # Is this a test file? - if not sess.isinitpath(fn): - # We have to be very careful here because imports in this code can - # trigger a cycle. - self.session = None - try: - for pat in self.fnpats: - if fn_pypath.fnmatch(pat): - state.trace("matched test file %r" % (fn,)) - break - else: - return None - finally: - self.session = sess - else: - state.trace("matched test file (was specified on cmdline): %r" % - (fn,)) + if not self._should_rewrite(name, fn_pypath, state): + return None + + self._rewritten_names.add(name) + # The requested module looks like a test file, so rewrite it. This is # the most magical part of the process: load the source, rewrite the # asserts, and load the rewritten source. We also cache the rewritten @@ -140,7 +136,7 @@ def find_module(self, name, path=None): co = _read_pyc(fn_pypath, pyc, state.trace) if co is None: state.trace("rewriting %r" % (fn,)) - source_stat, co = _rewrite_test(state, fn_pypath) + source_stat, co = _rewrite_test(self.config, fn_pypath) if co is None: # Probably a SyntaxError in the test. return None @@ -151,6 +147,51 @@ def find_module(self, name, path=None): self.modules[name] = co, pyc return self + def _should_rewrite(self, name, fn_pypath, state): + # always rewrite conftest files + fn = str(fn_pypath) + if fn_pypath.basename == 'conftest.py': + state.trace("rewriting conftest file: %r" % (fn,)) + return True + + if self.session is not None: + if self.session.isinitpath(fn): + state.trace("matched test file (was specified on cmdline): %r" % + (fn,)) + return True + + # modules not passed explicitly on the command line are only + # rewritten if they match the naming convention for test files + for pat in self.fnpats: + if fn_pypath.fnmatch(pat): + state.trace("matched test file %r" % (fn,)) + return True + + for marked in self._must_rewrite: + if name == marked or name.startswith(marked + '.'): + state.trace("matched marked file %r (from %r)" % (name, marked)) + return True + + return False + + def mark_rewrite(self, *names): + """Mark import names as needing to be rewritten. + + The named module or package as well as any nested modules will + be rewritten on import. + """ + already_imported = set(names).intersection(set(sys.modules)) + if already_imported: + for name in already_imported: + if name not in self._rewritten_names: + self._warn_already_imported(name) + self._must_rewrite.update(names) + + def _warn_already_imported(self, name): + self.config.warn( + 'P1', + 'Module already imported so cannot be rewritten: %s' % name) + def load_module(self, name): # If there is an existing module object named 'fullname' in # sys.modules, the loader must use that existing module. (Otherwise, @@ -169,13 +210,12 @@ def load_module(self, name): mod.__cached__ = pyc mod.__loader__ = self py.builtin.exec_(co, mod.__dict__) - except: - del sys.modules[name] + except: # noqa + if name in sys.modules: + del sys.modules[name] raise return sys.modules[name] - - def is_package(self, name): try: fd, fn, desc = imp.find_module(name) @@ -220,7 +260,7 @@ def _write_pyc(state, co, source_stat, pyc): fp = open(pyc, "wb") except IOError: err = sys.exc_info()[1].errno - state.trace("error writing pyc file at %s: errno=%s" %(pyc, err)) + state.trace("error writing pyc file at %s: errno=%s" % (pyc, err)) # we ignore any failure to write the cache file # there are many reasons, permission-denied, __pycache__ being a # file etc. @@ -235,14 +275,17 @@ def _write_pyc(state, co, source_stat, pyc): fp.close() return True + RN = "\r\n".encode("utf-8") N = "\n".encode("utf-8") cookie_re = re.compile(r"^[ \t\f]*#.*coding[:=][ \t]*[-\w.]+") BOM_UTF8 = '\xef\xbb\xbf' -def _rewrite_test(state, fn): + +def _rewrite_test(config, fn): """Try to read and rewrite *fn* and return the code object.""" + state = config._assertstate try: stat = fn.stat() source = fn.read("rb") @@ -264,7 +307,7 @@ def _rewrite_test(state, fn): end2 = source.find("\n", end1 + 1) if (not source.startswith(BOM_UTF8) and cookie_re.match(source[0:end1]) is None and - cookie_re.match(source[end1 + 1:end2]) is None): + cookie_re.match(source[end1 + 1:end2]) is None): if hasattr(state, "_indecode"): # encodings imported us again, so don't rewrite. return None, None @@ -277,19 +320,15 @@ def _rewrite_test(state, fn): return None, None finally: del state._indecode - # On Python versions which are not 2.7 and less than or equal to 3.1, the - # parser expects *nix newlines. - if REWRITE_NEWLINES: - source = source.replace(RN, N) + N try: tree = ast.parse(source) except SyntaxError: # Let this pop up again in the real import. state.trace("failed to parse: %r" % (fn,)) return None, None - rewrite_asserts(tree) + rewrite_asserts(tree, fn, config) try: - co = compile(tree, fn.strpath, "exec") + co = compile(tree, fn.strpath, "exec", dont_inherit=True) except SyntaxError: # It's possible that this error is from some bug in the # assertion rewriting, but I don't know of a fast way to tell. @@ -297,6 +336,7 @@ def _rewrite_test(state, fn): return None, None return stat, co + def _make_rewritten_pyc(state, source_stat, pyc, co): """Try to dump rewritten code to *pyc*.""" if sys.platform.startswith("win"): @@ -310,6 +350,7 @@ def _make_rewritten_pyc(state, source_stat, pyc, co): if _write_pyc(state, co, source_stat, proc_pyc): os.rename(proc_pyc, pyc) + def _read_pyc(source, pyc, trace=lambda x: None): """Possibly read a pytest pyc containing rewritten code. @@ -343,9 +384,9 @@ def _read_pyc(source, pyc, trace=lambda x: None): return co -def rewrite_asserts(mod): +def rewrite_asserts(mod, module_path=None, config=None): """Rewrite the assert statements in mod.""" - AssertionRewriter().run(mod) + AssertionRewriter(module_path, config).run(mod) def _saferepr(obj): @@ -360,14 +401,15 @@ def _saferepr(obj): """ repr = py.io.saferepr(obj) - if py.builtin._istext(repr): - t = py.builtin.text + if isinstance(repr, six.text_type): + t = six.text_type else: - t = py.builtin.bytes + t = six.binary_type return repr.replace(t("\n"), t("\\n")) -from _pytest.assertion.util import format_explanation as _format_explanation # noqa +from _pytest.assertion.util import format_explanation as _format_explanation # noqa + def _format_assertmsg(obj): """Format the custom assertion message given. @@ -381,32 +423,35 @@ def _format_assertmsg(obj): # contains a newline it gets escaped, however if an object has a # .__repr__() which contains newlines it does not get escaped. # However in either case we want to preserve the newline. - if py.builtin._istext(obj) or py.builtin._isbytes(obj): + if isinstance(obj, six.text_type) or isinstance(obj, six.binary_type): s = obj is_repr = False else: s = py.io.saferepr(obj) is_repr = True - if py.builtin._istext(s): - t = py.builtin.text + if isinstance(s, six.text_type): + t = six.text_type else: - t = py.builtin.bytes + t = six.binary_type s = s.replace(t("\n"), t("\n~")).replace(t("%"), t("%%")) if is_repr: s = s.replace(t("\\n"), t("\n~")) return s + def _should_repr_global_name(obj): - return not hasattr(obj, "__name__") and not py.builtin.callable(obj) + return not hasattr(obj, "__name__") and not callable(obj) + def _format_boolop(explanations, is_or): explanation = "(" + (is_or and " or " or " and ").join(explanations) + ")" - if py.builtin._istext(explanation): - t = py.builtin.text + if isinstance(explanation, six.text_type): + t = six.text_type else: - t = py.builtin.bytes + t = six.binary_type return explanation.replace(t('%'), t('%%')) + def _call_reprcompare(ops, results, expls, each_obj): for i, res, expl in zip(range(len(ops)), results, expls): try: @@ -440,7 +485,7 @@ def _call_reprcompare(ops, results, expls, each_obj): ast.Mult: "*", ast.Div: "/", ast.FloorDiv: "//", - ast.Mod: "%%", # escaped for string formatting + ast.Mod: "%%", # escaped for string formatting ast.Eq: "==", ast.NotEq: "!=", ast.Lt: "<", @@ -484,7 +529,7 @@ class AssertionRewriter(ast.NodeVisitor): """Assertion rewriting implementation. The main entrypoint is to call .run() with an ast.Module instance, - this will then find all the assert statements and re-write them to + this will then find all the assert statements and rewrite them to provide intermediate values and a detailed assertion error. See http://pybites.blogspot.be/2011/07/behind-scenes-of-pytests-new-assertion.html for an overview of how this works. @@ -493,7 +538,7 @@ class AssertionRewriter(ast.NodeVisitor): statements in an ast.Module and for each ast.Assert statement it finds call .visit() with it. Then .visit_Assert() takes over and is responsible for creating new ast statements to replace the - original assert statement: it re-writes the test of an assertion + original assert statement: it rewrites the test of an assertion to provide intermediate values and replace it with an if statement which raises an assertion error with a detailed explanation in case the expression is false. @@ -532,6 +577,11 @@ class AssertionRewriter(ast.NodeVisitor): """ + def __init__(self, module_path, config): + super(AssertionRewriter, self).__init__() + self.module_path = module_path + self.config = config + def run(self, mod): """Find all assert statements in *mod* and rewrite them.""" if not mod.body: @@ -541,23 +591,26 @@ def run(self, mod): # docstrings and __future__ imports. aliases = [ast.alias(py.builtin.builtins.__name__, "@py_builtins"), ast.alias("_pytest.assertion.rewrite", "@pytest_ar")] - expect_docstring = True + doc = getattr(mod, "docstring", None) + expect_docstring = doc is None + if doc is not None and self.is_rewrite_disabled(doc): + return pos = 0 - lineno = 0 + lineno = 1 for item in mod.body: if (expect_docstring and isinstance(item, ast.Expr) and isinstance(item.value, ast.Str)): doc = item.value.s - if "PYTEST_DONT_REWRITE" in doc: - # The module has disabled assertion rewriting. + if self.is_rewrite_disabled(doc): return - lineno += len(doc) - 1 expect_docstring = False elif (not isinstance(item, ast.ImportFrom) or item.level > 0 or item.module != "__future__"): lineno = item.lineno break pos += 1 + else: + lineno = item.lineno imports = [ast.Import([alias], lineno=lineno, col_offset=0) for alias in aliases] mod.body[pos:pos] = imports @@ -583,6 +636,9 @@ def run(self, mod): not isinstance(field, ast.expr)): nodes.append(field) + def is_rewrite_disabled(self, docstring): + return "PYTEST_DONT_REWRITE" in docstring + def variable(self): """Get a new variable.""" # Use a character invalid in python identifiers to avoid clashing. @@ -666,12 +722,16 @@ def generic_visit(self, node): def visit_Assert(self, assert_): """Return the AST statements to replace the ast.Assert instance. - This re-writes the test of an assertion to provide + This rewrites the test of an assertion to provide intermediate values and replace it with an if statement which raises an assertion error with a detailed explanation in case the expression is false. """ + if isinstance(assert_.test, ast.Tuple) and self.config is not None: + fslocation = (self.module_path, assert_.lineno) + self.config.warn('R1', 'assertion is always true, perhaps ' + 'remove parentheses?', fslocation=fslocation) self.statements = [] self.variables = [] self.variable_counter = itertools.count() @@ -735,7 +795,7 @@ def visit_BoolOp(self, boolop): if i: fail_inner = [] # cond is set in a prior loop iteration below - self.on_failure.append(ast.If(cond, fail_inner, [])) # noqa + self.on_failure.append(ast.If(cond, fail_inner, [])) # noqa self.on_failure = fail_inner self.push_format_context() res, expl = self.visit(v) @@ -787,7 +847,7 @@ def visit_Call_35(self, call): new_kwargs.append(ast.keyword(keyword.arg, res)) if keyword.arg: arg_expls.append(keyword.arg + "=" + expl) - else: ## **args have `arg` keywords with an .arg of None + else: # **args have `arg` keywords with an .arg of None arg_expls.append("**" + expl) expl = "%s(%s)" % (func_expl, ', '.join(arg_expls)) @@ -841,7 +901,6 @@ def visit_Call_legacy(self, call): else: visit_Call = visit_Call_legacy - def visit_Attribute(self, attr): if not isinstance(attr.ctx, ast.Load): return self.generic_visit(attr) @@ -855,6 +914,8 @@ def visit_Attribute(self, attr): def visit_Compare(self, comp): self.push_format_context() left_res, left_expl = self.visit(comp.left) + if isinstance(comp.left, (_ast.Compare, _ast.BoolOp)): + left_expl = "({0})".format(left_expl) res_variables = [self.variable() for i in range(len(comp.ops))] load_names = [ast.Name(v, ast.Load()) for v in res_variables] store_names = [ast.Name(v, ast.Store()) for v in res_variables] @@ -864,6 +925,8 @@ def visit_Compare(self, comp): results = [left_res] for i, op, next_operand in it: next_res, next_expl = self.visit(next_operand) + if isinstance(next_operand, (_ast.Compare, _ast.BoolOp)): + next_expl = "({0})".format(next_expl) results.append(next_res) sym = binop_map[op.__class__] syms.append(ast.Str(sym)) diff --git a/_pytest/assertion/truncate.py b/_pytest/assertion/truncate.py new file mode 100644 index 000000000000000..2ed12e2e5a960ca --- /dev/null +++ b/_pytest/assertion/truncate.py @@ -0,0 +1,102 @@ +""" +Utilities for truncating assertion output. + +Current default behaviour is to truncate assertion explanations at +~8 terminal lines, unless running in "-vv" mode or running on CI. +""" +from __future__ import absolute_import, division, print_function +import os + +import six + + +DEFAULT_MAX_LINES = 8 +DEFAULT_MAX_CHARS = 8 * 80 +USAGE_MSG = "use '-vv' to show" + + +def truncate_if_required(explanation, item, max_length=None): + """ + Truncate this assertion explanation if the given test item is eligible. + """ + if _should_truncate_item(item): + return _truncate_explanation(explanation) + return explanation + + +def _should_truncate_item(item): + """ + Whether or not this test item is eligible for truncation. + """ + verbose = item.config.option.verbose + return verbose < 2 and not _running_on_ci() + + +def _running_on_ci(): + """Check if we're currently running on a CI system.""" + env_vars = ['CI', 'BUILD_NUMBER'] + return any(var in os.environ for var in env_vars) + + +def _truncate_explanation(input_lines, max_lines=None, max_chars=None): + """ + Truncate given list of strings that makes up the assertion explanation. + + Truncates to either 8 lines, or 640 characters - whichever the input reaches + first. The remaining lines will be replaced by a usage message. + """ + + if max_lines is None: + max_lines = DEFAULT_MAX_LINES + if max_chars is None: + max_chars = DEFAULT_MAX_CHARS + + # Check if truncation required + input_char_count = len("".join(input_lines)) + if len(input_lines) <= max_lines and input_char_count <= max_chars: + return input_lines + + # Truncate first to max_lines, and then truncate to max_chars if max_chars + # is exceeded. + truncated_explanation = input_lines[:max_lines] + truncated_explanation = _truncate_by_char_count(truncated_explanation, max_chars) + + # Add ellipsis to final line + truncated_explanation[-1] = truncated_explanation[-1] + "..." + + # Append useful message to explanation + truncated_line_count = len(input_lines) - len(truncated_explanation) + truncated_line_count += 1 # Account for the part-truncated final line + msg = '...Full output truncated' + if truncated_line_count == 1: + msg += ' ({0} line hidden)'.format(truncated_line_count) + else: + msg += ' ({0} lines hidden)'.format(truncated_line_count) + msg += ", {0}" .format(USAGE_MSG) + truncated_explanation.extend([ + six.text_type(""), + six.text_type(msg), + ]) + return truncated_explanation + + +def _truncate_by_char_count(input_lines, max_chars): + # Check if truncation required + if len("".join(input_lines)) <= max_chars: + return input_lines + + # Find point at which input length exceeds total allowed length + iterated_char_count = 0 + for iterated_index, input_line in enumerate(input_lines): + if iterated_char_count + len(input_line) > max_chars: + break + iterated_char_count += len(input_line) + + # Create truncated explanation with modified final line + truncated_result = input_lines[:iterated_index] + final_line = input_lines[iterated_index] + if final_line: + final_line_truncate_point = max_chars - iterated_char_count + final_line = final_line[:final_line_truncate_point] + truncated_result.append(final_line) + return truncated_result diff --git a/_pytest/assertion/util.py b/_pytest/assertion/util.py index f2f23efea27558e..511d98ef1fdf196 100644 --- a/_pytest/assertion/util.py +++ b/_pytest/assertion/util.py @@ -1,15 +1,17 @@ """Utilities for assertion debugging""" +from __future__ import absolute_import, division, print_function import pprint import _pytest._code import py +import six try: from collections import Sequence except ImportError: Sequence = list -BuiltinAssertionError = py.builtin.builtins.AssertionError -u = py.builtin._totext + +u = six.text_type # The _reprcompare attribute on the util module is used by the new assertion # interpretation code and assertion rewriter to detect this plugin was @@ -38,44 +40,11 @@ def format_explanation(explanation): displaying diffs. """ explanation = ecu(explanation) - explanation = _collapse_false(explanation) lines = _split_explanation(explanation) result = _format_lines(lines) return u('\n').join(result) -def _collapse_false(explanation): - """Collapse expansions of False - - So this strips out any "assert False\n{where False = ...\n}" - blocks. - """ - where = 0 - while True: - start = where = explanation.find("False\n{False = ", where) - if where == -1: - break - level = 0 - prev_c = explanation[start] - for i, c in enumerate(explanation[start:]): - if prev_c + c == "\n{": - level += 1 - elif prev_c + c == "\n}": - level -= 1 - if not level: - break - prev_c = c - else: - raise AssertionError("unbalanced braces: %r" % (explanation,)) - end = start + i - where = end - if explanation[end - 1] == '\n': - explanation = (explanation[:start] + explanation[start+15:end-1] + - explanation[end+1:]) - where -= 17 - return explanation - - def _split_explanation(explanation): """Return a list of individual lines in the explanation @@ -85,11 +54,11 @@ def _split_explanation(explanation): """ raw_lines = (explanation or u('')).split('\n') lines = [raw_lines[0]] - for l in raw_lines[1:]: - if l and l[0] in ['{', '}', '~', '>']: - lines.append(l) + for values in raw_lines[1:]: + if values and values[0] in ['{', '}', '~', '>']: + lines.append(values) else: - lines[-1] += '\\n' + l + lines[-1] += '\\n' + values return lines @@ -114,7 +83,7 @@ def _format_lines(lines): stack.append(len(result)) stackcnt[-1] += 1 stackcnt.append(0) - result.append(u(' +') + u(' ')*(len(stack)-1) + s + line[1:]) + result.append(u(' +') + u(' ') * (len(stack) - 1) + s + line[1:]) elif line.startswith('}'): stack.pop() stackcnt.pop() @@ -123,7 +92,7 @@ def _format_lines(lines): assert line[0] in ['~', '>'] stack[-1] += 1 indent = len(stack) if line.startswith('~') else len(stack) - 1 - result.append(u(' ')*indent + line[1:]) + result.append(u(' ') * indent + line[1:]) assert len(stack) == 1 return result @@ -138,16 +107,22 @@ def _format_lines(lines): def assertrepr_compare(config, op, left, right): """Return specialised explanations for some operators/operands""" width = 80 - 15 - len(op) - 2 # 15 chars indentation, 1 space around op - left_repr = py.io.saferepr(left, maxsize=int(width/2)) - right_repr = py.io.saferepr(right, maxsize=width-len(left_repr)) + left_repr = py.io.saferepr(left, maxsize=int(width // 2)) + right_repr = py.io.saferepr(right, maxsize=width - len(left_repr)) summary = u('%s %s %s') % (ecu(left_repr), op, ecu(right_repr)) - issequence = lambda x: (isinstance(x, (list, tuple, Sequence)) and - not isinstance(x, basestring)) - istext = lambda x: isinstance(x, basestring) - isdict = lambda x: isinstance(x, dict) - isset = lambda x: isinstance(x, (set, frozenset)) + def issequence(x): + return (isinstance(x, (list, tuple, Sequence)) and not isinstance(x, basestring)) + + def istext(x): + return isinstance(x, basestring) + + def isdict(x): + return isinstance(x, dict) + + def isset(x): + return isinstance(x, (set, frozenset)) def isiterable(obj): try: @@ -200,9 +175,9 @@ def _diff_text(left, right, verbose=False): """ from difflib import ndiff explanation = [] - if isinstance(left, py.builtin.bytes): + if isinstance(left, six.binary_type): left = u(repr(left)[1:-1]).replace(r'\n', '\n') - if isinstance(right, py.builtin.bytes): + if isinstance(right, six.binary_type): right = u(repr(right)[1:-1]).replace(r'\n', '\n') if not verbose: i = 0 # just in case left or right has zero length @@ -225,9 +200,10 @@ def _diff_text(left, right, verbose=False): 'characters in diff, use -v to show') % i] left = left[:-i] right = right[:-i] + keepends = True explanation += [line.strip('\n') - for line in ndiff(left.splitlines(), - right.splitlines())] + for line in ndiff(left.splitlines(keepends), + right.splitlines(keepends))] return explanation @@ -288,8 +264,8 @@ def _compare_eq_dict(left, right, verbose=False): explanation = [] common = set(left).intersection(set(right)) same = dict((k, left[k]) for k in common if left[k] == right[k]) - if same and not verbose: - explanation += [u('Omitting %s identical items, use -v to show') % + if same and verbose < 2: + explanation += [u('Omitting %s identical items, use -vv to show') % len(same)] elif same: explanation += [u('Common items:')] @@ -316,7 +292,7 @@ def _compare_eq_dict(left, right, verbose=False): def _notin_text(term, text, verbose=False): index = text.find(term) head = text[:index] - tail = text[index+len(term):] + tail = text[index + len(term):] correct_text = head + tail diff = _diff_text(correct_text, text, verbose) newdiff = [u('%s is contained here:') % py.io.saferepr(term, maxsize=42)] diff --git a/_pytest/cacheprovider.py b/_pytest/cacheprovider.py index 0657001f2d4dbc9..c537c14472b1ce9 100755 --- a/_pytest/cacheprovider.py +++ b/_pytest/cacheprovider.py @@ -1,20 +1,21 @@ """ merged implementation of the cache provider -the name cache was not choosen to ensure pluggy automatically +the name cache was not chosen to ensure pluggy automatically ignores the external pytest-cache """ - +from __future__ import absolute_import, division, print_function import py import pytest import json +import os from os.path import sep as _sep, altsep as _altsep class Cache(object): def __init__(self, config): self.config = config - self._cachedir = config.rootdir.join(".cache") + self._cachedir = Cache.cache_dir_from_config(config) self.trace = config.trace.root.get("cache") if config.getvalue("cacheclear"): self.trace("clearing cachedir") @@ -22,6 +23,16 @@ def __init__(self, config): self._cachedir.remove() self._cachedir.mkdir() + @staticmethod + def cache_dir_from_config(config): + cache_dir = config.getini("cache_dir") + cache_dir = os.path.expanduser(cache_dir) + cache_dir = os.path.expandvars(cache_dir) + if os.path.isabs(cache_dir): + return py.path.local(cache_dir) + else: + return config.rootdir.join(cache_dir) + def makedir(self, name): """ return a directory path object with the given name. If the directory does not yet exist, it will be created. You can use it @@ -89,31 +100,31 @@ def set(self, key, value): class LFPlugin: """ Plugin which implements the --lf (run last-failing) option """ + def __init__(self, config): self.config = config active_keys = 'lf', 'failedfirst' self.active = any(config.getvalue(key) for key in active_keys) - if self.active: - self.lastfailed = config.cache.get("cache/lastfailed", {}) - else: - self.lastfailed = {} + self.lastfailed = config.cache.get("cache/lastfailed", {}) + self._previously_failed_count = None - def pytest_report_header(self): + def pytest_report_collectionfinish(self): if self.active: - if not self.lastfailed: + if not self._previously_failed_count: mode = "run all (no recorded failures)" else: - mode = "rerun last %d failures%s" % ( - len(self.lastfailed), - " first" if self.config.getvalue("failedfirst") else "") + noun = 'failure' if self._previously_failed_count == 1 else 'failures' + suffix = " first" if self.config.getvalue("failedfirst") else "" + mode = "rerun previous {count} {noun}{suffix}".format( + count=self._previously_failed_count, suffix=suffix, noun=noun + ) return "run-last-failure: %s" % mode def pytest_runtest_logreport(self, report): - if report.failed and "xfail" not in report.keywords: + if (report.when == 'call' and report.passed) or report.skipped: + self.lastfailed.pop(report.nodeid, None) + elif report.failed: self.lastfailed[report.nodeid] = True - elif not report.failed: - if report.when == "call": - self.lastfailed.pop(report.nodeid, None) def pytest_collectreport(self, report): passed = report.outcome in ('passed', 'skipped') @@ -135,22 +146,24 @@ def pytest_collection_modifyitems(self, session, config, items): previously_failed.append(item) else: previously_passed.append(item) - if not previously_failed and previously_passed: + self._previously_failed_count = len(previously_failed) + if not previously_failed: # running a subset of all tests with recorded failures outside # of the set of tests currently executing - pass - elif self.config.getvalue("failedfirst"): - items[:] = previously_failed + previously_passed - else: + return + if self.config.getvalue("lf"): items[:] = previously_failed config.hook.pytest_deselected(items=previously_passed) + else: + items[:] = previously_failed + previously_passed def pytest_sessionfinish(self, session): config = self.config if config.getvalue("cacheshow") or hasattr(config, "slaveinput"): return - prev_failed = config.cache.get("cache/lastfailed", None) is not None - if (session.testscollected and prev_failed) or self.lastfailed: + + saved_lastfailed = config.cache.get("cache/lastfailed", {}) + if saved_lastfailed != self.lastfailed: config.cache.set("cache/lastfailed", self.lastfailed) @@ -171,6 +184,9 @@ def pytest_addoption(parser): group.addoption( '--cache-clear', action='store_true', dest="cacheclear", help="remove all cache contents at start of test run.") + parser.addini( + "cache_dir", default='.cache', + help="cache directory path.") def pytest_cmdline_main(config): @@ -179,7 +195,6 @@ def pytest_cmdline_main(config): return wrap_session(config, cacheshow) - @pytest.hookimpl(tryfirst=True) def pytest_configure(config): config.cache = Cache(config) @@ -219,12 +234,12 @@ def cacheshow(config, session): basedir = config.cache._cachedir vdir = basedir.join("v") tw.sep("-", "cache values") - for valpath in vdir.visit(lambda x: x.isfile()): + for valpath in sorted(vdir.visit(lambda x: x.isfile())): key = valpath.relto(vdir).replace(valpath.sep, "/") val = config.cache.get(key, dummy) if val is dummy: tw.line("%s contains unreadable content, " - "will be ignored" % key) + "will be ignored" % key) else: tw.line("%s contains:" % key) stream = py.io.TextIO() @@ -235,8 +250,8 @@ def cacheshow(config, session): ddir = basedir.join("d") if ddir.isdir() and ddir.listdir(): tw.sep("-", "cache directories") - for p in basedir.join("d").visit(): - #if p.check(dir=1): + for p in sorted(basedir.join("d").visit()): + # if p.check(dir=1): # print("%s/" % p.relto(basedir)) if p.isfile(): key = p.relto(basedir) diff --git a/_pytest/capture.py b/_pytest/capture.py index 3895a714aa03f00..f2ebe38c8c09028 100644 --- a/_pytest/capture.py +++ b/_pytest/capture.py @@ -2,17 +2,20 @@ per-test stdout/stderr capturing mechanism. """ -from __future__ import with_statement +from __future__ import absolute_import, division, print_function +import collections +import contextlib import sys import os +import io +from io import UnsupportedOperation from tempfile import TemporaryFile -import py +import six import pytest +from _pytest.compat import CaptureIO -from py.io import TextIO -unicode = py.builtin.text patchsysdict = {0: 'stdin', 1: 'stdout', 2: 'stderr'} @@ -31,14 +34,17 @@ def pytest_addoption(parser): @pytest.hookimpl(hookwrapper=True) def pytest_load_initial_conftests(early_config, parser, args): - _readline_workaround() ns = early_config.known_args_namespace + if ns.capture == "fd": + _py36_windowsconsoleio_workaround(sys.stdout) + _colorama_workaround() + _readline_workaround() pluginmanager = early_config.pluginmanager capman = CaptureManager(ns.capture) pluginmanager.register(capman, "capturemanager") # make sure that capturemanager is properly reset at final shutdown - early_config.add_cleanup(capman.reset_capturings) + early_config.add_cleanup(capman.stop_global_capturing) # make sure logging does not raise exceptions at the end def silence_logging_at_shutdown(): @@ -47,17 +53,30 @@ def silence_logging_at_shutdown(): early_config.add_cleanup(silence_logging_at_shutdown) # finally trigger conftest loading but while capturing (issue93) - capman.init_capturings() + capman.start_global_capturing() outcome = yield - out, err = capman.suspendcapture() + out, err = capman.suspend_global_capture() if outcome.excinfo is not None: sys.stdout.write(out) sys.stderr.write(err) class CaptureManager: + """ + Capture plugin, manages that the appropriate capture method is enabled/disabled during collection and each + test phase (setup, call, teardown). After each of those points, the captured output is obtained and + attached to the collection/runtest report. + + There are two levels of capture: + * global: which is enabled by default and can be suppressed by the ``-s`` option. This is always enabled/disabled + during collection and each test phase. + * fixture: when a test function or one of its fixture depend on the ``capsys`` or ``capfd`` fixtures. In this + case special handling is needed to ensure the fixtures take precedence over the global capture. + """ + def __init__(self, method): self._method = method + self._global_capturing = None def _getcapture(self, method): if method == "fd": @@ -69,23 +88,24 @@ def _getcapture(self, method): else: raise ValueError("unknown capturing method: %r" % method) - def init_capturings(self): - assert not hasattr(self, "_capturing") - self._capturing = self._getcapture(self._method) - self._capturing.start_capturing() + def start_global_capturing(self): + assert self._global_capturing is None + self._global_capturing = self._getcapture(self._method) + self._global_capturing.start_capturing() - def reset_capturings(self): - cap = self.__dict__.pop("_capturing", None) - if cap is not None: - cap.pop_outerr_to_orig() - cap.stop_capturing() + def stop_global_capturing(self): + if self._global_capturing is not None: + self._global_capturing.pop_outerr_to_orig() + self._global_capturing.stop_capturing() + self._global_capturing = None - def resumecapture(self): - self._capturing.resume_capturing() + def resume_global_capture(self): + self._global_capturing.resume_capturing() - def suspendcapture(self, in_=False): - self.deactivate_funcargs() - cap = getattr(self, "_capturing", None) + def suspend_global_capture(self, item=None, in_=False): + if item is not None: + self.deactivate_fixture(item) + cap = getattr(self, "_global_capturing", None) if cap is not None: try: outerr = cap.readouterr() @@ -93,23 +113,26 @@ def suspendcapture(self, in_=False): cap.suspend_capturing(in_=in_) return outerr - def activate_funcargs(self, pyfuncitem): - capfuncarg = pyfuncitem.__dict__.pop("_capfuncarg", None) - if capfuncarg is not None: - capfuncarg._start() - self._capfuncarg = capfuncarg + def activate_fixture(self, item): + """If the current item is using ``capsys`` or ``capfd``, activate them so they take precedence over + the global capture. + """ + fixture = getattr(item, "_capture_fixture", None) + if fixture is not None: + fixture._start() - def deactivate_funcargs(self): - capfuncarg = self.__dict__.pop("_capfuncarg", None) - if capfuncarg is not None: - capfuncarg.close() + def deactivate_fixture(self, item): + """Deactivates the ``capsys`` or ``capfd`` fixture of this item, if any.""" + fixture = getattr(item, "_capture_fixture", None) + if fixture is not None: + fixture.close() @pytest.hookimpl(hookwrapper=True) def pytest_make_collect_report(self, collector): if isinstance(collector, pytest.File): - self.resumecapture() + self.resume_global_capture() outcome = yield - out, err = self.suspendcapture() + out, err = self.suspend_global_capture() rep = outcome.get_result() if out: rep.sections.append(("Captured stdout", out)) @@ -120,72 +143,142 @@ def pytest_make_collect_report(self, collector): @pytest.hookimpl(hookwrapper=True) def pytest_runtest_setup(self, item): - self.resumecapture() + self.resume_global_capture() + # no need to activate a capture fixture because they activate themselves during creation; this + # only makes sense when a fixture uses a capture fixture, otherwise the capture fixture will + # be activated during pytest_runtest_call yield - self.suspendcapture_item(item, "setup") + self.suspend_capture_item(item, "setup") @pytest.hookimpl(hookwrapper=True) def pytest_runtest_call(self, item): - self.resumecapture() - self.activate_funcargs(item) + self.resume_global_capture() + # it is important to activate this fixture during the call phase so it overwrites the "global" + # capture + self.activate_fixture(item) yield - #self.deactivate_funcargs() called from suspendcapture() - self.suspendcapture_item(item, "call") + self.suspend_capture_item(item, "call") @pytest.hookimpl(hookwrapper=True) def pytest_runtest_teardown(self, item): - self.resumecapture() + self.resume_global_capture() + self.activate_fixture(item) yield - self.suspendcapture_item(item, "teardown") + self.suspend_capture_item(item, "teardown") @pytest.hookimpl(tryfirst=True) def pytest_keyboard_interrupt(self, excinfo): - self.reset_capturings() + self.stop_global_capturing() @pytest.hookimpl(tryfirst=True) def pytest_internalerror(self, excinfo): - self.reset_capturings() + self.stop_global_capturing() - def suspendcapture_item(self, item, when): - out, err = self.suspendcapture() + def suspend_capture_item(self, item, when, in_=False): + out, err = self.suspend_global_capture(item, in_=in_) item.add_report_section(when, "stdout", out) item.add_report_section(when, "stderr", err) -error_capsysfderror = "cannot use capsys and capfd at the same time" + +capture_fixtures = {'capfd', 'capfdbinary', 'capsys', 'capsysbinary'} + + +def _ensure_only_one_capture_fixture(request, name): + fixtures = set(request.fixturenames) & capture_fixtures - set((name,)) + if fixtures: + fixtures = sorted(fixtures) + fixtures = fixtures[0] if len(fixtures) == 1 else fixtures + raise request.raiseerror( + "cannot use {0} and {1} at the same time".format( + fixtures, name, + ), + ) @pytest.fixture def capsys(request): - """enables capturing of writes to sys.stdout/sys.stderr and makes + """Enable capturing of writes to sys.stdout/sys.stderr and make captured output available via ``capsys.readouterr()`` method calls - which return a ``(out, err)`` tuple. + which return a ``(out, err)`` tuple. ``out`` and ``err`` will be ``text`` + objects. """ - if "capfd" in request._funcargs: - raise request.raiseerror(error_capsysfderror) - request.node._capfuncarg = c = CaptureFixture(SysCapture) - return c + _ensure_only_one_capture_fixture(request, 'capsys') + with _install_capture_fixture_on_item(request, SysCapture) as fixture: + yield fixture + + +@pytest.fixture +def capsysbinary(request): + """Enable capturing of writes to sys.stdout/sys.stderr and make + captured output available via ``capsys.readouterr()`` method calls + which return a ``(out, err)`` tuple. ``out`` and ``err`` will be ``bytes`` + objects. + """ + _ensure_only_one_capture_fixture(request, 'capsysbinary') + # Currently, the implementation uses the python3 specific `.buffer` + # property of CaptureIO. + if sys.version_info < (3,): + raise request.raiseerror('capsysbinary is only supported on python 3') + with _install_capture_fixture_on_item(request, SysCaptureBinary) as fixture: + yield fixture + @pytest.fixture def capfd(request): - """enables capturing of writes to file descriptors 1 and 2 and makes + """Enable capturing of writes to file descriptors 1 and 2 and make captured output available via ``capfd.readouterr()`` method calls - which return a ``(out, err)`` tuple. + which return a ``(out, err)`` tuple. ``out`` and ``err`` will be ``text`` + objects. """ - if "capsys" in request._funcargs: - request.raiseerror(error_capsysfderror) + _ensure_only_one_capture_fixture(request, 'capfd') if not hasattr(os, 'dup'): - pytest.skip("capfd funcarg needs os.dup") - request.node._capfuncarg = c = CaptureFixture(FDCapture) - return c + pytest.skip("capfd fixture needs os.dup function which is not available in this system") + with _install_capture_fixture_on_item(request, FDCapture) as fixture: + yield fixture + + +@pytest.fixture +def capfdbinary(request): + """Enable capturing of write to file descriptors 1 and 2 and make + captured output available via ``capfdbinary.readouterr`` method calls + which return a ``(out, err)`` tuple. ``out`` and ``err`` will be + ``bytes`` objects. + """ + _ensure_only_one_capture_fixture(request, 'capfdbinary') + if not hasattr(os, 'dup'): + pytest.skip("capfdbinary fixture needs os.dup function which is not available in this system") + with _install_capture_fixture_on_item(request, FDCaptureBinary) as fixture: + yield fixture + + +@contextlib.contextmanager +def _install_capture_fixture_on_item(request, capture_class): + """ + Context manager which creates a ``CaptureFixture`` instance and "installs" it on + the item/node of the given request. Used by ``capsys`` and ``capfd``. + + The CaptureFixture is added as attribute of the item because it needs to accessed + by ``CaptureManager`` during its ``pytest_runtest_*`` hooks. + """ + request.node._capture_fixture = fixture = CaptureFixture(capture_class, request) + capmanager = request.config.pluginmanager.getplugin('capturemanager') + # need to active this fixture right away in case it is being used by another fixture (setup phase) + # if this fixture is being used only by a test function (call phase), then we wouldn't need this + # activation, but it doesn't hurt + capmanager.activate_fixture(request.node) + yield fixture + fixture.close() + del request.node._capture_fixture class CaptureFixture: - def __init__(self, captureclass): + def __init__(self, captureclass, request): self.captureclass = captureclass + self.request = request def _start(self): self._capture = MultiCapture(out=True, err=True, in_=False, - Capture=self.captureclass) + Capture=self.captureclass) self._capture.start_capturing() def close(self): @@ -200,6 +293,17 @@ def readouterr(self): except AttributeError: return self._outerr + @contextlib.contextmanager + def disabled(self): + self._capture.suspend_capturing() + capmanager = self.request.config.pluginmanager.getplugin('capturemanager') + capmanager.suspend_global_capture(item=None, in_=False) + try: + yield + finally: + capmanager.resume_global_capture() + self._capture.resume_capturing() + def safe_text_dupfile(f, mode, default_encoding="UTF8"): """ return a open text file object that's a duplicate of f on the @@ -222,12 +326,13 @@ def safe_text_dupfile(f, mode, default_encoding="UTF8"): class EncodedFile(object): errors = "strict" # possibly needed by py3 code (issue555) + def __init__(self, buffer, encoding): self.buffer = buffer self.encoding = encoding def write(self, obj): - if isinstance(obj, unicode): + if isinstance(obj, six.text_type): obj = obj.encode(self.encoding, "replace") self.buffer.write(obj) @@ -235,10 +340,18 @@ def writelines(self, linelist): data = ''.join(linelist) self.write(data) + @property + def name(self): + """Ensure that file.name is a string.""" + return repr(self.buffer) + def __getattr__(self, name): return getattr(object.__getattribute__(self, "buffer"), name) +CaptureResult = collections.namedtuple("CaptureResult", ["out", "err"]) + + class MultiCapture(object): out = err = in_ = None @@ -299,14 +412,19 @@ def stop_capturing(self): def readouterr(self): """ return snapshot unicode value of stdout/stderr capturings. """ - return (self.out.snap() if self.out is not None else "", - self.err.snap() if self.err is not None else "") + return CaptureResult(self.out.snap() if self.out is not None else "", + self.err.snap() if self.err is not None else "") + class NoCapture: __init__ = start = done = suspend = resume = lambda *args: None -class FDCapture: - """ Capture IO to/from a given os-level filedescriptor. """ + +class FDCaptureBinary: + """Capture IO to/from a given os-level filedescriptor. + + snap() produces `bytes` + """ def __init__(self, targetfd, tmpfile=None): self.targetfd = targetfd @@ -345,17 +463,11 @@ def start(self): self.syscapture.start() def snap(self): - f = self.tmpfile - f.seek(0) - res = f.read() - if res: - enc = getattr(f, "encoding", None) - if enc and isinstance(res, bytes): - res = py.builtin._totext(res, enc, "replace") - f.truncate(0) - f.seek(0) - return res - return '' + self.tmpfile.seek(0) + res = self.tmpfile.read() + self.tmpfile.seek(0) + self.tmpfile.truncate() + return res def done(self): """ stop capturing, restore streams, return original capture file, @@ -376,11 +488,24 @@ def resume(self): def writeorg(self, data): """ write to original file descriptor. """ - if py.builtin._istext(data): - data = data.encode("utf8") # XXX use encoding of original stream + if isinstance(data, six.text_type): + data = data.encode("utf8") # XXX use encoding of original stream os.write(self.targetfd_save, data) +class FDCapture(FDCaptureBinary): + """Capture IO to/from a given os-level filedescriptor. + + snap() produces text + """ + def snap(self): + res = FDCaptureBinary.snap(self) + enc = getattr(self.tmpfile, "encoding", None) + if enc and isinstance(res, bytes): + res = six.text_type(res, enc, "replace") + return res + + class SysCapture: def __init__(self, fd, tmpfile=None): name = patchsysdict[fd] @@ -390,17 +515,16 @@ def __init__(self, fd, tmpfile=None): if name == "stdin": tmpfile = DontReadFromInput() else: - tmpfile = TextIO() + tmpfile = CaptureIO() self.tmpfile = tmpfile def start(self): setattr(sys, self.name, self.tmpfile) def snap(self): - f = self.tmpfile - res = f.getvalue() - f.truncate(0) - f.seek(0) + res = self.tmpfile.getvalue() + self.tmpfile.seek(0) + self.tmpfile.truncate() return res def done(self): @@ -419,6 +543,14 @@ def writeorg(self, data): self._old.flush() +class SysCaptureBinary(SysCapture): + def snap(self): + res = self.tmpfile.buffer.getvalue() + self.tmpfile.seek(0) + self.tmpfile.truncate() + return res + + class DontReadFromInput: """Temporary stub class. Ideally when stdin is accessed, the capturing should be turned off, with possibly all data captured @@ -436,7 +568,8 @@ def read(self, *args): __iter__ = read def fileno(self): - raise ValueError("redirected Stdin is pseudofile, has no fileno()") + raise UnsupportedOperation("redirected stdin is pseudofile, " + "has no fileno()") def isatty(self): return False @@ -444,6 +577,31 @@ def isatty(self): def close(self): pass + @property + def buffer(self): + if sys.version_info >= (3, 0): + return self + else: + raise AttributeError('redirected stdin has no attribute buffer') + + +def _colorama_workaround(): + """ + Ensure colorama is imported so that it attaches to the correct stdio + handles on Windows. + + colorama uses the terminal on import time. So if something does the + first import of colorama while I/O capture is active, colorama will + fail in various ways. + """ + + if not sys.platform.startswith('win32'): + return + try: + import colorama # noqa + except ImportError: + pass + def _readline_workaround(): """ @@ -452,7 +610,7 @@ def _readline_workaround(): Pdb uses readline support where available--when not running from the Python prompt, the readline module is not imported until running the pdb REPL. If - running py.test with the --pdb option this means the readline module is not + running pytest with the --pdb option this means the readline module is not imported until after I/O capture has been started. This is a problem for pyreadline, which is often used to implement readline @@ -470,3 +628,56 @@ def _readline_workaround(): import readline # noqa except ImportError: pass + + +def _py36_windowsconsoleio_workaround(stream): + """ + Python 3.6 implemented unicode console handling for Windows. This works + by reading/writing to the raw console handle using + ``{Read,Write}ConsoleW``. + + The problem is that we are going to ``dup2`` over the stdio file + descriptors when doing ``FDCapture`` and this will ``CloseHandle`` the + handles used by Python to write to the console. Though there is still some + weirdness and the console handle seems to only be closed randomly and not + on the first call to ``CloseHandle``, or maybe it gets reopened with the + same handle value when we suspend capturing. + + The workaround in this case will reopen stdio with a different fd which + also means a different handle by replicating the logic in + "Py_lifecycle.c:initstdio/create_stdio". + + :param stream: in practice ``sys.stdout`` or ``sys.stderr``, but given + here as parameter for unittesting purposes. + + See https://github.com/pytest-dev/py/issues/103 + """ + if not sys.platform.startswith('win32') or sys.version_info[:2] < (3, 6): + return + + # bail out if ``stream`` doesn't seem like a proper ``io`` stream (#2666) + if not hasattr(stream, 'buffer'): + return + + buffered = hasattr(stream.buffer, 'raw') + raw_stdout = stream.buffer.raw if buffered else stream.buffer + + if not isinstance(raw_stdout, io._WindowsConsoleIO): + return + + def _reopen_stdio(f, mode): + if not buffered and mode[0] == 'w': + buffering = 0 + else: + buffering = -1 + + return io.TextIOWrapper( + open(os.dup(f.fileno()), mode, buffering), + f.encoding, + f.errors, + f.newlines, + f.line_buffering) + + sys.__stdin__ = sys.stdin = _reopen_stdio(sys.stdin, 'rb') + sys.__stdout__ = sys.stdout = _reopen_stdio(sys.stdout, 'wb') + sys.__stderr__ = sys.stderr = _reopen_stdio(sys.stderr, 'wb') diff --git a/_pytest/compat.py b/_pytest/compat.py new file mode 100644 index 000000000000000..7560fbec3975557 --- /dev/null +++ b/_pytest/compat.py @@ -0,0 +1,322 @@ +""" +python version compatibility code +""" +from __future__ import absolute_import, division, print_function + +import codecs +import functools +import inspect +import re +import sys + +import py + +import _pytest +from _pytest.outcomes import TEST_OUTCOME + +try: + import enum +except ImportError: # pragma: no cover + # Only available in Python 3.4+ or as a backport + enum = None + + +_PY3 = sys.version_info > (3, 0) +_PY2 = not _PY3 + + +if _PY3: + from inspect import signature, Parameter as Parameter +else: + from funcsigs import signature, Parameter as Parameter + + +NoneType = type(None) +NOTSET = object() + +PY35 = sys.version_info[:2] >= (3, 5) +PY36 = sys.version_info[:2] >= (3, 6) +MODULE_NOT_FOUND_ERROR = 'ModuleNotFoundError' if PY36 else 'ImportError' + + +def _format_args(func): + return str(signature(func)) + + +isfunction = inspect.isfunction +isclass = inspect.isclass +# used to work around a python2 exception info leak +exc_clear = getattr(sys, 'exc_clear', lambda: None) +# The type of re.compile objects is not exposed in Python. +REGEX_TYPE = type(re.compile('')) + + +def is_generator(func): + genfunc = inspect.isgeneratorfunction(func) + return genfunc and not iscoroutinefunction(func) + + +def iscoroutinefunction(func): + """Return True if func is a decorated coroutine function. + + Note: copied and modified from Python 3.5's builtin couroutines.py to avoid import asyncio directly, + which in turns also initializes the "logging" module as side-effect (see issue #8). + """ + return (getattr(func, '_is_coroutine', False) or + (hasattr(inspect, 'iscoroutinefunction') and inspect.iscoroutinefunction(func))) + + +def getlocation(function, curdir): + fn = py.path.local(inspect.getfile(function)) + lineno = py.builtin._getcode(function).co_firstlineno + if fn.relto(curdir): + fn = fn.relto(curdir) + return "%s:%d" % (fn, lineno + 1) + + +def num_mock_patch_args(function): + """ return number of arguments used up by mock arguments (if any) """ + patchings = getattr(function, "patchings", None) + if not patchings: + return 0 + mock = sys.modules.get("mock", sys.modules.get("unittest.mock", None)) + if mock is not None: + return len([p for p in patchings + if not p.attribute_name and p.new is mock.DEFAULT]) + return len(patchings) + + +def getfuncargnames(function, is_method=False, cls=None): + """Returns the names of a function's mandatory arguments. + + This should return the names of all function arguments that: + * Aren't bound to an instance or type as in instance or class methods. + * Don't have default values. + * Aren't bound with functools.partial. + * Aren't replaced with mocks. + + The is_method and cls arguments indicate that the function should + be treated as a bound method even though it's not unless, only in + the case of cls, the function is a static method. + + @RonnyPfannschmidt: This function should be refactored when we + revisit fixtures. The fixture mechanism should ask the node for + the fixture names, and not try to obtain directly from the + function object well after collection has occurred. + + """ + # The parameters attribute of a Signature object contains an + # ordered mapping of parameter names to Parameter instances. This + # creates a tuple of the names of the parameters that don't have + # defaults. + arg_names = tuple(p.name for p in signature(function).parameters.values() + if (p.kind is Parameter.POSITIONAL_OR_KEYWORD or + p.kind is Parameter.KEYWORD_ONLY) and + p.default is Parameter.empty) + # If this function should be treated as a bound method even though + # it's passed as an unbound method or function, remove the first + # parameter name. + if (is_method or + (cls and not isinstance(cls.__dict__.get(function.__name__, None), + staticmethod))): + arg_names = arg_names[1:] + # Remove any names that will be replaced with mocks. + if hasattr(function, "__wrapped__"): + arg_names = arg_names[num_mock_patch_args(function):] + return arg_names + + +if _PY3: + STRING_TYPES = bytes, str + UNICODE_TYPES = str, + + if PY35: + def _bytes_to_ascii(val): + return val.decode('ascii', 'backslashreplace') + else: + def _bytes_to_ascii(val): + if val: + # source: http://goo.gl/bGsnwC + encoded_bytes, _ = codecs.escape_encode(val) + return encoded_bytes.decode('ascii') + else: + # empty bytes crashes codecs.escape_encode (#1087) + return '' + + def ascii_escaped(val): + """If val is pure ascii, returns it as a str(). Otherwise, escapes + bytes objects into a sequence of escaped bytes: + + b'\xc3\xb4\xc5\xd6' -> u'\\xc3\\xb4\\xc5\\xd6' + + and escapes unicode objects into a sequence of escaped unicode + ids, e.g.: + + '4\\nV\\U00043efa\\x0eMXWB\\x1e\\u3028\\u15fd\\xcd\\U0007d944' + + note: + the obvious "v.decode('unicode-escape')" will return + valid utf-8 unicode if it finds them in bytes, but we + want to return escaped bytes for any byte, even if they match + a utf-8 string. + + """ + if isinstance(val, bytes): + return _bytes_to_ascii(val) + else: + return val.encode('unicode_escape').decode('ascii') +else: + STRING_TYPES = bytes, str, unicode + UNICODE_TYPES = unicode, + + def ascii_escaped(val): + """In py2 bytes and str are the same type, so return if it's a bytes + object, return it unchanged if it is a full ascii string, + otherwise escape it into its binary form. + + If it's a unicode string, change the unicode characters into + unicode escapes. + + """ + if isinstance(val, bytes): + try: + return val.encode('ascii') + except UnicodeDecodeError: + return val.encode('string-escape') + else: + return val.encode('unicode-escape') + + +def get_real_func(obj): + """ gets the real function object of the (possibly) wrapped object by + functools.wraps or functools.partial. + """ + start_obj = obj + for i in range(100): + new_obj = getattr(obj, '__wrapped__', None) + if new_obj is None: + break + obj = new_obj + else: + raise ValueError( + ("could not find real function of {start}" + "\nstopped at {current}").format( + start=py.io.saferepr(start_obj), + current=py.io.saferepr(obj))) + if isinstance(obj, functools.partial): + obj = obj.func + return obj + + +def getfslineno(obj): + # xxx let decorators etc specify a sane ordering + obj = get_real_func(obj) + if hasattr(obj, 'place_as'): + obj = obj.place_as + fslineno = _pytest._code.getfslineno(obj) + assert isinstance(fslineno[1], int), obj + return fslineno + + +def getimfunc(func): + try: + return func.__func__ + except AttributeError: + return func + + +def safe_getattr(object, name, default): + """ Like getattr but return default upon any Exception or any OutcomeException. + + Attribute access can potentially fail for 'evil' Python objects. + See issue #214. + It catches OutcomeException because of #2490 (issue #580), new outcomes are derived from BaseException + instead of Exception (for more details check #2707) + """ + try: + return getattr(object, name, default) + except TEST_OUTCOME: + return default + + +def _is_unittest_unexpected_success_a_failure(): + """Return if the test suite should fail if a @expectedFailure unittest test PASSES. + + From https://docs.python.org/3/library/unittest.html?highlight=unittest#unittest.TestResult.wasSuccessful: + Changed in version 3.4: Returns False if there were any + unexpectedSuccesses from tests marked with the expectedFailure() decorator. + """ + return sys.version_info >= (3, 4) + + +if _PY3: + def safe_str(v): + """returns v as string""" + return str(v) +else: + def safe_str(v): + """returns v as string, converting to ascii if necessary""" + try: + return str(v) + except UnicodeError: + if not isinstance(v, unicode): + v = unicode(v) + errors = 'replace' + return v.encode('utf-8', errors) + + +COLLECT_FAKEMODULE_ATTRIBUTES = ( + 'Collector', + 'Module', + 'Generator', + 'Function', + 'Instance', + 'Session', + 'Item', + 'Class', + 'File', + '_fillfuncargs', +) + + +def _setup_collect_fakemodule(): + from types import ModuleType + import pytest + pytest.collect = ModuleType('pytest.collect') + pytest.collect.__all__ = [] # used for setns + for attr in COLLECT_FAKEMODULE_ATTRIBUTES: + setattr(pytest.collect, attr, getattr(pytest, attr)) + + +if _PY2: + # Without this the test_dupfile_on_textio will fail, otherwise CaptureIO could directly inherit from StringIO. + from py.io import TextIO + + class CaptureIO(TextIO): + + @property + def encoding(self): + return getattr(self, '_encoding', 'UTF-8') + +else: + import io + + class CaptureIO(io.TextIOWrapper): + def __init__(self): + super(CaptureIO, self).__init__( + io.BytesIO(), + encoding='UTF-8', newline='', write_through=True, + ) + + def getvalue(self): + return self.buffer.getvalue().decode('UTF-8') + + +class FuncargnamesCompatAttr(object): + """ helper class so that Metafunc, Function and FixtureRequest + don't need to each define the "funcargnames" compatibility attribute. + """ + @property + def funcargnames(self): + """ alias attribute for ``fixturenames`` for pre-2.3 compatibility""" + return self.fixturenames diff --git a/_pytest/config.py b/_pytest/config.py index fb7b1774f6853f8..499c8079d415986 100644 --- a/_pytest/config.py +++ b/_pytest/config.py @@ -1,16 +1,21 @@ """ command line options, ini-file and conftest.py processing. """ +from __future__ import absolute_import, division, print_function import argparse import shlex import traceback import types import warnings +import six import py # DON't import pytest here because it causes import cycle troubles -import sys, os +import sys +import os import _pytest._code import _pytest.hookspec # the extension point definitions -from _pytest._pluggy import PluginManager, HookimplMarker, HookspecMarker +import _pytest.assertion +from pluggy import PluginManager, HookimplMarker, HookspecMarker +from _pytest.compat import safe_str hookimpl = HookimplMarker("pytest") hookspec = HookspecMarker("pytest") @@ -25,6 +30,12 @@ def __init__(self, path, excinfo): self.path = path self.excinfo = excinfo + def __str__(self): + etype, evalue, etb = self.excinfo + formatted = traceback.format_tb(etb) + # The level of the tracebacks we want to print is hand crafted :( + return repr(evalue) + '\n' + ''.join(formatted[2:]) + def main(args=None, plugins=None): """ return exit code, after performing an in-process test run. @@ -45,39 +56,63 @@ def main(args=None, plugins=None): return 4 else: try: - config.pluginmanager.check_pending() return config.hook.pytest_cmdline_main(config=config) finally: config._ensure_unconfigure() except UsageError as e: for msg in e.args: - sys.stderr.write("ERROR: %s\n" %(msg,)) + sys.stderr.write("ERROR: %s\n" % (msg,)) return 4 + class cmdline: # compatibility namespace main = staticmethod(main) + class UsageError(Exception): """ error in pytest usage or invocation""" -_preinit = [] + +class PrintHelp(Exception): + """Raised when pytest should print it's help to skip the rest of the + argument parsing and validation.""" + pass + + +def filename_arg(path, optname): + """ Argparse type validator for filename arguments. + + :path: path of filename + :optname: name of the option + """ + if os.path.isdir(path): + raise UsageError("{0} must be a filename, given: {1}".format(optname, path)) + return path + + +def directory_arg(path, optname): + """Argparse type validator for directory arguments. + + :path: path of directory + :optname: name of the option + """ + if not os.path.isdir(path): + raise UsageError("{0} must be a directory, given: {1}".format(optname, path)) + return path + default_plugins = ( - "mark main terminal runner python pdb unittest capture skipping " - "tmpdir monkeypatch recwarn pastebin helpconfig nose assertion genscript " - "junitxml resultlog doctest cacheprovider").split() + "mark main terminal runner python fixtures debugging unittest capture skipping " + "tmpdir monkeypatch recwarn pastebin helpconfig nose assertion " + "junitxml resultlog doctest cacheprovider freeze_support " + "setuponly setupplan warnings logging").split() + builtin_plugins = set(default_plugins) builtin_plugins.add("pytester") -def _preloadplugins(): - assert not _preinit - _preinit.append(get_config()) - def get_config(): - if _preinit: - return _preinit.pop(0) # subsequent calls to main will create a fresh instance pluginmanager = PytestPluginManager() config = Config(pluginmanager) @@ -85,6 +120,7 @@ def get_config(): pluginmanager.import_plugin(spec) return config + def get_plugin_manager(): """ Obtain a new instance of the @@ -96,7 +132,9 @@ def get_plugin_manager(): """ return get_config().pluginmanager + def _prepareconfig(args=None, plugins=None): + warning = None if args is None: args = sys.argv[1:] elif isinstance(args, py.path.local): @@ -104,18 +142,22 @@ def _prepareconfig(args=None, plugins=None): elif not isinstance(args, (tuple, list)): if not isinstance(args, str): raise ValueError("not a string or argument list: %r" % (args,)) - args = shlex.split(args) + args = shlex.split(args, posix=sys.platform != "win32") + from _pytest import deprecated + warning = deprecated.MAIN_STR_ARGS config = get_config() pluginmanager = config.pluginmanager try: if plugins: for plugin in plugins: - if isinstance(plugin, py.builtin._basestring): + if isinstance(plugin, six.string_types): pluginmanager.consider_pluginarg(plugin) else: pluginmanager.register(plugin) + if warning: + config.warn('C1', warning) return pluginmanager.hook.pytest_cmdline_parse( - pluginmanager=pluginmanager, args=args) + pluginmanager=pluginmanager, args=args) except BaseException: config._ensure_unconfigure() raise @@ -123,13 +165,14 @@ def _prepareconfig(args=None, plugins=None): class PytestPluginManager(PluginManager): """ - Overwrites :py:class:`pluggy.PluginManager` to add pytest-specific + Overwrites :py:class:`pluggy.PluginManager ` to add pytest-specific functionality: * loading plugins from the command line, ``PYTEST_PLUGIN`` env variable and ``pytest_plugins`` global variables found in plugins being loaded; * ``conftest.py`` loading during start-up; """ + def __init__(self): super(PytestPluginManager, self).__init__("pytest", implprefix="pytest_") self._conftest_plugins = set() @@ -139,6 +182,7 @@ def __init__(self): self._conftestpath2mod = {} self._confcutdir = None self._noconftest = False + self._duplicatepaths = set() self.add_hookspecs(_pytest.hookspec) self.register(self) @@ -152,11 +196,15 @@ def __init__(self): self.trace.root.setwriter(err.write) self.enable_tracing() + # Config._consider_importhook will set a real object if required. + self.rewrite_hook = _pytest.assertion.DummyRewriteHook() + def addhooks(self, module_or_class): """ .. deprecated:: 2.8 - Use :py:meth:`pluggy.PluginManager.add_hookspecs` instead. + Use :py:meth:`pluggy.PluginManager.add_hookspecs ` + instead. """ warning = dict(code="I2", fslocation=_pytest._code.getfslineno(sys._getframe(1)), @@ -185,7 +233,7 @@ def parse_hookimpl_opts(self, plugin, name): def parse_hookspec_opts(self, module_or_class, name): opts = super(PytestPluginManager, self).parse_hookspec_opts( - module_or_class, name) + module_or_class, name) if opts is None: method = getattr(module_or_class, name) if name.startswith("pytest_"): @@ -193,22 +241,18 @@ def parse_hookspec_opts(self, module_or_class, name): "historic": hasattr(method, "historic")} return opts - def _verify_hook(self, hook, hookmethod): - super(PytestPluginManager, self)._verify_hook(hook, hookmethod) - if "__multicall__" in hookmethod.argnames: - fslineno = _pytest._code.getfslineno(hookmethod.function) - warning = dict(code="I1", - fslocation=fslineno, - nodeid=None, - message="%r hook uses deprecated __multicall__ " - "argument" % (hook.name)) - self._warn(warning) - def register(self, plugin, name=None): + if name == 'pytest_catchlog': + self._warn('pytest-catchlog plugin has been merged into the core, ' + 'please remove it from your requirements.') + return ret = super(PytestPluginManager, self).register(plugin, name) if ret: self.hook.pytest_plugin_registered.call_historic( - kwargs=dict(plugin=plugin, manager=self)) + kwargs=dict(plugin=plugin, manager=self)) + + if isinstance(plugin, types.ModuleType): + self.consider_module(plugin) return ret def getplugin(self, name): @@ -223,11 +267,11 @@ def pytest_configure(self, config): # XXX now that the pluginmanager exposes hookimpl(tryfirst...) # we should remove tryfirst/trylast as markers config.addinivalue_line("markers", - "tryfirst: mark a hook implementation function such that the " - "plugin machinery will try to call it first/as early as possible.") + "tryfirst: mark a hook implementation function such that the " + "plugin machinery will try to call it first/as early as possible.") config.addinivalue_line("markers", - "trylast: mark a hook implementation function such that the " - "plugin machinery will try to call it last/as late as possible.") + "trylast: mark a hook implementation function such that the " + "plugin machinery will try to call it last/as late as possible.") def _warn(self, message): kwargs = message if isinstance(message, dict) else { @@ -251,7 +295,7 @@ def _set_initial_conftests(self, namespace): """ current = py.path.local() self._confcutdir = current.join(namespace.confcutdir, abs=True) \ - if namespace.confcutdir else None + if namespace.confcutdir else None self._noconftest = namespace.noconftest testpaths = namespace.file_or_dir foundanchor = False @@ -262,7 +306,7 @@ def _set_initial_conftests(self, namespace): if i != -1: path = path[:i] anchor = current.join(path, abs=1) - if exists(anchor): # we found some file object + if exists(anchor): # we found some file object self._try_load_conftest(anchor) foundanchor = True if not foundanchor: @@ -329,7 +373,7 @@ def _importconftest(self, conftestpath): if path and path.relto(dirpath) or path == dirpath: assert mod not in mods mods.append(mod) - self.trace("loaded conftestmodule %r" %(mod)) + self.trace("loaded conftestmodule %r" % (mod)) self.consider_conftest(mod) return mod @@ -339,7 +383,7 @@ def _importconftest(self, conftestpath): # def consider_preparse(self, args): - for opt1,opt2 in zip(args, args[1:]): + for opt1, opt2 in zip(args, args[1:]): if opt1 == "-p": self.consider_pluginarg(opt2) @@ -353,52 +397,68 @@ def consider_pluginarg(self, arg): self.import_plugin(arg) def consider_conftest(self, conftestmodule): - if self.register(conftestmodule, name=conftestmodule.__file__): - self.consider_module(conftestmodule) + self.register(conftestmodule, name=conftestmodule.__file__) def consider_env(self): self._import_plugin_specs(os.environ.get("PYTEST_PLUGINS")) def consider_module(self, mod): - self._import_plugin_specs(getattr(mod, "pytest_plugins", None)) + self._import_plugin_specs(getattr(mod, 'pytest_plugins', [])) def _import_plugin_specs(self, spec): - if spec: - if isinstance(spec, str): - spec = spec.split(",") - for import_spec in spec: - self.import_plugin(import_spec) + plugins = _get_plugin_specs_as_list(spec) + for import_spec in plugins: + self.import_plugin(import_spec) def import_plugin(self, modname): # most often modname refers to builtin modules, e.g. "pytester", # "terminal" or "capture". Those plugins are registered under their # basename for historic purposes but must be imported with the # _pytest prefix. - assert isinstance(modname, str) - if self.get_plugin(modname) is not None: + assert isinstance(modname, (six.text_type, str)), "module name as text required, got %r" % modname + modname = str(modname) + if self.is_blocked(modname) or self.get_plugin(modname) is not None: return if modname in builtin_plugins: importspec = "_pytest." + modname else: importspec = modname + self.rewrite_hook.mark_rewrite(importspec) try: __import__(importspec) except ImportError as e: - new_exc = ImportError('Error importing plugin "%s": %s' % (modname, e)) - # copy over name and path attributes - for attr in ('name', 'path'): - if hasattr(e, attr): - setattr(new_exc, attr, getattr(e, attr)) - raise new_exc + new_exc_type = ImportError + new_exc_message = 'Error importing plugin "%s": %s' % (modname, safe_str(e.args[0])) + new_exc = new_exc_type(new_exc_message) + + six.reraise(new_exc_type, new_exc, sys.exc_info()[2]) + except Exception as e: import pytest if not hasattr(pytest, 'skip') or not isinstance(e, pytest.skip.Exception): raise - self._warn("skipped plugin %r: %s" %((modname, e.msg))) + self._warn("skipped plugin %r: %s" % ((modname, e.msg))) else: mod = sys.modules[importspec] self.register(mod, modname) - self.consider_module(mod) + + +def _get_plugin_specs_as_list(specs): + """ + Parses a list of "plugin specs" and returns a list of plugin names. + + Plugin specs can be given as a list of strings separated by "," or already as a list/tuple in + which case it is returned as a list. Specs can also be `None` in which case an + empty list is returned. + """ + if specs is not None: + if isinstance(specs, str): + specs = specs.split(',') if specs else [] + if not isinstance(specs, (list, tuple)): + raise UsageError("Plugin specs must be a ','-separated string or a " + "list/tuple of strings for plugin names. Given: %r" % specs) + return list(specs) + return [] class Parser: @@ -442,7 +502,7 @@ def getgroup(self, name, description="", after=None): for i, grp in enumerate(self._groups): if grp.name == after: break - self._groups.insert(i+1, group) + self._groups.insert(i + 1, group) return group def addoption(self, *opts, **attrs): @@ -480,7 +540,7 @@ def _getparser(self): a = option.attrs() arggroup.add_argument(*n, **a) # bash like autocompletion for dirs (appending '/') - optparser.add_argument(FILE_OR_DIR, nargs='*').completer=filescompleter + optparser.add_argument(FILE_OR_DIR, nargs='*').completer = filescompleter return optparser def parse_setoption(self, args, option, namespace=None): @@ -537,13 +597,18 @@ def __str__(self): class Argument: - """class that mimics the necessary behaviour of optparse.Option """ + """class that mimics the necessary behaviour of optparse.Option + + its currently a least effort implementation + and ignoring choices and integer prefixes + https://docs.python.org/3/library/optparse.html#optparse-standard-option-types + """ _typ_map = { 'int': int, 'string': str, - } - # enable after some grace period for plugin writers - TYPE_WARN = False + 'float': float, + 'complex': complex, + } def __init__(self, *names, **attrs): """store parms in private vars for use in add_argument""" @@ -551,44 +616,37 @@ def __init__(self, *names, **attrs): self._short_opts = [] self._long_opts = [] self.dest = attrs.get('dest') - if self.TYPE_WARN: - try: - help = attrs['help'] - if '%default' in help: - warnings.warn( - 'pytest now uses argparse. "%default" should be' - ' changed to "%(default)s" ', - FutureWarning, - stacklevel=3) - except KeyError: - pass + if '%default' in (attrs.get('help') or ''): + warnings.warn( + 'pytest now uses argparse. "%default" should be' + ' changed to "%(default)s" ', + DeprecationWarning, + stacklevel=3) try: typ = attrs['type'] except KeyError: pass else: # this might raise a keyerror as well, don't want to catch that - if isinstance(typ, py.builtin._basestring): + if isinstance(typ, six.string_types): if typ == 'choice': - if self.TYPE_WARN: - warnings.warn( - 'type argument to addoption() is a string %r.' - ' For parsearg this is optional and when supplied ' - ' should be a type.' - ' (options: %s)' % (typ, names), - FutureWarning, - stacklevel=3) + warnings.warn( + 'type argument to addoption() is a string %r.' + ' For parsearg this is optional and when supplied' + ' should be a type.' + ' (options: %s)' % (typ, names), + DeprecationWarning, + stacklevel=3) # argparse expects a type here take it from # the type of the first element attrs['type'] = type(attrs['choices'][0]) else: - if self.TYPE_WARN: - warnings.warn( - 'type argument to addoption() is a string %r.' - ' For parsearg this should be a type.' - ' (options: %s)' % (typ, names), - FutureWarning, - stacklevel=3) + warnings.warn( + 'type argument to addoption() is a string %r.' + ' For parsearg this should be a type.' + ' (options: %s)' % (typ, names), + DeprecationWarning, + stacklevel=3) attrs['type'] = Argument._typ_map[typ] # used in test_parseopt -> test_parse_defaultgetter self.type = attrs['type'] @@ -626,7 +684,7 @@ def attrs(self): if self._attrs.get('help'): a = self._attrs['help'] a = a.replace('%default', '%(default)s') - #a = a.replace('%prog', '%(prog)s') + # a = a.replace('%prog', '%(prog)s') self._attrs['help'] = a return self._attrs @@ -655,20 +713,17 @@ def _set_opt_strings(self, opts): self._long_opts.append(opt) def __repr__(self): - retval = 'Argument(' + args = [] if self._short_opts: - retval += '_short_opts: ' + repr(self._short_opts) + ', ' + args += ['_short_opts: ' + repr(self._short_opts)] if self._long_opts: - retval += '_long_opts: ' + repr(self._long_opts) + ', ' - retval += 'dest: ' + repr(self.dest) + ', ' + args += ['_long_opts: ' + repr(self._long_opts)] + args += ['dest: ' + repr(self.dest)] if hasattr(self, 'type'): - retval += 'type: ' + repr(self.type) + ', ' + args += ['type: ' + repr(self.type)] if hasattr(self, 'default'): - retval += 'default: ' + repr(self.default) + ', ' - if retval[-2:] == ', ': # always long enough to test ("Argument(" ) - retval = retval[:-2] - retval += ')' - return retval + args += ['default: ' + repr(self.default)] + return 'Argument({0})'.format(', '.join(args)) class OptionGroup: @@ -686,6 +741,10 @@ def addoption(self, *optnames, **attrs): results in help showing '--two-words' only, but --twowords gets accepted **and** the automatic destination is in args.twowords """ + conflict = set(optnames).intersection( + name for opt in self.options for name in opt.names()) + if conflict: + raise ValueError("option names %s already added" % conflict) option = Argument(*optnames, **attrs) self._addoption_instance(option, shortupper=False) @@ -709,7 +768,7 @@ def __init__(self, parser, extra_info=None): extra_info = {} self._parser = parser argparse.ArgumentParser.__init__(self, usage=parser._usage, - add_help=False, formatter_class=DropShorterLongHelpFormatter) + add_help=False, formatter_class=DropShorterLongHelpFormatter) # extra_info is a dict of (param -> value) to display if there's # an usage error to provide more contextual information to the user self.extra_info = extra_info @@ -737,9 +796,10 @@ class DropShorterLongHelpFormatter(argparse.HelpFormatter): - shortcut if there are only two options and one of them is a short one - cache result on action object as this is called at least 2 times """ + def _format_action_invocation(self, action): orgstr = argparse.HelpFormatter._format_action_invocation(self, action) - if orgstr and orgstr[0] != '-': # only optional arguments + if orgstr and orgstr[0] != '-': # only optional arguments return orgstr res = getattr(action, '_formatted_action_invocation', None) if res: @@ -750,7 +810,7 @@ def _format_action_invocation(self, action): action._formatted_action_invocation = orgstr return orgstr return_list = [] - option_map = getattr(action, 'map_long_option', {}) + option_map = getattr(action, 'map_long_option', {}) if option_map is None: option_map = {} short_long = {} @@ -768,38 +828,56 @@ def _format_action_invocation(self, action): short_long[shortened] = xxoption # now short_long has been filled out to the longest with dashes # **and** we keep the right option ordering from add_argument - for option in options: # + for option in options: if len(option) == 2 or option[2] == ' ': return_list.append(option) if option[2:] == short_long.get(option.replace('-', '')): - return_list.append(option.replace(' ', '=')) + return_list.append(option.replace(' ', '=', 1)) action._formatted_action_invocation = ', '.join(return_list) return action._formatted_action_invocation - def _ensure_removed_sysmodule(modname): try: del sys.modules[modname] except KeyError: pass + class CmdOptions(object): """ holds cmdline options as attributes.""" + def __init__(self, values=()): self.__dict__.update(values) + def __repr__(self): - return "" %(self.__dict__,) + return "" % (self.__dict__,) + def copy(self): return CmdOptions(self.__dict__) + class Notset: def __repr__(self): return "" + notset = Notset() FILE_OR_DIR = 'file_or_dir' + +def _iter_rewritable_modules(package_files): + for fn in package_files: + is_simple_module = '/' not in fn and fn.endswith('.py') + is_package = fn.count('/') == 1 and fn.endswith('__init__.py') + if is_simple_module: + module_name, _ = os.path.splitext(fn) + yield module_name + elif is_package: + package_name = os.path.dirname(fn) + yield package_name + + class Config(object): """ access to configuration values, pluginmanager and plugin hooks. """ @@ -817,14 +895,17 @@ def __init__(self, pluginmanager): self.trace = self.pluginmanager.trace.root.get("config") self.hook = self.pluginmanager.hook self._inicache = {} + self._override_ini = () self._opt2dest = {} self._cleanup = [] self._warn = self.pluginmanager._warn self.pluginmanager.register(self, "pytestconfig") self._configured = False + def do_setns(dic): import pytest setns(pytest, dic) + self.hook.pytest_namespace.call_historic(do_setns, {}) self.hook.pytest_addoption.call_historic(kwargs=dict(parser=self._parser)) @@ -847,11 +928,11 @@ def _ensure_unconfigure(self): fin = self._cleanup.pop() fin() - def warn(self, code, message, fslocation=None): + def warn(self, code, message, fslocation=None, nodeid=None): """ generate a warning for this test session. """ self.hook.pytest_logwarning.call_historic(kwargs=dict( code=code, message=message, - fslocation=fslocation, nodeid=None)) + fslocation=fslocation, nodeid=nodeid)) def get_terminal_writer(self): return self.pluginmanager.get_plugin("terminalreporter")._tw @@ -867,14 +948,14 @@ def notify_exception(self, excinfo, option=None): else: style = "native" excrepr = excinfo.getrepr(funcargs=True, - showlocals=getattr(option, 'showlocals', False), - style=style, - ) + showlocals=getattr(option, 'showlocals', False), + style=style, + ) res = self.hook.pytest_internalerror(excrepr=excrepr, excinfo=excinfo) - if not py.builtin.any(res): + if not any(res): for line in str(excrepr).split("\n"): - sys.stderr.write("INTERNALERROR> %s\n" %line) + sys.stderr.write("INTERNALERROR> %s\n" % line) sys.stderr.flush() def cwd_relative_nodeid(self, nodeid): @@ -908,25 +989,85 @@ def pytest_load_initial_conftests(self, early_config): def _initini(self, args): ns, unknown_args = self._parser.parse_known_and_unknown_args(args, namespace=self.option.copy()) - r = determine_setup(ns.inifilename, ns.file_or_dir + unknown_args) + r = determine_setup(ns.inifilename, ns.file_or_dir + unknown_args, warnfunc=self.warn) self.rootdir, self.inifile, self.inicfg = r self._parser.extra_info['rootdir'] = self.rootdir self._parser.extra_info['inifile'] = self.inifile self.invocation_dir = py.path.local() self._parser.addini('addopts', 'extra command line options', 'args') self._parser.addini('minversion', 'minimally required pytest version') + self._override_ini = ns.override_ini or () + + def _consider_importhook(self, args): + """Install the PEP 302 import hook if using assertion rewriting. + + Needs to parse the --assert= option from the commandline + and find all the installed plugins to mark them for rewriting + by the importhook. + """ + ns, unknown_args = self._parser.parse_known_and_unknown_args(args) + mode = ns.assertmode + if mode == 'rewrite': + try: + hook = _pytest.assertion.install_importhook(self) + except SystemError: + mode = 'plain' + else: + self._mark_plugins_for_rewrite(hook) + self._warn_about_missing_assertion(mode) + + def _mark_plugins_for_rewrite(self, hook): + """ + Given an importhook, mark for rewrite any top-level + modules or packages in the distribution package for + all pytest plugins. + """ + import pkg_resources + self.pluginmanager.rewrite_hook = hook + + # 'RECORD' available for plugins installed normally (pip install) + # 'SOURCES.txt' available for plugins installed in dev mode (pip install -e) + # for installed plugins 'SOURCES.txt' returns an empty list, and vice-versa + # so it shouldn't be an issue + metadata_files = 'RECORD', 'SOURCES.txt' + + package_files = ( + entry.split(',')[0] + for entrypoint in pkg_resources.iter_entry_points('pytest11') + for metadata in metadata_files + for entry in entrypoint.dist._get_metadata(metadata) + ) + + for name in _iter_rewritable_modules(package_files): + hook.mark_rewrite(name) + + def _warn_about_missing_assertion(self, mode): + try: + assert False + except AssertionError: + pass + else: + if mode == 'plain': + sys.stderr.write("WARNING: ASSERTIONS ARE NOT EXECUTED" + " and FAILING TESTS WILL PASS. Are you" + " using python -O?") + else: + sys.stderr.write("WARNING: assertions not in test modules or" + " plugins will be ignored" + " because assert statements are not executed " + "by the underlying Python interpreter " + "(are you using python -O?)\n") def _preparse(self, args, addopts=True): - self._initini(args) if addopts: args[:] = shlex.split(os.environ.get('PYTEST_ADDOPTS', '')) + args + self._initini(args) + if addopts: args[:] = self.getini("addopts") + args self._checkversion() + self._consider_importhook(args) self.pluginmanager.consider_preparse(args) - try: - self.pluginmanager.load_setuptools_entrypoints("pytest11") - except ImportError as e: - self.warn("I2", "could not load setuptools entry import: %s" % (e,)) + self.pluginmanager.load_setuptools_entrypoints('pytest11') self.pluginmanager.consider_env() self.known_args_namespace = ns = self._parser.parse_known_args(args, namespace=self.option.copy()) if self.known_args_namespace.confcutdir is None and self.inifile: @@ -934,7 +1075,7 @@ def _preparse(self, args, addopts=True): self.known_args_namespace.confcutdir = confcutdir try: self.hook.pytest_load_initial_conftests(early_config=self, - args=args, parser=self._parser) + args=args, parser=self._parser) except ConftestImportFailure: e = sys.exc_info()[1] if ns.help or ns.version: @@ -952,28 +1093,32 @@ def _checkversion(self): myver = pytest.__version__.split(".") if myver < ver: raise pytest.UsageError( - "%s:%d: requires pytest-%s, actual pytest-%s'" %( - self.inicfg.config.path, self.inicfg.lineof('minversion'), - minver, pytest.__version__)) + "%s:%d: requires pytest-%s, actual pytest-%s'" % ( + self.inicfg.config.path, self.inicfg.lineof('minversion'), + minver, pytest.__version__)) def parse(self, args, addopts=True): # parse given cmdline arguments into this config object. assert not hasattr(self, 'args'), ( - "can only parse cmdline args at most once per Config object") + "can only parse cmdline args at most once per Config object") self._origargs = args self.hook.pytest_addhooks.call_historic( - kwargs=dict(pluginmanager=self.pluginmanager)) + kwargs=dict(pluginmanager=self.pluginmanager)) self._preparse(args, addopts=addopts) # XXX deprecated hook: self.hook.pytest_cmdline_preparse(config=self, args=args) - args = self._parser.parse_setoption(args, self.option, namespace=self.option) - if not args: - cwd = os.getcwd() - if cwd == self.rootdir: - args = self.getini('testpaths') + self._parser.after_preparse = True + try: + args = self._parser.parse_setoption(args, self.option, namespace=self.option) if not args: - args = [cwd] - self.args = args + cwd = os.getcwd() + if cwd == self.rootdir: + args = self.getini('testpaths') + if not args: + args = [cwd] + self.args = args + except PrintHelp: + pass def addinivalue_line(self, name, line): """ add a line to an ini-file option. The option must have been @@ -981,12 +1126,12 @@ def addinivalue_line(self, name, line): the first line in its value. """ x = self.getini(name) assert isinstance(x, list) - x.append(line) # modifies the cached list inline + x.append(line) # modifies the cached list inline def getini(self, name): """ return configuration value from an :ref:`ini file `. If the specified name hasn't been registered through a prior - :py:func:`parser.addini ` + :py:func:`parser.addini <_pytest.config.Parser.addini>` call (usually from a plugin), a ValueError is raised. """ try: return self._inicache[name] @@ -998,21 +1143,23 @@ def _getini(self, name): try: description, type, default = self._parser._inidict[name] except KeyError: - raise ValueError("unknown configuration value: %r" %(name,)) - try: - value = self.inicfg[name] - except KeyError: - if default is not None: - return default - if type is None: - return '' - return [] + raise ValueError("unknown configuration value: %r" % (name,)) + value = self._get_override_ini_value(name) + if value is None: + try: + value = self.inicfg[name] + except KeyError: + if default is not None: + return default + if type is None: + return '' + return [] if type == "pathlist": dp = py.path.local(self.inicfg.config.path).dirpath() - l = [] + values = [] for relpath in shlex.split(value): - l.append(dp.join(relpath, abs=True)) - return l + values.append(dp.join(relpath, abs=True)) + return values elif type == "args": return shlex.split(value) elif type == "linelist": @@ -1029,13 +1176,29 @@ def _getconftest_pathlist(self, name, path): except KeyError: return None modpath = py.path.local(mod.__file__).dirpath() - l = [] + values = [] for relroot in relroots: if not isinstance(relroot, py.path.local): relroot = relroot.replace("/", py.path.local.sep) relroot = modpath.join(relroot, abs=True) - l.append(relroot) - return l + values.append(relroot) + return values + + def _get_override_ini_value(self, name): + value = None + # override_ini is a list of list, to support both -o foo1=bar1 foo2=bar2 and + # and -o foo1=bar1 -o foo2=bar2 options + # always use the last item if multiple value set for same ini-name, + # e.g. -o foo=bar1 -o foo=bar2 will set foo to bar2 + for ini_config_list in self._override_ini: + for ini_config in ini_config_list: + try: + (key, user_ini_value) = ini_config.split("=", 1) + except ValueError: + raise UsageError("-o/--override-ini expects option=value style.") + if key == name: + value = user_ini_value + return value def getoption(self, name, default=notset, skip=False): """ return command line option value. @@ -1057,7 +1220,7 @@ def getoption(self, name, default=notset, skip=False): return default if skip: import pytest - pytest.skip("no %r option found" %(name,)) + pytest.skip("no %r option found" % (name,)) raise ValueError("no option named %r" % (name,)) def getvalue(self, name, path=None): @@ -1068,13 +1231,26 @@ def getvalueorskip(self, name, path=None): """ (deprecated, use getoption(skip=True)) """ return self.getoption(name, skip=True) + def exists(path, ignore=EnvironmentError): try: return path.check() except ignore: return False -def getcfg(args, inibasenames): + +def getcfg(args, warnfunc=None): + """ + Search the list of arguments for a valid ini-file for pytest, + and return a tuple of (rootdir, inifile, cfg-dict). + + note: warnfunc is an optional function used to warn + about ini-files that use deprecated features. + This parameter should be removed when pytest + adopts standard deprecation warnings (#1804). + """ + from _pytest.deprecated import SETUP_CFG_PYTEST + inibasenames = ["pytest.ini", "tox.ini", "setup.cfg"] args = [x for x in args if not str(x).startswith("-")] if not args: args = [py.path.local()] @@ -1086,57 +1262,89 @@ def getcfg(args, inibasenames): if exists(p): iniconfig = py.iniconfig.IniConfig(p) if 'pytest' in iniconfig.sections: + if inibasename == 'setup.cfg' and warnfunc: + warnfunc('C1', SETUP_CFG_PYTEST) return base, p, iniconfig['pytest'] + if inibasename == 'setup.cfg' and 'tool:pytest' in iniconfig.sections: + return base, p, iniconfig['tool:pytest'] elif inibasename == "pytest.ini": # allowed to be empty return base, p, {} return None, None, None -def get_common_ancestor(args): - # args are what we get after early command line parsing (usually - # strings, but can be py.path.local objects as well) +def get_common_ancestor(paths): common_ancestor = None - for arg in args: - if str(arg)[0] == "-": + for path in paths: + if not path.exists(): continue - p = py.path.local(arg) if common_ancestor is None: - common_ancestor = p + common_ancestor = path else: - if p.relto(common_ancestor) or p == common_ancestor: + if path.relto(common_ancestor) or path == common_ancestor: continue - elif common_ancestor.relto(p): - common_ancestor = p + elif common_ancestor.relto(path): + common_ancestor = path else: - shared = p.common(common_ancestor) + shared = path.common(common_ancestor) if shared is not None: common_ancestor = shared if common_ancestor is None: common_ancestor = py.path.local() - elif not common_ancestor.isdir(): + elif common_ancestor.isfile(): common_ancestor = common_ancestor.dirpath() return common_ancestor -def determine_setup(inifile, args): +def get_dirs_from_args(args): + def is_option(x): + return str(x).startswith('-') + + def get_file_part_from_node_id(x): + return str(x).split('::')[0] + + def get_dir_from_path(path): + if path.isdir(): + return path + return py.path.local(path.dirname) + + # These look like paths but may not exist + possible_paths = ( + py.path.local(get_file_part_from_node_id(arg)) + for arg in args + if not is_option(arg) + ) + + return [ + get_dir_from_path(path) + for path in possible_paths + if path.exists() + ] + + +def determine_setup(inifile, args, warnfunc=None): + dirs = get_dirs_from_args(args) if inifile: iniconfig = py.iniconfig.IniConfig(inifile) try: inicfg = iniconfig["pytest"] except KeyError: inicfg = None - rootdir = get_common_ancestor(args) + rootdir = get_common_ancestor(dirs) else: - ancestor = get_common_ancestor(args) - rootdir, inifile, inicfg = getcfg( - [ancestor], ["pytest.ini", "tox.ini", "setup.cfg"]) + ancestor = get_common_ancestor(dirs) + rootdir, inifile, inicfg = getcfg([ancestor], warnfunc=warnfunc) if rootdir is None: for rootdir in ancestor.parts(reverse=True): if rootdir.join("setup.py").exists(): break else: - rootdir = ancestor + rootdir, inifile, inicfg = getcfg(dirs, warnfunc=warnfunc) + if rootdir is None: + rootdir = get_common_ancestor([py.path.local(), ancestor]) + is_fs_root = os.path.splitdrive(str(rootdir))[1] == '/' + if is_fs_root: + rootdir = ancestor return rootdir, inifile, inicfg or {} @@ -1156,7 +1364,7 @@ def setns(obj, dic): else: setattr(obj, name, value) obj.__all__.append(name) - #if obj != pytest: + # if obj != pytest: # pytest.__all__.append(name) setattr(pytest, name, value) diff --git a/_pytest/pdb.py b/_pytest/debugging.py similarity index 66% rename from _pytest/pdb.py rename to _pytest/debugging.py index 84c920d172cb91a..d7dca780956ec39 100644 --- a/_pytest/pdb.py +++ b/_pytest/debugging.py @@ -1,65 +1,79 @@ """ interactive debugging with PDB, the Python Debugger. """ -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function import pdb import sys -import pytest - def pytest_addoption(parser): group = parser.getgroup("general") - group._addoption('--pdb', - action="store_true", dest="usepdb", default=False, - help="start the interactive Python debugger on errors.") + group._addoption( + '--pdb', dest="usepdb", action="store_true", + help="start the interactive Python debugger on errors.") + group._addoption( + '--pdbcls', dest="usepdb_cls", metavar="modulename:classname", + help="start a custom interactive Python debugger on errors. " + "For example: --pdbcls=IPython.terminal.debugger:TerminalPdb") -def pytest_namespace(): - return {'set_trace': pytestPDB().set_trace} def pytest_configure(config): + if config.getvalue("usepdb_cls"): + modname, classname = config.getvalue("usepdb_cls").split(":") + __import__(modname) + pdb_cls = getattr(sys.modules[modname], classname) + else: + pdb_cls = pdb.Pdb + if config.getvalue("usepdb"): config.pluginmanager.register(PdbInvoke(), 'pdbinvoke') old = (pdb.set_trace, pytestPDB._pluginmanager) + def fin(): pdb.set_trace, pytestPDB._pluginmanager = old pytestPDB._config = None - pdb.set_trace = pytest.set_trace + pytestPDB._pdb_cls = pdb.Pdb + + pdb.set_trace = pytestPDB.set_trace pytestPDB._pluginmanager = config.pluginmanager pytestPDB._config = config + pytestPDB._pdb_cls = pdb_cls config._cleanup.append(fin) + class pytestPDB: """ Pseudo PDB that defers to the real pdb. """ _pluginmanager = None _config = None + _pdb_cls = pdb.Pdb - def set_trace(self): + @classmethod + def set_trace(cls): """ invoke PDB set_trace debugging, dropping any IO capturing. """ import _pytest.config frame = sys._getframe().f_back - if self._pluginmanager is not None: - capman = self._pluginmanager.getplugin("capturemanager") + if cls._pluginmanager is not None: + capman = cls._pluginmanager.getplugin("capturemanager") if capman: - capman.suspendcapture(in_=True) - tw = _pytest.config.create_terminal_writer(self._config) + capman.suspend_global_capture(in_=True) + tw = _pytest.config.create_terminal_writer(cls._config) tw.line() tw.sep(">", "PDB set_trace (IO-capturing turned off)") - self._pluginmanager.hook.pytest_enter_pdb(config=self._config) - pdb.Pdb().set_trace(frame) + cls._pluginmanager.hook.pytest_enter_pdb(config=cls._config) + cls._pdb_cls().set_trace(frame) class PdbInvoke: def pytest_exception_interact(self, node, call, report): capman = node.config.pluginmanager.getplugin("capturemanager") if capman: - out, err = capman.suspendcapture(in_=True) + out, err = capman.suspend_global_capture(in_=True) sys.stdout.write(out) sys.stdout.write(err) _enter_pdb(node, call.excinfo, report) def pytest_internalerror(self, excrepr, excinfo): for line in str(excrepr).split("\n"): - sys.stderr.write("INTERNALERROR> %s\n" %line) + sys.stderr.write("INTERNALERROR> %s\n" % line) sys.stderr.flush() tb = _postmortem_traceback(excinfo) post_mortem(tb) @@ -98,7 +112,7 @@ def _find_last_non_hidden_frame(stack): def post_mortem(t): - class Pdb(pdb.Pdb): + class Pdb(pytestPDB._pdb_cls): def get_stack(self, f, t): stack, i = pdb.Pdb.get_stack(self, f, t) if f is None: diff --git a/_pytest/deprecated.py b/_pytest/deprecated.py new file mode 100644 index 000000000000000..9c0fbeca7bc9e97 --- /dev/null +++ b/_pytest/deprecated.py @@ -0,0 +1,52 @@ +""" +This module contains deprecation messages and bits of code used elsewhere in the codebase +that is planned to be removed in the next pytest release. + +Keeping it in a central location makes it easy to track what is deprecated and should +be removed when the time comes. +""" +from __future__ import absolute_import, division, print_function + + +class RemovedInPytest4Warning(DeprecationWarning): + """warning class for features removed in pytest 4.0""" + + +MAIN_STR_ARGS = 'passing a string to pytest.main() is deprecated, ' \ + 'pass a list of arguments instead.' + +YIELD_TESTS = 'yield tests are deprecated, and scheduled to be removed in pytest 4.0' + +FUNCARG_PREFIX = ( + '{name}: declaring fixtures using "pytest_funcarg__" prefix is deprecated ' + 'and scheduled to be removed in pytest 4.0. ' + 'Please remove the prefix and use the @pytest.fixture decorator instead.') + +SETUP_CFG_PYTEST = '[pytest] section in setup.cfg files is deprecated, use [tool:pytest] instead.' + +GETFUNCARGVALUE = "use of getfuncargvalue is deprecated, use getfixturevalue" + +RESULT_LOG = ( + '--result-log is deprecated and scheduled for removal in pytest 4.0.\n' + 'See https://docs.pytest.org/en/latest/usage.html#creating-resultlog-format-files for more information.' +) + +MARK_INFO_ATTRIBUTE = RemovedInPytest4Warning( + "MarkInfo objects are deprecated as they contain the merged marks" +) + +MARK_PARAMETERSET_UNPACKING = RemovedInPytest4Warning( + "Applying marks directly to parameters is deprecated," + " please use pytest.param(..., marks=...) instead.\n" + "For more details, see: https://docs.pytest.org/en/latest/parametrize.html" +) + +COLLECTOR_MAKEITEM = RemovedInPytest4Warning( + "pycollector makeitem was removed " + "as it is an accidentially leaked internal api" +) + +METAFUNC_ADD_CALL = ( + "Metafunc.addcall is deprecated and scheduled to be removed in pytest 4.0.\n" + "Please use Metafunc.parametrize instead." +) diff --git a/_pytest/doctest.py b/_pytest/doctest.py index a57f7a4949e3b4b..bba90e551c505ae 100644 --- a/_pytest/doctest.py +++ b/_pytest/doctest.py @@ -1,41 +1,68 @@ """ discover and run doctests in modules and test files.""" -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function import traceback import pytest -from _pytest._code.code import TerminalRepr, ReprFileLocation, ExceptionInfo -from _pytest.python import FixtureRequest +from _pytest._code.code import ExceptionInfo, ReprFileLocation, TerminalRepr +from _pytest.fixtures import FixtureRequest +DOCTEST_REPORT_CHOICE_NONE = 'none' +DOCTEST_REPORT_CHOICE_CDIFF = 'cdiff' +DOCTEST_REPORT_CHOICE_NDIFF = 'ndiff' +DOCTEST_REPORT_CHOICE_UDIFF = 'udiff' +DOCTEST_REPORT_CHOICE_ONLY_FIRST_FAILURE = 'only_first_failure' + +DOCTEST_REPORT_CHOICES = ( + DOCTEST_REPORT_CHOICE_NONE, + DOCTEST_REPORT_CHOICE_CDIFF, + DOCTEST_REPORT_CHOICE_NDIFF, + DOCTEST_REPORT_CHOICE_UDIFF, + DOCTEST_REPORT_CHOICE_ONLY_FIRST_FAILURE, +) + def pytest_addoption(parser): parser.addini('doctest_optionflags', 'option flags for doctests', - type="args", default=["ELLIPSIS"]) + type="args", default=["ELLIPSIS"]) + parser.addini("doctest_encoding", 'encoding used for doctest files', default="utf-8") group = parser.getgroup("collect") group.addoption("--doctest-modules", - action="store_true", default=False, - help="run doctests in all .py modules", - dest="doctestmodules") + action="store_true", default=False, + help="run doctests in all .py modules", + dest="doctestmodules") + group.addoption("--doctest-report", + type=str.lower, default="udiff", + help="choose another output format for diffs on doctest failure", + choices=DOCTEST_REPORT_CHOICES, + dest="doctestreport") group.addoption("--doctest-glob", - action="append", default=[], metavar="pat", - help="doctests file matching pattern, default: test*.txt", - dest="doctestglob") + action="append", default=[], metavar="pat", + help="doctests file matching pattern, default: test*.txt", + dest="doctestglob") group.addoption("--doctest-ignore-import-errors", - action="store_true", default=False, - help="ignore doctest ImportErrors", - dest="doctest_ignore_import_errors") + action="store_true", default=False, + help="ignore doctest ImportErrors", + dest="doctest_ignore_import_errors") def pytest_collect_file(path, parent): config = parent.config if path.ext == ".py": - if config.option.doctestmodules: + if config.option.doctestmodules and not _is_setup_py(config, path, parent): return DoctestModule(path, parent) elif _is_doctest(config, path, parent): return DoctestTextfile(path, parent) +def _is_setup_py(config, path, parent): + if path.basename != "setup.py": + return False + contents = path.read() + return 'setuptools' in contents or 'distutils' in contents + + def _is_doctest(config, path, parent): if path.ext in ('.txt', '.rst') and parent.session.isinitpath(path): return True @@ -59,7 +86,6 @@ def toterminal(self, tw): class DoctestItem(pytest.Item): - def __init__(self, name, parent, runner=None, dtest=None): super(DoctestItem, self).__init__(name, parent) self.runner = runner @@ -70,7 +96,9 @@ def __init__(self, name, parent, runner=None, dtest=None): def setup(self): if self.dtest is not None: self.fixture_request = _setup_fixtures(self) - globs = dict(getfixture=self.fixture_request.getfuncargvalue) + globs = dict(getfixture=self.fixture_request.getfixturevalue) + for name, value in self.fixture_request.getfixturevalue('doctest_namespace').items(): + globs[name] = value self.dtest.globs.update(globs) def runtest(self): @@ -92,14 +120,14 @@ def repr_failure(self, excinfo): message = excinfo.type.__name__ reprlocation = ReprFileLocation(filename, lineno, message) checker = _get_checker() - REPORT_UDIFF = doctest.REPORT_UDIFF + report_choice = _get_report_choice(self.config.getoption("doctestreport")) if lineno is not None: lines = doctestfailure.test.docstring.splitlines(False) # add line numbers to the left of the error message lines = ["%03d %s" % (i + test.lineno + 1, x) for (i, x) in enumerate(lines)] # trim docstring error lines to 10 - lines = lines[example.lineno - 9:example.lineno + 1] + lines = lines[max(example.lineno - 9, 0):example.lineno + 1] else: lines = ['EXAMPLE LOCATION UNKNOWN, not showing all tests of that example'] indent = '>>>' @@ -108,18 +136,18 @@ def repr_failure(self, excinfo): indent = '...' if excinfo.errisinstance(doctest.DocTestFailure): lines += checker.output_difference(example, - doctestfailure.got, REPORT_UDIFF).split("\n") + doctestfailure.got, report_choice).split("\n") else: inner_excinfo = ExceptionInfo(excinfo.value.exc_info) lines += ["UNEXPECTED EXCEPTION: %s" % - repr(inner_excinfo.value)] + repr(inner_excinfo.value)] lines += traceback.format_exception(*excinfo.value.exc_info) return ReprFailDoctest(reprlocation, lines) else: return super(DoctestItem, self).repr_failure(excinfo) def reportinfo(self): - return self.fspath, None, "[doctest] %s" % self.name + return self.fspath, self.dtest.lineno, "[doctest] %s" % self.name def _get_flag_lookup(): @@ -144,29 +172,29 @@ def get_optionflags(parent): return flag_acc -class DoctestTextfile(DoctestItem, pytest.Module): +class DoctestTextfile(pytest.Module): + obj = None - def runtest(self): + def collect(self): import doctest - fixture_request = _setup_fixtures(self) # inspired by doctest.testfile; ideally we would use it directly, # but it doesn't support passing a custom checker - text = self.fspath.read() + encoding = self.config.getini("doctest_encoding") + text = self.fspath.read_text(encoding) filename = str(self.fspath) name = self.fspath.basename - globs = dict(getfixture=fixture_request.getfuncargvalue) - if '__name__' not in globs: - globs['__name__'] = '__main__' + globs = {'__name__': '__main__'} optionflags = get_optionflags(self) runner = doctest.DebugRunner(verbose=0, optionflags=optionflags, checker=_get_checker()) + _fix_spoof_python2(runner, encoding) parser = doctest.DocTestParser() test = parser.get_doctest(text, globs, name, filename, 0) - _check_all_skipped(test) - runner.run(test) + if test.examples: + yield DoctestItem(test.name, self, runner, test) def _check_all_skipped(test): @@ -197,6 +225,7 @@ def collect(self): optionflags = get_optionflags(self) runner = doctest.DebugRunner(verbose=0, optionflags=optionflags, checker=_get_checker()) + for test in finder.find(module, module.__name__): if test.examples: # skip empty doctests yield DoctestItem(test.name, self, runner, test) @@ -288,3 +317,53 @@ def _get_allow_bytes_flag(): """ import doctest return doctest.register_optionflag('ALLOW_BYTES') + + +def _get_report_choice(key): + """ + This function returns the actual `doctest` module flag value, we want to do it as late as possible to avoid + importing `doctest` and all its dependencies when parsing options, as it adds overhead and breaks tests. + """ + import doctest + + return { + DOCTEST_REPORT_CHOICE_UDIFF: doctest.REPORT_UDIFF, + DOCTEST_REPORT_CHOICE_CDIFF: doctest.REPORT_CDIFF, + DOCTEST_REPORT_CHOICE_NDIFF: doctest.REPORT_NDIFF, + DOCTEST_REPORT_CHOICE_ONLY_FIRST_FAILURE: doctest.REPORT_ONLY_FIRST_FAILURE, + DOCTEST_REPORT_CHOICE_NONE: 0, + }[key] + + +def _fix_spoof_python2(runner, encoding): + """ + Installs a "SpoofOut" into the given DebugRunner so it properly deals with unicode output. This + should patch only doctests for text files because they don't have a way to declare their + encoding. Doctests in docstrings from Python modules don't have the same problem given that + Python already decoded the strings. + + This fixes the problem related in issue #2434. + """ + from _pytest.compat import _PY2 + if not _PY2: + return + + from doctest import _SpoofOut + + class UnicodeSpoof(_SpoofOut): + + def getvalue(self): + result = _SpoofOut.getvalue(self) + if encoding: + result = result.decode(encoding) + return result + + runner._fakeout = UnicodeSpoof() + + +@pytest.fixture(scope='session') +def doctest_namespace(): + """ + Inject names into the doctest namespace. + """ + return dict() diff --git a/_pytest/fixtures.py b/_pytest/fixtures.py new file mode 100644 index 000000000000000..e09ffaddba73ff8 --- /dev/null +++ b/_pytest/fixtures.py @@ -0,0 +1,1147 @@ +from __future__ import absolute_import, division, print_function + +import functools +import inspect +import sys +import warnings +from collections import OrderedDict + +import attr +import py +from py._code.code import FormattedExcinfo + +import _pytest +from _pytest import nodes +from _pytest._code.code import TerminalRepr +from _pytest.compat import ( + NOTSET, exc_clear, _format_args, + getfslineno, get_real_func, + is_generator, isclass, getimfunc, + getlocation, getfuncargnames, + safe_getattr, + FuncargnamesCompatAttr, +) +from _pytest.outcomes import fail, TEST_OUTCOME + + +def pytest_sessionstart(session): + import _pytest.python + + scopename2class.update({ + 'class': _pytest.python.Class, + 'module': _pytest.python.Module, + 'function': _pytest.main.Item, + 'session': _pytest.main.Session, + }) + session._fixturemanager = FixtureManager(session) + + +scopename2class = {} + + +scope2props = dict(session=()) +scope2props["module"] = ("fspath", "module") +scope2props["class"] = scope2props["module"] + ("cls",) +scope2props["instance"] = scope2props["class"] + ("instance", ) +scope2props["function"] = scope2props["instance"] + ("function", "keywords") + + +def scopeproperty(name=None, doc=None): + def decoratescope(func): + scopename = name or func.__name__ + + def provide(self): + if func.__name__ in scope2props[self.scope]: + return func(self) + raise AttributeError("%s not available in %s-scoped context" % ( + scopename, self.scope)) + + return property(provide, None, None, func.__doc__) + return decoratescope + + +def get_scope_node(node, scope): + cls = scopename2class.get(scope) + if cls is None: + raise ValueError("unknown scope") + return node.getparent(cls) + + +def add_funcarg_pseudo_fixture_def(collector, metafunc, fixturemanager): + # this function will transform all collected calls to a functions + # if they use direct funcargs (i.e. direct parametrization) + # because we want later test execution to be able to rely on + # an existing FixtureDef structure for all arguments. + # XXX we can probably avoid this algorithm if we modify CallSpec2 + # to directly care for creating the fixturedefs within its methods. + if not metafunc._calls[0].funcargs: + return # this function call does not have direct parametrization + # collect funcargs of all callspecs into a list of values + arg2params = {} + arg2scope = {} + for callspec in metafunc._calls: + for argname, argvalue in callspec.funcargs.items(): + assert argname not in callspec.params + callspec.params[argname] = argvalue + arg2params_list = arg2params.setdefault(argname, []) + callspec.indices[argname] = len(arg2params_list) + arg2params_list.append(argvalue) + if argname not in arg2scope: + scopenum = callspec._arg2scopenum.get(argname, + scopenum_function) + arg2scope[argname] = scopes[scopenum] + callspec.funcargs.clear() + + # register artificial FixtureDef's so that later at test execution + # time we can rely on a proper FixtureDef to exist for fixture setup. + arg2fixturedefs = metafunc._arg2fixturedefs + for argname, valuelist in arg2params.items(): + # if we have a scope that is higher than function we need + # to make sure we only ever create an according fixturedef on + # a per-scope basis. We thus store and cache the fixturedef on the + # node related to the scope. + scope = arg2scope[argname] + node = None + if scope != "function": + node = get_scope_node(collector, scope) + if node is None: + assert scope == "class" and isinstance(collector, _pytest.python.Module) + # use module-level collector for class-scope (for now) + node = collector + if node and argname in node._name2pseudofixturedef: + arg2fixturedefs[argname] = [node._name2pseudofixturedef[argname]] + else: + fixturedef = FixtureDef(fixturemanager, '', argname, + get_direct_param_fixture_func, + arg2scope[argname], + valuelist, False, False) + arg2fixturedefs[argname] = [fixturedef] + if node is not None: + node._name2pseudofixturedef[argname] = fixturedef + + +def getfixturemarker(obj): + """ return fixturemarker or None if it doesn't exist or raised + exceptions.""" + try: + return getattr(obj, "_pytestfixturefunction", None) + except TEST_OUTCOME: + # some objects raise errors like request (from flask import request) + # we don't expect them to be fixture functions + return None + + +def get_parametrized_fixture_keys(item, scopenum): + """ return list of keys for all parametrized arguments which match + the specified scope. """ + assert scopenum < scopenum_function # function + try: + cs = item.callspec + except AttributeError: + pass + else: + # cs.indices.items() is random order of argnames. Need to + # sort this so that different calls to + # get_parametrized_fixture_keys will be deterministic. + for argname, param_index in sorted(cs.indices.items()): + if cs._arg2scopenum[argname] != scopenum: + continue + if scopenum == 0: # session + key = (argname, param_index) + elif scopenum == 1: # module + key = (argname, param_index, item.fspath) + elif scopenum == 2: # class + key = (argname, param_index, item.fspath, item.cls) + yield key + + +# algorithm for sorting on a per-parametrized resource setup basis +# it is called for scopenum==0 (session) first and performs sorting +# down to the lower scopes such as to minimize number of "high scope" +# setups and teardowns + +def reorder_items(items): + argkeys_cache = {} + for scopenum in range(0, scopenum_function): + argkeys_cache[scopenum] = d = {} + for item in items: + keys = OrderedDict.fromkeys(get_parametrized_fixture_keys(item, scopenum)) + if keys: + d[item] = keys + return reorder_items_atscope(items, set(), argkeys_cache, 0) + + +def reorder_items_atscope(items, ignore, argkeys_cache, scopenum): + if scopenum >= scopenum_function or len(items) < 3: + return items + items_done = [] + while 1: + items_before, items_same, items_other, newignore = \ + slice_items(items, ignore, argkeys_cache[scopenum]) + items_before = reorder_items_atscope( + items_before, ignore, argkeys_cache, scopenum + 1) + if items_same is None: + # nothing to reorder in this scope + assert items_other is None + return items_done + items_before + items_done.extend(items_before) + items = items_same + items_other + ignore = newignore + + +def slice_items(items, ignore, scoped_argkeys_cache): + # we pick the first item which uses a fixture instance in the + # requested scope and which we haven't seen yet. We slice the input + # items list into a list of items_nomatch, items_same and + # items_other + if scoped_argkeys_cache: # do we need to do work at all? + it = iter(items) + # first find a slicing key + for i, item in enumerate(it): + argkeys = scoped_argkeys_cache.get(item) + if argkeys is not None: + newargkeys = OrderedDict.fromkeys(k for k in argkeys if k not in ignore) + if newargkeys: # found a slicing key + slicing_argkey, _ = newargkeys.popitem() + items_before = items[:i] + items_same = [item] + items_other = [] + # now slice the remainder of the list + for item in it: + argkeys = scoped_argkeys_cache.get(item) + if argkeys and slicing_argkey in argkeys and \ + slicing_argkey not in ignore: + items_same.append(item) + else: + items_other.append(item) + newignore = ignore.copy() + newignore.add(slicing_argkey) + return (items_before, items_same, items_other, newignore) + return items, None, None, None + + +def fillfixtures(function): + """ fill missing funcargs for a test function. """ + try: + request = function._request + except AttributeError: + # XXX this special code path is only expected to execute + # with the oejskit plugin. It uses classes with funcargs + # and we thus have to work a bit to allow this. + fm = function.session._fixturemanager + fi = fm.getfixtureinfo(function.parent, function.obj, None) + function._fixtureinfo = fi + request = function._request = FixtureRequest(function) + request._fillfixtures() + # prune out funcargs for jstests + newfuncargs = {} + for name in fi.argnames: + newfuncargs[name] = function.funcargs[name] + function.funcargs = newfuncargs + else: + request._fillfixtures() + + +def get_direct_param_fixture_func(request): + return request.param + + +class FuncFixtureInfo: + def __init__(self, argnames, names_closure, name2fixturedefs): + self.argnames = argnames + self.names_closure = names_closure + self.name2fixturedefs = name2fixturedefs + + +class FixtureRequest(FuncargnamesCompatAttr): + """ A request for a fixture from a test or fixture function. + + A request object gives access to the requesting test context + and has an optional ``param`` attribute in case + the fixture is parametrized indirectly. + """ + + def __init__(self, pyfuncitem): + self._pyfuncitem = pyfuncitem + #: fixture for which this request is being performed + self.fixturename = None + #: Scope string, one of "function", "class", "module", "session" + self.scope = "function" + self._fixture_values = {} # argname -> fixture value + self._fixture_defs = {} # argname -> FixtureDef + fixtureinfo = pyfuncitem._fixtureinfo + self._arg2fixturedefs = fixtureinfo.name2fixturedefs.copy() + self._arg2index = {} + self._fixturemanager = pyfuncitem.session._fixturemanager + + @property + def fixturenames(self): + # backward incompatible note: now a readonly property + return list(self._pyfuncitem._fixtureinfo.names_closure) + + @property + def node(self): + """ underlying collection node (depends on current request scope)""" + return self._getscopeitem(self.scope) + + def _getnextfixturedef(self, argname): + fixturedefs = self._arg2fixturedefs.get(argname, None) + if fixturedefs is None: + # we arrive here because of a a dynamic call to + # getfixturevalue(argname) usage which was naturally + # not known at parsing/collection time + parentid = self._pyfuncitem.parent.nodeid + fixturedefs = self._fixturemanager.getfixturedefs(argname, parentid) + self._arg2fixturedefs[argname] = fixturedefs + # fixturedefs list is immutable so we maintain a decreasing index + index = self._arg2index.get(argname, 0) - 1 + if fixturedefs is None or (-index > len(fixturedefs)): + raise FixtureLookupError(argname, self) + self._arg2index[argname] = index + return fixturedefs[index] + + @property + def config(self): + """ the pytest config object associated with this request. """ + return self._pyfuncitem.config + + @scopeproperty() + def function(self): + """ test function object if the request has a per-function scope. """ + return self._pyfuncitem.obj + + @scopeproperty("class") + def cls(self): + """ class (can be None) where the test function was collected. """ + clscol = self._pyfuncitem.getparent(_pytest.python.Class) + if clscol: + return clscol.obj + + @property + def instance(self): + """ instance (can be None) on which test function was collected. """ + # unittest support hack, see _pytest.unittest.TestCaseFunction + try: + return self._pyfuncitem._testcase + except AttributeError: + function = getattr(self, "function", None) + if function is not None: + return py.builtin._getimself(function) + + @scopeproperty() + def module(self): + """ python module object where the test function was collected. """ + return self._pyfuncitem.getparent(_pytest.python.Module).obj + + @scopeproperty() + def fspath(self): + """ the file system path of the test module which collected this test. """ + return self._pyfuncitem.fspath + + @property + def keywords(self): + """ keywords/markers dictionary for the underlying node. """ + return self.node.keywords + + @property + def session(self): + """ pytest session object. """ + return self._pyfuncitem.session + + def addfinalizer(self, finalizer): + """ add finalizer/teardown function to be called after the + last test within the requesting test context finished + execution. """ + # XXX usually this method is shadowed by fixturedef specific ones + self._addfinalizer(finalizer, scope=self.scope) + + def _addfinalizer(self, finalizer, scope): + colitem = self._getscopeitem(scope) + self._pyfuncitem.session._setupstate.addfinalizer( + finalizer=finalizer, colitem=colitem) + + def applymarker(self, marker): + """ Apply a marker to a single test function invocation. + This method is useful if you don't want to have a keyword/marker + on all function invocations. + + :arg marker: a :py:class:`_pytest.mark.MarkDecorator` object + created by a call to ``pytest.mark.NAME(...)``. + """ + try: + self.node.keywords[marker.markname] = marker + except AttributeError: + raise ValueError(marker) + + def raiseerror(self, msg): + """ raise a FixtureLookupError with the given message. """ + raise self._fixturemanager.FixtureLookupError(None, self, msg) + + def _fillfixtures(self): + item = self._pyfuncitem + fixturenames = getattr(item, "fixturenames", self.fixturenames) + for argname in fixturenames: + if argname not in item.funcargs: + item.funcargs[argname] = self.getfixturevalue(argname) + + def cached_setup(self, setup, teardown=None, scope="module", extrakey=None): + """ (deprecated) Return a testing resource managed by ``setup`` & + ``teardown`` calls. ``scope`` and ``extrakey`` determine when the + ``teardown`` function will be called so that subsequent calls to + ``setup`` would recreate the resource. With pytest-2.3 you often + do not need ``cached_setup()`` as you can directly declare a scope + on a fixture function and register a finalizer through + ``request.addfinalizer()``. + + :arg teardown: function receiving a previously setup resource. + :arg setup: a no-argument function creating a resource. + :arg scope: a string value out of ``function``, ``class``, ``module`` + or ``session`` indicating the caching lifecycle of the resource. + :arg extrakey: added to internal caching key of (funcargname, scope). + """ + if not hasattr(self.config, '_setupcache'): + self.config._setupcache = {} # XXX weakref? + cachekey = (self.fixturename, self._getscopeitem(scope), extrakey) + cache = self.config._setupcache + try: + val = cache[cachekey] + except KeyError: + self._check_scope(self.fixturename, self.scope, scope) + val = setup() + cache[cachekey] = val + if teardown is not None: + def finalizer(): + del cache[cachekey] + teardown(val) + self._addfinalizer(finalizer, scope=scope) + return val + + def getfixturevalue(self, argname): + """ Dynamically run a named fixture function. + + Declaring fixtures via function argument is recommended where possible. + But if you can only decide whether to use another fixture at test + setup time, you may use this function to retrieve it inside a fixture + or test function body. + """ + return self._get_active_fixturedef(argname).cached_result[0] + + def getfuncargvalue(self, argname): + """ Deprecated, use getfixturevalue. """ + from _pytest import deprecated + warnings.warn( + deprecated.GETFUNCARGVALUE, + DeprecationWarning, + stacklevel=2) + return self.getfixturevalue(argname) + + def _get_active_fixturedef(self, argname): + try: + return self._fixture_defs[argname] + except KeyError: + try: + fixturedef = self._getnextfixturedef(argname) + except FixtureLookupError: + if argname == "request": + class PseudoFixtureDef: + cached_result = (self, [0], None) + scope = "function" + return PseudoFixtureDef + raise + # remove indent to prevent the python3 exception + # from leaking into the call + result = self._getfixturevalue(fixturedef) + self._fixture_values[argname] = result + self._fixture_defs[argname] = fixturedef + return fixturedef + + def _get_fixturestack(self): + current = self + values = [] + while 1: + fixturedef = getattr(current, "_fixturedef", None) + if fixturedef is None: + values.reverse() + return values + values.append(fixturedef) + current = current._parent_request + + def _getfixturevalue(self, fixturedef): + # prepare a subrequest object before calling fixture function + # (latter managed by fixturedef) + argname = fixturedef.argname + funcitem = self._pyfuncitem + scope = fixturedef.scope + try: + param = funcitem.callspec.getparam(argname) + except (AttributeError, ValueError): + param = NOTSET + param_index = 0 + if fixturedef.params is not None: + frame = inspect.stack()[3] + frameinfo = inspect.getframeinfo(frame[0]) + source_path = frameinfo.filename + source_lineno = frameinfo.lineno + source_path = py.path.local(source_path) + if source_path.relto(funcitem.config.rootdir): + source_path = source_path.relto(funcitem.config.rootdir) + msg = ( + "The requested fixture has no parameter defined for the " + "current test.\n\nRequested fixture '{0}' defined in:\n{1}" + "\n\nRequested here:\n{2}:{3}".format( + fixturedef.argname, + getlocation(fixturedef.func, funcitem.config.rootdir), + source_path, + source_lineno, + ) + ) + fail(msg) + else: + # indices might not be set if old-style metafunc.addcall() was used + param_index = funcitem.callspec.indices.get(argname, 0) + # if a parametrize invocation set a scope it will override + # the static scope defined with the fixture function + paramscopenum = funcitem.callspec._arg2scopenum.get(argname) + if paramscopenum is not None: + scope = scopes[paramscopenum] + + subrequest = SubRequest(self, scope, param, param_index, fixturedef) + + # check if a higher-level scoped fixture accesses a lower level one + subrequest._check_scope(argname, self.scope, scope) + + # clear sys.exc_info before invoking the fixture (python bug?) + # if its not explicitly cleared it will leak into the call + exc_clear() + try: + # call the fixture function + val = fixturedef.execute(request=subrequest) + finally: + # if fixture function failed it might have registered finalizers + self.session._setupstate.addfinalizer(functools.partial(fixturedef.finish, request=subrequest), + subrequest.node) + return val + + def _check_scope(self, argname, invoking_scope, requested_scope): + if argname == "request": + return + if scopemismatch(invoking_scope, requested_scope): + # try to report something helpful + lines = self._factorytraceback() + fail("ScopeMismatch: You tried to access the %r scoped " + "fixture %r with a %r scoped request object, " + "involved factories\n%s" % ( + (requested_scope, argname, invoking_scope, "\n".join(lines))), + pytrace=False) + + def _factorytraceback(self): + lines = [] + for fixturedef in self._get_fixturestack(): + factory = fixturedef.func + fs, lineno = getfslineno(factory) + p = self._pyfuncitem.session.fspath.bestrelpath(fs) + args = _format_args(factory) + lines.append("%s:%d: def %s%s" % ( + p, lineno, factory.__name__, args)) + return lines + + def _getscopeitem(self, scope): + if scope == "function": + # this might also be a non-function Item despite its attribute name + return self._pyfuncitem + node = get_scope_node(self._pyfuncitem, scope) + if node is None and scope == "class": + # fallback to function item itself + node = self._pyfuncitem + assert node, 'Could not obtain a node for scope "{}" for function {!r}'.format(scope, self._pyfuncitem) + return node + + def __repr__(self): + return "" % (self.node) + + +class SubRequest(FixtureRequest): + """ a sub request for handling getting a fixture from a + test function/fixture. """ + + def __init__(self, request, scope, param, param_index, fixturedef): + self._parent_request = request + self.fixturename = fixturedef.argname + if param is not NOTSET: + self.param = param + self.param_index = param_index + self.scope = scope + self._fixturedef = fixturedef + self._pyfuncitem = request._pyfuncitem + self._fixture_values = request._fixture_values + self._fixture_defs = request._fixture_defs + self._arg2fixturedefs = request._arg2fixturedefs + self._arg2index = request._arg2index + self._fixturemanager = request._fixturemanager + + def __repr__(self): + return "" % (self.fixturename, self._pyfuncitem) + + def addfinalizer(self, finalizer): + self._fixturedef.addfinalizer(finalizer) + + +class ScopeMismatchError(Exception): + """ A fixture function tries to use a different fixture function which + which has a lower scope (e.g. a Session one calls a function one) + """ + + +scopes = "session module class function".split() +scopenum_function = scopes.index("function") + + +def scopemismatch(currentscope, newscope): + return scopes.index(newscope) > scopes.index(currentscope) + + +def scope2index(scope, descr, where=None): + """Look up the index of ``scope`` and raise a descriptive value error + if not defined. + """ + try: + return scopes.index(scope) + except ValueError: + raise ValueError( + "{0} {1}has an unsupported scope value '{2}'".format( + descr, 'from {0} '.format(where) if where else '', + scope) + ) + + +class FixtureLookupError(LookupError): + """ could not return a requested Fixture (missing or invalid). """ + + def __init__(self, argname, request, msg=None): + self.argname = argname + self.request = request + self.fixturestack = request._get_fixturestack() + self.msg = msg + + def formatrepr(self): + tblines = [] + addline = tblines.append + stack = [self.request._pyfuncitem.obj] + stack.extend(map(lambda x: x.func, self.fixturestack)) + msg = self.msg + if msg is not None: + # the last fixture raise an error, let's present + # it at the requesting side + stack = stack[:-1] + for function in stack: + fspath, lineno = getfslineno(function) + try: + lines, _ = inspect.getsourcelines(get_real_func(function)) + except (IOError, IndexError, TypeError): + error_msg = "file %s, line %s: source code not available" + addline(error_msg % (fspath, lineno + 1)) + else: + addline("file %s, line %s" % (fspath, lineno + 1)) + for i, line in enumerate(lines): + line = line.rstrip() + addline(" " + line) + if line.lstrip().startswith('def'): + break + + if msg is None: + fm = self.request._fixturemanager + available = [] + parentid = self.request._pyfuncitem.parent.nodeid + for name, fixturedefs in fm._arg2fixturedefs.items(): + faclist = list(fm._matchfactories(fixturedefs, parentid)) + if faclist and name not in available: + available.append(name) + msg = "fixture %r not found" % (self.argname,) + msg += "\n available fixtures: %s" % (", ".join(sorted(available)),) + msg += "\n use 'pytest --fixtures [testpath]' for help on them." + + return FixtureLookupErrorRepr(fspath, lineno, tblines, msg, self.argname) + + +class FixtureLookupErrorRepr(TerminalRepr): + def __init__(self, filename, firstlineno, tblines, errorstring, argname): + self.tblines = tblines + self.errorstring = errorstring + self.filename = filename + self.firstlineno = firstlineno + self.argname = argname + + def toterminal(self, tw): + # tw.line("FixtureLookupError: %s" %(self.argname), red=True) + for tbline in self.tblines: + tw.line(tbline.rstrip()) + lines = self.errorstring.split("\n") + if lines: + tw.line('{0} {1}'.format(FormattedExcinfo.fail_marker, + lines[0].strip()), red=True) + for line in lines[1:]: + tw.line('{0} {1}'.format(FormattedExcinfo.flow_marker, + line.strip()), red=True) + tw.line() + tw.line("%s:%d" % (self.filename, self.firstlineno + 1)) + + +def fail_fixturefunc(fixturefunc, msg): + fs, lineno = getfslineno(fixturefunc) + location = "%s:%s" % (fs, lineno + 1) + source = _pytest._code.Source(fixturefunc) + fail(msg + ":\n\n" + str(source.indent()) + "\n" + location, + pytrace=False) + + +def call_fixture_func(fixturefunc, request, kwargs): + yieldctx = is_generator(fixturefunc) + if yieldctx: + it = fixturefunc(**kwargs) + res = next(it) + + def teardown(): + try: + next(it) + except StopIteration: + pass + else: + fail_fixturefunc(fixturefunc, + "yield_fixture function has more than one 'yield'") + + request.addfinalizer(teardown) + else: + res = fixturefunc(**kwargs) + return res + + +class FixtureDef: + """ A container for a factory definition. """ + + def __init__(self, fixturemanager, baseid, argname, func, scope, params, + unittest=False, ids=None): + self._fixturemanager = fixturemanager + self.baseid = baseid or '' + self.has_location = baseid is not None + self.func = func + self.argname = argname + self.scope = scope + self.scopenum = scope2index( + scope or "function", + descr='fixture {0}'.format(func.__name__), + where=baseid + ) + self.params = params + self.argnames = getfuncargnames(func, is_method=unittest) + self.unittest = unittest + self.ids = ids + self._finalizers = [] + + def addfinalizer(self, finalizer): + self._finalizers.append(finalizer) + + def finish(self, request): + exceptions = [] + try: + while self._finalizers: + try: + func = self._finalizers.pop() + func() + except: # noqa + exceptions.append(sys.exc_info()) + if exceptions: + e = exceptions[0] + del exceptions # ensure we don't keep all frames alive because of the traceback + py.builtin._reraise(*e) + + finally: + hook = self._fixturemanager.session.gethookproxy(request.node.fspath) + hook.pytest_fixture_post_finalizer(fixturedef=self, request=request) + # even if finalization fails, we invalidate + # the cached fixture value and remove + # all finalizers because they may be bound methods which will + # keep instances alive + if hasattr(self, "cached_result"): + del self.cached_result + self._finalizers = [] + + def execute(self, request): + # get required arguments and register our own finish() + # with their finalization + for argname in self.argnames: + fixturedef = request._get_active_fixturedef(argname) + if argname != "request": + fixturedef.addfinalizer(functools.partial(self.finish, request=request)) + + my_cache_key = request.param_index + cached_result = getattr(self, "cached_result", None) + if cached_result is not None: + result, cache_key, err = cached_result + if my_cache_key == cache_key: + if err is not None: + py.builtin._reraise(*err) + else: + return result + # we have a previous but differently parametrized fixture instance + # so we need to tear it down before creating a new one + self.finish(request) + assert not hasattr(self, "cached_result") + + hook = self._fixturemanager.session.gethookproxy(request.node.fspath) + return hook.pytest_fixture_setup(fixturedef=self, request=request) + + def __repr__(self): + return ("" % + (self.argname, self.scope, self.baseid)) + + +def pytest_fixture_setup(fixturedef, request): + """ Execution of fixture setup. """ + kwargs = {} + for argname in fixturedef.argnames: + fixdef = request._get_active_fixturedef(argname) + result, arg_cache_key, exc = fixdef.cached_result + request._check_scope(argname, request.scope, fixdef.scope) + kwargs[argname] = result + + fixturefunc = fixturedef.func + if fixturedef.unittest: + if request.instance is not None: + # bind the unbound method to the TestCase instance + fixturefunc = fixturedef.func.__get__(request.instance) + else: + # the fixture function needs to be bound to the actual + # request.instance so that code working with "fixturedef" behaves + # as expected. + if request.instance is not None: + fixturefunc = getimfunc(fixturedef.func) + if fixturefunc != fixturedef.func: + fixturefunc = fixturefunc.__get__(request.instance) + my_cache_key = request.param_index + try: + result = call_fixture_func(fixturefunc, request, kwargs) + except TEST_OUTCOME: + fixturedef.cached_result = (None, my_cache_key, sys.exc_info()) + raise + fixturedef.cached_result = (result, my_cache_key, None) + return result + + +def _ensure_immutable_ids(ids): + if ids is None: + return + if callable(ids): + return ids + return tuple(ids) + + +@attr.s(frozen=True) +class FixtureFunctionMarker(object): + scope = attr.ib() + params = attr.ib(convert=attr.converters.optional(tuple)) + autouse = attr.ib(default=False) + ids = attr.ib(default=None, convert=_ensure_immutable_ids) + name = attr.ib(default=None) + + def __call__(self, function): + if isclass(function): + raise ValueError( + "class fixtures not supported (may be in the future)") + function._pytestfixturefunction = self + return function + + +def fixture(scope="function", params=None, autouse=False, ids=None, name=None): + """ (return a) decorator to mark a fixture factory function. + + This decorator can be used (with or without parameters) to define a + fixture function. The name of the fixture function can later be + referenced to cause its invocation ahead of running tests: test + modules or classes can use the pytest.mark.usefixtures(fixturename) + marker. Test functions can directly use fixture names as input + arguments in which case the fixture instance returned from the fixture + function will be injected. + + :arg scope: the scope for which this fixture is shared, one of + "function" (default), "class", "module" or "session". + + :arg params: an optional list of parameters which will cause multiple + invocations of the fixture function and all of the tests + using it. + + :arg autouse: if True, the fixture func is activated for all tests that + can see it. If False (the default) then an explicit + reference is needed to activate the fixture. + + :arg ids: list of string ids each corresponding to the params + so that they are part of the test id. If no ids are provided + they will be generated automatically from the params. + + :arg name: the name of the fixture. This defaults to the name of the + decorated function. If a fixture is used in the same module in + which it is defined, the function name of the fixture will be + shadowed by the function arg that requests the fixture; one way + to resolve this is to name the decorated function + ``fixture_`` and then use + ``@pytest.fixture(name='')``. + + Fixtures can optionally provide their values to test functions using a ``yield`` statement, + instead of ``return``. In this case, the code block after the ``yield`` statement is executed + as teardown code regardless of the test outcome. A fixture function must yield exactly once. + """ + if callable(scope) and params is None and autouse is False: + # direct decoration + return FixtureFunctionMarker( + "function", params, autouse, name=name)(scope) + if params is not None and not isinstance(params, (list, tuple)): + params = list(params) + return FixtureFunctionMarker(scope, params, autouse, ids=ids, name=name) + + +def yield_fixture(scope="function", params=None, autouse=False, ids=None, name=None): + """ (return a) decorator to mark a yield-fixture factory function. + + .. deprecated:: 3.0 + Use :py:func:`pytest.fixture` directly instead. + """ + if callable(scope) and params is None and not autouse: + # direct decoration + return FixtureFunctionMarker( + "function", params, autouse, ids=ids, name=name)(scope) + else: + return FixtureFunctionMarker(scope, params, autouse, ids=ids, name=name) + + +defaultfuncargprefixmarker = fixture() + + +@fixture(scope="session") +def pytestconfig(request): + """ the pytest config object with access to command line opts.""" + return request.config + + +class FixtureManager: + """ + pytest fixtures definitions and information is stored and managed + from this class. + + During collection fm.parsefactories() is called multiple times to parse + fixture function definitions into FixtureDef objects and internal + data structures. + + During collection of test functions, metafunc-mechanics instantiate + a FuncFixtureInfo object which is cached per node/func-name. + This FuncFixtureInfo object is later retrieved by Function nodes + which themselves offer a fixturenames attribute. + + The FuncFixtureInfo object holds information about fixtures and FixtureDefs + relevant for a particular function. An initial list of fixtures is + assembled like this: + + - ini-defined usefixtures + - autouse-marked fixtures along the collection chain up from the function + - usefixtures markers at module/class/function level + - test function funcargs + + Subsequently the funcfixtureinfo.fixturenames attribute is computed + as the closure of the fixtures needed to setup the initial fixtures, + i. e. fixtures needed by fixture functions themselves are appended + to the fixturenames list. + + Upon the test-setup phases all fixturenames are instantiated, retrieved + by a lookup of their FuncFixtureInfo. + """ + + _argprefix = "pytest_funcarg__" + FixtureLookupError = FixtureLookupError + FixtureLookupErrorRepr = FixtureLookupErrorRepr + + def __init__(self, session): + self.session = session + self.config = session.config + self._arg2fixturedefs = {} + self._holderobjseen = set() + self._arg2finish = {} + self._nodeid_and_autousenames = [("", self.config.getini("usefixtures"))] + session.config.pluginmanager.register(self, "funcmanage") + + def getfixtureinfo(self, node, func, cls, funcargs=True): + if funcargs and not hasattr(node, "nofuncargs"): + argnames = getfuncargnames(func, cls=cls) + else: + argnames = () + usefixtures = getattr(func, "usefixtures", None) + initialnames = argnames + if usefixtures is not None: + initialnames = usefixtures.args + initialnames + fm = node.session._fixturemanager + names_closure, arg2fixturedefs = fm.getfixtureclosure(initialnames, + node) + return FuncFixtureInfo(argnames, names_closure, arg2fixturedefs) + + def pytest_plugin_registered(self, plugin): + nodeid = None + try: + p = py.path.local(plugin.__file__) + except AttributeError: + pass + else: + # construct the base nodeid which is later used to check + # what fixtures are visible for particular tests (as denoted + # by their test id) + if p.basename.startswith("conftest.py"): + nodeid = p.dirpath().relto(self.config.rootdir) + if p.sep != nodes.SEP: + nodeid = nodeid.replace(p.sep, nodes.SEP) + self.parsefactories(plugin, nodeid) + + def _getautousenames(self, nodeid): + """ return a tuple of fixture names to be used. """ + autousenames = [] + for baseid, basenames in self._nodeid_and_autousenames: + if nodeid.startswith(baseid): + if baseid: + i = len(baseid) + nextchar = nodeid[i:i + 1] + if nextchar and nextchar not in ":/": + continue + autousenames.extend(basenames) + # make sure autousenames are sorted by scope, scopenum 0 is session + autousenames.sort( + key=lambda x: self._arg2fixturedefs[x][-1].scopenum) + return autousenames + + def getfixtureclosure(self, fixturenames, parentnode): + # collect the closure of all fixtures , starting with the given + # fixturenames as the initial set. As we have to visit all + # factory definitions anyway, we also return a arg2fixturedefs + # mapping so that the caller can reuse it and does not have + # to re-discover fixturedefs again for each fixturename + # (discovering matching fixtures for a given name/node is expensive) + + parentid = parentnode.nodeid + fixturenames_closure = self._getautousenames(parentid) + + def merge(otherlist): + for arg in otherlist: + if arg not in fixturenames_closure: + fixturenames_closure.append(arg) + + merge(fixturenames) + arg2fixturedefs = {} + lastlen = -1 + while lastlen != len(fixturenames_closure): + lastlen = len(fixturenames_closure) + for argname in fixturenames_closure: + if argname in arg2fixturedefs: + continue + fixturedefs = self.getfixturedefs(argname, parentid) + if fixturedefs: + arg2fixturedefs[argname] = fixturedefs + merge(fixturedefs[-1].argnames) + return fixturenames_closure, arg2fixturedefs + + def pytest_generate_tests(self, metafunc): + for argname in metafunc.fixturenames: + faclist = metafunc._arg2fixturedefs.get(argname) + if faclist: + fixturedef = faclist[-1] + if fixturedef.params is not None: + parametrize_func = getattr(metafunc.function, 'parametrize', None) + func_params = getattr(parametrize_func, 'args', [[None]]) + func_kwargs = getattr(parametrize_func, 'kwargs', {}) + # skip directly parametrized arguments + if "argnames" in func_kwargs: + argnames = parametrize_func.kwargs["argnames"] + else: + argnames = func_params[0] + if not isinstance(argnames, (tuple, list)): + argnames = [x.strip() for x in argnames.split(",") if x.strip()] + if argname not in func_params and argname not in argnames: + metafunc.parametrize(argname, fixturedef.params, + indirect=True, scope=fixturedef.scope, + ids=fixturedef.ids) + else: + continue # will raise FixtureLookupError at setup time + + def pytest_collection_modifyitems(self, items): + # separate parametrized setups + items[:] = reorder_items(items) + + def parsefactories(self, node_or_obj, nodeid=NOTSET, unittest=False): + if nodeid is not NOTSET: + holderobj = node_or_obj + else: + holderobj = node_or_obj.obj + nodeid = node_or_obj.nodeid + if holderobj in self._holderobjseen: + return + self._holderobjseen.add(holderobj) + autousenames = [] + for name in dir(holderobj): + # The attribute can be an arbitrary descriptor, so the attribute + # access below can raise. safe_getatt() ignores such exceptions. + obj = safe_getattr(holderobj, name, None) + # fixture functions have a pytest_funcarg__ prefix (pre-2.3 style) + # or are "@pytest.fixture" marked + marker = getfixturemarker(obj) + if marker is None: + if not name.startswith(self._argprefix): + continue + if not callable(obj): + continue + marker = defaultfuncargprefixmarker + from _pytest import deprecated + self.config.warn('C1', deprecated.FUNCARG_PREFIX.format(name=name), nodeid=nodeid) + name = name[len(self._argprefix):] + elif not isinstance(marker, FixtureFunctionMarker): + # magic globals with __getattr__ might have got us a wrong + # fixture attribute + continue + else: + if marker.name: + name = marker.name + msg = 'fixtures cannot have "pytest_funcarg__" prefix ' \ + 'and be decorated with @pytest.fixture:\n%s' % name + assert not name.startswith(self._argprefix), msg + + fixture_def = FixtureDef(self, nodeid, name, obj, + marker.scope, marker.params, + unittest=unittest, ids=marker.ids) + + faclist = self._arg2fixturedefs.setdefault(name, []) + if fixture_def.has_location: + faclist.append(fixture_def) + else: + # fixturedefs with no location are at the front + # so this inserts the current fixturedef after the + # existing fixturedefs from external plugins but + # before the fixturedefs provided in conftests. + i = len([f for f in faclist if not f.has_location]) + faclist.insert(i, fixture_def) + if marker.autouse: + autousenames.append(name) + + if autousenames: + self._nodeid_and_autousenames.append((nodeid or '', autousenames)) + + def getfixturedefs(self, argname, nodeid): + """ + Gets a list of fixtures which are applicable to the given node id. + + :param str argname: name of the fixture to search for + :param str nodeid: full node id of the requesting test. + :return: list[FixtureDef] + """ + try: + fixturedefs = self._arg2fixturedefs[argname] + except KeyError: + return None + else: + return tuple(self._matchfactories(fixturedefs, nodeid)) + + def _matchfactories(self, fixturedefs, nodeid): + for fixturedef in fixturedefs: + if nodes.ischildnode(fixturedef.baseid, nodeid): + yield fixturedef diff --git a/_pytest/freeze_support.py b/_pytest/freeze_support.py new file mode 100644 index 000000000000000..97147a88250a355 --- /dev/null +++ b/_pytest/freeze_support.py @@ -0,0 +1,43 @@ +""" +Provides a function to report all internal modules for using freezing tools +pytest +""" +from __future__ import absolute_import, division, print_function + + +def freeze_includes(): + """ + Returns a list of module names used by py.test that should be + included by cx_freeze. + """ + import py + import _pytest + result = list(_iter_all_modules(py)) + result += list(_iter_all_modules(_pytest)) + return result + + +def _iter_all_modules(package, prefix=''): + """ + Iterates over the names of all modules that can be found in the given + package, recursively. + Example: + _iter_all_modules(_pytest) -> + ['_pytest.assertion.newinterpret', + '_pytest.capture', + '_pytest.core', + ... + ] + """ + import os + import pkgutil + if type(package) is not str: + path, prefix = package.__path__[0], package.__name__ + '.' + else: + path = package + for _, name, is_package in pkgutil.iter_modules([path]): + if is_package: + for m in _iter_all_modules(os.path.join(path, name), prefix=name + '.'): + yield prefix + m + else: + yield prefix + name diff --git a/_pytest/genscript.py b/_pytest/genscript.py deleted file mode 100755 index d2962d8fc82f020..000000000000000 --- a/_pytest/genscript.py +++ /dev/null @@ -1,132 +0,0 @@ -""" (deprecated) generate a single-file self-contained version of pytest """ -import os -import sys -import pkgutil - -import py -import _pytest - - - -def find_toplevel(name): - for syspath in sys.path: - base = py.path.local(syspath) - lib = base/name - if lib.check(dir=1): - return lib - mod = base.join("%s.py" % name) - if mod.check(file=1): - return mod - raise LookupError(name) - -def pkgname(toplevel, rootpath, path): - parts = path.parts()[len(rootpath.parts()):] - return '.'.join([toplevel] + [x.purebasename for x in parts]) - -def pkg_to_mapping(name): - toplevel = find_toplevel(name) - name2src = {} - if toplevel.check(file=1): # module - name2src[toplevel.purebasename] = toplevel.read() - else: # package - for pyfile in toplevel.visit('*.py'): - pkg = pkgname(name, toplevel, pyfile) - name2src[pkg] = pyfile.read() - # with wheels py source code might be not be installed - # and the resulting genscript is useless, just bail out. - assert name2src, "no source code found for %r at %r" %(name, toplevel) - return name2src - -def compress_mapping(mapping): - import base64, pickle, zlib - data = pickle.dumps(mapping, 2) - data = zlib.compress(data, 9) - data = base64.encodestring(data) - data = data.decode('ascii') - return data - - -def compress_packages(names): - mapping = {} - for name in names: - mapping.update(pkg_to_mapping(name)) - return compress_mapping(mapping) - -def generate_script(entry, packages): - data = compress_packages(packages) - tmpl = py.path.local(__file__).dirpath().join('standalonetemplate.py') - exe = tmpl.read() - exe = exe.replace('@SOURCES@', data) - exe = exe.replace('@ENTRY@', entry) - return exe - - -def pytest_addoption(parser): - group = parser.getgroup("debugconfig") - group.addoption("--genscript", action="store", default=None, - dest="genscript", metavar="path", - help="create standalone pytest script at given target path.") - -def pytest_cmdline_main(config): - import _pytest.config - genscript = config.getvalue("genscript") - if genscript: - tw = _pytest.config.create_terminal_writer(config) - tw.line("WARNING: usage of genscript is deprecated.", - red=True) - deps = ['py', '_pytest', 'pytest'] # pluggy is vendored - if sys.version_info < (2,7): - deps.append("argparse") - tw.line("generated script will run on python2.6-python3.3++") - else: - tw.line("WARNING: generated script will not run on python2.6 " - "due to 'argparse' dependency. Use python2.6 " - "to generate a python2.6 compatible script", red=True) - script = generate_script( - 'import pytest; raise SystemExit(pytest.cmdline.main())', - deps, - ) - genscript = py.path.local(genscript) - genscript.write(script) - tw.line("generated pytest standalone script: %s" % genscript, - bold=True) - return 0 - - -def pytest_namespace(): - return {'freeze_includes': freeze_includes} - - -def freeze_includes(): - """ - Returns a list of module names used by py.test that should be - included by cx_freeze. - """ - result = list(_iter_all_modules(py)) - result += list(_iter_all_modules(_pytest)) - return result - - -def _iter_all_modules(package, prefix=''): - """ - Iterates over the names of all modules that can be found in the given - package, recursively. - - Example: - _iter_all_modules(_pytest) -> - ['_pytest.assertion.newinterpret', - '_pytest.capture', - '_pytest.core', - ... - ] - """ - if type(package) is not str: - path, prefix = package.__path__[0], package.__name__ + '.' - else: - path = package - for _, name, is_package in pkgutil.iter_modules([path]): - if is_package: - for m in _iter_all_modules(os.path.join(path, name), prefix=name + '.'): - yield prefix + m - else: - yield prefix + name diff --git a/_pytest/helpconfig.py b/_pytest/helpconfig.py index 1df0c56ac7b862a..e744637f8660f27 100644 --- a/_pytest/helpconfig.py +++ b/_pytest/helpconfig.py @@ -1,25 +1,65 @@ """ version info, help messages, tracing configuration. """ +from __future__ import absolute_import, division, print_function + import py import pytest -import os, sys +from _pytest.config import PrintHelp +import os +import sys +from argparse import Action + + +class HelpAction(Action): + """This is an argparse Action that will raise an exception in + order to skip the rest of the argument parsing when --help is passed. + This prevents argparse from quitting due to missing required arguments + when any are defined, for example by ``pytest_addoption``. + This is similar to the way that the builtin argparse --help option is + implemented by raising SystemExit. + """ + + def __init__(self, + option_strings, + dest=None, + default=False, + help=None): + super(HelpAction, self).__init__( + option_strings=option_strings, + dest=dest, + const=True, + default=default, + nargs=0, + help=help) + + def __call__(self, parser, namespace, values, option_string=None): + setattr(namespace, self.dest, self.const) + + # We should only skip the rest of the parsing after preparse is done + if getattr(parser._parser, 'after_preparse', False): + raise PrintHelp + def pytest_addoption(parser): group = parser.getgroup('debugconfig') group.addoption('--version', action="store_true", - help="display pytest lib version and import information.") - group._addoption("-h", "--help", action="store_true", dest="help", - help="show help message and configuration info") - group._addoption('-p', action="append", dest="plugins", default = [], - metavar="name", - help="early-load given plugin (multi-allowed). " - "To avoid loading of plugins, use the `no:` prefix, e.g. " - "`no:doctest`.") + help="display pytest lib version and import information.") + group._addoption("-h", "--help", action=HelpAction, dest="help", + help="show help message and configuration info") + group._addoption('-p', action="append", dest="plugins", default=[], + metavar="name", + help="early-load given plugin (multi-allowed). " + "To avoid loading of plugins, use the `no:` prefix, e.g. " + "`no:doctest`.") group.addoption('--traceconfig', '--trace-config', - action="store_true", default=False, - help="trace considerations of conftest.py files."), + action="store_true", default=False, + help="trace considerations of conftest.py files."), group.addoption('--debug', - action="store_true", dest="debug", default=False, - help="store internal tracing debug information in 'pytestdebug.log'.") + action="store_true", dest="debug", default=False, + help="store internal tracing debug information in 'pytestdebug.log'.") + group._addoption( + '-o', '--override-ini', nargs='*', dest="override_ini", + action="append", + help="override config option with option=value style, e.g. `-o xfail_strict=True`.") @pytest.hookimpl(hookwrapper=True) @@ -30,26 +70,29 @@ def pytest_cmdline_parse(): path = os.path.abspath("pytestdebug.log") debugfile = open(path, 'w') debugfile.write("versions pytest-%s, py-%s, " - "python-%s\ncwd=%s\nargs=%s\n\n" %( - pytest.__version__, py.__version__, - ".".join(map(str, sys.version_info)), - os.getcwd(), config._origargs)) + "python-%s\ncwd=%s\nargs=%s\n\n" % ( + pytest.__version__, py.__version__, + ".".join(map(str, sys.version_info)), + os.getcwd(), config._origargs)) config.trace.root.setwriter(debugfile.write) undo_tracing = config.pluginmanager.enable_tracing() sys.stderr.write("writing pytestdebug information to %s\n" % path) + def unset_tracing(): debugfile.close() sys.stderr.write("wrote pytestdebug information to %s\n" % debugfile.name) config.trace.root.setwriter(None) undo_tracing() + config.add_cleanup(unset_tracing) + def pytest_cmdline_main(config): if config.option.version: p = py.path.local(pytest.__file__) sys.stderr.write("This is pytest version %s, imported from %s\n" % - (pytest.__version__, p)) + (pytest.__version__, p)) plugininfo = getpluginversioninfo(config) if plugininfo: for line in plugininfo: @@ -61,15 +104,15 @@ def pytest_cmdline_main(config): config._ensure_unconfigure() return 0 + def showhelp(config): reporter = config.pluginmanager.get_plugin('terminalreporter') tw = reporter._tw tw.write(config._parser.optparser.format_help()) tw.line() tw.line() - #tw.sep( "=", "config file settings") - tw.line("[pytest] ini-options in the next " - "pytest.ini|tox.ini|setup.cfg file:") + tw.line("[pytest] ini-options in the first " + "pytest.ini|tox.ini|setup.cfg file found:") tw.line() for name in config._parser._ininames: @@ -77,7 +120,7 @@ def showhelp(config): if type is None: type = "string" spec = "%s (%s)" % (name, type) - line = " %-24s %s" %(spec, help) + line = " %-24s %s" % (spec, help) tw.line(line[:tw.fullwidth]) tw.line() @@ -92,8 +135,8 @@ def showhelp(config): tw.line() tw.line() - tw.line("to see available markers type: py.test --markers") - tw.line("to see available fixtures type: py.test --fixtures") + tw.line("to see available markers type: pytest --markers") + tw.line("to see available fixtures type: pytest --fixtures") tw.line("(shown according to specified file_or_dir or current dir " "if not specified)") @@ -106,6 +149,7 @@ def showhelp(config): ('pytest_plugins', 'list of plugin names to load'), ] + def getpluginversioninfo(config): lines = [] plugininfo = config.pluginmanager.list_plugin_distinfo() @@ -117,11 +161,12 @@ def getpluginversioninfo(config): lines.append(" " + content) return lines + def pytest_report_header(config): lines = [] if config.option.debug or config.option.traceconfig: lines.append("using: pytest-%s pylib-%s" % - (pytest.__version__,py.__version__)) + (pytest.__version__, py.__version__)) verinfo = getpluginversioninfo(config) if verinfo: @@ -135,5 +180,5 @@ def pytest_report_header(config): r = plugin.__file__ else: r = repr(plugin) - lines.append(" %-20s: %s" %(name, r)) + lines.append(" %-20s: %s" % (name, r)) return lines diff --git a/_pytest/hookspec.py b/_pytest/hookspec.py index 60e9b47d2621cb4..440bf99d375aa6c 100644 --- a/_pytest/hookspec.py +++ b/_pytest/hookspec.py @@ -1,6 +1,6 @@ """ hook specifications for pytest plugins, invoked from main.py and builtin plugins. """ -from _pytest._pluggy import HookspecMarker +from pluggy import HookspecMarker hookspec = HookspecMarker("pytest") @@ -8,6 +8,7 @@ # Initialization hooks called for every plugin # ------------------------------------------------------------------------- + @hookspec(historic=True) def pytest_addhooks(pluginmanager): """called at plugin registration time to allow adding new hooks via a call to @@ -16,11 +17,14 @@ def pytest_addhooks(pluginmanager): @hookspec(historic=True) def pytest_namespace(): - """return dict of name->object to be made globally available in + """ + DEPRECATED: this hook causes direct monkeypatching on pytest, its use is strongly discouraged + return dict of name->object to be made globally available in the pytest namespace. This hook is called at plugin registration time. """ + @hookspec(historic=True) def pytest_plugin_registered(plugin, manager): """ a new pytest plugin got registered. """ @@ -34,7 +38,7 @@ def pytest_addoption(parser): .. note:: This function should be implemented only in plugins or ``conftest.py`` - files situated at the tests root directory due to how py.test + files situated at the tests root directory due to how pytest :ref:`discovers plugins during startup `. :arg parser: To add command line options, call @@ -56,11 +60,20 @@ def pytest_addoption(parser): via (deprecated) ``pytest.config``. """ + @hookspec(historic=True) def pytest_configure(config): - """ called after command line options have been parsed - and all plugins and initial conftest files been loaded. - This hook is called for every plugin. + """ + Allows plugins and conftest files to perform initial configuration. + + This hook is called for every plugin and initial conftest file + after command line options have been parsed. + + After that, the hook is called for other conftest files as they are + imported. + + :arg config: pytest config object + :type config: _pytest.config.Config """ # ------------------------------------------------------------------------- @@ -69,17 +82,25 @@ def pytest_configure(config): # discoverable conftest.py local plugins. # ------------------------------------------------------------------------- + @hookspec(firstresult=True) def pytest_cmdline_parse(pluginmanager, args): - """return initialized config object, parsing the specified args. """ + """return initialized config object, parsing the specified args. + + Stops at first non-None result, see :ref:`firstresult` """ + def pytest_cmdline_preparse(config, args): """(deprecated) modify command line arguments before option parsing. """ + @hookspec(firstresult=True) def pytest_cmdline_main(config): """ called for performing the main command line action. The default - implementation will invoke the configure hooks and runtest_mainloop. """ + implementation will invoke the configure hooks and runtest_mainloop. + + Stops at first non-None result, see :ref:`firstresult` """ + def pytest_load_initial_conftests(early_config, parser, args): """ implements the loading of initial conftest files ahead @@ -92,82 +113,124 @@ def pytest_load_initial_conftests(early_config, parser, args): @hookspec(firstresult=True) def pytest_collection(session): - """ perform the collection protocol for the given session. """ + """ perform the collection protocol for the given session. + + Stops at first non-None result, see :ref:`firstresult` """ + def pytest_collection_modifyitems(session, config, items): """ called after collection has been performed, may filter or re-order the items in-place.""" + def pytest_collection_finish(session): """ called after collection has been performed and modified. """ + @hookspec(firstresult=True) def pytest_ignore_collect(path, config): """ return True to prevent considering this path for collection. This hook is consulted for all files and directories prior to calling more specific hooks. + + Stops at first non-None result, see :ref:`firstresult` """ + @hookspec(firstresult=True) def pytest_collect_directory(path, parent): - """ called before traversing a directory for collection files. """ + """ called before traversing a directory for collection files. + + Stops at first non-None result, see :ref:`firstresult` """ + def pytest_collect_file(path, parent): """ return collection Node or None for the given path. Any new node needs to have the specified ``parent`` as a parent.""" # logging hooks for collection + + def pytest_collectstart(collector): """ collector starts collecting. """ + def pytest_itemcollected(item): """ we just collected a test item. """ + def pytest_collectreport(report): """ collector finished collecting. """ + def pytest_deselected(items): """ called for test items deselected by keyword. """ + @hookspec(firstresult=True) def pytest_make_collect_report(collector): - """ perform ``collector.collect()`` and return a CollectReport. """ + """ perform ``collector.collect()`` and return a CollectReport. + + Stops at first non-None result, see :ref:`firstresult` """ # ------------------------------------------------------------------------- # Python test function related hooks # ------------------------------------------------------------------------- + @hookspec(firstresult=True) def pytest_pycollect_makemodule(path, parent): """ return a Module collector or None for the given path. This hook will be called for each matching test module path. The pytest_collect_file hook needs to be used if you want to create test modules for files that do not match as a test module. - """ + + Stops at first non-None result, see :ref:`firstresult` """ + @hookspec(firstresult=True) def pytest_pycollect_makeitem(collector, name, obj): - """ return custom item/collector for a python object in a module, or None. """ + """ return custom item/collector for a python object in a module, or None. + + Stops at first non-None result, see :ref:`firstresult` """ + @hookspec(firstresult=True) def pytest_pyfunc_call(pyfuncitem): - """ call underlying test function. """ + """ call underlying test function. + + Stops at first non-None result, see :ref:`firstresult` """ + def pytest_generate_tests(metafunc): """ generate (multiple) parametrized calls to a test function.""" + +@hookspec(firstresult=True) +def pytest_make_parametrize_id(config, val, argname): + """Return a user-friendly string representation of the given ``val`` that will be used + by @pytest.mark.parametrize calls. Return None if the hook doesn't know about ``val``. + The parameter name is available as ``argname``, if required. + + Stops at first non-None result, see :ref:`firstresult` """ + # ------------------------------------------------------------------------- # generic runtest related hooks # ------------------------------------------------------------------------- + @hookspec(firstresult=True) def pytest_runtestloop(session): """ called for performing the main runtest loop - (after collection finished). """ + (after collection finished). + + Stops at first non-None result, see :ref:`firstresult` """ + def pytest_itemstart(item, node): """ (deprecated, use pytest_runtest_logstart). """ + @hookspec(firstresult=True) def pytest_runtest_protocol(item, nextitem): """ implements the runtest_setup/call/teardown protocol for @@ -181,17 +244,23 @@ def pytest_runtest_protocol(item, nextitem): :py:func:`pytest_runtest_teardown`. :return boolean: True if no further hook implementations should be invoked. - """ + + + Stops at first non-None result, see :ref:`firstresult` """ + def pytest_runtest_logstart(nodeid, location): """ signal the start of running a single test item. """ + def pytest_runtest_setup(item): """ called before ``pytest_runtest_call(item)``. """ + def pytest_runtest_call(item): """ called to execute the test ``item``. """ + def pytest_runtest_teardown(item, nextitem): """ called after ``pytest_runtest_call``. @@ -201,33 +270,56 @@ def pytest_runtest_teardown(item, nextitem): so that nextitem only needs to call setup-functions. """ + @hookspec(firstresult=True) def pytest_runtest_makereport(item, call): """ return a :py:class:`_pytest.runner.TestReport` object - for the given :py:class:`pytest.Item` and + for the given :py:class:`pytest.Item <_pytest.main.Item>` and :py:class:`_pytest.runner.CallInfo`. - """ + + Stops at first non-None result, see :ref:`firstresult` """ + def pytest_runtest_logreport(report): """ process a test setup/call/teardown report relating to the respective phase of executing a test. """ +# ------------------------------------------------------------------------- +# Fixture related hooks +# ------------------------------------------------------------------------- + + +@hookspec(firstresult=True) +def pytest_fixture_setup(fixturedef, request): + """ performs fixture setup execution. + + Stops at first non-None result, see :ref:`firstresult` """ + + +def pytest_fixture_post_finalizer(fixturedef, request): + """ called after fixture teardown, but before the cache is cleared so + the fixture result cache ``fixturedef.cached_result`` can + still be accessed.""" + # ------------------------------------------------------------------------- # test session related hooks # ------------------------------------------------------------------------- + def pytest_sessionstart(session): """ before session.main() is called. """ + def pytest_sessionfinish(session, exitstatus): """ whole test run finishes. """ + def pytest_unconfigure(config): """ called before test process is exited. """ # ------------------------------------------------------------------------- -# hooks for customising the assert methods +# hooks for customizing the assert methods # ------------------------------------------------------------------------- def pytest_assertrepr_compare(config, op, left, right): @@ -236,21 +328,50 @@ def pytest_assertrepr_compare(config, op, left, right): Return None for no custom explanation, otherwise return a list of strings. The strings will be joined by newlines but any newlines *in* a string will be escaped. Note that all but the first line will - be indented sligthly, the intention is for the first line to be a summary. + be indented slightly, the intention is for the first line to be a summary. """ # ------------------------------------------------------------------------- # hooks for influencing reporting (invoked from _pytest_terminal) # ------------------------------------------------------------------------- + def pytest_report_header(config, startdir): - """ return a string to be displayed as header info for terminal reporting.""" + """ return a string or list of strings to be displayed as header info for terminal reporting. + + :param config: the pytest config object. + :param startdir: py.path object with the starting dir + + .. note:: + + This function should be implemented only in plugins or ``conftest.py`` + files situated at the tests root directory due to how pytest + :ref:`discovers plugins during startup `. + """ + + +def pytest_report_collectionfinish(config, startdir, items): + """ + .. versionadded:: 3.2 + + return a string or list of strings to be displayed after collection has finished successfully. + + This strings will be displayed after the standard "collected X items" message. + + :param config: the pytest config object. + :param startdir: py.path object with the starting dir + :param items: list of pytest items that are going to be executed; this list should not be modified. + """ + @hookspec(firstresult=True) def pytest_report_teststatus(report): - """ return result-category, shortletter and verbose word for reporting.""" + """ return result-category, shortletter and verbose word for reporting. -def pytest_terminal_summary(terminalreporter): + Stops at first non-None result, see :ref:`firstresult` """ + + +def pytest_terminal_summary(terminalreporter, exitstatus): """ add additional section in terminal summary reporting. """ @@ -264,20 +385,26 @@ def pytest_logwarning(message, code, nodeid, fslocation): # doctest hooks # ------------------------------------------------------------------------- + @hookspec(firstresult=True) def pytest_doctest_prepare_content(content): - """ return processed content for a given doctest""" + """ return processed content for a given doctest + + Stops at first non-None result, see :ref:`firstresult` """ # ------------------------------------------------------------------------- # error handling and internal debugging hooks # ------------------------------------------------------------------------- + def pytest_internalerror(excrepr, excinfo): """ called for internal errors. """ + def pytest_keyboard_interrupt(excinfo): """ called for keyboard interrupt. """ + def pytest_exception_interact(node, call, report): """called when an exception was raised which can potentially be interactively handled. @@ -286,6 +413,7 @@ def pytest_exception_interact(node, call, report): that is not an internal exception like ``skip.Exception``. """ + def pytest_enter_pdb(config): """ called upon pdb.set_trace(), can be used by plugins to take special action just before the python debugger enters in interactive mode. diff --git a/_pytest/impl b/_pytest/impl deleted file mode 100644 index 889e37e5abcdfc4..000000000000000 --- a/_pytest/impl +++ /dev/null @@ -1,254 +0,0 @@ -Sorting per-resource ------------------------------ - -for any given set of items: - -- collect items per session-scoped parametrized funcarg -- re-order until items no parametrizations are mixed - - examples: - - test() - test1(s1) - test1(s2) - test2() - test3(s1) - test3(s2) - - gets sorted to: - - test() - test2() - test1(s1) - test3(s1) - test1(s2) - test3(s2) - - -the new @setup functions --------------------------------------- - -Consider a given @setup-marked function:: - - @pytest.mark.setup(maxscope=SCOPE) - def mysetup(request, arg1, arg2, ...) - ... - request.addfinalizer(fin) - ... - -then FUNCARGSET denotes the set of (arg1, arg2, ...) funcargs and -all of its dependent funcargs. The mysetup function will execute -for any matching test item once per scope. - -The scope is determined as the minimum scope of all scopes of the args -in FUNCARGSET and the given "maxscope". - -If mysetup has been called and no finalizers have been called it is -called "active". - -Furthermore the following rules apply: - -- if an arg value in FUNCARGSET is about to be torn down, the - mysetup-registered finalizers will execute as well. - -- There will never be two active mysetup invocations. - -Example 1, session scope:: - - @pytest.mark.funcarg(scope="session", params=[1,2]) - def db(request): - request.addfinalizer(db_finalize) - - @pytest.mark.setup - def mysetup(request, db): - request.addfinalizer(mysetup_finalize) - ... - -And a given test module: - - def test_something(): - ... - def test_otherthing(): - pass - -Here is what happens:: - - db(request) executes with request.param == 1 - mysetup(request, db) executes - test_something() executes - test_otherthing() executes - mysetup_finalize() executes - db_finalize() executes - db(request) executes with request.param == 2 - mysetup(request, db) executes - test_something() executes - test_otherthing() executes - mysetup_finalize() executes - db_finalize() executes - -Example 2, session/function scope:: - - @pytest.mark.funcarg(scope="session", params=[1,2]) - def db(request): - request.addfinalizer(db_finalize) - - @pytest.mark.setup(scope="function") - def mysetup(request, db): - ... - request.addfinalizer(mysetup_finalize) - ... - -And a given test module: - - def test_something(): - ... - def test_otherthing(): - pass - -Here is what happens:: - - db(request) executes with request.param == 1 - mysetup(request, db) executes - test_something() executes - mysetup_finalize() executes - mysetup(request, db) executes - test_otherthing() executes - mysetup_finalize() executes - db_finalize() executes - db(request) executes with request.param == 2 - mysetup(request, db) executes - test_something() executes - mysetup_finalize() executes - mysetup(request, db) executes - test_otherthing() executes - mysetup_finalize() executes - db_finalize() executes - - -Example 3 - funcargs session-mix ----------------------------------------- - -Similar with funcargs, an example:: - - @pytest.mark.funcarg(scope="session", params=[1,2]) - def db(request): - request.addfinalizer(db_finalize) - - @pytest.mark.funcarg(scope="function") - def table(request, db): - ... - request.addfinalizer(table_finalize) - ... - -And a given test module: - - def test_something(table): - ... - def test_otherthing(table): - pass - def test_thirdthing(): - pass - -Here is what happens:: - - db(request) executes with param == 1 - table(request, db) - test_something(table) - table_finalize() - table(request, db) - test_otherthing(table) - table_finalize() - db_finalize - db(request) executes with param == 2 - table(request, db) - test_something(table) - table_finalize() - table(request, db) - test_otherthing(table) - table_finalize() - db_finalize - test_thirdthing() - -Data structures --------------------- - -pytest internally maintains a dict of active funcargs with cache, param, -finalizer, (scopeitem?) information: - - active_funcargs = dict() - -if a parametrized "db" is activated: - - active_funcargs["db"] = FuncargInfo(dbvalue, paramindex, - FuncargFinalize(...), scopeitem) - -if a test is torn down and the next test requires a differently -parametrized "db": - - for argname in item.callspec.params: - if argname in active_funcargs: - funcarginfo = active_funcargs[argname] - if funcarginfo.param != item.callspec.params[argname]: - funcarginfo.callfinalizer() - del node2funcarg[funcarginfo.scopeitem] - del active_funcargs[argname] - nodes_to_be_torn_down = ... - for node in nodes_to_be_torn_down: - if node in node2funcarg: - argname = node2funcarg[node] - active_funcargs[argname].callfinalizer() - del node2funcarg[node] - del active_funcargs[argname] - -if a test is setup requiring a "db" funcarg: - - if "db" in active_funcargs: - return active_funcargs["db"][0] - funcarginfo = setup_funcarg() - active_funcargs["db"] = funcarginfo - node2funcarg[funcarginfo.scopeitem] = "db" - -Implementation plan for resources ------------------------------------------- - -1. Revert FuncargRequest to the old form, unmerge item/request - (done) -2. make funcarg factories be discovered at collection time -3. Introduce funcarg marker -4. Introduce funcarg scope parameter -5. Introduce funcarg parametrize parameter -6. make setup functions be discovered at collection time -7. (Introduce a pytest_fixture_protocol/setup_funcargs hook) - -methods and data structures --------------------------------- - -A FuncarcManager holds all information about funcarg definitions -including parametrization and scope definitions. It implements -a pytest_generate_tests hook which performs parametrization as appropriate. - -as a simple example, let's consider a tree where a test function requires -a "abc" funcarg and its factory defines it as parametrized and scoped -for Modules. When collections hits the function item, it creates -the metafunc object, and calls funcargdb.pytest_generate_tests(metafunc) -which looks up available funcarg factories and their scope and parametrization. -This information is equivalent to what can be provided today directly -at the function site and it should thus be relatively straight forward -to implement the additional way of defining parametrization/scoping. - -conftest loading: - each funcarg-factory will populate the session.funcargmanager - -When a test item is collected, it grows a dictionary -(funcargname2factorycalllist). A factory lookup is performed -for each required funcarg. The resulting factory call is stored -with the item. If a function is parametrized multiple items are -created with respective factory calls. Else if a factory is parametrized -multiple items and calls to the factory function are created as well. - -At setup time, an item populates a funcargs mapping, mapping names -to values. If a value is funcarg factories are queried for a given item -test functions and setup functions are put in a class -which looks up required funcarg factories. - - diff --git a/_pytest/junitxml.py b/_pytest/junitxml.py index 660d718a6a89e6c..7fb40dc354874f0 100644 --- a/_pytest/junitxml.py +++ b/_pytest/junitxml.py @@ -4,16 +4,21 @@ Based on initial code from Ross Lawley. + +Output conforms to https://github.com/jenkinsci/xunit-plugin/blob/master/ +src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd """ -# Output conforms to https://github.com/jenkinsci/xunit-plugin/blob/master/ -# src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd +from __future__ import absolute_import, division, print_function +import functools import py import os import re import sys import time import pytest +from _pytest import nodes +from _pytest.config import filename_arg # Python 2.X and 3.X compatibility if sys.version_info[0] < 3: @@ -27,6 +32,7 @@ class Junit(py.xml.Namespace): pass + # We need to get the subset of the invalid unicode ranges according to # XML 1.0 which are valid in this python build. Hence we calculate # this dynamically instead of hardcoding it. The spec range of valid @@ -102,6 +108,8 @@ def record_testreport(self, testreport): } if testreport.location[1] is not None: attrs["line"] = testreport.location[1] + if hasattr(testreport, "url"): + attrs["url"] = testreport.url self.attrs = attrs def to_xml(self): @@ -116,19 +124,15 @@ def _add_simple(self, kind, message, data=None): node = kind(data, message=message) self.append(node) - def _write_captured_output(self, report): + def write_captured_output(self, report): for capname in ('out', 'err'): - allcontent = "" - for name, content in report.get_sections("Captured std%s" % - capname): - allcontent += content - if allcontent: + content = getattr(report, 'capstd' + capname) + if content: tag = getattr(Junit, 'system-' + capname) - self.append(tag(bin_xml_escape(allcontent))) + self.append(tag(bin_xml_escape(content))) def append_pass(self, report): self.add_stats('passed') - self._write_captured_output(report) def append_failure(self, report): # msg = str(report.longrepr.reprtraceback.extraline) @@ -147,7 +151,6 @@ def append_failure(self, report): fail = Junit.failure(message=message) fail.append(bin_xml_escape(report.longrepr)) self.append(fail) - self._write_captured_output(report) def append_collect_error(self, report): # msg = str(report.longrepr.reprtraceback.extraline) @@ -159,9 +162,12 @@ def append_collect_skipped(self, report): Junit.skipped, "collection skipped", report.longrepr) def append_error(self, report): + if getattr(report, 'when', None) == 'teardown': + msg = "test teardown failure" + else: + msg = "test setup failure" self._add_simple( - Junit.error, "test setup failure", report.longrepr) - self._write_captured_output(report) + Junit.error, msg, report.longrepr) def append_skipped(self, report): if hasattr(report, "wasxfail"): @@ -176,7 +182,7 @@ def append_skipped(self, report): Junit.skipped("%s:%s: %s" % (filename, lineno, skipreason), type="pytest.skip", message=skipreason)) - self._write_captured_output(report) + self.write_captured_output(report) def finalize(self): data = self.to_xml().unicode(indent=0) @@ -186,8 +192,8 @@ def finalize(self): @pytest.fixture def record_xml_property(request): - """Fixture that adds extra xml properties to the tag for the calling test. - The fixture is callable with (name, value), with value being automatically + """Add extra xml properties to the tag for the calling test. + The fixture is callable with ``(name, value)``, with value being automatically xml-encoded. """ request.node.warn( @@ -212,6 +218,7 @@ def pytest_addoption(parser): action="store", dest="xmlpath", metavar="path", + type=functools.partial(filename_arg, optname="--junitxml"), default=None, help="create junit-xml style report file at given path.") group.addoption( @@ -220,13 +227,14 @@ def pytest_addoption(parser): metavar="str", default=None, help="prepend prefix to classnames in junit-xml output") + parser.addini("junit_suite_name", "Test suite name for JUnit report", default="pytest") def pytest_configure(config): xmlpath = config.option.xmlpath # prevent opening xmllog on slave nodes (xdist) if xmlpath and not hasattr(config, 'slaveinput'): - config._xml = LogXML(xmlpath, config.option.junitprefix) + config._xml = LogXML(xmlpath, config.option.junitprefix, config.getini("junit_suite_name")) config.pluginmanager.register(config._xml) @@ -245,7 +253,7 @@ def mangle_test_address(address): except ValueError: pass # convert file path to dotted path - names[0] = names[0].replace("/", '.') + names[0] = names[0].replace(nodes.SEP, '.') names[0] = _py_ext_re.sub("", names[0]) # put any params back names[-1] += possible_open_bracket + params @@ -253,10 +261,11 @@ def mangle_test_address(address): class LogXML(object): - def __init__(self, logfile, prefix): + def __init__(self, logfile, prefix, suite_name="pytest"): logfile = os.path.expanduser(os.path.expandvars(logfile)) self.logfile = os.path.normpath(os.path.abspath(logfile)) self.prefix = prefix + self.suite_name = suite_name self.stats = dict.fromkeys([ 'error', 'passed', @@ -265,6 +274,10 @@ def __init__(self, logfile, prefix): ], 0) self.node_reporters = {} # nodeid -> _NodeReporter self.node_reporters_ordered = [] + self.global_properties = [] + # List of reports that failed on call but teardown is pending. + self.open_reports = [] + self.cnt_double_fail_tests = 0 def finalize(self, report): nodeid = getattr(report, 'nodeid', report) @@ -284,9 +297,12 @@ def node_reporter(self, report): if key in self.node_reporters: # TODO: breasks for --dist=each return self.node_reporters[key] + reporter = _NodeReporter(nodeid, self) + self.node_reporters[key] = reporter self.node_reporters_ordered.append(reporter) + return reporter def add_stats(self, key): @@ -321,14 +337,33 @@ def pytest_runtest_logreport(self, report): -> teardown node2 -> teardown node1 """ + close_report = None if report.passed: if report.when == "call": # ignore setup/teardown reporter = self._opentestcase(report) reporter.append_pass(report) elif report.failed: + if report.when == "teardown": + # The following vars are needed when xdist plugin is used + report_wid = getattr(report, "worker_id", None) + report_ii = getattr(report, "item_index", None) + close_report = next( + (rep for rep in self.open_reports + if (rep.nodeid == report.nodeid and + getattr(rep, "item_index", None) == report_ii and + getattr(rep, "worker_id", None) == report_wid + ) + ), None) + if close_report: + # We need to open new testcase in case we have failure in + # call and error in teardown in order to follow junit + # schema + self.finalize(close_report) + self.cnt_double_fail_tests += 1 reporter = self._opentestcase(report) if report.when == "call": reporter.append_failure(report) + self.open_reports.append(report) else: reporter.append_error(report) elif report.skipped: @@ -336,7 +371,20 @@ def pytest_runtest_logreport(self, report): reporter.append_skipped(report) self.update_testcase_duration(report) if report.when == "teardown": + reporter = self._opentestcase(report) + reporter.write_captured_output(report) self.finalize(report) + report_wid = getattr(report, "worker_id", None) + report_ii = getattr(report, "item_index", None) + close_report = next( + (rep for rep in self.open_reports + if (rep.nodeid == report.nodeid and + getattr(rep, "item_index", None) == report_ii and + getattr(rep, "worker_id", None) == report_wid + ) + ), None) + if close_report: + self.open_reports.remove(close_report) def update_testcase_duration(self, report): """accumulates total duration for nodeid from given report and updates @@ -369,12 +417,15 @@ def pytest_sessionfinish(self): suite_stop_time = time.time() suite_time_delta = suite_stop_time - self.suite_start_time - numtests = self.stats['passed'] + self.stats['failure'] - + numtests = (self.stats['passed'] + self.stats['failure'] + + self.stats['skipped'] + self.stats['error'] - + self.cnt_double_fail_tests) logfile.write('') + logfile.write(Junit.testsuite( + self._get_global_properties_node(), [x.to_xml() for x in self.node_reporters_ordered], - name="pytest", + name=self.suite_name, errors=self.stats['error'], failures=self.stats['failure'], skips=self.stats['skipped'], @@ -385,3 +436,18 @@ def pytest_sessionfinish(self): def pytest_terminal_summary(self, terminalreporter): terminalreporter.write_sep("-", "generated xml file: %s" % (self.logfile)) + + def add_global_property(self, name, value): + self.global_properties.append((str(name), bin_xml_escape(value))) + + def _get_global_properties_node(self): + """Return a Junit node containing custom properties, if any. + """ + if self.global_properties: + return Junit.properties( + [ + Junit.property(name=name, value=value) + for name, value in self.global_properties + ] + ) + return '' diff --git a/_pytest/logging.py b/_pytest/logging.py new file mode 100644 index 000000000000000..ed4db25ad4431ef --- /dev/null +++ b/_pytest/logging.py @@ -0,0 +1,337 @@ +from __future__ import absolute_import, division, print_function + +import logging +from contextlib import closing, contextmanager +import sys +import six + +import pytest +import py + + +DEFAULT_LOG_FORMAT = '%(filename)-25s %(lineno)4d %(levelname)-8s %(message)s' +DEFAULT_LOG_DATE_FORMAT = '%H:%M:%S' + + +def get_option_ini(config, *names): + for name in names: + ret = config.getoption(name) # 'default' arg won't work as expected + if ret is None: + ret = config.getini(name) + if ret: + return ret + + +def pytest_addoption(parser): + """Add options to control log capturing.""" + group = parser.getgroup('logging') + + def add_option_ini(option, dest, default=None, type=None, **kwargs): + parser.addini(dest, default=default, type=type, + help='default value for ' + option) + group.addoption(option, dest=dest, **kwargs) + + add_option_ini( + '--no-print-logs', + dest='log_print', action='store_const', const=False, default=True, + type='bool', + help='disable printing caught logs on failed tests.') + add_option_ini( + '--log-level', + dest='log_level', default=None, + help='logging level used by the logging module') + add_option_ini( + '--log-format', + dest='log_format', default=DEFAULT_LOG_FORMAT, + help='log format as used by the logging module.') + add_option_ini( + '--log-date-format', + dest='log_date_format', default=DEFAULT_LOG_DATE_FORMAT, + help='log date format as used by the logging module.') + add_option_ini( + '--log-cli-level', + dest='log_cli_level', default=None, + help='cli logging level.') + add_option_ini( + '--log-cli-format', + dest='log_cli_format', default=None, + help='log format as used by the logging module.') + add_option_ini( + '--log-cli-date-format', + dest='log_cli_date_format', default=None, + help='log date format as used by the logging module.') + add_option_ini( + '--log-file', + dest='log_file', default=None, + help='path to a file when logging will be written to.') + add_option_ini( + '--log-file-level', + dest='log_file_level', default=None, + help='log file logging level.') + add_option_ini( + '--log-file-format', + dest='log_file_format', default=DEFAULT_LOG_FORMAT, + help='log format as used by the logging module.') + add_option_ini( + '--log-file-date-format', + dest='log_file_date_format', default=DEFAULT_LOG_DATE_FORMAT, + help='log date format as used by the logging module.') + + +@contextmanager +def logging_using_handler(handler, logger=None): + """Context manager that safely registers a given handler.""" + logger = logger or logging.getLogger(logger) + + if handler in logger.handlers: # reentrancy + # Adding the same handler twice would confuse logging system. + # Just don't do that. + yield + else: + logger.addHandler(handler) + try: + yield + finally: + logger.removeHandler(handler) + + +@contextmanager +def catching_logs(handler, formatter=None, + level=logging.NOTSET, logger=None): + """Context manager that prepares the whole logging machinery properly.""" + logger = logger or logging.getLogger(logger) + + if formatter is not None: + handler.setFormatter(formatter) + handler.setLevel(level) + + with logging_using_handler(handler, logger): + orig_level = logger.level + logger.setLevel(min(orig_level, level)) + try: + yield handler + finally: + logger.setLevel(orig_level) + + +class LogCaptureHandler(logging.StreamHandler): + """A logging handler that stores log records and the log text.""" + + def __init__(self): + """Creates a new log handler.""" + logging.StreamHandler.__init__(self, py.io.TextIO()) + self.records = [] + + def emit(self, record): + """Keep the log records in a list in addition to the log text.""" + self.records.append(record) + logging.StreamHandler.emit(self, record) + + +class LogCaptureFixture(object): + """Provides access and control of log capturing.""" + + def __init__(self, item): + """Creates a new funcarg.""" + self._item = item + + @property + def handler(self): + return self._item.catch_log_handler + + @property + def text(self): + """Returns the log text.""" + return self.handler.stream.getvalue() + + @property + def records(self): + """Returns the list of log records.""" + return self.handler.records + + @property + def record_tuples(self): + """Returns a list of a striped down version of log records intended + for use in assertion comparison. + + The format of the tuple is: + + (logger_name, log_level, message) + """ + return [(r.name, r.levelno, r.getMessage()) for r in self.records] + + def clear(self): + """Reset the list of log records.""" + self.handler.records = [] + + def set_level(self, level, logger=None): + """Sets the level for capturing of logs. + + By default, the level is set on the handler used to capture + logs. Specify a logger name to instead set the level of any + logger. + """ + if logger is None: + logger = self.handler + else: + logger = logging.getLogger(logger) + logger.setLevel(level) + + @contextmanager + def at_level(self, level, logger=None): + """Context manager that sets the level for capturing of logs. + + By default, the level is set on the handler used to capture + logs. Specify a logger name to instead set the level of any + logger. + """ + if logger is None: + logger = self.handler + else: + logger = logging.getLogger(logger) + + orig_level = logger.level + logger.setLevel(level) + try: + yield + finally: + logger.setLevel(orig_level) + + +@pytest.fixture +def caplog(request): + """Access and control log capturing. + + Captured logs are available through the following methods:: + + * caplog.text() -> string containing formatted log output + * caplog.records() -> list of logging.LogRecord instances + * caplog.record_tuples() -> list of (logger_name, level, message) tuples + """ + return LogCaptureFixture(request.node) + + +def get_actual_log_level(config, *setting_names): + """Return the actual logging level.""" + + for setting_name in setting_names: + log_level = config.getoption(setting_name) + if log_level is None: + log_level = config.getini(setting_name) + if log_level: + break + else: + return + + if isinstance(log_level, six.string_types): + log_level = log_level.upper() + try: + return int(getattr(logging, log_level, log_level)) + except ValueError: + # Python logging does not recognise this as a logging level + raise pytest.UsageError( + "'{0}' is not recognized as a logging level name for " + "'{1}'. Please consider passing the " + "logging level num instead.".format( + log_level, + setting_name)) + + +def pytest_configure(config): + config.pluginmanager.register(LoggingPlugin(config), + 'logging-plugin') + + +class LoggingPlugin(object): + """Attaches to the logging module and captures log messages for each test. + """ + + def __init__(self, config): + """Creates a new plugin to capture log messages. + + The formatter can be safely shared across all handlers so + create a single one for the entire test session here. + """ + self.log_cli_level = get_actual_log_level( + config, 'log_cli_level', 'log_level') or logging.WARNING + + self.print_logs = get_option_ini(config, 'log_print') + self.formatter = logging.Formatter( + get_option_ini(config, 'log_format'), + get_option_ini(config, 'log_date_format')) + + log_cli_handler = logging.StreamHandler(sys.stderr) + log_cli_format = get_option_ini( + config, 'log_cli_format', 'log_format') + log_cli_date_format = get_option_ini( + config, 'log_cli_date_format', 'log_date_format') + log_cli_formatter = logging.Formatter( + log_cli_format, + datefmt=log_cli_date_format) + self.log_cli_handler = log_cli_handler # needed for a single unittest + self.live_logs = catching_logs(log_cli_handler, + formatter=log_cli_formatter, + level=self.log_cli_level) + + log_file = get_option_ini(config, 'log_file') + if log_file: + self.log_file_level = get_actual_log_level( + config, 'log_file_level') or logging.WARNING + + log_file_format = get_option_ini( + config, 'log_file_format', 'log_format') + log_file_date_format = get_option_ini( + config, 'log_file_date_format', 'log_date_format') + self.log_file_handler = logging.FileHandler( + log_file, + # Each pytest runtests session will write to a clean logfile + mode='w') + log_file_formatter = logging.Formatter( + log_file_format, + datefmt=log_file_date_format) + self.log_file_handler.setFormatter(log_file_formatter) + else: + self.log_file_handler = None + + @contextmanager + def _runtest_for(self, item, when): + """Implements the internals of pytest_runtest_xxx() hook.""" + with catching_logs(LogCaptureHandler(), + formatter=self.formatter) as log_handler: + item.catch_log_handler = log_handler + try: + yield # run test + finally: + del item.catch_log_handler + + if self.print_logs: + # Add a captured log section to the report. + log = log_handler.stream.getvalue().strip() + item.add_report_section(when, 'log', log) + + @pytest.hookimpl(hookwrapper=True) + def pytest_runtest_setup(self, item): + with self._runtest_for(item, 'setup'): + yield + + @pytest.hookimpl(hookwrapper=True) + def pytest_runtest_call(self, item): + with self._runtest_for(item, 'call'): + yield + + @pytest.hookimpl(hookwrapper=True) + def pytest_runtest_teardown(self, item): + with self._runtest_for(item, 'teardown'): + yield + + @pytest.hookimpl(hookwrapper=True) + def pytest_runtestloop(self, session): + """Runs all collected test items.""" + with self.live_logs: + if self.log_file_handler is not None: + with closing(self.log_file_handler): + with catching_logs(self.log_file_handler, + level=self.log_file_level): + yield # run all the tests + else: + yield # run all the tests diff --git a/_pytest/main.py b/_pytest/main.py index 8654d7af627d9e7..25554098dac299f 100644 --- a/_pytest/main.py +++ b/_pytest/main.py @@ -1,18 +1,22 @@ """ core implementation of testing process: init, session, runtest loop. """ -import imp +from __future__ import absolute_import, division, print_function + +import functools import os -import re +import six import sys import _pytest +from _pytest import nodes import _pytest._code import py -import pytest try: from collections import MutableMapping as MappingMixin except ImportError: from UserDict import DictMixin as MappingMixin +from _pytest.config import directory_arg, UsageError, hookimpl +from _pytest.outcomes import exit from _pytest.runner import collect_one_node tracebackcutdir = py.path.local(_pytest.__file__).dirpath() @@ -25,60 +29,65 @@ EXIT_USAGEERROR = 4 EXIT_NOTESTSCOLLECTED = 5 -name_re = re.compile("^[a-zA-Z_]\w*$") def pytest_addoption(parser): parser.addini("norecursedirs", "directory patterns to avoid for recursion", - type="args", default=['.*', 'CVS', '_darcs', '{arch}', '*.egg']) - parser.addini("testpaths", "directories to search for tests when no files or directories are given in the command line.", - type="args", default=[]) - #parser.addini("dirpatterns", + type="args", default=['.*', 'build', 'dist', 'CVS', '_darcs', '{arch}', '*.egg', 'venv']) + parser.addini("testpaths", "directories to search for tests when no files or directories are given in the " + "command line.", + type="args", default=[]) + # parser.addini("dirpatterns", # "patterns specifying possible locations of test files", # type="linelist", default=["**/test_*.txt", # "**/test_*.py", "**/*_test.py"] - #) + # ) group = parser.getgroup("general", "running and selection options") - group._addoption('-x', '--exitfirst', action="store_true", default=False, - dest="exitfirst", - help="exit instantly on first error or failed test."), + group._addoption('-x', '--exitfirst', action="store_const", + dest="maxfail", const=1, + help="exit instantly on first error or failed test."), group._addoption('--maxfail', metavar="num", - action="store", type=int, dest="maxfail", default=0, - help="exit after first num failures or errors.") + action="store", type=int, dest="maxfail", default=0, + help="exit after first num failures or errors.") group._addoption('--strict', action="store_true", - help="run pytest in strict mode, warnings become errors.") + help="marks not registered in configuration file raise errors.") group._addoption("-c", metavar="file", type=str, dest="inifilename", - help="load configuration from `file` instead of trying to locate one of the implicit configuration files.") + help="load configuration from `file` instead of trying to locate one of the implicit " + "configuration files.") + group._addoption("--continue-on-collection-errors", action="store_true", + default=False, dest="continue_on_collection_errors", + help="Force test execution even if collection errors occur.") group = parser.getgroup("collect", "collection") group.addoption('--collectonly', '--collect-only', action="store_true", - help="only collect tests, don't execute them."), + help="only collect tests, don't execute them."), group.addoption('--pyargs', action="store_true", - help="try to interpret all arguments as python packages.") + help="try to interpret all arguments as python packages.") group.addoption("--ignore", action="append", metavar="path", - help="ignore path during collection (multi-allowed).") + help="ignore path during collection (multi-allowed).") # when changing this to --conf-cut-dir, config.py Conftest.setinitial # needs upgrading as well group.addoption('--confcutdir', dest="confcutdir", default=None, - metavar="dir", - help="only load conftest.py's relative to specified dir.") + metavar="dir", type=functools.partial(directory_arg, optname="--confcutdir"), + help="only load conftest.py's relative to specified dir.") group.addoption('--noconftest', action="store_true", - dest="noconftest", default=False, - help="Don't load any conftest.py files.") + dest="noconftest", default=False, + help="Don't load any conftest.py files.") + group.addoption('--keepduplicates', '--keep-duplicates', action="store_true", + dest="keepduplicates", default=False, + help="Keep duplicate tests.") + group.addoption('--collect-in-virtualenv', action='store_true', + dest='collect_in_virtualenv', default=False, + help="Don't ignore tests in a local virtualenv directory") group = parser.getgroup("debugconfig", - "test session debugging and configuration") + "test session debugging and configuration") group.addoption('--basetemp', dest="basetemp", default=None, metavar="dir", - help="base temporary directory for this test run.") - + help="base temporary directory for this test run.") -def pytest_namespace(): - collect = dict(Item=Item, Collector=Collector, File=File, Session=Session) - return dict(collect=collect) def pytest_configure(config): - pytest.config = config # compatibiltiy - if config.option.exitfirst: - config.option.maxfail = 1 + __import__('pytest').config = config # compatibiltiy + def wrap_session(config, doit): """Skeleton command line program""" @@ -92,13 +101,18 @@ def wrap_session(config, doit): config.hook.pytest_sessionstart(session=session) initstate = 2 session.exitstatus = doit(config, session) or 0 - except pytest.UsageError: + except UsageError: raise + except Failed: + session.exitstatus = EXIT_TESTSFAILED except KeyboardInterrupt: excinfo = _pytest._code.ExceptionInfo() + if initstate < 2 and isinstance(excinfo.value, exit.Exception): + sys.stderr.write('{0}: {1}\n'.format( + excinfo.typename, excinfo.value.msg)) config.hook.pytest_keyboard_interrupt(excinfo=excinfo) session.exitstatus = EXIT_INTERRUPTED - except: + except: # noqa excinfo = _pytest._code.ExceptionInfo() config.notify_exception(excinfo, config.option) session.exitstatus = EXIT_INTERNALERROR @@ -115,9 +129,11 @@ def wrap_session(config, doit): config._ensure_unconfigure() return session.exitstatus + def pytest_cmdline_main(config): return wrap_session(config, _main) + def _main(config, session): """ default command line protocol for initialization, session, running tests and reporting. """ @@ -129,37 +145,66 @@ def _main(config, session): elif session.testscollected == 0: return EXIT_NOTESTSCOLLECTED + def pytest_collection(session): return session.perform_collect() + def pytest_runtestloop(session): + if (session.testsfailed and + not session.config.option.continue_on_collection_errors): + raise session.Interrupted( + "%d errors during collection" % session.testsfailed) + if session.config.option.collectonly: return True - def getnextitem(i): - # this is a function to avoid python2 - # keeping sys.exc_info set when calling into a test - # python2 keeps sys.exc_info till the frame is left - try: - return session.items[i+1] - except IndexError: - return None - for i, item in enumerate(session.items): - nextitem = getnextitem(i) + nextitem = session.items[i + 1] if i + 1 < len(session.items) else None item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem) + if session.shouldfail: + raise session.Failed(session.shouldfail) if session.shouldstop: raise session.Interrupted(session.shouldstop) return True + +def _in_venv(path): + """Attempts to detect if ``path`` is the root of a Virtual Environment by + checking for the existence of the appropriate activate script""" + bindir = path.join('Scripts' if sys.platform.startswith('win') else 'bin') + if not bindir.exists(): + return False + activates = ('activate', 'activate.csh', 'activate.fish', + 'Activate', 'Activate.bat', 'Activate.ps1') + return any([fname.basename in activates for fname in bindir.listdir()]) + + def pytest_ignore_collect(path, config): - p = path.dirpath() - ignore_paths = config._getconftest_pathlist("collect_ignore", path=p) + ignore_paths = config._getconftest_pathlist("collect_ignore", path=path.dirpath()) ignore_paths = ignore_paths or [] excludeopt = config.getoption("ignore") if excludeopt: ignore_paths.extend([py.path.local(x) for x in excludeopt]) - return path in ignore_paths + + if py.path.local(path) in ignore_paths: + return True + + allow_in_venv = config.getoption("collect_in_virtualenv") + if _in_venv(path) and not allow_in_venv: + return True + + # Skip duplicate paths. + keepduplicates = config.getoption("keepduplicates") + duplicate_paths = config.pluginmanager._duplicatepaths + if not keepduplicates: + if path in duplicate_paths: + return True + else: + duplicate_paths.add(path) + + return False + class FSHookProxy: def __init__(self, fspath, pm, remove_mods): @@ -172,12 +217,22 @@ def __getattr__(self, name): self.__dict__[name] = x return x -def compatproperty(name): - def fget(self): - # deprecated - use pytest.name - return getattr(pytest, name) - return property(fget) +class _CompatProperty(object): + def __init__(self, name): + self.name = name + + def __get__(self, obj, owner): + if obj is None: + return self + + # TODO: reenable in the features branch + # warnings.warn( + # "usage of {owner!r}.{name} is deprecated, please use pytest.{name} instead".format( + # name=self.name, owner=type(owner).__name__), + # PendingDeprecationWarning, stacklevel=2) + return getattr(__import__('pytest'), self.name) + class NodeKeywords(MappingMixin): def __init__(self, node): @@ -249,24 +304,28 @@ def ihook(self): """ fspath sensitive hook proxy used to call pytest hooks""" return self.session.gethookproxy(self.fspath) - Module = compatproperty("Module") - Class = compatproperty("Class") - Instance = compatproperty("Instance") - Function = compatproperty("Function") - File = compatproperty("File") - Item = compatproperty("Item") + Module = _CompatProperty("Module") + Class = _CompatProperty("Class") + Instance = _CompatProperty("Instance") + Function = _CompatProperty("Function") + File = _CompatProperty("File") + Item = _CompatProperty("Item") def _getcustomclass(self, name): - cls = getattr(self, name) - if cls != getattr(pytest, name): - py.log._apiwarn("2.0", "use of node.%s is deprecated, " - "use pytest_pycollect_makeitem(...) to create custom " - "collection nodes" % name) + maybe_compatprop = getattr(type(self), name) + if isinstance(maybe_compatprop, _CompatProperty): + return getattr(__import__('pytest'), name) + else: + cls = getattr(self, name) + # TODO: reenable in the features branch + # warnings.warn("use of node.%s is deprecated, " + # "use pytest_pycollect_makeitem(...) to create custom " + # "collection nodes" % name, category=DeprecationWarning) return cls def __repr__(self): - return "<%s %r>" %(self.__class__.__name__, - getattr(self, 'name', None)) + return "<%s %r>" % (self.__class__.__name__, + getattr(self, 'name', None)) def warn(self, code, message): """ generate a warning with the given code and message for this @@ -275,9 +334,6 @@ def warn(self, code, message): fslocation = getattr(self, "location", None) if fslocation is None: fslocation = getattr(self, "fspath", None) - else: - fslocation = "%s:%s" % fslocation[:2] - self.ihook.pytest_logwarning.call_historic(kwargs=dict( code=code, message=message, nodeid=self.nodeid, fslocation=fslocation)) @@ -304,24 +360,6 @@ def setup(self): def teardown(self): pass - def _memoizedcall(self, attrname, function): - exattrname = "_ex_" + attrname - failure = getattr(self, exattrname, None) - if failure is not None: - py.builtin._reraise(failure[0], failure[1], failure[2]) - if hasattr(self, attrname): - return getattr(self, attrname) - try: - res = function() - except py.builtin._sysex: - raise - except: - failure = sys.exc_info() - setattr(self, exattrname, failure) - raise - setattr(self, attrname, res) - return res - def listchain(self): """ return list of all parent collectors up to self, starting from root of collection tree. """ @@ -338,9 +376,9 @@ def add_marker(self, marker): ``marker`` can be a string or pytest.mark.* instance. """ - from _pytest.mark import MarkDecorator - if isinstance(marker, py.builtin._basestring): - marker = MarkDecorator(marker) + from _pytest.mark import MarkDecorator, MARK_GEN + if isinstance(marker, six.string_types): + marker = getattr(MARK_GEN, marker) elif not isinstance(marker, MarkDecorator): raise ValueError("is not a string or pytest.mark.* Marker") self.keywords[marker.name] = marker @@ -390,9 +428,12 @@ def _repr_failure_py(self, excinfo, style=None): return excinfo.value.formatrepr() tbfilter = True if self.config.option.fulltrace: - style="long" + style = "long" else: + tb = _pytest._code.Traceback([excinfo.traceback[-1]]) self._prunetraceback(excinfo) + if len(excinfo.traceback) == 0: + excinfo.traceback = tb tbfilter = False # prunetraceback already does it if style == "auto": style = "long" @@ -403,12 +444,19 @@ def _repr_failure_py(self, excinfo, style=None): else: style = "long" - return excinfo.getrepr(funcargs=True, + try: + os.getcwd() + abspath = False + except OSError: + abspath = True + + return excinfo.getrepr(funcargs=True, abspath=abspath, showlocals=self.config.option.showlocals, style=style, tbfilter=tbfilter) repr_failure = _repr_failure_py + class Collector(Node): """ Collector instances create children through collect() and thus iteratively build a tree. @@ -430,10 +478,6 @@ def repr_failure(self, excinfo): return str(exc.args[0]) return self._repr_failure_py(excinfo, style="short") - def _memocollect(self): - """ internal helper method to cache results of calling collect(). """ - return self._memoizedcall('_collected', lambda: list(self.collect())) - def _prunetraceback(self, excinfo): if hasattr(self, 'fspath'): traceback = excinfo.traceback @@ -442,27 +486,38 @@ def _prunetraceback(self, excinfo): ntraceback = ntraceback.cut(excludepath=tracebackcutdir) excinfo.traceback = ntraceback.filter() + class FSCollector(Collector): def __init__(self, fspath, parent=None, config=None, session=None): - fspath = py.path.local(fspath) # xxx only for test_resultlog.py? + fspath = py.path.local(fspath) # xxx only for test_resultlog.py? name = fspath.basename if parent is not None: rel = fspath.relto(parent.fspath) if rel: name = rel - name = name.replace(os.sep, "/") + name = name.replace(os.sep, nodes.SEP) super(FSCollector, self).__init__(name, parent, config, session) self.fspath = fspath + def _check_initialpaths_for_relpath(self): + for initialpath in self.session._initialpaths: + if self.fspath.common(initialpath) == initialpath: + return self.fspath.relto(initialpath.dirname) + def _makeid(self): relpath = self.fspath.relto(self.config.rootdir) - if os.sep != "/": - relpath = relpath.replace(os.sep, "/") + + if not relpath: + relpath = self._check_initialpaths_for_relpath() + if os.sep != nodes.SEP: + relpath = relpath.replace(os.sep, nodes.SEP) return relpath + class File(FSCollector): """ base class for collecting tests from a file. """ + class Item(Node): """ a basic test invocation item. Note that for a single function there might be multiple test invocation items. @@ -474,6 +529,21 @@ def __init__(self, name, parent=None, config=None, session=None): self._report_sections = [] def add_report_section(self, when, key, content): + """ + Adds a new report section, similar to what's done internally to add stdout and + stderr captured output:: + + item.add_report_section("call", "stdout", "report section contents") + + :param str when: + One of the possible capture states, ``"setup"``, ``"call"``, ``"teardown"``. + :param str key: + Name of the section, can be customized at will. Pytest uses ``"stdout"`` and + ``"stderr"`` internally. + + :param str content: + The full contents as a string. + """ if content: self._report_sections.append((when, key, content)) @@ -497,23 +567,31 @@ def location(self): self._location = location return location + class NoMatch(Exception): """ raised if matching cannot locate a matching names. """ + class Interrupted(KeyboardInterrupt): """ signals an interrupted test run. """ - __module__ = 'builtins' # for py3 + __module__ = 'builtins' # for py3 + + +class Failed(Exception): + """ signals an stop as failed test run. """ + class Session(FSCollector): Interrupted = Interrupted + Failed = Failed def __init__(self, config): FSCollector.__init__(self, config.rootdir, parent=None, config=config, session=self) - self._fs2hookproxy = {} self.testsfailed = 0 self.testscollected = 0 self.shouldstop = False + self.shouldfail = False self.trace = config.trace.root.get("collection") self._norecursepatterns = config.getini("norecursedirs") self.startdir = py.path.local() @@ -522,18 +600,20 @@ def __init__(self, config): def _makeid(self): return "" - @pytest.hookimpl(tryfirst=True) + @hookimpl(tryfirst=True) def pytest_collectstart(self): + if self.shouldfail: + raise self.Failed(self.shouldfail) if self.shouldstop: raise self.Interrupted(self.shouldstop) - @pytest.hookimpl(tryfirst=True) + @hookimpl(tryfirst=True) def pytest_runtest_logreport(self, report): if report.failed and not hasattr(report, 'wasxfail'): self.testsfailed += 1 maxfail = self.config.getvalue("maxfail") if maxfail and self.testsfailed >= maxfail: - self.shouldstop = "stopping after %d failures" % ( + self.shouldfail = "stopping after %d failures" % ( self.testsfailed) pytest_collectreport = pytest_runtest_logreport @@ -541,30 +621,26 @@ def isinitpath(self, path): return path in self._initialpaths def gethookproxy(self, fspath): - try: - return self._fs2hookproxy[fspath] - except KeyError: - # check if we have the common case of running - # hooks with all conftest.py filesall conftest.py - pm = self.config.pluginmanager - my_conftestmodules = pm._getconftestmodules(fspath) - remove_mods = pm._conftest_plugins.difference(my_conftestmodules) - if remove_mods: - # one or more conftests are not in use at this fspath - proxy = FSHookProxy(fspath, pm, remove_mods) - else: - # all plugis are active for this fspath - proxy = self.config.hook - - self._fs2hookproxy[fspath] = proxy - return proxy + # check if we have the common case of running + # hooks with all conftest.py filesall conftest.py + pm = self.config.pluginmanager + my_conftestmodules = pm._getconftestmodules(fspath) + remove_mods = pm._conftest_plugins.difference(my_conftestmodules) + if remove_mods: + # one or more conftests are not in use at this fspath + proxy = FSHookProxy(fspath, pm, remove_mods) + else: + # all plugis are active for this fspath + proxy = self.config.hook + return proxy def perform_collect(self, args=None, genitems=True): hook = self.config.hook try: items = self._perform_collect(args, genitems) + self.config.pluginmanager.check_pending() hook.pytest_collection_modifyitems(session=self, - config=self.config, items=items) + config=self.config, items=items) finally: hook.pytest_collection_finish(session=self) self.testscollected = len(items) @@ -591,8 +667,8 @@ def _perform_collect(self, args, genitems): for arg, exc in self._notfound: line = "(no name %r in any of %r)" % (arg, exc.args[0]) errors.append("not found: %s\n%s" % (arg, line)) - #XXX: test this - raise pytest.UsageError(*errors) + # XXX: test this + raise UsageError(*errors) if not genitems: return rep.result else: @@ -620,7 +696,7 @@ def _collect(self, arg): names = self._parsearg(arg) path = names.pop(0) if path.check(dir=1): - assert not names, "invalid arg %r" %(arg,) + assert not names, "invalid arg %r" % (arg,) for path in path.visit(fil=lambda x: x.check(file=1), rec=self._recurse, bf=True, sort=True): for x in self._collectfile(path): @@ -649,44 +725,41 @@ def _recurse(self, path): return True def _tryconvertpyarg(self, x): - mod = None - path = [os.path.abspath('.')] + sys.path - for name in x.split('.'): - # ignore anything that's not a proper name here - # else something like --pyargs will mess up '.' - # since imp.find_module will actually sometimes work for it - # but it's supposed to be considered a filesystem path - # not a package - if name_re.match(name) is None: - return x - try: - fd, mod, type_ = imp.find_module(name, path) - except ImportError: - return x - else: - if fd is not None: - fd.close() + """Convert a dotted module name to path. - if type_[2] != imp.PKG_DIRECTORY: - path = [os.path.dirname(mod)] - else: - path = [mod] - return mod + """ + import pkgutil + try: + loader = pkgutil.find_loader(x) + except ImportError: + return x + if loader is None: + return x + # This method is sometimes invoked when AssertionRewritingHook, which + # does not define a get_filename method, is already in place: + try: + path = loader.get_filename(x) + except AttributeError: + # Retrieve path from AssertionRewritingHook: + path = loader.modules[x][0].co_filename + if loader.is_package(x): + path = os.path.dirname(path) + return path def _parsearg(self, arg): """ return (fspath, names) tuple after checking the file exists. """ - arg = str(arg) - if self.config.option.pyargs: - arg = self._tryconvertpyarg(arg) parts = str(arg).split("::") + if self.config.option.pyargs: + parts[0] = self._tryconvertpyarg(parts[0]) relpath = parts[0].replace("/", os.sep) path = self.config.invocation_dir.join(relpath, abs=True) if not path.check(): if self.config.option.pyargs: - msg = "file or package not found: " + raise UsageError( + "file or package not found: " + arg + + " (missing __init__.py?)") else: - msg = "file not found: " - raise pytest.UsageError(msg + arg) + raise UsageError("file not found: " + arg) parts[0] = path return parts @@ -709,11 +782,11 @@ def _matchnodes(self, matching, names): nextnames = names[1:] resultnodes = [] for node in matching: - if isinstance(node, pytest.Item): + if isinstance(node, Item): if not names: resultnodes.append(node) continue - assert isinstance(node, pytest.Collector) + assert isinstance(node, Collector) rep = collect_one_node(node) if rep.passed: has_matched = False @@ -726,16 +799,20 @@ def _matchnodes(self, matching, names): if not has_matched and len(rep.result) == 1 and x.name == "()": nextnames.insert(0, name) resultnodes.extend(self.matchnodes([x], nextnames)) - node.ihook.pytest_collectreport(report=rep) + else: + # report collection failures here to avoid failing to run some test + # specified in the command line because the module could not be + # imported (#134) + node.ihook.pytest_collectreport(report=rep) return resultnodes def genitems(self, node): self.trace("genitems", node) - if isinstance(node, pytest.Item): + if isinstance(node, Item): node.ihook.pytest_itemcollected(item=node) yield node else: - assert isinstance(node, pytest.Collector) + assert isinstance(node, Collector) rep = collect_one_node(node) if rep.passed: for subnode in rep.result: diff --git a/_pytest/mark.py b/_pytest/mark.py index 1a76354024014ed..3f1f01b1a2e34db 100644 --- a/_pytest/mark.py +++ b/_pytest/mark.py @@ -1,5 +1,97 @@ """ generic mechanism for marking and selecting python functions. """ +from __future__ import absolute_import, division, print_function + import inspect +import warnings +import attr +from collections import namedtuple +from operator import attrgetter +from six.moves import map +from .deprecated import MARK_PARAMETERSET_UNPACKING +from .compat import NOTSET, getfslineno + + +def alias(name, warning=None): + getter = attrgetter(name) + + def warned(self): + warnings.warn(warning, stacklevel=2) + return getter(self) + + return property(getter if warning is None else warned, doc='alias for ' + name) + + +class ParameterSet(namedtuple('ParameterSet', 'values, marks, id')): + @classmethod + def param(cls, *values, **kw): + marks = kw.pop('marks', ()) + if isinstance(marks, MarkDecorator): + marks = marks, + else: + assert isinstance(marks, (tuple, list, set)) + + def param_extract_id(id=None): + return id + + id = param_extract_id(**kw) + return cls(values, marks, id) + + @classmethod + def extract_from(cls, parameterset, legacy_force_tuple=False): + """ + :param parameterset: + a legacy style parameterset that may or may not be a tuple, + and may or may not be wrapped into a mess of mark objects + + :param legacy_force_tuple: + enforce tuple wrapping so single argument tuple values + don't get decomposed and break tests + + """ + + if isinstance(parameterset, cls): + return parameterset + if not isinstance(parameterset, MarkDecorator) and legacy_force_tuple: + return cls.param(parameterset) + + newmarks = [] + argval = parameterset + while isinstance(argval, MarkDecorator): + newmarks.append(MarkDecorator(Mark( + argval.markname, argval.args[:-1], argval.kwargs))) + argval = argval.args[-1] + assert not isinstance(argval, ParameterSet) + if legacy_force_tuple: + argval = argval, + + if newmarks: + warnings.warn(MARK_PARAMETERSET_UNPACKING) + + return cls(argval, marks=newmarks, id=None) + + @classmethod + def _for_parameterize(cls, argnames, argvalues, function): + if not isinstance(argnames, (tuple, list)): + argnames = [x.strip() for x in argnames.split(",") if x.strip()] + force_tuple = len(argnames) == 1 + else: + force_tuple = False + parameters = [ + ParameterSet.extract_from(x, legacy_force_tuple=force_tuple) + for x in argvalues] + del argvalues + + if not parameters: + fs, lineno = getfslineno(function) + reason = "got empty parameter set %r, function %s at %s:%d" % ( + argnames, function.__name__, fs, lineno) + mark = MARK_GEN.skip(reason=reason) + parameters.append(ParameterSet( + values=(NOTSET,) * len(argnames), + marks=[mark], + id=None, + )) + return argnames, parameters class MarkerError(Exception): @@ -7,8 +99,8 @@ class MarkerError(Exception): """Error in use of a pytest marker/attribute.""" -def pytest_namespace(): - return {'mark': MarkGenerator()} +def param(*values, **kw): + return ParameterSet.param(*values, **kw) def pytest_addoption(parser): @@ -19,9 +111,10 @@ def pytest_addoption(parser): help="only run tests which match the given substring expression. " "An expression is a python evaluatable expression " "where all names are substring-matched against test names " - "and their parent classes. Example: -k 'test_method or test " + "and their parent classes. Example: -k 'test_method or test_" "other' matches all test functions and classes whose name " - "contains 'test_method' or 'test_other'. " + "contains 'test_method' or 'test_other', while -k 'not test_method' " + "matches those that don't contain 'test_method' in their names. " "Additionally keywords are matched to classes and functions " "containing extra names in their 'extra_keyword_matches' set, " "as well as functions which have names assigned directly to them." @@ -48,23 +141,27 @@ def pytest_cmdline_main(config): config._do_configure() tw = _pytest.config.create_terminal_writer(config) for line in config.getini("markers"): - name, rest = line.split(":", 1) + parts = line.split(":", 1) + name = parts[0] + rest = parts[1] if len(parts) == 2 else '' tw.write("@pytest.mark.%s:" % name, bold=True) tw.line(rest) tw.line() config._ensure_unconfigure() return 0 + + pytest_cmdline_main.tryfirst = True def pytest_collection_modifyitems(items, config): - keywordexpr = config.option.keyword + keywordexpr = config.option.keyword.lstrip() matchexpr = config.option.markexpr if not keywordexpr and not matchexpr: return # pytest used to allow "-" for negating # but today we just allow "-" at the beginning, use "not" instead - # we probably remove "-" alltogether soon + # we probably remove "-" altogether soon if keywordexpr.startswith("-"): keywordexpr = "not " + keywordexpr[1:] selectuntil = False @@ -91,24 +188,30 @@ def pytest_collection_modifyitems(items, config): items[:] = remaining -class MarkMapping: +@attr.s +class MarkMapping(object): """Provides a local mapping for markers where item access resolves to True if the marker is present. """ - def __init__(self, keywords): - mymarks = set() + + own_mark_names = attr.ib() + + @classmethod + def from_keywords(cls, keywords): + mark_names = set() for key, value in keywords.items(): if isinstance(value, MarkInfo) or isinstance(value, MarkDecorator): - mymarks.add(key) - self._mymarks = mymarks + mark_names.add(key) + return cls(mark_names) def __getitem__(self, name): - return name in self._mymarks + return name in self.own_mark_names -class KeywordMapping: +class KeywordMapping(object): """Provides a local mapping for keywords. Given a list of names, map any substring of one of these names to True. """ + def __init__(self, names): self._names = names @@ -121,7 +224,7 @@ def __getitem__(self, subname): def matchmark(colitem, markexpr): """Tries to match on any marker names, attached to the given colitem.""" - return eval(markexpr, {}, MarkMapping(colitem.keywords)) + return eval(markexpr, {}, MarkMapping.from_keywords(colitem.keywords)) def matchkeyword(colitem, keywordexpr): @@ -160,9 +263,13 @@ def matchkeyword(colitem, keywordexpr): def pytest_configure(config): - import pytest + config._old_mark_config = MARK_GEN._config if config.option.strict: - pytest.mark._config = config + MARK_GEN._config = config + + +def pytest_unconfigure(config): + MARK_GEN._config = getattr(config, '_old_mark_config', None) class MarkGenerator: @@ -176,13 +283,14 @@ def test_function(): will set a 'slowtest' :class:`MarkInfo` object on the ``test_function`` object. """ + _config = None def __getattr__(self, name): if name[0] == "_": raise AttributeError("Marker name must NOT start with underscore") - if hasattr(self, '_config'): + if self._config is not None: self._check(name) - return MarkDecorator(name) + return MarkDecorator(Mark(name, (), {})) def _check(self, name): try: @@ -190,19 +298,36 @@ def _check(self, name): return except AttributeError: pass - self._markers = l = set() + self._markers = values = set() for line in self._config.getini("markers"): - beginning = line.split(":", 1) - x = beginning[0].split("(", 1)[0] - l.add(x) + marker = line.split(":", 1)[0] + marker = marker.rstrip() + x = marker.split("(", 1)[0] + values.add(x) if name not in self._markers: raise AttributeError("%r not a registered marker" % (name,)) + def istestfunc(func): return hasattr(func, "__call__") and \ getattr(func, "__name__", "") != "" -class MarkDecorator: + +@attr.s(frozen=True) +class Mark(object): + name = attr.ib() + args = attr.ib() + kwargs = attr.ib() + + def combined_with(self, other): + assert self.name == other.name + return Mark( + self.name, self.args + other.args, + dict(self.kwargs, **other.kwargs)) + + +@attr.s +class MarkDecorator(object): """ A decorator for test functions and test classes. When applied it will create :class:`MarkInfo` objects which may be :ref:`retrieved by hooks as item keywords `. @@ -235,19 +360,33 @@ def test_function(): additional keyword or positional arguments. """ - def __init__(self, name, args=None, kwargs=None): - self.name = name - self.args = args or () - self.kwargs = kwargs or {} + + mark = attr.ib(validator=attr.validators.instance_of(Mark)) + + name = alias('mark.name') + args = alias('mark.args') + kwargs = alias('mark.kwargs') @property def markname(self): - return self.name # for backward-compat (2.4.1 had this attr) + return self.name # for backward-compat (2.4.1 had this attr) + + def __eq__(self, other): + return self.mark == other.mark if isinstance(other, MarkDecorator) else False def __repr__(self): - d = self.__dict__.copy() - name = d.pop('name') - return "" % (name, d) + return "" % (self.mark,) + + def with_args(self, *args, **kwargs): + """ return a MarkDecorator with extra arguments added + + unlike call this can be used even if the sole argument is a callable/class + + :return: MarkDecorator + """ + + mark = Mark(self.name, args, kwargs) + return self.__class__(self.mark.combined_with(mark)) def __call__(self, *args, **kwargs): """ if passed a single callable argument: decorate it with mark info. @@ -257,55 +396,101 @@ def __call__(self, *args, **kwargs): is_class = inspect.isclass(func) if len(args) == 1 and (istestfunc(func) or is_class): if is_class: - if hasattr(func, 'pytestmark'): - mark_list = func.pytestmark - if not isinstance(mark_list, list): - mark_list = [mark_list] - # always work on a copy to avoid updating pytestmark - # from a superclass by accident - mark_list = mark_list + [self] - func.pytestmark = mark_list - else: - func.pytestmark = [self] + store_mark(func, self.mark) else: - holder = getattr(func, self.name, None) - if holder is None: - holder = MarkInfo( - self.name, self.args, self.kwargs - ) - setattr(func, self.name, holder) - else: - holder.add(self.args, self.kwargs) + store_legacy_markinfo(func, self.mark) + store_mark(func, self.mark) return func - kw = self.kwargs.copy() - kw.update(kwargs) - args = self.args + args - return self.__class__(self.name, args=args, kwargs=kw) + return self.with_args(*args, **kwargs) + + +def get_unpacked_marks(obj): + """ + obtain the unpacked marks that are stored on a object + """ + mark_list = getattr(obj, 'pytestmark', []) + + if not isinstance(mark_list, list): + mark_list = [mark_list] + return [ + getattr(mark, 'mark', mark) # unpack MarkDecorator + for mark in mark_list + ] -class MarkInfo: +def store_mark(obj, mark): + """store a Mark on a object + this is used to implement the Mark declarations/decorators correctly + """ + assert isinstance(mark, Mark), mark + # always reassign name to avoid updating pytestmark + # in a reference that was only borrowed + obj.pytestmark = get_unpacked_marks(obj) + [mark] + + +def store_legacy_markinfo(func, mark): + """create the legacy MarkInfo objects and put them onto the function + """ + if not isinstance(mark, Mark): + raise TypeError("got {mark!r} instead of a Mark".format(mark=mark)) + holder = getattr(func, mark.name, None) + if holder is None: + holder = MarkInfo(mark) + setattr(func, mark.name, holder) + else: + holder.add_mark(mark) + + +class MarkInfo(object): """ Marking object created by :class:`MarkDecorator` instances. """ - def __init__(self, name, args, kwargs): - #: name of attribute - self.name = name - #: positional argument list, empty if none specified - self.args = args - #: keyword argument dictionary, empty if nothing specified - self.kwargs = kwargs.copy() - self._arglist = [(args, kwargs.copy())] + + def __init__(self, mark): + assert isinstance(mark, Mark), repr(mark) + self.combined = mark + self._marks = [mark] + + name = alias('combined.name') + args = alias('combined.args') + kwargs = alias('combined.kwargs') def __repr__(self): - return "" % ( - self.name, self.args, self.kwargs - ) + return "".format(self.combined) - def add(self, args, kwargs): + def add_mark(self, mark): """ add a MarkInfo with the given args and kwargs. """ - self._arglist.append((args, kwargs)) - self.args += args - self.kwargs.update(kwargs) + self._marks.append(mark) + self.combined = self.combined.combined_with(mark) def __iter__(self): """ yield MarkInfo objects each relating to a marking-call. """ - for args, kwargs in self._arglist: - yield MarkInfo(self.name, args, kwargs) + return map(MarkInfo, self._marks) + + +MARK_GEN = MarkGenerator() + + +def _marked(func, mark): + """ Returns True if :func: is already marked with :mark:, False otherwise. + This can happen if marker is applied to class and the test file is + invoked more than once. + """ + try: + func_mark = getattr(func, mark.name) + except AttributeError: + return False + return mark.args == func_mark.args and mark.kwargs == func_mark.kwargs + + +def transfer_markers(funcobj, cls, mod): + """ + this function transfers class level markers and module level markers + into function level markinfo objects + + this is the main reason why marks are so broken + the resolution will involve phasing out function level MarkInfo objects + + """ + for obj in (cls, mod): + for mark in get_unpacked_marks(obj): + if not _marked(funcobj, mark): + store_legacy_markinfo(funcobj, mark) diff --git a/_pytest/monkeypatch.py b/_pytest/monkeypatch.py index d4c169d37a9ae3c..40ae560f0702b13 100644 --- a/_pytest/monkeypatch.py +++ b/_pytest/monkeypatch.py @@ -1,15 +1,18 @@ """ monkeypatching and mocking functionality. """ +from __future__ import absolute_import, division, print_function -import os, sys +import os +import sys import re - -from py.builtin import _basestring +import six +from _pytest.fixtures import fixture RE_IMPORT_ERROR_NAME = re.compile("^No module named (.*)$") -def pytest_funcarg__monkeypatch(request): - """The returned ``monkeypatch`` funcarg provides these +@fixture +def monkeypatch(): + """The returned ``monkeypatch`` fixture provides these helper methods to modify objects, dictionaries or os.environ:: monkeypatch.setattr(obj, name, value, raising=True) @@ -22,13 +25,13 @@ def pytest_funcarg__monkeypatch(request): monkeypatch.chdir(path) All modifications will be undone after the requesting - test function has finished. The ``raising`` + test function or fixture has finished. The ``raising`` parameter determines if a KeyError or AttributeError will be raised if the set/deletion operation has no target. """ - mpatch = monkeypatch() - request.addfinalizer(mpatch.undo) - return mpatch + mpatch = MonkeyPatch() + yield mpatch + mpatch.undo() def resolve(name): @@ -67,15 +70,15 @@ def annotated_getattr(obj, name, ann): obj = getattr(obj, name) except AttributeError: raise AttributeError( - '%r object at %s has no attribute %r' % ( - type(obj).__name__, ann, name - ) + '%r object at %s has no attribute %r' % ( + type(obj).__name__, ann, name + ) ) return obj def derive_importpath(import_path, raising): - if not isinstance(import_path, _basestring) or "." not in import_path: + if not isinstance(import_path, six.string_types) or "." not in import_path: raise TypeError("must be absolute import path string, not %r" % (import_path,)) module, attr = import_path.rsplit('.', 1) @@ -93,8 +96,9 @@ def __repr__(self): notset = Notset() -class monkeypatch: - """ Object keeping a record of setattr/item/env/syspath changes. """ +class MonkeyPatch: + """ Object returned by the ``monkeypatch`` fixture keeping a record of setattr/item/env/syspath changes. + """ def __init__(self): self._setattr = [] @@ -120,7 +124,7 @@ def setattr(self, target, name, value=notset, raising=True): import inspect if value is notset: - if not isinstance(target, _basestring): + if not isinstance(target, six.string_types): raise TypeError("use setattr(target, name, value) or " "setattr(target, value) with target being a dotted " "import string") @@ -150,7 +154,7 @@ def delattr(self, target, name=notset, raising=True): """ __tracebackhide__ = True if name is notset: - if not isinstance(target, _basestring): + if not isinstance(target, six.string_types): raise TypeError("use delattr(target, name) or " "delattr(target) with target being a dotted " "import string") @@ -220,10 +224,10 @@ def undo(self): """ Undo previous changes. This call consumes the undo stack. Calling it a second time has no effect unless you do more monkeypatching after the undo call. - + There is generally no need to call `undo()`, since it is called automatically during tear-down. - + Note that the same `monkeypatch` fixture is used across a single test function invocation. If `monkeypatch` is used both by the test function itself and one of the test fixtures, diff --git a/_pytest/nodes.py b/_pytest/nodes.py new file mode 100644 index 000000000000000..ad3af2ce67c8958 --- /dev/null +++ b/_pytest/nodes.py @@ -0,0 +1,37 @@ +SEP = "/" + + +def _splitnode(nodeid): + """Split a nodeid into constituent 'parts'. + + Node IDs are strings, and can be things like: + '' + 'testing/code' + 'testing/code/test_excinfo.py' + 'testing/code/test_excinfo.py::TestFormattedExcinfo::()' + + Return values are lists e.g. + [] + ['testing', 'code'] + ['testing', 'code', 'test_excinfo.py'] + ['testing', 'code', 'test_excinfo.py', 'TestFormattedExcinfo', '()'] + """ + if nodeid == '': + # If there is no root node at all, return an empty list so the caller's logic can remain sane + return [] + parts = nodeid.split(SEP) + # Replace single last element 'test_foo.py::Bar::()' with multiple elements 'test_foo.py', 'Bar', '()' + parts[-1:] = parts[-1].split("::") + return parts + + +def ischildnode(baseid, nodeid): + """Return True if the nodeid is a child node of the baseid. + + E.g. 'foo/bar::Baz::()' is a child of 'foo', 'foo/bar' and 'foo/bar::Baz', but not of 'foo/blorp' + """ + base_parts = _splitnode(baseid) + node_parts = _splitnode(nodeid) + if len(node_parts) < len(base_parts): + return False + return node_parts[:len(base_parts)] == base_parts diff --git a/_pytest/nose.py b/_pytest/nose.py index 038746868607675..c81542eadf377d1 100644 --- a/_pytest/nose.py +++ b/_pytest/nose.py @@ -1,10 +1,10 @@ """ run test suites written for nose. """ +from __future__ import absolute_import, division, print_function import sys -import py -import pytest -from _pytest import unittest +from _pytest import unittest, runner, python +from _pytest.config import hookimpl def get_skip_exceptions(): @@ -19,52 +19,53 @@ def get_skip_exceptions(): def pytest_runtest_makereport(item, call): if call.excinfo and call.excinfo.errisinstance(get_skip_exceptions()): # let's substitute the excinfo with a pytest.skip one - call2 = call.__class__(lambda: - pytest.skip(str(call.excinfo.value)), call.when) + call2 = call.__class__( + lambda: runner.skip(str(call.excinfo.value)), call.when) call.excinfo = call2.excinfo -@pytest.hookimpl(trylast=True) +@hookimpl(trylast=True) def pytest_runtest_setup(item): if is_potential_nosetest(item): - if isinstance(item.parent, pytest.Generator): + if isinstance(item.parent, python.Generator): gen = item.parent if not hasattr(gen, '_nosegensetup'): call_optional(gen.obj, 'setup') - if isinstance(gen.parent, pytest.Instance): + if isinstance(gen.parent, python.Instance): call_optional(gen.parent.obj, 'setup') gen._nosegensetup = True if not call_optional(item.obj, 'setup'): # call module level setup if there is no object level one call_optional(item.parent.obj, 'setup') - #XXX this implies we only call teardown when setup worked + # XXX this implies we only call teardown when setup worked item.session._setupstate.addfinalizer((lambda: teardown_nose(item)), item) + def teardown_nose(item): if is_potential_nosetest(item): if not call_optional(item.obj, 'teardown'): call_optional(item.parent.obj, 'teardown') - #if hasattr(item.parent, '_nosegensetup'): + # if hasattr(item.parent, '_nosegensetup'): # #call_optional(item._nosegensetup, 'teardown') # del item.parent._nosegensetup def pytest_make_collect_report(collector): - if isinstance(collector, pytest.Generator): + if isinstance(collector, python.Generator): call_optional(collector.obj, 'setup') def is_potential_nosetest(item): # extra check needed since we do not do nose style setup/teardown # on direct unittest style classes - return isinstance(item, pytest.Function) and \ + return isinstance(item, python.Function) and \ not isinstance(item, unittest.TestCaseFunction) def call_optional(obj, name): method = getattr(obj, name, None) isfixture = hasattr(method, "_pytestfixturefunction") - if method is not None and not isfixture and py.builtin.callable(method): + if method is not None and not isfixture and callable(method): # If there's any problems allow the exception to raise rather than # silently ignoring them method() diff --git a/_pytest/outcomes.py b/_pytest/outcomes.py new file mode 100644 index 000000000000000..7f0c18fa6c1b487 --- /dev/null +++ b/_pytest/outcomes.py @@ -0,0 +1,147 @@ +""" +exception classes and constants handling test outcomes +as well as functions creating them +""" +from __future__ import absolute_import, division, print_function +import py +import sys + + +class OutcomeException(BaseException): + """ OutcomeException and its subclass instances indicate and + contain info about test and collection outcomes. + """ + def __init__(self, msg=None, pytrace=True): + BaseException.__init__(self, msg) + self.msg = msg + self.pytrace = pytrace + + def __repr__(self): + if self.msg: + val = self.msg + if isinstance(val, bytes): + val = py._builtin._totext(val, errors='replace') + return val + return "<%s instance>" % (self.__class__.__name__,) + __str__ = __repr__ + + +TEST_OUTCOME = (OutcomeException, Exception) + + +class Skipped(OutcomeException): + # XXX hackish: on 3k we fake to live in the builtins + # in order to have Skipped exception printing shorter/nicer + __module__ = 'builtins' + + def __init__(self, msg=None, pytrace=True, allow_module_level=False): + OutcomeException.__init__(self, msg=msg, pytrace=pytrace) + self.allow_module_level = allow_module_level + + +class Failed(OutcomeException): + """ raised from an explicit call to pytest.fail() """ + __module__ = 'builtins' + + +class Exit(KeyboardInterrupt): + """ raised for immediate program exits (no tracebacks/summaries)""" + def __init__(self, msg="unknown reason"): + self.msg = msg + KeyboardInterrupt.__init__(self, msg) + +# exposed helper methods + + +def exit(msg): + """ exit testing process as if KeyboardInterrupt was triggered. """ + __tracebackhide__ = True + raise Exit(msg) + + +exit.Exception = Exit + + +def skip(msg="", **kwargs): + """ skip an executing test with the given message. Note: it's usually + better to use the pytest.mark.skipif marker to declare a test to be + skipped under certain conditions like mismatching platforms or + dependencies. See the pytest_skipping plugin for details. + + :kwarg bool allow_module_level: allows this function to be called at + module level, skipping the rest of the module. Default to False. + """ + __tracebackhide__ = True + allow_module_level = kwargs.pop('allow_module_level', False) + if kwargs: + keys = [k for k in kwargs.keys()] + raise TypeError('unexpected keyword arguments: {0}'.format(keys)) + raise Skipped(msg=msg, allow_module_level=allow_module_level) + + +skip.Exception = Skipped + + +def fail(msg="", pytrace=True): + """ explicitly fail an currently-executing test with the given Message. + + :arg pytrace: if false the msg represents the full failure information + and no python traceback will be reported. + """ + __tracebackhide__ = True + raise Failed(msg=msg, pytrace=pytrace) + + +fail.Exception = Failed + + +class XFailed(fail.Exception): + """ raised from an explicit call to pytest.xfail() """ + + +def xfail(reason=""): + """ xfail an executing test or setup functions with the given reason.""" + __tracebackhide__ = True + raise XFailed(reason) + + +xfail.Exception = XFailed + + +def importorskip(modname, minversion=None): + """ return imported module if it has at least "minversion" as its + __version__ attribute. If no minversion is specified the a skip + is only triggered if the module can not be imported. + """ + import warnings + __tracebackhide__ = True + compile(modname, '', 'eval') # to catch syntaxerrors + should_skip = False + + with warnings.catch_warnings(): + # make sure to ignore ImportWarnings that might happen because + # of existing directories with the same name we're trying to + # import but without a __init__.py file + warnings.simplefilter('ignore') + try: + __import__(modname) + except ImportError: + # Do not raise chained exception here(#1485) + should_skip = True + if should_skip: + raise Skipped("could not import %r" % (modname,), allow_module_level=True) + mod = sys.modules[modname] + if minversion is None: + return mod + verattr = getattr(mod, '__version__', None) + if minversion is not None: + try: + from pkg_resources import parse_version as pv + except ImportError: + raise Skipped("we have a required version for %r but can not import " + "pkg_resources to parse version strings." % (modname,), + allow_module_level=True) + if verattr is None or pv(verattr) < pv(minversion): + raise Skipped("module %r has __version__ %r, required is: %r" % ( + modname, verattr, minversion), allow_module_level=True) + return mod diff --git a/_pytest/pastebin.py b/_pytest/pastebin.py index 4ec62d022805002..b588b021b12422f 100644 --- a/_pytest/pastebin.py +++ b/_pytest/pastebin.py @@ -1,5 +1,8 @@ """ submit failure or test session information to a pastebin service. """ +from __future__ import absolute_import, division, print_function + import pytest +import six import sys import tempfile @@ -7,13 +10,13 @@ def pytest_addoption(parser): group = parser.getgroup("terminal reporting") group._addoption('--pastebin', metavar="mode", - action='store', dest="pastebin", default=None, - choices=['failed', 'all'], - help="send failed|all info to bpaste.net pastebin service.") + action='store', dest="pastebin", default=None, + choices=['failed', 'all'], + help="send failed|all info to bpaste.net pastebin service.") + @pytest.hookimpl(trylast=True) def pytest_configure(config): - import py if config.option.pastebin == "all": tr = config.pluginmanager.getplugin('terminalreporter') # if no terminal reporter plugin is present, nothing we can do here; @@ -23,13 +26,16 @@ def pytest_configure(config): # pastebin file will be utf-8 encoded binary file config._pastebinfile = tempfile.TemporaryFile('w+b') oldwrite = tr._tw.write + def tee_write(s, **kwargs): oldwrite(s, **kwargs) - if py.builtin._istext(s): + if isinstance(s, six.text_type): s = s.encode('utf-8') config._pastebinfile.write(s) + tr._tw.write = tee_write + def pytest_unconfigure(config): if hasattr(config, '_pastebinfile'): # get terminal contents and delete file @@ -45,6 +51,7 @@ def pytest_unconfigure(config): pastebinurl = create_new_paste(sessionlog) tr.write_line("pastebin session-log: %s\n" % pastebinurl) + def create_new_paste(contents): """ Creates a new paste using bpaste.net service. @@ -72,6 +79,7 @@ def create_new_paste(contents): else: return 'bad response: ' + response + def pytest_terminal_summary(terminalreporter): import _pytest.config if terminalreporter.config.option.pastebin != "failed": @@ -89,4 +97,4 @@ def pytest_terminal_summary(terminalreporter): s = tw.stringio.getvalue() assert len(s) pastebinurl = create_new_paste(s) - tr.write_line("%s --> %s" %(msg, pastebinurl)) + tr.write_line("%s --> %s" % (msg, pastebinurl)) diff --git a/_pytest/pytester.py b/_pytest/pytester.py index faed7f581c9d025..f2dd5994f1bbfb4 100644 --- a/_pytest/pytester.py +++ b/_pytest/pytester.py @@ -1,44 +1,44 @@ """ (disabled by default) support for testing pytest and pytest plugins. """ +from __future__ import absolute_import, division, print_function + import codecs import gc import os import platform import re import subprocess +import six import sys import time import traceback from fnmatch import fnmatch -from py.builtin import print_ +from weakref import WeakKeyDictionary +from _pytest.capture import MultiCapture, SysCapture from _pytest._code import Source import py import pytest from _pytest.main import Session, EXIT_OK +from _pytest.assertion.rewrite import AssertionRewritingHook + + +PYTEST_FULLPATH = os.path.abspath(pytest.__file__.rstrip("oc")).replace("$py.class", ".py") def pytest_addoption(parser): # group = parser.getgroup("pytester", "pytester (self-tests) options") parser.addoption('--lsof', - action="store_true", dest="lsof", default=False, - help=("run FD checks if lsof is available")) + action="store_true", dest="lsof", default=False, + help=("run FD checks if lsof is available")) parser.addoption('--runpytest', default="inprocess", dest="runpytest", - choices=("inprocess", "subprocess", ), - help=("run pytest sub runs in tests using an 'inprocess' " - "or 'subprocess' (python -m main) method")) + choices=("inprocess", "subprocess", ), + help=("run pytest sub runs in tests using an 'inprocess' " + "or 'subprocess' (python -m main) method")) def pytest_configure(config): - # This might be called multiple times. Only take the first. - global _pytest_fullpath - try: - _pytest_fullpath - except NameError: - _pytest_fullpath = os.path.abspath(pytest.__file__.rstrip("oc")) - _pytest_fullpath = _pytest_fullpath.replace("$py.class", ".py") - if config.getvalue("lsof"): checker = LsofFdLeakChecker() if checker.matching_platform(): @@ -58,7 +58,7 @@ def _exec_lsof(self): def _parse_lsof_output(self, out): def isopen(line): return line.startswith('f') and ("deleted" not in line and - 'mem' not in line and "txt" not in line and 'cwd' not in line) + 'mem' not in line and "txt" not in line and 'cwd' not in line) open_files = [] @@ -84,7 +84,7 @@ def matching_platform(self): return True @pytest.hookimpl(hookwrapper=True, tryfirst=True) - def pytest_runtest_item(self, item): + def pytest_runtest_protocol(self, item): lines1 = self.get_open_files() yield if hasattr(sys, "pypy_version_info"): @@ -103,40 +103,42 @@ def pytest_runtest_item(self, item): error.extend([str(f) for f in lines2]) error.append(error[0]) error.append("*** function %s:%s: %s " % item.location) - pytest.fail("\n".join(error), pytrace=False) + error.append("See issue #2366") + item.warn('', "\n".join(error)) # XXX copied from execnet's conftest.py - needs to be merged winpymap = { 'python2.7': r'C:\Python27\python.exe', - 'python2.6': r'C:\Python26\python.exe', - 'python3.1': r'C:\Python31\python.exe', - 'python3.2': r'C:\Python32\python.exe', - 'python3.3': r'C:\Python33\python.exe', 'python3.4': r'C:\Python34\python.exe', 'python3.5': r'C:\Python35\python.exe', + 'python3.6': r'C:\Python36\python.exe', } + def getexecutable(name, cache={}): try: return cache[name] except KeyError: executable = py.path.local.sysfind(name) if executable: + import subprocess + popen = subprocess.Popen([str(executable), "--version"], + universal_newlines=True, stderr=subprocess.PIPE) + out, err = popen.communicate() if name == "jython": - import subprocess - popen = subprocess.Popen([str(executable), "--version"], - universal_newlines=True, stderr=subprocess.PIPE) - out, err = popen.communicate() if not err or "2.5" not in err: executable = None if "2.5.2" in err: - executable = None # http://bugs.jython.org/issue1790 + executable = None # http://bugs.jython.org/issue1790 + elif popen.returncode != 0: + # Handle pyenv's 127. + executable = None cache[name] = executable return executable -@pytest.fixture(params=['python2.6', 'python2.7', 'python3.3', "python3.4", - 'pypy', 'pypy3']) + +@pytest.fixture(params=['python2.7', 'python3.4', 'pypy', 'pypy3']) def anypython(request): name = request.param executable = getexecutable(name) @@ -151,6 +153,8 @@ def anypython(request): return executable # used at least by pytest-xdist plugin + + @pytest.fixture def _pytest(request): """ Return a helper which offers a gethookrecorder(hook) @@ -159,6 +163,7 @@ def _pytest(request): """ return PytestArg(request) + class PytestArg: def __init__(self, request): self.request = request @@ -169,9 +174,9 @@ def gethookrecorder(self, hook): return hookrecorder -def get_public_names(l): - """Only return names from iterator l without a leading underscore.""" - return [x for x in l if x[0] != "_"] +def get_public_names(values): + """Only return names from iterator values without a leading underscore.""" + return [x for x in values if x[0] != "_"] class ParsedCall: @@ -182,7 +187,7 @@ def __init__(self, name, kwargs): def __repr__(self): d = self.__dict__.copy() del d['_name'] - return "" %(self._name, d) + return "" % (self._name, d) class HookRecorder: @@ -222,15 +227,15 @@ def assert_contains(self, entries): name, check = entries.pop(0) for ind, call in enumerate(self.calls[i:]): if call._name == name: - print_("NAMEMATCH", name, call) + print("NAMEMATCH", name, call) if eval(check, backlocals, call.__dict__): - print_("CHECKERMATCH", repr(check), "->", call) + print("CHECKERMATCH", repr(check), "->", call) else: - print_("NOCHECKERMATCH", repr(check), "-", call) + print("NOCHECKERMATCH", repr(check), "-", call) continue i += ind + 1 break - print_("NONAMEMATCH", name, "with", call) + print("NONAMEMATCH", name, "with", call) else: pytest.fail("could not find %r check %r" % (name, check)) @@ -245,9 +250,9 @@ def popcall(self, name): pytest.fail("\n".join(lines)) def getcall(self, name): - l = self.getcalls(name) - assert len(l) == 1, (name, l) - return l[0] + values = self.getcalls(name) + assert len(values) == 1, (name, values) + return values[0] # functionality for test reports @@ -256,9 +261,9 @@ def getreports(self, return [x.report for x in self.getcalls(names)] def matchreport(self, inamepart="", - names="pytest_runtest_logreport pytest_collectreport", when=None): + names="pytest_runtest_logreport pytest_collectreport", when=None): """ return a testreport whose dotted import path matches """ - l = [] + values = [] for rep in self.getreports(names=names): try: if not when and rep.when != "call" and rep.passed: @@ -269,14 +274,14 @@ def matchreport(self, inamepart="", if when and getattr(rep, 'when', None) != when: continue if not inamepart or inamepart in rep.nodeid.split("::"): - l.append(rep) - if not l: + values.append(rep) + if not values: raise ValueError("could not find test report matching %r: " "no test reports at all!" % (inamepart,)) - if len(l) > 1: + if len(values) > 1: raise ValueError( - "found 2 or more testreports matching %r: %s" %(inamepart, l)) - return l[0] + "found 2 or more testreports matching %r: %s" % (inamepart, values)) + return values[0] def getfailures(self, names='pytest_runtest_logreport pytest_collectreport'): @@ -290,7 +295,7 @@ def listoutcomes(self): skipped = [] failed = [] for rep in self.getreports( - "pytest_collectreport pytest_runtest_logreport"): + "pytest_collectreport pytest_runtest_logreport"): if rep.passed: if getattr(rep, "when", None) == "call": passed.append(rep) @@ -318,7 +323,8 @@ def linecomp(request): return LineComp() -def pytest_funcarg__LineMatcher(request): +@pytest.fixture(name='LineMatcher') +def LineMatcher_fixture(request): return LineMatcher @@ -327,7 +333,9 @@ def testdir(request, tmpdir_factory): return Testdir(request, tmpdir_factory) -rex_outcome = re.compile("(\d+) ([\w-]+)") +rex_outcome = re.compile(r"(\d+) ([\w-]+)") + + class RunResult: """The result of running a command. @@ -343,6 +351,7 @@ class RunResult: :duration: Duration in seconds. """ + def __init__(self, ret, outlines, errlines, duration): self.ret = ret self.outlines = outlines @@ -362,22 +371,26 @@ def parseoutcomes(self): for num, cat in outcomes: d[cat] = int(num) return d + raise ValueError("Pytest terminal report not found") - def assert_outcomes(self, passed=0, skipped=0, failed=0): + def assert_outcomes(self, passed=0, skipped=0, failed=0, error=0): """ assert that the specified outcomes appear with the respective numbers (0 means it didn't occur) in the text output from a test run.""" d = self.parseoutcomes() - assert passed == d.get("passed", 0) - assert skipped == d.get("skipped", 0) - assert failed == d.get("failed", 0) - + obtained = { + 'passed': d.get('passed', 0), + 'skipped': d.get('skipped', 0), + 'failed': d.get('failed', 0), + 'error': d.get('error', 0), + } + assert obtained == dict(passed=passed, skipped=skipped, failed=failed, error=error) class Testdir: - """Temporary test directory with tools to test/run py.test itself. + """Temporary test directory with tools to test/run pytest itself. This is based on the ``tmpdir`` fixture but provides a number of - methods which aid with testing py.test itself. Unless + methods which aid with testing pytest itself. Unless :py:meth:`chdir` is used all methods will use :py:attr:`tmpdir` as current working directory. @@ -396,20 +409,13 @@ class Testdir: def __init__(self, request, tmpdir_factory): self.request = request - # XXX remove duplication with tmpdir plugin - basetmp = tmpdir_factory.ensuretemp("testdir") + self._mod_collections = WeakKeyDictionary() name = request.function.__name__ - for i in range(100): - try: - tmpdir = basetmp.mkdir(name + str(i)) - except py.error.EEXIST: - continue - break - self.tmpdir = tmpdir + self.tmpdir = tmpdir_factory.mktemp(name, numbered=True) self.plugins = [] self._savesyspath = (list(sys.path), list(sys.meta_path)) self._savemodulekeys = set(sys.modules) - self.chdir() # always chdir + self.chdir() # always chdir self.request.addfinalizer(self.finalize) method = self.request.config.getoption("--runpytest") if method == "inprocess": @@ -441,9 +447,10 @@ def delete_loaded_modules(self): the module is re-imported. """ for name in set(sys.modules).difference(self._savemodulekeys): - # it seems zope.interfaces is keeping some state - # (used by twisted related tests) - if name != "zope.interface": + # some zope modules used by twisted-related tests keeps internal + # state and can't be deleted; we had some trouble in the past + # with zope.interface for example + if not name.startswith("zope"): del sys.modules[name] def make_hook_recorder(self, pluginmanager): @@ -463,26 +470,24 @@ def chdir(self): if not hasattr(self, '_olddir'): self._olddir = old - def _makefile(self, ext, args, kwargs): + def _makefile(self, ext, args, kwargs, encoding='utf-8'): items = list(kwargs.items()) + + def to_text(s): + return s.decode(encoding) if isinstance(s, bytes) else six.text_type(s) + if args: - source = py.builtin._totext("\n").join( - map(py.builtin._totext, args)) + py.builtin._totext("\n") + source = u"\n".join(to_text(x) for x in args) basename = self.request.function.__name__ items.insert(0, (basename, source)) + ret = None - for name, value in items: - p = self.tmpdir.join(name).new(ext=ext) + for basename, value in items: + p = self.tmpdir.join(basename).new(ext=ext) + p.dirpath().ensure_dir() source = Source(value) - def my_totext(s, encoding="utf-8"): - if py.builtin._isbytes(s): - s = py.builtin._totext(s, encoding=encoding) - return s - source_unicode = "\n".join([my_totext(line) for line in source.lines]) - source = py.builtin._totext(source_unicode) - content = source.strip().encode("utf-8") # + "\n" - #content = content.rstrip() + "\n" - p.write(content, "wb") + source = u"\n".join(to_text(line) for line in source.lines) + p.write(source.strip().encode(encoding), "wb") if ret is None: ret = p return ret @@ -557,7 +562,7 @@ def mkdir(self, name): def mkpydir(self, name): """Create a new python package. - This creates a (sub)direcotry with an empty ``__init__.py`` + This creates a (sub)directory with an empty ``__init__.py`` file so that is recognised as a python package. """ @@ -566,6 +571,7 @@ def mkpydir(self, name): return p Session = Session + def getnode(self, config, arg): """Return the collection node of a file. @@ -588,7 +594,7 @@ def getpathnode(self, path): """Return the collection node of a file. This is like :py:meth:`getnode` but uses - :py:meth:`parseconfigure` to create the (configured) py.test + :py:meth:`parseconfigure` to create the (configured) pytest Config instance. :param path: A :py:class:`py.path.local` instance of the file. @@ -646,17 +652,17 @@ def inline_runsource(self, source, *cmdlineargs): """ p = self.makepyfile(source) - l = list(cmdlineargs) + [p] - return self.inline_run(*l) + values = list(cmdlineargs) + [p] + return self.inline_run(*values) def inline_genitems(self, *args): """Run ``pytest.main(['--collectonly'])`` in-process. - Retuns a tuple of the collected items and a + Returns a tuple of the collected items and a :py:class:`HookRecorder` instance. This runs the :py:func:`pytest.main` function to run all of - py.test inside the test process itself like + pytest inside the test process itself like :py:meth:`inline_run`. However the return value is a tuple of the collection items and a :py:class:`HookRecorder` instance. @@ -669,7 +675,7 @@ def inline_run(self, *args, **kwargs): """Run ``pytest.main()`` in-process, returning a HookRecorder. This runs the :py:func:`pytest.main` function to run all of - py.test inside the test process itself. This means it can + pytest inside the test process itself. This means it can return a :py:class:`HookRecorder` instance which gives more detailed results from then run then can be done by matching stdout/stderr from :py:meth:`runpytest`. @@ -681,9 +687,21 @@ def inline_run(self, *args, **kwargs): ``pytest.main()`` instance should use. :return: A :py:class:`HookRecorder` instance. - """ + # When running py.test inline any plugins active in the main + # test process are already imported. So this disables the + # warning which will trigger to say they can no longer be + # rewritten, which is fine as they are already rewritten. + orig_warn = AssertionRewritingHook._warn_already_imported + + def revert(): + AssertionRewritingHook._warn_already_imported = orig_warn + + self.request.addfinalizer(revert) + AssertionRewritingHook._warn_already_imported = lambda *a: None + rec = [] + class Collect: def pytest_configure(x, config): rec.append(self.make_hook_recorder(config.pluginmanager)) @@ -713,25 +731,30 @@ def runpytest_inprocess(self, *args, **kwargs): if kwargs.get("syspathinsert"): self.syspathinsert() now = time.time() - capture = py.io.StdCapture() + capture = MultiCapture(Capture=SysCapture) + capture.start_capturing() try: try: reprec = self.inline_run(*args, **kwargs) except SystemExit as e: + class reprec: ret = e.args[0] + except Exception: traceback.print_exc() + class reprec: ret = 3 finally: - out, err = capture.reset() + out, err = capture.readouterr() + capture.stop_capturing() sys.stdout.write(out) sys.stderr.write(err) res = RunResult(reprec.ret, out.split("\n"), err.split("\n"), - time.time()-now) + time.time() - now) res.reprec = reprec return res @@ -747,17 +770,17 @@ def _ensure_basetemp(self, args): args = [str(x) for x in args] for x in args: if str(x).startswith('--basetemp'): - #print ("basedtemp exists: %s" %(args,)) + # print("basedtemp exists: %s" %(args,)) break else: args.append("--basetemp=%s" % self.tmpdir.dirpath('basetemp')) - #print ("added basetemp: %s" %(args,)) + # print("added basetemp: %s" %(args,)) return args def parseconfig(self, *args): - """Return a new py.test Config instance from given commandline args. + """Return a new pytest Config instance from given commandline args. - This invokes the py.test bootstrapping code in _pytest.config + This invokes the pytest bootstrapping code in _pytest.config to create a new :py:class:`_pytest.core.PluginManager` and call the pytest_cmdline_parse hook to create new :py:class:`_pytest.config.Config` instance. @@ -777,7 +800,7 @@ def parseconfig(self, *args): return config def parseconfigure(self, *args): - """Return a new py.test configured Config instance. + """Return a new pytest configured Config instance. This returns a new :py:class:`_pytest.config.Config` instance like :py:meth:`parseconfig`, but also calls the @@ -789,10 +812,10 @@ def parseconfigure(self, *args): self.request.addfinalizer(config._ensure_unconfigure) return config - def getitem(self, source, funcname="test_func"): + def getitem(self, source, funcname="test_func"): """Return the test item for a test function. - This writes the source to a python file and runs py.test's + This writes the source to a python file and runs pytest's collection on the resulting module, returning the test item for the requested function name. @@ -806,13 +829,13 @@ def getitem(self, source, funcname="test_func"): for item in items: if item.name == funcname: return item - assert 0, "%r item not found in module:\n%s\nitems: %s" %( + assert 0, "%r item not found in module:\n%s\nitems: %s" % ( funcname, source, items) - def getitems(self, source): + def getitems(self, source): """Return all test items collected from the module. - This writes the source to a python file and runs py.test's + This writes the source to a python file and runs pytest's collection on the resulting module, returning all test items contained within. @@ -820,11 +843,11 @@ def getitems(self, source): modcol = self.getmodulecol(source) return self.genitems([modcol]) - def getmodulecol(self, source, configargs=(), withinit=False): + def getmodulecol(self, source, configargs=(), withinit=False): """Return the module collection node for ``source``. This writes ``source`` to a file using :py:meth:`makepyfile` - and then runs the py.test collection on it, returning the + and then runs the pytest collection on it, returning the collection node for the test module. :param source: The source code of the module to collect. @@ -833,15 +856,16 @@ def getmodulecol(self, source, configargs=(), withinit=False): :py:meth:`parseconfigure`. :param withinit: Whether to also write a ``__init__.py`` file - to the temporarly directory to ensure it is a package. + to the temporary directory to ensure it is a package. """ kw = {self.request.function.__name__: Source(source).strip()} path = self.makepyfile(**kw) if withinit: - self.makepyfile(__init__ = "#") + self.makepyfile(__init__="#") self.config = config = self.parseconfigure(path, *configargs) node = self.getnode(config, path) + return node def collect_by_name(self, modcol, name): @@ -856,7 +880,9 @@ def collect_by_name(self, modcol, name): :param name: The name of the node to return. """ - for colitem in modcol._memocollect(): + if modcol not in self._mod_collections: + self._mod_collections[modcol] = list(modcol.collect()) + for colitem in self._mod_collections[modcol]: if colitem.name == name: return colitem @@ -873,8 +899,11 @@ def popen(self, cmdargs, stdout, stderr, **kw): env['PYTHONPATH'] = os.pathsep.join(filter(None, [ str(os.getcwd()), env.get('PYTHONPATH', '')])) kw['env'] = env - return subprocess.Popen(cmdargs, - stdout=stdout, stderr=stderr, **kw) + + popen = subprocess.Popen(cmdargs, stdin=subprocess.PIPE, stdout=stdout, stderr=stderr, **kw) + popen.stdin.close() + + return popen def run(self, *cmdargs): """Run a command with arguments. @@ -891,14 +920,14 @@ def _run(self, *cmdargs): cmdargs = [str(x) for x in cmdargs] p1 = self.tmpdir.join("stdout") p2 = self.tmpdir.join("stderr") - print_("running:", ' '.join(cmdargs)) - print_(" in:", str(py.path.local())) + print("running:", ' '.join(cmdargs)) + print(" in:", str(py.path.local())) f1 = codecs.open(str(p1), "w", encoding="utf8") f2 = codecs.open(str(p2), "w", encoding="utf8") try: now = time.time() popen = self.popen(cmdargs, stdout=f1, stderr=f2, - close_fds=(sys.platform != "win32")) + close_fds=(sys.platform != "win32")) ret = popen.wait() finally: f1.close() @@ -913,19 +942,19 @@ def _run(self, *cmdargs): f2.close() self._dump_lines(out, sys.stdout) self._dump_lines(err, sys.stderr) - return RunResult(ret, out, err, time.time()-now) + return RunResult(ret, out, err, time.time() - now) def _dump_lines(self, lines, fp): try: for line in lines: - py.builtin.print_(line, file=fp) + print(line, file=fp) except UnicodeEncodeError: print("couldn't print to %s because of encoding" % (fp,)) def _getpytestargs(self): # we cannot use "(sys.executable,script)" - # because on windows the script is e.g. a py.test.exe - return (sys.executable, _pytest_fullpath,) # noqa + # because on windows the script is e.g. a pytest.exe + return (sys.executable, PYTEST_FULLPATH) # noqa def runpython(self, script): """Run a python script using sys.executable as interpreter. @@ -939,7 +968,7 @@ def runpython_c(self, command): return self.run(sys.executable, "-c", command) def runpytest_subprocess(self, *args, **kwargs): - """Run py.test as a subprocess with given arguments. + """Run pytest as a subprocess with given arguments. Any plugins added to the :py:attr:`plugins` list will added using the ``-p`` command line option. Addtionally @@ -952,12 +981,12 @@ def runpytest_subprocess(self, *args, **kwargs): """ p = py.path.local.make_numbered_dir(prefix="runpytest-", - keep=None, rootdir=self.tmpdir) + keep=None, rootdir=self.tmpdir) args = ('--basetemp=%s' % p, ) + args - #for x in args: + # for x in args: # if '--confcutdir' in str(x): # break - #else: + # else: # pass # args = ('--confcutdir=.',) + args plugins = [x for x in self.plugins if isinstance(x, str)] @@ -967,15 +996,15 @@ def runpytest_subprocess(self, *args, **kwargs): return self.run(*args) def spawn_pytest(self, string, expect_timeout=10.0): - """Run py.test using pexpect. + """Run pytest using pexpect. - This makes sure to use the right py.test and sets up the + This makes sure to use the right pytest and sets up the temporary directory locations. The pexpect child is returned. """ - basetemp = self.tmpdir.mkdir("pexpect") + basetemp = self.tmpdir.mkdir("temp-pexpect") invoke = " ".join(map(str, self._getpytestargs())) cmd = "%s --basetemp=%s %s" % (invoke, basetemp, string) return self.spawn(cmd, expect_timeout=expect_timeout) @@ -988,8 +1017,6 @@ def spawn(self, cmd, expect_timeout=10.0): pexpect = pytest.importorskip("pexpect", "3.0") if hasattr(sys, 'pypy_version_info') and '64' in platform.machine(): pytest.skip("pypy-64 bit not supported") - if sys.platform == "darwin": - pytest.xfail("pexpect does not work reliably on darwin?!") if sys.platform.startswith("freebsd"): pytest.xfail("pexpect does not work reliably on freebsd") logfile = self.tmpdir.join("spawn.out").open("wb") @@ -998,12 +1025,13 @@ def spawn(self, cmd, expect_timeout=10.0): child.timeout = expect_timeout return child + def getdecoded(out): - try: - return out.decode("utf-8") - except UnicodeDecodeError: - return "INTERNAL not-utf8-decodeable, truncated string:\n%s" % ( - py.io.saferepr(out),) + try: + return out.decode("utf-8") + except UnicodeDecodeError: + return "INTERNAL not-utf8-decodeable, truncated string:\n%s" % ( + py.io.saferepr(out),) class LineComp: @@ -1033,8 +1061,9 @@ class LineMatcher: """ - def __init__(self, lines): + def __init__(self, lines): self.lines = lines + self._log_output = [] def str(self): """Return the entire original text.""" @@ -1048,6 +1077,23 @@ def _getlines(self, lines2): return lines2 def fnmatch_lines_random(self, lines2): + """Check lines exist in the output using ``fnmatch.fnmatch``, in any order. + + The argument is a list of lines which have to occur in the + output, in any order. + """ + self._match_lines_random(lines2, fnmatch) + + def re_match_lines_random(self, lines2): + """Check lines exist in the output using ``re.match``, in any order. + + The argument is a list of lines which have to occur in the + output, in any order. + + """ + self._match_lines_random(lines2, lambda name, pat: re.match(pat, name)) + + def _match_lines_random(self, lines2, match_func): """Check lines exist in the output. The argument is a list of lines which have to occur in the @@ -1057,11 +1103,12 @@ def fnmatch_lines_random(self, lines2): lines2 = self._getlines(lines2) for line in lines2: for x in self.lines: - if line == x or fnmatch(x, line): - print_("matched: ", repr(line)) + if line == x or match_func(x, line): + self._log("matched: ", repr(line)) break else: - raise ValueError("line %r not found in output" % line) + self._log("line %r not found in output" % line) + raise ValueError(self._log_text) def get_lines_after(self, fnline): """Return all lines following the given line in the text. @@ -1070,20 +1117,49 @@ def get_lines_after(self, fnline): """ for i, line in enumerate(self.lines): if fnline == line or fnmatch(line, fnline): - return self.lines[i+1:] + return self.lines[i + 1:] raise ValueError("line %r not found in output" % fnline) + def _log(self, *args): + self._log_output.append(' '.join((str(x) for x in args))) + + @property + def _log_text(self): + return '\n'.join(self._log_output) + def fnmatch_lines(self, lines2): - """Search the text for matching lines. + """Search captured text for matching lines using ``fnmatch.fnmatch``. The argument is a list of lines which have to match and can - use glob wildcards. If they do not match an pytest.fail() is + use glob wildcards. If they do not match a pytest.fail() is called. The matches and non-matches are also printed on stdout. """ - def show(arg1, arg2): - py.builtin.print_(arg1, arg2, file=sys.stderr) + self._match_lines(lines2, fnmatch, 'fnmatch') + + def re_match_lines(self, lines2): + """Search captured text for matching lines using ``re.match``. + + The argument is a list of lines which have to match using ``re.match``. + If they do not match a pytest.fail() is called. + + The matches and non-matches are also printed on + stdout. + """ + self._match_lines(lines2, lambda name, pat: re.match(pat, name), 're.match') + + def _match_lines(self, lines2, match_func, match_nickname): + """Underlying implementation of ``fnmatch_lines`` and ``re_match_lines``. + + :param list[str] lines2: list of string patterns to match. The actual format depends on + ``match_func``. + :param match_func: a callable ``match_func(line, pattern)`` where line is the captured + line from stdout/stderr and pattern is the matching pattern. + + :param str match_nickname: the nickname for the match function that will be logged + to stdout when a match occurs. + """ lines2 = self._getlines(lines2) lines1 = self.lines[:] nextline = None @@ -1094,17 +1170,18 @@ def show(arg1, arg2): while lines1: nextline = lines1.pop(0) if line == nextline: - show("exact match:", repr(line)) + self._log("exact match:", repr(line)) break - elif fnmatch(nextline, line): - show("fnmatch:", repr(line)) - show(" with:", repr(nextline)) + elif match_func(nextline, line): + self._log("%s:" % match_nickname, repr(line)) + self._log(" with:", repr(nextline)) break else: if not nomatchprinted: - show("nomatch:", repr(line)) + self._log("nomatch:", repr(line)) nomatchprinted = True - show(" and:", repr(nextline)) + self._log(" and:", repr(nextline)) extralines.append(nextline) else: - pytest.fail("remains unmatched: %r, see stderr" % (line,)) + self._log("remains unmatched: %r" % (line,)) + pytest.fail(self._log_text) diff --git a/_pytest/python.py b/_pytest/python.py index 3580eae074fd93b..650171a9e9c9090 100644 --- a/_pytest/python.py +++ b/_pytest/python.py @@ -1,64 +1,45 @@ """ Python test discovery, setup and run of test functions. """ +from __future__ import absolute_import, division, print_function + import fnmatch -import functools import inspect -import re -import types import sys +import os +import collections +import warnings +from textwrap import dedent +from itertools import count -import py -import pytest -from _pytest._code.code import TerminalRepr -from _pytest.mark import MarkDecorator, MarkerError -try: - import enum -except ImportError: # pragma: no cover - # Only available in Python 3.4+ or as a backport - enum = None +import py +import six +from _pytest.mark import MarkerError +from _pytest.config import hookimpl import _pytest -import _pytest._pluggy as pluggy +import pluggy +from _pytest import fixtures +from _pytest import main +from _pytest import deprecated +from _pytest.compat import ( + isclass, isfunction, is_generator, ascii_escaped, + REGEX_TYPE, STRING_TYPES, NoneType, NOTSET, + get_real_func, getfslineno, safe_getattr, + safe_str, getlocation, enum, +) +from _pytest.outcomes import fail +from _pytest.mark import transfer_markers -cutdir2 = py.path.local(_pytest.__file__).dirpath() cutdir1 = py.path.local(pluggy.__file__.rstrip("oc")) - - -NoneType = type(None) -NOTSET = object() -isfunction = inspect.isfunction -isclass = inspect.isclass -callable = py.builtin.callable -# used to work around a python2 exception info leak -exc_clear = getattr(sys, 'exc_clear', lambda: None) -# The type of re.compile objects is not exposed in Python. -REGEX_TYPE = type(re.compile('')) - -_PY3 = sys.version_info > (3, 0) -_PY2 = not _PY3 - - -if hasattr(inspect, 'signature'): - def _format_args(func): - return str(inspect.signature(func)) -else: - def _format_args(func): - return inspect.formatargspec(*inspect.getargspec(func)) - -if sys.version_info[:2] == (2, 6): - def isclass(object): - """ Return true if the object is a class. Overrides inspect.isclass for - python 2.6 because it will return True for objects which always return - something on __getattr__ calls (see #1035). - Backport of https://hg.python.org/cpython/rev/35bf8f7a8edc - """ - return isinstance(object, (type, types.ClassType)) - -def _has_positional_arg(func): - return func.__code__.co_argcount +cutdir2 = py.path.local(_pytest.__file__).dirpath() +cutdir3 = py.path.local(py.__file__).dirpath() def filter_traceback(entry): + """Return True if a TracebackEntry instance should be removed from tracebacks: + * dynamically generated code (no code to show up for it); + * internal traceback from pytest or its internal libraries, py and pluggy. + """ # entry.path might sometimes return a str object when the entry # points to dynamically generated code # see https://bitbucket.org/pytest-dev/py/issues/71 @@ -69,123 +50,12 @@ def filter_traceback(entry): # entry.path might point to an inexisting file, in which case it will # alsso return a str object. see #1133 p = py.path.local(entry.path) - return p != cutdir1 and not p.relto(cutdir2) - - -def get_real_func(obj): - """ gets the real function object of the (possibly) wrapped object by - functools.wraps or functools.partial. - """ - while hasattr(obj, "__wrapped__"): - obj = obj.__wrapped__ - if isinstance(obj, functools.partial): - obj = obj.func - return obj - -def getfslineno(obj): - # xxx let decorators etc specify a sane ordering - obj = get_real_func(obj) - if hasattr(obj, 'place_as'): - obj = obj.place_as - fslineno = _pytest._code.getfslineno(obj) - assert isinstance(fslineno[1], int), obj - return fslineno - -def getimfunc(func): - try: - return func.__func__ - except AttributeError: - try: - return func.im_func - except AttributeError: - return func - -def safe_getattr(object, name, default): - """ Like getattr but return default upon any Exception. + return p != cutdir1 and not p.relto(cutdir2) and not p.relto(cutdir3) - Attribute access can potentially fail for 'evil' Python objects. - See issue214 - """ - try: - return getattr(object, name, default) - except Exception: - return default - - -class FixtureFunctionMarker: - def __init__(self, scope, params, - autouse=False, yieldctx=False, ids=None): - self.scope = scope - self.params = params - self.autouse = autouse - self.yieldctx = yieldctx - self.ids = ids - - def __call__(self, function): - if isclass(function): - raise ValueError( - "class fixtures not supported (may be in the future)") - function._pytestfixturefunction = self - return function - - -def fixture(scope="function", params=None, autouse=False, ids=None): - """ (return a) decorator to mark a fixture factory function. - - This decorator can be used (with or or without parameters) to define - a fixture function. The name of the fixture function can later be - referenced to cause its invocation ahead of running tests: test - modules or classes can use the pytest.mark.usefixtures(fixturename) - marker. Test functions can directly use fixture names as input - arguments in which case the fixture instance returned from the fixture - function will be injected. - - :arg scope: the scope for which this fixture is shared, one of - "function" (default), "class", "module", "session". - - :arg params: an optional list of parameters which will cause multiple - invocations of the fixture function and all of the tests - using it. - - :arg autouse: if True, the fixture func is activated for all tests that - can see it. If False (the default) then an explicit - reference is needed to activate the fixture. - - :arg ids: list of string ids each corresponding to the params - so that they are part of the test id. If no ids are provided - they will be generated automatically from the params. - - """ - if callable(scope) and params is None and autouse == False: - # direct decoration - return FixtureFunctionMarker( - "function", params, autouse)(scope) - if params is not None and not isinstance(params, (list, tuple)): - params = list(params) - return FixtureFunctionMarker(scope, params, autouse, ids=ids) - -def yield_fixture(scope="function", params=None, autouse=False, ids=None): - """ (return a) decorator to mark a yield-fixture factory function - (EXPERIMENTAL). - - This takes the same arguments as :py:func:`pytest.fixture` but - expects a fixture function to use a ``yield`` instead of a ``return`` - statement to provide a fixture. See - http://pytest.org/en/latest/yieldfixture.html for more info. - """ - if callable(scope) and params is None and autouse == False: - # direct decoration - return FixtureFunctionMarker( - "function", params, autouse, yieldctx=True)(scope) - else: - return FixtureFunctionMarker(scope, params, autouse, - yieldctx=True, ids=ids) - -defaultfuncargprefixmarker = fixture() def pyobj_property(name): def get(self): - node = self.getparent(getattr(pytest, name)) + node = self.getparent(getattr(__import__('pytest'), name)) if node is not None: return node.obj doc = "python %s object this node was collected from (can be None)." % ( @@ -196,29 +66,39 @@ def get(self): def pytest_addoption(parser): group = parser.getgroup("general") group.addoption('--fixtures', '--funcargs', - action="store_true", dest="showfixtures", default=False, - help="show available fixtures, sorted by plugin appearance") + action="store_true", dest="showfixtures", default=False, + help="show available fixtures, sorted by plugin appearance") + group.addoption( + '--fixtures-per-test', + action="store_true", + dest="show_fixtures_per_test", + default=False, + help="show fixtures per test", + ) parser.addini("usefixtures", type="args", default=[], - help="list of default fixtures to be used with this project") + help="list of default fixtures to be used with this project") parser.addini("python_files", type="args", - default=['test_*.py', '*_test.py'], - help="glob-style file patterns for Python test module discovery") - parser.addini("python_classes", type="args", default=["Test",], - help="prefixes or glob names for Python test class discovery") - parser.addini("python_functions", type="args", default=["test",], - help="prefixes or glob names for Python test function and " - "method discovery") + default=['test_*.py', '*_test.py'], + help="glob-style file patterns for Python test module discovery") + parser.addini("python_classes", type="args", default=["Test", ], + help="prefixes or glob names for Python test class discovery") + parser.addini("python_functions", type="args", default=["test", ], + help="prefixes or glob names for Python test function and " + "method discovery") group.addoption("--import-mode", default="prepend", - choices=["prepend", "append"], dest="importmode", - help="prepend/append to sys.path when importing test modules, " - "default is to prepend.") + choices=["prepend", "append"], dest="importmode", + help="prepend/append to sys.path when importing test modules, " + "default is to prepend.") def pytest_cmdline_main(config): if config.option.showfixtures: showfixtures(config) return 0 + if config.option.show_fixtures_per_test: + show_fixtures_per_test(config) + return 0 def pytest_generate_tests(metafunc): @@ -236,45 +116,25 @@ def pytest_generate_tests(metafunc): for marker in markers: metafunc.parametrize(*marker.args, **marker.kwargs) + def pytest_configure(config): config.addinivalue_line("markers", - "parametrize(argnames, argvalues): call a test function multiple " - "times passing in different arguments in turn. argvalues generally " - "needs to be a list of values if argnames specifies only one name " - "or a list of tuples of values if argnames specifies multiple names. " - "Example: @parametrize('arg1', [1,2]) would lead to two calls of the " - "decorated test function, one with arg1=1 and another with arg1=2." - "see http://pytest.org/latest/parametrize.html for more info and " - "examples." - ) + "parametrize(argnames, argvalues): call a test function multiple " + "times passing in different arguments in turn. argvalues generally " + "needs to be a list of values if argnames specifies only one name " + "or a list of tuples of values if argnames specifies multiple names. " + "Example: @parametrize('arg1', [1,2]) would lead to two calls of the " + "decorated test function, one with arg1=1 and another with arg1=2." + "see http://pytest.org/latest/parametrize.html for more info and " + "examples." + ) config.addinivalue_line("markers", - "usefixtures(fixturename1, fixturename2, ...): mark tests as needing " - "all of the specified fixtures. see http://pytest.org/latest/fixture.html#usefixtures " - ) + "usefixtures(fixturename1, fixturename2, ...): mark tests as needing " + "all of the specified fixtures. see http://pytest.org/latest/fixture.html#usefixtures " + ) -def pytest_sessionstart(session): - session._fixturemanager = FixtureManager(session) - -@pytest.hookimpl(trylast=True) -def pytest_namespace(): - raises.Exception = pytest.fail.Exception - return { - 'fixture': fixture, - 'yield_fixture': yield_fixture, - 'raises' : raises, - 'collect': { - 'Module': Module, 'Class': Class, 'Instance': Instance, - 'Function': Function, 'Generator': Generator, - '_fillfuncargs': fillfixtures} - } - -@fixture(scope="session") -def pytestconfig(request): - """ the pytest config object with access to command line opts.""" - return request.config - - -@pytest.hookimpl(trylast=True) + +@hookimpl(trylast=True) def pytest_pyfunc_call(pyfuncitem): testfunction = pyfuncitem.obj if pyfuncitem._isyieldedfunction(): @@ -287,6 +147,7 @@ def pytest_pyfunc_call(pyfuncitem): testfunction(**testargs) return True + def pytest_collect_file(path, parent): ext = path.ext if ext == ".py": @@ -295,19 +156,21 @@ def pytest_collect_file(path, parent): if path.fnmatch(pat): break else: - return + return ihook = parent.session.gethookproxy(path) return ihook.pytest_pycollect_makemodule(path=path, parent=parent) + def pytest_pycollect_makemodule(path, parent): return Module(path, parent) -@pytest.hookimpl(hookwrapper=True) + +@hookimpl(hookwrapper=True) def pytest_pycollect_makeitem(collector, name, obj): outcome = yield res = outcome.get_result() if res is not None: - raise StopIteration + return # nothing was collected elsewhere, let's do it here if isclass(obj): if collector.istestclass(obj, name): @@ -320,9 +183,8 @@ def pytest_pycollect_makeitem(collector, name, obj): # or a funtools.wrapped. # We musn't if it's been wrapped with mock.patch (python 2 only) if not (isfunction(obj) or isfunction(get_real_func(obj))): - collector.warn(code="C2", message= - "cannot collect %r because it is not a function." - % name, ) + collector.warn(code="C2", message="cannot collect %r because it is not a function." + % name, ) elif getattr(obj, "__test__", True): if is_generator(obj): res = Generator(name, parent=collector) @@ -330,29 +192,30 @@ def pytest_pycollect_makeitem(collector, name, obj): res = list(collector._genfunctions(name, obj)) outcome.force_result(res) -def is_generator(func): - try: - return _pytest._code.getrawcode(func).co_flags & 32 # generator function - except AttributeError: # builtin functions have no bytecode - # assume them to not be generators - return False + +def pytest_make_parametrize_id(config, val, argname=None): + return None + class PyobjContext(object): module = pyobj_property("Module") cls = pyobj_property("Class") instance = pyobj_property("Instance") + class PyobjMixin(PyobjContext): def obj(): def fget(self): - try: - return self._obj - except AttributeError: + obj = getattr(self, '_obj', None) + if obj is None: self._obj = obj = self._getobj() - return obj + return obj + def fset(self, value): self._obj = value + return property(fget, fset, None, "underlying python object") + obj = obj() def _getobj(self): @@ -368,8 +231,7 @@ def getmodpath(self, stopatmodule=True, includemodule=False): continue name = node.name if isinstance(node, Module): - assert name.endswith(".py") - name = name[:-3] + name = os.path.splitext(name)[0] if stopatmodule: if includemodule: parts.append(name) @@ -398,7 +260,8 @@ def reportinfo(self): assert isinstance(lineno, int) return fspath, lineno, modpath -class PyCollector(PyobjMixin, pytest.Collector): + +class PyCollector(PyobjMixin, main.Collector): def funcnamefilter(self, name): return self._matches_prefix_or_glob_option('python_functions', name) @@ -416,10 +279,22 @@ def classnamefilter(self, name): return self._matches_prefix_or_glob_option('python_classes', name) def istestfunction(self, obj, name): - return ( - (self.funcnamefilter(name) or self.isnosetest(obj)) and - safe_getattr(obj, "__call__", False) and getfixturemarker(obj) is None - ) + if self.funcnamefilter(name) or self.isnosetest(obj): + if isinstance(obj, staticmethod): + # static methods need to be unwrapped + obj = safe_getattr(obj, '__func__', False) + if obj is False: + # Python 2.6 wraps in a different way that we won't try to handle + msg = "cannot collect static method %r because " \ + "it is not a function (always the case in Python 2.6)" + self.warn( + code="C2", message=msg % name) + return False + return ( + safe_getattr(obj, "__call__", False) and fixtures.getfixturemarker(obj) is None + ) + else: + return False def istestclass(self, obj, name): return self.classnamefilter(name) or self.isnosetest(obj) @@ -450,23 +325,27 @@ def collect(self): for basecls in inspect.getmro(self.obj.__class__): dicts.append(basecls.__dict__) seen = {} - l = [] + values = [] for dic in dicts: for name, obj in list(dic.items()): if name in seen: continue seen[name] = True - res = self.makeitem(name, obj) + res = self._makeitem(name, obj) if res is None: continue if not isinstance(res, list): res = [res] - l.extend(res) - l.sort(key=lambda item: item.reportinfo()[:2]) - return l + values.extend(res) + values.sort(key=lambda item: item.reportinfo()[:2]) + return values def makeitem(self, name, obj): - #assert self.ihook.fspath == self.fspath, self + warnings.warn(deprecated.COLLECTOR_MAKEITEM, stacklevel=2) + self._makeitem(name, obj) + + def _makeitem(self, name, obj): + # assert self.ihook.fspath == self.fspath, self return self.ihook.pytest_pycollect_makeitem( collector=self, name=name, obj=obj) @@ -495,110 +374,23 @@ def _genfunctions(self, name, funcobj): yield Function(name, parent=self, fixtureinfo=fixtureinfo) else: # add funcargs() as fixturedefs to fixtureinfo.arg2fixturedefs - add_funcarg_pseudo_fixture_def(self, metafunc, fm) + fixtures.add_funcarg_pseudo_fixture_def(self, metafunc, fm) for callspec in metafunc._calls: - subname = "%s[%s]" %(name, callspec.id) + subname = "%s[%s]" % (name, callspec.id) yield Function(name=subname, parent=self, callspec=callspec, callobj=funcobj, fixtureinfo=fixtureinfo, - keywords={callspec.id:True}) - -def add_funcarg_pseudo_fixture_def(collector, metafunc, fixturemanager): - # this function will transform all collected calls to a functions - # if they use direct funcargs (i.e. direct parametrization) - # because we want later test execution to be able to rely on - # an existing FixtureDef structure for all arguments. - # XXX we can probably avoid this algorithm if we modify CallSpec2 - # to directly care for creating the fixturedefs within its methods. - if not metafunc._calls[0].funcargs: - return # this function call does not have direct parametrization - # collect funcargs of all callspecs into a list of values - arg2params = {} - arg2scope = {} - for callspec in metafunc._calls: - for argname, argvalue in callspec.funcargs.items(): - assert argname not in callspec.params - callspec.params[argname] = argvalue - arg2params_list = arg2params.setdefault(argname, []) - callspec.indices[argname] = len(arg2params_list) - arg2params_list.append(argvalue) - if argname not in arg2scope: - scopenum = callspec._arg2scopenum.get(argname, - scopenum_function) - arg2scope[argname] = scopes[scopenum] - callspec.funcargs.clear() - - # register artificial FixtureDef's so that later at test execution - # time we can rely on a proper FixtureDef to exist for fixture setup. - arg2fixturedefs = metafunc._arg2fixturedefs - for argname, valuelist in arg2params.items(): - # if we have a scope that is higher than function we need - # to make sure we only ever create an according fixturedef on - # a per-scope basis. We thus store and cache the fixturedef on the - # node related to the scope. - scope = arg2scope[argname] - node = None - if scope != "function": - node = get_scope_node(collector, scope) - if node is None: - assert scope == "class" and isinstance(collector, Module) - # use module-level collector for class-scope (for now) - node = collector - if node and argname in node._name2pseudofixturedef: - arg2fixturedefs[argname] = [node._name2pseudofixturedef[argname]] - else: - fixturedef = FixtureDef(fixturemanager, '', argname, - get_direct_param_fixture_func, - arg2scope[argname], - valuelist, False, False) - arg2fixturedefs[argname] = [fixturedef] - if node is not None: - node._name2pseudofixturedef[argname] = fixturedef - - -def get_direct_param_fixture_func(request): - return request.param + keywords={callspec.id: True}, + originalname=name, + ) -class FuncFixtureInfo: - def __init__(self, argnames, names_closure, name2fixturedefs): - self.argnames = argnames - self.names_closure = names_closure - self.name2fixturedefs = name2fixturedefs - -def _marked(func, mark): - """ Returns True if :func: is already marked with :mark:, False otherwise. - This can happen if marker is applied to class and the test file is - invoked more than once. - """ - try: - func_mark = getattr(func, mark.name) - except AttributeError: - return False - return mark.args == func_mark.args and mark.kwargs == func_mark.kwargs - - -def transfer_markers(funcobj, cls, mod): - # XXX this should rather be code in the mark plugin or the mark - # plugin should merge with the python plugin. - for holder in (cls, mod): - try: - pytestmark = holder.pytestmark - except AttributeError: - continue - if isinstance(pytestmark, list): - for mark in pytestmark: - if not _marked(funcobj, mark): - mark(funcobj) - else: - if not _marked(funcobj, pytestmark): - pytestmark(funcobj) - -class Module(pytest.File, PyCollector): +class Module(main.File, PyCollector): """ Collector for test classes and functions. """ + def _getobj(self): - return self._memoizedcall('_obj', self._importtestmodule) + return self._importtestmodule() def collect(self): self.session._fixturemanager.parsefactories(self) @@ -622,49 +414,95 @@ def _importtestmodule(self): " %s\n" "HINT: remove __pycache__ / .pyc files and/or use a " "unique basename for your test file modules" - % e.args + % e.args + ) + except ImportError: + from _pytest._code.code import ExceptionInfo + exc_info = ExceptionInfo() + if self.config.getoption('verbose') < 2: + exc_info.traceback = exc_info.traceback.filter(filter_traceback) + exc_repr = exc_info.getrepr(style='short') if exc_info.traceback else exc_info.exconly() + formatted_tb = safe_str(exc_repr) + raise self.CollectError( + "ImportError while importing test module '{fspath}'.\n" + "Hint: make sure your test modules/packages have valid Python names.\n" + "Traceback:\n" + "{traceback}".format(fspath=self.fspath, traceback=formatted_tb) + ) + except _pytest.runner.Skipped as e: + if e.allow_module_level: + raise + raise self.CollectError( + "Using pytest.skip outside of a test is not allowed. " + "To decorate a test function, use the @pytest.mark.skip " + "or @pytest.mark.skipif decorators instead, and to skip a " + "module use `pytestmark = pytest.mark.{skip,skipif}." ) - #print "imported test module", mod self.config.pluginmanager.consider_module(mod) return mod def setup(self): - setup_module = xunitsetup(self.obj, "setUpModule") + setup_module = _get_xunit_setup_teardown(self.obj, "setUpModule") if setup_module is None: - setup_module = xunitsetup(self.obj, "setup_module") + setup_module = _get_xunit_setup_teardown(self.obj, "setup_module") if setup_module is not None: - #XXX: nose compat hack, move to nose plugin - # if it takes a positional arg, its probably a pytest style one - # so we pass the current module object - if _has_positional_arg(setup_module): - setup_module(self.obj) - else: - setup_module() - fin = getattr(self.obj, 'tearDownModule', None) - if fin is None: - fin = getattr(self.obj, 'teardown_module', None) - if fin is not None: - #XXX: nose compat hack, move to nose plugin - # if it takes a positional arg, it's probably a pytest style one - # so we pass the current module object - if _has_positional_arg(fin): - finalizer = lambda: fin(self.obj) - else: - finalizer = fin - self.addfinalizer(finalizer) + setup_module() + + teardown_module = _get_xunit_setup_teardown(self.obj, 'tearDownModule') + if teardown_module is None: + teardown_module = _get_xunit_setup_teardown(self.obj, 'teardown_module') + if teardown_module is not None: + self.addfinalizer(teardown_module) + + +def _get_xunit_setup_teardown(holder, attr_name, param_obj=None): + """ + Return a callable to perform xunit-style setup or teardown if + the function exists in the ``holder`` object. + The ``param_obj`` parameter is the parameter which will be passed to the function + when the callable is called without arguments, defaults to the ``holder`` object. + Return ``None`` if a suitable callable is not found. + """ + param_obj = param_obj if param_obj is not None else holder + result = _get_xunit_func(holder, attr_name) + if result is not None: + arg_count = result.__code__.co_argcount + if inspect.ismethod(result): + arg_count -= 1 + if arg_count: + return lambda: result(param_obj) + else: + return result + + +def _get_xunit_func(obj, name): + """Return the attribute from the given object to be used as a setup/teardown + xunit-style function, but only if not marked as a fixture to + avoid calling it twice. + """ + meth = getattr(obj, name, None) + if fixtures.getfixturemarker(meth) is None: + return meth class Class(PyCollector): """ Collector for test methods. """ + def collect(self): + if not safe_getattr(self.obj, "__test__", True): + return [] if hasinit(self.obj): self.warn("C1", "cannot collect test class %r because it has a " - "__init__ constructor" % self.obj.__name__) + "__init__ constructor" % self.obj.__name__) + return [] + elif hasnew(self.obj): + self.warn("C1", "cannot collect test class %r because it has a " + "__new__ constructor" % self.obj.__name__) return [] return [self._getcustomclass("Instance")(name="()", parent=self)] def setup(self): - setup_class = xunitsetup(self.obj, 'setup_class') + setup_class = _get_xunit_func(self.obj, 'setup_class') if setup_class is not None: setup_class = getattr(setup_class, 'im_func', setup_class) setup_class = getattr(setup_class, '__func__', setup_class) @@ -676,10 +514,10 @@ def setup(self): fin_class = getattr(fin_class, '__func__', fin_class) self.addfinalizer(lambda: fin_class(self.obj)) + class Instance(PyCollector): def _getobj(self): - obj = self.parent.obj() - return obj + return self.parent.obj() def collect(self): self.session._fixturemanager.parsefactories(self) @@ -689,6 +527,7 @@ def newinstance(self): self.obj = self._getobj() return self.obj + class FunctionMixin(PyobjMixin): """ mixin for the code common to Function and Generator. """ @@ -708,12 +547,12 @@ def setup(self): else: setup_name = 'setup_function' teardown_name = 'teardown_function' - setup_func_or_method = xunitsetup(obj, setup_name) + setup_func_or_method = _get_xunit_setup_teardown(obj, setup_name, param_obj=self.obj) if setup_func_or_method is not None: - setup_func_or_method(self.obj) - fin = getattr(obj, teardown_name, None) - if fin is not None: - self.addfinalizer(lambda: fin(self.obj)) + setup_func_or_method() + teardown_func_or_method = _get_xunit_setup_teardown(obj, teardown_name, param_obj=self.obj) + if teardown_func_or_method is not None: + self.addfinalizer(teardown_func_or_method) def _prunetraceback(self, excinfo): if hasattr(self, '_obj') and not self.config.option.fulltrace: @@ -724,7 +563,7 @@ def _prunetraceback(self, excinfo): if ntraceback == traceback: ntraceback = ntraceback.cut(path=path) if ntraceback == traceback: - #ntraceback = ntraceback.cut(excludepath=cutdir2) + # ntraceback = ntraceback.cut(excludepath=cutdir2) ntraceback = ntraceback.filter(filter_traceback) if not ntraceback: ntraceback = traceback @@ -738,11 +577,11 @@ def _prunetraceback(self, excinfo): entry.set_repr_style('short') def _repr_failure_py(self, excinfo, style="long"): - if excinfo.errisinstance(pytest.fail.Exception): + if excinfo.errisinstance(fail.Exception): if not excinfo.value.pytrace: return py._builtin._totext(excinfo.value) return super(FunctionMixin, self)._repr_failure_py(excinfo, - style=style) + style=style) def repr_failure(self, excinfo, outerr=None): assert outerr is None, "XXX outerr usage is deprecated" @@ -757,30 +596,32 @@ def collect(self): # test generators are seen as collectors but they also # invoke setup/teardown on popular request # (induced by the common "test_*" naming shared with normal tests) + from _pytest import deprecated self.session._setupstate.prepare(self) # see FunctionMixin.setup and test_setupstate_is_preserved_134 self._preservedparent = self.parent.obj - l = [] + values = [] seen = {} for i, x in enumerate(self.obj()): name, call, args = self.getcallargs(x) if not callable(call): - raise TypeError("%r yielded non callable test %r" %(self.obj, call,)) + raise TypeError("%r yielded non callable test %r" % (self.obj, call,)) if name is None: name = "[%d]" % i else: name = "['%s']" % name if name in seen: - raise ValueError("%r generated tests with non-unique name %r" %(self, name)) + raise ValueError("%r generated tests with non-unique name %r" % (self, name)) seen[name] = True - l.append(self.Function(name, self, args=args, callobj=call)) - return l + values.append(self.Function(name, self, args=args, callobj=call)) + self.warn('C1', deprecated.YIELD_TESTS) + return values def getcallargs(self, obj): if not isinstance(obj, (tuple, list)): obj = (obj,) - # explict naming - if isinstance(obj[0], py.builtin._basestring): + # explicit naming + if isinstance(obj[0], six.string_types): name = obj[0] obj = obj[1:] else: @@ -792,34 +633,14 @@ def getcallargs(self, obj): def hasinit(obj): init = getattr(obj, '__init__', None) if init: - if init != object.__init__: - return True + return init != object.__init__ +def hasnew(obj): + new = getattr(obj, '__new__', None) + if new: + return new != object.__new__ -def fillfixtures(function): - """ fill missing funcargs for a test function. """ - try: - request = function._request - except AttributeError: - # XXX this special code path is only expected to execute - # with the oejskit plugin. It uses classes with funcargs - # and we thus have to work a bit to allow this. - fm = function.session._fixturemanager - fi = fm.getfixtureinfo(function.parent, function.obj, None) - function._fixtureinfo = fi - request = function._request = FixtureRequest(function) - request._fillfixtures() - # prune out funcargs for jstests - newfuncargs = {} - for name in fi.argnames: - newfuncargs[name] = function.funcargs[name] - function.funcargs = newfuncargs - else: - request._fillfixtures() - - -_notexists = object() class CallSpec2(object): def __init__(self, metafunc): @@ -827,18 +648,18 @@ def __init__(self, metafunc): self.funcargs = {} self._idlist = [] self.params = {} - self._globalid = _notexists + self._globalid = NOTSET self._globalid_args = set() - self._globalparam = _notexists + self._globalparam = NOTSET self._arg2scopenum = {} # used for sorting parametrized resources - self.keywords = {} + self.marks = [] self.indices = {} def copy(self, metafunc): cs = CallSpec2(self.metafunc) cs.funcargs.update(self.funcargs) cs.params.update(self.params) - cs.keywords.update(self.keywords) + cs.marks.extend(self.marks) cs.indices.update(self.indices) cs._arg2scopenum.update(self._arg2scopenum) cs._idlist = list(self._idlist) @@ -849,13 +670,13 @@ def copy(self, metafunc): def _checkargnotcontained(self, arg): if arg in self.params or arg in self.funcargs: - raise ValueError("duplicate %r" %(arg,)) + raise ValueError("duplicate %r" % (arg,)) def getparam(self, name): try: return self.params[name] except KeyError: - if self._globalparam is _notexists: + if self._globalparam is NOTSET: raise ValueError(name) return self._globalparam @@ -863,75 +684,60 @@ def getparam(self, name): def id(self): return "-".join(map(str, filter(None, self._idlist))) - def setmulti(self, valtypes, argnames, valset, id, keywords, scopenum, - param_index): - for arg,val in zip(argnames, valset): + def setmulti2(self, valtypes, argnames, valset, id, marks, scopenum, + param_index): + for arg, val in zip(argnames, valset): self._checkargnotcontained(arg) valtype_for_arg = valtypes[arg] getattr(self, valtype_for_arg)[arg] = val self.indices[arg] = param_index self._arg2scopenum[arg] = scopenum - if val is _notexists: - self._emptyparamspecified = True self._idlist.append(id) - self.keywords.update(keywords) + self.marks.extend(marks) def setall(self, funcargs, id, param): for x in funcargs: self._checkargnotcontained(x) self.funcargs.update(funcargs) - if id is not _notexists: + if id is not NOTSET: self._idlist.append(id) - if param is not _notexists: - assert self._globalparam is _notexists + if param is not NOTSET: + assert self._globalparam is NOTSET self._globalparam = param for arg in funcargs: - self._arg2scopenum[arg] = scopenum_function + self._arg2scopenum[arg] = fixtures.scopenum_function -class FuncargnamesCompatAttr: - """ helper class so that Metafunc, Function and FixtureRequest - don't need to each define the "funcargnames" compatibility attribute. - """ - @property - def funcargnames(self): - """ alias attribute for ``fixturenames`` for pre-2.3 compatibility""" - return self.fixturenames - -class Metafunc(FuncargnamesCompatAttr): +class Metafunc(fixtures.FuncargnamesCompatAttr): """ Metafunc objects are passed to the ``pytest_generate_tests`` hook. They help to inspect a test function and to generate tests according to test configuration or values specified in the class or module where a test function is defined. - - :ivar fixturenames: set of fixture names required by the test function - - :ivar function: underlying python test function - - :ivar cls: class object where the test function is defined in or ``None``. - - :ivar module: the module object where the test function is defined in. - - :ivar config: access to the :class:`_pytest.config.Config` object for the - test session. - - :ivar funcargnames: - .. deprecated:: 2.3 - Use ``fixturenames`` instead. """ + def __init__(self, function, fixtureinfo, config, cls=None, module=None): + #: access to the :class:`_pytest.config.Config` object for the test session self.config = config + + #: the module object where the test function is defined in. self.module = module + + #: underlying python test function self.function = function + + #: set of fixture names required by the test function self.fixturenames = fixtureinfo.names_closure - self._arg2fixturedefs = fixtureinfo.name2fixturedefs + + #: class object where the test function is defined in or ``None``. self.cls = cls + self._calls = [] - self._ids = py.builtin.set() + self._ids = set() + self._arg2fixturedefs = fixtureinfo.name2fixturedefs def parametrize(self, argnames, argvalues, indirect=False, ids=None, - scope=None): + scope=None): """ Add new invocations to the underlying test function using the list of argvalues for the given argnames. Parametrization is performed during the collection phase. If you need to setup expensive resources @@ -956,7 +762,8 @@ def parametrize(self, argnames, argvalues, indirect=False, ids=None, :arg ids: list of string ids, or a callable. If strings, each is corresponding to the argvalues so that they are - part of the test id. + part of the test id. If None is given as id of specific test, the + automatically generated id for that argument will be used. If callable, it should take one argument (a single argvalue) and return a string or return None. If None, the automatically generated id for that argument will be used. @@ -968,36 +775,27 @@ def parametrize(self, argnames, argvalues, indirect=False, ids=None, It will also override any fixture-function defined scope, allowing to set a dynamic scope using test context or configuration. """ - - # individual parametrized argument sets can be wrapped in a series - # of markers in which case we unwrap the values and apply the mark - # at Function init - newkeywords = {} - unwrapped_argvalues = [] - for i, argval in enumerate(argvalues): - while isinstance(argval, MarkDecorator): - newmark = MarkDecorator(argval.markname, - argval.args[:-1], argval.kwargs) - newmarks = newkeywords.setdefault(i, {}) - newmarks[newmark.markname] = newmark - argval = argval.args[-1] - unwrapped_argvalues.append(argval) - argvalues = unwrapped_argvalues - - if not isinstance(argnames, (tuple, list)): - argnames = [x.strip() for x in argnames.split(",") if x.strip()] - if len(argnames) == 1: - argvalues = [(val,) for val in argvalues] - if not argvalues: - argvalues = [(_notexists,) * len(argnames)] + from _pytest.fixtures import scope2index + from _pytest.mark import ParameterSet + from py.io import saferepr + argnames, parameters = ParameterSet._for_parameterize( + argnames, argvalues, self.function) + del argvalues if scope is None: - scope = "function" - scopenum = scopes.index(scope) + scope = _find_parametrized_scope(argnames, self._arg2fixturedefs, indirect) + + scopenum = scope2index(scope, descr='call to {0}'.format(self.parametrize)) valtypes = {} for arg in argnames: if arg not in self.fixturenames: - raise ValueError("%r uses no fixture %r" %(self.function, arg)) + if isinstance(indirect, (tuple, list)): + name = 'fixture' if arg in indirect else 'argument' + else: + name = 'fixture' if indirect else 'argument' + raise ValueError( + "%r uses no %s %r" % ( + self.function, name, arg)) if indirect is True: valtypes = dict.fromkeys(argnames, "params") @@ -1007,33 +805,45 @@ def parametrize(self, argnames, argvalues, indirect=False, ids=None, valtypes = dict.fromkeys(argnames, "funcargs") for arg in indirect: if arg not in argnames: - raise ValueError("indirect given to %r: fixture %r doesn't exist" %( + raise ValueError("indirect given to %r: fixture %r doesn't exist" % ( self.function, arg)) valtypes[arg] = "params" idfn = None if callable(ids): idfn = ids ids = None - if ids and len(ids) != len(argvalues): - raise ValueError('%d tests specified with %d ids' %( - len(argvalues), len(ids))) - if not ids: - ids = idmaker(argnames, argvalues, idfn) + if ids: + if len(ids) != len(parameters): + raise ValueError('%d tests specified with %d ids' % ( + len(parameters), len(ids))) + for id_value in ids: + if id_value is not None and not isinstance(id_value, six.string_types): + msg = 'ids must be list of strings, found: %s (type: %s)' + raise ValueError(msg % (saferepr(id_value), type(id_value).__name__)) + ids = idmaker(argnames, parameters, idfn, ids, self.config) newcalls = [] for callspec in self._calls or [CallSpec2(self)]: - for param_index, valset in enumerate(argvalues): - assert len(valset) == len(argnames) + elements = zip(ids, parameters, count()) + for a_id, param, param_index in elements: + if len(param.values) != len(argnames): + raise ValueError( + 'In "parametrize" the number of values ({0}) must be ' + 'equal to the number of names ({1})'.format( + param.values, argnames)) newcallspec = callspec.copy(self) - newcallspec.setmulti(valtypes, argnames, valset, ids[param_index], - newkeywords.get(param_index, {}), scopenum, - param_index) + newcallspec.setmulti2(valtypes, argnames, param.values, a_id, + param.marks, scopenum, param_index) newcalls.append(newcallspec) self._calls = newcalls - def addcall(self, funcargs=None, id=_notexists, param=_notexists): - """ (deprecated, use parametrize) Add a new call to the underlying - test function during the collection phase of a test run. Note that - request.addcall() is called during the test collection phase prior and + def addcall(self, funcargs=None, id=NOTSET, param=NOTSET): + """ Add a new call to the underlying test function during the collection phase of a test run. + + .. deprecated:: 3.3 + + Use :meth:`parametrize` instead. + + Note that request.addcall() is called during the test collection phase prior and independently to actual test execution. You should only use addcall() if you need to specify multiple arguments of a test function. @@ -1046,16 +856,18 @@ def addcall(self, funcargs=None, id=_notexists, param=_notexists): :arg param: a parameter which will be exposed to a later fixture function invocation through the ``request.param`` attribute. """ + if self.config: + self.config.warn('C1', message=deprecated.METAFUNC_ADD_CALL, fslocation=None) assert funcargs is None or isinstance(funcargs, dict) if funcargs is not None: for name in funcargs: if name not in self.fixturenames: - pytest.fail("funcarg %r not used in this function." % name) + fail("funcarg %r not used in this function." % name) else: funcargs = {} if id is None: raise ValueError("id=None not allowed") - if id is _notexists: + if id is NOTSET: id = len(self._calls) id = str(id) if id in self._ids: @@ -1067,86 +879,151 @@ def addcall(self, funcargs=None, id=_notexists, param=_notexists): self._calls.append(cs) -if _PY3: - import codecs +def _find_parametrized_scope(argnames, arg2fixturedefs, indirect): + """Find the most appropriate scope for a parametrized call based on its arguments. - def _escape_bytes(val): - """ - If val is pure ascii, returns it as a str(), otherwise escapes - into a sequence of escaped bytes: - b'\xc3\xb4\xc5\xd6' -> u'\\xc3\\xb4\\xc5\\xd6' - - note: - the obvious "v.decode('unicode-escape')" will return - valid utf-8 unicode if it finds them in the string, but we - want to return escaped bytes for any byte, even if they match - a utf-8 string. - """ - if val: - # source: http://goo.gl/bGsnwC - encoded_bytes, _ = codecs.escape_encode(val) - return encoded_bytes.decode('ascii') - else: - # empty bytes crashes codecs.escape_encode (#1087) - return '' -else: - def _escape_bytes(val): - """ - In py2 bytes and str are the same type, so return it unchanged if it - is a full ascii string, otherwise escape it into its binary form. - """ - try: - return val.decode('ascii') - except UnicodeDecodeError: - return val.encode('string-escape') + When there's at least one direct argument, always use "function" scope. + When a test function is parametrized and all its arguments are indirect + (e.g. fixtures), return the most narrow scope based on the fixtures used. -def _idval(val, argname, idx, idfn): + Related to issue #1832, based on code posted by @Kingdread. + """ + from _pytest.fixtures import scopes + indirect_as_list = isinstance(indirect, (list, tuple)) + all_arguments_are_fixtures = indirect is True or \ + indirect_as_list and len(indirect) == argnames + if all_arguments_are_fixtures: + fixturedefs = arg2fixturedefs or {} + used_scopes = [fixturedef[0].scope for name, fixturedef in fixturedefs.items()] + if used_scopes: + # Takes the most narrow scope from used fixtures + for scope in reversed(scopes): + if scope in used_scopes: + return scope + + return 'function' + + +def _idval(val, argname, idx, idfn, config=None): if idfn: + s = None try: s = idfn(val) - if s: - return s except Exception: - pass - - if isinstance(val, bytes): - return _escape_bytes(val) - elif isinstance(val, (float, int, str, bool, NoneType)): + # See issue https://github.com/pytest-dev/pytest/issues/2169 + import warnings + msg = "Raised while trying to determine id of parameter %s at position %d." % (argname, idx) + msg += '\nUpdate your code as this will raise an error in pytest-4.0.' + warnings.warn(msg, DeprecationWarning) + if s: + return ascii_escaped(s) + + if config: + hook_id = config.hook.pytest_make_parametrize_id( + config=config, val=val, argname=argname) + if hook_id: + return hook_id + + if isinstance(val, STRING_TYPES): + return ascii_escaped(val) + elif isinstance(val, (float, int, bool, NoneType)): return str(val) elif isinstance(val, REGEX_TYPE): - return _escape_bytes(val.pattern) if isinstance(val.pattern, bytes) else val.pattern + return ascii_escaped(val.pattern) elif enum is not None and isinstance(val, enum.Enum): return str(val) elif isclass(val) and hasattr(val, '__name__'): return val.__name__ - elif _PY2 and isinstance(val, unicode): - # special case for python 2: if a unicode string is - # convertible to ascii, return it as an str() object instead - try: - return str(val) - except UnicodeError: - # fallthrough - pass - return str(argname)+str(idx) - -def _idvalset(idx, valset, argnames, idfn): - this_id = [_idval(val, argname, idx, idfn) - for val, argname in zip(valset, argnames)] - return "-".join(this_id) - -def idmaker(argnames, argvalues, idfn=None): - ids = [_idvalset(valindex, valset, argnames, idfn) - for valindex, valset in enumerate(argvalues)] - if len(set(ids)) < len(ids): - # user may have provided a bad idfn which means the ids are not unique - ids = [str(i) + testid for i, testid in enumerate(ids)] + return str(argname) + str(idx) + + +def _idvalset(idx, parameterset, argnames, idfn, ids, config=None): + if parameterset.id is not None: + return parameterset.id + if ids is None or (idx >= len(ids) or ids[idx] is None): + this_id = [_idval(val, argname, idx, idfn, config) + for val, argname in zip(parameterset.values, argnames)] + return "-".join(this_id) + else: + return ascii_escaped(ids[idx]) + + +def idmaker(argnames, parametersets, idfn=None, ids=None, config=None): + ids = [_idvalset(valindex, parameterset, argnames, idfn, ids, config) + for valindex, parameterset in enumerate(parametersets)] + if len(set(ids)) != len(ids): + # The ids are not unique + duplicates = [testid for testid in ids if ids.count(testid) > 1] + counters = collections.defaultdict(lambda: 0) + for index, testid in enumerate(ids): + if testid in duplicates: + ids[index] = testid + str(counters[testid]) + counters[testid] += 1 return ids + +def show_fixtures_per_test(config): + from _pytest.main import wrap_session + return wrap_session(config, _show_fixtures_per_test) + + +def _show_fixtures_per_test(config, session): + import _pytest.config + session.perform_collect() + curdir = py.path.local() + tw = _pytest.config.create_terminal_writer(config) + verbose = config.getvalue("verbose") + + def get_best_relpath(func): + loc = getlocation(func, curdir) + return curdir.bestrelpath(loc) + + def write_fixture(fixture_def): + argname = fixture_def.argname + if verbose <= 0 and argname.startswith("_"): + return + if verbose > 0: + bestrel = get_best_relpath(fixture_def.func) + funcargspec = "{0} -- {1}".format(argname, bestrel) + else: + funcargspec = argname + tw.line(funcargspec, green=True) + fixture_doc = fixture_def.func.__doc__ + if fixture_doc: + write_docstring(tw, fixture_doc) + else: + tw.line(' no docstring available', red=True) + + def write_item(item): + try: + info = item._fixtureinfo + except AttributeError: + # doctests items have no _fixtureinfo attribute + return + if not info.name2fixturedefs: + # this test item does not use any fixtures + return + tw.line() + tw.sep('-', 'fixtures used by {0}'.format(item.name)) + tw.sep('-', '({0})'.format(get_best_relpath(item.function))) + # dict key not used in loop but needed for sorting + for _, fixturedefs in sorted(info.name2fixturedefs.items()): + assert fixturedefs is not None + if not fixturedefs: + continue + # last item is expected to be the one used by the test item + write_fixture(fixturedefs[-1]) + + for session_item in session.items: + write_item(session_item) + + def showfixtures(config): from _pytest.main import wrap_session return wrap_session(config, _showfixtures_main) + def _showfixtures_main(config, session): import _pytest.config session.perform_collect() @@ -1157,16 +1034,21 @@ def _showfixtures_main(config, session): fm = session._fixturemanager available = [] + seen = set() + for argname, fixturedefs in fm._arg2fixturedefs.items(): assert fixturedefs is not None if not fixturedefs: continue - fixturedef = fixturedefs[-1] - loc = getlocation(fixturedef.func, curdir) - available.append((len(fixturedef.baseid), - fixturedef.func.__module__, - curdir.bestrelpath(loc), - fixturedef.argname, fixturedef)) + for fixturedef in fixturedefs: + loc = getlocation(fixturedef.func, curdir) + if (fixturedef.argname, loc) in seen: + continue + seen.add((fixturedef.argname, loc)) + available.append((len(fixturedef.baseid), + fixturedef.func.__module__, + curdir.bestrelpath(loc), + fixturedef.argname, fixturedef)) available.sort() currentmodule = None @@ -1174,180 +1056,49 @@ def _showfixtures_main(config, session): if currentmodule != module: if not module.startswith("_pytest."): tw.line() - tw.sep("-", "fixtures defined from %s" %(module,)) + tw.sep("-", "fixtures defined from %s" % (module,)) currentmodule = module if verbose <= 0 and argname[0] == "_": continue if verbose > 0: - funcargspec = "%s -- %s" %(argname, bestrel,) + funcargspec = "%s -- %s" % (argname, bestrel,) else: funcargspec = argname tw.line(funcargspec, green=True) loc = getlocation(fixturedef.func, curdir) doc = fixturedef.func.__doc__ or "" if doc: - for line in doc.strip().split("\n"): - tw.line(" " + line.strip()) + write_docstring(tw, doc) else: - tw.line(" %s: no docstring available" %(loc,), - red=True) - -def getlocation(function, curdir): - import inspect - fn = py.path.local(inspect.getfile(function)) - lineno = py.builtin._getcode(function).co_firstlineno - if fn.relto(curdir): - fn = fn.relto(curdir) - return "%s:%d" %(fn, lineno+1) - -# builtin pytest.raises helper - -def raises(expected_exception, *args, **kwargs): - """ assert that a code block/function call raises ``expected_exception`` - and raise a failure exception otherwise. - - This helper produces a ``ExceptionInfo()`` object (see below). - - If using Python 2.5 or above, you may use this function as a - context manager:: - - >>> with raises(ZeroDivisionError): - ... 1/0 - - .. note:: - - When using ``pytest.raises`` as a context manager, it's worthwhile to - note that normal context manager rules apply and that the exception - raised *must* be the final line in the scope of the context manager. - Lines of code after that, within the scope of the context manager will - not be executed. For example:: - - >>> with raises(OSError) as exc_info: - assert 1 == 1 # this will execute as expected - raise OSError(errno.EEXISTS, 'directory exists') - assert exc_info.value.errno == errno.EEXISTS # this will not execute - - Instead, the following approach must be taken (note the difference in - scope):: - - >>> with raises(OSError) as exc_info: - assert 1 == 1 # this will execute as expected - raise OSError(errno.EEXISTS, 'directory exists') - - assert exc_info.value.errno == errno.EEXISTS # this will now execute - - Or you can specify a callable by passing a to-be-called lambda:: - - >>> raises(ZeroDivisionError, lambda: 1/0) - - - or you can specify an arbitrary callable with arguments:: - - >>> def f(x): return 1/x - ... - >>> raises(ZeroDivisionError, f, 0) - - >>> raises(ZeroDivisionError, f, x=0) - + tw.line(" %s: no docstring available" % (loc,), + red=True) - A third possibility is to use a string to be executed:: - >>> raises(ZeroDivisionError, "f(0)") - +def write_docstring(tw, doc): + INDENT = " " + doc = doc.rstrip() + if "\n" in doc: + firstline, rest = doc.split("\n", 1) + else: + firstline, rest = doc, "" - .. autoclass:: _pytest._code.ExceptionInfo - :members: + if firstline.strip(): + tw.line(INDENT + firstline.strip()) - .. note:: - Similar to caught exception objects in Python, explicitly clearing - local references to returned ``ExceptionInfo`` objects can - help the Python interpreter speed up its garbage collection. + if rest: + for line in dedent(rest).split("\n"): + tw.write(INDENT + line + "\n") - Clearing those references breaks a reference cycle - (``ExceptionInfo`` --> caught exception --> frame stack raising - the exception --> current frame stack --> local variables --> - ``ExceptionInfo``) which makes Python keep all objects referenced - from that cycle (including all local variables in the current - frame) alive until the next cyclic garbage collection run. See the - official Python ``try`` statement documentation for more detailed - information. - """ - __tracebackhide__ = True - if expected_exception is AssertionError: - # we want to catch a AssertionError - # replace our subclass with the builtin one - # see https://github.com/pytest-dev/pytest/issues/176 - from _pytest.assertion.util import BuiltinAssertionError \ - as expected_exception - msg = ("exceptions must be old-style classes or" - " derived from BaseException, not %s") - if isinstance(expected_exception, tuple): - for exc in expected_exception: - if not isclass(exc): - raise TypeError(msg % type(exc)) - elif not isclass(expected_exception): - raise TypeError(msg % type(expected_exception)) - - if not args: - return RaisesContext(expected_exception) - elif isinstance(args[0], str): - code, = args - assert isinstance(code, str) - frame = sys._getframe(1) - loc = frame.f_locals.copy() - loc.update(kwargs) - #print "raises frame scope: %r" % frame.f_locals - try: - code = _pytest._code.Source(code).compile() - py.builtin.exec_(code, frame.f_globals, loc) - # XXX didn'T mean f_globals == f_locals something special? - # this is destroyed here ... - except expected_exception: - return _pytest._code.ExceptionInfo() - else: - func = args[0] - try: - func(*args[1:], **kwargs) - except expected_exception: - return _pytest._code.ExceptionInfo() - pytest.fail("DID NOT RAISE {0}".format(expected_exception)) - -class RaisesContext(object): - def __init__(self, expected_exception): - self.expected_exception = expected_exception - self.excinfo = None - - def __enter__(self): - self.excinfo = object.__new__(_pytest._code.ExceptionInfo) - return self.excinfo - - def __exit__(self, *tp): - __tracebackhide__ = True - if tp[0] is None: - pytest.fail("DID NOT RAISE") - if sys.version_info < (2, 7): - # py26: on __exit__() exc_value often does not contain the - # exception value. - # http://bugs.python.org/issue7853 - if not isinstance(tp[1], BaseException): - exc_type, value, traceback = tp - tp = exc_type, exc_type(value), traceback - self.excinfo.__init__(tp) - return issubclass(self.excinfo.type, self.expected_exception) - -# -# the basic pytest Function item -# - -class Function(FunctionMixin, pytest.Item, FuncargnamesCompatAttr): +class Function(FunctionMixin, main.Item, fixtures.FuncargnamesCompatAttr): """ a Function Item is responsible for setting up and executing a Python test function. """ _genid = None + def __init__(self, name, parent, args=None, config=None, callspec=None, callobj=NOTSET, keywords=None, session=None, - fixtureinfo=None): + fixtureinfo=None, originalname=None): super(Function, self).__init__(name, parent, config=config, session=session) self._args = args @@ -1357,7 +1108,13 @@ def __init__(self, name, parent, args=None, config=None, self.keywords.update(self.obj.__dict__) if callspec: self.callspec = callspec - self.keywords.update(callspec.keywords) + # this is total hostile and a mess + # keywords are broken by design by now + # this will be redeemed later + for mark in callspec.marks: + # feel free to cry, this was broken for years before + # and keywords cant fix it per design + self.keywords[mark.name] = mark if keywords: self.keywords.update(keywords) @@ -1369,6 +1126,12 @@ def __init__(self, name, parent, args=None, config=None, self.fixturenames = fixtureinfo.names_closure self._initrequest() + #: original function name, without any decorations (for example + #: parametrization adds a ``"[...]"`` suffix to function names). + #: + #: .. versionadded:: 3.0 + self.originalname = originalname + def _initrequest(self): self.funcargs = {} if self._isyieldedfunction(): @@ -1381,7 +1144,7 @@ def _initrequest(self): self._genid = callspec.id if hasattr(callspec, "param"): self.param = callspec.param - self._request = FixtureRequest(self) + self._request = fixtures.FixtureRequest(self) @property def function(self): @@ -1390,7 +1153,7 @@ def function(self): def _getobj(self): name = self.name - i = name.find("[") # parametrization + i = name.find("[") # parametrization if i != -1: name = name[:i] return getattr(self.parent.obj, name) @@ -1408,895 +1171,5 @@ def runtest(self): self.ihook.pytest_pyfunc_call(pyfuncitem=self) def setup(self): - # check if parametrization happend with an empty list - try: - self.callspec._emptyparamspecified - except AttributeError: - pass - else: - fs, lineno = self._getfslineno() - pytest.skip("got empty parameter set, function %s at %s:%d" %( - self.function.__name__, fs, lineno)) super(Function, self).setup() - fillfixtures(self) - - -scope2props = dict(session=()) -scope2props["module"] = ("fspath", "module") -scope2props["class"] = scope2props["module"] + ("cls",) -scope2props["instance"] = scope2props["class"] + ("instance", ) -scope2props["function"] = scope2props["instance"] + ("function", "keywords") - -def scopeproperty(name=None, doc=None): - def decoratescope(func): - scopename = name or func.__name__ - def provide(self): - if func.__name__ in scope2props[self.scope]: - return func(self) - raise AttributeError("%s not available in %s-scoped context" % ( - scopename, self.scope)) - return property(provide, None, None, func.__doc__) - return decoratescope - - -class FixtureRequest(FuncargnamesCompatAttr): - """ A request for a fixture from a test or fixture function. - - A request object gives access to the requesting test context - and has an optional ``param`` attribute in case - the fixture is parametrized indirectly. - """ - - def __init__(self, pyfuncitem): - self._pyfuncitem = pyfuncitem - #: fixture for which this request is being performed - self.fixturename = None - #: Scope string, one of "function", "cls", "module", "session" - self.scope = "function" - self._funcargs = {} - self._fixturedefs = {} - fixtureinfo = pyfuncitem._fixtureinfo - self._arg2fixturedefs = fixtureinfo.name2fixturedefs.copy() - self._arg2index = {} - self.fixturenames = fixtureinfo.names_closure - self._fixturemanager = pyfuncitem.session._fixturemanager - - @property - def node(self): - """ underlying collection node (depends on current request scope)""" - return self._getscopeitem(self.scope) - - - def _getnextfixturedef(self, argname): - fixturedefs = self._arg2fixturedefs.get(argname, None) - if fixturedefs is None: - # we arrive here because of a a dynamic call to - # getfuncargvalue(argname) usage which was naturally - # not known at parsing/collection time - fixturedefs = self._fixturemanager.getfixturedefs( - argname, self._pyfuncitem.parent.nodeid) - self._arg2fixturedefs[argname] = fixturedefs - # fixturedefs list is immutable so we maintain a decreasing index - index = self._arg2index.get(argname, 0) - 1 - if fixturedefs is None or (-index > len(fixturedefs)): - raise FixtureLookupError(argname, self) - self._arg2index[argname] = index - return fixturedefs[index] - - @property - def config(self): - """ the pytest config object associated with this request. """ - return self._pyfuncitem.config - - - @scopeproperty() - def function(self): - """ test function object if the request has a per-function scope. """ - return self._pyfuncitem.obj - - @scopeproperty("class") - def cls(self): - """ class (can be None) where the test function was collected. """ - clscol = self._pyfuncitem.getparent(pytest.Class) - if clscol: - return clscol.obj - - @property - def instance(self): - """ instance (can be None) on which test function was collected. """ - # unittest support hack, see _pytest.unittest.TestCaseFunction - try: - return self._pyfuncitem._testcase - except AttributeError: - function = getattr(self, "function", None) - if function is not None: - return py.builtin._getimself(function) - - @scopeproperty() - def module(self): - """ python module object where the test function was collected. """ - return self._pyfuncitem.getparent(pytest.Module).obj - - @scopeproperty() - def fspath(self): - """ the file system path of the test module which collected this test. """ - return self._pyfuncitem.fspath - - @property - def keywords(self): - """ keywords/markers dictionary for the underlying node. """ - return self.node.keywords - - @property - def session(self): - """ pytest session object. """ - return self._pyfuncitem.session - - def addfinalizer(self, finalizer): - """ add finalizer/teardown function to be called after the - last test within the requesting test context finished - execution. """ - # XXX usually this method is shadowed by fixturedef specific ones - self._addfinalizer(finalizer, scope=self.scope) - - def _addfinalizer(self, finalizer, scope): - colitem = self._getscopeitem(scope) - self._pyfuncitem.session._setupstate.addfinalizer( - finalizer=finalizer, colitem=colitem) - - def applymarker(self, marker): - """ Apply a marker to a single test function invocation. - This method is useful if you don't want to have a keyword/marker - on all function invocations. - - :arg marker: a :py:class:`_pytest.mark.MarkDecorator` object - created by a call to ``pytest.mark.NAME(...)``. - """ - try: - self.node.keywords[marker.markname] = marker - except AttributeError: - raise ValueError(marker) - - def raiseerror(self, msg): - """ raise a FixtureLookupError with the given message. """ - raise self._fixturemanager.FixtureLookupError(None, self, msg) - - def _fillfixtures(self): - item = self._pyfuncitem - fixturenames = getattr(item, "fixturenames", self.fixturenames) - for argname in fixturenames: - if argname not in item.funcargs: - item.funcargs[argname] = self.getfuncargvalue(argname) - - def cached_setup(self, setup, teardown=None, scope="module", extrakey=None): - """ (deprecated) Return a testing resource managed by ``setup`` & - ``teardown`` calls. ``scope`` and ``extrakey`` determine when the - ``teardown`` function will be called so that subsequent calls to - ``setup`` would recreate the resource. With pytest-2.3 you often - do not need ``cached_setup()`` as you can directly declare a scope - on a fixture function and register a finalizer through - ``request.addfinalizer()``. - - :arg teardown: function receiving a previously setup resource. - :arg setup: a no-argument function creating a resource. - :arg scope: a string value out of ``function``, ``class``, ``module`` - or ``session`` indicating the caching lifecycle of the resource. - :arg extrakey: added to internal caching key of (funcargname, scope). - """ - if not hasattr(self.config, '_setupcache'): - self.config._setupcache = {} # XXX weakref? - cachekey = (self.fixturename, self._getscopeitem(scope), extrakey) - cache = self.config._setupcache - try: - val = cache[cachekey] - except KeyError: - self._check_scope(self.fixturename, self.scope, scope) - val = setup() - cache[cachekey] = val - if teardown is not None: - def finalizer(): - del cache[cachekey] - teardown(val) - self._addfinalizer(finalizer, scope=scope) - return val - - def getfuncargvalue(self, argname): - """ Dynamically retrieve a named fixture function argument. - - As of pytest-2.3, it is easier and usually better to access other - fixture values by stating it as an input argument in the fixture - function. If you only can decide about using another fixture at test - setup time, you may use this function to retrieve it inside a fixture - function body. - """ - return self._get_active_fixturedef(argname).cached_result[0] - - def _get_active_fixturedef(self, argname): - try: - return self._fixturedefs[argname] - except KeyError: - try: - fixturedef = self._getnextfixturedef(argname) - except FixtureLookupError: - if argname == "request": - class PseudoFixtureDef: - cached_result = (self, [0], None) - scope = "function" - return PseudoFixtureDef - raise - # remove indent to prevent the python3 exception - # from leaking into the call - result = self._getfuncargvalue(fixturedef) - self._funcargs[argname] = result - self._fixturedefs[argname] = fixturedef - return fixturedef - - def _get_fixturestack(self): - current = self - l = [] - while 1: - fixturedef = getattr(current, "_fixturedef", None) - if fixturedef is None: - l.reverse() - return l - l.append(fixturedef) - current = current._parent_request - - def _getfuncargvalue(self, fixturedef): - # prepare a subrequest object before calling fixture function - # (latter managed by fixturedef) - argname = fixturedef.argname - funcitem = self._pyfuncitem - scope = fixturedef.scope - try: - param = funcitem.callspec.getparam(argname) - except (AttributeError, ValueError): - param = NOTSET - param_index = 0 - else: - # indices might not be set if old-style metafunc.addcall() was used - param_index = funcitem.callspec.indices.get(argname, 0) - # if a parametrize invocation set a scope it will override - # the static scope defined with the fixture function - paramscopenum = funcitem.callspec._arg2scopenum.get(argname) - if paramscopenum is not None: - scope = scopes[paramscopenum] - - subrequest = SubRequest(self, scope, param, param_index, fixturedef) - - # check if a higher-level scoped fixture accesses a lower level one - subrequest._check_scope(argname, self.scope, scope) - - # clear sys.exc_info before invoking the fixture (python bug?) - # if its not explicitly cleared it will leak into the call - exc_clear() - try: - # call the fixture function - val = fixturedef.execute(request=subrequest) - finally: - # if fixture function failed it might have registered finalizers - self.session._setupstate.addfinalizer(fixturedef.finish, - subrequest.node) - return val - - def _check_scope(self, argname, invoking_scope, requested_scope): - if argname == "request": - return - if scopemismatch(invoking_scope, requested_scope): - # try to report something helpful - lines = self._factorytraceback() - pytest.fail("ScopeMismatch: You tried to access the %r scoped " - "fixture %r with a %r scoped request object, " - "involved factories\n%s" %( - (requested_scope, argname, invoking_scope, "\n".join(lines))), - pytrace=False) - - def _factorytraceback(self): - lines = [] - for fixturedef in self._get_fixturestack(): - factory = fixturedef.func - fs, lineno = getfslineno(factory) - p = self._pyfuncitem.session.fspath.bestrelpath(fs) - args = _format_args(factory) - lines.append("%s:%d: def %s%s" %( - p, lineno, factory.__name__, args)) - return lines - - def _getscopeitem(self, scope): - if scope == "function": - # this might also be a non-function Item despite its attribute name - return self._pyfuncitem - node = get_scope_node(self._pyfuncitem, scope) - if node is None and scope == "class": - # fallback to function item itself - node = self._pyfuncitem - assert node - return node - - def __repr__(self): - return "" %(self.node) - - -class SubRequest(FixtureRequest): - """ a sub request for handling getting a fixture from a - test function/fixture. """ - def __init__(self, request, scope, param, param_index, fixturedef): - self._parent_request = request - self.fixturename = fixturedef.argname - if param is not NOTSET: - self.param = param - self.param_index = param_index - self.scope = scope - self._fixturedef = fixturedef - self.addfinalizer = fixturedef.addfinalizer - self._pyfuncitem = request._pyfuncitem - self._funcargs = request._funcargs - self._fixturedefs = request._fixturedefs - self._arg2fixturedefs = request._arg2fixturedefs - self._arg2index = request._arg2index - self.fixturenames = request.fixturenames - self._fixturemanager = request._fixturemanager - - def __repr__(self): - return "" % (self.fixturename, self._pyfuncitem) - - -class ScopeMismatchError(Exception): - """ A fixture function tries to use a different fixture function which - which has a lower scope (e.g. a Session one calls a function one) - """ - -scopes = "session module class function".split() -scopenum_function = scopes.index("function") -def scopemismatch(currentscope, newscope): - return scopes.index(newscope) > scopes.index(currentscope) - - -class FixtureLookupError(LookupError): - """ could not return a requested Fixture (missing or invalid). """ - def __init__(self, argname, request, msg=None): - self.argname = argname - self.request = request - self.fixturestack = request._get_fixturestack() - self.msg = msg - - def formatrepr(self): - tblines = [] - addline = tblines.append - stack = [self.request._pyfuncitem.obj] - stack.extend(map(lambda x: x.func, self.fixturestack)) - msg = self.msg - if msg is not None: - # the last fixture raise an error, let's present - # it at the requesting side - stack = stack[:-1] - for function in stack: - fspath, lineno = getfslineno(function) - try: - lines, _ = inspect.getsourcelines(get_real_func(function)) - except (IOError, IndexError): - error_msg = "file %s, line %s: source code not available" - addline(error_msg % (fspath, lineno+1)) - else: - addline("file %s, line %s" % (fspath, lineno+1)) - for i, line in enumerate(lines): - line = line.rstrip() - addline(" " + line) - if line.lstrip().startswith('def'): - break - - if msg is None: - fm = self.request._fixturemanager - available = [] - for name, fixturedef in fm._arg2fixturedefs.items(): - parentid = self.request._pyfuncitem.parent.nodeid - faclist = list(fm._matchfactories(fixturedef, parentid)) - if faclist: - available.append(name) - msg = "fixture %r not found" % (self.argname,) - msg += "\n available fixtures: %s" %(", ".join(available),) - msg += "\n use 'py.test --fixtures [testpath]' for help on them." - - return FixtureLookupErrorRepr(fspath, lineno, tblines, msg, self.argname) - -class FixtureLookupErrorRepr(TerminalRepr): - def __init__(self, filename, firstlineno, tblines, errorstring, argname): - self.tblines = tblines - self.errorstring = errorstring - self.filename = filename - self.firstlineno = firstlineno - self.argname = argname - - def toterminal(self, tw): - #tw.line("FixtureLookupError: %s" %(self.argname), red=True) - for tbline in self.tblines: - tw.line(tbline.rstrip()) - for line in self.errorstring.split("\n"): - tw.line(" " + line.strip(), red=True) - tw.line() - tw.line("%s:%d" % (self.filename, self.firstlineno+1)) - -class FixtureManager: - """ - pytest fixtures definitions and information is stored and managed - from this class. - - During collection fm.parsefactories() is called multiple times to parse - fixture function definitions into FixtureDef objects and internal - data structures. - - During collection of test functions, metafunc-mechanics instantiate - a FuncFixtureInfo object which is cached per node/func-name. - This FuncFixtureInfo object is later retrieved by Function nodes - which themselves offer a fixturenames attribute. - - The FuncFixtureInfo object holds information about fixtures and FixtureDefs - relevant for a particular function. An initial list of fixtures is - assembled like this: - - - ini-defined usefixtures - - autouse-marked fixtures along the collection chain up from the function - - usefixtures markers at module/class/function level - - test function funcargs - - Subsequently the funcfixtureinfo.fixturenames attribute is computed - as the closure of the fixtures needed to setup the initial fixtures, - i. e. fixtures needed by fixture functions themselves are appended - to the fixturenames list. - - Upon the test-setup phases all fixturenames are instantiated, retrieved - by a lookup of their FuncFixtureInfo. - """ - - _argprefix = "pytest_funcarg__" - FixtureLookupError = FixtureLookupError - FixtureLookupErrorRepr = FixtureLookupErrorRepr - - def __init__(self, session): - self.session = session - self.config = session.config - self._arg2fixturedefs = {} - self._holderobjseen = set() - self._arg2finish = {} - self._nodeid_and_autousenames = [("", self.config.getini("usefixtures"))] - session.config.pluginmanager.register(self, "funcmanage") - - - def getfixtureinfo(self, node, func, cls, funcargs=True): - if funcargs and not hasattr(node, "nofuncargs"): - if cls is not None: - startindex = 1 - else: - startindex = None - argnames = getfuncargnames(func, startindex) - else: - argnames = () - usefixtures = getattr(func, "usefixtures", None) - initialnames = argnames - if usefixtures is not None: - initialnames = usefixtures.args + initialnames - fm = node.session._fixturemanager - names_closure, arg2fixturedefs = fm.getfixtureclosure(initialnames, - node) - return FuncFixtureInfo(argnames, names_closure, arg2fixturedefs) - - def pytest_plugin_registered(self, plugin): - nodeid = None - try: - p = py.path.local(plugin.__file__) - except AttributeError: - pass - else: - # construct the base nodeid which is later used to check - # what fixtures are visible for particular tests (as denoted - # by their test id) - if p.basename.startswith("conftest.py"): - nodeid = p.dirpath().relto(self.config.rootdir) - if p.sep != "/": - nodeid = nodeid.replace(p.sep, "/") - self.parsefactories(plugin, nodeid) - - def _getautousenames(self, nodeid): - """ return a tuple of fixture names to be used. """ - autousenames = [] - for baseid, basenames in self._nodeid_and_autousenames: - if nodeid.startswith(baseid): - if baseid: - i = len(baseid) - nextchar = nodeid[i:i+1] - if nextchar and nextchar not in ":/": - continue - autousenames.extend(basenames) - # make sure autousenames are sorted by scope, scopenum 0 is session - autousenames.sort( - key=lambda x: self._arg2fixturedefs[x][-1].scopenum) - return autousenames - - def getfixtureclosure(self, fixturenames, parentnode): - # collect the closure of all fixtures , starting with the given - # fixturenames as the initial set. As we have to visit all - # factory definitions anyway, we also return a arg2fixturedefs - # mapping so that the caller can reuse it and does not have - # to re-discover fixturedefs again for each fixturename - # (discovering matching fixtures for a given name/node is expensive) - - parentid = parentnode.nodeid - fixturenames_closure = self._getautousenames(parentid) - def merge(otherlist): - for arg in otherlist: - if arg not in fixturenames_closure: - fixturenames_closure.append(arg) - merge(fixturenames) - arg2fixturedefs = {} - lastlen = -1 - while lastlen != len(fixturenames_closure): - lastlen = len(fixturenames_closure) - for argname in fixturenames_closure: - if argname in arg2fixturedefs: - continue - fixturedefs = self.getfixturedefs(argname, parentid) - if fixturedefs: - arg2fixturedefs[argname] = fixturedefs - merge(fixturedefs[-1].argnames) - return fixturenames_closure, arg2fixturedefs - - def pytest_generate_tests(self, metafunc): - for argname in metafunc.fixturenames: - faclist = metafunc._arg2fixturedefs.get(argname) - if faclist: - fixturedef = faclist[-1] - if fixturedef.params is not None: - func_params = getattr(getattr(metafunc.function, 'parametrize', None), 'args', [[None]]) - # skip directly parametrized arguments - argnames = func_params[0] - if not isinstance(argnames, (tuple, list)): - argnames = [x.strip() for x in argnames.split(",") if x.strip()] - if argname not in func_params and argname not in argnames: - metafunc.parametrize(argname, fixturedef.params, - indirect=True, scope=fixturedef.scope, - ids=fixturedef.ids) - else: - continue # will raise FixtureLookupError at setup time - - def pytest_collection_modifyitems(self, items): - # separate parametrized setups - items[:] = reorder_items(items) - - def parsefactories(self, node_or_obj, nodeid=NOTSET, unittest=False): - if nodeid is not NOTSET: - holderobj = node_or_obj - else: - holderobj = node_or_obj.obj - nodeid = node_or_obj.nodeid - if holderobj in self._holderobjseen: - return - self._holderobjseen.add(holderobj) - autousenames = [] - for name in dir(holderobj): - obj = getattr(holderobj, name, None) - # fixture functions have a pytest_funcarg__ prefix (pre-2.3 style) - # or are "@pytest.fixture" marked - marker = getfixturemarker(obj) - if marker is None: - if not name.startswith(self._argprefix): - continue - if not callable(obj): - continue - marker = defaultfuncargprefixmarker - name = name[len(self._argprefix):] - elif not isinstance(marker, FixtureFunctionMarker): - # magic globals with __getattr__ might have got us a wrong - # fixture attribute - continue - else: - assert not name.startswith(self._argprefix) - fixturedef = FixtureDef(self, nodeid, name, obj, - marker.scope, marker.params, - yieldctx=marker.yieldctx, - unittest=unittest, ids=marker.ids) - faclist = self._arg2fixturedefs.setdefault(name, []) - if fixturedef.has_location: - faclist.append(fixturedef) - else: - # fixturedefs with no location are at the front - # so this inserts the current fixturedef after the - # existing fixturedefs from external plugins but - # before the fixturedefs provided in conftests. - i = len([f for f in faclist if not f.has_location]) - faclist.insert(i, fixturedef) - if marker.autouse: - autousenames.append(name) - if autousenames: - self._nodeid_and_autousenames.append((nodeid or '', autousenames)) - - def getfixturedefs(self, argname, nodeid): - try: - fixturedefs = self._arg2fixturedefs[argname] - except KeyError: - return None - else: - return tuple(self._matchfactories(fixturedefs, nodeid)) - - def _matchfactories(self, fixturedefs, nodeid): - for fixturedef in fixturedefs: - if nodeid.startswith(fixturedef.baseid): - yield fixturedef - - -def fail_fixturefunc(fixturefunc, msg): - fs, lineno = getfslineno(fixturefunc) - location = "%s:%s" % (fs, lineno+1) - source = _pytest._code.Source(fixturefunc) - pytest.fail(msg + ":\n\n" + str(source.indent()) + "\n" + location, - pytrace=False) - -def call_fixture_func(fixturefunc, request, kwargs, yieldctx): - if yieldctx: - if not is_generator(fixturefunc): - fail_fixturefunc(fixturefunc, - msg="yield_fixture requires yield statement in function") - iter = fixturefunc(**kwargs) - next = getattr(iter, "__next__", None) - if next is None: - next = getattr(iter, "next") - res = next() - def teardown(): - try: - next() - except StopIteration: - pass - else: - fail_fixturefunc(fixturefunc, - "yield_fixture function has more than one 'yield'") - request.addfinalizer(teardown) - else: - if is_generator(fixturefunc): - fail_fixturefunc(fixturefunc, - msg="pytest.fixture functions cannot use ``yield``. " - "Instead write and return an inner function/generator " - "and let the consumer call and iterate over it.") - res = fixturefunc(**kwargs) - return res - -class FixtureDef: - """ A container for a factory definition. """ - def __init__(self, fixturemanager, baseid, argname, func, scope, params, - yieldctx, unittest=False, ids=None): - self._fixturemanager = fixturemanager - self.baseid = baseid or '' - self.has_location = baseid is not None - self.func = func - self.argname = argname - self.scope = scope - self.scopenum = scopes.index(scope or "function") - self.params = params - startindex = unittest and 1 or None - self.argnames = getfuncargnames(func, startindex=startindex) - self.yieldctx = yieldctx - self.unittest = unittest - self.ids = ids - self._finalizer = [] - - def addfinalizer(self, finalizer): - self._finalizer.append(finalizer) - - def finish(self): - try: - while self._finalizer: - func = self._finalizer.pop() - func() - finally: - # even if finalization fails, we invalidate - # the cached fixture value - if hasattr(self, "cached_result"): - del self.cached_result - - def execute(self, request): - # get required arguments and register our own finish() - # with their finalization - kwargs = {} - for argname in self.argnames: - fixturedef = request._get_active_fixturedef(argname) - result, arg_cache_key, exc = fixturedef.cached_result - request._check_scope(argname, request.scope, fixturedef.scope) - kwargs[argname] = result - if argname != "request": - fixturedef.addfinalizer(self.finish) - - my_cache_key = request.param_index - cached_result = getattr(self, "cached_result", None) - if cached_result is not None: - result, cache_key, err = cached_result - if my_cache_key == cache_key: - if err is not None: - py.builtin._reraise(*err) - else: - return result - # we have a previous but differently parametrized fixture instance - # so we need to tear it down before creating a new one - self.finish() - assert not hasattr(self, "cached_result") - - fixturefunc = self.func - - if self.unittest: - if request.instance is not None: - # bind the unbound method to the TestCase instance - fixturefunc = self.func.__get__(request.instance) - else: - # the fixture function needs to be bound to the actual - # request.instance so that code working with "self" behaves - # as expected. - if request.instance is not None: - fixturefunc = getimfunc(self.func) - if fixturefunc != self.func: - fixturefunc = fixturefunc.__get__(request.instance) - - try: - result = call_fixture_func(fixturefunc, request, kwargs, - self.yieldctx) - except Exception: - self.cached_result = (None, my_cache_key, sys.exc_info()) - raise - self.cached_result = (result, my_cache_key, None) - return result - - def __repr__(self): - return ("" % - (self.argname, self.scope, self.baseid)) - -def num_mock_patch_args(function): - """ return number of arguments used up by mock arguments (if any) """ - patchings = getattr(function, "patchings", None) - if not patchings: - return 0 - mock = sys.modules.get("mock", sys.modules.get("unittest.mock", None)) - if mock is not None: - return len([p for p in patchings - if not p.attribute_name and p.new is mock.DEFAULT]) - return len(patchings) - - -def getfuncargnames(function, startindex=None): - # XXX merge with main.py's varnames - #assert not isclass(function) - realfunction = function - while hasattr(realfunction, "__wrapped__"): - realfunction = realfunction.__wrapped__ - if startindex is None: - startindex = inspect.ismethod(function) and 1 or 0 - if realfunction != function: - startindex += num_mock_patch_args(function) - function = realfunction - if isinstance(function, functools.partial): - argnames = inspect.getargs(_pytest._code.getrawcode(function.func))[0] - partial = function - argnames = argnames[len(partial.args):] - if partial.keywords: - for kw in partial.keywords: - argnames.remove(kw) - else: - argnames = inspect.getargs(_pytest._code.getrawcode(function))[0] - defaults = getattr(function, 'func_defaults', - getattr(function, '__defaults__', None)) or () - numdefaults = len(defaults) - if numdefaults: - return tuple(argnames[startindex:-numdefaults]) - return tuple(argnames[startindex:]) - -# algorithm for sorting on a per-parametrized resource setup basis -# it is called for scopenum==0 (session) first and performs sorting -# down to the lower scopes such as to minimize number of "high scope" -# setups and teardowns - -def reorder_items(items): - argkeys_cache = {} - for scopenum in range(0, scopenum_function): - argkeys_cache[scopenum] = d = {} - for item in items: - keys = set(get_parametrized_fixture_keys(item, scopenum)) - if keys: - d[item] = keys - return reorder_items_atscope(items, set(), argkeys_cache, 0) - -def reorder_items_atscope(items, ignore, argkeys_cache, scopenum): - if scopenum >= scopenum_function or len(items) < 3: - return items - items_done = [] - while 1: - items_before, items_same, items_other, newignore = \ - slice_items(items, ignore, argkeys_cache[scopenum]) - items_before = reorder_items_atscope( - items_before, ignore, argkeys_cache,scopenum+1) - if items_same is None: - # nothing to reorder in this scope - assert items_other is None - return items_done + items_before - items_done.extend(items_before) - items = items_same + items_other - ignore = newignore - - -def slice_items(items, ignore, scoped_argkeys_cache): - # we pick the first item which uses a fixture instance in the - # requested scope and which we haven't seen yet. We slice the input - # items list into a list of items_nomatch, items_same and - # items_other - if scoped_argkeys_cache: # do we need to do work at all? - it = iter(items) - # first find a slicing key - for i, item in enumerate(it): - argkeys = scoped_argkeys_cache.get(item) - if argkeys is not None: - argkeys = argkeys.difference(ignore) - if argkeys: # found a slicing key - slicing_argkey = argkeys.pop() - items_before = items[:i] - items_same = [item] - items_other = [] - # now slice the remainder of the list - for item in it: - argkeys = scoped_argkeys_cache.get(item) - if argkeys and slicing_argkey in argkeys and \ - slicing_argkey not in ignore: - items_same.append(item) - else: - items_other.append(item) - newignore = ignore.copy() - newignore.add(slicing_argkey) - return (items_before, items_same, items_other, newignore) - return items, None, None, None - -def get_parametrized_fixture_keys(item, scopenum): - """ return list of keys for all parametrized arguments which match - the specified scope. """ - assert scopenum < scopenum_function # function - try: - cs = item.callspec - except AttributeError: - pass - else: - # cs.indictes.items() is random order of argnames but - # then again different functions (items) can change order of - # arguments so it doesn't matter much probably - for argname, param_index in cs.indices.items(): - if cs._arg2scopenum[argname] != scopenum: - continue - if scopenum == 0: # session - key = (argname, param_index) - elif scopenum == 1: # module - key = (argname, param_index, item.fspath) - elif scopenum == 2: # class - key = (argname, param_index, item.fspath, item.cls) - yield key - - -def xunitsetup(obj, name): - meth = getattr(obj, name, None) - if getfixturemarker(meth) is None: - return meth - -def getfixturemarker(obj): - """ return fixturemarker or None if it doesn't exist or raised - exceptions.""" - try: - return getattr(obj, "_pytestfixturefunction", None) - except KeyboardInterrupt: - raise - except Exception: - # some objects raise errors like request (from flask import request) - # we don't expect them to be fixture functions - return None - -scopename2class = { - 'class': Class, - 'module': Module, - 'function': pytest.Item, -} -def get_scope_node(node, scope): - cls = scopename2class.get(scope) - if cls is None: - if scope == "session": - return node.session - raise ValueError("unknown scope") - return node.getparent(cls) + fixtures.fillfixtures(self) diff --git a/_pytest/python_api.py b/_pytest/python_api.py new file mode 100644 index 000000000000000..81960295b381bd0 --- /dev/null +++ b/_pytest/python_api.py @@ -0,0 +1,619 @@ +import math +import sys + +import py +from six.moves import zip + +from _pytest.compat import isclass +from _pytest.outcomes import fail +import _pytest._code + + +def _cmp_raises_type_error(self, other): + """__cmp__ implementation which raises TypeError. Used + by Approx base classes to implement only == and != and raise a + TypeError for other comparisons. + + Needed in Python 2 only, Python 3 all it takes is not implementing the + other operators at all. + """ + __tracebackhide__ = True + raise TypeError('Comparison operators other than == and != not supported by approx objects') + + +# builtin pytest.approx helper + + +class ApproxBase(object): + """ + Provide shared utilities for making approximate comparisons between numbers + or sequences of numbers. + """ + + def __init__(self, expected, rel=None, abs=None, nan_ok=False): + self.expected = expected + self.abs = abs + self.rel = rel + self.nan_ok = nan_ok + + def __repr__(self): + raise NotImplementedError + + def __eq__(self, actual): + return all( + a == self._approx_scalar(x) + for a, x in self._yield_comparisons(actual)) + + __hash__ = None + + def __ne__(self, actual): + return not (actual == self) + + if sys.version_info[0] == 2: + __cmp__ = _cmp_raises_type_error + + def _approx_scalar(self, x): + return ApproxScalar(x, rel=self.rel, abs=self.abs, nan_ok=self.nan_ok) + + def _yield_comparisons(self, actual): + """ + Yield all the pairs of numbers to be compared. This is used to + implement the `__eq__` method. + """ + raise NotImplementedError + + +class ApproxNumpy(ApproxBase): + """ + Perform approximate comparisons for numpy arrays. + """ + + # Tell numpy to use our `__eq__` operator instead of its. + __array_priority__ = 100 + + def __repr__(self): + # It might be nice to rewrite this function to account for the + # shape of the array... + return "approx({0!r})".format(list( + self._approx_scalar(x) for x in self.expected)) + + if sys.version_info[0] == 2: + __cmp__ = _cmp_raises_type_error + + def __eq__(self, actual): + import numpy as np + + try: + actual = np.asarray(actual) + except: # noqa + raise TypeError("cannot compare '{0}' to numpy.ndarray".format(actual)) + + if actual.shape != self.expected.shape: + return False + + return ApproxBase.__eq__(self, actual) + + def _yield_comparisons(self, actual): + import numpy as np + + # We can be sure that `actual` is a numpy array, because it's + # casted in `__eq__` before being passed to `ApproxBase.__eq__`, + # which is the only method that calls this one. + for i in np.ndindex(self.expected.shape): + yield actual[i], self.expected[i] + + +class ApproxMapping(ApproxBase): + """ + Perform approximate comparisons for mappings where the values are numbers + (the keys can be anything). + """ + + def __repr__(self): + return "approx({0!r})".format(dict( + (k, self._approx_scalar(v)) + for k, v in self.expected.items())) + + def __eq__(self, actual): + if set(actual.keys()) != set(self.expected.keys()): + return False + + return ApproxBase.__eq__(self, actual) + + def _yield_comparisons(self, actual): + for k in self.expected.keys(): + yield actual[k], self.expected[k] + + +class ApproxSequence(ApproxBase): + """ + Perform approximate comparisons for sequences of numbers. + """ + + # Tell numpy to use our `__eq__` operator instead of its. + __array_priority__ = 100 + + def __repr__(self): + seq_type = type(self.expected) + if seq_type not in (tuple, list, set): + seq_type = list + return "approx({0!r})".format(seq_type( + self._approx_scalar(x) for x in self.expected)) + + def __eq__(self, actual): + if len(actual) != len(self.expected): + return False + return ApproxBase.__eq__(self, actual) + + def _yield_comparisons(self, actual): + return zip(actual, self.expected) + + +class ApproxScalar(ApproxBase): + """ + Perform approximate comparisons for single numbers only. + """ + + def __repr__(self): + """ + Return a string communicating both the expected value and the tolerance + for the comparison being made, e.g. '1.0 +- 1e-6'. Use the unicode + plus/minus symbol if this is python3 (it's too hard to get right for + python2). + """ + if isinstance(self.expected, complex): + return str(self.expected) + + # Infinities aren't compared using tolerances, so don't show a + # tolerance. + if math.isinf(self.expected): + return str(self.expected) + + # If a sensible tolerance can't be calculated, self.tolerance will + # raise a ValueError. In this case, display '???'. + try: + vetted_tolerance = '{:.1e}'.format(self.tolerance) + except ValueError: + vetted_tolerance = '???' + + if sys.version_info[0] == 2: + return '{0} +- {1}'.format(self.expected, vetted_tolerance) + else: + return u'{0} \u00b1 {1}'.format(self.expected, vetted_tolerance) + + def __eq__(self, actual): + """ + Return true if the given value is equal to the expected value within + the pre-specified tolerance. + """ + + # Short-circuit exact equality. + if actual == self.expected: + return True + + # Allow the user to control whether NaNs are considered equal to each + # other or not. The abs() calls are for compatibility with complex + # numbers. + if math.isnan(abs(self.expected)): + return self.nan_ok and math.isnan(abs(actual)) + + # Infinity shouldn't be approximately equal to anything but itself, but + # if there's a relative tolerance, it will be infinite and infinity + # will seem approximately equal to everything. The equal-to-itself + # case would have been short circuited above, so here we can just + # return false if the expected value is infinite. The abs() call is + # for compatibility with complex numbers. + if math.isinf(abs(self.expected)): + return False + + # Return true if the two numbers are within the tolerance. + return abs(self.expected - actual) <= self.tolerance + + __hash__ = None + + @property + def tolerance(self): + """ + Return the tolerance for the comparison. This could be either an + absolute tolerance or a relative tolerance, depending on what the user + specified or which would be larger. + """ + def set_default(x, default): + return x if x is not None else default + + # Figure out what the absolute tolerance should be. ``self.abs`` is + # either None or a value specified by the user. + absolute_tolerance = set_default(self.abs, 1e-12) + + if absolute_tolerance < 0: + raise ValueError("absolute tolerance can't be negative: {}".format(absolute_tolerance)) + if math.isnan(absolute_tolerance): + raise ValueError("absolute tolerance can't be NaN.") + + # If the user specified an absolute tolerance but not a relative one, + # just return the absolute tolerance. + if self.rel is None: + if self.abs is not None: + return absolute_tolerance + + # Figure out what the relative tolerance should be. ``self.rel`` is + # either None or a value specified by the user. This is done after + # we've made sure the user didn't ask for an absolute tolerance only, + # because we don't want to raise errors about the relative tolerance if + # we aren't even going to use it. + relative_tolerance = set_default(self.rel, 1e-6) * abs(self.expected) + + if relative_tolerance < 0: + raise ValueError("relative tolerance can't be negative: {}".format(absolute_tolerance)) + if math.isnan(relative_tolerance): + raise ValueError("relative tolerance can't be NaN.") + + # Return the larger of the relative and absolute tolerances. + return max(relative_tolerance, absolute_tolerance) + + +def approx(expected, rel=None, abs=None, nan_ok=False): + """ + Assert that two numbers (or two sets of numbers) are equal to each other + within some tolerance. + + Due to the `intricacies of floating-point arithmetic`__, numbers that we + would intuitively expect to be equal are not always so:: + + >>> 0.1 + 0.2 == 0.3 + False + + __ https://docs.python.org/3/tutorial/floatingpoint.html + + This problem is commonly encountered when writing tests, e.g. when making + sure that floating-point values are what you expect them to be. One way to + deal with this problem is to assert that two floating-point numbers are + equal to within some appropriate tolerance:: + + >>> abs((0.1 + 0.2) - 0.3) < 1e-6 + True + + However, comparisons like this are tedious to write and difficult to + understand. Furthermore, absolute comparisons like the one above are + usually discouraged because there's no tolerance that works well for all + situations. ``1e-6`` is good for numbers around ``1``, but too small for + very big numbers and too big for very small ones. It's better to express + the tolerance as a fraction of the expected value, but relative comparisons + like that are even more difficult to write correctly and concisely. + + The ``approx`` class performs floating-point comparisons using a syntax + that's as intuitive as possible:: + + >>> from pytest import approx + >>> 0.1 + 0.2 == approx(0.3) + True + + The same syntax also works for sequences of numbers:: + + >>> (0.1 + 0.2, 0.2 + 0.4) == approx((0.3, 0.6)) + True + + Dictionary *values*:: + + >>> {'a': 0.1 + 0.2, 'b': 0.2 + 0.4} == approx({'a': 0.3, 'b': 0.6}) + True + + And ``numpy`` arrays:: + + >>> import numpy as np # doctest: +SKIP + >>> np.array([0.1, 0.2]) + np.array([0.2, 0.4]) == approx(np.array([0.3, 0.6])) # doctest: +SKIP + True + + By default, ``approx`` considers numbers within a relative tolerance of + ``1e-6`` (i.e. one part in a million) of its expected value to be equal. + This treatment would lead to surprising results if the expected value was + ``0.0``, because nothing but ``0.0`` itself is relatively close to ``0.0``. + To handle this case less surprisingly, ``approx`` also considers numbers + within an absolute tolerance of ``1e-12`` of its expected value to be + equal. Infinity and NaN are special cases. Infinity is only considered + equal to itself, regardless of the relative tolerance. NaN is not + considered equal to anything by default, but you can make it be equal to + itself by setting the ``nan_ok`` argument to True. (This is meant to + facilitate comparing arrays that use NaN to mean "no data".) + + Both the relative and absolute tolerances can be changed by passing + arguments to the ``approx`` constructor:: + + >>> 1.0001 == approx(1) + False + >>> 1.0001 == approx(1, rel=1e-3) + True + >>> 1.0001 == approx(1, abs=1e-3) + True + + If you specify ``abs`` but not ``rel``, the comparison will not consider + the relative tolerance at all. In other words, two numbers that are within + the default relative tolerance of ``1e-6`` will still be considered unequal + if they exceed the specified absolute tolerance. If you specify both + ``abs`` and ``rel``, the numbers will be considered equal if either + tolerance is met:: + + >>> 1 + 1e-8 == approx(1) + True + >>> 1 + 1e-8 == approx(1, abs=1e-12) + False + >>> 1 + 1e-8 == approx(1, rel=1e-6, abs=1e-12) + True + + If you're thinking about using ``approx``, then you might want to know how + it compares to other good ways of comparing floating-point numbers. All of + these algorithms are based on relative and absolute tolerances and should + agree for the most part, but they do have meaningful differences: + + - ``math.isclose(a, b, rel_tol=1e-9, abs_tol=0.0)``: True if the relative + tolerance is met w.r.t. either ``a`` or ``b`` or if the absolute + tolerance is met. Because the relative tolerance is calculated w.r.t. + both ``a`` and ``b``, this test is symmetric (i.e. neither ``a`` nor + ``b`` is a "reference value"). You have to specify an absolute tolerance + if you want to compare to ``0.0`` because there is no tolerance by + default. Only available in python>=3.5. `More information...`__ + + __ https://docs.python.org/3/library/math.html#math.isclose + + - ``numpy.isclose(a, b, rtol=1e-5, atol=1e-8)``: True if the difference + between ``a`` and ``b`` is less that the sum of the relative tolerance + w.r.t. ``b`` and the absolute tolerance. Because the relative tolerance + is only calculated w.r.t. ``b``, this test is asymmetric and you can + think of ``b`` as the reference value. Support for comparing sequences + is provided by ``numpy.allclose``. `More information...`__ + + __ http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.isclose.html + + - ``unittest.TestCase.assertAlmostEqual(a, b)``: True if ``a`` and ``b`` + are within an absolute tolerance of ``1e-7``. No relative tolerance is + considered and the absolute tolerance cannot be changed, so this function + is not appropriate for very large or very small numbers. Also, it's only + available in subclasses of ``unittest.TestCase`` and it's ugly because it + doesn't follow PEP8. `More information...`__ + + __ https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertAlmostEqual + + - ``a == pytest.approx(b, rel=1e-6, abs=1e-12)``: True if the relative + tolerance is met w.r.t. ``b`` or if the absolute tolerance is met. + Because the relative tolerance is only calculated w.r.t. ``b``, this test + is asymmetric and you can think of ``b`` as the reference value. In the + special case that you explicitly specify an absolute tolerance but not a + relative tolerance, only the absolute tolerance is considered. + + .. warning:: + + .. versionchanged:: 3.2 + + In order to avoid inconsistent behavior, ``TypeError`` is + raised for ``>``, ``>=``, ``<`` and ``<=`` comparisons. + The example below illustrates the problem:: + + assert approx(0.1) > 0.1 + 1e-10 # calls approx(0.1).__gt__(0.1 + 1e-10) + assert 0.1 + 1e-10 > approx(0.1) # calls approx(0.1).__lt__(0.1 + 1e-10) + + In the second example one expects ``approx(0.1).__le__(0.1 + 1e-10)`` + to be called. But instead, ``approx(0.1).__lt__(0.1 + 1e-10)`` is used to + comparison. This is because the call hierarchy of rich comparisons + follows a fixed behavior. `More information...`__ + + __ https://docs.python.org/3/reference/datamodel.html#object.__ge__ + """ + + from collections import Mapping, Sequence + from _pytest.compat import STRING_TYPES as String + + # Delegate the comparison to a class that knows how to deal with the type + # of the expected value (e.g. int, float, list, dict, numpy.array, etc). + # + # This architecture is really driven by the need to support numpy arrays. + # The only way to override `==` for arrays without requiring that approx be + # the left operand is to inherit the approx object from `numpy.ndarray`. + # But that can't be a general solution, because it requires (1) numpy to be + # installed and (2) the expected value to be a numpy array. So the general + # solution is to delegate each type of expected value to a different class. + # + # This has the advantage that it made it easy to support mapping types + # (i.e. dict). The old code accepted mapping types, but would only compare + # their keys, which is probably not what most people would expect. + + if _is_numpy_array(expected): + cls = ApproxNumpy + elif isinstance(expected, Mapping): + cls = ApproxMapping + elif isinstance(expected, Sequence) and not isinstance(expected, String): + cls = ApproxSequence + else: + cls = ApproxScalar + + return cls(expected, rel, abs, nan_ok) + + +def _is_numpy_array(obj): + """ + Return true if the given object is a numpy array. Make a special effort to + avoid importing numpy unless it's really necessary. + """ + import inspect + + for cls in inspect.getmro(type(obj)): + if cls.__module__ == 'numpy': + try: + import numpy as np + return isinstance(obj, np.ndarray) + except ImportError: + pass + + return False + + +# builtin pytest.raises helper + +def raises(expected_exception, *args, **kwargs): + """ + Assert that a code block/function call raises ``expected_exception`` + and raise a failure exception otherwise. + + This helper produces a ``ExceptionInfo()`` object (see below). + + You may use this function as a context manager:: + + >>> with raises(ZeroDivisionError): + ... 1/0 + + .. versionchanged:: 2.10 + + In the context manager form you may use the keyword argument + ``message`` to specify a custom failure message:: + + >>> with raises(ZeroDivisionError, message="Expecting ZeroDivisionError"): + ... pass + Traceback (most recent call last): + ... + Failed: Expecting ZeroDivisionError + + .. note:: + + When using ``pytest.raises`` as a context manager, it's worthwhile to + note that normal context manager rules apply and that the exception + raised *must* be the final line in the scope of the context manager. + Lines of code after that, within the scope of the context manager will + not be executed. For example:: + + >>> value = 15 + >>> with raises(ValueError) as exc_info: + ... if value > 10: + ... raise ValueError("value must be <= 10") + ... assert exc_info.type == ValueError # this will not execute + + Instead, the following approach must be taken (note the difference in + scope):: + + >>> with raises(ValueError) as exc_info: + ... if value > 10: + ... raise ValueError("value must be <= 10") + ... + >>> assert exc_info.type == ValueError + + + Since version ``3.1`` you can use the keyword argument ``match`` to assert that the + exception matches a text or regex:: + + >>> with raises(ValueError, match='must be 0 or None'): + ... raise ValueError("value must be 0 or None") + + >>> with raises(ValueError, match=r'must be \d+$'): + ... raise ValueError("value must be 42") + + **Legacy forms** + + The forms below are fully supported but are discouraged for new code because the + context manager form is regarded as more readable and less error-prone. + + It is possible to specify a callable by passing a to-be-called lambda:: + + >>> raises(ZeroDivisionError, lambda: 1/0) + + + or you can specify an arbitrary callable with arguments:: + + >>> def f(x): return 1/x + ... + >>> raises(ZeroDivisionError, f, 0) + + >>> raises(ZeroDivisionError, f, x=0) + + + It is also possible to pass a string to be evaluated at runtime:: + + >>> raises(ZeroDivisionError, "f(0)") + + + The string will be evaluated using the same ``locals()`` and ``globals()`` + at the moment of the ``raises`` call. + + .. autoclass:: _pytest._code.ExceptionInfo + :members: + + .. note:: + Similar to caught exception objects in Python, explicitly clearing + local references to returned ``ExceptionInfo`` objects can + help the Python interpreter speed up its garbage collection. + + Clearing those references breaks a reference cycle + (``ExceptionInfo`` --> caught exception --> frame stack raising + the exception --> current frame stack --> local variables --> + ``ExceptionInfo``) which makes Python keep all objects referenced + from that cycle (including all local variables in the current + frame) alive until the next cyclic garbage collection run. See the + official Python ``try`` statement documentation for more detailed + information. + + """ + __tracebackhide__ = True + msg = ("exceptions must be old-style classes or" + " derived from BaseException, not %s") + if isinstance(expected_exception, tuple): + for exc in expected_exception: + if not isclass(exc): + raise TypeError(msg % type(exc)) + elif not isclass(expected_exception): + raise TypeError(msg % type(expected_exception)) + + message = "DID NOT RAISE {0}".format(expected_exception) + match_expr = None + + if not args: + if "message" in kwargs: + message = kwargs.pop("message") + if "match" in kwargs: + match_expr = kwargs.pop("match") + message += " matching '{0}'".format(match_expr) + return RaisesContext(expected_exception, message, match_expr) + elif isinstance(args[0], str): + code, = args + assert isinstance(code, str) + frame = sys._getframe(1) + loc = frame.f_locals.copy() + loc.update(kwargs) + # print "raises frame scope: %r" % frame.f_locals + try: + code = _pytest._code.Source(code).compile() + py.builtin.exec_(code, frame.f_globals, loc) + # XXX didn'T mean f_globals == f_locals something special? + # this is destroyed here ... + except expected_exception: + return _pytest._code.ExceptionInfo() + else: + func = args[0] + try: + func(*args[1:], **kwargs) + except expected_exception: + return _pytest._code.ExceptionInfo() + fail(message) + + +raises.Exception = fail.Exception + + +class RaisesContext(object): + def __init__(self, expected_exception, message, match_expr): + self.expected_exception = expected_exception + self.message = message + self.match_expr = match_expr + self.excinfo = None + + def __enter__(self): + self.excinfo = object.__new__(_pytest._code.ExceptionInfo) + return self.excinfo + + def __exit__(self, *tp): + __tracebackhide__ = True + if tp[0] is None: + fail(self.message) + self.excinfo.__init__(tp) + suppress_exception = issubclass(self.excinfo.type, self.expected_exception) + if sys.version_info[0] == 2 and suppress_exception: + sys.exc_clear() + if self.match_expr: + self.excinfo.match(self.match_expr) + return suppress_exception diff --git a/_pytest/recwarn.py b/_pytest/recwarn.py index a89474c036aa5b3..4fceb10a7f34043 100644 --- a/_pytest/recwarn.py +++ b/_pytest/recwarn.py @@ -1,4 +1,5 @@ """ recording warnings during test function execution. """ +from __future__ import absolute_import, division, print_function import inspect @@ -6,11 +7,15 @@ import py import sys import warnings -import pytest +import re -@pytest.yield_fixture -def recwarn(request): +from _pytest.fixtures import yield_fixture +from _pytest.outcomes import fail + + +@yield_fixture +def recwarn(): """Return a WarningsRecorder instance that provides these methods: * ``pop(category=None)``: return last warning matching the category. @@ -25,54 +30,59 @@ def recwarn(request): yield wrec -def pytest_namespace(): - return {'deprecated_call': deprecated_call, - 'warns': warns} - - def deprecated_call(func=None, *args, **kwargs): - """ assert that calling ``func(*args, **kwargs)`` triggers a - ``DeprecationWarning`` or ``PendingDeprecationWarning``. + """context manager that can be used to ensure a block of code triggers a + ``DeprecationWarning`` or ``PendingDeprecationWarning``:: - This function can be used as a context manager:: + >>> import warnings + >>> def api_call_v2(): + ... warnings.warn('use v3 of this api', DeprecationWarning) + ... return 200 >>> with deprecated_call(): - ... myobject.deprecated_method() + ... assert api_call_v2() == 200 - Note: we cannot use WarningsRecorder here because it is still subject - to the mechanism that prevents warnings of the same type from being - triggered twice for the same module. See #1190. + ``deprecated_call`` can also be used by passing a function and ``*args`` and ``*kwargs``, + in which case it will ensure calling ``func(*args, **kwargs)`` produces one of the warnings + types above. """ if not func: - return WarningsChecker(expected_warning=DeprecationWarning) + return _DeprecatedCallContext() + else: + __tracebackhide__ = True + with _DeprecatedCallContext(): + return func(*args, **kwargs) - categories = [] - def warn_explicit(message, category, *args, **kwargs): - categories.append(category) - old_warn_explicit(message, category, *args, **kwargs) +class _DeprecatedCallContext(object): + """Implements the logic to capture deprecation warnings as a context manager.""" - def warn(message, category=None, *args, **kwargs): + def __enter__(self): + self._captured_categories = [] + self._old_warn = warnings.warn + self._old_warn_explicit = warnings.warn_explicit + warnings.warn_explicit = self._warn_explicit + warnings.warn = self._warn + + def _warn_explicit(self, message, category, *args, **kwargs): + self._captured_categories.append(category) + + def _warn(self, message, category=None, *args, **kwargs): if isinstance(message, Warning): - categories.append(message.__class__) + self._captured_categories.append(message.__class__) else: - categories.append(category) - old_warn(message, category, *args, **kwargs) - - old_warn = warnings.warn - old_warn_explicit = warnings.warn_explicit - warnings.warn_explicit = warn_explicit - warnings.warn = warn - try: - ret = func(*args, **kwargs) - finally: - warnings.warn_explicit = old_warn_explicit - warnings.warn = old_warn - deprecation_categories = (DeprecationWarning, PendingDeprecationWarning) - if not any(issubclass(c, deprecation_categories) for c in categories): - __tracebackhide__ = True - raise AssertionError("%r did not produce DeprecationWarning" % (func,)) - return ret + self._captured_categories.append(category) + + def __exit__(self, exc_type, exc_val, exc_tb): + warnings.warn_explicit = self._old_warn_explicit + warnings.warn = self._old_warn + + if exc_type is None: + deprecation_categories = (DeprecationWarning, PendingDeprecationWarning) + if not any(issubclass(c, deprecation_categories) for c in self._captured_categories): + __tracebackhide__ = True + msg = "Did not produce DeprecationWarning or PendingDeprecationWarning" + raise AssertionError(msg) def warns(expected_warning, *args, **kwargs): @@ -90,10 +100,28 @@ def warns(expected_warning, *args, **kwargs): >>> with warns(RuntimeWarning): ... warnings.warn("my warning", RuntimeWarning) + + In the context manager form you may use the keyword argument ``match`` to assert + that the exception matches a text or regex:: + + >>> with warns(UserWarning, match='must be 0 or None'): + ... warnings.warn("value must be 0 or None", UserWarning) + + >>> with warns(UserWarning, match=r'must be \d+$'): + ... warnings.warn("value must be 42", UserWarning) + + >>> with warns(UserWarning, match=r'must be \d+$'): + ... warnings.warn("this is not here", UserWarning) + Traceback (most recent call last): + ... + Failed: DID NOT WARN. No warnings of type ...UserWarning... was emitted... + """ - wcheck = WarningsChecker(expected_warning) + match_expr = None if not args: - return wcheck + if "match" in kwargs: + match_expr = kwargs.pop("match") + return WarningsChecker(expected_warning, match_expr=match_expr) elif isinstance(args[0], str): code, = args assert isinstance(code, str) @@ -101,33 +129,23 @@ def warns(expected_warning, *args, **kwargs): loc = frame.f_locals.copy() loc.update(kwargs) - with wcheck: + with WarningsChecker(expected_warning, match_expr=match_expr): code = _pytest._code.Source(code).compile() py.builtin.exec_(code, frame.f_globals, loc) else: func = args[0] - with wcheck: + with WarningsChecker(expected_warning, match_expr=match_expr): return func(*args[1:], **kwargs) -class RecordedWarning(object): - def __init__(self, message, category, filename, lineno, file, line): - self.message = message - self.category = category - self.filename = filename - self.lineno = lineno - self.file = file - self.line = line - - -class WarningsRecorder(object): +class WarningsRecorder(warnings.catch_warnings): """A context manager to record raised warnings. Adapted from `warnings.catch_warnings`. """ - def __init__(self, module=None): - self._module = sys.modules['warnings'] if module is None else module + def __init__(self): + super(WarningsRecorder, self).__init__(record=True) self._entered = False self._list = [] @@ -164,38 +182,20 @@ def __enter__(self): if self._entered: __tracebackhide__ = True raise RuntimeError("Cannot enter %r twice" % self) - self._entered = True - self._filters = self._module.filters - self._module.filters = self._filters[:] - self._showwarning = self._module.showwarning - - def showwarning(message, category, filename, lineno, - file=None, line=None): - self._list.append(RecordedWarning( - message, category, filename, lineno, file, line)) - - # still perform old showwarning functionality - self._showwarning( - message, category, filename, lineno, file=file, line=line) - - self._module.showwarning = showwarning - - # allow the same warning to be raised more than once - - self._module.simplefilter('always') + self._list = super(WarningsRecorder, self).__enter__() + warnings.simplefilter('always') return self def __exit__(self, *exc_info): if not self._entered: __tracebackhide__ = True raise RuntimeError("Cannot exit %r without entering first" % self) - self._module.filters = self._filters - self._module.showwarning = self._showwarning + super(WarningsRecorder, self).__exit__(*exc_info) class WarningsChecker(WarningsRecorder): - def __init__(self, expected_warning=None, module=None): - super(WarningsChecker, self).__init__(module=module) + def __init__(self, expected_warning=None, match_expr=None): + super(WarningsChecker, self).__init__() msg = ("exceptions must be old-style classes or " "derived from Warning, not %s") @@ -209,6 +209,7 @@ def __init__(self, expected_warning=None, module=None): raise TypeError(msg % type(expected_warning)) self.expected_warning = expected_warning + self.match_expr = match_expr def __exit__(self, *exc_info): super(WarningsChecker, self).__exit__(*exc_info) @@ -216,6 +217,20 @@ def __exit__(self, *exc_info): # only check if we're not currently handling an exception if all(a is None for a in exc_info): if self.expected_warning is not None: - if not any(r.category in self.expected_warning for r in self): + if not any(issubclass(r.category, self.expected_warning) + for r in self): __tracebackhide__ = True - pytest.fail("DID NOT WARN") + fail("DID NOT WARN. No warnings of type {0} was emitted. " + "The list of emitted warnings is: {1}.".format( + self.expected_warning, + [each.message for each in self])) + elif self.match_expr is not None: + for r in self: + if issubclass(r.category, self.expected_warning): + if re.compile(self.match_expr).search(str(r.message)): + break + else: + fail("DID NOT WARN. No warnings of type {0} matching" + " ('{1}') was emitted. The list of emitted warnings" + " is: {2}.".format(self.expected_warning, self.match_expr, + [each.message for each in self])) diff --git a/_pytest/resultlog.py b/_pytest/resultlog.py index 3670f0214c9df3c..9f9c2d1f65360f3 100644 --- a/_pytest/resultlog.py +++ b/_pytest/resultlog.py @@ -1,15 +1,18 @@ """ log machine-parseable test session result information in a plain text file. """ +from __future__ import absolute_import, division, print_function import py import os + def pytest_addoption(parser): group = parser.getgroup("terminal reporting", "resultlog plugin options") group.addoption('--resultlog', '--result-log', action="store", - metavar="path", default=None, - help="path for machine-readable result log.") + metavar="path", default=None, + help="DEPRECATED path for machine-readable result log.") + def pytest_configure(config): resultlog = config.option.resultlog @@ -18,10 +21,14 @@ def pytest_configure(config): dirname = os.path.dirname(os.path.abspath(resultlog)) if not os.path.isdir(dirname): os.makedirs(dirname) - logfile = open(resultlog, 'w', 1) # line buffered + logfile = open(resultlog, 'w', 1) # line buffered config._resultlog = ResultLog(config, logfile) config.pluginmanager.register(config._resultlog) + from _pytest.deprecated import RESULT_LOG + config.warn('C1', RESULT_LOG) + + def pytest_unconfigure(config): resultlog = getattr(config, '_resultlog', None) if resultlog: @@ -29,6 +36,7 @@ def pytest_unconfigure(config): del config._resultlog config.pluginmanager.unregister(resultlog) + def generic_path(item): chain = item.listchain() gpath = [chain[0].name] @@ -52,15 +60,16 @@ def generic_path(item): fspath = newfspath return ''.join(gpath) + class ResultLog(object): def __init__(self, config, logfile): self.config = config - self.logfile = logfile # preferably line buffered + self.logfile = logfile # preferably line buffered def write_log_entry(self, testpath, lettercode, longrepr): - py.builtin.print_("%s %s" % (lettercode, testpath), file=self.logfile) + print("%s %s" % (lettercode, testpath), file=self.logfile) for line in longrepr.splitlines(): - py.builtin.print_(" %s" % line, file=self.logfile) + print(" %s" % line, file=self.logfile) def log_outcome(self, report, lettercode, longrepr): testpath = getattr(report, 'nodeid', None) diff --git a/_pytest/runner.py b/_pytest/runner.py index cde94c8c89e18ee..e07ed2a248b3223 100644 --- a/_pytest/runner.py +++ b/_pytest/runner.py @@ -1,29 +1,25 @@ """ basic collect and runtest protocol implementations """ +from __future__ import absolute_import, division, print_function + import bdb +import os import sys from time import time import py -import pytest from _pytest._code.code import TerminalRepr, ExceptionInfo - - -def pytest_namespace(): - return { - 'fail' : fail, - 'skip' : skip, - 'importorskip' : importorskip, - 'exit' : exit, - } +from _pytest.outcomes import skip, Skipped, TEST_OUTCOME # # pytest plugin hooks + def pytest_addoption(parser): group = parser.getgroup("terminal reporting", "reporting", after="general") group.addoption('--durations', - action="store", type=int, default=None, metavar="N", - help="show N slowest setup/test durations (N=0 for all)."), + action="store", type=int, default=None, metavar="N", + help="show N slowest setup/test durations (N=0 for all)."), + def pytest_terminal_summary(terminalreporter): durations = terminalreporter.config.option.durations @@ -48,16 +44,16 @@ def pytest_terminal_summary(terminalreporter): for rep in dlist: nodeid = rep.nodeid.replace("::()::", "::") tr.write_line("%02.2fs %-8s %s" % - (rep.duration, rep.when, nodeid)) + (rep.duration, rep.when, nodeid)) + def pytest_sessionstart(session): session._setupstate = SetupState() + + def pytest_sessionfinish(session): session._setupstate.teardown_all() -class NodeInfo: - def __init__(self, location): - self.location = location def pytest_runtest_protocol(item, nextitem): item.ihook.pytest_runtest_logstart( @@ -66,6 +62,7 @@ def pytest_runtest_protocol(item, nextitem): runtestprotocol(item, nextitem=nextitem) return True + def runtestprotocol(item, log=True, nextitem=None): hasrequest = hasattr(item, "_request") if hasrequest and not item._request: @@ -73,9 +70,12 @@ def runtestprotocol(item, log=True, nextitem=None): rep = call_and_report(item, "setup", log) reports = [rep] if rep.passed: - reports.append(call_and_report(item, "call", log)) + if item.config.option.setupshow: + show_test_item(item) + if not item.config.option.setuponly: + reports.append(call_and_report(item, "call", log)) reports.append(call_and_report(item, "teardown", log, - nextitem=nextitem)) + nextitem=nextitem)) # after all teardown hooks have been called # want funcargs and request info to go away if hasrequest: @@ -83,10 +83,25 @@ def runtestprotocol(item, log=True, nextitem=None): item.funcargs = None return reports + +def show_test_item(item): + """Show test function, parameters and the fixtures of the test item.""" + tw = item.config.get_terminal_writer() + tw.line() + tw.write(' ' * 8) + tw.write(item._nodeid) + used_fixtures = sorted(item._fixtureinfo.name2fixturedefs.keys()) + if used_fixtures: + tw.write(' (fixtures used: {0})'.format(', '.join(used_fixtures))) + + def pytest_runtest_setup(item): + _update_current_test_var(item, 'setup') item.session._setupstate.prepare(item) + def pytest_runtest_call(item): + _update_current_test_var(item, 'call') try: item.runtest() except Exception: @@ -99,8 +114,28 @@ def pytest_runtest_call(item): del tb # Get rid of it in this namespace raise + def pytest_runtest_teardown(item, nextitem): + _update_current_test_var(item, 'teardown') item.session._setupstate.teardown_exact(item, nextitem) + _update_current_test_var(item, None) + + +def _update_current_test_var(item, when): + """ + Update PYTEST_CURRENT_TEST to reflect the current item and stage. + + If ``when`` is None, delete PYTEST_CURRENT_TEST from the environment. + """ + var_name = 'PYTEST_CURRENT_TEST' + if when: + value = '{0} ({1})'.format(item.nodeid, when) + # don't allow null bytes on environment variables (see #2644, #2957) + value = value.replace('\x00', '(null)') + os.environ[var_name] = value + else: + os.environ.pop(var_name) + def pytest_report_teststatus(report): if report.when in ("setup", "teardown"): @@ -126,21 +161,25 @@ def call_and_report(item, when, log=True, **kwds): hook.pytest_exception_interact(node=item, call=call, report=report) return report + def check_interactive_exception(call, report): return call.excinfo and not ( - hasattr(report, "wasxfail") or - call.excinfo.errisinstance(skip.Exception) or - call.excinfo.errisinstance(bdb.BdbQuit)) + hasattr(report, "wasxfail") or + call.excinfo.errisinstance(skip.Exception) or + call.excinfo.errisinstance(bdb.BdbQuit)) + def call_runtest_hook(item, when, **kwds): hookname = "pytest_runtest_" + when ihook = getattr(item.ihook, hookname) return CallInfo(lambda: ihook(item=item, **kwds), when=when) + class CallInfo: """ Result/Exception info a function invocation. """ #: None or ExceptionInfo object. excinfo = None + def __init__(self, func, when): #: context of invocation: one of "setup", "call", #: "teardown", "memocollect" @@ -151,7 +190,7 @@ def __init__(self, func, when): except KeyboardInterrupt: self.stop = time() raise - except: + except: # noqa self.excinfo = ExceptionInfo() self.stop = time() @@ -162,6 +201,7 @@ def __repr__(self): status = "result: %r" % (self.result,) return "" % (self.when, status) + def getslaveinfoline(node): try: return node._slaveinfocache @@ -172,6 +212,7 @@ def getslaveinfoline(node): d['id'], d['sysplatform'], ver, d['executable']) return s + class BaseReport(object): def __init__(self, **kw): @@ -198,6 +239,36 @@ def get_sections(self, prefix): if name.startswith(prefix): yield prefix, content + @property + def longreprtext(self): + """ + Read-only property that returns the full string representation + of ``longrepr``. + + .. versionadded:: 3.0 + """ + tw = py.io.TerminalWriter(stringio=True) + tw.hasmarkup = False + self.toterminal(tw) + exc = tw.stringio.getvalue() + return exc.strip() + + @property + def capstdout(self): + """Return captured text from stdout, if capturing is enabled + + .. versionadded:: 3.0 + """ + return ''.join(content for (prefix, content) in self.get_sections('Captured stdout')) + + @property + def capstderr(self): + """Return captured text from stderr, if capturing is enabled + + .. versionadded:: 3.0 + """ + return ''.join(content for (prefix, content) in self.get_sections('Captured stderr')) + passed = property(lambda x: x.outcome == "passed") failed = property(lambda x: x.outcome == "failed") skipped = property(lambda x: x.outcome == "skipped") @@ -206,10 +277,11 @@ def get_sections(self, prefix): def fspath(self): return self.nodeid.split("::")[0] + def pytest_runtest_makereport(item, call): when = call.when - duration = call.stop-call.start - keywords = dict([(x,1) for x in item.keywords]) + duration = call.stop - call.start + keywords = dict([(x, 1) for x in item.keywords]) excinfo = call.excinfo sections = [] if not call.excinfo: @@ -219,7 +291,7 @@ def pytest_runtest_makereport(item, call): if not isinstance(excinfo, ExceptionInfo): outcome = "failed" longrepr = excinfo - elif excinfo.errisinstance(pytest.skip.Exception): + elif excinfo.errisinstance(skip.Exception): outcome = "skipped" r = excinfo._getreprcrash() longrepr = (str(r.path), r.lineno, r.message) @@ -227,19 +299,21 @@ def pytest_runtest_makereport(item, call): outcome = "failed" if call.when == "call": longrepr = item.repr_failure(excinfo) - else: # exception in setup or teardown + else: # exception in setup or teardown longrepr = item._repr_failure_py(excinfo, - style=item.config.option.tbstyle) + style=item.config.option.tbstyle) for rwhen, key, content in item._report_sections: - sections.append(("Captured %s %s" %(key, rwhen), content)) + sections.append(("Captured %s %s" % (key, rwhen), content)) return TestReport(item.nodeid, item.location, keywords, outcome, longrepr, when, sections, duration) + class TestReport(BaseReport): """ Basic test report object (also used for setup and teardown calls if they fail). """ + def __init__(self, nodeid, location, keywords, outcome, longrepr, when, sections=(), duration=0, **extra): #: normalized collection node id @@ -263,8 +337,10 @@ def __init__(self, nodeid, location, keywords, outcome, #: one of 'setup', 'call', 'teardown' to indicate runtest phase. self.when = when - #: list of (secname, data) extra information which needs to - #: marshallable + #: list of pairs ``(str, str)`` of extra information which needs to + #: marshallable. Used by pytest to add captured text + #: from ``stdout`` and ``stderr``, but may be used by other plugins + #: to add arbitrary information to reports. self.sections = list(sections) #: time it took to run just the test @@ -276,16 +352,21 @@ def __repr__(self): return "" % ( self.nodeid, self.when, self.outcome) + class TeardownErrorReport(BaseReport): outcome = "failed" when = "teardown" + def __init__(self, longrepr, **extra): self.longrepr = longrepr self.sections = [] self.__dict__.update(extra) + def pytest_make_collect_report(collector): - call = CallInfo(collector._memocollect, "memocollect") + call = CallInfo( + lambda: list(collector.collect()), + 'collect') longrepr = None if not call.excinfo: outcome = "passed" @@ -303,7 +384,7 @@ def pytest_make_collect_report(collector): errorinfo = CollectErrorRepr(errorinfo) longrepr = errorinfo rep = CollectReport(collector.nodeid, outcome, longrepr, - getattr(call, 'result', None)) + getattr(call, 'result', None)) rep.call = call # see collect_one_node return rep @@ -324,16 +405,20 @@ def location(self): def __repr__(self): return "" % ( - self.nodeid, len(self.result), self.outcome) + self.nodeid, len(self.result), self.outcome) + class CollectErrorRepr(TerminalRepr): def __init__(self, msg): self.longrepr = msg + def toterminal(self, out): out.line(self.longrepr, red=True) + class SetupState(object): """ shared state for setting up/tearing down test items or collectors. """ + def __init__(self): self.stack = [] self._finalizers = {} @@ -344,8 +429,8 @@ def addfinalizer(self, finalizer, colitem): is called at the end of teardown_all(). """ assert colitem and not isinstance(colitem, tuple) - assert py.builtin.callable(finalizer) - #assert colitem in self.stack # some unit tests don't setup stack :/ + assert callable(finalizer) + # assert colitem in self.stack # some unit tests don't setup stack :/ self._finalizers.setdefault(colitem, []).append(finalizer) def _pop_and_teardown(self): @@ -359,7 +444,7 @@ def _callfinalizers(self, colitem): fin = finalizers.pop() try: fin() - except Exception: + except TEST_OUTCOME: # XXX Only first exception will be seen by user, # ideally all should be reported. if exc is None: @@ -373,7 +458,7 @@ def _teardown_with_finalization(self, colitem): colitem.teardown() for colitem in self._finalizers: assert colitem is None or colitem in self.stack \ - or isinstance(colitem, tuple) + or isinstance(colitem, tuple) def teardown_all(self): while self.stack: @@ -406,10 +491,11 @@ def prepare(self, colitem): self.stack.append(col) try: col.setup() - except Exception: + except TEST_OUTCOME: col._prepare_exc = sys.exc_info() raise + def collect_one_node(collector): ihook = collector.ihook ihook.pytest_collectstart(collector=collector) @@ -418,98 +504,3 @@ def collect_one_node(collector): if call and check_interactive_exception(call, rep): ihook.pytest_exception_interact(node=collector, call=call, report=rep) return rep - - -# ============================================================= -# Test OutcomeExceptions and helpers for creating them. - - -class OutcomeException(Exception): - """ OutcomeException and its subclass instances indicate and - contain info about test and collection outcomes. - """ - def __init__(self, msg=None, pytrace=True): - Exception.__init__(self, msg) - self.msg = msg - self.pytrace = pytrace - - def __repr__(self): - if self.msg: - val = self.msg - if isinstance(val, bytes): - val = py._builtin._totext(val, errors='replace') - return val - return "<%s instance>" %(self.__class__.__name__,) - __str__ = __repr__ - -class Skipped(OutcomeException): - # XXX hackish: on 3k we fake to live in the builtins - # in order to have Skipped exception printing shorter/nicer - __module__ = 'builtins' - -class Failed(OutcomeException): - """ raised from an explicit call to pytest.fail() """ - __module__ = 'builtins' - -class Exit(KeyboardInterrupt): - """ raised for immediate program exits (no tracebacks/summaries)""" - def __init__(self, msg="unknown reason"): - self.msg = msg - KeyboardInterrupt.__init__(self, msg) - -# exposed helper methods - -def exit(msg): - """ exit testing process as if KeyboardInterrupt was triggered. """ - __tracebackhide__ = True - raise Exit(msg) - -exit.Exception = Exit - -def skip(msg=""): - """ skip an executing test with the given message. Note: it's usually - better to use the pytest.mark.skipif marker to declare a test to be - skipped under certain conditions like mismatching platforms or - dependencies. See the pytest_skipping plugin for details. - """ - __tracebackhide__ = True - raise Skipped(msg=msg) -skip.Exception = Skipped - -def fail(msg="", pytrace=True): - """ explicitly fail an currently-executing test with the given Message. - - :arg pytrace: if false the msg represents the full failure information - and no python traceback will be reported. - """ - __tracebackhide__ = True - raise Failed(msg=msg, pytrace=pytrace) -fail.Exception = Failed - - -def importorskip(modname, minversion=None): - """ return imported module if it has at least "minversion" as its - __version__ attribute. If no minversion is specified the a skip - is only triggered if the module can not be imported. - """ - __tracebackhide__ = True - compile(modname, '', 'eval') # to catch syntaxerrors - try: - __import__(modname) - except ImportError: - skip("could not import %r" %(modname,)) - mod = sys.modules[modname] - if minversion is None: - return mod - verattr = getattr(mod, '__version__', None) - if minversion is not None: - try: - from pkg_resources import parse_version as pv - except ImportError: - skip("we have a required version for %r but can not import " - "no pkg_resources to parse version strings." %(modname,)) - if verattr is None or pv(verattr) < pv(minversion): - skip("module %r has __version__ %r, required is: %r" %( - modname, verattr, minversion)) - return mod - diff --git a/_pytest/setuponly.py b/_pytest/setuponly.py new file mode 100644 index 000000000000000..a1c7457d7e506c0 --- /dev/null +++ b/_pytest/setuponly.py @@ -0,0 +1,74 @@ +from __future__ import absolute_import, division, print_function + +import pytest +import sys + + +def pytest_addoption(parser): + group = parser.getgroup("debugconfig") + group.addoption('--setuponly', '--setup-only', action="store_true", + help="only setup fixtures, do not execute tests.") + group.addoption('--setupshow', '--setup-show', action="store_true", + help="show setup of fixtures while executing tests.") + + +@pytest.hookimpl(hookwrapper=True) +def pytest_fixture_setup(fixturedef, request): + yield + config = request.config + if config.option.setupshow: + if hasattr(request, 'param'): + # Save the fixture parameter so ._show_fixture_action() can + # display it now and during the teardown (in .finish()). + if fixturedef.ids: + if callable(fixturedef.ids): + fixturedef.cached_param = fixturedef.ids(request.param) + else: + fixturedef.cached_param = fixturedef.ids[ + request.param_index] + else: + fixturedef.cached_param = request.param + _show_fixture_action(fixturedef, 'SETUP') + + +def pytest_fixture_post_finalizer(fixturedef): + if hasattr(fixturedef, "cached_result"): + config = fixturedef._fixturemanager.config + if config.option.setupshow: + _show_fixture_action(fixturedef, 'TEARDOWN') + if hasattr(fixturedef, "cached_param"): + del fixturedef.cached_param + + +def _show_fixture_action(fixturedef, msg): + config = fixturedef._fixturemanager.config + capman = config.pluginmanager.getplugin('capturemanager') + if capman: + out, err = capman.suspend_global_capture() + + tw = config.get_terminal_writer() + tw.line() + tw.write(' ' * 2 * fixturedef.scopenum) + tw.write('{step} {scope} {fixture}'.format( + step=msg.ljust(8), # align the output to TEARDOWN + scope=fixturedef.scope[0].upper(), + fixture=fixturedef.argname)) + + if msg == 'SETUP': + deps = sorted(arg for arg in fixturedef.argnames if arg != 'request') + if deps: + tw.write(' (fixtures used: {0})'.format(', '.join(deps))) + + if hasattr(fixturedef, 'cached_param'): + tw.write('[{0}]'.format(fixturedef.cached_param)) + + if capman: + capman.resume_global_capture() + sys.stdout.write(out) + sys.stderr.write(err) + + +@pytest.hookimpl(tryfirst=True) +def pytest_cmdline_main(config): + if config.option.setuponly: + config.option.setupshow = True diff --git a/_pytest/setupplan.py b/_pytest/setupplan.py new file mode 100644 index 000000000000000..e11bd40698b772b --- /dev/null +++ b/_pytest/setupplan.py @@ -0,0 +1,25 @@ +from __future__ import absolute_import, division, print_function + +import pytest + + +def pytest_addoption(parser): + group = parser.getgroup("debugconfig") + group.addoption('--setupplan', '--setup-plan', action="store_true", + help="show what fixtures and tests would be executed but " + "don't execute anything.") + + +@pytest.hookimpl(tryfirst=True) +def pytest_fixture_setup(fixturedef, request): + # Will return a dummy fixture if the setuponly option is provided. + if request.config.option.setupplan: + fixturedef.cached_result = (None, None, None) + return fixturedef.cached_result + + +@pytest.hookimpl(tryfirst=True) +def pytest_cmdline_main(config): + if config.option.setupplan: + config.option.setuponly = True + config.option.setupshow = True diff --git a/_pytest/skipping.py b/_pytest/skipping.py index 69157f485a01188..a1e5b43800b54f8 100644 --- a/_pytest/skipping.py +++ b/_pytest/skipping.py @@ -1,18 +1,21 @@ """ support for skip/xfail functions and markers. """ +from __future__ import absolute_import, division, print_function + import os +import six import sys import traceback -import py -import pytest +from _pytest.config import hookimpl from _pytest.mark import MarkInfo, MarkDecorator +from _pytest.outcomes import fail, skip, xfail, TEST_OUTCOME def pytest_addoption(parser): group = parser.getgroup("general") group.addoption('--runxfail', - action="store_true", dest="runxfail", default=False, - help="run tests even if they are marked xfail") + action="store_true", dest="runxfail", default=False, + help="run tests even if they are marked xfail") parser.addini("xfail_strict", "default for the strict parameter of xfail " "markers when not given explicitly (default: " @@ -23,62 +26,65 @@ def pytest_addoption(parser): def pytest_configure(config): if config.option.runxfail: + # yay a hack + import pytest old = pytest.xfail config._cleanup.append(lambda: setattr(pytest, "xfail", old)) + def nop(*args, **kwargs): pass - nop.Exception = XFailed + + nop.Exception = xfail.Exception setattr(pytest, "xfail", nop) config.addinivalue_line("markers", - "skipif(condition): skip the given test function if eval(condition) " - "results in a True value. Evaluation happens within the " - "module global context. Example: skipif('sys.platform == \"win32\"') " - "skips the test if we are on the win32 platform. see " - "http://pytest.org/latest/skipping.html" - ) + "skip(reason=None): skip the given test function with an optional reason. " + "Example: skip(reason=\"no way of currently testing this\") skips the " + "test." + ) config.addinivalue_line("markers", - "xfail(condition, reason=None, run=True, raises=None): mark the the test function " - "as an expected failure if eval(condition) has a True value. " - "Optionally specify a reason for better reporting and run=False if " - "you don't even want to execute the test function. If only specific " - "exception(s) are expected, you can list them in raises, and if the test fails " - "in other ways, it will be reported as a true failure. " - "See http://pytest.org/latest/skipping.html" - ) - - -def pytest_namespace(): - return dict(xfail=xfail) - - -class XFailed(pytest.fail.Exception): - """ raised from an explicit call to pytest.xfail() """ - - -def xfail(reason=""): - """ xfail an executing test or setup functions with the given reason.""" - __tracebackhide__ = True - raise XFailed(reason) -xfail.Exception = XFailed + "skipif(condition): skip the given test function if eval(condition) " + "results in a True value. Evaluation happens within the " + "module global context. Example: skipif('sys.platform == \"win32\"') " + "skips the test if we are on the win32 platform. see " + "http://pytest.org/latest/skipping.html" + ) + config.addinivalue_line("markers", + "xfail(condition, reason=None, run=True, raises=None, strict=False): " + "mark the test function as an expected failure if eval(condition) " + "has a True value. Optionally specify a reason for better reporting " + "and run=False if you don't even want to execute the test function. " + "If only specific exception(s) are expected, you can list them in " + "raises, and if the test fails in other ways, it will be reported as " + "a true failure. See http://pytest.org/latest/skipping.html" + ) -class MarkEvaluator: +class MarkEvaluator(object): def __init__(self, item, name): self.item = item - self.name = name - - @property - def holder(self): - return self.item.keywords.get(self.name) + self._marks = None + self._mark = None + self._mark_name = name def __bool__(self): - return bool(self.holder) + self._marks = self._get_marks() + return bool(self._marks) __nonzero__ = __bool__ def wasvalid(self): return not hasattr(self, 'exc') + def _get_marks(self): + + keyword = self.item.keywords.get(self._mark_name) + if isinstance(keyword, MarkDecorator): + return [keyword.mark] + elif isinstance(keyword, MarkInfo): + return [x.combined for x in keyword] + else: + return [] + def invalidraise(self, exc): raises = self.get('raises') if not raises: @@ -88,65 +94,67 @@ def invalidraise(self, exc): def istrue(self): try: return self._istrue() - except Exception: + except TEST_OUTCOME: self.exc = sys.exc_info() if isinstance(self.exc[1], SyntaxError): - msg = [" " * (self.exc[1].offset + 4) + "^",] + msg = [" " * (self.exc[1].offset + 4) + "^", ] msg.append("SyntaxError: invalid syntax") else: msg = traceback.format_exception_only(*self.exc[:2]) - pytest.fail("Error evaluating %r expression\n" - " %s\n" - "%s" - %(self.name, self.expr, "\n".join(msg)), - pytrace=False) + fail("Error evaluating %r expression\n" + " %s\n" + "%s" + % (self._mark_name, self.expr, "\n".join(msg)), + pytrace=False) def _getglobals(self): d = {'os': os, 'sys': sys, 'config': self.item.config} - func = self.item.obj - try: - d.update(func.__globals__) - except AttributeError: - d.update(func.func_globals) + if hasattr(self.item, 'obj'): + d.update(self.item.obj.__globals__) return d def _istrue(self): if hasattr(self, 'result'): return self.result - if self.holder: - d = self._getglobals() - if self.holder.args: - self.result = False - # "holder" might be a MarkInfo or a MarkDecorator; only - # MarkInfo keeps track of all parameters it received in an - # _arglist attribute - if hasattr(self.holder, '_arglist'): - arglist = self.holder._arglist + self._marks = self._get_marks() + + if self._marks: + self.result = False + for mark in self._marks: + self._mark = mark + if 'condition' in mark.kwargs: + args = (mark.kwargs['condition'],) else: - arglist = [(self.holder.args, self.holder.kwargs)] - for args, kwargs in arglist: - for expr in args: + args = mark.args + + for expr in args: + self.expr = expr + if isinstance(expr, six.string_types): + d = self._getglobals() + result = cached_eval(self.item.config, expr, d) + else: + if "reason" not in mark.kwargs: + # XXX better be checked at collection time + msg = "you need to specify reason=STRING " \ + "when using booleans as conditions." + fail(msg) + result = bool(expr) + if result: + self.result = True + self.reason = mark.kwargs.get('reason', None) self.expr = expr - if isinstance(expr, py.builtin._basestring): - result = cached_eval(self.item.config, expr, d) - else: - if "reason" not in kwargs: - # XXX better be checked at collection time - msg = "you need to specify reason=STRING " \ - "when using booleans as conditions." - pytest.fail(msg) - result = bool(expr) - if result: - self.result = True - self.reason = kwargs.get('reason', None) - self.expr = expr - return self.result - else: - self.result = True - return getattr(self, 'result', False) + return self.result + + if not args: + self.result = True + self.reason = mark.kwargs.get('reason', None) + return self.result + return False def get(self, attr, default=None): - return self.holder.kwargs.get(attr, default) + if self._mark is None: + return default + return self._mark.kwargs.get(attr, default) def getexplanation(self): expl = getattr(self, 'reason', None) or self.get('reason', None) @@ -158,32 +166,32 @@ def getexplanation(self): return expl -@pytest.hookimpl(tryfirst=True) +@hookimpl(tryfirst=True) def pytest_runtest_setup(item): # Check if skip or skipif are specified as pytest marks - + item._skipped_by_mark = False skipif_info = item.keywords.get('skipif') if isinstance(skipif_info, (MarkInfo, MarkDecorator)): eval_skipif = MarkEvaluator(item, 'skipif') if eval_skipif.istrue(): - item._evalskip = eval_skipif - pytest.skip(eval_skipif.getexplanation()) + item._skipped_by_mark = True + skip(eval_skipif.getexplanation()) skip_info = item.keywords.get('skip') if isinstance(skip_info, (MarkInfo, MarkDecorator)): - item._evalskip = True + item._skipped_by_mark = True if 'reason' in skip_info.kwargs: - pytest.skip(skip_info.kwargs['reason']) + skip(skip_info.kwargs['reason']) elif skip_info.args: - pytest.skip(skip_info.args[0]) + skip(skip_info.args[0]) else: - pytest.skip("unconditional skip") + skip("unconditional skip") item._evalxfail = MarkEvaluator(item, 'xfail') check_xfail_no_run(item) -@pytest.mark.hookwrapper +@hookimpl(hookwrapper=True) def pytest_pyfunc_call(pyfuncitem): check_xfail_no_run(pyfuncitem) outcome = yield @@ -198,7 +206,7 @@ def check_xfail_no_run(item): evalxfail = item._evalxfail if evalxfail.istrue(): if not evalxfail.get('run', True): - pytest.xfail("[NOTRUN] " + evalxfail.getexplanation()) + xfail("[NOTRUN] " + evalxfail.getexplanation()) def check_strict_xfail(pyfuncitem): @@ -210,27 +218,33 @@ def check_strict_xfail(pyfuncitem): if is_strict_xfail: del pyfuncitem._evalxfail explanation = evalxfail.getexplanation() - pytest.fail('[XPASS(strict)] ' + explanation, pytrace=False) + fail('[XPASS(strict)] ' + explanation, pytrace=False) -@pytest.hookimpl(hookwrapper=True) +@hookimpl(hookwrapper=True) def pytest_runtest_makereport(item, call): outcome = yield rep = outcome.get_result() evalxfail = getattr(item, '_evalxfail', None) - evalskip = getattr(item, '_evalskip', None) # unitttest special case, see setting of _unexpectedsuccess if hasattr(item, '_unexpectedsuccess') and rep.when == "call": - # we need to translate into how pytest encodes xpass - rep.wasxfail = "reason: " + repr(item._unexpectedsuccess) - rep.outcome = "failed" + from _pytest.compat import _is_unittest_unexpected_success_a_failure + if item._unexpectedsuccess: + rep.longrepr = "Unexpected success: {0}".format(item._unexpectedsuccess) + else: + rep.longrepr = "Unexpected success" + if _is_unittest_unexpected_success_a_failure(): + rep.outcome = "failed" + else: + rep.outcome = "passed" + rep.wasxfail = rep.longrepr elif item.config.option.runxfail: pass # don't interefere - elif call.excinfo and call.excinfo.errisinstance(pytest.xfail.Exception): + elif call.excinfo and call.excinfo.errisinstance(xfail.Exception): rep.wasxfail = "reason: " + call.excinfo.value.msg rep.outcome = "skipped" elif evalxfail and not rep.skipped and evalxfail.wasvalid() and \ - evalxfail.istrue(): + evalxfail.istrue(): if call.excinfo: if evalxfail.invalidraise(call.excinfo.value): rep.outcome = "failed" @@ -238,9 +252,16 @@ def pytest_runtest_makereport(item, call): rep.outcome = "skipped" rep.wasxfail = evalxfail.getexplanation() elif call.when == "call": - rep.outcome = "failed" # xpass outcome - rep.wasxfail = evalxfail.getexplanation() - elif evalskip is not None and rep.skipped and type(rep.longrepr) is tuple: + strict_default = item.config.getini('xfail_strict') + is_strict_xfail = evalxfail.get('strict', strict_default) + explanation = evalxfail.getexplanation() + if is_strict_xfail: + rep.outcome = "failed" + rep.longrepr = "[XPASS(strict)] {0}".format(explanation) + else: + rep.outcome = "passed" + rep.wasxfail = explanation + elif item._skipped_by_mark and rep.skipped and type(rep.longrepr) is tuple: # skipped by mark.skipif; change the location of the failure # to point to the item definition, otherwise it will display # the location of where the skip exception was raised within pytest @@ -249,18 +270,22 @@ def pytest_runtest_makereport(item, call): rep.longrepr = filename, line, reason # called by terminalreporter progress reporting + + def pytest_report_teststatus(report): if hasattr(report, "wasxfail"): if report.skipped: return "xfailed", "x", "xfail" - elif report.failed: + elif report.passed: return "xpassed", "X", ("XPASS", {'yellow': True}) # called by the terminalreporter instance/plugin + + def pytest_terminal_summary(terminalreporter): tr = terminalreporter if not tr.reportchars: - #for name in "xfailed skipped failed xpassed": + # for name in "xfailed skipped failed xpassed": # if not tr.stats.get(name, 0): # tr.write_line("HINT: use '-r' option to see extra " # "summary info about tests") @@ -287,12 +312,14 @@ def pytest_terminal_summary(terminalreporter): for line in lines: tr._tw.line(line) + def show_simple(terminalreporter, lines, stat, format): failed = terminalreporter.stats.get(stat) if failed: for rep in failed: pos = terminalreporter.config.cwd_relative_nodeid(rep.nodeid) - lines.append(format %(pos,)) + lines.append(format % (pos,)) + def show_xfailed(terminalreporter, lines): xfailed = terminalreporter.stats.get("xfailed") @@ -304,13 +331,15 @@ def show_xfailed(terminalreporter, lines): if reason: lines.append(" " + str(reason)) + def show_xpassed(terminalreporter, lines): xpassed = terminalreporter.stats.get("xpassed") if xpassed: for rep in xpassed: pos = terminalreporter.config.cwd_relative_nodeid(rep.nodeid) reason = rep.wasxfail - lines.append("XPASS %s %s" %(pos, reason)) + lines.append("XPASS %s %s" % (pos, reason)) + def cached_eval(config, expr, d): if not hasattr(config, '_evalcache'): @@ -329,26 +358,40 @@ def folded_skips(skipped): for event in skipped: key = event.longrepr assert len(key) == 3, (event, key) + keywords = getattr(event, 'keywords', {}) + # folding reports with global pytestmark variable + # this is workaround, because for now we cannot identify the scope of a skip marker + # TODO: revisit after marks scope would be fixed + when = getattr(event, 'when', None) + if when == 'setup' and 'skip' in keywords and 'pytestmark' not in keywords: + key = (key[0], None, key[2], ) d.setdefault(key, []).append(event) - l = [] + values = [] for key, events in d.items(): - l.append((len(events),) + key) - return l + values.append((len(events),) + key) + return values + def show_skipped(terminalreporter, lines): tr = terminalreporter skipped = tr.stats.get('skipped', []) if skipped: - #if not tr.hasopt('skipped'): + # if not tr.hasopt('skipped'): # tr.write_line( # "%d skipped tests, specify -rs for more info" % # len(skipped)) # return fskips = folded_skips(skipped) if fskips: - #tr.write_sep("_", "skipped test summary") + # tr.write_sep("_", "skipped test summary") for num, fspath, lineno, reason in fskips: if reason.startswith("Skipped: "): reason = reason[9:] - lines.append("SKIP [%d] %s:%d: %s" % - (num, fspath, lineno, reason)) + if lineno is not None: + lines.append( + "SKIP [%d] %s:%d: %s" % + (num, fspath, lineno + 1, reason)) + else: + lines.append( + "SKIP [%d] %s: %s" % + (num, fspath, reason)) diff --git a/_pytest/standalonetemplate.py b/_pytest/standalonetemplate.py deleted file mode 100755 index 484d5d1b25f0da9..000000000000000 --- a/_pytest/standalonetemplate.py +++ /dev/null @@ -1,89 +0,0 @@ -#! /usr/bin/env python - -# Hi There! -# You may be wondering what this giant blob of binary data here is, you might -# even be worried that we're up to something nefarious (good for you for being -# paranoid!). This is a base64 encoding of a zip file, this zip file contains -# a fully functional basic pytest script. -# -# Pytest is a thing that tests packages, pytest itself is a package that some- -# one might want to install, especially if they're looking to run tests inside -# some package they want to install. Pytest has a lot of code to collect and -# execute tests, and other such sort of "tribal knowledge" that has been en- -# coded in its code base. Because of this we basically include a basic copy -# of pytest inside this blob. We do this because it let's you as a maintainer -# or application developer who wants people who don't deal with python much to -# easily run tests without installing the complete pytest package. -# -# If you're wondering how this is created: you can create it yourself if you -# have a complete pytest installation by using this command on the command- -# line: ``py.test --genscript=runtests.py``. - -sources = """ -@SOURCES@""" - -import sys -import base64 -import zlib - -class DictImporter(object): - def __init__(self, sources): - self.sources = sources - - def find_module(self, fullname, path=None): - if fullname == "argparse" and sys.version_info >= (2,7): - # we were generated with = (3, 0): - exec("def do_exec(co, loc): exec(co, loc)\n") - import pickle - sources = sources.encode("ascii") # ensure bytes - sources = pickle.loads(zlib.decompress(base64.decodebytes(sources))) - else: - import cPickle as pickle - exec("def do_exec(co, loc): exec co in loc\n") - sources = pickle.loads(zlib.decompress(base64.decodestring(sources))) - - importer = DictImporter(sources) - sys.meta_path.insert(0, importer) - entry = "@ENTRY@" - do_exec(entry, locals()) # noqa diff --git a/_pytest/terminal.py b/_pytest/terminal.py index 825f553ef2cd689..c5cbe14a1051759 100644 --- a/_pytest/terminal.py +++ b/_pytest/terminal.py @@ -2,45 +2,58 @@ This is a good source for looking at the various reporting hooks. """ -from _pytest.main import EXIT_OK, EXIT_TESTSFAILED, EXIT_INTERRUPTED, \ - EXIT_USAGEERROR, EXIT_NOTESTSCOLLECTED -import pytest -import py +from __future__ import absolute_import, division, print_function + +import itertools +import platform import sys import time -import platform -import _pytest._pluggy as pluggy +import pluggy +import py +import six + +import pytest +from _pytest import nodes +from _pytest.main import EXIT_OK, EXIT_TESTSFAILED, EXIT_INTERRUPTED, \ + EXIT_USAGEERROR, EXIT_NOTESTSCOLLECTED def pytest_addoption(parser): group = parser.getgroup("terminal reporting", "reporting", after="general") group._addoption('-v', '--verbose', action="count", - dest="verbose", default=0, help="increase verbosity."), + dest="verbose", default=0, help="increase verbosity."), group._addoption('-q', '--quiet', action="count", - dest="quiet", default=0, help="decrease verbosity."), + dest="quiet", default=0, help="decrease verbosity."), group._addoption('-r', - action="store", dest="reportchars", default=None, metavar="chars", - help="show extra test summary info as specified by chars (f)ailed, " - "(E)error, (s)skipped, (x)failed, (X)passed (w)pytest-warnings " - "(p)passed, (P)passed with output, (a)all except pP.") + action="store", dest="reportchars", default='', metavar="chars", + help="show extra test summary info as specified by chars (f)ailed, " + "(E)error, (s)skipped, (x)failed, (X)passed, " + "(p)passed, (P)passed with output, (a)all except pP. " + "Warnings are displayed at all times except when " + "--disable-warnings is set") + group._addoption('--disable-warnings', '--disable-pytest-warnings', default=False, + dest='disable_warnings', action='store_true', + help='disable warnings summary') group._addoption('-l', '--showlocals', - action="store_true", dest="showlocals", default=False, - help="show locals in tracebacks (disabled by default).") - group._addoption('--report', - action="store", dest="report", default=None, metavar="opts", - help="(deprecated, use -r)") + action="store_true", dest="showlocals", default=False, + help="show locals in tracebacks (disabled by default).") group._addoption('--tb', metavar="style", - action="store", dest="tbstyle", default='auto', - choices=['auto', 'long', 'short', 'no', 'line', 'native'], - help="traceback print mode (auto/long/short/line/native/no).") + action="store", dest="tbstyle", default='auto', + choices=['auto', 'long', 'short', 'no', 'line', 'native'], + help="traceback print mode (auto/long/short/line/native/no).") group._addoption('--fulltrace', '--full-trace', - action="store_true", default=False, - help="don't cut any tracebacks (default is to cut).") + action="store_true", default=False, + help="don't cut any tracebacks (default is to cut).") group._addoption('--color', metavar="color", - action="store", dest="color", default='auto', - choices=['yes', 'no', 'auto'], - help="color terminal output (yes/no/auto).") + action="store", dest="color", default='auto', + choices=['yes', 'no', 'auto'], + help="color terminal output (yes/no/auto).") + + parser.addini("console_output_style", + help="console output: classic or with additional progress information (classic|progress).", + default='progress') + def pytest_configure(config): config.option.verbose -= config.option.quiet @@ -52,20 +65,14 @@ def mywriter(tags, args): reporter.write_line("[traceconfig] " + msg) config.trace.root.setprocessor("pytest:config", mywriter) + def getreportopt(config): reportopts = "" - optvalue = config.option.report - if optvalue: - py.builtin.print_("DEPRECATED: use -r instead of --report option.", - file=sys.stderr) - if optvalue: - for setting in optvalue.split(","): - setting = setting.strip() - if setting == "skipped": - reportopts += "s" - elif setting == "xfailed": - reportopts += "x" reportchars = config.option.reportchars + if not config.option.disable_warnings and 'w' not in reportchars: + reportchars += 'w' + elif config.option.disable_warnings and 'w' in reportchars: + reportchars = reportchars.replace('w', '') if reportchars: for char in reportchars: if char not in reportopts and char != 'a': @@ -74,6 +81,7 @@ def getreportopt(config): reportopts = 'fEsxXw' return reportopts + def pytest_report_teststatus(report): if report.passed: letter = "." @@ -85,13 +93,41 @@ def pytest_report_teststatus(report): letter = "f" return report.outcome, letter, report.outcome.upper() + class WarningReport: + """ + Simple structure to hold warnings information captured by ``pytest_logwarning``. + """ + def __init__(self, code, message, nodeid=None, fslocation=None): + """ + :param code: unused + :param str message: user friendly message about the warning + :param str|None nodeid: node id that generated the warning (see ``get_location``). + :param tuple|py.path.local fslocation: + file system location of the source of the warning (see ``get_location``). + """ self.code = code self.message = message self.nodeid = nodeid self.fslocation = fslocation + def get_location(self, config): + """ + Returns the more user-friendly information about the location + of a warning, or None. + """ + if self.nodeid: + return self.nodeid + if self.fslocation: + if isinstance(self.fslocation, tuple) and len(self.fslocation) >= 2: + filename, linenum = self.fslocation[:2] + relpath = py.path.local(filename).relto(config.invocation_dir) + return '%s:%s' % (relpath, linenum) + else: + return str(self.fslocation) + return None + class TerminalReporter: def __init__(self, config, file=None): @@ -102,17 +138,20 @@ def __init__(self, config, file=None): self.showfspath = self.verbosity >= 0 self.showlongtestinfo = self.verbosity > 0 self._numcollected = 0 + self._session = None self.stats = {} self.startdir = py.path.local() if file is None: file = sys.stdout - self._tw = self.writer = _pytest.config.create_terminal_writer(config, - file) + self._tw = _pytest.config.create_terminal_writer(config, file) + self._screen_width = self._tw.fullwidth self.currentfspath = None self.reportchars = getreportopt(config) self.hasmarkup = self._tw.hasmarkup self.isatty = file.isatty() + self._progress_items_reported = 0 + self._show_progress_info = self.config.getini('console_output_style') == 'progress' def hasopt(self, char): char = {'xfailed': 'x', 'skipped': 's'}.get(char, char) @@ -121,6 +160,8 @@ def hasopt(self, char): def write_fspath_result(self, nodeid, res): fspath = self.config.rootdir.join(nodeid.split("::")[0]) if fspath != self.currentfspath: + if self.currentfspath is not None: + self._write_progress_information_filling_space() self.currentfspath = fspath fspath = self.startdir.bestrelpath(fspath) self._tw.line() @@ -135,6 +176,7 @@ def write_ensure_prefix(self, prefix, extra="", **kwargs): if extra: self._tw.write(extra, **kwargs) self.currentfspath = -2 + self._write_progress_information_filling_space() def ensure_newline(self): if self.currentfspath: @@ -145,14 +187,28 @@ def write(self, content, **markup): self._tw.write(content, **markup) def write_line(self, line, **markup): - if not py.builtin._istext(line): - line = py.builtin.text(line, errors="replace") + if not isinstance(line, six.text_type): + line = six.text_type(line, errors="replace") self.ensure_newline() self._tw.line(line, **markup) def rewrite(self, line, **markup): + """ + Rewinds the terminal cursor to the beginning and writes the given line. + + :kwarg erase: if True, will also add spaces until the full terminal width to ensure + previous lines are properly erased. + + The rest of the keyword arguments are markup instructions. + """ + erase = markup.pop('erase', False) + if erase: + fill_count = self._tw.fullwidth - len(line) - 1 + fill = ' ' * fill_count + else: + fill = '' line = str(line) - self._tw.write("\r" + line, **markup) + self._tw.write("\r" + line + fill, **markup) def write_sep(self, sep, title=None, **markup): self.ensure_newline() @@ -165,14 +221,12 @@ def line(self, msg, **kw): self._tw.line(msg, **kw) def pytest_internalerror(self, excrepr): - for line in py.builtin.text(excrepr).split("\n"): + for line in six.text_type(excrepr).split("\n"): self.write_line("INTERNALERROR> " + line) return 1 def pytest_logwarning(self, code, fslocation, message, nodeid): warnings = self.stats.setdefault("warnings", []) - if isinstance(fslocation, tuple): - fslocation = "%s:%d" % fslocation warning = WarningReport(code=code, fslocation=fslocation, message=message, nodeid=nodeid) warnings.append(warning) @@ -202,38 +256,76 @@ def pytest_runtest_logreport(self, report): rep = report res = self.config.hook.pytest_report_teststatus(report=rep) cat, letter, word = res + if isinstance(word, tuple): + word, markup = word + else: + markup = None self.stats.setdefault(cat, []).append(rep) self._tests_ran = True if not letter and not word: # probably passed setup/teardown return + running_xdist = hasattr(rep, 'node') + self._progress_items_reported += 1 if self.verbosity <= 0: - if not hasattr(rep, 'node') and self.showfspath: + if not running_xdist and self.showfspath: self.write_fspath_result(rep.nodeid, letter) else: self._tw.write(letter) + self._write_progress_if_past_edge() else: - if isinstance(word, tuple): - word, markup = word - else: + if markup is None: if rep.passed: - markup = {'green':True} + markup = {'green': True} elif rep.failed: - markup = {'red':True} + markup = {'red': True} elif rep.skipped: - markup = {'yellow':True} + markup = {'yellow': True} + else: + markup = {} line = self._locationline(rep.nodeid, *rep.location) - if not hasattr(rep, 'node'): + if not running_xdist: self.write_ensure_prefix(line, word, **markup) - #self._tw.write(word, **markup) else: self.ensure_newline() - if hasattr(rep, 'node'): - self._tw.write("[%s] " % rep.node.gateway.id) + self._tw.write("[%s]" % rep.node.gateway.id) + if self._show_progress_info: + self._tw.write(self._get_progress_information_message() + " ", cyan=True) + else: + self._tw.write(' ') self._tw.write(word, **markup) self._tw.write(" " + line) self.currentfspath = -2 + def _write_progress_if_past_edge(self): + if not self._show_progress_info: + return + last_item = self._progress_items_reported == self._session.testscollected + if last_item: + self._write_progress_information_filling_space() + return + + past_edge = self._tw.chars_on_current_line + self._PROGRESS_LENGTH + 1 >= self._screen_width + if past_edge: + msg = self._get_progress_information_message() + self._tw.write(msg + '\n', cyan=True) + + _PROGRESS_LENGTH = len(' [100%]') + + def _get_progress_information_message(self): + collected = self._session.testscollected + if collected: + progress = self._progress_items_reported * 100 // collected + return ' [{:3d}%]'.format(progress) + return ' [100%]' + + def _write_progress_information_filling_space(self): + if not self._show_progress_info: + return + msg = self._get_progress_information_message() + fill = ' ' * (self._tw.fullwidth - self._tw.chars_on_current_line - len(msg) - 1) + self.write(fill + msg, cyan=True) + def pytest_collection(self): if not self.isatty and self.config.option.verbose >= 1: self.write("collecting ... ", bold=True) @@ -246,7 +338,7 @@ def pytest_collectreport(self, report): items = [x for x in report.result if isinstance(x, pytest.Item)] self._numcollected += len(items) if self.isatty: - #self.write_fspath_result(report.nodeid, 'E') + # self.write_fspath_result(report.nodeid, 'E') self.report_collect() def report_collect(self, final=False): @@ -259,15 +351,15 @@ def report_collect(self, final=False): line = "collected " else: line = "collecting " - line += str(self._numcollected) + " items" + line += str(self._numcollected) + " item" + ('' if self._numcollected == 1 else 's') if errors: line += " / %d errors" % errors if skipped: line += " / %d skipped" % skipped if self.isatty: + self.rewrite(line, bold=True, erase=True) if final: - line += " \n" - self.rewrite(line, bold=True) + self.write('\n') else: self.write_line(line) @@ -276,6 +368,7 @@ def pytest_collection_modifyitems(self): @pytest.hookimpl(trylast=True) def pytest_sessionstart(self, session): + self._session = session self._sessionstarttime = time.time() if not self.showheader: return @@ -293,6 +386,9 @@ def pytest_sessionstart(self, session): self.write_line(msg) lines = self.config.hook.pytest_report_header( config=self.config, startdir=self.startdir) + self._write_report_lines_from_hooks(lines) + + def _write_report_lines_from_hooks(self, lines): lines.reverse() for line in flatten(lines): self.write_line(line) @@ -300,8 +396,8 @@ def pytest_sessionstart(self, session): def pytest_report_header(self, config): inifile = "" if config.inifile: - inifile = config.rootdir.bestrelpath(config.inifile) - lines = ["rootdir: %s, inifile: %s" %(config.rootdir, inifile)] + inifile = " " + config.rootdir.bestrelpath(config.inifile) + lines = ["rootdir: %s, inifile:%s" % (config.rootdir, inifile)] plugininfo = config.pluginmanager.list_plugin_distinfo() if plugininfo: @@ -319,10 +415,9 @@ def pytest_collection_finish(self, session): rep.toterminal(self._tw) return 1 return 0 - if not self.showheader: - return - #for i, testarg in enumerate(self.config.args): - # self.write_line("test path %d: %s" %(i+1, testarg)) + lines = self.config.hook.pytest_report_collectionfinish( + config=self.config, startdir=self.startdir, items=session.items) + self._write_report_lines_from_hooks(lines) def _printcollecteditems(self, items): # to print out items and their parent collectors @@ -345,14 +440,14 @@ def _printcollecteditems(self, items): stack = [] indent = "" for item in items: - needed_collectors = item.listchain()[1:] # strip root node + needed_collectors = item.listchain()[1:] # strip root node while stack: if stack == needed_collectors[:len(stack)]: break stack.pop() for col in needed_collectors[len(stack):]: stack.append(col) - #if col.name == "()": + # if col.name == "()": # continue indent = (len(stack) - 1) * " " self._tw.line("%s%s" % (indent, col)) @@ -366,7 +461,8 @@ def pytest_sessionfinish(self, exitstatus): EXIT_OK, EXIT_TESTSFAILED, EXIT_INTERRUPTED, EXIT_USAGEERROR, EXIT_NOTESTSCOLLECTED) if exitstatus in summary_exit_codes: - self.config.hook.pytest_terminal_summary(terminalreporter=self) + self.config.hook.pytest_terminal_summary(terminalreporter=self, + exitstatus=exitstatus) self.summary_errors() self.summary_failures() self.summary_warnings() @@ -400,15 +496,15 @@ def mkrel(nodeid): line = self.config.cwd_relative_nodeid(nodeid) if domain and line.endswith(domain): line = line[:-len(domain)] - l = domain.split("[") - l[0] = l[0].replace('.', '::') # don't replace '.' in params - line += "[".join(l) + values = domain.split("[") + values[0] = values[0].replace('.', '::') # don't replace '.' in params + line += "[".join(values) return line # collect_fspath comes from testid which has a "/"-normalized path if fspath: res = mkrel(nodeid).replace("::()", "") # parens-normalization - if nodeid.split("::")[0] != fspath.replace("\\", "/"): + if nodeid.split("::")[0] != fspath.replace("\\", nodes.SEP): res += " <- " + self.startdir.bestrelpath(fspath) else: res = "[location]" @@ -419,7 +515,7 @@ def _getfailureheadline(self, rep): fspath, lineno, domain = rep.location return domain else: - return "test session" # XXX? + return "test session" # XXX? def _getcrashline(self, rep): try: @@ -434,21 +530,29 @@ def _getcrashline(self, rep): # summaries for sessionfinish # def getreports(self, name): - l = [] + values = [] for x in self.stats.get(name, []): if not hasattr(x, '_pdbshown'): - l.append(x) - return l + values.append(x) + return values def summary_warnings(self): if self.hasopt("w"): - warnings = self.stats.get("warnings") - if not warnings: + all_warnings = self.stats.get("warnings") + if not all_warnings: return - self.write_sep("=", "pytest-warning summary") - for w in warnings: - self._tw.line("W%s %s %s" % (w.code, - w.fslocation, w.message)) + + grouped = itertools.groupby(all_warnings, key=lambda wr: wr.get_location(self.config)) + + self.write_sep("=", "warnings summary", yellow=True, bold=False) + for location, warning_records in grouped: + self._tw.line(str(location) or '') + for w in warning_records: + lines = w.message.splitlines() + indented = '\n'.join(' ' + x for x in lines) + self._tw.line(indented) + self._tw.line() + self._tw.line('-- Docs: http://doc.pytest.org/en/latest/warnings.html') def summary_passes(self): if self.config.option.tbstyle != "no": @@ -462,6 +566,14 @@ def summary_passes(self): self.write_sep("_", msg) self._outrep_summary(rep) + def print_teardown_sections(self, rep): + for secname, content in rep.sections: + if 'teardown' in secname: + self._tw.sep('-', secname) + if content[-1:] == "\n": + content = content[:-1] + self._tw.line(content) + def summary_failures(self): if self.config.option.tbstyle != "no": reports = self.getreports('failed') @@ -477,6 +589,9 @@ def summary_failures(self): markup = {'red': True, 'bold': True} self.write_sep("_", msg, **markup) self._outrep_summary(rep) + for report in self.getreports(''): + if report.nodeid == rep.nodeid and report.when == 'teardown': + self.print_teardown_sections(report) def summary_errors(self): if self.config.option.tbstyle != "no": @@ -517,16 +632,9 @@ def summary_stats(self): def summary_deselected(self): if 'deselected' in self.stats: - l = [] - k = self.config.option.keyword - if k: - l.append("-k%s" % k) - m = self.config.option.markexpr - if m: - l.append("-m %r" % m) - if l: - self.write_sep("=", "%d tests deselected by %r" % ( - len(self.stats['deselected']), " ".join(l)), bold=True) + self.write_sep("=", "%d tests deselected" % ( + len(self.stats['deselected'])), bold=True) + def repr_pythonversion(v=None): if v is None: @@ -536,30 +644,30 @@ def repr_pythonversion(v=None): except (TypeError, ValueError): return str(v) -def flatten(l): - for x in l: + +def flatten(values): + for x in values: if isinstance(x, (list, tuple)): for y in flatten(x): yield y else: yield x + def build_summary_stats_line(stats): keys = ("failed passed skipped deselected " - "xfailed xpassed warnings error").split() - key_translation = {'warnings': 'pytest-warnings'} + "xfailed xpassed warnings error").split() unknown_key_seen = False for key in stats.keys(): if key not in keys: - if key: # setup/teardown reports have an empty key, ignore them + if key: # setup/teardown reports have an empty key, ignore them keys.append(key) unknown_key_seen = True parts = [] for key in keys: val = stats.get(key, None) if val: - key_name = key_translation.get(key, key) - parts.append("%d %s" % (len(val), key_name)) + parts.append("%d %s" % (len(val), key)) if parts: line = ", ".join(parts) @@ -579,7 +687,7 @@ def build_summary_stats_line(stats): def _plugin_nameversions(plugininfo): - l = [] + values = [] for plugin, dist in plugininfo: # gets us name and version! name = '{dist.project_name}-{dist.version}'.format(dist=dist) @@ -588,6 +696,6 @@ def _plugin_nameversions(plugininfo): name = name[7:] # we decided to print python package names # they can have more than one plugin - if name not in l: - l.append(name) - return l + if name not in values: + values.append(name) + return values diff --git a/_pytest/tmpdir.py b/_pytest/tmpdir.py index ebc48dbe5b24ba0..da1b032237afbf4 100644 --- a/_pytest/tmpdir.py +++ b/_pytest/tmpdir.py @@ -1,9 +1,11 @@ """ support for providing temporary directories to test functions. """ +from __future__ import absolute_import, division, print_function + import re import pytest import py -from _pytest.monkeypatch import monkeypatch +from _pytest.monkeypatch import MonkeyPatch class TempdirFactory: @@ -23,7 +25,7 @@ def ensuretemp(self, string, dir=1): provides an empty unique-per-test-invocation directory and is guaranteed to be empty. """ - #py.log._apiwarn(">1.1", "use tmpdir function argument") + # py.log._apiwarn(">1.1", "use tmpdir function argument") return self.getbasetemp().ensure(string, dir=dir) def mktemp(self, basename, numbered=True): @@ -36,7 +38,7 @@ def mktemp(self, basename, numbered=True): p = basetemp.mkdir(basename) else: p = py.path.local.make_numbered_dir(prefix=basename, - keep=0, rootdir=basetemp, lock_timeout=None) + keep=0, rootdir=basetemp, lock_timeout=None) self.trace("mktemp", p) return p @@ -81,6 +83,7 @@ def get_user(): except (ImportError, KeyError): return None + # backward compatibility TempdirHandler = TempdirFactory @@ -92,7 +95,7 @@ def pytest_configure(config): available at pytest_configure time, but ideally should be moved entirely to the tmpdir_factory session fixture. """ - mp = monkeypatch() + mp = MonkeyPatch() t = TempdirFactory(config) config._cleanup.extend([mp.undo, t.finish]) mp.setattr(config, '_tmpdirhandler', t, raising=False) @@ -108,14 +111,14 @@ def tmpdir_factory(request): @pytest.fixture def tmpdir(request, tmpdir_factory): - """return a temporary directory path object + """Return a temporary directory path object which is unique to each test function invocation, created as a sub directory of the base temporary directory. The returned object is a `py.path.local`_ path object. """ name = request.node.name - name = re.sub("[\W]", "_", name) + name = re.sub(r"[\W]", "_", name) MAXVAL = 30 if len(name) > MAXVAL: name = name[:MAXVAL] diff --git a/_pytest/unittest.py b/_pytest/unittest.py index 8120e94fbf41be2..3ddb39495e39703 100644 --- a/_pytest/unittest.py +++ b/_pytest/unittest.py @@ -1,14 +1,14 @@ """ discovery and running of std-library "unittest" style tests. """ -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function import sys import traceback -import pytest -# for transfering markers +# for transferring markers import _pytest._code -from _pytest.python import transfer_markers -from _pytest.skipping import MarkEvaluator +from _pytest.config import hookimpl +from _pytest.outcomes import fail, skip, xfail +from _pytest.python import transfer_markers, Class, Module, Function def pytest_pycollect_makeitem(collector, name, obj): @@ -22,11 +22,11 @@ def pytest_pycollect_makeitem(collector, name, obj): return UnitTestCase(name, parent=collector) -class UnitTestCase(pytest.Class): +class UnitTestCase(Class): # marker for fixturemanger.getfixtureinfo() # to declare that our children do not support funcargs nofuncargs = True - + def setup(self): cls = self.obj if getattr(cls, '__unittest_skip__', False): @@ -46,10 +46,12 @@ def collect(self): return self.session._fixturemanager.parsefactories(self, unittest=True) loader = TestLoader() - module = self.getparent(pytest.Module).obj + module = self.getparent(Module).obj foundsomething = False for name in loader.getTestCaseNames(self.obj): x = getattr(self.obj, name) + if not getattr(x, '__test__', True): + continue funcobj = getattr(x, 'im_func', x) transfer_markers(funcobj, cls, module) yield TestCaseFunction(name, parent=self) @@ -63,8 +65,7 @@ def collect(self): yield TestCaseFunction('runTest', parent=self) - -class TestCaseFunction(pytest.Function): +class TestCaseFunction(Function): _excinfo = None def setup(self): @@ -92,6 +93,9 @@ def _fix_unittest_skip_decorator(self): def teardown(self): if hasattr(self._testcase, 'teardown_method'): self._testcase.teardown_method(self._obj) + # Allow garbage collection on TestCase instance attributes. + self._testcase = None + self._obj = None def startTest(self, testcase): pass @@ -104,38 +108,38 @@ def _addexcinfo(self, rawexcinfo): except TypeError: try: try: - l = traceback.format_exception(*rawexcinfo) - l.insert(0, "NOTE: Incompatible Exception Representation, " - "displaying natively:\n\n") - pytest.fail("".join(l), pytrace=False) - except (pytest.fail.Exception, KeyboardInterrupt): + values = traceback.format_exception(*rawexcinfo) + values.insert(0, "NOTE: Incompatible Exception Representation, " + "displaying natively:\n\n") + fail("".join(values), pytrace=False) + except (fail.Exception, KeyboardInterrupt): raise - except: - pytest.fail("ERROR: Unknown Incompatible Exception " - "representation:\n%r" %(rawexcinfo,), pytrace=False) + except: # noqa + fail("ERROR: Unknown Incompatible Exception " + "representation:\n%r" % (rawexcinfo,), pytrace=False) except KeyboardInterrupt: raise - except pytest.fail.Exception: + except fail.Exception: excinfo = _pytest._code.ExceptionInfo() self.__dict__.setdefault('_excinfo', []).append(excinfo) def addError(self, testcase, rawexcinfo): self._addexcinfo(rawexcinfo) + def addFailure(self, testcase, rawexcinfo): self._addexcinfo(rawexcinfo) def addSkip(self, testcase, reason): try: - pytest.skip(reason) - except pytest.skip.Exception: - self._evalskip = MarkEvaluator(self, 'SkipTest') - self._evalskip.result = True + skip(reason) + except skip.Exception: + self._skipped_by_mark = True self._addexcinfo(sys.exc_info()) def addExpectedFailure(self, testcase, rawexcinfo, reason=""): try: - pytest.xfail(str(reason)) - except pytest.xfail.Exception: + xfail(str(reason)) + except xfail.Exception: self._addexcinfo(sys.exc_info()) def addUnexpectedSuccess(self, testcase, reason=""): @@ -147,17 +151,42 @@ def addSuccess(self, testcase): def stopTest(self, testcase): pass + def _handle_skip(self): + # implements the skipping machinery (see #2137) + # analog to pythons Lib/unittest/case.py:run + testMethod = getattr(self._testcase, self._testcase._testMethodName) + if (getattr(self._testcase.__class__, "__unittest_skip__", False) or + getattr(testMethod, "__unittest_skip__", False)): + # If the class or method was skipped. + skip_why = (getattr(self._testcase.__class__, '__unittest_skip_why__', '') or + getattr(testMethod, '__unittest_skip_why__', '')) + try: # PY3, unittest2 on PY2 + self._testcase._addSkip(self, self._testcase, skip_why) + except TypeError: # PY2 + if sys.version_info[0] != 2: + raise + self._testcase._addSkip(self, skip_why) + return True + return False + def runtest(self): - self._testcase(result=self) + if self.config.pluginmanager.get_plugin("pdbinvoke") is None: + self._testcase(result=self) + else: + # disables tearDown and cleanups for post mortem debugging (see #1890) + if self._handle_skip(): + return + self._testcase.debug() def _prunetraceback(self, excinfo): - pytest.Function._prunetraceback(self, excinfo) + Function._prunetraceback(self, excinfo) traceback = excinfo.traceback.filter( - lambda x:not x.frame.f_globals.get('__unittest')) + lambda x: not x.frame.f_globals.get('__unittest')) if traceback: excinfo.traceback = traceback -@pytest.hookimpl(tryfirst=True) + +@hookimpl(tryfirst=True) def pytest_runtest_makereport(item, call): if isinstance(item, TestCaseFunction): if item._excinfo: @@ -169,15 +198,17 @@ def pytest_runtest_makereport(item, call): # twisted trial support -@pytest.hookimpl(hookwrapper=True) + +@hookimpl(hookwrapper=True) def pytest_runtest_protocol(item): if isinstance(item, TestCaseFunction) and \ 'twisted.trial.unittest' in sys.modules: ut = sys.modules['twisted.python.failure'] Failure__init__ = ut.Failure.__init__ check_testcase_implements_trial_reporter() + def excstore(self, exc_value=None, exc_type=None, exc_tb=None, - captureVars=None): + captureVars=None): if exc_value is None: self._rawexcinfo = sys.exc_info() else: @@ -186,9 +217,10 @@ def excstore(self, exc_value=None, exc_type=None, exc_tb=None, self._rawexcinfo = (exc_type, exc_value, exc_tb) try: Failure__init__(self, exc_value, exc_type, exc_tb, - captureVars=captureVars) + captureVars=captureVars) except TypeError: Failure__init__(self, exc_value, exc_type, exc_tb) + ut.Failure.__init__ = excstore yield ut.Failure.__init__ = Failure__init__ diff --git a/_pytest/vendored_packages/README.md b/_pytest/vendored_packages/README.md deleted file mode 100644 index eab7c714fb01e3a..000000000000000 --- a/_pytest/vendored_packages/README.md +++ /dev/null @@ -1,13 +0,0 @@ -This directory vendors the `pluggy` module. - -For a more detailed discussion for the reasons to vendoring this -package, please see [this issue](https://github.com/pytest-dev/pytest/issues/944). - -To update the current version, execute: - -``` -$ pip install -U pluggy== --no-compile --target=_pytest/vendored_packages -``` - -And commit the modified files. The `pluggy-.dist-info` directory -created by `pip` should be ignored. diff --git a/_pytest/vendored_packages/__init__.py b/_pytest/vendored_packages/__init__.py deleted file mode 100644 index e69de29bb2d1d64..000000000000000 diff --git a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/DESCRIPTION.rst b/_pytest/vendored_packages/pluggy-0.3.1.dist-info/DESCRIPTION.rst deleted file mode 100644 index aa3bbf812975588..000000000000000 --- a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/DESCRIPTION.rst +++ /dev/null @@ -1,10 +0,0 @@ -Plugin registration and hook calling for Python -=============================================== - -This is the plugin manager as used by pytest but stripped -of pytest specific details. - -During the 0.x series this plugin does not have much documentation -except extensive docstrings in the pluggy.py module. - - diff --git a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/METADATA b/_pytest/vendored_packages/pluggy-0.3.1.dist-info/METADATA deleted file mode 100644 index ec81f0a6be0ae28..000000000000000 --- a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/METADATA +++ /dev/null @@ -1,39 +0,0 @@ -Metadata-Version: 2.0 -Name: pluggy -Version: 0.3.1 -Summary: plugin and hook calling mechanisms for python -Home-page: UNKNOWN -Author: Holger Krekel -Author-email: holger at merlinux.eu -License: MIT license -Platform: unix -Platform: linux -Platform: osx -Platform: win32 -Classifier: Development Status :: 4 - Beta -Classifier: Intended Audience :: Developers -Classifier: License :: OSI Approved :: MIT License -Classifier: Operating System :: POSIX -Classifier: Operating System :: Microsoft :: Windows -Classifier: Operating System :: MacOS :: MacOS X -Classifier: Topic :: Software Development :: Testing -Classifier: Topic :: Software Development :: Libraries -Classifier: Topic :: Utilities -Classifier: Programming Language :: Python :: 2 -Classifier: Programming Language :: Python :: 2.6 -Classifier: Programming Language :: Python :: 2.7 -Classifier: Programming Language :: Python :: 3 -Classifier: Programming Language :: Python :: 3.3 -Classifier: Programming Language :: Python :: 3.4 -Classifier: Programming Language :: Python :: 3.5 - -Plugin registration and hook calling for Python -=============================================== - -This is the plugin manager as used by pytest but stripped -of pytest specific details. - -During the 0.x series this plugin does not have much documentation -except extensive docstrings in the pluggy.py module. - - diff --git a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/RECORD b/_pytest/vendored_packages/pluggy-0.3.1.dist-info/RECORD deleted file mode 100644 index 9626673c43ce274..000000000000000 --- a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/RECORD +++ /dev/null @@ -1,8 +0,0 @@ -pluggy.py,sha256=v_RfWzyW6DPU1cJu_EFoL_OHq3t13qloVdR6UaMCXQA,29862 -pluggy-0.3.1.dist-info/top_level.txt,sha256=xKSCRhai-v9MckvMuWqNz16c1tbsmOggoMSwTgcpYHE,7 -pluggy-0.3.1.dist-info/pbr.json,sha256=xX3s6__wOcAyF-AZJX1sdZyW6PUXT-FkfBlM69EEUCg,47 -pluggy-0.3.1.dist-info/RECORD,, -pluggy-0.3.1.dist-info/metadata.json,sha256=nLKltOT78dMV-00uXD6Aeemp4xNsz2q59j6ORSDeLjw,1027 -pluggy-0.3.1.dist-info/METADATA,sha256=1b85Ho2u4iK30M099k7axMzcDDhLcIMb-A82JUJZnSo,1334 -pluggy-0.3.1.dist-info/WHEEL,sha256=AvR0WeTpDaxT645bl5FQxUK6NPsTls2ttpcGJg3j1Xg,110 -pluggy-0.3.1.dist-info/DESCRIPTION.rst,sha256=P5Akh1EdIBR6CeqtV2P8ZwpGSpZiTKPw0NyS7jEiD-g,306 diff --git a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/WHEEL b/_pytest/vendored_packages/pluggy-0.3.1.dist-info/WHEEL deleted file mode 100644 index 9dff69d86102d36..000000000000000 --- a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/WHEEL +++ /dev/null @@ -1,6 +0,0 @@ -Wheel-Version: 1.0 -Generator: bdist_wheel (0.24.0) -Root-Is-Purelib: true -Tag: py2-none-any -Tag: py3-none-any - diff --git a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/metadata.json b/_pytest/vendored_packages/pluggy-0.3.1.dist-info/metadata.json deleted file mode 100644 index 426a3a7ade14808..000000000000000 --- a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/metadata.json +++ /dev/null @@ -1 +0,0 @@ -{"license": "MIT license", "name": "pluggy", "metadata_version": "2.0", "generator": "bdist_wheel (0.24.0)", "summary": "plugin and hook calling mechanisms for python", "platform": "unix", "version": "0.3.1", "extensions": {"python.details": {"document_names": {"description": "DESCRIPTION.rst"}, "contacts": [{"role": "author", "email": "holger at merlinux.eu", "name": "Holger Krekel"}]}}, "classifiers": ["Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: POSIX", "Operating System :: Microsoft :: Windows", "Operating System :: MacOS :: MacOS X", "Topic :: Software Development :: Testing", "Topic :: Software Development :: Libraries", "Topic :: Utilities", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5"]} \ No newline at end of file diff --git a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/pbr.json b/_pytest/vendored_packages/pluggy-0.3.1.dist-info/pbr.json deleted file mode 100644 index d6b798640195967..000000000000000 --- a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/pbr.json +++ /dev/null @@ -1 +0,0 @@ -{"is_release": false, "git_version": "7d4c9cd"} \ No newline at end of file diff --git a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/top_level.txt b/_pytest/vendored_packages/pluggy-0.3.1.dist-info/top_level.txt deleted file mode 100644 index 11bdb5c1f5fcdd9..000000000000000 --- a/_pytest/vendored_packages/pluggy-0.3.1.dist-info/top_level.txt +++ /dev/null @@ -1 +0,0 @@ -pluggy diff --git a/_pytest/vendored_packages/pluggy.py b/_pytest/vendored_packages/pluggy.py deleted file mode 100644 index 2f848b23d35ed9b..000000000000000 --- a/_pytest/vendored_packages/pluggy.py +++ /dev/null @@ -1,777 +0,0 @@ -""" -PluginManager, basic initialization and tracing. - -pluggy is the cristallized core of plugin management as used -by some 150 plugins for pytest. - -Pluggy uses semantic versioning. Breaking changes are only foreseen for -Major releases (incremented X in "X.Y.Z"). If you want to use pluggy in -your project you should thus use a dependency restriction like -"pluggy>=0.1.0,<1.0" to avoid surprises. - -pluggy is concerned with hook specification, hook implementations and hook -calling. For any given hook specification a hook call invokes up to N implementations. -A hook implementation can influence its position and type of execution: -if attributed "tryfirst" or "trylast" it will be tried to execute -first or last. However, if attributed "hookwrapper" an implementation -can wrap all calls to non-hookwrapper implementations. A hookwrapper -can thus execute some code ahead and after the execution of other hooks. - -Hook specification is done by way of a regular python function where -both the function name and the names of all its arguments are significant. -Each hook implementation function is verified against the original specification -function, including the names of all its arguments. To allow for hook specifications -to evolve over the livetime of a project, hook implementations can -accept less arguments. One can thus add new arguments and semantics to -a hook specification by adding another argument typically without breaking -existing hook implementations. - -The chosen approach is meant to let a hook designer think carefuly about -which objects are needed by an extension writer. By contrast, subclass-based -extension mechanisms often expose a lot more state and behaviour than needed, -thus restricting future developments. - -Pluggy currently consists of functionality for: - -- a way to register new hook specifications. Without a hook - specification no hook calling can be performed. - -- a registry of plugins which contain hook implementation functions. It - is possible to register plugins for which a hook specification is not yet - known and validate all hooks when the system is in a more referentially - consistent state. Setting an "optionalhook" attribution to a hook - implementation will avoid PluginValidationError's if a specification - is missing. This allows to have optional integration between plugins. - -- a "hook" relay object from which you can launch 1:N calls to - registered hook implementation functions - -- a mechanism for ordering hook implementation functions - -- mechanisms for two different type of 1:N calls: "firstresult" for when - the call should stop when the first implementation returns a non-None result. - And the other (default) way of guaranteeing that all hook implementations - will be called and their non-None result collected. - -- mechanisms for "historic" extension points such that all newly - registered functions will receive all hook calls that happened - before their registration. - -- a mechanism for discovering plugin objects which are based on - setuptools based entry points. - -- a simple tracing mechanism, including tracing of plugin calls and - their arguments. - -""" -import sys -import inspect - -__version__ = '0.3.1' -__all__ = ["PluginManager", "PluginValidationError", - "HookspecMarker", "HookimplMarker"] - -_py3 = sys.version_info > (3, 0) - - -class HookspecMarker: - """ Decorator helper class for marking functions as hook specifications. - - You can instantiate it with a project_name to get a decorator. - Calling PluginManager.add_hookspecs later will discover all marked functions - if the PluginManager uses the same project_name. - """ - - def __init__(self, project_name): - self.project_name = project_name - - def __call__(self, function=None, firstresult=False, historic=False): - """ if passed a function, directly sets attributes on the function - which will make it discoverable to add_hookspecs(). If passed no - function, returns a decorator which can be applied to a function - later using the attributes supplied. - - If firstresult is True the 1:N hook call (N being the number of registered - hook implementation functions) will stop at I<=N when the I'th function - returns a non-None result. - - If historic is True calls to a hook will be memorized and replayed - on later registered plugins. - - """ - def setattr_hookspec_opts(func): - if historic and firstresult: - raise ValueError("cannot have a historic firstresult hook") - setattr(func, self.project_name + "_spec", - dict(firstresult=firstresult, historic=historic)) - return func - - if function is not None: - return setattr_hookspec_opts(function) - else: - return setattr_hookspec_opts - - -class HookimplMarker: - """ Decorator helper class for marking functions as hook implementations. - - You can instantiate with a project_name to get a decorator. - Calling PluginManager.register later will discover all marked functions - if the PluginManager uses the same project_name. - """ - def __init__(self, project_name): - self.project_name = project_name - - def __call__(self, function=None, hookwrapper=False, optionalhook=False, - tryfirst=False, trylast=False): - - """ if passed a function, directly sets attributes on the function - which will make it discoverable to register(). If passed no function, - returns a decorator which can be applied to a function later using - the attributes supplied. - - If optionalhook is True a missing matching hook specification will not result - in an error (by default it is an error if no matching spec is found). - - If tryfirst is True this hook implementation will run as early as possible - in the chain of N hook implementations for a specfication. - - If trylast is True this hook implementation will run as late as possible - in the chain of N hook implementations. - - If hookwrapper is True the hook implementations needs to execute exactly - one "yield". The code before the yield is run early before any non-hookwrapper - function is run. The code after the yield is run after all non-hookwrapper - function have run. The yield receives an ``_CallOutcome`` object representing - the exception or result outcome of the inner calls (including other hookwrapper - calls). - - """ - def setattr_hookimpl_opts(func): - setattr(func, self.project_name + "_impl", - dict(hookwrapper=hookwrapper, optionalhook=optionalhook, - tryfirst=tryfirst, trylast=trylast)) - return func - - if function is None: - return setattr_hookimpl_opts - else: - return setattr_hookimpl_opts(function) - - -def normalize_hookimpl_opts(opts): - opts.setdefault("tryfirst", False) - opts.setdefault("trylast", False) - opts.setdefault("hookwrapper", False) - opts.setdefault("optionalhook", False) - - -class _TagTracer: - def __init__(self): - self._tag2proc = {} - self.writer = None - self.indent = 0 - - def get(self, name): - return _TagTracerSub(self, (name,)) - - def format_message(self, tags, args): - if isinstance(args[-1], dict): - extra = args[-1] - args = args[:-1] - else: - extra = {} - - content = " ".join(map(str, args)) - indent = " " * self.indent - - lines = [ - "%s%s [%s]\n" % (indent, content, ":".join(tags)) - ] - - for name, value in extra.items(): - lines.append("%s %s: %s\n" % (indent, name, value)) - return lines - - def processmessage(self, tags, args): - if self.writer is not None and args: - lines = self.format_message(tags, args) - self.writer(''.join(lines)) - try: - self._tag2proc[tags](tags, args) - except KeyError: - pass - - def setwriter(self, writer): - self.writer = writer - - def setprocessor(self, tags, processor): - if isinstance(tags, str): - tags = tuple(tags.split(":")) - else: - assert isinstance(tags, tuple) - self._tag2proc[tags] = processor - - -class _TagTracerSub: - def __init__(self, root, tags): - self.root = root - self.tags = tags - - def __call__(self, *args): - self.root.processmessage(self.tags, args) - - def setmyprocessor(self, processor): - self.root.setprocessor(self.tags, processor) - - def get(self, name): - return self.__class__(self.root, self.tags + (name,)) - - -def _raise_wrapfail(wrap_controller, msg): - co = wrap_controller.gi_code - raise RuntimeError("wrap_controller at %r %s:%d %s" % - (co.co_name, co.co_filename, co.co_firstlineno, msg)) - - -def _wrapped_call(wrap_controller, func): - """ Wrap calling to a function with a generator which needs to yield - exactly once. The yield point will trigger calling the wrapped function - and return its _CallOutcome to the yield point. The generator then needs - to finish (raise StopIteration) in order for the wrapped call to complete. - """ - try: - next(wrap_controller) # first yield - except StopIteration: - _raise_wrapfail(wrap_controller, "did not yield") - call_outcome = _CallOutcome(func) - try: - wrap_controller.send(call_outcome) - _raise_wrapfail(wrap_controller, "has second yield") - except StopIteration: - pass - return call_outcome.get_result() - - -class _CallOutcome: - """ Outcome of a function call, either an exception or a proper result. - Calling the ``get_result`` method will return the result or reraise - the exception raised when the function was called. """ - excinfo = None - - def __init__(self, func): - try: - self.result = func() - except BaseException: - self.excinfo = sys.exc_info() - - def force_result(self, result): - self.result = result - self.excinfo = None - - def get_result(self): - if self.excinfo is None: - return self.result - else: - ex = self.excinfo - if _py3: - raise ex[1].with_traceback(ex[2]) - _reraise(*ex) # noqa - -if not _py3: - exec(""" -def _reraise(cls, val, tb): - raise cls, val, tb -""") - - -class _TracedHookExecution: - def __init__(self, pluginmanager, before, after): - self.pluginmanager = pluginmanager - self.before = before - self.after = after - self.oldcall = pluginmanager._inner_hookexec - assert not isinstance(self.oldcall, _TracedHookExecution) - self.pluginmanager._inner_hookexec = self - - def __call__(self, hook, hook_impls, kwargs): - self.before(hook.name, hook_impls, kwargs) - outcome = _CallOutcome(lambda: self.oldcall(hook, hook_impls, kwargs)) - self.after(outcome, hook.name, hook_impls, kwargs) - return outcome.get_result() - - def undo(self): - self.pluginmanager._inner_hookexec = self.oldcall - - -class PluginManager(object): - """ Core Pluginmanager class which manages registration - of plugin objects and 1:N hook calling. - - You can register new hooks by calling ``addhooks(module_or_class)``. - You can register plugin objects (which contain hooks) by calling - ``register(plugin)``. The Pluginmanager is initialized with a - prefix that is searched for in the names of the dict of registered - plugin objects. An optional excludefunc allows to blacklist names which - are not considered as hooks despite a matching prefix. - - For debugging purposes you can call ``enable_tracing()`` - which will subsequently send debug information to the trace helper. - """ - - def __init__(self, project_name, implprefix=None): - """ if implprefix is given implementation functions - will be recognized if their name matches the implprefix. """ - self.project_name = project_name - self._name2plugin = {} - self._plugin2hookcallers = {} - self._plugin_distinfo = [] - self.trace = _TagTracer().get("pluginmanage") - self.hook = _HookRelay(self.trace.root.get("hook")) - self._implprefix = implprefix - self._inner_hookexec = lambda hook, methods, kwargs: \ - _MultiCall(methods, kwargs, hook.spec_opts).execute() - - def _hookexec(self, hook, methods, kwargs): - # called from all hookcaller instances. - # enable_tracing will set its own wrapping function at self._inner_hookexec - return self._inner_hookexec(hook, methods, kwargs) - - def register(self, plugin, name=None): - """ Register a plugin and return its canonical name or None if the name - is blocked from registering. Raise a ValueError if the plugin is already - registered. """ - plugin_name = name or self.get_canonical_name(plugin) - - if plugin_name in self._name2plugin or plugin in self._plugin2hookcallers: - if self._name2plugin.get(plugin_name, -1) is None: - return # blocked plugin, return None to indicate no registration - raise ValueError("Plugin already registered: %s=%s\n%s" % - (plugin_name, plugin, self._name2plugin)) - - # XXX if an error happens we should make sure no state has been - # changed at point of return - self._name2plugin[plugin_name] = plugin - - # register matching hook implementations of the plugin - self._plugin2hookcallers[plugin] = hookcallers = [] - for name in dir(plugin): - hookimpl_opts = self.parse_hookimpl_opts(plugin, name) - if hookimpl_opts is not None: - normalize_hookimpl_opts(hookimpl_opts) - method = getattr(plugin, name) - hookimpl = HookImpl(plugin, plugin_name, method, hookimpl_opts) - hook = getattr(self.hook, name, None) - if hook is None: - hook = _HookCaller(name, self._hookexec) - setattr(self.hook, name, hook) - elif hook.has_spec(): - self._verify_hook(hook, hookimpl) - hook._maybe_apply_history(hookimpl) - hook._add_hookimpl(hookimpl) - hookcallers.append(hook) - return plugin_name - - def parse_hookimpl_opts(self, plugin, name): - method = getattr(plugin, name) - res = getattr(method, self.project_name + "_impl", None) - if res is not None and not isinstance(res, dict): - # false positive - res = None - elif res is None and self._implprefix and name.startswith(self._implprefix): - res = {} - return res - - def unregister(self, plugin=None, name=None): - """ unregister a plugin object and all its contained hook implementations - from internal data structures. """ - if name is None: - assert plugin is not None, "one of name or plugin needs to be specified" - name = self.get_name(plugin) - - if plugin is None: - plugin = self.get_plugin(name) - - # if self._name2plugin[name] == None registration was blocked: ignore - if self._name2plugin.get(name): - del self._name2plugin[name] - - for hookcaller in self._plugin2hookcallers.pop(plugin, []): - hookcaller._remove_plugin(plugin) - - return plugin - - def set_blocked(self, name): - """ block registrations of the given name, unregister if already registered. """ - self.unregister(name=name) - self._name2plugin[name] = None - - def is_blocked(self, name): - """ return True if the name blogs registering plugins of that name. """ - return name in self._name2plugin and self._name2plugin[name] is None - - def add_hookspecs(self, module_or_class): - """ add new hook specifications defined in the given module_or_class. - Functions are recognized if they have been decorated accordingly. """ - names = [] - for name in dir(module_or_class): - spec_opts = self.parse_hookspec_opts(module_or_class, name) - if spec_opts is not None: - hc = getattr(self.hook, name, None) - if hc is None: - hc = _HookCaller(name, self._hookexec, module_or_class, spec_opts) - setattr(self.hook, name, hc) - else: - # plugins registered this hook without knowing the spec - hc.set_specification(module_or_class, spec_opts) - for hookfunction in (hc._wrappers + hc._nonwrappers): - self._verify_hook(hc, hookfunction) - names.append(name) - - if not names: - raise ValueError("did not find any %r hooks in %r" % - (self.project_name, module_or_class)) - - def parse_hookspec_opts(self, module_or_class, name): - method = getattr(module_or_class, name) - return getattr(method, self.project_name + "_spec", None) - - def get_plugins(self): - """ return the set of registered plugins. """ - return set(self._plugin2hookcallers) - - def is_registered(self, plugin): - """ Return True if the plugin is already registered. """ - return plugin in self._plugin2hookcallers - - def get_canonical_name(self, plugin): - """ Return canonical name for a plugin object. Note that a plugin - may be registered under a different name which was specified - by the caller of register(plugin, name). To obtain the name - of an registered plugin use ``get_name(plugin)`` instead.""" - return getattr(plugin, "__name__", None) or str(id(plugin)) - - def get_plugin(self, name): - """ Return a plugin or None for the given name. """ - return self._name2plugin.get(name) - - def get_name(self, plugin): - """ Return name for registered plugin or None if not registered. """ - for name, val in self._name2plugin.items(): - if plugin == val: - return name - - def _verify_hook(self, hook, hookimpl): - if hook.is_historic() and hookimpl.hookwrapper: - raise PluginValidationError( - "Plugin %r\nhook %r\nhistoric incompatible to hookwrapper" % - (hookimpl.plugin_name, hook.name)) - - for arg in hookimpl.argnames: - if arg not in hook.argnames: - raise PluginValidationError( - "Plugin %r\nhook %r\nargument %r not available\n" - "plugin definition: %s\n" - "available hookargs: %s" % - (hookimpl.plugin_name, hook.name, arg, - _formatdef(hookimpl.function), ", ".join(hook.argnames))) - - def check_pending(self): - """ Verify that all hooks which have not been verified against - a hook specification are optional, otherwise raise PluginValidationError""" - for name in self.hook.__dict__: - if name[0] != "_": - hook = getattr(self.hook, name) - if not hook.has_spec(): - for hookimpl in (hook._wrappers + hook._nonwrappers): - if not hookimpl.optionalhook: - raise PluginValidationError( - "unknown hook %r in plugin %r" % - (name, hookimpl.plugin)) - - def load_setuptools_entrypoints(self, entrypoint_name): - """ Load modules from querying the specified setuptools entrypoint name. - Return the number of loaded plugins. """ - from pkg_resources import iter_entry_points, DistributionNotFound - for ep in iter_entry_points(entrypoint_name): - # is the plugin registered or blocked? - if self.get_plugin(ep.name) or self.is_blocked(ep.name): - continue - try: - plugin = ep.load() - except DistributionNotFound: - continue - self.register(plugin, name=ep.name) - self._plugin_distinfo.append((plugin, ep.dist)) - return len(self._plugin_distinfo) - - def list_plugin_distinfo(self): - """ return list of distinfo/plugin tuples for all setuptools registered - plugins. """ - return list(self._plugin_distinfo) - - def list_name_plugin(self): - """ return list of name/plugin pairs. """ - return list(self._name2plugin.items()) - - def get_hookcallers(self, plugin): - """ get all hook callers for the specified plugin. """ - return self._plugin2hookcallers.get(plugin) - - def add_hookcall_monitoring(self, before, after): - """ add before/after tracing functions for all hooks - and return an undo function which, when called, - will remove the added tracers. - - ``before(hook_name, hook_impls, kwargs)`` will be called ahead - of all hook calls and receive a hookcaller instance, a list - of HookImpl instances and the keyword arguments for the hook call. - - ``after(outcome, hook_name, hook_impls, kwargs)`` receives the - same arguments as ``before`` but also a :py:class:`_CallOutcome`` object - which represents the result of the overall hook call. - """ - return _TracedHookExecution(self, before, after).undo - - def enable_tracing(self): - """ enable tracing of hook calls and return an undo function. """ - hooktrace = self.hook._trace - - def before(hook_name, methods, kwargs): - hooktrace.root.indent += 1 - hooktrace(hook_name, kwargs) - - def after(outcome, hook_name, methods, kwargs): - if outcome.excinfo is None: - hooktrace("finish", hook_name, "-->", outcome.result) - hooktrace.root.indent -= 1 - - return self.add_hookcall_monitoring(before, after) - - def subset_hook_caller(self, name, remove_plugins): - """ Return a new _HookCaller instance for the named method - which manages calls to all registered plugins except the - ones from remove_plugins. """ - orig = getattr(self.hook, name) - plugins_to_remove = [plug for plug in remove_plugins if hasattr(plug, name)] - if plugins_to_remove: - hc = _HookCaller(orig.name, orig._hookexec, orig._specmodule_or_class, - orig.spec_opts) - for hookimpl in (orig._wrappers + orig._nonwrappers): - plugin = hookimpl.plugin - if plugin not in plugins_to_remove: - hc._add_hookimpl(hookimpl) - # we also keep track of this hook caller so it - # gets properly removed on plugin unregistration - self._plugin2hookcallers.setdefault(plugin, []).append(hc) - return hc - return orig - - -class _MultiCall: - """ execute a call into multiple python functions/methods. """ - - # XXX note that the __multicall__ argument is supported only - # for pytest compatibility reasons. It was never officially - # supported there and is explicitly deprecated since 2.8 - # so we can remove it soon, allowing to avoid the below recursion - # in execute() and simplify/speed up the execute loop. - - def __init__(self, hook_impls, kwargs, specopts={}): - self.hook_impls = hook_impls - self.kwargs = kwargs - self.kwargs["__multicall__"] = self - self.specopts = specopts - - def execute(self): - all_kwargs = self.kwargs - self.results = results = [] - firstresult = self.specopts.get("firstresult") - - while self.hook_impls: - hook_impl = self.hook_impls.pop() - args = [all_kwargs[argname] for argname in hook_impl.argnames] - if hook_impl.hookwrapper: - return _wrapped_call(hook_impl.function(*args), self.execute) - res = hook_impl.function(*args) - if res is not None: - if firstresult: - return res - results.append(res) - - if not firstresult: - return results - - def __repr__(self): - status = "%d meths" % (len(self.hook_impls),) - if hasattr(self, "results"): - status = ("%d results, " % len(self.results)) + status - return "<_MultiCall %s, kwargs=%r>" % (status, self.kwargs) - - -def varnames(func, startindex=None): - """ return argument name tuple for a function, method, class or callable. - - In case of a class, its "__init__" method is considered. - For methods the "self" parameter is not included unless you are passing - an unbound method with Python3 (which has no supports for unbound methods) - """ - cache = getattr(func, "__dict__", {}) - try: - return cache["_varnames"] - except KeyError: - pass - if inspect.isclass(func): - try: - func = func.__init__ - except AttributeError: - return () - startindex = 1 - else: - if not inspect.isfunction(func) and not inspect.ismethod(func): - func = getattr(func, '__call__', func) - if startindex is None: - startindex = int(inspect.ismethod(func)) - - try: - rawcode = func.__code__ - except AttributeError: - return () - try: - x = rawcode.co_varnames[startindex:rawcode.co_argcount] - except AttributeError: - x = () - else: - defaults = func.__defaults__ - if defaults: - x = x[:-len(defaults)] - try: - cache["_varnames"] = x - except TypeError: - pass - return x - - -class _HookRelay: - """ hook holder object for performing 1:N hook calls where N is the number - of registered plugins. - - """ - - def __init__(self, trace): - self._trace = trace - - -class _HookCaller(object): - def __init__(self, name, hook_execute, specmodule_or_class=None, spec_opts=None): - self.name = name - self._wrappers = [] - self._nonwrappers = [] - self._hookexec = hook_execute - if specmodule_or_class is not None: - assert spec_opts is not None - self.set_specification(specmodule_or_class, spec_opts) - - def has_spec(self): - return hasattr(self, "_specmodule_or_class") - - def set_specification(self, specmodule_or_class, spec_opts): - assert not self.has_spec() - self._specmodule_or_class = specmodule_or_class - specfunc = getattr(specmodule_or_class, self.name) - argnames = varnames(specfunc, startindex=inspect.isclass(specmodule_or_class)) - assert "self" not in argnames # sanity check - self.argnames = ["__multicall__"] + list(argnames) - self.spec_opts = spec_opts - if spec_opts.get("historic"): - self._call_history = [] - - def is_historic(self): - return hasattr(self, "_call_history") - - def _remove_plugin(self, plugin): - def remove(wrappers): - for i, method in enumerate(wrappers): - if method.plugin == plugin: - del wrappers[i] - return True - if remove(self._wrappers) is None: - if remove(self._nonwrappers) is None: - raise ValueError("plugin %r not found" % (plugin,)) - - def _add_hookimpl(self, hookimpl): - if hookimpl.hookwrapper: - methods = self._wrappers - else: - methods = self._nonwrappers - - if hookimpl.trylast: - methods.insert(0, hookimpl) - elif hookimpl.tryfirst: - methods.append(hookimpl) - else: - # find last non-tryfirst method - i = len(methods) - 1 - while i >= 0 and methods[i].tryfirst: - i -= 1 - methods.insert(i + 1, hookimpl) - - def __repr__(self): - return "<_HookCaller %r>" % (self.name,) - - def __call__(self, **kwargs): - assert not self.is_historic() - return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs) - - def call_historic(self, proc=None, kwargs=None): - self._call_history.append((kwargs or {}, proc)) - # historizing hooks don't return results - self._hookexec(self, self._nonwrappers + self._wrappers, kwargs) - - def call_extra(self, methods, kwargs): - """ Call the hook with some additional temporarily participating - methods using the specified kwargs as call parameters. """ - old = list(self._nonwrappers), list(self._wrappers) - for method in methods: - opts = dict(hookwrapper=False, trylast=False, tryfirst=False) - hookimpl = HookImpl(None, "", method, opts) - self._add_hookimpl(hookimpl) - try: - return self(**kwargs) - finally: - self._nonwrappers, self._wrappers = old - - def _maybe_apply_history(self, method): - if self.is_historic(): - for kwargs, proc in self._call_history: - res = self._hookexec(self, [method], kwargs) - if res and proc is not None: - proc(res[0]) - - -class HookImpl: - def __init__(self, plugin, plugin_name, function, hook_impl_opts): - self.function = function - self.argnames = varnames(self.function) - self.plugin = plugin - self.opts = hook_impl_opts - self.plugin_name = plugin_name - self.__dict__.update(hook_impl_opts) - - -class PluginValidationError(Exception): - """ plugin failed validation. """ - - -if hasattr(inspect, 'signature'): - def _formatdef(func): - return "%s%s" % ( - func.__name__, - str(inspect.signature(func)) - ) -else: - def _formatdef(func): - return "%s%s" % ( - func.__name__, - inspect.formatargspec(*inspect.getargspec(func)) - ) diff --git a/_pytest/warnings.py b/_pytest/warnings.py new file mode 100644 index 000000000000000..3c2b1914fb69164 --- /dev/null +++ b/_pytest/warnings.py @@ -0,0 +1,96 @@ +from __future__ import absolute_import, division, print_function + +import warnings +from contextlib import contextmanager + +import pytest + +from _pytest import compat + + +def _setoption(wmod, arg): + """ + Copy of the warning._setoption function but does not escape arguments. + """ + parts = arg.split(':') + if len(parts) > 5: + raise wmod._OptionError("too many fields (max 5): %r" % (arg,)) + while len(parts) < 5: + parts.append('') + action, message, category, module, lineno = [s.strip() + for s in parts] + action = wmod._getaction(action) + category = wmod._getcategory(category) + if lineno: + try: + lineno = int(lineno) + if lineno < 0: + raise ValueError + except (ValueError, OverflowError): + raise wmod._OptionError("invalid lineno %r" % (lineno,)) + else: + lineno = 0 + wmod.filterwarnings(action, message, category, module, lineno) + + +def pytest_addoption(parser): + group = parser.getgroup("pytest-warnings") + group.addoption( + '-W', '--pythonwarnings', action='append', + help="set which warnings to report, see -W option of python itself.") + parser.addini("filterwarnings", type="linelist", + help="Each line specifies a pattern for " + "warnings.filterwarnings. " + "Processed after -W and --pythonwarnings.") + + +@contextmanager +def catch_warnings_for_item(item): + """ + catches the warnings generated during setup/call/teardown execution + of the given item and after it is done posts them as warnings to this + item. + """ + args = item.config.getoption('pythonwarnings') or [] + inifilters = item.config.getini("filterwarnings") + with warnings.catch_warnings(record=True) as log: + for arg in args: + warnings._setoption(arg) + + for arg in inifilters: + _setoption(warnings, arg) + + mark = item.get_marker('filterwarnings') + if mark: + for arg in mark.args: + warnings._setoption(arg) + + yield + + for warning in log: + warn_msg = warning.message + unicode_warning = False + + if compat._PY2 and any(isinstance(m, compat.UNICODE_TYPES) for m in warn_msg.args): + new_args = [] + for m in warn_msg.args: + new_args.append(compat.ascii_escaped(m) if isinstance(m, compat.UNICODE_TYPES) else m) + unicode_warning = list(warn_msg.args) != new_args + warn_msg.args = new_args + + msg = warnings.formatwarning( + warn_msg, warning.category, + warning.filename, warning.lineno, warning.line) + item.warn("unused", msg) + + if unicode_warning: + warnings.warn( + "Warning is using unicode non convertible to ascii, " + "converting to a safe representation:\n %s" % msg, + UnicodeWarning) + + +@pytest.hookimpl(hookwrapper=True) +def pytest_runtest_protocol(item): + with catch_warnings_for_item(item): + yield diff --git a/appveyor.yml b/appveyor.yml index 4b73645f735b61b..4f4afe15c38334e 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -5,24 +5,40 @@ environment: # using pytestbot account as detailed here: # https://www.appveyor.com/docs/build-configuration#secure-variables + matrix: + # coveralls is not in the default env list + - TOXENV: "coveralls" + # note: please use "tox --listenvs" to populate the build matrix below + - TOXENV: "linting" + - TOXENV: "py27" + - TOXENV: "py34" + - TOXENV: "py35" + - TOXENV: "py36" + - TOXENV: "pypy" + - TOXENV: "py27-pexpect" + - TOXENV: "py27-xdist" + - TOXENV: "py27-trial" + - TOXENV: "py27-numpy" + - TOXENV: "py27-pluggymaster" + - TOXENV: "py36-pexpect" + - TOXENV: "py36-xdist" + - TOXENV: "py36-trial" + - TOXENV: "py36-numpy" + - TOXENV: "py36-pluggymaster" + - TOXENV: "py27-nobyte" + - TOXENV: "doctesting" + - TOXENV: "py35-freeze" + - TOXENV: "docs" + install: - echo Installed Pythons - dir c:\Python* - # install pypy using choco (redirect to a file and write to console in case - # choco install returns non-zero, because choco install python.pypy is too - # noisy) - - choco install python.pypy > pypy-inst.log 2>&1 || (type pypy-inst.log & exit /b 1) - - set PATH=C:\tools\pypy\pypy;%PATH% # so tox can find pypy - - echo PyPy installed - - pypy --version + - if "%TOXENV%" == "pypy" call scripts\install-pypy.bat - - C:\Python35\python -m pip install tox + - C:\Python36\python -m pip install --upgrade --pre tox build: false # Not a C# project, build stuff at the test step instead. test_script: - - C:\Python35\python -m tox - # coveralls is not in tox's envlist, plus for PRs the secure variable - # is not defined so we have to check for it - - if defined COVERALLS_REPO_TOKEN C:\Python35\python -m tox -e coveralls + - call scripts\call-tox.bat diff --git a/changelog/2920.bugfix b/changelog/2920.bugfix new file mode 100644 index 000000000000000..9c5217278aed7e1 --- /dev/null +++ b/changelog/2920.bugfix @@ -0,0 +1 @@ +Fix issue about ``-p no:`` having no effect. diff --git a/changelog/2949.trivial b/changelog/2949.trivial new file mode 100644 index 000000000000000..39789e72b7a9ab4 --- /dev/null +++ b/changelog/2949.trivial @@ -0,0 +1 @@ +Update github "bugs" link in CONTRIBUTING.rst diff --git a/changelog/2956.bugfix b/changelog/2956.bugfix new file mode 100644 index 000000000000000..13717657bf16815 --- /dev/null +++ b/changelog/2956.bugfix @@ -0,0 +1 @@ +Fix regression with warnings that contained non-strings in their arguments in Python 2. diff --git a/changelog/2957.bugfix b/changelog/2957.bugfix new file mode 100644 index 000000000000000..589665b692ae73b --- /dev/null +++ b/changelog/2957.bugfix @@ -0,0 +1 @@ +Always escape null bytes when setting ``PYTEST_CURRENT_TEST``. diff --git a/changelog/2963.doc b/changelog/2963.doc new file mode 100644 index 000000000000000..c9a1d661b64183b --- /dev/null +++ b/changelog/2963.doc @@ -0,0 +1 @@ +Fix broken link to plugin pytest-localserver. diff --git a/changelog/2971.bugfix b/changelog/2971.bugfix new file mode 100644 index 000000000000000..36684e8c880334c --- /dev/null +++ b/changelog/2971.bugfix @@ -0,0 +1 @@ +Fix ``ZeroDivisionError`` when using the ``testmon`` plugin when no tests were actually collected. diff --git a/changelog/_template.rst b/changelog/_template.rst new file mode 100644 index 000000000000000..a898abc15af1712 --- /dev/null +++ b/changelog/_template.rst @@ -0,0 +1,40 @@ +{% for section in sections %} +{% set underline = "-" %} +{% if section %} +{{section}} +{{ underline * section|length }}{% set underline = "~" %} + +{% endif %} +{% if sections[section] %} +{% for category, val in definitions.items() if category in sections[section] %} + +{{ definitions[category]['name'] }} +{{ underline * definitions[category]['name']|length }} + +{% if definitions[category]['showcontent'] %} +{% for text, values in sections[section][category]|dictsort(by='value') %} +{% set issue_joiner = joiner(', ') %} +- {{ text }}{% if category != 'vendor' %} ({% for value in values|sort %}{{ issue_joiner() }}`{{ value }} `_{% endfor %}){% endif %} + + +{% endfor %} +{% else %} +- {{ sections[section][category]['']|sort|join(', ') }} + + +{% endif %} +{% if sections[section][category]|length == 0 %} + +No significant changes. + + +{% else %} +{% endif %} +{% endfor %} +{% else %} + +No significant changes. + + +{% endif %} +{% endfor %} diff --git a/doc/en/Makefile b/doc/en/Makefile index 8621f779c70e275..fa8e8266a29e71f 100644 --- a/doc/en/Makefile +++ b/doc/en/Makefile @@ -13,16 +13,18 @@ PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . REGENDOC_ARGS := \ - --normalize "/={8,} (.*) ={8,}/======= \1 ========/" \ - --normalize "/_{8,} (.*) _{8,}/_______ \1 ________/" \ --normalize "/in \d+.\d+ seconds/in 0.12 seconds/" \ --normalize "@/tmp/pytest-of-.*/pytest-\d+@PYTEST_TMPDIR@" \ - - + --normalize "@pytest-(\d+)\\.[^ ,]+@pytest-\1.x.y@" \ + --normalize "@(This is pytest version )(\d+)\\.[^ ,]+@\1\2.x.y@" \ + --normalize "@py-(\d+)\\.[^ ,]+@py-\1.x.y@" \ + --normalize "@pluggy-(\d+)\\.[.\d,]+@pluggy-\1.x.y@" \ + --normalize "@hypothesis-(\d+)\\.[.\d,]+@hypothesis-\1.x.y@" \ + --normalize "@Python (\d+)\\.[^ ,]+@Python \1.x.y@" .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest - + help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @@ -36,24 +38,8 @@ help: clean: -rm -rf $(BUILDDIR)/* -SITETARGET=$(shell ./_getdoctarget.py) - -showtarget: - @echo $(SITETARGET) - -install: html - # for access talk to someone with login rights to - # pytest-dev@pytest.org to add your ssh key - rsync -avz _build/html/ pytest-dev@pytest.org:pytest.org/$(SITETARGET) - -installpdf: latexpdf - @scp $(BUILDDIR)/latex/pytest.pdf pytest-dev@pytest.org:pytest.org/$(SITETARGET) - -installall: clean install installpdf - @echo "done" - regen: - PYTHONDONTWRITEBYTECODE=1 COLUMNS=76 regendoc --update *.rst */*.rst ${REGENDOC_ARGS} + PYTHONDONTWRITEBYTECODE=1 PYTEST_ADDOPT=-pno:hypothesis COLUMNS=76 regendoc --update *.rst */*.rst ${REGENDOC_ARGS} html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html diff --git a/doc/en/_getdoctarget.py b/doc/en/_getdoctarget.py deleted file mode 100755 index 20e487bb7386781..000000000000000 --- a/doc/en/_getdoctarget.py +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env python - -import py - -def get_version_string(): - fn = py.path.local(__file__).join("..", "..", "..", - "_pytest", "__init__.py") - for line in fn.readlines(): - if "version" in line and not line.strip().startswith('#'): - return eval(line.split("=")[-1]) - -def get_minor_version_string(): - return ".".join(get_version_string().split(".")[:2]) - -if __name__ == "__main__": - print (get_minor_version_string()) diff --git a/doc/en/_templates/globaltoc.html b/doc/en/_templates/globaltoc.html index af427198a626c5f..fdd4dd59b324bdb 100644 --- a/doc/en/_templates/globaltoc.html +++ b/doc/en/_templates/globaltoc.html @@ -9,6 +9,7 @@

{{ _('Table Of Contents') }}

  • Contact
  • Talks/Posts
  • Changelog
  • +
  • Backwards Compatibility
  • License
  • diff --git a/doc/en/_templates/layout.html b/doc/en/_templates/layout.html index 0ce480be300ee53..2fc8e2a7fb4c026 100644 --- a/doc/en/_templates/layout.html +++ b/doc/en/_templates/layout.html @@ -1,19 +1,5 @@ {% extends "!layout.html" %} {% block header %} -
    -

    - Want to help improve pytest? Please - - contribute to - - or - - join - - our upcoming sprint in June 2016! - -

    -
    {{super()}} {% endblock %} {% block footer %} diff --git a/doc/en/_templates/links.html b/doc/en/_templates/links.html index 200258e165ff27a..d855a013f3406f5 100644 --- a/doc/en/_templates/links.html +++ b/doc/en/_templates/links.html @@ -1,16 +1,11 @@

    Useful Links

    diff --git a/doc/en/adopt.rst b/doc/en/adopt.rst index aead96e7f3d3d5a..710f431be30bcdf 100644 --- a/doc/en/adopt.rst +++ b/doc/en/adopt.rst @@ -1,3 +1,7 @@ +:orphan: + +.. warnings about this file not being included in any toctree will be suppressed by :orphan: + April 2015 is "adopt pytest month" ============================================= diff --git a/doc/en/announce/index.rst b/doc/en/announce/index.rst index 877afff77775fa4..1a5f3760b68e054 100644 --- a/doc/en/announce/index.rst +++ b/doc/en/announce/index.rst @@ -5,8 +5,29 @@ Release announcements .. toctree:: :maxdepth: 2 - + + release-3.3.0 + release-3.2.5 + release-3.2.4 + release-3.2.3 + release-3.2.2 + release-3.2.1 + release-3.2.0 + release-3.1.3 + release-3.1.2 + release-3.1.1 + release-3.1.0 + release-3.0.7 + release-3.0.6 + release-3.0.5 + release-3.0.4 + release-3.0.3 + release-3.0.2 + release-3.0.1 + release-3.0.0 sprint2016 + release-2.9.2 + release-2.9.1 release-2.9.0 release-2.8.7 release-2.8.6 @@ -45,4 +66,3 @@ Release announcements release-2.0.2 release-2.0.1 release-2.0.0 - diff --git a/doc/en/announce/release-2.0.2.rst b/doc/en/announce/release-2.0.2.rst index 733a9f7bdd89eba..f1f44f34f4f7df2 100644 --- a/doc/en/announce/release-2.0.2.rst +++ b/doc/en/announce/release-2.0.2.rst @@ -63,9 +63,9 @@ Changes between 2.0.1 and 2.0.2 this. - fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular - thanks to Laura Creighton who also revieved parts of the documentation. + thanks to Laura Creighton who also reviewed parts of the documentation. -- fix slighly wrong output of verbose progress reporting for classes +- fix slightly wrong output of verbose progress reporting for classes (thanks Amaury) - more precise (avoiding of) deprecation warnings for node.Class|Function accesses diff --git a/doc/en/announce/release-2.0.3.rst b/doc/en/announce/release-2.0.3.rst index ed746e8519b73d0..9bbfdaab361aa69 100644 --- a/doc/en/announce/release-2.0.3.rst +++ b/doc/en/announce/release-2.0.3.rst @@ -13,7 +13,7 @@ If you want to install or upgrade pytest, just type one of:: easy_install -U pytest There also is a bugfix release 1.6 of pytest-xdist, the plugin -that enables seemless distributed and "looponfail" testing for Python. +that enables seamless distributed and "looponfail" testing for Python. best, holger krekel @@ -33,7 +33,7 @@ Changes between 2.0.2 and 2.0.3 - don't require zlib (and other libs) for genscript plugin without --genscript actually being used. -- speed up skips (by not doing a full traceback represenation +- speed up skips (by not doing a full traceback representation internally) - fix issue37: avoid invalid characters in junitxml's output diff --git a/doc/en/announce/release-2.2.1.rst b/doc/en/announce/release-2.2.1.rst index f9764634c72578a..5d28bcb01f4a675 100644 --- a/doc/en/announce/release-2.2.1.rst +++ b/doc/en/announce/release-2.2.1.rst @@ -2,7 +2,7 @@ pytest-2.2.1: bug fixes, perfect teardowns =========================================================================== -pytest-2.2.1 is a minor backward-compatible release of the the py.test +pytest-2.2.1 is a minor backward-compatible release of the py.test testing tool. It contains bug fixes and little improvements, including documentation fixes. If you are using the distributed testing pluginmake sure to upgrade it to pytest-xdist-1.8. diff --git a/doc/en/announce/release-2.2.4.rst b/doc/en/announce/release-2.2.4.rst index 8720bdb28a72552..67f0feb27c7fd02 100644 --- a/doc/en/announce/release-2.2.4.rst +++ b/doc/en/announce/release-2.2.4.rst @@ -29,7 +29,7 @@ Changes between 2.2.3 and 2.2.4 - fix issue with unittest: now @unittest.expectedFailure markers should be processed correctly (you can also use @pytest.mark markers) - document integration with the extended distribute/setuptools test commands -- fix issue 140: propperly get the real functions +- fix issue 140: properly get the real functions of bound classmethods for setup/teardown_class - fix issue #141: switch from the deceased paste.pocoo.org to bpaste.net - fix issue #143: call unconfigure/sessionfinish always when diff --git a/doc/en/announce/release-2.3.0.rst b/doc/en/announce/release-2.3.0.rst index 54fe3961fd83395..f863aad0ace945f 100644 --- a/doc/en/announce/release-2.3.0.rst +++ b/doc/en/announce/release-2.3.0.rst @@ -89,7 +89,7 @@ Changes between 2.2.4 and 2.3.0 - fix issue128: show captured output when capsys/capfd are used -- fix issue179: propperly show the dependency chain of factories +- fix issue179: properly show the dependency chain of factories - pluginmanager.register(...) now raises ValueError if the plugin has been already registered or the name is taken @@ -130,5 +130,5 @@ Changes between 2.2.4 and 2.3.0 - don't show deselected reason line if there is none - - py.test -vv will show all of assert comparisations instead of truncating + - py.test -vv will show all of assert comparisons instead of truncating diff --git a/doc/en/announce/release-2.3.2.rst b/doc/en/announce/release-2.3.2.rst index 948b374d43b5b73..75312b429cdd746 100644 --- a/doc/en/announce/release-2.3.2.rst +++ b/doc/en/announce/release-2.3.2.rst @@ -1,7 +1,7 @@ pytest-2.3.2: some fixes and more traceback-printing speed =========================================================================== -pytest-2.3.2 is a another stabilization release: +pytest-2.3.2 is another stabilization release: - issue 205: fixes a regression with conftest detection - issue 208/29: fixes traceback-printing speed in some bad cases diff --git a/doc/en/announce/release-2.3.3.rst b/doc/en/announce/release-2.3.3.rst index 1d7c7027bed545a..3a48b6ac4baf919 100644 --- a/doc/en/announce/release-2.3.3.rst +++ b/doc/en/announce/release-2.3.3.rst @@ -1,7 +1,7 @@ -pytest-2.3.3: integration fixes, py24 suport, ``*/**`` shown in traceback +pytest-2.3.3: integration fixes, py24 support, ``*/**`` shown in traceback =========================================================================== -pytest-2.3.3 is a another stabilization release of the py.test tool +pytest-2.3.3 is another stabilization release of the py.test tool which offers uebersimple assertions, scalable fixture mechanisms and deep customization for testing with Python. Particularly, this release provides: @@ -46,7 +46,7 @@ Changes between 2.3.2 and 2.3.3 - fix issue209 - reintroduce python2.4 support by depending on newer pylib which re-introduced statement-finding for pre-AST interpreters -- nose support: only call setup if its a callable, thanks Andrew +- nose support: only call setup if it's a callable, thanks Andrew Taumoefolau - fix issue219 - add py2.4-3.3 classifiers to TROVE list diff --git a/doc/en/announce/release-2.3.5.rst b/doc/en/announce/release-2.3.5.rst index c4e91e0e61429b5..112399ef3ca7e91 100644 --- a/doc/en/announce/release-2.3.5.rst +++ b/doc/en/announce/release-2.3.5.rst @@ -44,11 +44,11 @@ Changes between 2.3.4 and 2.3.5 (thanks Adam Goucher) - Issue 265 - integrate nose setup/teardown with setupstate - so it doesnt try to teardown if it did not setup + so it doesn't try to teardown if it did not setup -- issue 271 - dont write junitxml on slave nodes +- issue 271 - don't write junitxml on slave nodes -- Issue 274 - dont try to show full doctest example +- Issue 274 - don't try to show full doctest example when doctest does not know the example location - issue 280 - disable assertion rewriting on buggy CPython 2.6.0 @@ -84,7 +84,7 @@ Changes between 2.3.4 and 2.3.5 - allow to specify prefixes starting with "_" when customizing python_functions test discovery. (thanks Graham Horler) -- improve PYTEST_DEBUG tracing output by puting +- improve PYTEST_DEBUG tracing output by putting extra data on a new lines with additional indent - ensure OutcomeExceptions like skip/fail have initialized exception attributes diff --git a/doc/en/announce/release-2.4.0.rst b/doc/en/announce/release-2.4.0.rst index 88130c481f146b3..be3aaedb09ff7f6 100644 --- a/doc/en/announce/release-2.4.0.rst +++ b/doc/en/announce/release-2.4.0.rst @@ -36,7 +36,7 @@ a full list of details. A few feature highlights: - reporting: color the last line red or green depending if failures/errors occurred or everything passed. -The documentation has been updated to accomodate the changes, +The documentation has been updated to accommodate the changes, see `http://pytest.org `_ To install or upgrade pytest:: @@ -118,7 +118,7 @@ new features: - fix issue322: tearDownClass is not run if setUpClass failed. Thanks Mathieu Agopian for the initial fix. Also make all of pytest/nose - finalizer mimick the same generic behaviour: if a setupX exists and + finalizer mimic the same generic behaviour: if a setupX exists and fails, don't run teardownX. This internally introduces a new method "node.addfinalizer()" helper which can only be called during the setup phase of a node. diff --git a/doc/en/announce/release-2.5.0.rst b/doc/en/announce/release-2.5.0.rst index b8f28d6fd5ec285..b04a825cd8e4ff1 100644 --- a/doc/en/announce/release-2.5.0.rst +++ b/doc/en/announce/release-2.5.0.rst @@ -70,7 +70,7 @@ holger krekel to problems for more than >966 non-function scoped parameters). - fix issue290 - there is preliminary support now for parametrizing - with repeated same values (sometimes useful to to test if calling + with repeated same values (sometimes useful to test if calling a second time works as with the first time). - close issue240 - document precisely how pytest module importing @@ -149,7 +149,7 @@ holger krekel would not work correctly because pytest assumes @pytest.mark.some gets a function to be decorated already. We now at least detect if this - arg is an lambda and thus the example will work. Thanks Alex Gaynor + arg is a lambda and thus the example will work. Thanks Alex Gaynor for bringing it up. - xfail a test on pypy that checks wrong encoding/ascii (pypy does diff --git a/doc/en/announce/release-2.5.2.rst b/doc/en/announce/release-2.5.2.rst index 9308ffdd6261abd..d5cfca2dbdad61a 100644 --- a/doc/en/announce/release-2.5.2.rst +++ b/doc/en/announce/release-2.5.2.rst @@ -60,5 +60,5 @@ holger krekel - fix issue429: comparing byte strings with non-ascii chars in assert expressions now work better. Thanks Floris Bruynooghe. -- make capfd/capsys.capture private, its unused and shouldnt be exposed +- make capfd/capsys.capture private, its unused and shouldn't be exposed diff --git a/doc/en/announce/release-2.6.3.rst b/doc/en/announce/release-2.6.3.rst index 13fae31b8a16fe2..ee0d2692c4745c3 100644 --- a/doc/en/announce/release-2.6.3.rst +++ b/doc/en/announce/release-2.6.3.rst @@ -41,8 +41,8 @@ Changes 2.6.3 dep). Thanks Charles Cloud for analysing the issue. - fix conftest related fixture visibility issue: when running with a - CWD outside a test package pytest would get fixture discovery wrong. - Thanks to Wolfgang Schnerring for figuring out a reproducable example. + CWD outside of a test package pytest would get fixture discovery wrong. + Thanks to Wolfgang Schnerring for figuring out a reproducible example. - Introduce pytest_enter_pdb hook (needed e.g. by pytest_timeout to cancel the timeout when interactively entering pdb). Thanks Wolfgang Schnerring. diff --git a/doc/en/announce/release-2.7.0.rst b/doc/en/announce/release-2.7.0.rst index 07ae44ca1acc5d0..4e317ff8f3475aa 100644 --- a/doc/en/announce/release-2.7.0.rst +++ b/doc/en/announce/release-2.7.0.rst @@ -62,7 +62,7 @@ holger krekel - fix issue655: work around different ways that cause python2/3 to leak sys.exc_info into fixtures/tests causing failures in 3rd party code -- fix issue615: assertion re-writing did not correctly escape % signs +- fix issue615: assertion rewriting did not correctly escape % signs when formatting boolean operations, which tripped over mixing booleans with modulo operators. Thanks to Tom Viner for the report, triaging and fix. diff --git a/doc/en/announce/release-2.7.1.rst b/doc/en/announce/release-2.7.1.rst index cd37cad0cb89a34..fdc71eebba95f33 100644 --- a/doc/en/announce/release-2.7.1.rst +++ b/doc/en/announce/release-2.7.1.rst @@ -32,7 +32,7 @@ The py.test Development Team explanations. Thanks Carl Meyer for the report and test case. - fix issue553: properly handling inspect.getsourcelines failures in - FixtureLookupError which would lead to to an internal error, + FixtureLookupError which would lead to an internal error, obfuscating the original problem. Thanks talljosh for initial diagnose/patch and Bruno Oliveira for final patch. diff --git a/doc/en/announce/release-2.9.0.rst b/doc/en/announce/release-2.9.0.rst index 99c1c631f17776e..011b1ffb9d29d7e 100644 --- a/doc/en/announce/release-2.9.0.rst +++ b/doc/en/announce/release-2.9.0.rst @@ -75,7 +75,7 @@ The py.test Development Team **Changes** -* **Important**: `py.code `_ has been +* **Important**: `py.code `_ has been merged into the ``pytest`` repository as ``pytest._code``. This decision was made because ``py.code`` had very few uses outside ``pytest`` and the fact that it was in a different repository made it difficult to fix bugs on @@ -88,7 +88,7 @@ The py.test Development Team **experimental**, so you definitely should not import it explicitly! Please note that the original ``py.code`` is still available in - `pylib `_. + `pylib `_. * ``pytest_enter_pdb`` now optionally receives the pytest config object. Thanks `@nicoddemus`_ for the PR. diff --git a/doc/en/announce/release-2.9.1.rst b/doc/en/announce/release-2.9.1.rst index 05a44843042b2d9..3277da1e9b0f155 100644 --- a/doc/en/announce/release-2.9.1.rst +++ b/doc/en/announce/release-2.9.1.rst @@ -63,3 +63,5 @@ The py.test Development Team .. _#649: https://github.com/pytest-dev/pytest/issues/649 .. _@asottile: https://github.com/asottile +.. _@nicoddemus: https://github.com/nicoddemus +.. _@tomviner: https://github.com/tomviner diff --git a/doc/en/announce/release-2.9.2.rst b/doc/en/announce/release-2.9.2.rst new file mode 100644 index 000000000000000..8f274cdf3981f05 --- /dev/null +++ b/doc/en/announce/release-2.9.2.rst @@ -0,0 +1,78 @@ +pytest-2.9.2 +============ + +pytest is a mature Python testing tool with more than a 1100 tests +against itself, passing on many different interpreters and platforms. + +See below for the changes and see docs at: + + http://pytest.org + +As usual, you can upgrade from pypi via:: + + pip install -U pytest + +Thanks to all who contributed to this release, among them: + + Adam Chainz + Benjamin Dopplinger + Bruno Oliveira + Florian Bruhin + John Towler + Martin Prusse + Meng Jue + MengJueM + Omar Kohl + Quentin Pradet + Ronny Pfannschmidt + Thomas Güttler + TomV + Tyler Goodlet + + +Happy testing, +The py.test Development Team + + +2.9.2 (compared to 2.9.1) +--------------------------- + +**Bug Fixes** + +* fix `#510`_: skip tests where one parameterize dimension was empty + thanks Alex Stapleton for the Report and `@RonnyPfannschmidt`_ for the PR + +* Fix Xfail does not work with condition keyword argument. + Thanks `@astraw38`_ for reporting the issue (`#1496`_) and `@tomviner`_ + for PR the (`#1524`_). + +* Fix win32 path issue when putting custom config file with absolute path + in ``pytest.main("-c your_absolute_path")``. + +* Fix maximum recursion depth detection when raised error class is not aware + of unicode/encoded bytes. + Thanks `@prusse-martin`_ for the PR (`#1506`_). + +* Fix ``pytest.mark.skip`` mark when used in strict mode. + Thanks `@pquentin`_ for the PR and `@RonnyPfannschmidt`_ for + showing how to fix the bug. + +* Minor improvements and fixes to the documentation. + Thanks `@omarkohl`_ for the PR. + +* Fix ``--fixtures`` to show all fixture definitions as opposed to just + one per fixture name. + Thanks to `@hackebrot`_ for the PR. + +.. _#510: https://github.com/pytest-dev/pytest/issues/510 +.. _#1506: https://github.com/pytest-dev/pytest/pull/1506 +.. _#1496: https://github.com/pytest-dev/pytest/issue/1496 +.. _#1524: https://github.com/pytest-dev/pytest/issue/1524 + +.. _@astraw38: https://github.com/astraw38 +.. _@hackebrot: https://github.com/hackebrot +.. _@omarkohl: https://github.com/omarkohl +.. _@pquentin: https://github.com/pquentin +.. _@prusse-martin: https://github.com/prusse-martin +.. _@RonnyPfannschmidt: https://github.com/RonnyPfannschmidt +.. _@tomviner: https://github.com/tomviner diff --git a/doc/en/announce/release-3.0.0.rst b/doc/en/announce/release-3.0.0.rst new file mode 100644 index 000000000000000..4bf1e8534ec385f --- /dev/null +++ b/doc/en/announce/release-3.0.0.rst @@ -0,0 +1,82 @@ +pytest-3.0.0 +============ + +The pytest team is proud to announce the 3.0.0 release! + +pytest is a mature Python testing tool with more than a 1600 tests +against itself, passing on many different interpreters and platforms. + +This release contains a lot of bugs fixes and improvements, and much of +the work done on it was possible because of the 2016 Sprint[1], which +was funded by an indiegogo campaign which raised over US$12,000 with +nearly 100 backers. + +There's a "What's new in pytest 3.0" [2] blog post highlighting the +major features in this release. + +To see the complete changelog and documentation, please visit: + + http://docs.pytest.org + +As usual, you can upgrade from pypi via: + + pip install -U pytest + +Thanks to all who contributed to this release, among them: + + AbdealiJK + Ana Ribeiro + Antony Lee + Brandon W Maister + Brianna Laugher + Bruno Oliveira + Ceridwen + Christian Boelsen + Daniel Hahler + Danielle Jenkins + Dave Hunt + Diego Russo + Dmitry Dygalo + Edoardo Batini + Eli Boyarski + Florian Bruhin + Floris Bruynooghe + Greg Price + Guyzmo + HEAD KANGAROO + JJ + Javi Romero + Javier Domingo Cansino + Kale Kundert + Kalle Bronsen + Marius Gedminas + Matt Williams + Mike Lundy + Oliver Bestwalter + Omar Kohl + Raphael Pierzina + RedBeardCode + Roberto Polli + Romain Dorgueil + Roman Bolshakov + Ronny Pfannschmidt + Stefan Zimmermann + Steffen Allner + Tareq Alayan + Ted Xiao + Thomas Grainger + Tom Viner + TomV + Vasily Kuznetsov + aostr + marscher + palaviv + satoru + taschini + + +Happy testing, +The Pytest Development Team + +[1] http://blog.pytest.org/2016/pytest-development-sprint/ +[2] http://blog.pytest.org/2016/whats-new-in-pytest-30/ diff --git a/doc/en/announce/release-3.0.1.rst b/doc/en/announce/release-3.0.1.rst new file mode 100644 index 000000000000000..9fb38047b9cae67 --- /dev/null +++ b/doc/en/announce/release-3.0.1.rst @@ -0,0 +1,26 @@ +pytest-3.0.1 +============ + +pytest 3.0.1 has just been released to PyPI. + +This release fixes some regressions reported in version 3.0.0, being a +drop-in replacement. To upgrade: + + pip install --upgrade pytest + +The changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + + Adam Chainz + Andrew Svetlov + Bruno Oliveira + Daniel Hahler + Dmitry Dygalo + Florian Bruhin + Marcin Bachry + Ronny Pfannschmidt + matthiasha + +Happy testing, +The py.test Development Team diff --git a/doc/en/announce/release-3.0.2.rst b/doc/en/announce/release-3.0.2.rst new file mode 100644 index 000000000000000..9d1c05f2d4531ff --- /dev/null +++ b/doc/en/announce/release-3.0.2.rst @@ -0,0 +1,24 @@ +pytest-3.0.2 +============ + +pytest 3.0.2 has just been released to PyPI. + +This release fixes some regressions and bugs reported in version 3.0.1, being a +drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Ahn Ki-Wook +* Bruno Oliveira +* Florian Bruhin +* Jordan Guymon +* Raphael Pierzina +* Ronny Pfannschmidt +* mbyt + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.0.3.rst b/doc/en/announce/release-3.0.3.rst new file mode 100644 index 000000000000000..f00172195db8cb3 --- /dev/null +++ b/doc/en/announce/release-3.0.3.rst @@ -0,0 +1,27 @@ +pytest-3.0.3 +============ + +pytest 3.0.3 has just been released to PyPI. + +This release fixes some regressions and bugs reported in the last version, +being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Bruno Oliveira +* Florian Bruhin +* Floris Bruynooghe +* Huayi Zhang +* Lev Maximov +* Raquel Alegre +* Ronny Pfannschmidt +* Roy Williams +* Tyler Goodlet +* mbyt + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.0.4.rst b/doc/en/announce/release-3.0.4.rst new file mode 100644 index 000000000000000..852057037dd4ff5 --- /dev/null +++ b/doc/en/announce/release-3.0.4.rst @@ -0,0 +1,29 @@ +pytest-3.0.4 +============ + +pytest 3.0.4 has just been released to PyPI. + +This release fixes some regressions and bugs reported in the last version, +being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Bruno Oliveira +* Dan Wandschneider +* Florian Bruhin +* Georgy Dyuldin +* Grigorii Eremeev +* Jason R. Coombs +* Manuel Jacob +* Mathieu Clabaut +* Michael Seifert +* Nikolaus Rath +* Ronny Pfannschmidt +* Tom V + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.0.5.rst b/doc/en/announce/release-3.0.5.rst new file mode 100644 index 000000000000000..3e2419d7e5dc702 --- /dev/null +++ b/doc/en/announce/release-3.0.5.rst @@ -0,0 +1,27 @@ +pytest-3.0.5 +============ + +pytest 3.0.5 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Ana Vojnovic +* Bruno Oliveira +* Daniel Hahler +* Duncan Betts +* Igor Starikov +* Ismail +* Luke Murphy +* Ned Batchelder +* Ronny Pfannschmidt +* Sebastian Ramacher +* nmundar + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.0.6.rst b/doc/en/announce/release-3.0.6.rst new file mode 100644 index 000000000000000..2988b9cb3b8f3ed --- /dev/null +++ b/doc/en/announce/release-3.0.6.rst @@ -0,0 +1,33 @@ +pytest-3.0.6 +============ + +pytest 3.0.6 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + + +Thanks to all who contributed to this release, among them: + +* Andreas Pelme +* Bruno Oliveira +* Dmitry Malinovsky +* Eli Boyarski +* Jakub Wilk +* Jeff Widman +* Loïc Estève +* Luke Murphy +* Miro Hrončok +* Oscar Hellström +* Peter Heatwole +* Philippe Ombredanne +* Ronny Pfannschmidt +* Rutger Prins +* Stefan Scherfke + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.0.7.rst b/doc/en/announce/release-3.0.7.rst new file mode 100644 index 000000000000000..591557aa7873d3c --- /dev/null +++ b/doc/en/announce/release-3.0.7.rst @@ -0,0 +1,33 @@ +pytest-3.0.7 +============ + +pytest 3.0.7 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Anthony Sottile +* Barney Gale +* Bruno Oliveira +* Florian Bruhin +* Floris Bruynooghe +* Ionel Cristian Mărieș +* Katerina Koukiou +* NODA, Kai +* Omer Hadari +* Patrick Hayes +* Ran Benita +* Ronny Pfannschmidt +* Victor Uriarte +* Vidar Tonaas Fauske +* Ville Skyttä +* fbjorn +* mbyt + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.1.0.rst b/doc/en/announce/release-3.1.0.rst new file mode 100644 index 000000000000000..99cc6bdbe2077f1 --- /dev/null +++ b/doc/en/announce/release-3.1.0.rst @@ -0,0 +1,61 @@ +pytest-3.1.0 +======================================= + +The pytest team is proud to announce the 3.1.0 release! + +pytest is a mature Python testing tool with more than a 1600 tests +against itself, passing on many different interpreters and platforms. + +This release contains a bugs fixes and improvements, so users are encouraged +to take a look at the CHANGELOG: + +http://doc.pytest.org/en/latest/changelog.html + +For complete documentation, please visit: + + http://docs.pytest.org + +As usual, you can upgrade from pypi via: + + pip install -U pytest + +Thanks to all who contributed to this release, among them: + +* Anthony Sottile +* Ben Lloyd +* Bruno Oliveira +* David Giese +* David Szotten +* Dmitri Pribysh +* Florian Bruhin +* Florian Schulze +* Floris Bruynooghe +* John Towler +* Jonas Obrist +* Katerina Koukiou +* Kodi Arfer +* Krzysztof Szularz +* Lev Maximov +* Loïc Estève +* Luke Murphy +* Manuel Krebber +* Matthew Duck +* Matthias Bussonnier +* Michael Howitz +* Michal Wajszczuk +* Paweł Adamczak +* Rafael Bertoldi +* Ravi Chandra +* Ronny Pfannschmidt +* Skylar Downes +* Thomas Kriechbaumer +* Vitaly Lashmanov +* Vlad Dragos +* Wheerd +* Xander Johnson +* mandeep +* reut + + +Happy testing, +The Pytest Development Team diff --git a/doc/en/announce/release-3.1.1.rst b/doc/en/announce/release-3.1.1.rst new file mode 100644 index 000000000000000..370b8fd7355026c --- /dev/null +++ b/doc/en/announce/release-3.1.1.rst @@ -0,0 +1,23 @@ +pytest-3.1.1 +======================================= + +pytest 3.1.1 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Bruno Oliveira +* Florian Bruhin +* Floris Bruynooghe +* Jason R. Coombs +* Ronny Pfannschmidt +* wanghui + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.1.2.rst b/doc/en/announce/release-3.1.2.rst new file mode 100644 index 000000000000000..60168a857babf6c --- /dev/null +++ b/doc/en/announce/release-3.1.2.rst @@ -0,0 +1,23 @@ +pytest-3.1.2 +======================================= + +pytest 3.1.2 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Andreas Pelme +* ApaDoctor +* Bruno Oliveira +* Florian Bruhin +* Ronny Pfannschmidt +* Segev Finer + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.1.3.rst b/doc/en/announce/release-3.1.3.rst new file mode 100644 index 000000000000000..a55280626ba98d6 --- /dev/null +++ b/doc/en/announce/release-3.1.3.rst @@ -0,0 +1,23 @@ +pytest-3.1.3 +======================================= + +pytest 3.1.3 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Antoine Legrand +* Bruno Oliveira +* Max Moroz +* Raphael Pierzina +* Ronny Pfannschmidt +* Ryan Fitzpatrick + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.2.0.rst b/doc/en/announce/release-3.2.0.rst new file mode 100644 index 000000000000000..4d2830edd2dede7 --- /dev/null +++ b/doc/en/announce/release-3.2.0.rst @@ -0,0 +1,48 @@ +pytest-3.2.0 +======================================= + +The pytest team is proud to announce the 3.2.0 release! + +pytest is a mature Python testing tool with more than a 1600 tests +against itself, passing on many different interpreters and platforms. + +This release contains a number of bugs fixes and improvements, so users are encouraged +to take a look at the CHANGELOG: + + http://doc.pytest.org/en/latest/changelog.html + +For complete documentation, please visit: + + http://docs.pytest.org + +As usual, you can upgrade from pypi via: + + pip install -U pytest + +Thanks to all who contributed to this release, among them: + +* Alex Hartoto +* Andras Tim +* Bruno Oliveira +* Daniel Hahler +* Florian Bruhin +* Floris Bruynooghe +* John Still +* Jordan Moldow +* Kale Kundert +* Lawrence Mitchell +* Llandy Riveron Del Risco +* Maik Figura +* Martin Altmayer +* Mihai Capotă +* Nathaniel Waisbrot +* Nguyễn Hồng Quân +* Pauli Virtanen +* Raphael Pierzina +* Ronny Pfannschmidt +* Segev Finer +* V.Kuznetsov + + +Happy testing, +The Pytest Development Team diff --git a/doc/en/announce/release-3.2.1.rst b/doc/en/announce/release-3.2.1.rst new file mode 100644 index 000000000000000..899ffcd4b4aacd8 --- /dev/null +++ b/doc/en/announce/release-3.2.1.rst @@ -0,0 +1,22 @@ +pytest-3.2.1 +======================================= + +pytest 3.2.1 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Alex Gaynor +* Bruno Oliveira +* Florian Bruhin +* Ronny Pfannschmidt +* Srinivas Reddy Thatiparthy + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.2.2.rst b/doc/en/announce/release-3.2.2.rst new file mode 100644 index 000000000000000..599bf872775cb2b --- /dev/null +++ b/doc/en/announce/release-3.2.2.rst @@ -0,0 +1,28 @@ +pytest-3.2.2 +======================================= + +pytest 3.2.2 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Andreas Pelme +* Antonio Hidalgo +* Bruno Oliveira +* Felipe Dau +* Fernando Macedo +* Jesús Espino +* Joan Massich +* Joe Talbott +* Kirill Pinchuk +* Ronny Pfannschmidt +* Xuan Luong + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.2.3.rst b/doc/en/announce/release-3.2.3.rst new file mode 100644 index 000000000000000..589374974d91d95 --- /dev/null +++ b/doc/en/announce/release-3.2.3.rst @@ -0,0 +1,23 @@ +pytest-3.2.3 +======================================= + +pytest 3.2.3 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Bruno Oliveira +* Evan +* Joe Hamman +* Oliver Bestwalter +* Ronny Pfannschmidt +* Xuan Luong + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.2.4.rst b/doc/en/announce/release-3.2.4.rst new file mode 100644 index 000000000000000..44bfcc27e29b5fe --- /dev/null +++ b/doc/en/announce/release-3.2.4.rst @@ -0,0 +1,36 @@ +pytest-3.2.4 +======================================= + +pytest 3.2.4 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Bruno Oliveira +* Christian Boelsen +* Christoph Buchner +* Daw-Ran Liou +* Florian Bruhin +* Franck Michea +* Leonard Lausen +* Matty G +* Owen Tuz +* Pavel Karateev +* Pierre GIRAUD +* Ronny Pfannschmidt +* Stephen Finucane +* Sviatoslav Abakumov +* Thomas Hisch +* Tom Dalton +* Xuan Luong +* Yorgos Pagles +* Семён Марьясин + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.2.5.rst b/doc/en/announce/release-3.2.5.rst new file mode 100644 index 000000000000000..a520ce2b333d56b --- /dev/null +++ b/doc/en/announce/release-3.2.5.rst @@ -0,0 +1,18 @@ +pytest-3.2.5 +======================================= + +pytest 3.2.5 has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +* Bruno Oliveira + + +Happy testing, +The pytest Development Team diff --git a/doc/en/announce/release-3.3.0.rst b/doc/en/announce/release-3.3.0.rst new file mode 100644 index 000000000000000..e0740e7d59295fe --- /dev/null +++ b/doc/en/announce/release-3.3.0.rst @@ -0,0 +1,50 @@ +pytest-3.3.0 +======================================= + +The pytest team is proud to announce the 3.3.0 release! + +pytest is a mature Python testing tool with more than a 1600 tests +against itself, passing on many different interpreters and platforms. + +This release contains a number of bugs fixes and improvements, so users are encouraged +to take a look at the CHANGELOG: + + http://doc.pytest.org/en/latest/changelog.html + +For complete documentation, please visit: + + http://docs.pytest.org + +As usual, you can upgrade from pypi via: + + pip install -U pytest + +Thanks to all who contributed to this release, among them: + +* Anthony Sottile +* Bruno Oliveira +* Ceridwen +* Daniel Hahler +* Dirk Thomas +* Dmitry Malinovsky +* Florian Bruhin +* George Y. Kussumoto +* Hugo +* Jesús Espino +* Joan Massich +* Ofir +* OfirOshir +* Ronny Pfannschmidt +* Samuel Dion-Girardeau +* Srinivas Reddy Thatiparthy +* Sviatoslav Abakumov +* Tarcisio Fischer +* Thomas Hisch +* Tyler Goodlet +* hugovk +* je +* prokaktus + + +Happy testing, +The Pytest Development Team diff --git a/doc/en/announce/sprint2016.rst b/doc/en/announce/sprint2016.rst index e59ccdda7725a1a..8e706589876710d 100644 --- a/doc/en/announce/sprint2016.rst +++ b/doc/en/announce/sprint2016.rst @@ -4,9 +4,9 @@ python testing sprint June 20th-26th 2016 .. image:: ../img/freiburg2.jpg :width: 400 -The pytest core group is heading towards the biggest sprint -in its history, to take place in the black forest town Freiburg -in Germany. As of February 2016 we have started a `funding +The pytest core group held the biggest sprint +in its history in June 2016, taking place in the black forest town Freiburg +in Germany. In February 2016 we started a `funding campaign on Indiegogo to cover expenses `_ The page also mentions some preliminary topics: @@ -35,71 +35,30 @@ some preliminary topics: Participants -------------- -Here are preliminary participants who said they are likely to come, -given some expenses funding:: - - Anatoly Bubenkoff, Netherlands - Andreas Pelme, Personalkollen, Sweden - Anthony Wang, Splunk, US - Brianna Laugher, Australia - Bruno Oliveira, Brazil - Danielle Jenkins, Splunk, US - Dave Hunt, UK - Florian Bruhin, Switzerland - Floris Bruynooghe, Cobe.io, UK - Holger Krekel, merlinux, Germany - Oliver Bestwalter, Avira, Germany - Omar Kohl, Germany - Raphael Pierzina, FanDuel, UK - Tom Viner, UK - - - -Other contributors and experienced newcomers are invited to join as well -but please send a mail to the pytest-dev mailing list if you intend to -do so somewhat soon, also how much funding you need if so. And if you -are working for a company and using pytest heavily you are welcome to -join and we encourage your company to provide some funding for the -sprint. They may see it, and rightfully so, as a very cheap and deep -training which brings you together with the experts in the field :) +Over 20 participants took part from 4 continents, including employees +from Splunk, Personalkollen, Cobe.io, FanDuel and Dolby. Some newcomers +mixed with developers who have worked on pytest since its beginning, and +of course everyone in between. Sprint organisation, schedule ------------------------------- -tentative schedule: +People arrived in Freiburg on the 19th, with sprint development taking +place on 20th, 21st, 22nd, 24th and 25th. On the 23rd we took a break +day for some hot hiking in the Black Forest. -- 19/20th arrival in Freiburg -- 20th social get together, initial hacking -- 21/22th full sprint days -- 23rd break day, hiking -- 24/25th full sprint days -- 26th departure +Sprint activity was organised heavily around pairing, with plenty of group +discusssions to take advantage of the high bandwidth, and lightning talks +as well. -We might adjust according to weather to make sure that if -we do some hiking or excursion we'll have good weather. -Freiburg is one of the sunniest places in Germany so -it shouldn't be too much of a constraint. - - -Accomodation ----------------- - -We'll see to arrange for renting a flat with multiple -beds/rooms. Hotels are usually below 100 per night. -The earlier we book the better. Money / funding --------------- -The Indiegogo campaign asks for 11000 USD which should cover -the costs for flights and accomodation, renting a sprint place -and maybe a bit of food as well. -If your organisation wants to support the sprint but prefers -to give money according to an invoice, get in contact with -holger at http://merlinux.eu who can invoice your organisation -properly. +The Indiegogo campaign aimed for 11000 USD and in the end raised over +12000, to reimburse travel costs, pay for a sprint venue and catering. -If we have excess money we'll use for further sprint/travel -funding for pytest/tox contributors. +Excess money is reserved for further sprint/travel funding for pytest/tox +contributors. diff --git a/doc/en/assert.rst b/doc/en/assert.rst index e7f14e8bd1ad4c9..4a852978ed219be 100644 --- a/doc/en/assert.rst +++ b/doc/en/assert.rst @@ -24,16 +24,16 @@ following:: to assert that your function returns a certain value. If this assertion fails you will see the return value of the function call:: - $ py.test test_assert1.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items + $ pytest test_assert1.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item - test_assert1.py F + test_assert1.py F [100%] - ======= FAILURES ======== - _______ test_function ________ + ================================= FAILURES ================================= + ______________________________ test_function _______________________________ def test_function(): > assert f() == 4 @@ -41,7 +41,7 @@ you will see the return value of the function call:: E + where 3 = f() test_assert1.py:5: AssertionError - ======= 1 failed in 0.12 seconds ======== + ========================= 1 failed in 0.12 seconds ========================= ``pytest`` has support for showing the values of the most common subexpressions including calls, attributes, comparisons, and binary and unary @@ -85,6 +85,15 @@ and if you need to have access to the actual exception info you may use:: the actual exception raised. The main attributes of interest are ``.type``, ``.value`` and ``.traceback``. +.. versionchanged:: 3.0 + +In the context manager form you may use the keyword argument +``message`` to specify a custom failure message:: + + >>> with raises(ZeroDivisionError, message="Expecting ZeroDivisionError"): + ... pass + ... Failed: Expecting ZeroDivisionError + If you want to write test code that works on Python 2.4 as well, you may also use two other ways to test for an expected exception:: @@ -110,6 +119,23 @@ exceptions your own code is deliberately raising, whereas using like documenting unfixed bugs (where the test describes what "should" happen) or bugs in dependencies. +Also, the context manager form accepts a ``match`` keyword parameter to test +that a regular expression matches on the string representation of an exception +(like the ``TestCase.assertRaisesRegexp`` method from ``unittest``):: + + import pytest + + def myfunc(): + raise ValueError("Exception 123 raised") + + def test_match(): + with pytest.raises(ValueError, match=r'.* 123 .*'): + myfunc() + +The regexp parameter of the ``match`` method is matched with the ``re.search`` +function. So in the above example ``match='123'`` would have worked as +well. + .. _`assertwarns`: @@ -141,22 +167,22 @@ when it encounters comparisons. For example:: if you run this module:: - $ py.test test_assert2.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items + $ pytest test_assert2.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item - test_assert2.py F + test_assert2.py F [100%] - ======= FAILURES ======== - _______ test_set_comparison ________ + ================================= FAILURES ================================= + ___________________________ test_set_comparison ____________________________ def test_set_comparison(): set1 = set("1308") set2 = set("8035") > assert set1 == set2 - E assert set(['0', '1', '3', '8']) == set(['0', '3', '5', '8']) + E AssertionError: assert {'0', '1', '3', '8'} == {'0', '3', '5', '8'} E Extra items in the left set: E '1' E Extra items in the right set: @@ -164,7 +190,7 @@ if you run this module:: E Use -v to get the full diff test_assert2.py:5: AssertionError - ======= 1 failed in 0.12 seconds ======== + ========================= 1 failed in 0.12 seconds ========================= Special comparisons are done for a number of cases: @@ -181,9 +207,10 @@ It is possible to add your own detailed explanations by implementing the ``pytest_assertrepr_compare`` hook. .. autofunction:: _pytest.hookspec.pytest_assertrepr_compare + :noindex: -As an example consider adding the following hook in a conftest.py which -provides an alternative explanation for ``Foo`` objects:: +As an example consider adding the following hook in a :ref:`conftest.py ` +file which provides an alternative explanation for ``Foo`` objects:: # content of conftest.py from test_foocompare import Foo @@ -195,7 +222,7 @@ provides an alternative explanation for ``Foo`` objects:: now, given this test module:: # content of test_foocompare.py - class Foo: + class Foo(object): def __init__(self, val): self.val = val @@ -210,10 +237,10 @@ now, given this test module:: you can run the test module and get the custom output defined in the conftest file:: - $ py.test -q test_foocompare.py - F - ======= FAILURES ======== - _______ test_compare ________ + $ pytest -q test_foocompare.py + F [100%] + ================================= FAILURES ================================= + _______________________________ test_compare _______________________________ def test_compare(): f1 = Foo(1) @@ -234,50 +261,29 @@ Advanced assertion introspection .. versionadded:: 2.1 -Reporting details about a failing assertion is achieved either by rewriting -assert statements before they are run or re-evaluating the assert expression and -recording the intermediate values. Which technique is used depends on the -location of the assert, ``pytest`` configuration, and Python version being used -to run ``pytest``. - -By default, ``pytest`` rewrites assert statements in test modules. -Rewritten assert statements put introspection information into the assertion failure message. -``pytest`` only rewrites test modules directly discovered by its test collection process, so -asserts in supporting modules which are not themselves test modules will not be -rewritten. +Reporting details about a failing assertion is achieved by rewriting assert +statements before they are run. Rewritten assert statements put introspection +information into the assertion failure message. ``pytest`` only rewrites test +modules directly discovered by its test collection process, so asserts in +supporting modules which are not themselves test modules will not be rewritten. .. note:: - ``pytest`` rewrites test modules on import. It does this by using an import - hook to write a new pyc files. Most of the time this works transparently. + ``pytest`` rewrites test modules on import by using an import + hook to write new ``pyc`` files. Most of the time this works transparently. However, if you are messing with import yourself, the import hook may - interfere. If this is the case, simply use ``--assert=reinterp`` or - ``--assert=plain``. Additionally, rewriting will fail silently if it cannot - write new pycs, i.e. in a read-only filesystem or a zipfile. - -If an assert statement has not been rewritten or the Python version is less than -2.6, ``pytest`` falls back on assert reinterpretation. In assert -reinterpretation, ``pytest`` walks the frame of the function containing the -assert statement to discover sub-expression results of the failing assert -statement. You can force ``pytest`` to always use assertion reinterpretation by -passing the ``--assert=reinterp`` option. + interfere. -Assert reinterpretation has a caveat not present with assert rewriting: If -evaluating the assert expression has side effects you may get a warning that the -intermediate values could not be determined safely. A common example of this -issue is an assertion which reads from a file:: + If this is the case you have two options: - assert f.read() != '...' + * Disable rewriting for a specific module by adding the string + ``PYTEST_DONT_REWRITE`` to its docstring. -If this assertion fails then the re-evaluation will probably succeed! -This is because ``f.read()`` will return an empty string when it is -called the second time during the re-evaluation. However, it is -easy to rewrite the assertion and avoid any trouble:: + * Disable rewriting for all modules by using ``--assert=plain``. - content = f.read() - assert content != '...' + Additionally, rewriting will fail silently if it cannot write new ``.pyc`` files, + i.e. in a read-only filesystem or a zipfile. -All assert introspection can be turned off by passing ``--assert=plain``. For further information, Benjamin Peterson wrote up `Behind the scenes of pytest's new assertion rewriting `_. @@ -287,3 +293,7 @@ For further information, Benjamin Peterson wrote up `Behind the scenes of pytest .. versionchanged:: 2.1 Introduce the ``--assert`` option. Deprecate ``--no-assert`` and ``--nomagic``. + +.. versionchanged:: 3.0 + Removes the ``--no-assert`` and ``--nomagic`` options. + Removes the ``--assert=reinterp`` option. diff --git a/doc/en/backwards-compatibility.rst b/doc/en/backwards-compatibility.rst new file mode 100644 index 000000000000000..84f2c43edaa1b09 --- /dev/null +++ b/doc/en/backwards-compatibility.rst @@ -0,0 +1,105 @@ +.. _backwards-compatibility: + +Backwards Compatibility Policy +============================== + +Keeping backwards compatibility has a very high priority in the pytest project. Although we have deprecated functionality over the years, most of it is still supported. All deprecations in pytest were done because simpler or more efficient ways of accomplishing the same tasks have emerged, making the old way of doing things unnecessary. + +With the pytest 3.0 release we introduced a clear communication scheme for when we will actually remove the old busted joint and politely ask you to use the new hotness instead, while giving you enough time to adjust your tests or raise concerns if there are valid reasons to keep deprecated functionality around. + +To communicate changes we are already issuing deprecation warnings, but they are not displayed by default. In pytest 3.0 we changed the default setting so that pytest deprecation warnings are displayed if not explicitly silenced (with ``--disable-pytest-warnings``). + +We will only remove deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we will not remove it in 4.0 but in 5.0). + + +Deprecation Roadmap +------------------- + +This page lists deprecated features and when we plan to remove them. It is important to list the feature, the version where it got deprecated and the version we plan to remove it. + +Following our deprecation policy, we should aim to keep features for *at least* two minor versions after it was considered deprecated. + + +Future Releases +~~~~~~~~~~~~~~~ + +3.4 +^^^ + +**Old style classes** + +Issue: `#2147 `_. + +Deprecated in ``3.2``. + +4.0 +^^^ + +**Yield tests** + +Deprecated in ``3.0``. + +**pytest-namespace hook** + +deprecated in ``3.2``. + +**Marks in parameter sets** + +Deprecated in ``3.2``. + +**--result-log** + +Deprecated in ``3.0``. + +See `#830 `_ for more information. Suggested alternative: `pytest-tap `_. + +**metafunc.addcall** + +Issue: `#2876 `_. + +Deprecated in ``3.3``. + +**pytest_plugins in non-toplevel conftests** + +There is a deep conceptual confusion as ``conftest.py`` files themselves are activated/deactivated based on path, but the plugins they depend on aren't. + +Issue: `#2639 `_. + +Not yet officially deprecated. + +**passing a single string to pytest.main()** + +Pass a list of strings to ``pytest.main()`` instead. + +Deprecated in ``3.1``. + +**[pytest] section in setup.cfg** + +Use ``[tool:pytest]`` instead for compatibility with other tools. + +Deprecated in ``3.0``. + +Past Releases +~~~~~~~~~~~~~ + +3.0 +^^^ + +* The following deprecated commandline options were removed: + + * ``--genscript``: no longer supported; + * ``--no-assert``: use ``--assert=plain`` instead; + * ``--nomagic``: use ``--assert=plain`` instead; + * ``--report``: use ``-r`` instead; + +* Removed all ``py.test-X*`` entry points. The versioned, suffixed entry points + were never documented and a leftover from a pre-virtualenv era. These entry + points also created broken entry points in wheels, so removing them also + removes a source of confusion for users. + + + +3.3 +^^^ + +* Dropped support for EOL Python 2.6 and 3.3. \ No newline at end of file diff --git a/doc/en/bash-completion.rst b/doc/en/bash-completion.rst index b2a52fa637b3667..81fe62183fb093e 100644 --- a/doc/en/bash-completion.rst +++ b/doc/en/bash-completion.rst @@ -5,7 +5,7 @@ Setting up bash completion ========================== When using bash as your shell, ``pytest`` can use argcomplete -(https://argcomplete.readthedocs.org/) for auto-completion. +(https://argcomplete.readthedocs.io/) for auto-completion. For this ``argcomplete`` needs to be installed **and** enabled. Install argcomplete using:: @@ -18,11 +18,11 @@ For global activation of all argcomplete enabled python applications run:: For permanent (but not global) ``pytest`` activation, use:: - register-python-argcomplete py.test >> ~/.bashrc + register-python-argcomplete pytest >> ~/.bashrc For one-time activation of argcomplete for ``pytest`` only, use:: - eval "$(register-python-argcomplete py.test)" + eval "$(register-python-argcomplete pytest)" diff --git a/doc/en/builtin.rst b/doc/en/builtin.rst index b18c3f8280a5297..d11eb5606e09e51 100644 --- a/doc/en/builtin.rst +++ b/doc/en/builtin.rst @@ -35,6 +35,11 @@ Examples at :ref:`assertraises`. .. autofunction:: deprecated_call +Comparing floating point numbers +-------------------------------- + +.. autofunction:: approx + Raising a specific test outcome -------------------------------------- @@ -42,24 +47,24 @@ You can use the following functions in your test, fixture or setup functions to force a certain test outcome. Note that most often you can rather use declarative marks, see :ref:`skipping`. -.. autofunction:: _pytest.runner.fail -.. autofunction:: _pytest.runner.skip -.. autofunction:: _pytest.runner.importorskip -.. autofunction:: _pytest.skipping.xfail -.. autofunction:: _pytest.runner.exit +.. autofunction:: _pytest.outcomes.fail +.. autofunction:: _pytest.outcomes.skip +.. autofunction:: _pytest.outcomes.importorskip +.. autofunction:: _pytest.outcomes.xfail +.. autofunction:: _pytest.outcomes.exit -fixtures and requests +Fixtures and requests ----------------------------------------------------- To mark a fixture function: -.. autofunction:: _pytest.python.fixture +.. autofunction:: _pytest.fixtures.fixture Tutorial at :ref:`fixtures`. The ``request`` object that can be used from fixture functions. -.. autoclass:: _pytest.python.FixtureRequest() +.. autoclass:: _pytest.fixtures.FixtureRequest() :members: @@ -72,7 +77,7 @@ Builtin fixtures/function arguments You can ask for available builtin or project-custom :ref:`fixtures ` by typing:: - $ py.test -q --fixtures + $ pytest -q --fixtures cache Return a cache object that can persist state between testing sessions. @@ -84,36 +89,58 @@ You can ask for available builtin or project-custom Values can be any object handled by the json stdlib module. capsys - enables capturing of writes to sys.stdout/sys.stderr and makes + Enable capturing of writes to sys.stdout/sys.stderr and make + captured output available via ``capsys.readouterr()`` method calls + which return a ``(out, err)`` tuple. ``out`` and ``err`` will be ``text`` + objects. + capsysbinary + Enable capturing of writes to sys.stdout/sys.stderr and make captured output available via ``capsys.readouterr()`` method calls - which return a ``(out, err)`` tuple. + which return a ``(out, err)`` tuple. ``out`` and ``err`` will be ``bytes`` + objects. capfd - enables capturing of writes to file descriptors 1 and 2 and makes + Enable capturing of writes to file descriptors 1 and 2 and make captured output available via ``capfd.readouterr()`` method calls - which return a ``(out, err)`` tuple. + which return a ``(out, err)`` tuple. ``out`` and ``err`` will be ``text`` + objects. + capfdbinary + Enable capturing of write to file descriptors 1 and 2 and make + captured output available via ``capfdbinary.readouterr`` method calls + which return a ``(out, err)`` tuple. ``out`` and ``err`` will be + ``bytes`` objects. + doctest_namespace + Inject names into the doctest namespace. + pytestconfig + the pytest config object with access to command line opts. record_xml_property - Fixture that adds extra xml properties to the tag for the calling test. - The fixture is callable with (name, value), with value being automatically + Add extra xml properties to the tag for the calling test. + The fixture is callable with ``(name, value)``, with value being automatically xml-encoded. + caplog + Access and control log capturing. + + Captured logs are available through the following methods:: + + * caplog.text() -> string containing formatted log output + * caplog.records() -> list of logging.LogRecord instances + * caplog.record_tuples() -> list of (logger_name, level, message) tuples monkeypatch - The returned ``monkeypatch`` funcarg provides these + The returned ``monkeypatch`` fixture provides these helper methods to modify objects, dictionaries or os.environ:: - monkeypatch.setattr(obj, name, value, raising=True) - monkeypatch.delattr(obj, name, raising=True) - monkeypatch.setitem(mapping, name, value) - monkeypatch.delitem(obj, name, raising=True) - monkeypatch.setenv(name, value, prepend=False) - monkeypatch.delenv(name, value, raising=True) - monkeypatch.syspath_prepend(path) - monkeypatch.chdir(path) + monkeypatch.setattr(obj, name, value, raising=True) + monkeypatch.delattr(obj, name, raising=True) + monkeypatch.setitem(mapping, name, value) + monkeypatch.delitem(obj, name, raising=True) + monkeypatch.setenv(name, value, prepend=False) + monkeypatch.delenv(name, value, raising=True) + monkeypatch.syspath_prepend(path) + monkeypatch.chdir(path) All modifications will be undone after the requesting - test function has finished. The ``raising`` + test function or fixture has finished. The ``raising`` parameter determines if a KeyError or AttributeError will be raised if the set/deletion operation has no target. - pytestconfig - the pytest config object with access to command line opts. recwarn Return a WarningsRecorder instance that provides these methods: @@ -125,7 +152,7 @@ You can ask for available builtin or project-custom tmpdir_factory Return a TempdirFactory instance for the test session. tmpdir - return a temporary directory path object + Return a temporary directory path object which is unique to each test function invocation, created as a sub directory of the base temporary directory. The returned object is a `py.path.local`_ diff --git a/doc/en/cache.rst b/doc/en/cache.rst index 52abb52a0df4cd0..c88721b11b089e1 100644 --- a/doc/en/cache.rst +++ b/doc/en/cache.rst @@ -1,21 +1,17 @@ +.. _`cache_provider`: +.. _cache: + + Cache: working with cross-testrun state ======================================= .. versionadded:: 2.8 -.. warning:: - - The functionality of this core plugin was previously distributed - as a third party plugin named ``pytest-cache``. The core plugin - is compatible regarding command line options and API usage except that you - can only store/receive data between test runs that is json-serializable. - - Usage --------- The plugin provides two command line options to rerun failures from the -last ``py.test`` invocation: +last ``pytest`` invocation: * ``--lf``, ``--last-failed`` - to only re-run the failures. * ``--ff``, ``--failed-first`` - to run the failures first and then the rest of @@ -25,7 +21,7 @@ For cleanup (usually not needed), a ``--cache-clear`` option allows to remove all cross-session cache contents ahead of a test run. Other plugins may access the `config.cache`_ object to set/get -**json encodable** values between ``py.test`` invocations. +**json encodable** values between ``pytest`` invocations. .. note:: @@ -49,10 +45,10 @@ First, let's create 50 test invocation of which only 2 fail:: If you run this for the first time you will see two failures:: - $ py.test -q - .................F.......F........................ - ======= FAILURES ======== - _______ test_num[17] ________ + $ pytest -q + .................F.......F........................ [100%] + ================================= FAILURES ================================= + _______________________________ test_num[17] _______________________________ i = 17 @@ -63,7 +59,7 @@ If you run this for the first time you will see two failures:: E Failed: bad luck test_50.py:6: Failed - _______ test_num[25] ________ + _______________________________ test_num[25] _______________________________ i = 25 @@ -78,17 +74,17 @@ If you run this for the first time you will see two failures:: If you then run it with ``--lf``:: - $ py.test --lf - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - run-last-failure: rerun last 2 failures - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest --lf + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 50 items + run-last-failure: rerun previous 2 failures - test_50.py FF + test_50.py FF [100%] - ======= FAILURES ======== - _______ test_num[17] ________ + ================================= FAILURES ================================= + _______________________________ test_num[17] _______________________________ i = 17 @@ -99,7 +95,7 @@ If you then run it with ``--lf``:: E Failed: bad luck test_50.py:6: Failed - _______ test_num[25] ________ + _______________________________ test_num[25] _______________________________ i = 25 @@ -110,7 +106,8 @@ If you then run it with ``--lf``:: E Failed: bad luck test_50.py:6: Failed - ======= 2 failed, 48 deselected in 0.12 seconds ======== + =========================== 48 tests deselected ============================ + ================= 2 failed, 48 deselected in 0.12 seconds ================== You have run only the two failing test from the last run, while 48 tests have not been run ("deselected"). @@ -119,17 +116,17 @@ Now, if you run with the ``--ff`` option, all tests will be run but the first previous failures will be executed first (as can be seen from the series of ``FF`` and dots):: - $ py.test --ff - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - run-last-failure: rerun last 2 failures first - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest --ff + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 50 items + run-last-failure: rerun previous 2 failures first - test_50.py FF................................................ + test_50.py FF................................................ [100%] - ======= FAILURES ======== - _______ test_num[17] ________ + ================================= FAILURES ================================= + _______________________________ test_num[17] _______________________________ i = 17 @@ -140,7 +137,7 @@ of ``FF`` and dots):: E Failed: bad luck test_50.py:6: Failed - _______ test_num[25] ________ + _______________________________ test_num[25] _______________________________ i = 25 @@ -151,7 +148,7 @@ of ``FF`` and dots):: E Failed: bad luck test_50.py:6: Failed - ======= 2 failed, 48 passed in 0.12 seconds ======== + =================== 2 failed, 48 passed in 0.12 seconds ==================== .. _`config.cache`: @@ -163,7 +160,7 @@ The new config.cache object Plugins or conftest.py support code can get a cached value using the pytest ``config`` object. Here is a basic example plugin which implements a :ref:`fixture` which re-uses previously created state -across py.test invocations:: +across pytest invocations:: # content of test_caching.py import pytest @@ -184,10 +181,10 @@ across py.test invocations:: If you run this command once, it will take a while because of the sleep:: - $ py.test -q - F - ======= FAILURES ======== - _______ test_function ________ + $ pytest -q + F [100%] + ================================= FAILURES ================================= + ______________________________ test_function _______________________________ mydata = 42 @@ -201,10 +198,10 @@ of the sleep:: If you run it a second time the value will be retrieved from the cache and this will be quick:: - $ py.test -q - F - ======= FAILURES ======== - _______ test_function ________ + $ pytest -q + F [100%] + ================================= FAILURES ================================= + ______________________________ test_function _______________________________ mydata = 42 @@ -222,27 +219,20 @@ Inspecting Cache content ------------------------------- You can always peek at the content of the cache using the -``--cache-clear`` command line option:: +``--cache-show`` command line option:: - $ py.test --cache-clear - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items - - test_caching.py F - - ======= FAILURES ======== - _______ test_function ________ + $ py.test --cache-show + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + cachedir: $REGENDOC_TMPDIR/.cache + ------------------------------- cache values ------------------------------- + cache/lastfailed contains: + {'test_caching.py::test_function': True} + example/value contains: + 42 - mydata = 42 - - def test_function(mydata): - > assert mydata == 23 - E assert 42 == 23 - - test_caching.py:14: AssertionError - ======= 1 failed in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= Clearing Cache content ------------------------------- @@ -250,9 +240,9 @@ Clearing Cache content You can instruct pytest to clear all cache files and values by adding the ``--cache-clear`` option like this:: - py.test --cache-clear + pytest --cache-clear -This is recommended for invocations from Continous Integration +This is recommended for invocations from Continuous Integration servers where isolation and correctness is more important than speed. diff --git a/doc/en/capture.rst b/doc/en/capture.rst index 8892f5be756dfa8..a87b57f8fc061b8 100644 --- a/doc/en/capture.rst +++ b/doc/en/capture.rst @@ -36,9 +36,9 @@ There are two ways in which ``pytest`` can perform capturing: You can influence output capturing mechanisms from the command line:: - py.test -s # disable all capturing - py.test --capture=sys # replace sys.stdout/stderr with in-mem files - py.test --capture=fd # also point filedescriptors 1 and 2 to temp file + pytest -s # disable all capturing + pytest --capture=sys # replace sys.stdout/stderr with in-mem files + pytest --capture=fd # also point filedescriptors 1 and 2 to temp file .. _printdebugging: @@ -62,16 +62,16 @@ is that you can use print statements for debugging:: and running this module will show you precisely the output of the failing function and hide the other one:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 2 items - test_module.py .F + test_module.py .F [100%] - ======= FAILURES ======== - _______ test_func2 ________ + ================================= FAILURES ================================= + ________________________________ test_func2 ________________________________ def test_func2(): > assert False @@ -80,14 +80,14 @@ of the failing function and hide the other one:: test_module.py:9: AssertionError -------------------------- Captured stdout setup --------------------------- setting up - ======= 1 failed, 1 passed in 0.12 seconds ======== + ==================== 1 failed, 1 passed in 0.12 seconds ==================== Accessing captured output from a test function --------------------------------------------------- -The ``capsys`` and ``capfd`` fixtures allow to access stdout/stderr -output created during test execution. Here is an example test function -that performs some output related checks: +The ``capsys``, ``capsysbinary``, ``capfd``, and ``capfdbinary`` fixtures +allow access to stdout/stderr output created during test execution. Here is +an example test function that performs some output related checks: .. code-block:: python @@ -97,7 +97,7 @@ that performs some output related checks: out, err = capsys.readouterr() assert out == "hello\n" assert err == "world\n" - print "next" + print ("next") out, err = capsys.readouterr() assert out == "next\n" @@ -110,9 +110,39 @@ output streams and also interacts well with pytest's own per-test capturing. If you want to capture on filedescriptor level you can use -the ``capfd`` function argument which offers the exact +the ``capfd`` fixture which offers the exact same interface but allows to also capture output from libraries or subprocesses that directly write to operating system level output streams (FD1 and FD2). +.. versionadded:: 3.3 + +If the code under test writes non-textual data, you can capture this using +the ``capsysbinary`` fixture which instead returns ``bytes`` from +the ``readouterr`` method. The ``capfsysbinary`` fixture is currently only +available in python 3. + + +.. versionadded:: 3.3 + +If the code under test writes non-textual data, you can capture this using +the ``capfdbinary`` fixture which instead returns ``bytes`` from +the ``readouterr`` method. The ``capfdbinary`` fixture operates on the +filedescriptor level. + + +.. versionadded:: 3.0 + +To temporarily disable capture within a test, both ``capsys`` +and ``capfd`` have a ``disabled()`` method that can be used +as a context manager, disabling capture inside the ``with`` block: + +.. code-block:: python + + def test_disabling_capturing(capsys): + print('this output is captured') + with capsys.disabled(): + print('output not captured, going directly to sys.stdout') + print('this output is also captured') + .. include:: links.inc diff --git a/doc/en/conf.py b/doc/en/conf.py index aca0442c5d65bfd..40f1e4165e44209 100644 --- a/doc/en/conf.py +++ b/doc/en/conf.py @@ -19,11 +19,8 @@ # The short X.Y version. import os, sys -sys.path.insert(0, os.path.dirname(__file__)) -import _getdoctarget - -version = _getdoctarget.get_minor_version_string() -release = _getdoctarget.get_version_string() +from _pytest import __version__ as version +release = ".".join(version.split(".")[:2]) # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the @@ -127,7 +124,7 @@ # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". -html_title = None +html_title = 'pytest documentation' # A shorter title for the navigation bar. Default is the same as html_title. html_short_title = "pytest-%s" % release @@ -144,7 +141,7 @@ # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['_static'] +# html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. @@ -306,7 +303,7 @@ ('Holger Krekel@*Benjamin Peterson@*Ronny Pfannschmidt@*' 'Floris Bruynooghe@*others'), 'pytest', - 'simple powerful testing with Pytho', + 'simple powerful testing with Python', 'Programming', 1), ] diff --git a/doc/en/contact.rst b/doc/en/contact.rst index d4a1a03dee34345..83d496640d5ebe7 100644 --- a/doc/en/contact.rst +++ b/doc/en/contact.rst @@ -19,9 +19,9 @@ Contact channels - `pytest-commit at python.org (mailing list)`_: for commits and new issues - :doc:`contribution guide ` for help on submitting pull - requests to bitbucket (including using git via gitifyhg). + requests to GitHub. -- #pylib on irc.freenode.net IRC channel for random questions. +- ``#pylib`` on irc.freenode.net IRC channel for random questions. - private mail to Holger.Krekel at gmail com if you want to communicate sensitive issues @@ -46,6 +46,5 @@ Contact channels .. _`py-dev`: .. _`development mailing list`: .. _`pytest-dev at python.org (mailing list)`: http://mail.python.org/mailman/listinfo/pytest-dev -.. _`py-svn`: .. _`pytest-commit at python.org (mailing list)`: http://mail.python.org/mailman/listinfo/pytest-commit diff --git a/doc/en/contents.rst b/doc/en/contents.rst index 48c3471b5e20d6b..7a6570e0bc5b924 100644 --- a/doc/en/contents.rst +++ b/doc/en/contents.rst @@ -3,37 +3,63 @@ Full pytest documentation =========================== -`Download latest version as PDF `_ +`Download latest version as PDF `_ .. `Download latest version as EPUB `_ .. toctree:: :maxdepth: 2 - overview - apiref - example/index + getting-started + usage + existingtestsuite + assert + builtin + fixture monkeypatch tmpdir capture - recwarn + warnings + doctest + mark + skipping + parametrize cache + unittest + nose + xunit_setup plugins + writing_plugins + logging + + goodpractices + pythonpath + customize + example/index + bash-completion + backwards-compatibility + historical-notes + license contributing + development_guide talks + projects + faq + contact .. only:: html .. toctree:: + :maxdepth: 1 - funcarg_compare announce/index .. only:: html .. toctree:: :hidden: + :maxdepth: 1 changelog diff --git a/doc/en/customize.rst b/doc/en/customize.rst index 34e319c246a820b..8133704a52c2e71 100644 --- a/doc/en/customize.rst +++ b/doc/en/customize.rst @@ -1,5 +1,5 @@ -Basic test configuration -=================================== +Configuration +============= Command line options and configuration file settings ----------------------------------------------------------------- @@ -7,7 +7,7 @@ Command line options and configuration file settings You can get help on command line options and values in INI-style configurations files by using the general help option:: - py.test -h # prints options _and_ config file settings + pytest -h # prints options _and_ config file settings This will display command line and configuration file settings which were registered by installed plugins. @@ -15,39 +15,67 @@ which were registered by installed plugins. .. _rootdir: .. _inifiles: -initialization: determining rootdir and inifile +Initialization: determining rootdir and inifile ----------------------------------------------- .. versionadded:: 2.7 -pytest determines a "rootdir" for each test run which depends on +pytest determines a ``rootdir`` for each test run which depends on the command line arguments (specified test files, paths) and on -the existence of inifiles. The determined rootdir and ini-file are -printed as part of the pytest header. The rootdir is used for constructing -"nodeids" during collection and may also be used by plugins to store -project/testrun-specific information. +the existence of *ini-files*. The determined ``rootdir`` and *ini-file* are +printed as part of the pytest header during startup. + +Here's a summary what ``pytest`` uses ``rootdir`` for: + +* Construct *nodeids* during collection; each test is assigned + a unique *nodeid* which is rooted at the ``rootdir`` and takes in account full path, + class name, function name and parametrization (if any). + +* Is used by plugins as a stable location to store project/test run specific information; + for example, the internal :ref:`cache ` plugin creates a ``.cache`` subdirectory + in ``rootdir`` to store its cross-test run state. + +Important to emphasize that ``rootdir`` is **NOT** used to modify ``sys.path``/``PYTHONPATH`` or +influence how modules are imported. See :ref:`pythonpath` for more details. + +Finding the ``rootdir`` +~~~~~~~~~~~~~~~~~~~~~~~ Here is the algorithm which finds the rootdir from ``args``: -- determine the common ancestor directory for the specified ``args``. +- determine the common ancestor directory for the specified ``args`` that are + recognised as paths that exist in the file system. If no such paths are + found, the common ancestor directory is set to the current working directory. -- look for ``pytest.ini``, ``tox.ini`` and ``setup.cfg`` files in the - ancestor directory and upwards. If one is matched, it becomes the - ini-file and its directory becomes the rootdir. An existing - ``pytest.ini`` file will always be considered a match whereas - ``tox.ini`` and ``setup.cfg`` will only match if they contain - a ``[pytest]`` section. +- look for ``pytest.ini``, ``tox.ini`` and ``setup.cfg`` files in the ancestor + directory and upwards. If one is matched, it becomes the ini-file and its + directory becomes the rootdir. -- if no ini-file was found, look for ``setup.py`` upwards from - the common ancestor directory to determine the ``rootdir``. +- if no ini-file was found, look for ``setup.py`` upwards from the common + ancestor directory to determine the ``rootdir``. -- if no ini-file and no ``setup.py`` was found, use the already - determined common ancestor as root directory. This allows to - work with pytest in structures that are not part of a package - and don't have any particular ini-file configuration. +- if no ``setup.py`` was found, look for ``pytest.ini``, ``tox.ini`` and + ``setup.cfg`` in each of the specified ``args`` and upwards. If one is + matched, it becomes the ini-file and its directory becomes the rootdir. -Note that options from multiple ini-files candidates are never merged, -the first one wins (``pytest.ini`` always wins even if it does not +- if no ini-file was found, use the already determined common ancestor as root + directory. This allows the use of pytest in structures that are not part of + a package and don't have any particular ini-file configuration. + +If no ``args`` are given, pytest collects test below the current working +directory and also starts determining the rootdir from there. + +:warning: custom pytest plugin commandline arguments may include a path, as in + ``pytest --log-output ../../test.log args``. Then ``args`` is mandatory, + otherwise pytest uses the folder of test.log for rootdir determination + (see also `issue 1435 `_). + A dot ``.`` for referencing to the current working directory is also + possible. + +Note that an existing ``pytest.ini`` file will always be considered a match, +whereas ``tox.ini`` and ``setup.cfg`` will only match if they contain a +``[pytest]`` or ``[tool:pytest]`` section, respectively. Options from multiple ini-files candidates are never +merged - the first one wins (``pytest.ini`` always wins, even if it does not contain a ``[pytest]`` section). The ``config`` object will subsequently carry these attributes: @@ -62,14 +90,14 @@ per-testrun information. Example:: - py.test path/to/testdir path/other/ + pytest path/to/testdir path/other/ will determine the common ancestor as ``path`` and then check for ini-files as follows:: # first look for pytest.ini files path/pytest.ini - path/setup.cfg # must also contain [pytest] section to match + path/setup.cfg # must also contain [tool:pytest] section to match path/tox.ini # must also contain [pytest] section to match pytest.ini ... # all the way down to the root @@ -83,6 +111,8 @@ check for ini-files as follows:: .. _`how to change command line options defaults`: .. _`adding default options`: + + How to change command line options defaults ------------------------------------------------ @@ -96,15 +126,27 @@ progress output, you can write it into a configuration file: # content of pytest.ini # (or tox.ini or setup.cfg) [pytest] - addopts = -rsxX -q + addopts = -ra -q -Alternatively, you can set a PYTEST_ADDOPTS environment variable to add command +Alternatively, you can set a ``PYTEST_ADDOPTS`` environment variable to add command line options while the environment is in use:: - export PYTEST_ADDOPTS="-rsxX -q" + export PYTEST_ADDOPTS="-v" -From now on, running ``pytest`` will add the specified options. +Here's how the command-line is built in the presence of ``addopts`` or the environment variable:: + $PYTEST_ADDOTPS + +So if the user executes in the command-line:: + + pytest -m slow + +The actual command line executed is:: + + pytest -ra -q -v -m slow + +Note that as usual for other command-line applications, in case of conflicting options the last one wins, so the example +above will show verbose output because ``-v`` overwrites ``-q``. Builtin configuration file options @@ -126,9 +168,9 @@ Builtin configuration file options [pytest] addopts = --maxfail=2 -rf # exit after 2 failures, report fail info - issuing ``py.test test_hello.py`` actually means:: + issuing ``pytest test_hello.py`` actually means:: - py.test --maxfail=2 -rf test_hello.py + pytest --maxfail=2 -rf test_hello.py Default is to add no options. @@ -144,18 +186,27 @@ Builtin configuration file options [seq] matches any character in seq [!seq] matches any char not in seq - Default patterns are ``'.*', 'CVS', '_darcs', '{arch}', '*.egg'``. + Default patterns are ``'.*', 'build', 'dist', 'CVS', '_darcs', '{arch}', '*.egg', 'venv'``. Setting a ``norecursedirs`` replaces the default. Here is an example of how to avoid certain directories: .. code-block:: ini - # content of setup.cfg + # content of pytest.ini [pytest] norecursedirs = .svn _build tmp* This would tell ``pytest`` to not look into typical subversion or - sphinx-build directories or into any ``tmp`` prefixed directory. + sphinx-build directories or into any ``tmp`` prefixed directory. + + Additionally, ``pytest`` will attempt to intelligently identify and ignore a + virtualenv by the presence of an activation script. Any directory deemed to + be the root of a virtual environment will not be considered during test + collection unless ``‑‑collect‑in‑virtualenv`` is given. Note also that + ``norecursedirs`` takes precedence over ``‑‑collect‑in‑virtualenv``; e.g. if + you intend to run tests in a virtualenv with a base directory that matches + ``'.*'`` you *must* override ``norecursedirs`` in addition to using the + ``‑‑collect‑in‑virtualenv`` flag. .. confval:: testpaths @@ -179,13 +230,16 @@ Builtin configuration file options .. confval:: python_files One or more Glob-style file patterns determining which python files - are considered as test modules. + are considered as test modules. By default, pytest will consider + any file matching with ``test_*.py`` and ``*_test.py`` globs as a test + module. .. confval:: python_classes One or more name prefixes or glob-style patterns determining which classes - are considered for test collection. Here is an example of how to collect - tests from classes that end in ``Suite``: + are considered for test collection. By default, pytest will consider any + class prefixed with ``Test`` as a test collection. Here is an example of how + to collect tests from classes that end in ``Suite``: .. code-block:: ini @@ -200,7 +254,8 @@ Builtin configuration file options .. confval:: python_functions One or more name prefixes or glob-patterns determining which test functions - and methods are considered tests. Here is an example of how + and methods are considered tests. By default, pytest will consider any + function prefixed with ``test`` as a test. Here is an example of how to collect test functions and methods that end in ``_test``: .. code-block:: ini @@ -218,7 +273,7 @@ Builtin configuration file options .. confval:: doctest_optionflags One or more doctest flag names from the standard ``doctest`` module. - :doc:`See how py.test handles doctests `. + :doc:`See how pytest handles doctests `. .. confval:: confcutdir @@ -226,3 +281,53 @@ Builtin configuration file options By default, pytest will stop searching for ``conftest.py`` files upwards from ``pytest.ini``/``tox.ini``/``setup.cfg`` of the project if any, or up to the file-system root. + + +.. confval:: filterwarnings + + .. versionadded:: 3.1 + + Sets a list of filters and actions that should be taken for matched + warnings. By default all warnings emitted during the test session + will be displayed in a summary at the end of the test session. + + .. code-block:: ini + + # content of pytest.ini + [pytest] + filterwarnings = + error + ignore::DeprecationWarning + + This tells pytest to ignore deprecation warnings and turn all other warnings + into errors. For more information please refer to :ref:`warnings`. + +.. confval:: cache_dir + + .. versionadded:: 3.2 + + Sets a directory where stores content of cache plugin. Default directory is + ``.cache`` which is created in :ref:`rootdir `. Directory may be + relative or absolute path. If setting relative path, then directory is created + relative to :ref:`rootdir `. Additionally path may contain environment + variables, that will be expanded. For more information about cache plugin + please refer to :ref:`cache_provider`. + + +.. confval:: console_output_style + + .. versionadded:: 3.3 + + Sets the console output style while running tests: + + * ``classic``: classic pytest output. + * ``progress``: like classic pytest output, but with a progress indicator. + + The default is ``progress``, but you can fallback to ``classic`` if you prefer or + the new mode is causing unexpected problems: + + .. code-block:: ini + + # content of pytest.ini + [pytest] + console_output_style = classic diff --git a/doc/en/development_guide.rst b/doc/en/development_guide.rst new file mode 100644 index 000000000000000..465e97de0f9ce09 --- /dev/null +++ b/doc/en/development_guide.rst @@ -0,0 +1,108 @@ +================= +Development Guide +================= + +Some general guidelines regarding development in pytest for core maintainers and general contributors. Nothing here +is set in stone and can't be changed, feel free to suggest improvements or changes in the workflow. + + +Code Style +---------- + +* `PEP-8 `_ +* `flake8 `_ for quality checks +* `invoke `_ to automate development tasks + + +Branches +-------- + +We have two long term branches: + +* ``master``: contains the code for the next bugfix release. +* ``features``: contains the code with new features for the next minor release. + +The official repository usually does not contain topic branches, developers and contributors should create topic +branches in their own forks. + +Exceptions can be made for cases where more than one contributor is working on the same +topic or where it makes sense to use some automatic capability of the main repository, such as automatic docs from +`readthedocs `_ for a branch dealing with documentation refactoring. + +Issues +------ + +Any question, feature, bug or proposal is welcome as an issue. Users are encouraged to use them whenever they need. + +GitHub issues should use labels to categorize them. Labels should be created sporadically, to fill a niche; we should +avoid creating labels just for the sake of creating them. + +Here is a list of labels and a brief description mentioning their intent. + + +**Type** + +* ``type: backward compatibility``: issue that will cause problems with old pytest versions. +* ``type: bug``: problem that needs to be addressed. +* ``type: deprecation``: feature that will be deprecated in the future. +* ``type: docs``: documentation missing or needing clarification. +* ``type: enhancement``: new feature or API change, should be merged into ``features``. +* ``type: feature-branch``: new feature or API change, should be merged into ``features``. +* ``type: infrastructure``: improvement to development/releases/CI structure. +* ``type: performance``: performance or memory problem/improvement. +* ``type: proposal``: proposal for a new feature, often to gather opinions or design the API around the new feature. +* ``type: question``: question regarding usage, installation, internals or how to test something. +* ``type: refactoring``: internal improvements to the code. +* ``type: regression``: indicates a problem that was introduced in a release which was working previously. + +**Status** + +* ``status: critical``: grave problem or usability issue that affects lots of users. +* ``status: easy``: easy issue that is friendly to new contributors. +* ``status: help wanted``: core developers need help from experts on this topic. +* ``status: needs information``: reporter needs to provide more information; can be closed after 2 or more weeks of inactivity. + +**Topic** + +* ``topic: collection`` +* ``topic: fixtures`` +* ``topic: parametrize`` +* ``topic: reporting`` +* ``topic: selection`` +* ``topic: tracebacks`` + +**Plugin (internal or external)** + +* ``plugin: cache`` +* ``plugin: capture`` +* ``plugin: doctests`` +* ``plugin: junitxml`` +* ``plugin: monkeypatch`` +* ``plugin: nose`` +* ``plugin: pastebin`` +* ``plugin: pytester`` +* ``plugin: tmpdir`` +* ``plugin: unittest`` +* ``plugin: warnings`` +* ``plugin: xdist`` + + +**OS** + +Issues specific to a single operating system. Do not use as a means to indicate where an issue originated from, only +for problems that happen **only** in that system. + +* ``os: linux`` +* ``os: mac`` +* ``os: windows`` + +**Temporary** + +Used to classify issues for limited time, to help find issues related in events for example. +They should be removed after they are no longer relevant. + +* ``temporary: EP2017 sprint``: +* ``temporary: sprint-candidate``: + + +.. include:: ../../HOWTORELEASE.rst diff --git a/doc/en/doctest.rst b/doc/en/doctest.rst index db764141ecb3683..4c5a878dd613eaf 100644 --- a/doc/en/doctest.rst +++ b/doc/en/doctest.rst @@ -6,16 +6,29 @@ By default all files matching the ``test*.txt`` pattern will be run through the python standard ``doctest`` module. You can change the pattern by issuing:: - py.test --doctest-glob='*.rst' + pytest --doctest-glob='*.rst' on the command line. Since version ``2.9``, ``--doctest-glob`` can be given multiple times in the command-line. +.. versionadded:: 3.1 + + You can specify the encoding that will be used for those doctest files + using the ``doctest_encoding`` ini option: + + .. code-block:: ini + + # content of pytest.ini + [pytest] + doctest_encoding = latin1 + + The default encoding is UTF-8. + You can also trigger running of doctests from docstrings in all python modules (including regular python test modules):: - py.test --doctest-modules + pytest --doctest-modules You can make these changes permanent in your project by putting them into a pytest.ini file like this: @@ -45,17 +58,17 @@ and another like this:: """ return 42 -then you can just invoke ``py.test`` without command line options:: +then you can just invoke ``pytest`` without command line options:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini - collected 1 items + collected 1 item - mymodule.py . + mymodule.py . [100%] - ======= 1 passed in 0.12 seconds ======== + ========================= 1 passed in 0.12 seconds ========================= It is possible to use fixtures using the ``getfixture`` helper:: @@ -68,7 +81,7 @@ Also, :ref:`usefixtures` and :ref:`autouse` fixtures are supported when executing text doctest files. The standard ``doctest`` module provides some setting flags to configure the -strictness of doctest tests. In py.test You can enable those flags those flags +strictness of doctest tests. In pytest You can enable those flags those flags using the configuration file. To make pytest ignore trailing whitespaces and ignore lengthy exception stack traces you can just write: @@ -77,7 +90,7 @@ ignore lengthy exception stack traces you can just write: [pytest] doctest_optionflags= NORMALIZE_WHITESPACE IGNORE_EXCEPTION_DETAIL -py.test also introduces new options to allow doctests to run in Python 2 and +pytest also introduces new options to allow doctests to run in Python 2 and Python 3 unchanged: * ``ALLOW_UNICODE``: when enabled, the ``u`` prefix is stripped from unicode @@ -103,3 +116,50 @@ itself:: 'Hello' +The 'doctest_namespace' fixture +------------------------------- + +.. versionadded:: 3.0 + +The ``doctest_namespace`` fixture can be used to inject items into the +namespace in which your doctests run. It is intended to be used within +your own fixtures to provide the tests that use them with context. + +``doctest_namespace`` is a standard ``dict`` object into which you +place the objects you want to appear in the doctest namespace:: + + # content of conftest.py + import numpy + @pytest.fixture(autouse=True) + def add_np(doctest_namespace): + doctest_namespace['np'] = numpy + +which can then be used in your doctests directly:: + + # content of numpy.py + def arange(): + """ + >>> a = np.arange(10) + >>> len(a) + 10 + """ + pass + + +Output format +------------- + +.. versionadded:: 3.0 + +You can change the diff output format on failure for your doctests +by using one of standard doctest modules format in options +(see :data:`python:doctest.REPORT_UDIFF`, :data:`python:doctest.REPORT_CDIFF`, +:data:`python:doctest.REPORT_NDIFF`, :data:`python:doctest.REPORT_ONLY_FIRST_FAILURE`):: + + pytest --doctest-modules --doctest-report none + pytest --doctest-modules --doctest-report udiff + pytest --doctest-modules --doctest-report cdiff + pytest --doctest-modules --doctest-report ndiff + pytest --doctest-modules --doctest-report only_first_failure + + diff --git a/doc/en/example/assertion/failure_demo.py b/doc/en/example/assertion/failure_demo.py index a4ff758b18c7739..d31fba2adaa2e8c 100644 --- a/doc/en/example/assertion/failure_demo.py +++ b/doc/en/example/assertion/failure_demo.py @@ -128,7 +128,7 @@ class Bar(object): def globf(x): return x+1 -class TestRaises: +class TestRaises(object): def test_raises(self): s = 'qwe' raises(TypeError, "int(s)") @@ -167,7 +167,7 @@ def test_dynamic_compile_shows_nicely(): -class TestMoreErrors: +class TestMoreErrors(object): def test_complex_error(self): def f(): return 44 @@ -213,23 +213,23 @@ def test_try_finally(self): x = 0 -class TestCustomAssertMsg: +class TestCustomAssertMsg(object): def test_single_line(self): - class A: + class A(object): a = 1 b = 2 assert A.a == b, "A.a appears not to be b" def test_multiline(self): - class A: + class A(object): a = 1 b = 2 assert A.a == b, "A.a appears not to be b\n" \ "or does not appear to be b\none of those" def test_custom_repr(self): - class JSON: + class JSON(object): a = 1 def __repr__(self): return "This is JSON\n{\n 'foo': 'bar'\n}" diff --git a/doc/en/example/assertion/test_setup_flow_example.py b/doc/en/example/assertion/test_setup_flow_example.py index 512330cb4c71b77..100effa499f3e4b 100644 --- a/doc/en/example/assertion/test_setup_flow_example.py +++ b/doc/en/example/assertion/test_setup_flow_example.py @@ -1,7 +1,7 @@ def setup_module(module): module.TestStateFullThing.classcount = 0 -class TestStateFullThing: +class TestStateFullThing(object): def setup_class(cls): cls.classcount += 1 diff --git a/doc/en/example/attic.rst b/doc/en/example/attic.rst index 1bc32b2837db9d9..9e124a5d09d35a3 100644 --- a/doc/en/example/attic.rst +++ b/doc/en/example/attic.rst @@ -15,9 +15,9 @@ example: specifying and selecting acceptance tests def pytest_funcarg__accept(request): return AcceptFixture(request) - class AcceptFixture: + class AcceptFixture(object): def __init__(self, request): - if not request.config.option.acceptance: + if not request.config.getoption('acceptance'): pytest.skip("specify -A to run acceptance tests") self.tmpdir = request.config.mktemp(request.function.__name__, numbered=True) @@ -61,7 +61,7 @@ extend the `accept example`_ by putting this in our test module: arg.tmpdir.mkdir("special") return arg - class TestSpecialAcceptance: + class TestSpecialAcceptance(object): def test_sometest(self, accept): assert accept.tmpdir.join("special").check() diff --git a/doc/en/example/costlysetup/conftest.py b/doc/en/example/costlysetup/conftest.py index d689c11b207f7d1..ea3c1cffb72fa52 100644 --- a/doc/en/example/costlysetup/conftest.py +++ b/doc/en/example/costlysetup/conftest.py @@ -4,10 +4,10 @@ @pytest.fixture("session") def setup(request): setup = CostlySetup() - request.addfinalizer(setup.finalize) - return setup + yield setup + setup.finalize() -class CostlySetup: +class CostlySetup(object): def __init__(self): import time print ("performing costly setup") diff --git a/doc/en/example/index.rst b/doc/en/example/index.rst index 363de5ab71482b3..f63cb822a41cbec 100644 --- a/doc/en/example/index.rst +++ b/doc/en/example/index.rst @@ -1,8 +1,8 @@ .. _examples: -Usages and Examples -=========================================== +Examples and customization tricks +================================= Here is a (growing) list of examples. :ref:`Contact ` us if you need more examples or have questions. Also take a look at the diff --git a/doc/en/example/layout1/setup.cfg b/doc/en/example/layout1/setup.cfg deleted file mode 100644 index 02d3750ee405b6f..000000000000000 --- a/doc/en/example/layout1/setup.cfg +++ /dev/null @@ -1,4 +0,0 @@ -[pytest] -testfilepatterns = - ${topdir}/tests/unit/test_${basename} - ${topdir}/tests/functional/*.py diff --git a/doc/en/example/markers.rst b/doc/en/example/markers.rst index 6bdc603479ca254..43c20d5b7dc18ca 100644 --- a/doc/en/example/markers.rst +++ b/doc/en/example/markers.rst @@ -21,7 +21,7 @@ You can "mark" a test function with custom metadata like this:: pass def test_another(): pass - class TestClass: + class TestClass(object): def test_method(self): pass @@ -29,33 +29,33 @@ You can "mark" a test function with custom metadata like this:: You can then restrict a test run to only run tests marked with ``webtest``:: - $ py.test -v -m webtest - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v -m webtest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collecting ... collected 4 items - test_server.py::test_send_http PASSED + test_server.py::test_send_http PASSED [100%] - ======= 3 tests deselected by "-m 'webtest'" ======== - ======= 1 passed, 3 deselected in 0.12 seconds ======== + ============================ 3 tests deselected ============================ + ================== 1 passed, 3 deselected in 0.12 seconds ================== Or the inverse, running all tests except the webtest ones:: - $ py.test -v -m "not webtest" - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v -m "not webtest" + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collecting ... collected 4 items - test_server.py::test_something_quick PASSED - test_server.py::test_another PASSED - test_server.py::TestClass::test_method PASSED + test_server.py::test_something_quick PASSED [ 33%] + test_server.py::test_another PASSED [ 66%] + test_server.py::TestClass::test_method PASSED [100%] - ======= 1 tests deselected by "-m 'not webtest'" ======== - ======= 3 passed, 1 deselected in 0.12 seconds ======== + ============================ 1 tests deselected ============================ + ================== 3 passed, 1 deselected in 0.12 seconds ================== Selecting tests based on their node ID -------------------------------------- @@ -64,43 +64,43 @@ You can provide one or more :ref:`node IDs ` as positional arguments to select only specified tests. This makes it easy to select tests based on their module, class, method, or function name:: - $ py.test -v test_server.py::TestClass::test_method - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v test_server.py::TestClass::test_method + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: - collecting ... collected 5 items + rootdir: $REGENDOC_TMPDIR, inifile: + collecting ... collected 1 item - test_server.py::TestClass::test_method PASSED + test_server.py::TestClass::test_method PASSED [100%] - ======= 1 passed in 0.12 seconds ======== + ========================= 1 passed in 0.12 seconds ========================= You can also select on the class:: - $ py.test -v test_server.py::TestClass - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v test_server.py::TestClass + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: - collecting ... collected 4 items + rootdir: $REGENDOC_TMPDIR, inifile: + collecting ... collected 1 item - test_server.py::TestClass::test_method PASSED + test_server.py::TestClass::test_method PASSED [100%] - ======= 1 passed in 0.12 seconds ======== + ========================= 1 passed in 0.12 seconds ========================= Or select multiple nodes:: - $ py.test -v test_server.py::TestClass test_server.py::test_send_http - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v test_server.py::TestClass test_server.py::test_send_http + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: - collecting ... collected 8 items + rootdir: $REGENDOC_TMPDIR, inifile: + collecting ... collected 2 items - test_server.py::TestClass::test_method PASSED - test_server.py::test_send_http PASSED + test_server.py::TestClass::test_method PASSED [ 50%] + test_server.py::test_send_http PASSED [100%] - ======= 2 passed in 0.12 seconds ======== + ========================= 2 passed in 0.12 seconds ========================= .. _node-id: @@ -115,8 +115,8 @@ Or select multiple nodes:: ``module.py::function[param]``. Node IDs for failing tests are displayed in the test summary info - when running py.test with the ``-rf`` option. You can also - construct Node IDs from the output of ``py.test --collectonly``. + when running pytest with the ``-rf`` option. You can also + construct Node IDs from the output of ``pytest --collectonly``. Using ``-k expr`` to select tests based on their name ------------------------------------------------------- @@ -128,59 +128,63 @@ which implements a substring match on the test names instead of the exact match on markers that ``-m`` provides. This makes it easy to select tests based on their names:: - $ py.test -v -k http # running with the above defined example module - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v -k http # running with the above defined example module + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collecting ... collected 4 items - test_server.py::test_send_http PASSED + test_server.py::test_send_http PASSED [100%] - ======= 3 tests deselected by '-khttp' ======== - ======= 1 passed, 3 deselected in 0.12 seconds ======== + ============================ 3 tests deselected ============================ + ================== 1 passed, 3 deselected in 0.12 seconds ================== And you can also run all tests except the ones that match the keyword:: - $ py.test -k "not send_http" -v - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -k "not send_http" -v + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collecting ... collected 4 items - test_server.py::test_something_quick PASSED - test_server.py::test_another PASSED - test_server.py::TestClass::test_method PASSED + test_server.py::test_something_quick PASSED [ 33%] + test_server.py::test_another PASSED [ 66%] + test_server.py::TestClass::test_method PASSED [100%] - ======= 1 tests deselected by '-knot send_http' ======== - ======= 3 passed, 1 deselected in 0.12 seconds ======== + ============================ 1 tests deselected ============================ + ================== 3 passed, 1 deselected in 0.12 seconds ================== Or to select "http" and "quick" tests:: - $ py.test -k "http or quick" -v - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -k "http or quick" -v + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collecting ... collected 4 items - test_server.py::test_send_http PASSED - test_server.py::test_something_quick PASSED + test_server.py::test_send_http PASSED [ 50%] + test_server.py::test_something_quick PASSED [100%] - ======= 2 tests deselected by '-khttp or quick' ======== - ======= 2 passed, 2 deselected in 0.12 seconds ======== + ============================ 2 tests deselected ============================ + ================== 2 passed, 2 deselected in 0.12 seconds ================== .. note:: - If you are using expressions such as "X and Y" then both X and Y - need to be simple non-keyword names. For example, "pass" or "from" - will result in SyntaxErrors because "-k" evaluates the expression. + If you are using expressions such as ``"X and Y"`` then both ``X`` and ``Y`` + need to be simple non-keyword names. For example, ``"pass"`` or ``"from"`` + will result in SyntaxErrors because ``"-k"`` evaluates the expression using + Python's `eval`_ function. + +.. _`eval`: https://docs.python.org/3.6/library/functions.html#eval - However, if the "-k" argument is a simple string, no such restrictions - apply. Also "-k 'not STRING'" has no restrictions. You can also - specify numbers like "-k 1.3" to match tests which are parametrized - with the float "1.3". + + However, if the ``"-k"`` argument is a simple string, no such restrictions + apply. Also ``"-k 'not STRING'"`` has no restrictions. You can also + specify numbers like ``"-k 1.3"`` to match tests which are parametrized + with the float ``"1.3"``. Registering markers ------------------------------------- @@ -198,12 +202,14 @@ Registering markers for your test suite is simple:: You can ask which markers exist for your test suite - the list includes our just defined ``webtest`` markers:: - $ py.test --markers + $ pytest --markers @pytest.mark.webtest: mark a test as a webtest. + @pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test. + @pytest.mark.skipif(condition): skip the given test function if eval(condition) results in a True value. Evaluation happens within the module global context. Example: skipif('sys.platform == "win32"') skips the test if we are on the win32 platform. see http://pytest.org/latest/skipping.html - @pytest.mark.xfail(condition, reason=None, run=True, raises=None): mark the the test function as an expected failure if eval(condition) has a True value. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See http://pytest.org/latest/skipping.html + @pytest.mark.xfail(condition, reason=None, run=True, raises=None, strict=False): mark the test function as an expected failure if eval(condition) has a True value. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See http://pytest.org/latest/skipping.html @pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see http://pytest.org/latest/parametrize.html for more info and examples. @@ -221,13 +227,12 @@ For an example on how to add and work with markers from a plugin, see It is recommended to explicitly register markers so that: - * there is one place in your test suite defining your markers + * There is one place in your test suite defining your markers - * asking for existing markers via ``py.test --markers`` gives good output + * Asking for existing markers via ``pytest --markers`` gives good output - * typos in function markers are treated as an error if you use - the ``--strict`` option. Future versions of ``pytest`` are probably - going to start treating non-registered markers as errors at some point. + * Typos in function markers are treated as an error if you use + the ``--strict`` option. .. _`scoped-marking`: @@ -240,7 +245,7 @@ its test methods:: # content of test_mark_classlevel.py import pytest @pytest.mark.webtest - class TestClass: + class TestClass(object): def test_startup(self): pass def test_startup_and_more(self): @@ -254,14 +259,14 @@ To remain backward-compatible with Python 2.4 you can also set a import pytest - class TestClass: + class TestClass(object): pytestmark = pytest.mark.webtest or if you need to use multiple markers you can use a list:: import pytest - class TestClass: + class TestClass(object): pytestmark = [pytest.mark.webtest, pytest.mark.slowtest] You can also set a module level marker:: @@ -348,36 +353,38 @@ A test file using this local plugin:: and an example invocations specifying a different environment than what the test needs:: - $ py.test -E stage2 - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items + $ pytest -E stage2 + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item - test_someenv.py s + test_someenv.py s [100%] - ======= 1 skipped in 0.12 seconds ======== + ======================== 1 skipped in 0.12 seconds ========================= and here is one that specifies exactly the environment needed:: - $ py.test -E stage1 - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items + $ pytest -E stage1 + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item - test_someenv.py . + test_someenv.py . [100%] - ======= 1 passed in 0.12 seconds ======== + ========================= 1 passed in 0.12 seconds ========================= The ``--markers`` option always gives you a list of available markers:: - $ py.test --markers + $ pytest --markers @pytest.mark.env(name): mark test to run only on named environment + @pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test. + @pytest.mark.skipif(condition): skip the given test function if eval(condition) results in a True value. Evaluation happens within the module global context. Example: skipif('sys.platform == "win32"') skips the test if we are on the win32 platform. see http://pytest.org/latest/skipping.html - @pytest.mark.xfail(condition, reason=None, run=True, raises=None): mark the the test function as an expected failure if eval(condition) has a True value. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See http://pytest.org/latest/skipping.html + @pytest.mark.xfail(condition, reason=None, run=True, raises=None, strict=False): mark the test function as an expected failure if eval(condition) has a True value. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See http://pytest.org/latest/skipping.html @pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see http://pytest.org/latest/parametrize.html for more info and examples. @@ -388,6 +395,49 @@ The ``--markers`` option always gives you a list of available markers:: @pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. +.. _`passing callables to custom markers`: + +Passing a callable to custom markers +-------------------------------------------- + +.. regendoc:wipe + +Below is the config file that will be used in the next examples:: + + # content of conftest.py + import sys + + def pytest_runtest_setup(item): + marker = item.get_marker('my_marker') + if marker is not None: + for info in marker: + print('Marker info name={} args={} kwars={}'.format(info.name, info.args, info.kwargs)) + sys.stdout.flush() + +A custom marker can have its argument set, i.e. ``args`` and ``kwargs`` properties, defined by either invoking it as a callable or using ``pytest.mark.MARKER_NAME.with_args``. These two methods achieve the same effect most of the time. + +However, if there is a callable as the single positional argument with no keyword arguments, using the ``pytest.mark.MARKER_NAME(c)`` will not pass ``c`` as a positional argument but decorate ``c`` with the custom marker (see :ref:`MarkDecorator `). Fortunately, ``pytest.mark.MARKER_NAME.with_args`` comes to the rescue:: + + # content of test_custom_marker.py + import pytest + + def hello_world(*args, **kwargs): + return 'Hello World' + + @pytest.mark.my_marker.with_args(hello_world) + def test_with_args(): + pass + +The output is as follows:: + + $ pytest -q -s + Marker info name=my_marker args=(,) kwars={} + . [100%] + 1 passed in 0.12 seconds + +We can see that the custom marker has its argument set extended with the function ``hello_world``. This is the key difference between creating a custom marker as a callable, which invokes ``__call__`` behind the scenes, and using ``with_args``. + + Reading markers which were set from multiple places ---------------------------------------------------- @@ -403,7 +453,7 @@ code you can read over all such settings. Example:: pytestmark = pytest.mark.glob("module", x=1) @pytest.mark.glob("class", x=2) - class TestClass: + class TestClass(object): @pytest.mark.glob("function", x=3) def test_something(self): pass @@ -423,11 +473,11 @@ test function. From a conftest file we can read it like this:: Let's run this without capturing output and see what we get:: - $ py.test -q -s + $ pytest -q -s glob args=('function',) kwargs={'x': 3} glob args=('class',) kwargs={'x': 2} glob args=('module',) kwargs={'x': 1} - . + . [100%] 1 passed in 0.12 seconds marking platform specific tests with pytest @@ -446,7 +496,7 @@ for your particular platform, you could use the following plugin:: import sys import pytest - ALL = set("darwin linux2 win32".split()) + ALL = set("darwin linux win32".split()) def pytest_runtest_setup(item): if isinstance(item, item.Function): @@ -466,7 +516,7 @@ Let's do a little test file to show how this looks like:: def test_if_apple_is_evil(): pass - @pytest.mark.linux2 + @pytest.mark.linux def test_if_linux_works(): pass @@ -477,32 +527,32 @@ Let's do a little test file to show how this looks like:: def test_runs_everywhere(): pass -then you will see two test skipped and two executed tests as expected:: +then you will see two tests skipped and two executed tests as expected:: - $ py.test -rs # this option reports skip reasons - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest -rs # this option reports skip reasons + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 4 items - test_plat.py sss. - ======= short test summary info ======== - SKIP [3] $REGENDOC_TMPDIR/conftest.py:12: cannot run on platform linux + test_plat.py s.s. [100%] + ========================= short test summary info ========================== + SKIP [2] $REGENDOC_TMPDIR/conftest.py:13: cannot run on platform linux - ======= 1 passed, 3 skipped in 0.12 seconds ======== + =================== 2 passed, 2 skipped in 0.12 seconds ==================== Note that if you specify a platform via the marker-command line option like this:: - $ py.test -m linux2 - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest -m linux + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 4 items - test_plat.py s + test_plat.py . [100%] - ======= 3 tests deselected by "-m 'linux2'" ======== - ======= 1 skipped, 3 deselected in 0.12 seconds ======== + ============================ 3 tests deselected ============================ + ================== 1 passed, 3 deselected in 0.12 seconds ================== then the unmarked-tests will not be run. It is thus a way to restrict the run to the specific tests. @@ -545,48 +595,48 @@ We want to dynamically define two markers and can do it in a We can now use the ``-m option`` to select one set:: - $ py.test -m interface --tb=short - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest -m interface --tb=short + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 4 items - test_module.py FF + test_module.py FF [100%] - ======= FAILURES ======== - _______ test_interface_simple ________ + ================================= FAILURES ================================= + __________________________ test_interface_simple ___________________________ test_module.py:3: in test_interface_simple assert 0 E assert 0 - _______ test_interface_complex ________ + __________________________ test_interface_complex __________________________ test_module.py:6: in test_interface_complex assert 0 E assert 0 - ======= 2 tests deselected by "-m 'interface'" ======== - ======= 2 failed, 2 deselected in 0.12 seconds ======== + ============================ 2 tests deselected ============================ + ================== 2 failed, 2 deselected in 0.12 seconds ================== or to select both "event" and "interface" tests:: - $ py.test -m "interface or event" --tb=short - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest -m "interface or event" --tb=short + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 4 items - test_module.py FFF + test_module.py FFF [100%] - ======= FAILURES ======== - _______ test_interface_simple ________ + ================================= FAILURES ================================= + __________________________ test_interface_simple ___________________________ test_module.py:3: in test_interface_simple assert 0 E assert 0 - _______ test_interface_complex ________ + __________________________ test_interface_complex __________________________ test_module.py:6: in test_interface_complex assert 0 E assert 0 - _______ test_event_simple ________ + ____________________________ test_event_simple _____________________________ test_module.py:9: in test_event_simple assert 0 E assert 0 - ======= 1 tests deselected by "-m 'interface or event'" ======== - ======= 3 failed, 1 deselected in 0.12 seconds ======== + ============================ 1 tests deselected ============================ + ================== 3 failed, 1 deselected in 0.12 seconds ================== diff --git a/doc/en/example/multipython.py b/doc/en/example/multipython.py index 66a368a1267aef9..66079be7e37c056 100644 --- a/doc/en/example/multipython.py +++ b/doc/en/example/multipython.py @@ -6,7 +6,7 @@ import pytest import _pytest._code -pythonlist = ['python2.6', 'python2.7', 'python3.3'] +pythonlist = ['python2.7', 'python3.4', 'python3.5'] @pytest.fixture(params=pythonlist) def python1(request, tmpdir): picklefile = tmpdir.join("data.pickle") @@ -16,7 +16,7 @@ def python1(request, tmpdir): def python2(request, python1): return Python(request.param, python1.picklefile) -class Python: +class Python(object): def __init__(self, version, picklefile): self.pythonpath = py.path.local.sysfind(version) if not self.pythonpath: diff --git a/doc/en/example/nonpython.rst b/doc/en/example/nonpython.rst index 6437e398457d8ee..cf72c7219e154d5 100644 --- a/doc/en/example/nonpython.rst +++ b/doc/en/example/nonpython.rst @@ -25,20 +25,20 @@ You can create a simple example file: and if you installed `PyYAML`_ or a compatible YAML-parser you can now execute the test specification:: - nonpython $ py.test test_simple.yml - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR/nonpython, inifile: + nonpython $ pytest test_simple.yml + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR/nonpython, inifile: collected 2 items - test_simple.yml F. + test_simple.yml F. [100%] - ======= FAILURES ======== - _______ usecase: hello ________ + ================================= FAILURES ================================= + ______________________________ usecase: hello ______________________________ usecase execution failed spec failed: 'some': 'other' no further details known at this point. - ======= 1 failed, 1 passed in 0.12 seconds ======== + ==================== 1 failed, 1 passed in 0.12 seconds ==================== .. regendoc:wipe @@ -57,35 +57,35 @@ your own domain specific testing language this way. ``reportinfo()`` is used for representing the test location and is also consulted when reporting in ``verbose`` mode:: - nonpython $ py.test -v - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + nonpython $ pytest -v + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR/nonpython, inifile: + rootdir: $REGENDOC_TMPDIR/nonpython, inifile: collecting ... collected 2 items - test_simple.yml::hello FAILED - test_simple.yml::ok PASSED + test_simple.yml::hello FAILED [ 50%] + test_simple.yml::ok PASSED [100%] - ======= FAILURES ======== - _______ usecase: hello ________ + ================================= FAILURES ================================= + ______________________________ usecase: hello ______________________________ usecase execution failed spec failed: 'some': 'other' no further details known at this point. - ======= 1 failed, 1 passed in 0.12 seconds ======== + ==================== 1 failed, 1 passed in 0.12 seconds ==================== .. regendoc:wipe While developing your custom test collection and execution it's also interesting to just look at the collection tree:: - nonpython $ py.test --collect-only - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR/nonpython, inifile: + nonpython $ pytest --collect-only + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR/nonpython, inifile: collected 2 items - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= diff --git a/doc/en/example/nonpython/conftest.py b/doc/en/example/nonpython/conftest.py index 2406e8f10bb9810..baff300155093e2 100644 --- a/doc/en/example/nonpython/conftest.py +++ b/doc/en/example/nonpython/conftest.py @@ -10,7 +10,7 @@ class YamlFile(pytest.File): def collect(self): import yaml # we need a yaml parser, e.g. PyYAML raw = yaml.safe_load(self.fspath.open()) - for name, spec in raw.items(): + for name, spec in sorted(raw.items()): yield YamlItem(name, self, spec) class YamlItem(pytest.Item): @@ -19,7 +19,7 @@ def __init__(self, name, parent, spec): self.spec = spec def runtest(self): - for name, value in self.spec.items(): + for name, value in sorted(self.spec.items()): # some custom test execution (dumb example follows) if name != value: raise YamlException(self, name, value) diff --git a/doc/en/example/parametrize.rst b/doc/en/example/parametrize.rst index 5d637ffcbf9f033..dd01b25277a8aa0 100644 --- a/doc/en/example/parametrize.rst +++ b/doc/en/example/parametrize.rst @@ -36,7 +36,7 @@ Now we add a test configuration like this:: def pytest_generate_tests(metafunc): if 'param1' in metafunc.fixturenames: - if metafunc.config.option.all: + if metafunc.config.getoption('all'): end = 5 else: end = 2 @@ -44,17 +44,17 @@ Now we add a test configuration like this:: This means that we only run 2 tests if we do not pass ``--all``:: - $ py.test -q test_compute.py - .. + $ pytest -q test_compute.py + .. [100%] 2 passed in 0.12 seconds We run only two computations, so we see two dots. let's run the full monty:: - $ py.test -q --all - ....F - ======= FAILURES ======== - _______ test_compute[4] ________ + $ pytest -q --all + ....F [100%] + ================================= FAILURES ================================= + _____________________________ test_compute[4] ______________________________ param1 = 4 @@ -116,6 +116,15 @@ the argument name:: diff = a - b assert diff == expected + @pytest.mark.parametrize("a,b,expected", [ + pytest.param(datetime(2001, 12, 12), datetime(2001, 12, 11), + timedelta(1), id='forward'), + pytest.param(datetime(2001, 12, 11), datetime(2001, 12, 12), + timedelta(-1), id='backward'), + ]) + def test_timedistance_v3(a, b, expected): + diff = a - b + assert diff == expected In ``test_timedistance_v0``, we let pytest generate the test IDs. @@ -128,11 +137,11 @@ label generated by ``idfn``, but because we didn't generate a label for ``timede objects, they are still using the default pytest representation:: - $ py.test test_time.py --collect-only - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 6 items + $ pytest test_time.py --collect-only + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 8 items @@ -140,8 +149,13 @@ objects, they are still using the default pytest representation:: + + - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= + +In ``test_timedistance_v3``, we used ``pytest.param`` to specify the test IDs +together with the actual data, instead of listing them separately. A quick port of "testscenarios" ------------------------------------ @@ -168,7 +182,7 @@ only have to work a bit to construct the correct arguments for pytest's scenario1 = ('basic', {'attribute': 'value'}) scenario2 = ('advanced', {'attribute': 'value2'}) - class TestSampleWithScenarios: + class TestSampleWithScenarios(object): scenarios = [scenario1, scenario2] def test_demo1(self, attribute): @@ -179,23 +193,23 @@ only have to work a bit to construct the correct arguments for pytest's this is a fully self-contained example which you can run with:: - $ py.test test_scenarios.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest test_scenarios.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 4 items - test_scenarios.py .... + test_scenarios.py .... [100%] - ======= 4 passed in 0.12 seconds ======== + ========================= 4 passed in 0.12 seconds ========================= If you just collect tests you'll also nicely see 'advanced' and 'basic' as variants for the test function:: - $ py.test --collect-only test_scenarios.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest --collect-only test_scenarios.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 4 items @@ -205,7 +219,7 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= Note that we told ``metafunc.parametrize()`` that your scenario values should be considered class-scoped. With pytest-2.3 this leads to a @@ -241,9 +255,9 @@ creates a database object for the actual test invocations:: if 'db' in metafunc.fixturenames: metafunc.parametrize("db", ['d1', 'd2'], indirect=True) - class DB1: + class DB1(object): "one database object" - class DB2: + class DB2(object): "alternative database object" @pytest.fixture @@ -257,23 +271,23 @@ creates a database object for the actual test invocations:: Let's first see how it looks like at collection time:: - $ py.test test_backends.py --collect-only - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest test_backends.py --collect-only + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 2 items - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= And then when we run the test:: - $ py.test -q test_backends.py - .F - ======= FAILURES ======== - _______ test_db_initialized[d2] ________ + $ pytest -q test_backends.py + .F [100%] + ================================= FAILURES ================================= + _________________________ test_db_initialized[d2] __________________________ db = @@ -318,15 +332,15 @@ will be passed to respective fixture function:: The result of this test will be successful:: - $ py.test test_indirect_list.py --collect-only - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items + $ pytest test_indirect_list.py --collect-only + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= .. regendoc:wipe @@ -336,7 +350,7 @@ Parametrizing test methods through per-class configuration .. _`unittest parametrizer`: https://github.com/testing-cabal/unittest-ext/blob/master/params.py -Here is an example ``pytest_generate_function`` function implementing a +Here is an example ``pytest_generate_tests`` function implementing a parametrization scheme similar to Michael Foord's `unittest parametrizer`_ but in a lot less code:: @@ -346,11 +360,11 @@ parametrizer`_ but in a lot less code:: def pytest_generate_tests(metafunc): # called once per each test function funcarglist = metafunc.cls.params[metafunc.function.__name__] - argnames = list(funcarglist[0]) + argnames = sorted(funcarglist[0]) metafunc.parametrize(argnames, [[funcargs[name] for name in argnames] for funcargs in funcarglist]) - class TestClass: + class TestClass(object): # a map specifying multiple argument sets for a test method params = { 'test_equals': [dict(a=1, b=2), dict(a=3, b=3), ], @@ -366,10 +380,10 @@ parametrizer`_ but in a lot less code:: Our test generator looks up a class-level definition which specifies which argument sets to use for each test function. Let's run it:: - $ py.test -q - F.. - ======= FAILURES ======== - _______ TestClass.test_equals[1-2] ________ + $ pytest -q + F.. [100%] + ================================= FAILURES ================================= + ________________________ TestClass.test_equals[1-2] ________________________ self = , a = 1, b = 2 @@ -396,12 +410,9 @@ is to be run with different sets of arguments for its three arguments: Running it results in some skips if we don't have all the python interpreters installed and otherwise runs all combinations (5 interpreters times 5 interpreters times 3 objects to serialize/deserialize):: - . $ py.test -rs -q multipython.py - ssssssssssss...ssssssssssss - ======= short test summary info ======== - SKIP [12] $REGENDOC_TMPDIR/CWD/multipython.py:23: 'python3.3' not found - SKIP [12] $REGENDOC_TMPDIR/CWD/multipython.py:23: 'python2.6' not found - 3 passed, 24 skipped in 0.12 seconds + . $ pytest -rs -q multipython.py + ........................... [100%] + 27 passed in 0.12 seconds Indirect parametrization of optional implementations/imports -------------------------------------------------------------------- @@ -446,17 +457,17 @@ And finally a little test module:: If you run this with reporting for skips enabled:: - $ py.test -rs test_module.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest -rs test_module.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 2 items - test_module.py .s - ======= short test summary info ======== - SKIP [1] $REGENDOC_TMPDIR/conftest.py:10: could not import 'opt2' + test_module.py .s [100%] + ========================= short test summary info ========================== + SKIP [1] $REGENDOC_TMPDIR/conftest.py:11: could not import 'opt2' - ======= 1 passed, 1 skipped in 0.12 seconds ======== + =================== 1 passed, 1 skipped in 0.12 seconds ==================== You'll see that we don't have a ``opt2`` module and thus the second test run of our ``test_func1`` was skipped. A few notes: @@ -472,4 +483,54 @@ of our ``test_func1`` was skipped. A few notes: values as well. +Set marks or test ID for individual parametrized test +-------------------------------------------------------------------- + +Use ``pytest.param`` to apply marks or set test ID to individual parametrized test. +For example:: + + # content of test_pytest_param_example.py + import pytest + @pytest.mark.parametrize('test_input,expected', [ + ('3+5', 8), + pytest.param('1+7', 8, + marks=pytest.mark.basic), + pytest.param('2+4', 6, + marks=pytest.mark.basic, + id='basic_2+4'), + pytest.param('6*9', 42, + marks=[pytest.mark.basic, pytest.mark.xfail], + id='basic_6*9'), + ]) + def test_eval(test_input, expected): + assert eval(test_input) == expected + +In this example, we have 4 parametrized tests. Except for the first test, +we mark the rest three parametrized tests with the custom marker ``basic``, +and for the fourth test we also use the built-in mark ``xfail`` to indicate this +test is expected to fail. For explicitness, we set test ids for some tests. + +Then run ``pytest`` with verbose mode and with only the ``basic`` marker:: + + pytest -v -m basic + ============================================ test session starts ============================================= + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 4 items + + test_pytest_param_example.py::test_eval[1+7-8] PASSED + test_pytest_param_example.py::test_eval[basic_2+4] PASSED + test_pytest_param_example.py::test_eval[basic_6*9] xfail + ========================================== short test summary info =========================================== + XFAIL test_pytest_param_example.py::test_eval[basic_6*9] + + ============================================= 1 tests deselected ============================================= + +As the result: +- Four tests were collected +- One test was deselected because it doesn't have the ``basic`` mark. +- Three tests with the ``basic`` mark was selected. +- The test ``test_eval[1+7-8]`` passed, but the name is autogenerated and confusing. +- The test ``test_eval[basic_2+4]`` passed. +- The test ``test_eval[basic_6*9]`` was expected to fail and did fail. diff --git a/doc/en/example/pythoncollection.py b/doc/en/example/pythoncollection.py index 05858eb854b49c9..9c4bd31cea026a8 100644 --- a/doc/en/example/pythoncollection.py +++ b/doc/en/example/pythoncollection.py @@ -1,10 +1,10 @@ -# run this with $ py.test --collect-only test_collectonly.py +# run this with $ pytest --collect-only test_collectonly.py # def test_function(): pass -class TestClass: +class TestClass(object): def test_method(self): pass def test_anothermethod(self): diff --git a/doc/en/example/pythoncollection.rst b/doc/en/example/pythoncollection.rst index 5faf4c6c8adbf57..c9d31d7c4208630 100644 --- a/doc/en/example/pythoncollection.rst +++ b/doc/en/example/pythoncollection.rst @@ -9,19 +9,19 @@ by passing the ``--ignore=path`` option on the cli. ``pytest`` allows multiple ``--ignore`` options. Example:: tests/ - ├── example - │   ├── test_example_01.py - │   ├── test_example_02.py - │   └── test_example_03.py - ├── foobar - │   ├── test_foobar_01.py - │   ├── test_foobar_02.py - │   └── test_foobar_03.py - └── hello - └── world - ├── test_world_01.py - ├── test_world_02.py - └── test_world_03.py + |-- example + | |-- test_example_01.py + | |-- test_example_02.py + | '-- test_example_03.py + |-- foobar + | |-- test_foobar_01.py + | |-- test_foobar_02.py + | '-- test_foobar_03.py + '-- hello + '-- world + |-- test_world_01.py + |-- test_world_02.py + '-- test_world_03.py Now if you invoke ``pytest`` with ``--ignore=tests/foobar/test_foobar_03.py --ignore=tests/hello/``, you will see that ``pytest`` only collects test-modules, which do not match the patterns specified:: @@ -40,12 +40,46 @@ you will see that ``pytest`` only collects test-modules, which do not match the ======= 5 passed in 0.02 seconds ======= +Keeping duplicate paths specified from command line +---------------------------------------------------- + +Default behavior of ``pytest`` is to ignore duplicate paths specified from the command line. +Example:: + + py.test path_a path_a + + ... + collected 1 item + ... + +Just collect tests once. + +To collect duplicate tests, use the ``--keep-duplicates`` option on the cli. +Example:: + + py.test --keep-duplicates path_a path_a + + ... + collected 2 items + ... + +As the collector just works on directories, if you specify twice a single test file, ``pytest`` will +still collect it twice, no matter if the ``--keep-duplicates`` is not specified. +Example:: + + py.test test_a.py test_a.py + + ... + collected 2 items + ... + + Changing directory recursion ----------------------------------------------------- -You can set the :confval:`norecursedirs` option in an ini-file, for example your ``setup.cfg`` in the project root directory:: +You can set the :confval:`norecursedirs` option in an ini-file, for example your ``pytest.ini`` in the project root directory:: - # content of setup.cfg + # content of pytest.ini [pytest] norecursedirs = .svn _build tmp* @@ -60,8 +94,9 @@ You can configure different naming conventions by setting the :confval:`python_files`, :confval:`python_classes` and :confval:`python_functions` configuration options. Example:: - # content of setup.cfg - # can also be defined in in tox.ini or pytest.ini file + # content of pytest.ini + # can also be defined in tox.ini or setup.cfg file, although the section + # name in setup.cfg files should be "tool:pytest" [pytest] python_files=check_*.py python_classes=Check @@ -72,7 +107,7 @@ This would make ``pytest`` look for tests in files that match the ``check_* that match ``*_check``. For example, if we have:: # content of check_myapp.py - class CheckMyApp: + class CheckMyApp(object): def simple_check(self): pass def complex_check(self): @@ -80,10 +115,10 @@ that match ``*_check``. For example, if we have:: then the test collection looks like this:: - $ py.test --collect-only - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: setup.cfg + $ pytest --collect-only + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini collected 2 items @@ -91,7 +126,7 @@ then the test collection looks like this:: - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= .. note:: @@ -107,7 +142,7 @@ interpreting arguments as python package names, deriving their file system path and then running the test. For example if you have unittest2 installed you can type:: - py.test --pyargs unittest2.test.test_skipping -q + pytest --pyargs unittest2.test.test_skipping -q which would run the respective test module. Like with other options, through an ini-file and the :confval:`addopts` option you @@ -117,7 +152,7 @@ can make this change more permanently:: [pytest] addopts = --pyargs -Now a simple invocation of ``py.test NAME`` will check +Now a simple invocation of ``pytest NAME`` will check if NAME exists as an importable package/module and otherwise treat it as a filesystem path. @@ -126,9 +161,9 @@ Finding out what is collected You can always peek at the collection tree without running tests like this:: - . $ py.test --collect-only pythoncollection.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 + . $ pytest --collect-only pythoncollection.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini collected 3 items @@ -138,23 +173,25 @@ You can always peek at the collection tree without running tests like this:: - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= -customizing test collection to find all .py files ---------------------------------------------------------- +.. _customizing-test-collection: -.. regendoc:wipe +Customizing test collection +--------------------------- -You can easily instruct ``pytest`` to discover tests from every python file:: +.. regendoc:wipe +You can easily instruct ``pytest`` to discover tests from every Python file:: # content of pytest.ini [pytest] python_files = *.py -However, many projects will have a ``setup.py`` which they don't want to be imported. Moreover, there may files only importable by a specific python version. -For such cases you can dynamically define files to be ignored by listing -them in a ``conftest.py`` file:: +However, many projects will have a ``setup.py`` which they don't want to be +imported. Moreover, there may files only importable by a specific python +version. For such cases you can dynamically define files to be ignored by +listing them in a ``conftest.py`` file:: # content of conftest.py import sys @@ -163,7 +200,7 @@ them in a ``conftest.py`` file:: if sys.version_info[0] > 2: collect_ignore.append("pkg/module_py2.py") -And then if you have a module file like this:: +and then if you have a module file like this:: # content of pkg/module_py2.py def test_only_on_python2(): @@ -172,21 +209,31 @@ And then if you have a module file like this:: except Exception, e: pass -and a setup.py dummy file like this:: +and a ``setup.py`` dummy file like this:: # content of setup.py 0/0 # will raise exception if imported -then a pytest run on python2 will find the one test when run with a python2 -interpreters and will leave out the setup.py file:: +If you run with a Python 2 interpreter then you will find the one test and will +leave out the ``setup.py`` file:: - $ py.test --collect-only - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 + #$ pytest --collect-only + ====== test session starts ====== + platform linux2 -- Python 2.7.10, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini - collected 0 items - - ======= no tests ran in 0.12 seconds ======== + collected 1 items + + + + ====== no tests ran in 0.04 seconds ====== -If you run with a Python3 interpreter the moduled added through the conftest.py file will not be considered for test collection. +If you run with a Python 3 interpreter both the one test and the ``setup.py`` +file will be left out:: + $ pytest --collect-only + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini + collected 0 items + + ======================= no tests ran in 0.12 seconds ======================= diff --git a/doc/en/example/reportingdemo.rst b/doc/en/example/reportingdemo.rst index 28624aa07b89106..9edc02b3cd4bd5e 100644 --- a/doc/en/example/reportingdemo.rst +++ b/doc/en/example/reportingdemo.rst @@ -7,20 +7,18 @@ Demo of Python failure reports with pytest Here is a nice run of several tens of failures and how ``pytest`` presents things (unfortunately not showing the nice colors here in the HTML that you -get on the terminal - we are working on that): +get on the terminal - we are working on that):: -.. code-block:: python - - assertion $ py.test failure_demo.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR/assertion, inifile: + assertion $ pytest failure_demo.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR/assertion, inifile: collected 42 items - failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF + failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [100%] - ======= FAILURES ======== - _______ test_generative[0] ________ + ================================= FAILURES ================================= + ____________________________ test_generative[0] ____________________________ param1 = 3, param2 = 6 @@ -29,7 +27,7 @@ get on the terminal - we are working on that): E assert (3 * 2) < 6 failure_demo.py:16: AssertionError - _______ TestFailing.test_simple ________ + _________________________ TestFailing.test_simple __________________________ self = @@ -45,7 +43,7 @@ get on the terminal - we are working on that): E + and 43 = .g at 0xdeadbeef>() failure_demo.py:29: AssertionError - _______ TestFailing.test_simple_multiline ________ + ____________________ TestFailing.test_simple_multiline _____________________ self = @@ -65,7 +63,7 @@ get on the terminal - we are working on that): E assert 42 == 54 failure_demo.py:12: AssertionError - _______ TestFailing.test_not ________ + ___________________________ TestFailing.test_not ___________________________ self = @@ -77,44 +75,44 @@ get on the terminal - we are working on that): E + where 42 = .f at 0xdeadbeef>() failure_demo.py:39: AssertionError - _______ TestSpecialisedExplanations.test_eq_text ________ + _________________ TestSpecialisedExplanations.test_eq_text _________________ self = def test_eq_text(self): > assert 'spam' == 'eggs' - E assert 'spam' == 'eggs' + E AssertionError: assert 'spam' == 'eggs' E - spam E + eggs failure_demo.py:43: AssertionError - _______ TestSpecialisedExplanations.test_eq_similar_text ________ + _____________ TestSpecialisedExplanations.test_eq_similar_text _____________ self = def test_eq_similar_text(self): > assert 'foo 1 bar' == 'foo 2 bar' - E assert 'foo 1 bar' == 'foo 2 bar' + E AssertionError: assert 'foo 1 bar' == 'foo 2 bar' E - foo 1 bar E ? ^ E + foo 2 bar E ? ^ failure_demo.py:46: AssertionError - _______ TestSpecialisedExplanations.test_eq_multiline_text ________ + ____________ TestSpecialisedExplanations.test_eq_multiline_text ____________ self = def test_eq_multiline_text(self): > assert 'foo\nspam\nbar' == 'foo\neggs\nbar' - E assert 'foo\nspam\nbar' == 'foo\neggs\nbar' + E AssertionError: assert 'foo\nspam\nbar' == 'foo\neggs\nbar' E foo E - spam E + eggs E bar failure_demo.py:49: AssertionError - _______ TestSpecialisedExplanations.test_eq_long_text ________ + ______________ TestSpecialisedExplanations.test_eq_long_text _______________ self = @@ -122,7 +120,7 @@ get on the terminal - we are working on that): a = '1'*100 + 'a' + '2'*100 b = '1'*100 + 'b' + '2'*100 > assert a == b - E assert '111111111111...2222222222222' == '1111111111111...2222222222222' + E AssertionError: assert '111111111111...2222222222222' == '1111111111111...2222222222222' E Skipping 90 identical leading characters in diff, use -v to show E Skipping 91 identical trailing characters in diff, use -v to show E - 1111111111a222222222 @@ -131,7 +129,7 @@ get on the terminal - we are working on that): E ? ^ failure_demo.py:54: AssertionError - _______ TestSpecialisedExplanations.test_eq_long_text_multiline ________ + _________ TestSpecialisedExplanations.test_eq_long_text_multiline __________ self = @@ -139,23 +137,19 @@ get on the terminal - we are working on that): a = '1\n'*100 + 'a' + '2\n'*100 b = '1\n'*100 + 'b' + '2\n'*100 > assert a == b - E assert '1\n1\n1\n1\n...n2\n2\n2\n2\n' == '1\n1\n1\n1\n1...n2\n2\n2\n2\n' + E AssertionError: assert '1\n1\n1\n1\n...n2\n2\n2\n2\n' == '1\n1\n1\n1\n1...n2\n2\n2\n2\n' E Skipping 190 identical leading characters in diff, use -v to show E Skipping 191 identical trailing characters in diff, use -v to show E 1 E 1 E 1 E 1 - E 1 - E - a2 - E + b2 - E 2 - E 2 - E 2 - E 2 + E 1... + E + E ...Full output truncated (7 lines hidden), use '-vv' to show failure_demo.py:59: AssertionError - _______ TestSpecialisedExplanations.test_eq_list ________ + _________________ TestSpecialisedExplanations.test_eq_list _________________ self = @@ -166,7 +160,7 @@ get on the terminal - we are working on that): E Use -v to get the full diff failure_demo.py:62: AssertionError - _______ TestSpecialisedExplanations.test_eq_list_long ________ + ______________ TestSpecialisedExplanations.test_eq_list_long _______________ self = @@ -179,41 +173,43 @@ get on the terminal - we are working on that): E Use -v to get the full diff failure_demo.py:67: AssertionError - _______ TestSpecialisedExplanations.test_eq_dict ________ + _________________ TestSpecialisedExplanations.test_eq_dict _________________ self = def test_eq_dict(self): > assert {'a': 0, 'b': 1, 'c': 0} == {'a': 0, 'b': 2, 'd': 0} - E assert {'a': 0, 'b': 1, 'c': 0} == {'a': 0, 'b': 2, 'd': 0} - E Omitting 1 identical items, use -v to show + E AssertionError: assert {'a': 0, 'b': 1, 'c': 0} == {'a': 0, 'b': 2, 'd': 0} + E Omitting 1 identical items, use -vv to show E Differing items: E {'b': 1} != {'b': 2} E Left contains more items: E {'c': 0} E Right contains more items: - E {'d': 0} - E Use -v to get the full diff + E {'d': 0}... + E + E ...Full output truncated (2 lines hidden), use '-vv' to show failure_demo.py:70: AssertionError - _______ TestSpecialisedExplanations.test_eq_set ________ + _________________ TestSpecialisedExplanations.test_eq_set __________________ self = def test_eq_set(self): > assert set([0, 10, 11, 12]) == set([0, 20, 21]) - E assert set([0, 10, 11, 12]) == set([0, 20, 21]) + E AssertionError: assert {0, 10, 11, 12} == {0, 20, 21} E Extra items in the left set: E 10 E 11 E 12 E Extra items in the right set: E 20 - E 21 - E Use -v to get the full diff + E 21... + E + E ...Full output truncated (2 lines hidden), use '-vv' to show failure_demo.py:73: AssertionError - _______ TestSpecialisedExplanations.test_eq_longer_list ________ + _____________ TestSpecialisedExplanations.test_eq_longer_list ______________ self = @@ -224,7 +220,7 @@ get on the terminal - we are working on that): E Use -v to get the full diff failure_demo.py:76: AssertionError - _______ TestSpecialisedExplanations.test_in_list ________ + _________________ TestSpecialisedExplanations.test_in_list _________________ self = @@ -233,45 +229,46 @@ get on the terminal - we are working on that): E assert 1 in [0, 2, 3, 4, 5] failure_demo.py:79: AssertionError - _______ TestSpecialisedExplanations.test_not_in_text_multiline ________ + __________ TestSpecialisedExplanations.test_not_in_text_multiline __________ self = def test_not_in_text_multiline(self): text = 'some multiline\ntext\nwhich\nincludes foo\nand a\ntail' > assert 'foo' not in text - E assert 'foo' not in 'some multiline\ntext\nw...ncludes foo\nand a\ntail' + E AssertionError: assert 'foo' not in 'some multiline\ntext\nw...ncludes foo\nand a\ntail' E 'foo' is contained here: E some multiline E text E which E includes foo E ? +++ - E and a - E tail + E and a... + E + E ...Full output truncated (2 lines hidden), use '-vv' to show failure_demo.py:83: AssertionError - _______ TestSpecialisedExplanations.test_not_in_text_single ________ + ___________ TestSpecialisedExplanations.test_not_in_text_single ____________ self = def test_not_in_text_single(self): text = 'single foo line' > assert 'foo' not in text - E assert 'foo' not in 'single foo line' + E AssertionError: assert 'foo' not in 'single foo line' E 'foo' is contained here: E single foo line E ? +++ failure_demo.py:87: AssertionError - _______ TestSpecialisedExplanations.test_not_in_text_single_long ________ + _________ TestSpecialisedExplanations.test_not_in_text_single_long _________ self = def test_not_in_text_single_long(self): text = 'head ' * 50 + 'foo ' + 'tail ' * 20 > assert 'foo' not in text - E assert 'foo' not in 'head head head head hea...ail tail tail tail tail ' + E AssertionError: assert 'foo' not in 'head head head head hea...ail tail tail tail tail ' E 'foo' is contained here: E head head foo tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail E ? +++ @@ -284,13 +281,13 @@ get on the terminal - we are working on that): def test_not_in_text_single_long_term(self): text = 'head ' * 50 + 'f'*70 + 'tail ' * 20 > assert 'f'*70 not in text - E assert 'fffffffffff...ffffffffffff' not in 'head head he...l tail tail ' + E AssertionError: assert 'fffffffffff...ffffffffffff' not in 'head head he...l tail tail ' E 'ffffffffffffffffff...fffffffffffffffffff' is contained here: E head head fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffftail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail E ? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ failure_demo.py:95: AssertionError - _______ test_attribute ________ + ______________________________ test_attribute ______________________________ def test_attribute(): class Foo(object): @@ -301,18 +298,18 @@ get on the terminal - we are working on that): E + where 1 = .Foo object at 0xdeadbeef>.b failure_demo.py:102: AssertionError - _______ test_attribute_instance ________ + _________________________ test_attribute_instance __________________________ def test_attribute_instance(): class Foo(object): b = 1 > assert Foo().b == 2 - E assert 1 == 2 + E AssertionError: assert 1 == 2 E + where 1 = .Foo object at 0xdeadbeef>.b E + where .Foo object at 0xdeadbeef> = .Foo'>() failure_demo.py:108: AssertionError - _______ test_attribute_failure ________ + __________________________ test_attribute_failure __________________________ def test_attribute_failure(): class Foo(object): @@ -332,7 +329,7 @@ get on the terminal - we are working on that): E Exception: Failed to get attrib failure_demo.py:114: Exception - _______ test_attribute_multiple ________ + _________________________ test_attribute_multiple __________________________ def test_attribute_multiple(): class Foo(object): @@ -340,14 +337,14 @@ get on the terminal - we are working on that): class Bar(object): b = 2 > assert Foo().b == Bar().b - E assert 1 == 2 + E AssertionError: assert 1 == 2 E + where 1 = .Foo object at 0xdeadbeef>.b E + where .Foo object at 0xdeadbeef> = .Foo'>() E + and 2 = .Bar object at 0xdeadbeef>.b E + where .Bar object at 0xdeadbeef> = .Bar'>() failure_demo.py:125: AssertionError - _______ TestRaises.test_raises ________ + __________________________ TestRaises.test_raises __________________________ self = @@ -361,8 +358,8 @@ get on the terminal - we are working on that): > int(s) E ValueError: invalid literal for int() with base 10: 'qwe' - <0-codegen $PYTHON_PREFIX/lib/python3.4/site-packages/_pytest/python.py:1302>:1: ValueError - _______ TestRaises.test_raises_doesnt ________ + <0-codegen $PYTHON_PREFIX/lib/python3.5/site-packages/_pytest/python_api.py:580>:1: ValueError + ______________________ TestRaises.test_raises_doesnt _______________________ self = @@ -371,7 +368,7 @@ get on the terminal - we are working on that): E Failed: DID NOT RAISE failure_demo.py:137: Failed - _______ TestRaises.test_raise ________ + __________________________ TestRaises.test_raise ___________________________ self = @@ -380,13 +377,13 @@ get on the terminal - we are working on that): E ValueError: demo error failure_demo.py:140: ValueError - _______ TestRaises.test_tupleerror ________ + ________________________ TestRaises.test_tupleerror ________________________ self = def test_tupleerror(self): > a,b = [1] - E ValueError: need more than 1 value to unpack + E ValueError: not enough values to unpack (expected 2, got 1) failure_demo.py:143: ValueError ______ TestRaises.test_reinterpret_fails_with_print_for_the_fun_of_it ______ @@ -402,7 +399,7 @@ get on the terminal - we are working on that): failure_demo.py:148: TypeError --------------------------- Captured stdout call --------------------------- l is [1, 2, 3] - _______ TestRaises.test_some_error ________ + ________________________ TestRaises.test_some_error ________________________ self = @@ -411,7 +408,7 @@ get on the terminal - we are working on that): E NameError: name 'namenotexi' is not defined failure_demo.py:151: NameError - _______ test_dynamic_compile_shows_nicely ________ + ____________________ test_dynamic_compile_shows_nicely _____________________ def test_dynamic_compile_shows_nicely(): src = 'def foo():\n assert 1 == 0\n' @@ -427,10 +424,10 @@ get on the terminal - we are working on that): def foo(): > assert 1 == 0 - E assert 1 == 0 + E AssertionError <2-codegen 'abc-123' $REGENDOC_TMPDIR/assertion/failure_demo.py:163>:2: AssertionError - _______ TestMoreErrors.test_complex_error ________ + ____________________ TestMoreErrors.test_complex_error _____________________ self = @@ -454,17 +451,17 @@ get on the terminal - we are working on that): E assert 44 == 43 failure_demo.py:6: AssertionError - _______ TestMoreErrors.test_z1_unpack_error ________ + ___________________ TestMoreErrors.test_z1_unpack_error ____________________ self = def test_z1_unpack_error(self): l = [] > a,b = l - E ValueError: need more than 0 values to unpack + E ValueError: not enough values to unpack (expected 2, got 0) failure_demo.py:180: ValueError - _______ TestMoreErrors.test_z2_type_error ________ + ____________________ TestMoreErrors.test_z2_type_error _____________________ self = @@ -474,7 +471,7 @@ get on the terminal - we are working on that): E TypeError: 'int' object is not iterable failure_demo.py:184: TypeError - _______ TestMoreErrors.test_startswith ________ + ______________________ TestMoreErrors.test_startswith ______________________ self = @@ -482,11 +479,12 @@ get on the terminal - we are working on that): s = "123" g = "456" > assert s.startswith(g) - E assert ('456') - E + where = '123'.startswith + E AssertionError: assert False + E + where False = ('456') + E + where = '123'.startswith failure_demo.py:189: AssertionError - _______ TestMoreErrors.test_startswith_nested ________ + __________________ TestMoreErrors.test_startswith_nested ___________________ self = @@ -496,23 +494,25 @@ get on the terminal - we are working on that): def g(): return "456" > assert f().startswith(g()) - E assert ('456') - E + where = '123'.startswith - E + where '123' = .f at 0xdeadbeef>() - E + and '456' = .g at 0xdeadbeef>() + E AssertionError: assert False + E + where False = ('456') + E + where = '123'.startswith + E + where '123' = .f at 0xdeadbeef>() + E + and '456' = .g at 0xdeadbeef>() failure_demo.py:196: AssertionError - _______ TestMoreErrors.test_global_func ________ + _____________________ TestMoreErrors.test_global_func ______________________ self = def test_global_func(self): > assert isinstance(globf(42), float) - E assert isinstance(43, float) - E + where 43 = globf(42) + E assert False + E + where False = isinstance(43, float) + E + where 43 = globf(42) failure_demo.py:199: AssertionError - _______ TestMoreErrors.test_instance ________ + _______________________ TestMoreErrors.test_instance _______________________ self = @@ -523,7 +523,7 @@ get on the terminal - we are working on that): E + where 42 = .x failure_demo.py:203: AssertionError - _______ TestMoreErrors.test_compare ________ + _______________________ TestMoreErrors.test_compare ________________________ self = @@ -533,7 +533,7 @@ get on the terminal - we are working on that): E + where 11 = globf(10) failure_demo.py:206: AssertionError - _______ TestMoreErrors.test_try_finally ________ + _____________________ TestMoreErrors.test_try_finally ______________________ self = @@ -544,12 +544,12 @@ get on the terminal - we are working on that): E assert 1 == 0 failure_demo.py:211: AssertionError - _______ TestCustomAssertMsg.test_single_line ________ + ___________________ TestCustomAssertMsg.test_single_line ___________________ self = def test_single_line(self): - class A: + class A(object): a = 1 b = 2 > assert A.a == b, "A.a appears not to be b" @@ -558,12 +558,12 @@ get on the terminal - we are working on that): E + where 1 = .A'>.a failure_demo.py:222: AssertionError - _______ TestCustomAssertMsg.test_multiline ________ + ____________________ TestCustomAssertMsg.test_multiline ____________________ self = def test_multiline(self): - class A: + class A(object): a = 1 b = 2 > assert A.a == b, "A.a appears not to be b\n" \ @@ -575,12 +575,12 @@ get on the terminal - we are working on that): E + where 1 = .A'>.a failure_demo.py:228: AssertionError - _______ TestCustomAssertMsg.test_custom_repr ________ + ___________________ TestCustomAssertMsg.test_custom_repr ___________________ self = def test_custom_repr(self): - class JSON: + class JSON(object): a = 1 def __repr__(self): return "This is JSON\n{\n 'foo': 'bar'\n}" @@ -595,4 +595,10 @@ get on the terminal - we are working on that): E + where 1 = This is JSON\n{\n 'foo': 'bar'\n}.a failure_demo.py:238: AssertionError - ======= 42 failed in 0.12 seconds ======== + ============================= warnings summary ============================= + None + Metafunc.addcall is deprecated and scheduled to be removed in pytest 4.0. + Please use Metafunc.parametrize instead. + + -- Docs: http://doc.pytest.org/en/latest/warnings.html + ================== 42 failed, 1 warnings in 0.12 seconds =================== diff --git a/doc/en/example/simple.rst b/doc/en/example/simple.rst index be12d2afe4152b8..678a0db0094b92c 100644 --- a/doc/en/example/simple.rst +++ b/doc/en/example/simple.rst @@ -1,5 +1,4 @@ -.. highlightlang:: python Basic patterns and examples ========================================================== @@ -10,7 +9,9 @@ Pass different values to a test function, depending on command line options .. regendoc:wipe Suppose we want to write a test that depends on a command line option. -Here is a basic pattern to achieve this:: +Here is a basic pattern to achieve this: + +.. code-block:: python # content of test_sample.py def test_answer(cmdopt): @@ -22,7 +23,9 @@ Here is a basic pattern to achieve this:: For this to work we need to add a command line option and -provide the ``cmdopt`` through a :ref:`fixture function `:: +provide the ``cmdopt`` through a :ref:`fixture function `: + +.. code-block:: python # content of conftest.py import pytest @@ -37,10 +40,10 @@ provide the ``cmdopt`` through a :ref:`fixture function `:: Let's run this without supplying our new option:: - $ py.test -q test_sample.py - F - ======= FAILURES ======== - _______ test_answer ________ + $ pytest -q test_sample.py + F [100%] + ================================= FAILURES ================================= + _______________________________ test_answer ________________________________ cmdopt = 'type1' @@ -59,10 +62,10 @@ Let's run this without supplying our new option:: And now with supplying a command line option:: - $ py.test -q --cmdopt=type2 - F - ======= FAILURES ======== - _______ test_answer ________ + $ pytest -q --cmdopt=type2 + F [100%] + ================================= FAILURES ================================= + _______________________________ test_answer ________________________________ cmdopt = 'type2' @@ -91,7 +94,9 @@ Dynamically adding command line options Through :confval:`addopts` you can statically add command line options for your project. You can also dynamically modify -the command line arguments before they get processed:: +the command line arguments before they get processed: + +.. code-block:: python # content of conftest.py import sys @@ -101,18 +106,18 @@ the command line arguments before they get processed:: num = max(multiprocessing.cpu_count() / 2, 1) args[:] = ["-n", str(num)] + args -If you have the :ref:`xdist plugin ` installed +If you have the `xdist plugin `_ installed you will now always perform test runs using a number of subprocesses close to your CPU. Running in an empty directory with the above conftest.py:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 0 items - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= .. _`excontrolskip`: @@ -122,61 +127,67 @@ Control skipping of tests according to command line option .. regendoc:wipe Here is a ``conftest.py`` file adding a ``--runslow`` command -line option to control skipping of ``slow`` marked tests:: +line option to control skipping of ``pytest.mark.slow`` marked tests: + +.. code-block:: python # content of conftest.py import pytest def pytest_addoption(parser): parser.addoption("--runslow", action="store_true", - help="run slow tests") + default=False, help="run slow tests") -We can now write a test module like this:: + def pytest_collection_modifyitems(config, items): + if config.getoption("--runslow"): + # --runslow given in cli: do not skip slow tests + return + skip_slow = pytest.mark.skip(reason="need --runslow option to run") + for item in items: + if "slow" in item.keywords: + item.add_marker(skip_slow) - # content of test_module.py - - import pytest +We can now write a test module like this: +.. code-block:: python - slow = pytest.mark.skipif( - not pytest.config.getoption("--runslow"), - reason="need --runslow option to run" - ) + # content of test_module.py + import pytest def test_func_fast(): pass - @slow + @pytest.mark.slow def test_func_slow(): pass and when running it will see a skipped "slow" test:: - $ py.test -rs # "-rs" means report details on the little 's' - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest -rs # "-rs" means report details on the little 's' + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 2 items - test_module.py .s - ======= short test summary info ======== - SKIP [1] test_module.py:14: need --runslow option to run + test_module.py .s [100%] + ========================= short test summary info ========================== + SKIP [1] test_module.py:8: need --runslow option to run - ======= 1 passed, 1 skipped in 0.12 seconds ======== + =================== 1 passed, 1 skipped in 0.12 seconds ==================== Or run it including the ``slow`` marked test:: - $ py.test --runslow - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest --runslow + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 2 items - test_module.py .. + test_module.py .. [100%] - ======= 2 passed in 0.12 seconds ======== + ========================= 2 passed in 0.12 seconds ========================= Writing well integrated assertion helpers -------------------------------------------------- @@ -187,7 +198,9 @@ If you have a test helper function called from a test you can use the ``pytest.fail`` marker to fail a test with a certain message. The test support function will not show up in the traceback if you set the ``__tracebackhide__`` option somewhere in the helper function. -Example:: +Example: + +.. code-block:: python # content of test_checkconfig.py import pytest @@ -201,13 +214,13 @@ Example:: The ``__tracebackhide__`` setting influences ``pytest`` showing of tracebacks: the ``checkconfig`` function will not be shown -unless the ``--fulltrace`` command line option is specified. +unless the ``--full-trace`` command line option is specified. Let's run our little function:: - $ py.test -q test_checkconfig.py - F - ======= FAILURES ======== - _______ test_something ________ + $ pytest -q test_checkconfig.py + F [100%] + ================================= FAILURES ================================= + ______________________________ test_something ______________________________ def test_something(): > checkconfig(42) @@ -216,6 +229,30 @@ Let's run our little function:: test_checkconfig.py:8: Failed 1 failed in 0.12 seconds +If you only want to hide certain exceptions, you can set ``__tracebackhide__`` +to a callable which gets the ``ExceptionInfo`` object. You can for example use +this to make sure unexpected exception types aren't hidden: + +.. code-block:: python + + import operator + import pytest + + class ConfigException(Exception): + pass + + def checkconfig(x): + __tracebackhide__ = operator.methodcaller('errisinstance', ConfigException) + if not hasattr(x, "config"): + raise ConfigException("not configured: %s" %(x,)) + + def test_something(): + checkconfig(42) + +This will avoid hiding the exception traceback on unrelated exceptions (i.e. +bugs in assertion helpers). + + Detect if running from within a pytest run -------------------------------------------------------------- @@ -224,7 +261,9 @@ Detect if running from within a pytest run Usually it is a bad idea to make application code behave differently if called from a test. But if you absolutely must find out if your application code is -running from a test you can do something like this:: +running from a test you can do something like this: + +.. code-block:: python # content of conftest.py @@ -233,9 +272,12 @@ running from a test you can do something like this:: sys._called_from_test = True def pytest_unconfigure(config): + import sys del sys._called_from_test -and then check for the ``sys._called_from_test`` flag:: +and then check for the ``sys._called_from_test`` flag: + +.. code-block:: python if hasattr(sys, '_called_from_test'): # called from within a test run @@ -251,7 +293,9 @@ Adding info to test report header .. regendoc:wipe -It's easy to present extra information in a ``pytest`` run:: +It's easy to present extra information in a ``pytest`` run: + +.. code-block:: python # content of conftest.py @@ -260,50 +304,51 @@ It's easy to present extra information in a ``pytest`` run:: which will add the string to the test header accordingly:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y project deps: mylib-1.1 - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collected 0 items - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= .. regendoc:wipe -You can also return a list of strings which will be considered as several -lines of information. You can of course also make the amount of reporting -information on e.g. the value of ``config.option.verbose`` so that -you present more information appropriately:: +It is also possible to return a list of strings which will be considered as several +lines of information. You may consider ``config.getoption('verbose')`` in order to +display more information if applicable: + +.. code-block:: python # content of conftest.py def pytest_report_header(config): - if config.option.verbose > 0: + if config.getoption('verbose') > 0: return ["info1: did you know that ...", "did you?"] which will add info only when run with "--v":: - $ py.test -v - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache info1: did you know that ... did you? - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collecting ... collected 0 items - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= and nothing when run plainly:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 0 items - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= profiling test duration -------------------------- @@ -313,36 +358,37 @@ profiling test duration .. versionadded: 2.2 If you have a slow running large test suite you might want to find -out which tests are the slowest. Let's make an artificial test suite:: +out which tests are the slowest. Let's make an artificial test suite: - # content of test_some_are_slow.py +.. code-block:: python + # content of test_some_are_slow.py import time def test_funcfast(): - pass + time.sleep(0.1) def test_funcslow1(): - time.sleep(0.1) + time.sleep(0.2) def test_funcslow2(): - time.sleep(0.2) + time.sleep(0.3) Now we can profile which test functions execute the slowest:: - $ py.test --durations=3 - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest --durations=3 + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 3 items - test_some_are_slow.py ... + test_some_are_slow.py ... [100%] - ======= slowest 3 test durations ======== - 0.20s call test_some_are_slow.py::test_funcslow2 - 0.10s call test_some_are_slow.py::test_funcslow1 - 0.00s setup test_some_are_slow.py::test_funcfast - ======= 3 passed in 0.12 seconds ======== + ========================= slowest 3 test durations ========================= + 0.30s call test_some_are_slow.py::test_funcslow2 + 0.20s call test_some_are_slow.py::test_funcslow1 + 0.10s call test_some_are_slow.py::test_funcfast + ========================= 3 passed in 0.12 seconds ========================= incremental testing - test steps --------------------------------------------------- @@ -353,7 +399,9 @@ Sometimes you may have a testing situation which consists of a series of test steps. If one step fails it makes no sense to execute further steps as they are all expected to fail anyway and their tracebacks add no insight. Here is a simple ``conftest.py`` file which introduces -an ``incremental`` marker which is to be used on classes:: +an ``incremental`` marker which is to be used on classes: + +.. code-block:: python # content of conftest.py @@ -372,14 +420,16 @@ an ``incremental`` marker which is to be used on classes:: pytest.xfail("previous test failed (%s)" %previousfailed.name) These two hook implementations work together to abort incremental-marked -tests in a class. Here is a test module example:: +tests in a class. Here is a test module example: + +.. code-block:: python # content of test_step.py import pytest @pytest.mark.incremental - class TestUserHandling: + class TestUserHandling(object): def test_login(self): pass def test_modification(self): @@ -392,19 +442,19 @@ tests in a class. Here is a test module example:: If we run this:: - $ py.test -rx - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest -rx + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 4 items - test_step.py .Fx. - ======= short test summary info ======== + test_step.py .Fx. [100%] + ========================= short test summary info ========================== XFAIL test_step.py::TestUserHandling::()::test_deletion reason: previous test failed (test_modification) - ======= FAILURES ======== - _______ TestUserHandling.test_modification ________ + ================================= FAILURES ================================= + ____________________ TestUserHandling.test_modification ____________________ self = @@ -413,7 +463,7 @@ If we run this:: E assert 0 test_step.py:9: AssertionError - ======= 1 failed, 2 passed, 1 xfailed in 0.12 seconds ======== + ============== 1 failed, 2 passed, 1 xfailed in 0.12 seconds =============== We'll see that ``test_deletion`` was not executed because ``test_modification`` failed. It is reported as an "expected failure". @@ -430,32 +480,40 @@ concept. It's however recommended to have explicit fixture references in your tests or test classes rather than relying on implicitly executing setup/teardown functions, especially if they are far away from the actual tests. -Here is a an example for making a ``db`` fixture available in a directory:: +Here is an example for making a ``db`` fixture available in a directory: + +.. code-block:: python # content of a/conftest.py import pytest - class DB: + class DB(object): pass @pytest.fixture(scope="session") def db(): return DB() -and then a test module in that directory:: +and then a test module in that directory: + +.. code-block:: python # content of a/test_db.py def test_a1(db): assert 0, db # to show value -another test module:: +another test module: + +.. code-block:: python # content of a/test_db2.py def test_a2(db): assert 0, db # to show value and then a module in a sister directory which will not see -the ``db`` fixture:: +the ``db`` fixture: + +.. code-block:: python # content of b/test_error.py def test_root(db): # no db here, will error out @@ -463,28 +521,28 @@ the ``db`` fixture:: We can run this:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 7 items - test_step.py .Fx. - a/test_db.py F - a/test_db2.py F - b/test_error.py E + test_step.py .Fx. [ 57%] + a/test_db.py F [ 71%] + a/test_db2.py F [ 85%] + b/test_error.py E [100%] - ======= ERRORS ======== - _______ ERROR at setup of test_root ________ + ================================== ERRORS ================================== + _______________________ ERROR at setup of test_root ________________________ file $REGENDOC_TMPDIR/b/test_error.py, line 1 def test_root(db): # no db here, will error out - fixture 'db' not found - available fixtures: record_xml_property, recwarn, cache, capsys, pytestconfig, tmpdir_factory, capfd, monkeypatch, tmpdir - use 'py.test --fixtures [testpath]' for help on them. + E fixture 'db' not found + > available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_xml_property, recwarn, tmpdir, tmpdir_factory + > use 'pytest --fixtures [testpath]' for help on them. $REGENDOC_TMPDIR/b/test_error.py:1 - ======= FAILURES ======== - _______ TestUserHandling.test_modification ________ + ================================= FAILURES ================================= + ____________________ TestUserHandling.test_modification ____________________ self = @@ -493,7 +551,7 @@ We can run this:: E assert 0 test_step.py:9: AssertionError - _______ test_a1 ________ + _________________________________ test_a1 __________________________________ db = @@ -503,7 +561,7 @@ We can run this:: E assert 0 a/test_db.py:2: AssertionError - _______ test_a2 ________ + _________________________________ test_a2 __________________________________ db = @@ -513,7 +571,7 @@ We can run this:: E assert 0 a/test_db2.py:2: AssertionError - ======= 3 failed, 2 passed, 1 xfailed, 1 error in 0.12 seconds ======== + ========== 3 failed, 2 passed, 1 xfailed, 1 error in 0.12 seconds ========== The two test modules in the ``a`` directory see the same ``db`` fixture instance while the one test in the sister-directory ``b`` doesn't see it. We could of course @@ -531,7 +589,9 @@ environment you can implement a hook that gets called when the test "report" object is about to be created. Here we write out all failing test calls and also access a fixture (if it was used by the test) in case you want to query/look at it during your post processing. In our -case we just write some informations out to a ``failures`` file:: +case we just write some information out to a ``failures`` file: + +.. code-block:: python # content of conftest.py @@ -557,7 +617,9 @@ case we just write some informations out to a ``failures`` file:: f.write(rep.nodeid + extra + "\n") -if you then have failing tests:: +if you then have failing tests: + +.. code-block:: python # content of test_module.py def test_fail1(tmpdir): @@ -567,16 +629,16 @@ if you then have failing tests:: and run them:: - $ py.test test_module.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest test_module.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 2 items - test_module.py FF + test_module.py FF [100%] - ======= FAILURES ======== - _______ test_fail1 ________ + ================================= FAILURES ================================= + ________________________________ test_fail1 ________________________________ tmpdir = local('PYTEST_TMPDIR/test_fail10') @@ -585,14 +647,14 @@ and run them:: E assert 0 test_module.py:2: AssertionError - _______ test_fail2 ________ + ________________________________ test_fail2 ________________________________ def test_fail2(): > assert 0 E assert 0 test_module.py:4: AssertionError - ======= 2 failed in 0.12 seconds ======== + ========================= 2 failed in 0.12 seconds ========================= you will have a "failures" file which contains the failing test ids:: @@ -606,7 +668,9 @@ Making test result information available in fixtures .. regendoc:wipe If you want to make test result reports available in fixture finalizers -here is a little example implemented via a local plugin:: +here is a little example implemented via a local plugin: + +.. code-block:: python # content of conftest.py @@ -618,7 +682,7 @@ here is a little example implemented via a local plugin:: outcome = yield rep = outcome.get_result() - # set an report attribute for each phase of a call, which can + # set a report attribute for each phase of a call, which can # be "setup", "call", "teardown" setattr(item, "rep_" + rep.when, rep) @@ -626,18 +690,19 @@ here is a little example implemented via a local plugin:: @pytest.fixture def something(request): - def fin(): - # request.node is an "item" because we use the default - # "function" scope - if request.node.rep_setup.failed: - print ("setting up a test failed!", request.node.nodeid) - elif request.node.rep_setup.passed: - if request.node.rep_call.failed: - print ("executing test failed", request.node.nodeid) - request.addfinalizer(fin) + yield + # request.node is an "item" because we use the default + # "function" scope + if request.node.rep_setup.failed: + print ("setting up a test failed!", request.node.nodeid) + elif request.node.rep_setup.passed: + if request.node.rep_call.failed: + print ("executing test failed", request.node.nodeid) -if you then have failing tests:: +if you then have failing tests: + +.. code-block:: python # content of test_module.py @@ -658,18 +723,18 @@ if you then have failing tests:: and run it:: - $ py.test -s test_module.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest -s test_module.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 3 items test_module.py Esetting up a test failed! test_module.py::test_setup_fails Fexecuting test failed test_module.py::test_call_fails - F + F [100%] - ======= ERRORS ======== - _______ ERROR at setup of test_setup_fails ________ + ================================== ERRORS ================================== + ____________________ ERROR at setup of test_setup_fails ____________________ @pytest.fixture def other(): @@ -677,8 +742,8 @@ and run it:: E assert 0 test_module.py:6: AssertionError - ======= FAILURES ======== - _______ test_call_fails ________ + ================================= FAILURES ================================= + _____________________________ test_call_fails ______________________________ something = None @@ -687,52 +752,82 @@ and run it:: E assert 0 test_module.py:12: AssertionError - _______ test_fail2 ________ + ________________________________ test_fail2 ________________________________ def test_fail2(): > assert 0 E assert 0 test_module.py:15: AssertionError - ======= 2 failed, 1 error in 0.12 seconds ======== + ==================== 2 failed, 1 error in 0.12 seconds ===================== You'll see that the fixture finalizers could use the precise reporting information. -Integrating pytest runner and cx_freeze ------------------------------------------------------------ +``PYTEST_CURRENT_TEST`` environment variable +-------------------------------------------- -If you freeze your application using a tool like -`cx_freeze `_ in order to distribute it -to your end-users, it is a good idea to also package your test runner and run -your tests using the frozen application. +.. versionadded:: 3.2 -This way packaging errors such as dependencies not being -included into the executable can be detected early while also allowing you to -send test files to users so they can run them in their machines, which can be -invaluable to obtain more information about a hard to reproduce bug. +Sometimes a test session might get stuck and there might be no easy way to figure out +which test got stuck, for example if pytest was run in quiet mode (``-q``) or you don't have access to the console +output. This is particularly a problem if the problem helps only sporadically, the famous "flaky" kind of tests. -Unfortunately ``cx_freeze`` can't discover them -automatically because of ``pytest``'s use of dynamic module loading, so you -must declare them explicitly by using ``pytest.freeze_includes()``:: +``pytest`` sets a ``PYTEST_CURRENT_TEST`` environment variable when running tests, which can be inspected +by process monitoring utilities or libraries like `psutil `_ to discover which +test got stuck if necessary: - # contents of setup.py - from cx_Freeze import setup, Executable - import pytest +.. code-block:: python + + import psutil + + for pid in psutil.pids(): + environ = psutil.Process(pid).environ() + if 'PYTEST_CURRENT_TEST' in environ: + print(f'pytest process {pid} running: {environ["PYTEST_CURRENT_TEST"]}') + +During the test session pytest will set ``PYTEST_CURRENT_TEST`` to the current test +:ref:`nodeid ` and the current stage, which can be ``setup``, ``call`` +and ``teardown``. - setup( - name="app_main", - executables=[Executable("app_main.py")], - options={"build_exe": - { - 'includes': pytest.freeze_includes()} - }, - # ... other options - ) +For example, when running a single test function named ``test_foo`` from ``foo_module.py``, +``PYTEST_CURRENT_TEST`` will be set to: -If you don't want to ship a different executable just in order to run your tests, -you can make your program check for a certain flag and pass control -over to ``pytest`` instead. For example:: +#. ``foo_module.py::test_foo (setup)`` +#. ``foo_module.py::test_foo (call)`` +#. ``foo_module.py::test_foo (teardown)`` + +In that order. + +.. note:: + + The contents of ``PYTEST_CURRENT_TEST`` is meant to be human readable and the actual format + can be changed between releases (even bug fixes) so it shouldn't be relied on for scripting + or automation. + +Freezing pytest +--------------- + +If you freeze your application using a tool like +`PyInstaller `_ +in order to distribute it to your end-users, it is a good idea to also package +your test runner and run your tests using the frozen application. This way packaging +errors such as dependencies not being included into the executable can be detected early +while also allowing you to send test files to users so they can run them in their +machines, which can be useful to obtain more information about a hard to reproduce bug. + +Fortunately recent ``PyInstaller`` releases already have a custom hook +for pytest, but if you are using another tool to freeze executables +such as ``cx_freeze`` or ``py2exe``, you can use ``pytest.freeze_includes()`` +to obtain the full list of internal pytest modules. How to configure the tools +to find the internal modules varies from tool to tool, however. + +Instead of freezing the pytest runner as a separate executable, you can make +your frozen program work as the pytest runner by some clever +argument handling during program startup. This allows you to +have a single executable, which is usually more convenient. + +.. code-block:: python # contents of app_main.py import sys @@ -745,7 +840,8 @@ over to ``pytest`` instead. For example:: # by your argument-parsing library of choice as usual ... -This makes it convenient to execute your tests from within your frozen -application, using standard ``py.test`` command-line options:: + +This allows you to execute tests using the frozen +application with standard ``pytest`` command-line options:: ./app_main --pytest --verbose --tb=long --junitxml=results.xml test-suite/ diff --git a/doc/en/example/special.rst b/doc/en/example/special.rst index 58e66d44e3442ec..4437e1cc30ea184 100644 --- a/doc/en/example/special.rst +++ b/doc/en/example/special.rst @@ -28,7 +28,7 @@ will be called ahead of running any tests:: # content of test_module.py - class TestHello: + class TestHello(object): @classmethod def callme(cls): print ("callme called!") @@ -39,7 +39,7 @@ will be called ahead of running any tests:: def test_method2(self): print ("test_method1 called") - class TestOther: + class TestOther(object): @classmethod def callme(cls): print ("callme other called") @@ -59,7 +59,7 @@ will be called ahead of running any tests:: If you run this without output capturing:: - $ py.test -q -s test_module.py + $ pytest -q -s test_module.py callattr_ahead_of_alltests called callme called! callme other called @@ -68,5 +68,5 @@ If you run this without output capturing:: .test_method1 called .test other .test_unit1 method called - . + . [100%] 4 passed in 0.12 seconds diff --git a/doc/en/existingtestsuite.rst b/doc/en/existingtestsuite.rst new file mode 100644 index 000000000000000..d304b30c9a60d63 --- /dev/null +++ b/doc/en/existingtestsuite.rst @@ -0,0 +1,34 @@ +.. _existingtestsuite: + +Using pytest with an existing test suite +=========================================== + +Pytest can be used with most existing test suites, but its +behavior differs from other test runners such as :ref:`nose ` or +Python's default unittest framework. + +Before using this section you will want to :ref:`install pytest `. + +Running an existing test suite with pytest +--------------------------------------------- + +Say you want to contribute to an existing repository somewhere. +After pulling the code into your development space using some +flavor of version control and (optionally) setting up a virtualenv +you will want to run:: + + cd + pip install -e . # Environment dependent alternatives include + # 'python setup.py develop' and 'conda develop' + +in your project root. This will set up a symlink to your code in +site-packages, allowing you to edit your code while your tests +run against it as if it were installed. + +Setting up your project in development mode lets you avoid having to +reinstall every time you want to run your tests, and is less brittle than +mucking about with sys.path to point your tests at local code. + +Also consider using :ref:`tox `. + +.. include:: links.inc diff --git a/doc/en/faq.rst b/doc/en/faq.rst index fd7ca35e9431c81..27d74e1148c5191 100644 --- a/doc/en/faq.rst +++ b/doc/en/faq.rst @@ -66,14 +66,6 @@ This completely avoids previous issues of confusing assertion-reporting. It also means, that you can use Python's ``-O`` optimization without losing assertions in test modules. -``pytest`` contains a second, mostly obsolete, assert debugging technique -invoked via ``--assert=reinterpret``: When an ``assert`` statement fails, ``pytest`` re-interprets -the expression part to show intermediate values. This technique suffers -from a caveat that the rewriting does not: If your expression has side -effects (better to avoid them anyway!) the intermediate values may not -be the same, confusing the reinterpreter and obfuscating the initial -error (this is also explained at the command line if it happens). - You can also turn off all assertion interaction using the ``--assert=plain`` option. @@ -81,18 +73,17 @@ You can also turn off all assertion interaction using the .. _`py/__init__.py`: http://bitbucket.org/hpk42/py-trunk/src/trunk/py/__init__.py -Why a ``py.test`` instead of a ``pytest`` command? -++++++++++++++++++++++++++++++++++++++++++++++++++ +Why can I use both ``pytest`` and ``py.test`` commands? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -Some of the reasons are historic, others are practical. ``pytest`` -used to be part of the ``py`` package which provided several developer -utilities, all starting with ``py.``, thus providing nice -TAB-completion. If -you install ``pip install pycmd`` you get these tools from a separate -package. These days the command line tool could be called ``pytest`` -but since many people have gotten used to the old name and there -is another tool named "pytest" we just decided to stick with -``py.test`` for now. +pytest used to be part of the py package, which provided several developer +utilities, all starting with ``py.``, thus providing nice TAB-completion. +If you install ``pip install pycmd`` you get these tools from a separate +package. Once ``pytest`` became a separate package, the ``py.test`` name was +retained due to avoid a naming conflict with another tool. This conflict was +eventually resolved, and the ``pytest`` command was therefore introduced. In +future versions of pytest, we may deprecate and later remove the ``py.test`` +command to avoid perpetuating the confusion. pytest fixtures, parametrized tests ------------------------------------------------------- diff --git a/doc/en/feedback.rst b/doc/en/feedback.rst deleted file mode 100644 index 9c63b7640e0ef2a..000000000000000 --- a/doc/en/feedback.rst +++ /dev/null @@ -1,8 +0,0 @@ - -What users say: - - `py.test is pretty much the best thing ever`_ (Alex Gaynor) - - -.. _`py.test is pretty much the best thing ever`_ (Alex Gaynor) - http://twitter.com/#!/alex_gaynor/status/22389410366 diff --git a/doc/en/fixture.rst b/doc/en/fixture.rst index f48607ae2ddd04b..01a941ddf656f2b 100644 --- a/doc/en/fixture.rst +++ b/doc/en/fixture.rst @@ -11,7 +11,7 @@ pytest fixtures: explicit, modular, scalable .. _`xUnit`: http://en.wikipedia.org/wiki/XUnit .. _`purpose of test fixtures`: http://en.wikipedia.org/wiki/Test_fixture#Software -.. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection#Definition +.. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection The `purpose of test fixtures`_ is to provide a fixed baseline upon which tests can reliably and repeatedly execute. pytest fixtures @@ -27,18 +27,13 @@ functions: * fixture management scales from simple unit to complex functional testing, allowing to parametrize fixtures and tests according to configuration and component options, or to re-use fixtures - across class, module or whole test session scopes. + across function, class, module or whole test session scopes. In addition, pytest continues to support :ref:`xunitsetup`. You can mix both styles, moving incrementally from classic to new style, as you prefer. You can also start out from existing :ref:`unittest.TestCase style ` or :ref:`nose based ` projects. -.. note:: - - pytest-2.4 introduced an additional experimental - :ref:`yield fixture mechanism ` for easier context manager - integration and more linear writing of teardown code. .. _`funcargs`: .. _`funcarg mechanism`: @@ -62,7 +57,7 @@ using it:: @pytest.fixture def smtp(): import smtplib - return smtplib.SMTP("smtp.gmail.com") + return smtplib.SMTP("smtp.gmail.com", 587, timeout=5) def test_ehlo(smtp): response, msg = smtp.ehlo() @@ -73,16 +68,16 @@ Here, the ``test_ehlo`` needs the ``smtp`` fixture value. pytest will discover and call the :py:func:`@pytest.fixture <_pytest.python.fixture>` marked ``smtp`` fixture function. Running the test looks like this:: - $ py.test test_smtpsimple.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items + $ pytest test_smtpsimple.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item - test_smtpsimple.py F + test_smtpsimple.py F [100%] - ======= FAILURES ======== - _______ test_ehlo ________ + ================================= FAILURES ================================= + ________________________________ test_ehlo _________________________________ smtp = @@ -93,7 +88,7 @@ marked ``smtp`` fixture function. Running the test looks like this:: E assert 0 test_smtpsimple.py:11: AssertionError - ======= 1 failed in 0.12 seconds ======== + ========================= 1 failed in 0.12 seconds ========================= In the failure traceback we see that the test function was called with a ``smtp`` argument, the ``smtplib.SMTP()`` instance created by the fixture @@ -114,41 +109,57 @@ Note that if you misspell a function argument or want to use one that isn't available, you'll see an error with a list of available function arguments. -.. Note:: +.. note:: You can always issue:: - py.test --fixtures test_simplefactory.py + pytest --fixtures test_simplefactory.py to see available fixtures. - In versions prior to 2.3 there was no ``@pytest.fixture`` marker - and you had to use a magic ``pytest_funcarg__NAME`` prefix - for the fixture factory. This remains and will remain supported - but is not anymore advertised as the primary means of declaring fixture - functions. - -"Funcargs" a prime example of dependency injection +Fixtures: a prime example of dependency injection --------------------------------------------------- -When injecting fixtures to test functions, pytest-2.0 introduced the -term "funcargs" or "funcarg mechanism" which continues to be present -also in docs today. It now refers to the specific case of injecting -fixture values as arguments to test functions. With pytest-2.3 there are -more possibilities to use fixtures but "funcargs" remain as the main way -as they allow to directly state the dependencies of a test function. - -As the following examples show in more detail, funcargs allow test -functions to easily receive and work against specific pre-initialized -application objects without having to care about import/setup/cleanup -details. It's a prime example of `dependency injection`_ where fixture +Fixtures allow test functions to easily receive and work +against specific pre-initialized application objects without having +to care about import/setup/cleanup details. +It's a prime example of `dependency injection`_ where fixture functions take the role of the *injector* and test functions are the *consumers* of fixture objects. +.. _`conftest.py`: +.. _`conftest`: + +``conftest.py``: sharing fixture functions +------------------------------------------ + +If during implementing your tests you realize that you +want to use a fixture function from multiple test files you can move it +to a ``conftest.py`` file. +You don't need to import the fixture you want to use in a test, it +automatically gets discovered by pytest. The discovery of +fixture functions starts at test classes, then test modules, then +``conftest.py`` files and finally builtin and third party plugins. + +You can also use the ``conftest.py`` file to implement +:ref:`local per-directory plugins `. + +Sharing test data +----------------- + +If you want to make test data from files available to your tests, a good way +to do this is by loading these data in a fixture for use by your tests. +This makes use of the automatic caching mechanisms of pytest. + +Another good approach is by adding the data files in the ``tests`` folder. +There are also community plugins available to help managing this aspect of +testing, e.g. `pytest-datadir `__ +and `pytest-datafiles `__. + .. _smtpshared: -Sharing a fixture across tests in a module (or class/session) ------------------------------------------------------------------ +Scope: sharing a fixture instance across tests in a class, module or session +---------------------------------------------------------------------------- .. regendoc:wipe @@ -157,10 +168,12 @@ usually time-expensive to create. Extending the previous example, we can add a ``scope='module'`` parameter to the :py:func:`@pytest.fixture <_pytest.python.fixture>` invocation to cause the decorated ``smtp`` fixture function to only be invoked once -per test module. Multiple test functions in a test module will thus -each receive the same ``smtp`` fixture instance. The next example puts -the fixture function into a separate ``conftest.py`` file so -that tests from multiple test modules in the directory can +per test *module* (the default is to invoke once per test *function*). +Multiple test functions in a test module will thus +each receive the same ``smtp`` fixture instance, thus saving time. + +The next example puts the fixture function into a separate ``conftest.py`` file +so that tests from multiple test modules in the directory can access the fixture function:: # content of conftest.py @@ -169,7 +182,7 @@ access the fixture function:: @pytest.fixture(scope="module") def smtp(): - return smtplib.SMTP("smtp.gmail.com") + return smtplib.SMTP("smtp.gmail.com", 587, timeout=5) The name of the fixture again is ``smtp`` and you can access its result by listing the name ``smtp`` as an input parameter in any test or fixture @@ -181,7 +194,7 @@ function (in or below the directory where ``conftest.py`` is located):: response, msg = smtp.ehlo() assert response == 250 assert b"smtp.gmail.com" in msg - assert 0 # for demo purposes + assert 0 # for demo purposes def test_noop(smtp): response, msg = smtp.noop() @@ -191,16 +204,16 @@ function (in or below the directory where ``conftest.py`` is located):: We deliberately insert failing ``assert 0`` statements in order to inspect what is going on and can now run the tests:: - $ py.test test_module.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest test_module.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 2 items - test_module.py FF + test_module.py FF [100%] - ======= FAILURES ======== - _______ test_ehlo ________ + ================================= FAILURES ================================= + ________________________________ test_ehlo _________________________________ smtp = @@ -212,7 +225,7 @@ inspect what is going on and can now run the tests:: E assert 0 test_module.py:6: AssertionError - _______ test_noop ________ + ________________________________ test_noop _________________________________ smtp = @@ -223,7 +236,7 @@ inspect what is going on and can now run the tests:: E assert 0 test_module.py:11: AssertionError - ======= 2 failed in 0.12 seconds ======== + ========================= 2 failed in 0.12 seconds ========================= You see the two ``assert 0`` failing and more importantly you can also see that the same (module-scoped) ``smtp`` object was passed into the two @@ -241,15 +254,18 @@ instance, you can simply declare it: # the returned fixture value will be shared for # all tests needing it +Finally, the ``class`` scope will invoke the fixture once per test *class*. + .. _`finalization`: Fixture finalization / executing teardown code ------------------------------------------------------------- pytest supports execution of fixture specific finalization code -when the fixture goes out of scope. By accepting a ``request`` object -into your fixture function you can call its ``request.addfinalizer`` one -or multiple times:: +when the fixture goes out of scope. By using a ``yield`` statement instead of ``return``, all +the code after the *yield* statement serves as the teardown code: + +.. code-block:: python # content of conftest.py @@ -257,21 +273,20 @@ or multiple times:: import pytest @pytest.fixture(scope="module") - def smtp(request): - smtp = smtplib.SMTP("smtp.gmail.com") - def fin(): - print ("teardown smtp") - smtp.close() - request.addfinalizer(fin) - return smtp # provide the fixture value + def smtp(): + smtp = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) + yield smtp # provide the fixture value + print("teardown smtp") + smtp.close() -The ``fin`` function will execute when the last test using -the fixture in the module has finished execution. +The ``print`` and ``smtp.close()`` statements will execute when the last test in +the module has finished execution, regardless of the exception status of the +tests. Let's execute it:: - $ py.test -s -q --tb=no - FFteardown smtp + $ pytest -s -q --tb=no + FF [100%]teardown smtp 2 failed in 0.12 seconds @@ -282,6 +297,72 @@ occur around each single test. In either case the test module itself does not need to change or know about these details of fixture setup. +Note that we can also seamlessly use the ``yield`` syntax with ``with`` statements: + +.. code-block:: python + + # content of test_yield2.py + + import smtplib + import pytest + + @pytest.fixture(scope="module") + def smtp(): + with smtplib.SMTP("smtp.gmail.com", 587, timeout=5) as smtp: + yield smtp # provide the fixture value + + +The ``smtp`` connection will be closed after the test finished execution +because the ``smtp`` object automatically closes when +the ``with`` statement ends. + +Note that if an exception happens during the *setup* code (before the ``yield`` keyword), the +*teardown* code (after the ``yield``) will not be called. + +An alternative option for executing *teardown* code is to +make use of the ``addfinalizer`` method of the `request-context`_ object to register +finalization functions. + +Here's the ``smtp`` fixture changed to use ``addfinalizer`` for cleanup: + +.. code-block:: python + + # content of conftest.py + import smtplib + import pytest + + @pytest.fixture(scope="module") + def smtp(request): + smtp = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) + def fin(): + print ("teardown smtp") + smtp.close() + request.addfinalizer(fin) + return smtp # provide the fixture value + + +Both ``yield`` and ``addfinalizer`` methods work similarly by calling their code after the test +ends, but ``addfinalizer`` has two key differences over ``yield``: + +1. It is possible to register multiple finalizer functions. + +2. Finalizers will always be called regardless if the fixture *setup* code raises an exception. + This is handy to properly close all resources created by a fixture even if one of them + fails to be created/acquired:: + + @pytest.fixture + def equipments(request): + r = [] + for port in ('C1', 'C3', 'C28'): + equip = connect(port) + request.addfinalizer(equip.disconnect) + r.append(equip) + return r + + In the example above, if ``"C28"`` fails with an exception, ``"C1"`` and ``"C3"`` will still + be properly closed. Of course, if an exception happens before the finalize function is + registered then it will not be executed. + .. _`request-context`: @@ -300,20 +381,17 @@ read an optional server URL from the test module which uses our fixture:: @pytest.fixture(scope="module") def smtp(request): server = getattr(request.module, "smtpserver", "smtp.gmail.com") - smtp = smtplib.SMTP(server) - - def fin(): - print ("finalizing %s (%s)" % (smtp, server)) - smtp.close() - request.addfinalizer(fin) - return smtp + smtp = smtplib.SMTP(server, 587, timeout=5) + yield smtp + print ("finalizing %s (%s)" % (smtp, server)) + smtp.close() We use the ``request.module`` attribute to optionally obtain an ``smtpserver`` attribute from the test module. If we just execute again, nothing much has changed:: - $ py.test -s -q --tb=no - FFfinalizing (smtp.gmail.com) + $ pytest -s -q --tb=no + FF [100%]finalizing (smtp.gmail.com) 2 failed in 0.12 seconds @@ -329,21 +407,23 @@ server URL in its module namespace:: Running it:: - $ py.test -qq --tb=short test_anothersmtp.py - F - ======= FAILURES ======== - _______ test_showhelo ________ + $ pytest -qq --tb=short test_anothersmtp.py + F [100%] + ================================= FAILURES ================================= + ______________________________ test_showhelo _______________________________ test_anothersmtp.py:5: in test_showhelo assert 0, smtp.helo() E AssertionError: (250, b'mail.python.org') E assert 0 + ------------------------- Captured stdout teardown ------------------------- + finalizing (mail.python.org) voila! The ``smtp`` fixture function picked up our mail server name from the module namespace. .. _`fixture-parametrize`: -Parametrizing a fixture +Parametrizing fixtures ----------------------------------------------------------------- Fixture functions can be parametrized in which case they will be called @@ -365,12 +445,10 @@ through the special :py:class:`request ` object:: @pytest.fixture(scope="module", params=["smtp.gmail.com", "mail.python.org"]) def smtp(request): - smtp = smtplib.SMTP(request.param) - def fin(): - print ("finalizing %s" % smtp) - smtp.close() - request.addfinalizer(fin) - return smtp + smtp = smtplib.SMTP(request.param, 587, timeout=5) + yield smtp + print ("finalizing %s" % smtp) + smtp.close() The main change is the declaration of ``params`` with :py:func:`@pytest.fixture <_pytest.python.fixture>`, a list of values @@ -378,10 +456,10 @@ for each of which the fixture function will execute and can access a value via ``request.param``. No test function code needs to change. So let's just do another run:: - $ py.test -q test_module.py - FFFF - ======= FAILURES ======== - _______ test_ehlo[smtp.gmail.com] ________ + $ pytest -q test_module.py + FFFF [100%] + ================================= FAILURES ================================= + ________________________ test_ehlo[smtp.gmail.com] _________________________ smtp = @@ -393,7 +471,7 @@ So let's just do another run:: E assert 0 test_module.py:6: AssertionError - _______ test_noop[smtp.gmail.com] ________ + ________________________ test_noop[smtp.gmail.com] _________________________ smtp = @@ -404,7 +482,7 @@ So let's just do another run:: E assert 0 test_module.py:11: AssertionError - _______ test_ehlo[mail.python.org] ________ + ________________________ test_ehlo[mail.python.org] ________________________ smtp = @@ -412,12 +490,12 @@ So let's just do another run:: response, msg = smtp.ehlo() assert response == 250 > assert b"smtp.gmail.com" in msg - E assert b'smtp.gmail.com' in b'mail.python.org\nSIZE 51200000\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nSMTPUTF8' + E AssertionError: assert b'smtp.gmail.com' in b'mail.python.org\nPIPELINING\nSIZE 51200000\nETRN\nSTARTTLS\nAUTH DIGEST-MD5 NTLM CRAM-MD5\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nSMTPUTF8' test_module.py:5: AssertionError -------------------------- Captured stdout setup --------------------------- finalizing - _______ test_noop[mail.python.org] ________ + ________________________ test_noop[mail.python.org] ________________________ smtp = @@ -428,6 +506,8 @@ So let's just do another run:: E assert 0 test_module.py:11: AssertionError + ------------------------- Captured stdout teardown ------------------------- + finalizing 4 failed in 0.12 seconds We see that our two test functions each ran twice, against the different @@ -478,10 +558,10 @@ return ``None`` then pytest's auto-generated ID will be used. Running the above tests results in the following test IDs being used:: - $ py.test --collect-only - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest --collect-only + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 10 items @@ -497,7 +577,7 @@ Running the above tests results in the following test IDs being used:: - ======= no tests ran in 0.12 seconds ======== + ======================= no tests ran in 0.12 seconds ======================= .. _`interdependent fixtures`: @@ -515,7 +595,7 @@ and instantiate an object ``app`` where we stick the already defined import pytest - class App: + class App(object): def __init__(self, smtp): self.smtp = smtp @@ -529,17 +609,17 @@ and instantiate an object ``app`` where we stick the already defined Here we declare an ``app`` fixture which receives the previously defined ``smtp`` fixture and instantiates an ``App`` object with it. Let's run it:: - $ py.test -v test_appsetup.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v test_appsetup.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collecting ... collected 2 items - test_appsetup.py::test_smtp_exists[smtp.gmail.com] PASSED - test_appsetup.py::test_smtp_exists[mail.python.org] PASSED + test_appsetup.py::test_smtp_exists[smtp.gmail.com] PASSED [ 50%] + test_appsetup.py::test_smtp_exists[mail.python.org] PASSED [100%] - ======= 2 passed in 0.12 seconds ======== + ========================= 2 passed in 0.12 seconds ========================= Due to the parametrization of ``smtp`` the test will run twice with two different ``App`` instances and respective smtp servers. There is no @@ -567,7 +647,7 @@ first execute with one instance and then finalizers are called before the next fixture instance is created. Among other things, this eases testing of applications which create and use global state. -The following example uses two parametrized funcargs, one of which is +The following example uses two parametrized fixture, one of which is scoped on a per-module basis, and all the functions perform ``print`` calls to show the setup/teardown flow:: @@ -577,55 +657,80 @@ to show the setup/teardown flow:: @pytest.fixture(scope="module", params=["mod1", "mod2"]) def modarg(request): param = request.param - print ("create", param) - def fin(): - print ("fin %s" % param) - return param + print (" SETUP modarg %s" % param) + yield param + print (" TEARDOWN modarg %s" % param) @pytest.fixture(scope="function", params=[1,2]) def otherarg(request): - return request.param + param = request.param + print (" SETUP otherarg %s" % param) + yield param + print (" TEARDOWN otherarg %s" % param) def test_0(otherarg): - print (" test0", otherarg) + print (" RUN test0 with otherarg %s" % otherarg) def test_1(modarg): - print (" test1", modarg) + print (" RUN test1 with modarg %s" % modarg) def test_2(otherarg, modarg): - print (" test2", otherarg, modarg) + print (" RUN test2 with otherarg %s and modarg %s" % (otherarg, modarg)) + Let's run the tests in verbose mode and with looking at the print-output:: - $ py.test -v -s test_module.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4 + $ pytest -v -s test_module.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 cachedir: .cache - rootdir: $REGENDOC_TMPDIR, inifile: + rootdir: $REGENDOC_TMPDIR, inifile: collecting ... collected 8 items - test_module.py::test_0[1] test0 1 - PASSED - test_module.py::test_0[2] test0 2 - PASSED - test_module.py::test_1[mod1] create mod1 - test1 mod1 - PASSED - test_module.py::test_2[1-mod1] test2 1 mod1 - PASSED - test_module.py::test_2[2-mod1] test2 2 mod1 - PASSED - test_module.py::test_1[mod2] create mod2 - test1 mod2 - PASSED - test_module.py::test_2[1-mod2] test2 1 mod2 - PASSED - test_module.py::test_2[2-mod2] test2 2 mod2 - PASSED - - ======= 8 passed in 0.12 seconds ======== - -You can see that the parametrized module-scoped ``modarg`` resource caused -an ordering of test execution that lead to the fewest possible "active" resources. The finalizer for the ``mod1`` parametrized resource was executed -before the ``mod2`` resource was setup. + test_module.py::test_0[1] SETUP otherarg 1 + RUN test0 with otherarg 1 + PASSED [ 12%] TEARDOWN otherarg 1 + + test_module.py::test_0[2] SETUP otherarg 2 + RUN test0 with otherarg 2 + PASSED [ 25%] TEARDOWN otherarg 2 + + test_module.py::test_1[mod1] SETUP modarg mod1 + RUN test1 with modarg mod1 + PASSED [ 37%] + test_module.py::test_2[1-mod1] SETUP otherarg 1 + RUN test2 with otherarg 1 and modarg mod1 + PASSED [ 50%] TEARDOWN otherarg 1 + + test_module.py::test_2[2-mod1] SETUP otherarg 2 + RUN test2 with otherarg 2 and modarg mod1 + PASSED [ 62%] TEARDOWN otherarg 2 + + test_module.py::test_1[mod2] TEARDOWN modarg mod1 + SETUP modarg mod2 + RUN test1 with modarg mod2 + PASSED [ 75%] + test_module.py::test_2[1-mod2] SETUP otherarg 1 + RUN test2 with otherarg 1 and modarg mod2 + PASSED [ 87%] TEARDOWN otherarg 1 + + test_module.py::test_2[2-mod2] SETUP otherarg 2 + RUN test2 with otherarg 2 and modarg mod2 + PASSED [100%] TEARDOWN otherarg 2 + TEARDOWN modarg mod2 + + + ========================= 8 passed in 0.12 seconds ========================= + +You can see that the parametrized module-scoped ``modarg`` resource caused an +ordering of test execution that lead to the fewest possible "active" resources. +The finalizer for the ``mod1`` parametrized resource was executed before the +``mod2`` resource was setup. + +In particular notice that test_0 is completely independent and finishes first. +Then test_1 is executed with ``mod1``, then test_2 with ``mod1``, then test_1 +with ``mod2`` and finally test_2 with ``mod2``. + +The ``otherarg`` parametrized resource (having function scope) was set up before +and teared down after every test that used it. .. _`usefixtures`: @@ -638,7 +743,7 @@ Using fixtures from classes, modules or projects Sometimes test functions do not directly need access to a fixture object. For example, tests may require to operate with an empty directory as the current working directory but otherwise do not care for the concrete -directory. Here is how you can can use the standard `tempfile +directory. Here is how you can use the standard `tempfile `_ and pytest fixtures to achieve it. We separate the creation of the fixture into a conftest.py file:: @@ -661,7 +766,7 @@ and declare its use in a test module via a ``usefixtures`` marker:: import pytest @pytest.mark.usefixtures("cleandir") - class TestDirectoryInit: + class TestDirectoryInit(object): def test_cwd_starts_empty(self): assert os.listdir(os.getcwd()) == [] with open("myfile", "w") as f: @@ -675,8 +780,8 @@ will be required for the execution of each test method, just as if you specified a "cleandir" function argument to each of them. Let's run it to verify our fixture is activated and the tests pass:: - $ py.test -q - .. + $ pytest -q + .. [100%] 2 passed in 0.12 seconds You can specify multiple fixtures like this: @@ -714,8 +819,8 @@ Autouse fixtures (xUnit setup on steroids) .. regendoc:wipe Occasionally, you may want to have fixtures get invoked automatically -without a `usefixtures`_ or `funcargs`_ reference. As a practical -example, suppose we have a database fixture which has a +without declaring a function argument explicitly or a `usefixtures`_ decorator. +As a practical example, suppose we have a database fixture which has a begin/rollback/commit architecture and we want to automatically surround each test method by a transaction and a rollback. Here is a dummy self-contained implementation of this idea:: @@ -724,7 +829,7 @@ self-contained implementation of this idea:: import pytest - class DB: + class DB(object): def __init__(self): self.intransaction = [] def begin(self, name): @@ -736,11 +841,12 @@ self-contained implementation of this idea:: def db(): return DB() - class TestClass: + class TestClass(object): @pytest.fixture(autouse=True) def transact(self, request, db): db.begin(request.function.__name__) - request.addfinalizer(db.rollback) + yield + db.rollback() def test_method1(self, db): assert db.intransaction == ["test_method1"] @@ -755,12 +861,16 @@ class-level ``usefixtures`` decorator. If we run it, we get two passing tests:: - $ py.test -q - .. + $ pytest -q + .. [100%] 2 passed in 0.12 seconds Here is how autouse fixtures work in other scopes: +- autouse fixtures obey the ``scope=`` keyword-argument: if an autouse fixture + has ``scope='session'`` it will only be run once, no matter where it is + defined. ``scope='class'`` means it will be run once per class, etc. + - if an autouse fixture is defined in a test module, all its test functions automatically use it. @@ -780,15 +890,16 @@ active. The canonical way to do that is to put the transact definition into a conftest.py file **without** using ``autouse``:: # content of conftest.py - @pytest.fixture() - def transact(self, request, db): + @pytest.fixture + def transact(request, db): db.begin() - request.addfinalizer(db.rollback) + yield + db.rollback() and then e.g. have a TestClass using it by declaring the need:: @pytest.mark.usefixtures("transact") - class TestClass: + class TestClass(object): def test_method1(self): ... @@ -796,16 +907,6 @@ All test methods in this TestClass will use the transaction fixture while other test classes or functions in the module will not use it unless they also add a ``transact`` reference. -Shifting (visibility of) fixture functions ----------------------------------------------------- - -If during implementing your tests you realize that you -want to use a fixture function from multiple test files you can move it -to a :ref:`conftest.py ` file or even separately installable -:ref:`plugins ` without changing test code. The discovery of -fixtures functions starts at test classes, then test modules, then -``conftest.py`` files and finally builtin and third party plugins. - Overriding fixtures on various levels ------------------------------------- @@ -928,7 +1029,7 @@ Given the tests file structure is: @pytest.mark.parametrize('username', ['directly-overridden-username-other']) def test_username_other(other_username): - assert username == 'other-directly-overridden-username-other' + assert other_username == 'other-directly-overridden-username-other' In the example above, a fixture value is overridden by the test parameter value. Note that the value of the fixture can be overridden this way even if the test doesn't use it directly (doesn't mention it in the function prototype). diff --git a/doc/en/funcarg_compare.rst b/doc/en/funcarg_compare.rst index 832922e1839d0d9..b857a014d318777 100644 --- a/doc/en/funcarg_compare.rst +++ b/doc/en/funcarg_compare.rst @@ -1,3 +1,4 @@ +:orphan: .. _`funcargcompare`: @@ -23,7 +24,7 @@ resources. Here is a basic example how we could implement a per-session Database object:: # content of conftest.py - class Database: + class Database(object): def __init__(self): print ("database instance created") def destroy(self): @@ -96,7 +97,7 @@ sets. pytest-2.3 introduces a decorator for use on the factory itself:: ... # use request.param Here the factory will be invoked twice (with the respective "mysql" -and "pg" values set as ``request.param`` attributes) and and all of +and "pg" values set as ``request.param`` attributes) and all of the tests requiring "db" will run twice as well. The "mysql" and "pg" values will also be used for reporting the test-invocation variants. @@ -172,17 +173,17 @@ to do this with parametrization as ``pytest_runtest_setup()`` is called during test execution and parametrization happens at collection time. It follows that pytest_configure/session/runtest_setup are often not -appropriate for implementing common fixture needs. Therefore, +appropriate for implementing common fixture needs. Therefore, pytest-2.3 introduces :ref:`autouse fixtures` which fully -integrate with the generic :ref:`fixture mechanism ` +integrate with the generic :ref:`fixture mechanism ` and obsolete many prior uses of pytest hooks. funcargs/fixture discovery now happens at collection time --------------------------------------------------------------------- -pytest-2.3 takes care to discover fixture/funcarg factories -at collection time. This is more efficient especially for large test suites. -Moreover, a call to "py.test --collect-only" should be able to in the future +Since pytest-2.3, discovery of fixture/funcarg factories are taken care of +at collection time. This is more efficient especially for large test suites. +Moreover, a call to "pytest --collect-only" should be able to in the future show a lot of setup-information and thus presents a nice method to get an overview of fixture management in your project. diff --git a/doc/en/genapi.py b/doc/en/genapi.py index f8cdda6cfbfb0a5..0ede44fa2dee730 100644 --- a/doc/en/genapi.py +++ b/doc/en/genapi.py @@ -1,7 +1,7 @@ import textwrap import inspect -class Writer: +class Writer(object): def __init__(self, clsname): self.clsname = clsname @@ -32,7 +32,7 @@ def docmethod(self, method): def pytest_funcarg__a(request): with Writer("request") as writer: - writer.docmethod(request.getfuncargvalue) + writer.docmethod(request.getfixturevalue) writer.docmethod(request.cached_setup) writer.docmethod(request.addfinalizer) writer.docmethod(request.applymarker) diff --git a/doc/en/getting-started.rst b/doc/en/getting-started.rst index 4a5b75aea0c833e..64b0108262d8ade 100644 --- a/doc/en/getting-started.rst +++ b/doc/en/getting-started.rst @@ -1,7 +1,7 @@ Installation and Getting Started =================================== -**Pythons**: Python 2.6,2.7,3.3,3.4,3.5, Jython, PyPy-2.3 +**Pythons**: Python 2.7, 3.4, 3.5, 3.6, Jython, PyPy-2.3 **Platforms**: Unix/Posix and Windows @@ -9,9 +9,8 @@ Installation and Getting Started **dependencies**: `py `_, `colorama (Windows) `_, -`argparse (py26) `_. -**documentation as PDF**: `download latest `_ +**documentation as PDF**: `download latest `_ .. _`getstarted`: .. _installation: @@ -19,17 +18,14 @@ Installation and Getting Started Installation ---------------------------------------- -Installation options:: +Installation:: - pip install -U pytest # or - easy_install -U pytest + pip install -U pytest To check your installation has installed the correct version:: - $ py.test --version - This is pytest version 2.9.1, imported from $PYTHON_PREFIX/lib/python3.4/site-packages/pytest.py - -If you get an error checkout :ref:`installation issues`. + $ pytest --version + This is pytest version 3.x.y, imported from $PYTHON_PREFIX/lib/python3.5/site-packages/pytest.py .. _`simpletest`: @@ -47,16 +43,16 @@ Let's create a first test file with a simple test function:: That's it. You can execute the test function now:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item - test_sample.py F + test_sample.py F [100%] - ======= FAILURES ======== - _______ test_answer ________ + ================================= FAILURES ================================= + _______________________________ test_answer ________________________________ def test_answer(): > assert func(3) == 5 @@ -64,7 +60,7 @@ That's it. You can execute the test function now:: E + where 4 = func(3) test_sample.py:5: AssertionError - ======= 1 failed in 0.12 seconds ======== + ========================= 1 failed in 0.12 seconds ========================= We got a failure report because our little ``func(3)`` call did not return ``5``. @@ -102,8 +98,8 @@ use the ``raises`` helper:: Running it with, this time in "quiet" reporting mode:: - $ py.test -q test_sysexit.py - . + $ pytest -q test_sysexit.py + . [100%] 1 passed in 0.12 seconds Grouping multiple tests in a class @@ -114,7 +110,7 @@ to group tests logically, in classes and modules. Let's write a class containing two tests:: # content of test_class.py - class TestClass: + class TestClass(object): def test_one(self): x = "this" assert 'h' in x @@ -127,17 +123,18 @@ The two tests are found because of the standard :ref:`test discovery`. There is no need to subclass anything. We can simply run the module by passing its filename:: - $ py.test -q test_class.py - .F - ======= FAILURES ======== - _______ TestClass.test_two ________ + $ pytest -q test_class.py + .F [100%] + ================================= FAILURES ================================= + ____________________________ TestClass.test_two ____________________________ self = def test_two(self): x = "hello" > assert hasattr(x, 'check') - E assert hasattr('hello', 'check') + E AssertionError: assert False + E + where False = hasattr('hello', 'check') test_class.py:8: AssertionError 1 failed, 1 passed in 0.12 seconds @@ -163,10 +160,10 @@ We list the name ``tmpdir`` in the test function signature and ``pytest`` will lookup and call a fixture factory to create the resource before performing the test function call. Let's just run it:: - $ py.test -q test_tmpdir.py - F - ======= FAILURES ======== - _______ test_needsfiles ________ + $ pytest -q test_tmpdir.py + F [100%] + ================================= FAILURES ================================= + _____________________________ test_needsfiles ______________________________ tmpdir = local('PYTEST_TMPDIR/test_needsfiles0') @@ -185,7 +182,7 @@ was created. More info at :ref:`tmpdir handling`. You can find out what kind of builtin :ref:`fixtures` exist by typing:: - py.test --fixtures # shows builtin and custom fixtures + pytest --fixtures # shows builtin and custom fixtures Where to go next ------------------------------------- @@ -193,45 +190,9 @@ Where to go next Here are a few suggestions where to go next: * :ref:`cmdline` for command line invocation examples -* :ref:`good practices ` for virtualenv, test layout, genscript support +* :ref:`good practices ` for virtualenv, test layout +* :ref:`existingtestsuite` for working with pre-existing tests * :ref:`fixtures` for providing a functional baseline to your tests -* :ref:`apiref` for documentation and examples on using ``pytest`` * :ref:`plugins` managing and writing plugins -.. _`installation issues`: - -Known Installation issues ------------------------------- - -easy_install or pip not found? -++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -.. _`install pip`: http://www.pip-installer.org/en/latest/index.html - -`Install pip`_ for a state of the art python package installer. - -Install `setuptools`_ to get ``easy_install`` which allows to install -``.egg`` binary format packages in addition to source-based ones. - -py.test not found on Windows despite installation? -++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -.. _`Python for Windows`: http://www.imladris.com/Scripts/PythonForWindows.html - -- **Windows**: If "easy_install" or "py.test" are not found - you need to add the Python script path to your ``PATH``, see here: - `Python for Windows`_. You may alternatively use an `ActivePython install`_ - which does this for you automatically. - -.. _`ActivePython install`: http://www.activestate.com/activepython/downloads - -.. _`Jython does not create command line launchers`: http://bugs.jython.org/issue1491 - -- **Jython2.5.1 on Windows XP**: `Jython does not create command line launchers`_ - so ``py.test`` will not work correctly. You may install py.test on - CPython and type ``py.test --genscript=mytest`` and then use - ``jython mytest`` to run your tests with Jython using ``pytest``. - - :ref:`examples` for more complex examples - .. include:: links.inc diff --git a/doc/en/goodpractices.rst b/doc/en/goodpractices.rst index 2d8050bd9ca154b..16fdd24c3927259 100644 --- a/doc/en/goodpractices.rst +++ b/doc/en/goodpractices.rst @@ -16,10 +16,12 @@ Conventions for Python test discovery * If no arguments are specified then collection starts from :confval:`testpaths` (if configured) or the current directory. Alternatively, command line arguments can be used in any combination of directories, file names or node ids. -* recurse into directories, unless they match :confval:`norecursedirs` -* ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_. -* ``Test`` prefixed test classes (without an ``__init__`` method) -* ``test_`` prefixed test functions or methods are test items +* Recurse into directories, unless they match :confval:`norecursedirs`. +* In those directories, search for ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_. +* From those files, collect test items: + + * ``test_`` prefixed test functions or methods outside of class + * ``test_`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method) For examples of how to customize your test discovery :doc:`example/pythoncollection`. @@ -28,68 +30,106 @@ Within Python modules, ``pytest`` also discovers tests using the standard Choosing a test layout / import rules ------------------------------------------- +------------------------------------- ``pytest`` supports two common test layouts: -* putting tests into an extra directory outside your actual application - code, useful if you have many functional tests or for other reasons - want to keep tests separate from actual application code (often a good - idea):: +Tests outside application code +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Putting tests into an extra directory outside your actual application code +might be useful if you have many functional tests or for other reasons want +to keep tests separate from actual application code (often a good idea):: - setup.py # your setuptools Python package metadata + setup.py mypkg/ __init__.py - appmodule.py + app.py + view.py tests/ test_app.py + test_view.py ... +This way your tests can run easily against an installed version +of ``mypkg``. + +Note that using this scheme your test files must have **unique names**, because +``pytest`` will import them as *top-level* modules since there are no packages +to derive a full package name from. In other words, the test files in the example above will +be imported as ``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to +``sys.path``. -* inlining test directories into your application package, useful if you - have direct relation between (unit-)test and application modules and - want to distribute your tests along with your application:: +If you need to have test modules with the same name, you might add ``__init__.py`` files to your +``tests`` folder and subfolders, changing them to packages:: - setup.py # your setuptools Python package metadata + setup.py mypkg/ - __init__.py - appmodule.py ... - test/ - test_app.py - ... + tests/ + __init__.py + foo/ + __init__.py + test_view.py + bar/ + __init__.py + test_view.py + +Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test_view``, allowing +you to have modules with the same name. But now this introduces a subtle problem: in order to load +the test modules from the ``tests`` directory, pytest prepends the root of the repository to +``sys.path``, which adds the side-effect that now ``mypkg`` is also importable. +This is problematic if you are using a tool like `tox`_ to test your package in a virtual environment, +because you want to test the *installed* version of your package, not the local code from the repository. + +In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a +sub-directory of your root:: + + setup.py + src/ + mypkg/ + __init__.py + app.py + view.py + tests/ + __init__.py + foo/ + __init__.py + test_view.py + bar/ + __init__.py + test_view.py + + +This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this excellent +`blog post by Ionel Cristian Mărieș `_. -Important notes relating to both schemes: +Tests as part of application code +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- **make sure that "mypkg" is importable**, for example by typing once:: +Inlining test directories into your application package +is useful if you have direct relation between tests and application modules and +want to distribute them along with your application:: - pip install -e . # install package using setup.py in editable mode + setup.py + mypkg/ + __init__.py + app.py + view.py + test/ + __init__.py + test_app.py + test_view.py + ... -- **avoid "__init__.py" files in your test directories**. - This way your tests can run easily against an installed version - of ``mypkg``, independently from the installed package if it contains - the tests or not. +In this scheme, it is easy to run your tests using the ``--pyargs`` option:: -- With inlined tests you might put ``__init__.py`` into test - directories and make them installable as part of your application. - Using the ``py.test --pyargs mypkg`` invocation pytest will - discover where mypkg is installed and collect tests from there. - With the "external" test you can still distribute tests but they - will not be installed or become importable. + pytest --pyargs mypkg -Typically you can run tests by pointing to test directories or modules:: +``pytest`` will discover where ``mypkg`` is installed and collect tests from there. - py.test tests/test_app.py # for external test dirs - py.test mypkg/test/test_app.py # for inlined test dirs - py.test mypkg # run tests in all below test directories - py.test # run all tests below current dir - ... +Note that this layout also works in conjunction with the ``src`` layout mentioned in the previous section. -Because of the above ``editable install`` mode you can change your -source code (both tests and the app) and rerun tests at will. -Once you are done with your work, you can `use tox`_ to make sure -that the package is really correct and tests pass in all -required configurations. .. note:: @@ -125,7 +165,7 @@ required configurations. The reason for this somewhat evolved importing technique is that in larger projects multiple test modules might import from each other and thus deriving a canonical import name helps - to avoid surprises such as a test modules getting imported twice. + to avoid surprises such as a test module getting imported twice. .. _`virtualenv`: http://pypi.python.org/pypi/virtualenv @@ -142,19 +182,24 @@ for installing your application and any dependencies as well as the ``pytest`` package itself. This ensures your code and dependencies are isolated from the system Python installation. -If you frequently release code and want to make sure that your actual +You can then install your package in "editable" mode:: + + pip install -e . + +which lets you change your source code (both tests and application) and rerun tests at will. +This is similar to running `python setup.py develop` or `conda develop` in that it installs +your package using a symlink to your development code. + +Once you are done with your work and want to make sure that your actual package passes all tests you may want to look into `tox`_, the virtualenv test automation tool and its `pytest support -`_. +`_. Tox helps you to setup virtualenv environments with pre-defined dependencies and then executing a pre-configured test command with options. It will run tests against the installed package and not against your source code checkout, helping to detect packaging glitches. -Continuous integration services such as Jenkins_ can make use of the -``--junitxml=PATH`` option to create a JUnitXML file and generate reports. - Integrating with setuptools / ``python setup.py test`` / ``pytest-runner`` -------------------------------------------------------------------------- @@ -191,9 +236,18 @@ If you now type:: this will execute your tests using ``pytest-runner``. As this is a standalone version of ``pytest`` no prior installation whatsoever is required for calling the test command. You can also pass additional -arguments to py.test such as your test directory or other +arguments to pytest such as your test directory or other options using ``--addopts``. +You can also specify other pytest-ini options in your ``setup.cfg`` file +by putting them into a ``[tool:pytest]`` section: + +.. code-block:: ini + + [tool:pytest] + addopts = --verbose + python_files = testing/*/*.py + Manual Integration ^^^^^^^^^^^^^^^^^^ @@ -209,16 +263,17 @@ your own setuptools Test command for invoking pytest. class PyTest(TestCommand): - user_options = [('pytest-args=', 'a', "Arguments to pass to py.test")] + user_options = [('pytest-args=', 'a', "Arguments to pass to pytest")] def initialize_options(self): TestCommand.initialize_options(self) - self.pytest_args = [] + self.pytest_args = '' def run_tests(self): + import shlex #import here, cause outside the eggs aren't loaded import pytest - errno = pytest.main(self.pytest_args) + errno = pytest.main(shlex.split(self.pytest_args)) sys.exit(errno) @@ -238,41 +293,7 @@ using the ``--pytest-args`` or ``-a`` command-line option. For example:: python setup.py test -a "--durations=5" -is equivalent to running ``py.test --durations=5``. - - -.. _standalone: -.. _`genscript method`: - -(deprecated) Create a pytest standalone script ------------------------------------------------ - -.. deprecated:: 2.8 - -.. note:: - - ``genscript`` has been deprecated because: - - * It cannot support plugins, rendering its usefulness extremely limited; - * Tooling has become much better since ``genscript`` was introduced; - * It is possible to build a zipped ``pytest`` application without the - shortcomings above. - - There's no planned version in which this command will be removed - at the moment of this writing, but its use is discouraged for new - applications. - -If you are a maintainer or application developer and want people -who don't deal with python much to easily run tests you may generate -a standalone ``pytest`` script:: - - py.test --genscript=runtests.py - -This generates a ``runtests.py`` script which is a fully functional basic -``pytest`` script, running unchanged under Python2 and Python3. -You can tell people to download the script and then e.g. run it like this:: - - python runtests.py +is equivalent to running ``pytest --durations=5``. .. include:: links.inc diff --git a/doc/en/historical-notes.rst b/doc/en/historical-notes.rst new file mode 100644 index 000000000000000..028ceff9b174861 --- /dev/null +++ b/doc/en/historical-notes.rst @@ -0,0 +1,177 @@ +Historical Notes +================ + +This page lists features or behavior from previous versions of pytest which have changed over the years. They are +kept here as a historical note so users looking at old code can find documentation related to them. + +cache plugin integrated into the core +------------------------------------- + +.. versionadded:: 2.8 + +The functionality of the :ref:`core cache ` plugin was previously distributed +as a third party plugin named ``pytest-cache``. The core plugin +is compatible regarding command line options and API usage except that you +can only store/receive data between test runs that is json-serializable. + + +funcargs and ``pytest_funcarg__`` +--------------------------------- + +.. versionchanged:: 2.3 + +In versions prior to 2.3 there was no ``@pytest.fixture`` marker +and you had to use a magic ``pytest_funcarg__NAME`` prefix +for the fixture factory. This remains and will remain supported +but is not anymore advertised as the primary means of declaring fixture +functions. + + +``@pytest.yield_fixture`` decorator +----------------------------------- + +.. versionchanged:: 2.10 + +Prior to version 2.10, in order to use a ``yield`` statement to execute teardown code one +had to mark a fixture using the ``yield_fixture`` marker. From 2.10 onward, normal +fixtures can use ``yield`` directly so the ``yield_fixture`` decorator is no longer needed +and considered deprecated. + + +``[pytest]`` header in ``setup.cfg`` +------------------------------------ + +.. versionchanged:: 3.0 + +Prior to 3.0, the supported section name was ``[pytest]``. Due to how +this may collide with some distutils commands, the recommended +section name for ``setup.cfg`` files is now ``[tool:pytest]``. + +Note that for ``pytest.ini`` and ``tox.ini`` files the section +name is ``[pytest]``. + + +Applying marks to ``@pytest.mark.parametrize`` parameters +--------------------------------------------------------- + +.. versionchanged:: 3.1 + +Prior to version 3.1 the supported mechanism for marking values +used the syntax:: + + import pytest + @pytest.mark.parametrize("test_input,expected", [ + ("3+5", 8), + ("2+4", 6), + pytest.mark.xfail(("6*9", 42),), + ]) + def test_eval(test_input, expected): + assert eval(test_input) == expected + + +This was an initial hack to support the feature but soon was demonstrated to be incomplete, +broken for passing functions or applying multiple marks with the same name but different parameters. + +The old syntax is planned to be removed in pytest-4.0. + + +``@pytest.mark.parametrize`` argument names as a tuple +------------------------------------------------------ + +.. versionchanged:: 2.4 + +In versions prior to 2.4 one needed to specify the argument +names as a tuple. This remains valid but the simpler ``"name1,name2,..."`` +comma-separated-string syntax is now advertised first because +it's easier to write and produces less line noise. + + +setup: is now an "autouse fixture" +---------------------------------- + +.. versionchanged:: 2.3 + +During development prior to the pytest-2.3 release the name +``pytest.setup`` was used but before the release it was renamed +and moved to become part of the general fixture mechanism, +namely :ref:`autouse fixtures` + + +.. _string conditions: + +Conditions as strings instead of booleans +----------------------------------------- + +.. versionchanged:: 2.4 + +Prior to pytest-2.4 the only way to specify skipif/xfail conditions was +to use strings:: + + import sys + @pytest.mark.skipif("sys.version_info >= (3,3)") + def test_function(): + ... + +During test function setup the skipif condition is evaluated by calling +``eval('sys.version_info >= (3,0)', namespace)``. The namespace contains +all the module globals, and ``os`` and ``sys`` as a minimum. + +Since pytest-2.4 :ref:`boolean conditions ` are considered preferable +because markers can then be freely imported between test modules. +With strings you need to import not only the marker but all variables +used by the marker, which violates encapsulation. + +The reason for specifying the condition as a string was that ``pytest`` can +report a summary of skip conditions based purely on the condition string. +With conditions as booleans you are required to specify a ``reason`` string. + +Note that string conditions will remain fully supported and you are free +to use them if you have no need for cross-importing markers. + +The evaluation of a condition string in ``pytest.mark.skipif(conditionstring)`` +or ``pytest.mark.xfail(conditionstring)`` takes place in a namespace +dictionary which is constructed as follows: + +* the namespace is initialized by putting the ``sys`` and ``os`` modules + and the pytest ``config`` object into it. + +* updated with the module globals of the test function for which the + expression is applied. + +The pytest ``config`` object allows you to skip based on a test +configuration value which you might have added:: + + @pytest.mark.skipif("not config.getvalue('db')") + def test_function(...): + ... + +The equivalent with "boolean conditions" is:: + + @pytest.mark.skipif(not pytest.config.getvalue("db"), + reason="--db was not specified") + def test_function(...): + pass + +.. note:: + + You cannot use ``pytest.config.getvalue()`` in code + imported before pytest's argument parsing takes place. For example, + ``conftest.py`` files are imported before command line parsing and thus + ``config.getvalue()`` will not execute correctly. + +``pytest.set_trace()`` +---------------------- + +.. versionchanged:: 2.4 + +Previous to version 2.4 to set a break point in code one needed to use ``pytest.set_trace()``:: + + import pytest + def test_function(): + ... + pytest.set_trace() # invoke PDB debugger and tracing + + +This is no longer needed and one can use the native ``import pdb;pdb.set_trace()`` call directly. + +For more details see :ref:`breakpoints`. diff --git a/doc/en/index.rst b/doc/en/index.rst index 04b4512da309e26..66c59f08d349ccb 100644 --- a/doc/en/index.rst +++ b/doc/en/index.rst @@ -1,61 +1,90 @@ +:orphan: .. _features: pytest: helps you write better programs -============================================= +======================================= -**a mature full-featured Python testing tool** - - runs on Posix/Windows, Python 2.6-3.5, PyPy and (possibly still) Jython-2.5.1 - - free and open source software, distributed under the terms of the :ref:`MIT license ` - - **well tested** with more than a thousand tests against itself - - **strict backward compatibility policy** for safe pytest upgrades - - :ref:`comprehensive online ` and `PDF documentation `_ - - many :ref:`third party plugins ` and :ref:`builtin helpers `, - - used in :ref:`many small and large projects and organisations ` - - comes with many :ref:`tested examples ` +The ``pytest`` framework makes it easy to write small tests, yet +scales to support complex functional testing for applications and libraries. -**provides easy no-boilerplate testing** +An example of a simple test: - - makes it :ref:`easy to get started `, - has many :ref:`usage options ` - - :ref:`assert with the assert statement` - - helpful :ref:`traceback and failing assertion reporting ` - - :ref:`print debugging ` and :ref:`the - capturing of standard output during test execution ` +.. code-block:: python -**scales from simple unit to complex functional testing** + # content of test_sample.py + def inc(x): + return x + 1 - - :ref:`modular parametrizeable fixtures ` (new in 2.3, - continuously improved) - - :ref:`parametrized test functions ` - - :ref:`mark` - - :ref:`skipping` (improved in 2.4) - - :ref:`distribute tests to multiple CPUs ` through :ref:`xdist plugin ` - - :ref:`continuously re-run failing tests ` - - :doc:`cache` - - flexible :ref:`Python test discovery` + def test_answer(): + assert inc(3) == 5 -**integrates with other testing methods and tools**: - - multi-paradigm: pytest can run ``nose``, ``unittest`` and - ``doctest`` style test suites, including running testcases made for - Django and trial - - supports :ref:`good integration practices ` - - supports extended :ref:`xUnit style setup ` - - supports domain-specific :ref:`non-python tests` - - supports generating `test coverage reports - `_ - - supports :pep:`8` compliant coding styles in tests +To execute it:: -**extensive plugin and customization system**: + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item + + test_sample.py F [100%] + + ================================= FAILURES ================================= + _______________________________ test_answer ________________________________ + + def test_answer(): + > assert inc(3) == 5 + E assert 4 == 5 + E + where 4 = inc(3) + + test_sample.py:5: AssertionError + ========================= 1 failed in 0.12 seconds ========================= - - all collection, reporting, running aspects are delegated to hook functions - - customizations can be per-directory, per-project or per PyPI released plugin - - it is easy to add command line options or customize existing behaviour - - :ref:`easy to write your own plugins ` +Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. +See :ref:`Getting Started ` for more examples. -.. _`easy`: http://bruynooghe.blogspot.com/2009/12/skipping-slow-test-by-default-in-pytest.html +Features +-------- +- Detailed info on failing :ref:`assert statements ` (no need to remember ``self.assert*`` names); +- :ref:`Auto-discovery ` of test modules and functions; + +- :ref:`Modular fixtures ` for managing small or parametrized long-lived test resources; + +- Can run :ref:`unittest ` (including trial) and :ref:`nose ` test suites out of the box; + +- Python 2.7, Python 3.4+, PyPy 2.3, Jython 2.5 (untested); + +- Rich plugin architecture, with over 315+ `external plugins `_ and thriving community; + + +Documentation +------------- + +Please see :ref:`Contents ` for full documentation, including installation, tutorials and PDF documents. + + +Bugs/Requests +------------- + +Please use the `GitHub issue tracker `_ to submit bugs or request features. + + +Changelog +--------- + +Consult the :ref:`Changelog ` page for fixes and enhancements of each version. + + +License +------- + +Copyright Holger Krekel and others, 2004-2017. + +Distributed under the terms of the `MIT`_ license, pytest is free and open source software. + +.. _`MIT`: https://github.com/pytest-dev/pytest/blob/master/LICENSE diff --git a/doc/en/license.rst b/doc/en/license.rst index 3fc1dad52fdbf3c..b8c0dce1bea8e17 100644 --- a/doc/en/license.rst +++ b/doc/en/license.rst @@ -9,7 +9,7 @@ Distributed under the terms of the `MIT`_ license, pytest is free and open sourc The MIT License (MIT) - Copyright (c) 2004-2016 Holger Krekel and others + Copyright (c) 2004-2017 Holger Krekel and others Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in diff --git a/doc/en/links.inc b/doc/en/links.inc index 3d7863751e2c9e1..b69390baa84bdbc 100644 --- a/doc/en/links.inc +++ b/doc/en/links.inc @@ -6,7 +6,7 @@ .. _`pytest_nose`: plugin/nose.html .. _`reStructured Text`: http://docutils.sourceforge.net .. _`Python debugger`: http://docs.python.org/lib/module-pdb.html -.. _nose: https://nose.readthedocs.org/en/latest/ +.. _nose: https://nose.readthedocs.io/en/latest/ .. _pytest: http://pypi.python.org/pypi/pytest .. _mercurial: http://mercurial.selenic.com/wiki/ .. _`setuptools`: http://pypi.python.org/pypi/setuptools @@ -18,4 +18,4 @@ .. _hudson: http://hudson-ci.org/ .. _jenkins: http://jenkins-ci.org/ .. _tox: http://testrun.org/tox -.. _pylib: http://py.readthedocs.org/en/latest/ +.. _pylib: https://py.readthedocs.io/en/latest/ diff --git a/doc/en/logging.rst b/doc/en/logging.rst new file mode 100644 index 000000000000000..e3bf56038872c2e --- /dev/null +++ b/doc/en/logging.rst @@ -0,0 +1,192 @@ +.. _logging: + +Logging +------- + +.. versionadded 3.3.0 + +.. note:: + + This feature is a drop-in replacement for the `pytest-catchlog + `_ plugin and they will conflict + with each other. The backward compatibility API with ``pytest-capturelog`` + has been dropped when this feature was introduced, so if for that reason you + still need ``pytest-catchlog`` you can disable the internal feature by + adding to your ``pytest.ini``: + + .. code-block:: ini + + [pytest] + addopts=-p no:logging + +Log messages are captured by default and for each failed test will be shown in +the same manner as captured stdout and stderr. + +Running without options:: + + pytest + +Shows failed tests like so:: + + ----------------------- Captured stdlog call ---------------------- + test_reporting.py 26 INFO text going to logger + ----------------------- Captured stdout call ---------------------- + text going to stdout + ----------------------- Captured stderr call ---------------------- + text going to stderr + ==================== 2 failed in 0.02 seconds ===================== + +By default each captured log message shows the module, line number, log level +and message. Showing the exact module and line number is useful for testing and +debugging. If desired the log format and date format can be specified to +anything that the logging module supports. + +Running pytest specifying formatting options:: + + pytest --log-format="%(asctime)s %(levelname)s %(message)s" \ + --log-date-format="%Y-%m-%d %H:%M:%S" + +Shows failed tests like so:: + + ----------------------- Captured stdlog call ---------------------- + 2010-04-10 14:48:44 INFO text going to logger + ----------------------- Captured stdout call ---------------------- + text going to stdout + ----------------------- Captured stderr call ---------------------- + text going to stderr + ==================== 2 failed in 0.02 seconds ===================== + +These options can also be customized through a configuration file: + +.. code-block:: ini + + [pytest] + log_format = %(asctime)s %(levelname)s %(message)s + log_date_format = %Y-%m-%d %H:%M:%S + +Further it is possible to disable reporting logs on failed tests completely +with:: + + pytest --no-print-logs + +Or in you ``pytest.ini``: + +.. code-block:: ini + + [pytest] + log_print = False + + +Shows failed tests in the normal manner as no logs were captured:: + + ----------------------- Captured stdout call ---------------------- + text going to stdout + ----------------------- Captured stderr call ---------------------- + text going to stderr + ==================== 2 failed in 0.02 seconds ===================== + +Inside tests it is possible to change the log level for the captured log +messages. This is supported by the ``caplog`` fixture:: + + def test_foo(caplog): + caplog.set_level(logging.INFO) + pass + +By default the level is set on the handler used to catch the log messages, +however as a convenience it is also possible to set the log level of any +logger:: + + def test_foo(caplog): + caplog.set_level(logging.CRITICAL, logger='root.baz') + pass + +It is also possible to use a context manager to temporarily change the log +level:: + + def test_bar(caplog): + with caplog.at_level(logging.INFO): + pass + +Again, by default the level of the handler is affected but the level of any +logger can be changed instead with:: + + def test_bar(caplog): + with caplog.at_level(logging.CRITICAL, logger='root.baz'): + pass + +Lastly all the logs sent to the logger during the test run are made available on +the fixture in the form of both the LogRecord instances and the final log text. +This is useful for when you want to assert on the contents of a message:: + + def test_baz(caplog): + func_under_test() + for record in caplog.records: + assert record.levelname != 'CRITICAL' + assert 'wally' not in caplog.text + +For all the available attributes of the log records see the +``logging.LogRecord`` class. + +You can also resort to ``record_tuples`` if all you want to do is to ensure, +that certain messages have been logged under a given logger name with a given +severity and message:: + + def test_foo(caplog): + logging.getLogger().info('boo %s', 'arg') + + assert caplog.record_tuples == [ + ('root', logging.INFO, 'boo arg'), + ] + +You can call ``caplog.clear()`` to reset the captured log records in a test:: + + def test_something_with_clearing_records(caplog): + some_method_that_creates_log_records() + caplog.clear() + your_test_method() + assert ['Foo'] == [rec.message for rec in caplog.records] + +Live Logs +^^^^^^^^^ + +By default, pytest will output any logging records with a level higher or +equal to WARNING. In order to actually see these logs in the console you have to +disable pytest output capture by passing ``-s``. + +You can specify the logging level for which log records with equal or higher +level are printed to the console by passing ``--log-cli-level``. This setting +accepts the logging level names as seen in python's documentation or an integer +as the logging level num. + +Additionally, you can also specify ``--log-cli-format`` and +``--log-cli-date-format`` which mirror and default to ``--log-format`` and +``--log-date-format`` if not provided, but are applied only to the console +logging handler. + +All of the CLI log options can also be set in the configuration INI file. The +option names are: + +* ``log_cli_level`` +* ``log_cli_format`` +* ``log_cli_date_format`` + +If you need to record the whole test suite logging calls to a file, you can pass +``--log-file=/path/to/log/file``. This log file is opened in write mode which +means that it will be overwritten at each run tests session. + +You can also specify the logging level for the log file by passing +``--log-file-level``. This setting accepts the logging level names as seen in +python's documentation(ie, uppercased level names) or an integer as the logging +level num. + +Additionally, you can also specify ``--log-file-format`` and +``--log-file-date-format`` which are equal to ``--log-format`` and +``--log-date-format`` but are applied to the log file logging handler. + +All of the log file options can also be set in the configuration INI file. The +option names are: + +* ``log_file`` +* ``log_file_level`` +* ``log_file_format`` +* ``log_file_date_format`` diff --git a/doc/en/mark.rst b/doc/en/mark.rst index ab9546d317c5d59..0b0e072a09bae29 100644 --- a/doc/en/mark.rst +++ b/doc/en/mark.rst @@ -10,6 +10,7 @@ By using the ``pytest.mark`` helper you can easily set metadata on your test functions. There are some builtin markers, for example: +* :ref:`skip ` - always skip a test function * :ref:`skipif ` - skip a test function if a certain condition is met * :ref:`xfail ` - produce an "expected failure" outcome if a certain condition is met diff --git a/doc/en/monkeypatch.rst b/doc/en/monkeypatch.rst index 4155a3a345b5bf6..0c07b2f44fcbb25 100644 --- a/doc/en/monkeypatch.rst +++ b/doc/en/monkeypatch.rst @@ -6,7 +6,7 @@ Monkeypatching/mocking modules and environments Sometimes tests need to invoke functionality which depends on global settings or which invokes code which cannot be easily -tested such as network access. The ``monkeypatch`` function argument +tested such as network access. The ``monkeypatch`` fixture helps you to safely set/delete an attribute, dictionary item or environment variable or to modify ``sys.path`` for importing. See the `monkeypatch blog post`_ for some introduction material @@ -14,6 +14,7 @@ and a discussion of its motivation. .. _`monkeypatch blog post`: http://tetamap.wordpress.com/2009/03/03/monkeypatching-in-unit-tests-done-right/ + Simple example: monkeypatching functions --------------------------------------------------- @@ -34,7 +35,7 @@ patch this function before calling into a function which uses it:: assert x == '/abc/.ssh' Here our test function monkeypatches ``os.path.expanduser`` and -then calls into an function that calls it. After the test function +then calls into a function that calls it. After the test function finishes the ``os.path.expanduser`` modification will be undone. example: preventing "requests" from remote operations @@ -53,28 +54,20 @@ This autouse fixture will be executed for each test function and it will delete the method ``request.session.Session.request`` so that any attempts within tests to create http requests will fail. -example: setting an attribute on some class ------------------------------------------------------- - -If you need to patch out ``os.getcwd()`` to return an artificial -value:: - - def test_some_interaction(monkeypatch): - monkeypatch.setattr("os.getcwd", lambda: "/") -which is equivalent to the long form:: - - def test_some_interaction(monkeypatch): - import os - monkeypatch.setattr(os, "getcwd", lambda: "/") +.. note:: + + Be advised that it is not recommended to patch builtin functions such as ``open``, + ``compile``, etc., because it might break pytest's internals. If that's + unavoidable, passing ``--tb=native``, ``--assert=plain`` and ``--capture=no`` might + help although there's no guarantee. +Method reference of the monkeypatch fixture +------------------------------------------- -Method reference of the monkeypatch function argument ------------------------------------------------------ - -.. autoclass:: monkeypatch - :members: setattr, replace, delattr, setitem, delitem, setenv, delenv, syspath_prepend, chdir, undo +.. autoclass:: MonkeyPatch + :members: ``monkeypatch.setattr/delattr/delitem/delenv()`` all by default raise an Exception if the target does not exist. diff --git a/doc/en/nose.rst b/doc/en/nose.rst index 3b92e04cffd0972..10a10633ab06e74 100644 --- a/doc/en/nose.rst +++ b/doc/en/nose.rst @@ -1,3 +1,5 @@ +.. _`noseintegration`: + Running tests written for nose ======================================= @@ -13,7 +15,7 @@ Usage After :ref:`installation` type:: python setup.py develop # make sure tests can import our package - py.test # instead of 'nosetests' + pytest # instead of 'nosetests' and you should be able to run your nose style tests and make use of pytest's capabilities. @@ -24,7 +26,7 @@ Supported nose Idioms * setup and teardown at module/class/method level * SkipTest exceptions and markers * setup/teardown decorators -* yield-based tests and their setup +* ``yield``-based tests and their setup (considered deprecated as of pytest 3.0) * ``__test__`` attribute on modules/classes/functions * general usage of nose utilities @@ -45,11 +47,29 @@ Unsupported idioms / known issues ``tests.test_mod``) but different file system paths (e.g. ``tests/test_mode.py`` and ``other/tests/test_mode.py``) by extending sys.path/import semantics. pytest does not do that - but there is discussion in `issue268 `_ for adding some support. Note that - `nose2 choose to avoid this sys.path/import hackery `_. + but there is discussion in `#268 `_ for adding some support. Note that + `nose2 choose to avoid this sys.path/import hackery `_. + + If you place a conftest.py file in the root directory of your project + (as determined by pytest) pytest will run tests "nose style" against + the code below that directory by adding it to your ``sys.path`` instead of + running against your installed code. + + You may find yourself wanting to do this if you ran ``python setup.py install`` + to set up your project, as opposed to ``python setup.py develop`` or any of + the package manager equivalents. Installing with develop in a + virtual environment like Tox is recommended over this pattern. - nose-style doctests are not collected and executed correctly, also doctest fixtures don't work. -- no nose-configuration is recognized +- no nose-configuration is recognized. + +- ``yield``-based methods don't support ``setup`` properly because + the ``setup`` method is always called in the same class instance. + There are no plans to fix this currently because ``yield``-tests + are deprecated in pytest 3.0, with ``pytest.mark.parametrize`` + being the recommended alternative. + + diff --git a/doc/en/overview.rst b/doc/en/overview.rst deleted file mode 100644 index eb261977514f2f2..000000000000000 --- a/doc/en/overview.rst +++ /dev/null @@ -1,13 +0,0 @@ -================================================== -Getting started basics -================================================== - -.. toctree:: - :maxdepth: 2 - - getting-started - usage - goodpractices - projects - faq - diff --git a/doc/en/parametrize.rst b/doc/en/parametrize.rst index 919ac93d25aeaa5..7a4ac2e1877e25d 100644 --- a/doc/en/parametrize.rst +++ b/doc/en/parametrize.rst @@ -9,17 +9,16 @@ Parametrizing fixtures and test functions ========================================================================== -pytest supports test parametrization in several well-integrated ways: +pytest enables test parametrization at several levels: -- :py:func:`pytest.fixture` allows to define :ref:`parametrization - at the level of fixture functions `. +- :py:func:`pytest.fixture` allows one to :ref:`parametrize fixture + functions `. -* `@pytest.mark.parametrize`_ allows to define parametrization at the - function or class level, provides multiple argument/fixture sets - for a particular test function or class. +* `@pytest.mark.parametrize`_ allows one to define multiple sets of + arguments and fixtures at the test function or class. -* `pytest_generate_tests`_ enables implementing your own custom - dynamic parametrization scheme or extensions. +* `pytest_generate_tests`_ allows one to define custom parametrization + schemes or extensions. .. _parametrizemark: .. _`@pytest.mark.parametrize`: @@ -53,16 +52,16 @@ Here, the ``@parametrize`` decorator defines three different ``(test_input,expec tuples so that the ``test_eval`` function will run three times using them in turn:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 3 items - test_expectation.py ..F + test_expectation.py ..F [100%] - ======= FAILURES ======== - _______ test_eval[6*9-42] ________ + ================================= FAILURES ================================= + ____________________________ test_eval[6*9-42] _____________________________ test_input = '6*9', expected = 42 @@ -73,11 +72,11 @@ them in turn:: ]) def test_eval(test_input, expected): > assert eval(test_input) == expected - E assert 54 == 42 + E AssertionError: assert 54 == 42 E + where 54 = eval('6*9') test_expectation.py:8: AssertionError - ======= 1 failed, 2 passed in 0.12 seconds ======== + ==================== 1 failed, 2 passed in 0.12 seconds ==================== As designed in this example, only one pair of input/output values fails the simple test function. And as usual with test function arguments, @@ -94,22 +93,23 @@ for example with the builtin ``mark.xfail``:: @pytest.mark.parametrize("test_input,expected", [ ("3+5", 8), ("2+4", 6), - pytest.mark.xfail(("6*9", 42)), + pytest.param("6*9", 42, + marks=pytest.mark.xfail), ]) def test_eval(test_input, expected): assert eval(test_input) == expected Let's run this:: - $ py.test - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 3 items - test_expectation.py ..x + test_expectation.py ..x [100%] - ======= 2 passed, 1 xfailed in 0.12 seconds ======== + =================== 2 passed, 1 xfailed in 0.12 seconds ==================== The one parameter set which caused a failure previously now shows up as an "xfailed (expected to fail)" test. @@ -123,15 +123,8 @@ To get all combinations of multiple parametrized arguments you can stack def test_foo(x, y): pass -This will run the test with the arguments set to x=0/y=2, x=0/y=3, x=1/y=2 and -x=1/y=3. - -.. note:: - - In versions prior to 2.4 one needed to specify the argument - names as a tuple. This remains valid but the simpler ``"name1,name2,..."`` - comma-separated-string syntax is now advertised first because - it's easier to write and produces less line noise. +This will run the test with the arguments set to ``x=0/y=2``, ``x=0/y=3``, ``x=1/y=2`` and +``x=1/y=3``. .. _`pytest_generate_tests`: @@ -167,27 +160,28 @@ command line option and the parametrization of our test function:: def pytest_generate_tests(metafunc): if 'stringinput' in metafunc.fixturenames: metafunc.parametrize("stringinput", - metafunc.config.option.stringinput) + metafunc.config.getoption('stringinput')) If we now pass two stringinput values, our test will run twice:: - $ py.test -q --stringinput="hello" --stringinput="world" test_strings.py - .. + $ pytest -q --stringinput="hello" --stringinput="world" test_strings.py + .. [100%] 2 passed in 0.12 seconds Let's also run with a stringinput that will lead to a failing test:: - $ py.test -q --stringinput="!" test_strings.py - F - ======= FAILURES ======== - _______ test_valid_string[!] ________ + $ pytest -q --stringinput="!" test_strings.py + F [100%] + ================================= FAILURES ================================= + ___________________________ test_valid_string[!] ___________________________ stringinput = '!' def test_valid_string(stringinput): > assert stringinput.isalpha() - E assert () - E + where = '!'.isalpha + E AssertionError: assert False + E + where False = () + E + where = '!'.isalpha test_strings.py:3: AssertionError 1 failed in 0.12 seconds @@ -198,12 +192,18 @@ If you don't specify a stringinput it will be skipped because ``metafunc.parametrize()`` will be called with an empty parameter list:: - $ py.test -q -rs test_strings.py - s - ======= short test summary info ======== - SKIP [1] $PYTHON_PREFIX/lib/python3.4/site-packages/_pytest/python.py:1419: got empty parameter set, function test_valid_string at $REGENDOC_TMPDIR/test_strings.py:1 + $ pytest -q -rs test_strings.py + s [100%] + ========================= short test summary info ========================== + SKIP [1] test_strings.py: got empty parameter set ['stringinput'], function test_valid_string at $REGENDOC_TMPDIR/test_strings.py:1 1 skipped in 0.12 seconds +Note that when calling ``metafunc.parametrize`` multiple times with different parameter sets, all parameter names across +those sets cannot be duplicated, otherwise an error will be raised. + +More examples +------------- + For further examples, you might want to look at :ref:`more parametrization examples `. @@ -214,6 +214,4 @@ The **metafunc** object .. currentmodule:: _pytest.python .. autoclass:: Metafunc - - .. automethod:: Metafunc.parametrize - .. automethod:: Metafunc.addcall(funcargs=None,id=_notexists,param=_notexists) + :members: diff --git a/doc/en/plugins.rst b/doc/en/plugins.rst index e9b3f460cc46a26..400418aee25867b 100644 --- a/doc/en/plugins.rst +++ b/doc/en/plugins.rst @@ -27,9 +27,6 @@ Here is a little annotated list for some popular plugins: for `twisted `_ apps, starting a reactor and processing deferreds from test functions. -* `pytest-catchlog `_: - to capture and assert about messages from the logging module - * `pytest-cov `_: coverage reporting, compatible with distributed testing @@ -37,7 +34,7 @@ Here is a little annotated list for some popular plugins: to distribute tests to CPUs and remote hosts, to run in boxed mode which allows to survive segmentation faults, to run in looponfailing mode, automatically re-running failing tests - on file changes, see also :ref:`xdist` + on file changes. * `pytest-instafail `_: to report failures while the test run is happening. @@ -59,7 +56,7 @@ Here is a little annotated list for some popular plugins: a plugin to run javascript unittests in live browsers. To see a complete list of all plugins with their latest testing -status against different py.test and Python versions, please visit +status against different pytest and Python versions, please visit `plugincompat `_. You may also discover more plugins through a `pytest- pypi.python.org search`_. @@ -90,11 +87,11 @@ Finding out which plugins are active If you want to find out which plugins are active in your environment you can type:: - py.test --traceconfig + pytest --trace-config and will get an extended test header which shows activated plugins and their names. It will also print local plugins aka -:ref:`conftest.py ` files when they are loaded. +:ref:`conftest.py ` files when they are loaded. .. _`cmdunregister`: @@ -103,7 +100,7 @@ Deactivating / unregistering a plugin by name You can prevent plugins from loading or unregister them:: - py.test -p no:NAME + pytest -p no:NAME This means that any subsequent try to activate/load the named plugin will not work. @@ -138,14 +135,13 @@ in the `pytest repository `_. _pytest.capture _pytest.config _pytest.doctest - _pytest.genscript _pytest.helpconfig _pytest.junitxml _pytest.mark _pytest.monkeypatch _pytest.nose _pytest.pastebin - _pytest.pdb + _pytest.debugging _pytest.pytester _pytest.python _pytest.recwarn @@ -156,4 +152,3 @@ in the `pytest repository `_. _pytest.terminal _pytest.tmpdir _pytest.unittest - diff --git a/doc/en/projects.rst b/doc/en/projects.rst index 76d004916cea94f..86df99ab2c5fc0b 100644 --- a/doc/en/projects.rst +++ b/doc/en/projects.rst @@ -57,8 +57,8 @@ Here are some examples of projects using ``pytest`` (please send notes via :ref: * `bu `_ a microscopic build system * `katcp `_ Telescope communication protocol over Twisted * `kss plugin timer `_ -* `pyudev `_ a pure Python binding to the Linux library libudev -* `pytest-localserver `_ a plugin for pytest that provides a httpserver and smtpserver +* `pyudev `_ a pure Python binding to the Linux library libudev +* `pytest-localserver `_ a plugin for pytest that provides an httpserver and smtpserver * `pytest-monkeyplus `_ a plugin that extends monkeypatch These projects help integrate ``pytest`` into other Python frameworks: diff --git a/doc/en/proposals/parametrize_with_fixtures.rst b/doc/en/proposals/parametrize_with_fixtures.rst new file mode 100644 index 000000000000000..146032aa4713e91 --- /dev/null +++ b/doc/en/proposals/parametrize_with_fixtures.rst @@ -0,0 +1,160 @@ +:orphan: + +=================================== +PROPOSAL: Parametrize with fixtures +=================================== + +.. warning:: + + This document outlines a proposal around using fixtures as input + of parametrized tests or fixtures. + +Problem +------- + +As a user I have functional tests that I would like to run against various +scenarios. + +In this particular example we want to generate a new project based on a +cookiecutter template. We want to test default values but also data that +emulates user input. + +- use default values + +- emulate user input + + - specify 'author' + + - specify 'project_slug' + + - specify 'author' and 'project_slug' + +This is how a functional test could look like: + +.. code-block:: python + + import pytest + + @pytest.fixture + def default_context(): + return {'extra_context': {}} + + + @pytest.fixture(params=[ + {'author': 'alice'}, + {'project_slug': 'helloworld'}, + {'author': 'bob', 'project_slug': 'foobar'}, + ]) + def extra_context(request): + return {'extra_context': request.param} + + + @pytest.fixture(params=['default', 'extra']) + def context(request): + if request.param == 'default': + return request.getfuncargvalue('default_context') + else: + return request.getfuncargvalue('extra_context') + + + def test_generate_project(cookies, context): + """Call the cookiecutter API to generate a new project from a + template. + """ + result = cookies.bake(extra_context=context) + + assert result.exit_code == 0 + assert result.exception is None + assert result.project.isdir() + + +Issues +------ + +* By using ``request.getfuncargvalue()`` we rely on actual fixture function + execution to know what fixtures are involved, due to it's dynamic nature +* More importantly, ``request.getfuncargvalue()`` cannot be combined with + parametrized fixtures, such as ``extra_context`` +* This is very inconvenient if you wish to extend an existing test suite by + certain parameters for fixtures that are already used by tests + +pytest version 3.0 reports an error if you try to run above code:: + + Failed: The requested fixture has no parameter defined for the current + test. + + Requested fixture 'extra_context' + + +Proposed solution +----------------- + +A new function that can be used in modules can be used to dynamically define +fixtures from existing ones. + +.. code-block:: python + + pytest.define_combined_fixture( + name='context', + fixtures=['default_context', 'extra_context'], + ) + +The new fixture ``context`` inherits the scope from the used fixtures and yield +the following values. + +- ``{}`` + +- ``{'author': 'alice'}`` + +- ``{'project_slug': 'helloworld'}`` + +- ``{'author': 'bob', 'project_slug': 'foobar'}`` + +Alternative approach +-------------------- + +A new helper function named ``fixture_request`` would tell pytest to yield +all parameters marked as a fixture. + +.. note:: + + The `pytest-lazy-fixture `_ plugin implements a very + similar solution to the proposal below, make sure to check it out. + +.. code-block:: python + + @pytest.fixture(params=[ + pytest.fixture_request('default_context'), + pytest.fixture_request('extra_context'), + ]) + def context(request): + """Returns all values for ``default_context``, one-by-one before it + does the same for ``extra_context``. + + request.param: + - {} + - {'author': 'alice'} + - {'project_slug': 'helloworld'} + - {'author': 'bob', 'project_slug': 'foobar'} + """ + return request.param + +The same helper can be used in combination with ``pytest.mark.parametrize``. + +.. code-block:: python + + + @pytest.mark.parametrize( + 'context, expected_response_code', + [ + (pytest.fixture_request('default_context'), 0), + (pytest.fixture_request('extra_context'), 0), + ], + ) + def test_generate_project(cookies, context, exit_code): + """Call the cookiecutter API to generate a new project from a + template. + """ + result = cookies.bake(extra_context=context) + + assert result.exit_code == exit_code diff --git a/doc/en/pythonpath.rst b/doc/en/pythonpath.rst new file mode 100644 index 000000000000000..b6474276887dbcb --- /dev/null +++ b/doc/en/pythonpath.rst @@ -0,0 +1,76 @@ +.. _pythonpath: + +pytest import mechanisms and ``sys.path``/``PYTHONPATH`` +======================================================== + +Here's a list of scenarios where pytest may need to change ``sys.path`` in order +to import test modules or ``conftest.py`` files. + +Test modules / ``conftest.py`` files inside packages +---------------------------------------------------- + +Consider this file and directory layout:: + + root/ + |- foo/ + |- __init__.py + |- conftest.py + |- bar/ + |- __init__.py + |- tests/ + |- __init__.py + |- test_foo.py + + +When executing:: + + pytest root/ + + + +pytest will find ``foo/bar/tests/test_foo.py`` and realize it is part of a package given that +there's an ``__init__.py`` file in the same folder. It will then search upwards until it can find the +last folder which still contains an ``__init__.py`` file in order to find the package *root* (in +this case ``foo/``). To load the module, it will insert ``root/`` to the front of +``sys.path`` (if not there already) in order to load +``test_foo.py`` as the *module* ``foo.bar.tests.test_foo``. + +The same logic applies to the ``conftest.py`` file: it will be imported as ``foo.conftest`` module. + +Preserving the full package name is important when tests live in a package to avoid problems +and allow test modules to have duplicated names. This is also discussed in details in +:ref:`test discovery`. + +Standalone test modules / ``conftest.py`` files +----------------------------------------------- + +Consider this file and directory layout:: + + root/ + |- foo/ + |- conftest.py + |- bar/ + |- tests/ + |- test_foo.py + + +When executing:: + + pytest root/ + +pytest will find ``foo/bar/tests/test_foo.py`` and realize it is NOT part of a package given that +there's no ``__init__.py`` file in the same folder. It will then add ``root/foo/bar/tests`` to +``sys.path`` in order to import ``test_foo.py`` as the *module* ``test_foo``. The same is done +with the ``conftest.py`` file by adding ``root/foo`` to ``sys.path`` to import it as ``conftest``. + +For this reason this layout cannot have test modules with the same name, as they all will be +imported in the global import namespace. + +This is also discussed in details in :ref:`test discovery`. + +Invoking ``pytest`` versus ``python -m pytest`` +----------------------------------------------- + +Running pytest with ``python -m pytest [...]`` instead of ``pytest [...]`` yields nearly +equivalent behaviour, except that the former call will add the current directory to ``sys.path``. +See also :ref:`cmdline`. diff --git a/doc/en/recwarn.rst b/doc/en/recwarn.rst index 3c42bfaaf94b892..513af0d450ed3c7 100644 --- a/doc/en/recwarn.rst +++ b/doc/en/recwarn.rst @@ -1,130 +1,3 @@ +:orphan: -Asserting Warnings -===================================================== - -.. _warns: - -Asserting warnings with the warns function ------------------------------------------------ - -.. versionadded:: 2.8 - -You can check that code raises a particular warning using ``pytest.warns``, -which works in a similar manner to :ref:`raises `:: - - import warnings - import pytest - - def test_warning(): - with pytest.warns(UserWarning): - warnings.warn("my warning", UserWarning) - -The test will fail if the warning in question is not raised. - -You can also call ``pytest.warns`` on a function or code string:: - - pytest.warns(expected_warning, func, *args, **kwargs) - pytest.warns(expected_warning, "func(*args, **kwargs)") - -The function also returns a list of all raised warnings (as -``warnings.WarningMessage`` objects), which you can query for -additional information:: - - with pytest.warns(RuntimeWarning) as record: - warnings.warn("another warning", RuntimeWarning) - - # check that only one warning was raised - assert len(record) == 1 - # check that the message matches - assert record[0].message.args[0] == "another warning" - -Alternatively, you can examine raised warnings in detail using the -:ref:`recwarn ` fixture (see below). - -.. note:: - ``DeprecationWarning`` and ``PendingDeprecationWarning`` are treated - differently; see :ref:`ensuring_function_triggers`. - -.. _recwarn: - -Recording warnings ------------------------- - -You can record raised warnings either using ``pytest.warns`` or with -the ``recwarn`` fixture. - -To record with ``pytest.warns`` without asserting anything about the warnings, -pass ``None`` as the expected warning type:: - - with pytest.warns(None) as record: - warnings.warn("user", UserWarning) - warnings.warn("runtime", RuntimeWarning) - - assert len(record) == 2 - assert str(record[0].message) == "user" - assert str(record[1].message) == "runtime" - -The ``recwarn`` fixture will record warnings for the whole function:: - - import warnings - - def test_hello(recwarn): - warnings.warn("hello", UserWarning) - assert len(recwarn) == 1 - w = recwarn.pop(UserWarning) - assert issubclass(w.category, UserWarning) - assert str(w.message) == "hello" - assert w.filename - assert w.lineno - -Both ``recwarn`` and ``pytest.warns`` return the same interface for recorded -warnings: a WarningsRecorder instance. To view the recorded warnings, you can -iterate over this instance, call ``len`` on it to get the number of recorded -warnings, or index into it to get a particular recorded warning. It also -provides these methods: - -.. autoclass:: _pytest.recwarn.WarningsRecorder() - :members: - -Each recorded warning has the attributes ``message``, ``category``, -``filename``, ``lineno``, ``file``, and ``line``. The ``category`` is the -class of the warning. The ``message`` is the warning itself; calling -``str(message)`` will return the actual message of the warning. - -.. note:: - ``DeprecationWarning`` and ``PendingDeprecationWarning`` are treated - differently; see :ref:`ensuring_function_triggers`. - -.. _ensuring_function_triggers: - -Ensuring a function triggers a deprecation warning -------------------------------------------------------- - -You can also call a global helper for checking -that a certain function call triggers a ``DeprecationWarning`` or -``PendingDeprecationWarning``:: - - import pytest - - def test_global(): - pytest.deprecated_call(myfunction, 17) - -By default, ``DeprecationWarning`` and ``PendingDeprecationWarning`` will not be -caught when using ``pytest.warns`` or ``recwarn`` because default Python warnings filters hide -them. If you wish to record them in your own code, use the -command ``warnings.simplefilter('always')``:: - - import warnings - import pytest - - def test_deprecation(recwarn): - warnings.simplefilter('always') - warnings.warn("deprecated", DeprecationWarning) - assert len(recwarn) == 1 - assert recwarn.pop(DeprecationWarning) - -You can also use it as a contextmanager:: - - def test_global(): - with pytest.deprecated_call(): - myobject.deprecated_method() +This page has been moved, please see :ref:`assertwarnings`. diff --git a/doc/en/requirements.txt b/doc/en/requirements.txt new file mode 100644 index 000000000000000..72bb60a811ff7b1 --- /dev/null +++ b/doc/en/requirements.txt @@ -0,0 +1,3 @@ +# pinning sphinx to 1.4.* due to search issues with rtd: +# https://github.com/rtfd/readthedocs-sphinx-ext/issues/25 +sphinx ==1.4.* diff --git a/doc/en/setup.rst b/doc/en/setup.rst deleted file mode 100644 index fe235346544e4c2..000000000000000 --- a/doc/en/setup.rst +++ /dev/null @@ -1,10 +0,0 @@ - -setup: is now an "autouse fixture" -======================================================== - -During development prior to the pytest-2.3 release the name -``pytest.setup`` was used but before the release it was renamed -and moved to become part of the general fixture mechanism, -namely :ref:`autouse fixtures` - - diff --git a/doc/en/skipping.rst b/doc/en/skipping.rst index 4282afb77ec3b24..7e001929b23755f 100644 --- a/doc/en/skipping.rst +++ b/doc/en/skipping.rst @@ -2,32 +2,40 @@ .. _skipping: -Skip and xfail: dealing with tests that can not succeed -===================================================================== +Skip and xfail: dealing with tests that cannot succeed +====================================================== -If you have test functions that cannot be run on certain platforms -or that you expect to fail you can mark them accordingly or you -may call helper functions during execution of setup or test functions. +You can mark test functions that cannot be run on certain platforms +or that you expect to fail so pytest can deal with them accordingly and +present a summary of the test session, while keeping the test suite *green*. -A *skip* means that you expect your test to pass unless the environment -(e.g. wrong Python interpreter, missing dependency) prevents it to run. -And *xfail* means that your test can run but you expect it to fail -because there is an implementation problem. +A **skip** means that you expect your test to pass only if some conditions are met, +otherwise pytest should skip running the test altogether. Common examples are skipping +windows-only tests on non-windows platforms, or skipping tests that depend on an external +resource which is not available at the moment (for example a database). + +A **xfail** means that you expect a test to fail for some reason. +A common example is a test for a feature not yet implemented, or a bug not yet fixed. +When a test passes despite being expected to fail (marked with ``pytest.mark.xfail``), +it's an **xpass** and will be reported in the test summary. ``pytest`` counts and lists *skip* and *xfail* tests separately. Detailed information about skipped/xfailed tests is not shown by default to avoid cluttering the output. You can use the ``-r`` option to see details corresponding to the "short" letters shown in the test progress:: - py.test -rxs # show extra info on skips and xfails + pytest -rxXs # show extra info on xfailed, xpassed, and skipped tests + +More details on the ``-r`` option can be found by running ``pytest -h``. (See :ref:`how to change command line options defaults`) .. _skipif: +.. _skip: .. _`condition booleans`: -Marking a test function to be skipped -------------------------------------------- +Skipping test functions +----------------------- .. versionadded:: 2.9 @@ -40,31 +48,50 @@ which may be passed an optional ``reason``: def test_the_unknown(): ... + +Alternatively, it is also possible to skip imperatively during test execution or setup +by calling the ``pytest.skip(reason)`` function: + +.. code-block:: python + + def test_function(): + if not valid_config(): + pytest.skip("unsupported configuration") + +It is also possible to skip the whole module using +``pytest.skip(reason, allow_module_level=True)`` at the module level: + +.. code-block:: python + + import pytest + + if not pytest.config.getoption("--custom-flag"): + pytest.skip("--custom-flag is missing, skipping tests", allow_module_level=True) + +The imperative method is useful when it is not possible to evaluate the skip condition +during import time. + ``skipif`` ~~~~~~~~~~ -.. versionadded:: 2.0, 2.4 +.. versionadded:: 2.0 If you wish to skip something conditionally then you can use ``skipif`` instead. Here is an example of marking a test function to be skipped -when run on a Python3.3 interpreter:: +when run on a Python3.6 interpreter:: import sys - @pytest.mark.skipif(sys.version_info < (3,3), - reason="requires python3.3") + @pytest.mark.skipif(sys.version_info < (3,6), + reason="requires python3.6") def test_function(): ... -During test function setup the condition ("sys.version_info >= (3,3)") is -checked. If it evaluates to True, the test function will be skipped -with the specified reason. Note that pytest enforces specifying a reason -in order to report meaningful "skip reasons" (e.g. when using ``-rs``). -If the condition is a string, it will be evaluated as python expression. +If the condition evaluates to ``True`` during collection, the test function will be skipped, +with the specified reason appearing in the summary when using ``-rs``. -You can share skipif markers between modules. Consider this test module:: +You can share ``skipif`` markers between modules. Consider this test module:: # content of test_mymodule.py - import mymodule minversion = pytest.mark.skipif(mymodule.__versioninfo__ < (1,1), reason="at least mymodule-1.1 required") @@ -72,7 +99,7 @@ You can share skipif markers between modules. Consider this test module:: def test_function(): ... -You can import it from another test module:: +You can import the marker and reuse it in another test module:: # test_myothermodule.py from test_mymodule import minversion @@ -85,28 +112,33 @@ For larger test suites it's usually a good idea to have one file where you define the markers which you then consistently apply throughout your test suite. -Alternatively, the pre pytest-2.4 way to specify :ref:`condition strings -` instead of booleans will remain fully supported in future -versions of pytest. It couldn't be easily used for importing markers -between test modules so it's no longer advertised as the primary method. +Alternatively, you can use :ref:`condition strings +` instead of booleans, but they can't be shared between modules easily +so they are supported mainly for backward compatibility reasons. Skip all test functions of a class or module ---------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can use the ``skipif`` decorator (and any other marker) on classes:: +You can use the ``skipif`` marker (as any other marker) on classes:: @pytest.mark.skipif(sys.platform == 'win32', reason="does not run on windows") - class TestPosixCalls: + class TestPosixCalls(object): def test_function(self): "will not be setup or run under 'win32' platform" -If the condition is true, this marker will produce a skip result for -each of the test methods. +If the condition is ``True``, this marker will produce a skip result for +each of the test methods of that class. + +.. warning:: + + The use of ``skipif`` on classes that use inheritance is strongly + discouraged. `A Known bug `_ + in pytest's markers may cause unexpected behavior in super classes. -If you want to skip all test functions of a module, you must use +If you want to skip all test functions of a module, you may use the ``pytestmark`` name on the global level: .. code-block:: python @@ -114,15 +146,67 @@ the ``pytestmark`` name on the global level: # test_module.py pytestmark = pytest.mark.skipif(...) -If multiple "skipif" decorators are applied to a test function, it +If multiple ``skipif`` decorators are applied to a test function, it will be skipped if any of the skip conditions is true. .. _`whole class- or module level`: mark.html#scoped-marking + +Skipping files or directories +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Sometimes you may need to skip an entire file or directory, for example if the +tests rely on Python version-specific features or contain code that you do not +wish pytest to run. In this case, you must exclude the files and directories +from collection. Refer to :ref:`customizing-test-collection` for more +information. + + +Skipping on a missing import dependency +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can use the following helper at module level +or within a test or test setup function:: + + docutils = pytest.importorskip("docutils") + +If ``docutils`` cannot be imported here, this will lead to a +skip outcome of the test. You can also skip based on the +version number of a library:: + + docutils = pytest.importorskip("docutils", minversion="0.3") + +The version will be read from the specified +module's ``__version__`` attribute. + +Summary +~~~~~~~ + +Here's a quick guide on how to skip tests in a module in different situations: + +1. Skip all tests in a module unconditionally: + + .. code-block:: python + + pytestmark = pytest.mark.skip('all tests still WIP') + +2. Skip all tests in a module based on some condition: + + .. code-block:: python + + pytestmark = pytest.mark.skipif(sys.platform == 'win32', 'tests for linux only') + +3. Skip all tests in a module if some import is missing: + + .. code-block:: python + + pexpect = pytest.importorskip('pexpect') + + .. _xfail: -Mark a test function as expected to fail -------------------------------------------------------- +XFail: mark test functions as expected to fail +---------------------------------------------- You can use the ``xfail`` marker to indicate that you expect a test to fail:: @@ -135,6 +219,29 @@ This test will be run but no traceback will be reported when it fails. Instead terminal reporting will list it in the "expected to fail" (``XFAIL``) or "unexpectedly passing" (``XPASS``) sections. +Alternatively, you can also mark a test as ``XFAIL`` from within a test or setup function +imperatively: + +.. code-block:: python + + def test_function(): + if not valid_config(): + pytest.xfail("failing configuration (but should work)") + +This will unconditionally make ``test_function`` ``XFAIL``. Note that no other code is executed +after ``pytest.xfail`` call, differently from the marker. That's because it is implemented +internally by raising a known exception. + +Here's the signature of the ``xfail`` **marker** (not the function), using Python 3 keyword-only +arguments syntax: + +.. code-block:: python + + def xfail(condition=None, *, reason=None, raises=None, run=True, strict=False): + + + + ``strict`` parameter ~~~~~~~~~~~~~~~~~~~~ @@ -167,8 +274,8 @@ You can change the default value of the ``strict`` parameter using the As with skipif_ you can also mark your expectation of a failure on a particular platform:: - @pytest.mark.xfail(sys.version_info >= (3,3), - reason="python3.3 api changes") + @pytest.mark.xfail(sys.version_info >= (3,6), + reason="python3.6 api changes") def test_function(): ... @@ -200,18 +307,19 @@ even executed, use the ``run`` parameter as ``False``: def test_function(): ... -This is specially useful for marking crashing tests for later inspection. +This is specially useful for xfailing tests that are crashing the interpreter and should be +investigated later. -Ignoring xfail marks -~~~~~~~~~~~~~~~~~~~~ +Ignoring xfail +~~~~~~~~~~~~~~ By specifying on the commandline:: pytest --runxfail you can force the running and reporting of an ``xfail`` marked test -as if it weren't marked at all. +as if it weren't marked at all. This also causes ``pytest.xfail`` to produce no effect. Examples ~~~~~~~~ @@ -222,14 +330,14 @@ Here is a simple test file with the several usages: Running it with the report-on-xfail option gives this output:: - example $ py.test -rx xfail_demo.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR/example, inifile: + example $ pytest -rx xfail_demo.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR/example, inifile: collected 7 items - xfail_demo.py xxxxxxx - ======= short test summary info ======== + xfail_demo.py xxxxxxx [100%] + ========================= short test summary info ========================== XFAIL xfail_demo.py::test_hello XFAIL xfail_demo.py::test_hello2 reason: [NOTRUN] @@ -243,19 +351,7 @@ Running it with the report-on-xfail option gives this output:: reason: reason XFAIL xfail_demo.py::test_hello7 - ======= 7 xfailed in 0.12 seconds ======== - -xfail signature summary -~~~~~~~~~~~~~~~~~~~~~~~ - -Here's the signature of the ``xfail`` marker, using Python 3 keyword-only -arguments syntax: - -.. code-block:: python - - def xfail(condition=None, *, reason=None, raises=None, run=True, strict=False): - - + ======================== 7 xfailed in 0.12 seconds ========================= .. _`skip/xfail with parametrize`: @@ -263,111 +359,20 @@ Skip/xfail with parametrize --------------------------- It is possible to apply markers like skip and xfail to individual -test instances when using parametrize:: +test instances when using parametrize: + +.. code-block:: python import pytest @pytest.mark.parametrize(("n", "expected"), [ (1, 2), - pytest.mark.xfail((1, 0)), - pytest.mark.xfail(reason="some bug")((1, 3)), + pytest.param(1, 0, marks=pytest.mark.xfail), + pytest.param(1, 3, marks=pytest.mark.xfail(reason="some bug")), (2, 3), (3, 4), (4, 5), - pytest.mark.skipif("sys.version_info >= (3,0)")((10, 11)), + pytest.param(10, 11, marks=pytest.mark.skipif(sys.version_info >= (3, 0), reason="py2k")), ]) def test_increment(n, expected): assert n + 1 == expected - - -Imperative xfail from within a test or setup function ------------------------------------------------------- - -If you cannot declare xfail- of skipif conditions at import -time you can also imperatively produce an according outcome -imperatively, in test or setup code:: - - def test_function(): - if not valid_config(): - pytest.xfail("failing configuration (but should work)") - # or - pytest.skip("unsupported configuration") - - -Skipping on a missing import dependency --------------------------------------------------- - -You can use the following import helper at module level -or within a test or test setup function:: - - docutils = pytest.importorskip("docutils") - -If ``docutils`` cannot be imported here, this will lead to a -skip outcome of the test. You can also skip based on the -version number of a library:: - - docutils = pytest.importorskip("docutils", minversion="0.3") - -The version will be read from the specified -module's ``__version__`` attribute. - - -.. _string conditions: - -specifying conditions as strings versus booleans ----------------------------------------------------------- - -Prior to pytest-2.4 the only way to specify skipif/xfail conditions was -to use strings:: - - import sys - @pytest.mark.skipif("sys.version_info >= (3,3)") - def test_function(): - ... - -During test function setup the skipif condition is evaluated by calling -``eval('sys.version_info >= (3,0)', namespace)``. The namespace contains -all the module globals, and ``os`` and ``sys`` as a minimum. - -Since pytest-2.4 `condition booleans`_ are considered preferable -because markers can then be freely imported between test modules. -With strings you need to import not only the marker but all variables -everything used by the marker, which violates encapsulation. - -The reason for specifying the condition as a string was that ``pytest`` can -report a summary of skip conditions based purely on the condition string. -With conditions as booleans you are required to specify a ``reason`` string. - -Note that string conditions will remain fully supported and you are free -to use them if you have no need for cross-importing markers. - -The evaluation of a condition string in ``pytest.mark.skipif(conditionstring)`` -or ``pytest.mark.xfail(conditionstring)`` takes place in a namespace -dictionary which is constructed as follows: - -* the namespace is initialized by putting the ``sys`` and ``os`` modules - and the pytest ``config`` object into it. - -* updated with the module globals of the test function for which the - expression is applied. - -The pytest ``config`` object allows you to skip based on a test -configuration value which you might have added:: - - @pytest.mark.skipif("not config.getvalue('db')") - def test_function(...): - ... - -The equivalent with "boolean conditions" is:: - - @pytest.mark.skipif(not pytest.config.getvalue("db"), - reason="--db was not specified") - def test_function(...): - pass - -.. note:: - - You cannot use ``pytest.config.getvalue()`` in code - imported before py.test's argument parsing takes place. For example, - ``conftest.py`` files are imported before command line parsing and thus - ``config.getvalue()`` will not execute correctly. diff --git a/doc/en/status.rst b/doc/en/status.rst deleted file mode 100644 index 3c7bf70eaa483bb..000000000000000 --- a/doc/en/status.rst +++ /dev/null @@ -1,5 +0,0 @@ -pytest development status -================================ - -https://travis-ci.org/pytest-dev/pytest - diff --git a/doc/en/talks.rst b/doc/en/talks.rst index 7a5221845ce9b7e..bf593db4b4b101b 100644 --- a/doc/en/talks.rst +++ b/doc/en/talks.rst @@ -2,17 +2,26 @@ Talks and Tutorials ========================== -.. sidebar:: Next Open Trainings +.. + .. sidebar:: Next Open Trainings - `professional testing with pytest and tox `_, 27-29th June 2016, Freiburg, Germany + `Professional Testing with Python + `_, + 26-28 April 2017, Leipzig, Germany. .. _`funcargs`: funcargs.html +Books +--------------------------------------------- + +- `Python Testing with pytest, by Brian Okken (2017) + `_. + Talks and blog postings --------------------------------------------- -.. _`tutorial1 repository`: http://bitbucket.org/pytest-dev/pytest-tutorial1/ -.. _`pycon 2010 tutorial PDF`: http://bitbucket.org/pytest-dev/pytest-tutorial1/raw/tip/pytest-basic.pdf +- `Pythonic testing, Igor Starikov (Russian, PyNsk, November 2016) + `_. - `pytest - Rapid Simple Testing, Florian Bruhin, Swiss Python Summit 2016 `_. @@ -52,12 +61,14 @@ Talks and blog postings - `pytest introduction from Brian Okken (January 2013) `_ -- `monkey patching done right`_ (blog post, consult `monkeypatch - plugin`_ for up-to-date API) +- pycon australia 2012 pytest talk from Brianna Laugher (`video `_, `slides `_, `code `_) +- `pycon 2012 US talk video from Holger Krekel `_ + +- `monkey patching done right`_ (blog post, consult `monkeypatch plugin`_ for up-to-date API) Test parametrization: -- `generating parametrized tests with funcargs`_ (uses deprecated ``addcall()`` API. +- `generating parametrized tests with fixtures`_. - `test generators and cached setup`_ - `parametrizing tests, generalized`_ (blog post) - `putting test-hooks into local or global plugins`_ (blog post) @@ -78,39 +89,17 @@ Plugin specific examples: - `many examples in the docs for plugins`_ .. _`skipping slow tests by default in pytest`: http://bruynooghe.blogspot.com/2009/12/skipping-slow-test-by-default-in-pytest.html -.. _`many examples in the docs for plugins`: plugin/index.html -.. _`monkeypatch plugin`: plugin/monkeypatch.html -.. _`application setup in test functions with funcargs`: funcargs.html#appsetup +.. _`many examples in the docs for plugins`: plugins.html +.. _`monkeypatch plugin`: monkeypatch.html +.. _`application setup in test functions with fixtures`: fixture.html#interdependent-fixtures .. _`simultaneously test your code on all platforms`: http://tetamap.wordpress.com/2009/03/23/new-simultanously-test-your-code-on-all-platforms/ .. _`monkey patching done right`: http://tetamap.wordpress.com/2009/03/03/monkeypatching-in-unit-tests-done-right/ .. _`putting test-hooks into local or global plugins`: http://tetamap.wordpress.com/2009/05/14/putting-test-hooks-into-local-and-global-plugins/ .. _`parametrizing tests, generalized`: http://tetamap.wordpress.com/2009/05/13/parametrizing-python-tests-generalized/ -.. _`generating parametrized tests with funcargs`: funcargs.html#test-generators +.. _`generating parametrized tests with fixtures`: parametrize.html#test-generators .. _`test generators and cached setup`: http://bruynooghe.blogspot.com/2010/06/pytest-test-generators-and-cached-setup.html -Older conference talks and tutorials ----------------------------------------- - -- `pycon australia 2012 pytest talk from Brianna Laugher - `_ (`video `_, `slides `_, `code `_) -- `pycon 2012 US talk video from Holger Krekel `_ -- `pycon 2010 tutorial PDF`_ and `tutorial1 repository`_ - -- `ep2009-rapidtesting.pdf`_ tutorial slides (July 2009): - - - testing terminology - - basic pytest usage, file system layout - - test function arguments (funcargs_) and test fixtures - - existing plugins - - distributed testing -- `ep2009-pytest.pdf`_ 60 minute pytest talk, highlighting unique features and a roadmap (July 2009) -- `pycon2009-pytest-introduction.zip`_ slides and files, extended version of pytest basic introduction, discusses more options, also introduces old-style xUnit setup, looponfailing and other features. -- `pycon2009-pytest-advanced.pdf`_ contain a slightly older version of funcargs and distributed testing, compared to the EuroPython 2009 slides. -.. _`ep2009-rapidtesting.pdf`: http://codespeak.net/download/py/ep2009-rapidtesting.pdf -.. _`ep2009-pytest.pdf`: http://codespeak.net/download/py/ep2009-pytest.pdf -.. _`pycon2009-pytest-introduction.zip`: http://codespeak.net/download/py/pycon2009-pytest-introduction.zip -.. _`pycon2009-pytest-advanced.pdf`: http://codespeak.net/download/py/pycon2009-pytest-advanced.pdf diff --git a/doc/en/test/attic.rst b/doc/en/test/attic.rst index 6408c722584d68a..06944661cb5ce71 100644 --- a/doc/en/test/attic.rst +++ b/doc/en/test/attic.rst @@ -21,7 +21,7 @@ but note that project specific settings will be considered first. There is a flag that helps you debugging your conftest.py configurations:: - py.test --traceconfig + pytest --trace-config customizing the collecting and running process @@ -110,7 +110,7 @@ If you want to disable a complete test class you can set the class-level attribute ``disabled``. For example, in order to avoid running some tests on Win32:: - class TestPosixOnly: + class TestPosixOnly(object): disabled = sys.platform == 'win32' def test_xxx(self): diff --git a/doc/en/test/mission.rst b/doc/en/test/mission.rst index cda8d9a7208f827..51c252dc0d86a4c 100644 --- a/doc/en/test/mission.rst +++ b/doc/en/test/mission.rst @@ -5,7 +5,7 @@ Mission ``pytest`` strives to make testing a fun and no-boilerplate effort. The tool is distributed as a `pytest` package. Its project independent -``py.test`` command line tool helps you to: +``pytest`` command line tool helps you to: * rapidly collect and run tests * run unit- or doctests, functional or integration tests diff --git a/doc/en/test/plugin/cov.rst b/doc/en/test/plugin/cov.rst index 355093f2586ebce..541c7ef947934ea 100644 --- a/doc/en/test/plugin/cov.rst +++ b/doc/en/test/plugin/cov.rst @@ -53,7 +53,7 @@ subprocesses. Running centralised testing:: - py.test --cov myproj tests/ + pytest --cov myproj tests/ Shows a terminal report:: @@ -76,7 +76,7 @@ file system. Each slave will have it's subprocesses measured. Running distributed testing with dist mode set to load:: - py.test --cov myproj -n 2 tests/ + pytest --cov myproj -n 2 tests/ Shows a terminal report:: @@ -92,7 +92,7 @@ Shows a terminal report:: Again but spread over different hosts and different directories:: - py.test --cov myproj --dist load + pytest --cov myproj --dist load --tx ssh=memedough@host1//chdir=testenv1 --tx ssh=memedough@host2//chdir=/tmp/testenv2//python=/tmp/env1/bin/python --rsyncdir myproj --rsyncdir tests --rsync examples @@ -119,7 +119,7 @@ environments. Running distributed testing with dist mode set to each:: - py.test --cov myproj --dist each + pytest --cov myproj --dist each --tx popen//chdir=/tmp/testenv3//python=/usr/local/python27/bin/python --tx ssh=memedough@host2//chdir=/tmp/testenv4//python=/tmp/env2/bin/python --rsyncdir myproj --rsyncdir tests --rsync examples @@ -149,7 +149,7 @@ annotated source code. The terminal report without line numbers (default):: - py.test --cov-report term --cov myproj tests/ + pytest --cov-report term --cov myproj tests/ -------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- Name Stmts Miss Cover @@ -163,7 +163,7 @@ The terminal report without line numbers (default):: The terminal report with line numbers:: - py.test --cov-report term-missing --cov myproj tests/ + pytest --cov-report term-missing --cov myproj tests/ -------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- Name Stmts Miss Cover Missing @@ -178,7 +178,7 @@ The terminal report with line numbers:: The remaining three reports output to files without showing anything on the terminal (useful for when the output is going to a continuous integration server):: - py.test --cov-report html --cov-report xml --cov-report annotate --cov myproj tests/ + pytest --cov-report html --cov-report xml --cov-report annotate --cov myproj tests/ Coverage Data File diff --git a/doc/en/test/plugin/coverage.rst b/doc/en/test/plugin/coverage.rst index 965b4a4ee078020..71139d008ba020a 100644 --- a/doc/en/test/plugin/coverage.rst +++ b/doc/en/test/plugin/coverage.rst @@ -26,7 +26,7 @@ Usage To get full test coverage reports for a particular package type:: - py.test --cover-report=report + pytest --cover-report=report command line options -------------------- diff --git a/doc/en/test/plugin/figleaf.rst b/doc/en/test/plugin/figleaf.rst index 86e0da65b6badc7..0c1603ade9fc3df 100644 --- a/doc/en/test/plugin/figleaf.rst +++ b/doc/en/test/plugin/figleaf.rst @@ -24,7 +24,7 @@ Usage After installation you can simply type:: - py.test --figleaf [...] + pytest --figleaf [...] to enable figleaf coverage in your test run. A default ".figleaf" data file and "html" directory will be created. You can use command line options diff --git a/doc/en/test/plugin/genscript.rst b/doc/en/test/plugin/genscript.rst deleted file mode 100644 index ee80f233fa06977..000000000000000 --- a/doc/en/test/plugin/genscript.rst +++ /dev/null @@ -1,28 +0,0 @@ - -(deprecated) generate standalone test script to be distributed along with an application. -============================================================================ - - -.. contents:: - :local: - - - -command line options --------------------- - - -``--genscript=path`` - create standalone ``pytest`` script at given target path. - -Start improving this plugin in 30 seconds -========================================= - - -1. Download `pytest_genscript.py`_ plugin source code -2. put it somewhere as ``pytest_genscript.py`` into your import path -3. a subsequent ``pytest`` run will use your local version - -Checkout customize_, other plugins_ or `get in contact`_. - -.. include:: links.txt diff --git a/doc/en/test/plugin/helpconfig.rst b/doc/en/test/plugin/helpconfig.rst index 9b5b8cddd9df0c2..326b75c4552e142 100644 --- a/doc/en/test/plugin/helpconfig.rst +++ b/doc/en/test/plugin/helpconfig.rst @@ -16,10 +16,8 @@ command line options display py lib version and import information. ``-p name`` early-load given plugin (multi-allowed). -``--traceconfig`` +``--trace-config`` trace considerations of conftest.py files. -``--nomagic`` - don't reinterpret asserts, no traceback cutting. ``--debug`` generate and show internal debugging information. ``--help-config`` diff --git a/doc/en/test/plugin/links.rst b/doc/en/test/plugin/links.rst index aa965e73049cf0c..6dec2b4848a69e5 100644 --- a/doc/en/test/plugin/links.rst +++ b/doc/en/test/plugin/links.rst @@ -2,10 +2,8 @@ .. _`pytest_recwarn.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_recwarn.py .. _`unittest`: unittest.html .. _`pytest_monkeypatch.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_monkeypatch.py -.. _`pytest_genscript.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_genscript.py .. _`pastebin`: pastebin.html .. _`skipping`: skipping.html -.. _`genscript`: genscript.html .. _`plugins`: index.html .. _`mark`: mark.html .. _`tmpdir`: tmpdir.html diff --git a/doc/en/test/plugin/nose.rst b/doc/en/test/plugin/nose.rst index f3aa7d705f5eb26..9eeae5ff697f05a 100644 --- a/doc/en/test/plugin/nose.rst +++ b/doc/en/test/plugin/nose.rst @@ -14,7 +14,7 @@ Usage type:: - py.test # instead of 'nosetests' + pytest # instead of 'nosetests' and you should be able to run nose style tests and at the same time can make full use of pytest's capabilities. @@ -38,7 +38,7 @@ Unsupported idioms / issues If you find other issues or have suggestions please run:: - py.test --pastebin=all + pytest --pastebin=all and send the resulting URL to a ``pytest`` contact channel, at best to the mailing list. diff --git a/doc/en/test/plugin/terminal.rst b/doc/en/test/plugin/terminal.rst index 214c24dfccd466f..e07d4f72183a9a2 100644 --- a/doc/en/test/plugin/terminal.rst +++ b/doc/en/test/plugin/terminal.rst @@ -18,11 +18,9 @@ command line options show extra test summary info as specified by chars (f)ailed, (s)skipped, (x)failed, (X)passed. ``-l, --showlocals`` show locals in tracebacks (disabled by default). -``--report=opts`` - (deprecated, use -r) ``--tb=style`` traceback print mode (long/short/line/no). -``--fulltrace`` +``--full-trace`` don't cut any tracebacks (default is to cut). ``--fixtures`` show available function arguments, sorted by plugin diff --git a/doc/en/test/plugin/xdist.rst b/doc/en/test/plugin/xdist.rst index 7ab6cdc8b831992..506d240aee13c7d 100644 --- a/doc/en/test/plugin/xdist.rst +++ b/doc/en/test/plugin/xdist.rst @@ -36,7 +36,7 @@ Speed up test runs by sending tests to multiple CPUs To send tests to multiple CPUs, type:: - py.test -n NUM + pytest -n NUM Especially for longer running tests or tests requiring a lot of IO this can lead to considerable speed ups. @@ -47,7 +47,7 @@ Running tests in a Python subprocess To instantiate a python2.4 sub process and send tests to it, you may type:: - py.test -d --tx popen//python=python2.4 + pytest -d --tx popen//python=python2.4 This will start a subprocess which is run with the "python2.4" Python interpreter, found in your system binary lookup path. @@ -68,10 +68,10 @@ tests that you can successfully run locally. And you have a ssh-reachable machine ``myhost``. Then you can ad-hoc distribute your tests by typing:: - py.test -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg + pytest -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg This will synchronize your ``mypkg`` package directory -to an remote ssh account and then locally collect tests +to a remote ssh account and then locally collect tests and send them to remote places for execution. You can specify multiple ``--rsyncdir`` directories @@ -97,7 +97,7 @@ It will tell you that it starts listening on the default port. You can now on your home machine specify this new socket host with something like this:: - py.test -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg + pytest -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg .. _`atonce`: @@ -107,7 +107,7 @@ Running tests on many platforms at once The basic command to run tests on multiple platforms is:: - py.test --dist=each --tx=spec1 --tx=spec2 + pytest --dist=each --tx=spec1 --tx=spec2 If you specify a windows host, an OSX host and a Linux environment this command will send each tests to all diff --git a/doc/en/tmpdir.rst b/doc/en/tmpdir.rst index f8935b8cef85b9e..b8174484e1d7613 100644 --- a/doc/en/tmpdir.rst +++ b/doc/en/tmpdir.rst @@ -27,16 +27,16 @@ and more. Here is an example test usage:: Running this would result in a passed test except for the last ``assert 0`` line which we use to look at values:: - $ py.test test_tmpdir.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: - collected 1 items + $ pytest test_tmpdir.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item - test_tmpdir.py F + test_tmpdir.py F [100%] - ======= FAILURES ======== - _______ test_create_file ________ + ================================= FAILURES ================================= + _____________________________ test_create_file _____________________________ tmpdir = local('PYTEST_TMPDIR/test_create_file0') @@ -49,7 +49,7 @@ Running this would result in a passed test except for the last E assert 0 test_tmpdir.py:7: AssertionError - ======= 1 failed in 0.12 seconds ======== + ========================= 1 failed in 0.12 seconds ========================= The 'tmpdir_factory' fixture ---------------------------- @@ -100,7 +100,7 @@ than 3 temporary directories will be removed. You can override the default temporary directory setting like this:: - py.test --basetemp=mydir + pytest --basetemp=mydir When distributing tests on the local machine, ``pytest`` takes care to configure a basetemp directory for the sub processes such that all temporary diff --git a/doc/en/unittest.rst b/doc/en/unittest.rst index ce99bd11867853d..b44bda44fa83384 100644 --- a/doc/en/unittest.rst +++ b/doc/en/unittest.rst @@ -1,39 +1,78 @@ .. _`unittest.TestCase`: +.. _`unittest`: -Support for unittest.TestCase / Integration of fixtures -===================================================================== +unittest.TestCase Support +========================= -.. _`unittest.py style`: http://docs.python.org/library/unittest.html +``pytest`` supports running Python ``unittest``-based tests out of the box. +It's meant for leveraging existing ``unittest``-based test suites +to use pytest as a test runner and also allow to incrementally adapt +the test suite to take full advantage of pytest's features. -``pytest`` has support for running Python `unittest.py style`_ tests. -It's meant for leveraging existing unittest-style projects -to use pytest features. Concretely, pytest will automatically -collect ``unittest.TestCase`` subclasses and their ``test`` methods in -test files. It will invoke typical setup/teardown methods and -generally try to make test suites written to run on unittest, to also -run using ``pytest``. We assume here that you are familiar with writing -``unittest.TestCase`` style tests and rather focus on -integration aspects. +To run an existing ``unittest``-style test suite using ``pytest``, type:: -Usage -------------------------------------------------------------------- + pytest tests -After :ref:`installation` type:: - py.test +pytest will automatically collect ``unittest.TestCase`` subclasses and +their ``test`` methods in ``test_*.py`` or ``*_test.py`` files. -and you should be able to run your unittest-style tests if they -are contained in ``test_*`` modules. If that works for you then -you can make use of most :ref:`pytest features `, for example -``--pdb`` debugging in failures, using :ref:`plain assert-statements `, -:ref:`more informative tracebacks `, stdout-capturing or -distributing tests to multiple CPUs via the ``-nNUM`` option if you -installed the ``pytest-xdist`` plugin. Please refer to -the general ``pytest`` documentation for many more examples. +Almost all ``unittest`` features are supported: -Mixing pytest fixtures into unittest.TestCase style tests ------------------------------------------------------------ +* ``@unittest.skip`` style decorators; +* ``setUp/tearDown``; +* ``setUpClass/tearDownClass()``; + +.. _`load_tests protocol`: https://docs.python.org/3/library/unittest.html#load-tests-protocol +.. _`setUpModule/tearDownModule`: https://docs.python.org/3/library/unittest.html#setupmodule-and-teardownmodule +.. _`subtests`: https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests + +Up to this point pytest does not have support for the following features: + +* `load_tests protocol`_; +* `setUpModule/tearDownModule`_; +* `subtests`_; + +Benefits out of the box +----------------------- + +By running your test suite with pytest you can make use of several features, +in most cases without having to modify existing code: + +* Obtain :ref:`more informative tracebacks `; +* :ref:`stdout and stderr ` capturing; +* :ref:`Test selection options ` using ``-k`` and ``-m`` flags; +* :ref:`maxfail`; +* :ref:`--pdb ` command-line option for debugging on test failures + (see :ref:`note ` below); +* Distribute tests to multiple CPUs using the `pytest-xdist `_ plugin; +* Use :ref:`plain assert-statements ` instead of ``self.assert*`` functions (`unittest2pytest + `__ is immensely helpful in this); + + +pytest features in ``unittest.TestCase`` subclasses +--------------------------------------------------- + +The following pytest features work in ``unittest.TestCase`` subclasses: + +* :ref:`Marks `: :ref:`skip `, :ref:`skipif `, :ref:`xfail `; +* :ref:`Auto-use fixtures `; + +The following pytest features **do not** work, and probably +never will due to different design philosophies: + +* :ref:`Fixtures ` (except for ``autouse`` fixtures, see :ref:`below `); +* :ref:`Parametrization `; +* :ref:`Custom hooks `; + + +Third party plugins may or may not work well, depending on the plugin and the test suite. + +.. _mixing-fixtures: + +Mixing pytest fixtures into ``unittest.TestCase`` subclasses using marks +------------------------------------------------------------------------ Running your unittest with ``pytest`` allows you to use its :ref:`fixture mechanism ` with ``unittest.TestCase`` style @@ -51,7 +90,7 @@ it from a unittest-style test:: @pytest.fixture(scope="class") def db_class(request): - class DummyDB: + class DummyDB(object): pass # set a class attribute on the invoking test context request.cls.db = DummyDB() @@ -86,16 +125,16 @@ the pytest fixture function ``db_class`` is called once per class. Due to the deliberately failing assert statements, we can take a look at the ``self.db`` values in the traceback:: - $ py.test test_unittest_db.py - ======= test session starts ======== - platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 - rootdir: $REGENDOC_TMPDIR, inifile: + $ pytest test_unittest_db.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: collected 2 items - test_unittest_db.py FF + test_unittest_db.py FF [100%] - ======= FAILURES ======== - _______ MyTest.test_method1 ________ + ================================= FAILURES ================================= + ___________________________ MyTest.test_method1 ____________________________ self = @@ -106,7 +145,7 @@ the ``self.db`` values in the traceback:: E assert 0 test_unittest_db.py:9: AssertionError - _______ MyTest.test_method2 ________ + ___________________________ MyTest.test_method2 ____________________________ self = @@ -116,15 +155,15 @@ the ``self.db`` values in the traceback:: E assert 0 test_unittest_db.py:12: AssertionError - ======= 2 failed in 0.12 seconds ======== + ========================= 2 failed in 0.12 seconds ========================= This default pytest traceback shows that the two test methods share the same ``self.db`` instance which was our intention when writing the class-scoped fixture function above. -autouse fixtures and accessing other fixtures -------------------------------------------------------------------- +Using autouse fixtures and accessing other fixtures +--------------------------------------------------- Although it's usually better to explicitly declare use of fixtures you need for a given test, you may sometimes want to have fixtures that are @@ -145,13 +184,15 @@ creation of a per-test temporary directory:: import unittest class MyTest(unittest.TestCase): + @pytest.fixture(autouse=True) def initdir(self, tmpdir): tmpdir.chdir() # change to pytest-provided temporary directory tmpdir.join("samplefile.ini").write("# testdata") def test_method(self): - s = open("samplefile.ini").read() + with open("samplefile.ini") as f: + s = f.read() assert "testdata" in s Due to the ``autouse`` flag the ``initdir`` fixture function will be @@ -161,8 +202,8 @@ on the class like in the previous example. Running this test module ...:: - $ py.test -q test_unittest_cleandir.py - . + $ pytest -q test_unittest_cleandir.py + . [100%] 1 passed in 0.12 seconds ... gives us one passed test because the ``initdir`` fixture function @@ -170,21 +211,35 @@ was executed ahead of the ``test_method``. .. note:: - While pytest supports receiving fixtures via :ref:`test function arguments ` for non-unittest test methods, ``unittest.TestCase`` methods cannot directly receive fixture - function arguments as implementing that is likely to inflict + ``unittest.TestCase`` methods cannot directly receive fixture + arguments as implementing that is likely to inflict on the ability to run general unittest.TestCase test suites. - Maybe optional support would be possible, though. If unittest finally - grows a plugin system that should help as well. In the meanwhile, the - above ``usefixtures`` and ``autouse`` examples should help to mix in - pytest fixtures into unittest suites. And of course you can also start - to selectively leave away the ``unittest.TestCase`` subclassing, use - plain asserts and get the unlimited pytest feature set. + The above ``usefixtures`` and ``autouse`` examples should help to mix in + pytest fixtures into unittest suites. + + You can also gradually move away from subclassing from ``unittest.TestCase`` to *plain asserts* + and then start to benefit from the full pytest feature set step by step. -Converting from unittest to pytest ---------------------------------------- +.. _pdb-unittest-note: + +.. note:: + + Running tests from ``unittest.TestCase`` subclasses with ``--pdb`` will + disable tearDown and cleanup methods for the case that an Exception + occurs. This allows proper post mortem debugging for all applications + which have significant logic in their tearDown machinery. However, + supporting this feature has the following side effect: If people + overwrite ``unittest.TestCase`` ``__call__`` or ``run``, they need to + to overwrite ``debug`` in the same way (this is also true for standard + unittest). + +.. note:: -If you want to convert your unittest testcases to pytest, there are -some helpers like `unittest2pytest -`__, which uses lib2to3 -and introspection for the transformation. + Due to architectural differences between the two frameworks, setup and + teardown for ``unittest``-based tests is performed during the ``call`` phase + of testing instead of in ``pytest``'s standard ``setup`` and ``teardown`` + stages. This can be important to understand in some situations, particularly + when reasoning about errors. For example, if a ``unittest``-based suite + exhibits errors during setup, ``pytest`` will report no errors during its + ``setup`` phase and will instead raise the error during ``call``. diff --git a/doc/en/usage.rst b/doc/en/usage.rst index 4b92fd1e150d042..6091db8be38ffcf 100644 --- a/doc/en/usage.rst +++ b/doc/en/usage.rst @@ -16,63 +16,131 @@ You can invoke testing through the Python interpreter from the command line:: python -m pytest [...] -This is equivalent to invoking the command line script ``py.test [...]`` -directly. +This is almost equivalent to invoking the command line script ``pytest [...]`` +directly, except that calling via ``python`` will also add the current directory to ``sys.path``. + +Possible exit codes +-------------------------------------------------------------- + +Running ``pytest`` can result in six different exit codes: + +:Exit code 0: All tests were collected and passed successfully +:Exit code 1: Tests were collected and run but some of the tests failed +:Exit code 2: Test execution was interrupted by the user +:Exit code 3: Internal error happened while executing tests +:Exit code 4: pytest command line usage error +:Exit code 5: No tests were collected Getting help on version, option names, environment variables -------------------------------------------------------------- :: - py.test --version # shows where pytest was imported from - py.test --fixtures # show available builtin function arguments - py.test -h | --help # show help on command line and config file options + pytest --version # shows where pytest was imported from + pytest --fixtures # show available builtin function arguments + pytest -h | --help # show help on command line and config file options + +.. _maxfail: Stopping after the first (or N) failures --------------------------------------------------- To stop the testing process after the first (N) failures:: - py.test -x # stop after first failure - py.test --maxfail=2 # stop after two failures + pytest -x # stop after first failure + pytest --maxfail=2 # stop after two failures + +.. _select-tests: Specifying tests / selecting tests --------------------------------------------------- -Several test run options:: +Pytest supports several ways to run and select tests from the command-line. + +**Run tests in a module** + +:: + + pytest test_mod.py + +**Run tests in a directory** + +:: + + pytest testing/ + +**Run tests by keyword expressions** + +:: + + pytest -k "MyClass and not method" + +This will run tests which contain names that match the given *string expression*, which can +include Python operators that use filenames, class names and function names as variables. +The example above will run ``TestMyClass.test_something`` but not ``TestMyClass.test_method_simple``. + +.. _nodeids: - py.test test_mod.py # run tests in module - py.test somepath # run all tests below somepath - py.test -k stringexpr # only run tests with names that match the - # "string expression", e.g. "MyClass and not method" - # will select TestMyClass.test_something - # but not TestMyClass.test_method_simple - py.test test_mod.py::test_func # only run tests that match the "node ID", - # e.g "test_mod.py::test_func" will select - # only test_func in test_mod.py - py.test test_mod.py::TestClass::test_method # run a single method in - # a single class +**Run tests by node ids** -Import 'pkg' and use its filesystem location to find and run tests:: +Each collected test is assigned a unique ``nodeid`` which consist of the module filename followed +by specifiers like class names, function names and parameters from parametrization, separated by ``::`` characters. - py.test --pyargs pkg # run all tests found below directory of pypkg +To run a specific test within a module:: + + pytest test_mod.py::test_func + + +Another example specifying a test method in the command line:: + + pytest test_mod.py::TestClass::test_method + +**Run tests by marker expressions** + +:: + + pytest -m slow + +Will run all tests which are decorated with the ``@pytest.mark.slow`` decorator. + +For more information see :ref:`marks `. + +**Run tests from packages** + +:: + + pytest --pyargs pkg.testing + +This will import ``pkg.testing`` and use its filesystem location to find and run tests from. + Modifying Python traceback printing ---------------------------------------------- Examples for modifying traceback printing:: - py.test --showlocals # show local variables in tracebacks - py.test -l # show local variables (shortcut) + pytest --showlocals # show local variables in tracebacks + pytest -l # show local variables (shortcut) - py.test --tb=auto # (default) 'long' tracebacks for the first and last + pytest --tb=auto # (default) 'long' tracebacks for the first and last # entry, but 'short' style for the other entries - py.test --tb=long # exhaustive, informative traceback formatting - py.test --tb=short # shorter traceback format - py.test --tb=line # only one line per failure - py.test --tb=native # Python standard library formatting - py.test --tb=no # no traceback at all + pytest --tb=long # exhaustive, informative traceback formatting + pytest --tb=short # shorter traceback format + pytest --tb=line # only one line per failure + pytest --tb=native # Python standard library formatting + pytest --tb=no # no traceback at all + +The ``--full-trace`` causes very long traces to be printed on error (longer +than ``--tb=long``). It also ensures that a stack trace is printed on +**KeyboardInterrupt** (Ctrl+C). +This is very useful if the tests are taking too long and you interrupt them +with Ctrl+C to find out where the tests are *hanging*. By default no output +will be shown (because KeyboardInterrupt is caught by pytest). By using this +option you make sure a trace is shown. + + +.. _pdb-option: Dropping to PDB_ (Python Debugger) on failures ----------------------------------------------- @@ -82,14 +150,14 @@ Dropping to PDB_ (Python Debugger) on failures Python comes with a builtin Python debugger called PDB_. ``pytest`` allows one to drop into the PDB_ prompt via a command line option:: - py.test --pdb + pytest --pdb This will invoke the Python debugger on every failure. Often you might only want to do this for the first failing test to understand a certain failure situation:: - py.test -x --pdb # drop to PDB on first failure, then end test session - py.test --pdb --maxfail=3 # drop to PDB for first three failures + pytest -x --pdb # drop to PDB on first failure, then end test session + pytest --pdb --maxfail=3 # drop to PDB for first three failures Note that on any failure the exception information is stored on ``sys.last_value``, ``sys.last_type`` and ``sys.last_traceback``. In @@ -103,22 +171,15 @@ for example:: >>> sys.last_value AssertionError('assert result == "ok"',) -Setting a breakpoint / aka ``set_trace()`` ----------------------------------------------------- - -If you want to set a breakpoint and enter the ``pdb.set_trace()`` you -can use a helper:: +.. _breakpoints: - import pytest - def test_function(): - ... - pytest.set_trace() # invoke PDB debugger and tracing +Setting breakpoints +------------------- -.. versionadded: 2.0.0 +.. versionadded: 2.4.0 -Prior to pytest version 2.0.0 you could only enter PDB_ tracing if you disabled -capturing on the command line via ``py.test -s``. In later versions, pytest -automatically disables its output capture when you enter PDB_ tracing: +To set a breakpoint in your code use the native Python ``import pdb;pdb.set_trace()`` call +in your code and pytest automatically disables its output capture for that test: * Output capture in other tests is not affected. * Any prior test output that has already been captured and will be processed as @@ -128,13 +189,6 @@ automatically disables its output capture when you enter PDB_ tracing: for test output occurring after you exit the interactive PDB_ tracing session and continue with the regular test run. -.. versionadded: 2.4.0 - -Since pytest version 2.4.0 you can also use the native Python -``import pdb;pdb.set_trace()`` call to enter PDB_ tracing without having to use -the ``pytest.set_trace()`` wrapper or explicitly disable pytest's output -capturing via ``py.test -s``. - .. _durations: Profiling test execution duration @@ -144,19 +198,28 @@ Profiling test execution duration To get a list of the slowest 10 test durations:: - py.test --durations=10 + pytest --durations=10 Creating JUnitXML format files ---------------------------------------------------- -To create result files which can be read by Hudson_ or other Continuous +To create result files which can be read by Jenkins_ or other Continuous integration servers, use this invocation:: - py.test --junitxml=path + pytest --junitxml=path to create an XML file at ``path``. +.. versionadded:: 3.1 + +To set the name of the root test suite xml item, you can configure the ``junit_suite_name`` option in your config file: + +.. code-block:: ini + + [pytest] + junit_suite_name = my_suite + record_xml_property ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -184,7 +247,7 @@ This will add an extra property ``example_key="1"`` to the generated .. warning:: - This is an experimental feature, and its interface might be replaced + ``record_xml_property`` is an experimental feature, and its interface might be replaced by something more powerful and general in future versions. The functionality per-se will be kept, however. @@ -193,12 +256,70 @@ This will add an extra property ``example_key="1"`` to the generated Also please note that using this feature will break any schema verification. This might be a problem when used with some CI servers. +LogXML: add_global_property +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. versionadded:: 3.0 + +If you want to add a properties node in the testsuite level, which may contains properties that are relevant +to all testcases you can use ``LogXML.add_global_properties`` + +.. code-block:: python + + import pytest + + @pytest.fixture(scope="session") + def log_global_env_facts(f): + + if pytest.config.pluginmanager.hasplugin('junitxml'): + my_junit = getattr(pytest.config, '_xml', None) + + my_junit.add_global_property('ARCH', 'PPC') + my_junit.add_global_property('STORAGE_TYPE', 'CEPH') + + @pytest.mark.usefixtures(log_global_env_facts) + def start_and_prepare_env(): + pass + + class TestMe(object): + def test_foo(self): + assert True + +This will add a property node below the testsuite node to the generated xml: + +.. code-block:: xml + + + + + + + + + +.. warning:: + + This is an experimental feature, and its interface might be replaced + by something more powerful and general in future versions. The + functionality per-se will be kept. + Creating resultlog format files ---------------------------------------------------- +.. deprecated:: 3.0 + + This option is rarely used and is scheduled for removal in 4.0. + + An alternative for users which still need similar functionality is to use the + `pytest-tap `_ plugin which provides + a stream of test data. + + If you have any concerns, please don't hesitate to + `open an issue `_. + To create plain-text machine-readable result files you can issue:: - py.test --resultlog=path + pytest --resultlog=path and look at the content at the ``path`` location. Such files are used e.g. by the `PyPy-test`_ web page to show test results over several revisions. @@ -211,7 +332,7 @@ Sending test report to online pastebin service **Creating a URL for each test failure**:: - py.test --pastebin=failed + pytest --pastebin=failed This will submit test run information to a remote Paste service and provide a URL for each failure. You may select tests as usual or add @@ -219,7 +340,7 @@ for example ``-x`` if you only want to send one particular failure. **Creating a URL for a whole test session log**:: - py.test --pastebin=all + pytest --pastebin=all Currently only pasting to the http://bpaste.net service is implemented. @@ -230,9 +351,9 @@ To disable loading specific plugins at invocation time, use the ``-p`` option together with the prefix ``no:``. Example: to disable loading the plugin ``doctest``, which is responsible for -executing doctest tests from text files, invoke py.test like this:: +executing doctest tests from text files, invoke pytest like this:: - py.test -p no:doctest + pytest -p no:doctest .. _`pytest.main-usage`: @@ -245,25 +366,21 @@ You can invoke ``pytest`` from Python code directly:: pytest.main() -this acts as if you would call "py.test" from the command line. +this acts as if you would call "pytest" from the command line. It will not raise ``SystemExit`` but return the exitcode instead. You can pass in options and arguments:: pytest.main(['-x', 'mytestdir']) -or pass in a string:: - - pytest.main("-x mytestdir") - You can specify additional plugins to ``pytest.main``:: # content of myinvoke.py import pytest - class MyPlugin: + class MyPlugin(object): def pytest_sessionfinish(self): print("*** test run reporting finishing") - pytest.main("-qq", plugins=[MyPlugin()]) + pytest.main(["-qq"], plugins=[MyPlugin()]) Running it will show that ``MyPlugin`` was added and its hook was invoked:: diff --git a/doc/en/warnings.rst b/doc/en/warnings.rst new file mode 100644 index 000000000000000..f249d7e3b2f82ef --- /dev/null +++ b/doc/en/warnings.rst @@ -0,0 +1,299 @@ +.. _`warnings`: + +Warnings Capture +================ + +.. versionadded:: 3.1 + +Starting from version ``3.1``, pytest now automatically catches warnings during test execution +and displays them at the end of the session:: + + # content of test_show_warnings.py + import warnings + + def api_v1(): + warnings.warn(UserWarning("api v1, should use functions from v2")) + return 1 + + def test_one(): + assert api_v1() == 1 + +Running pytest now produces this output:: + + $ pytest test_show_warnings.py + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 1 item + + test_show_warnings.py . [100%] + + ============================= warnings summary ============================= + test_show_warnings.py::test_one + $REGENDOC_TMPDIR/test_show_warnings.py:4: UserWarning: api v1, should use functions from v2 + warnings.warn(UserWarning("api v1, should use functions from v2")) + + -- Docs: http://doc.pytest.org/en/latest/warnings.html + =================== 1 passed, 1 warnings in 0.12 seconds =================== + +Pytest by default catches all warnings except for ``DeprecationWarning`` and ``PendingDeprecationWarning``. + +The ``-W`` flag can be passed to control which warnings will be displayed or even turn +them into errors:: + + $ pytest -q test_show_warnings.py -W error::UserWarning + F [100%] + ================================= FAILURES ================================= + _________________________________ test_one _________________________________ + + def test_one(): + > assert api_v1() == 1 + + test_show_warnings.py:8: + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + + def api_v1(): + > warnings.warn(UserWarning("api v1, should use functions from v2")) + E UserWarning: api v1, should use functions from v2 + + test_show_warnings.py:4: UserWarning + 1 failed in 0.12 seconds + +The same option can be set in the ``pytest.ini`` file using the ``filterwarnings`` ini option. +For example, the configuration below will ignore all user warnings, but will transform +all other warnings into errors. + +.. code-block:: ini + + [pytest] + filterwarnings = + error + ignore::UserWarning + + +When a warning matches more than one option in the list, the action for the last matching option +is performed. + +Both ``-W`` command-line option and ``filterwarnings`` ini option are based on Python's own +`-W option`_ and `warnings.simplefilter`_, so please refer to those sections in the Python +documentation for other examples and advanced usage. + +``@pytest.mark.filterwarnings`` +------------------------------- + +.. versionadded:: 3.2 + +You can use the ``@pytest.mark.filterwarnings`` to add warning filters to specific test items, +allowing you to have finer control of which warnings should be captured at test, class or +even module level: + +.. code-block:: python + + import warnings + + def api_v1(): + warnings.warn(UserWarning("api v1, should use functions from v2")) + return 1 + + @pytest.mark.filterwarnings('ignore:api v1') + def test_one(): + assert api_v1() == 1 + + +Filters applied using a mark take precedence over filters passed on the command line or configured +by the ``filterwarnings`` ini option. + +You may apply a filter to all tests of a class by using the ``filterwarnings`` mark as a class +decorator or to all tests in a module by setting the ``pytestmark`` variable: + +.. code-block:: python + + # turns all warnings into errors for this module + pytestmark = pytest.mark.filterwarnings('error') + + +.. note:: + + ``DeprecationWarning`` and ``PendingDeprecationWarning`` are hidden by the standard library + by default so you have to explicitly configure them to be displayed in your ``pytest.ini``: + + .. code-block:: ini + + [pytest] + filterwarnings = + once::DeprecationWarning + once::PendingDeprecationWarning + + +*Credits go to Florian Schulze for the reference implementation in the* `pytest-warnings`_ +*plugin.* + +.. _`-W option`: https://docs.python.org/3/using/cmdline.html?highlight=#cmdoption-W +.. _warnings.simplefilter: https://docs.python.org/3/library/warnings.html#warnings.simplefilter +.. _`pytest-warnings`: https://github.com/fschulze/pytest-warnings + + +Disabling warning capture +------------------------- + +This feature is enabled by default but can be disabled entirely in your ``pytest.ini`` file with: + + .. code-block:: ini + + [pytest] + addopts = -p no:warnings + +Or passing ``-p no:warnings`` in the command-line. + +.. _`asserting warnings`: + +.. _assertwarnings: + +.. _`asserting warnings with the warns function`: + +.. _warns: + +Asserting warnings with the warns function +----------------------------------------------- + +.. versionadded:: 2.8 + +You can check that code raises a particular warning using ``pytest.warns``, +which works in a similar manner to :ref:`raises `:: + + import warnings + import pytest + + def test_warning(): + with pytest.warns(UserWarning): + warnings.warn("my warning", UserWarning) + +The test will fail if the warning in question is not raised. The keyword +argument ``match`` to assert that the exception matches a text or regex:: + + >>> with warns(UserWarning, match='must be 0 or None'): + ... warnings.warn("value must be 0 or None", UserWarning) + + >>> with warns(UserWarning, match=r'must be \d+$'): + ... warnings.warn("value must be 42", UserWarning) + + >>> with warns(UserWarning, match=r'must be \d+$'): + ... warnings.warn("this is not here", UserWarning) + Traceback (most recent call last): + ... + Failed: DID NOT WARN. No warnings of type ...UserWarning... was emitted... + +You can also call ``pytest.warns`` on a function or code string:: + + pytest.warns(expected_warning, func, *args, **kwargs) + pytest.warns(expected_warning, "func(*args, **kwargs)") + +The function also returns a list of all raised warnings (as +``warnings.WarningMessage`` objects), which you can query for +additional information:: + + with pytest.warns(RuntimeWarning) as record: + warnings.warn("another warning", RuntimeWarning) + + # check that only one warning was raised + assert len(record) == 1 + # check that the message matches + assert record[0].message.args[0] == "another warning" + +Alternatively, you can examine raised warnings in detail using the +:ref:`recwarn ` fixture (see below). + +.. note:: + ``DeprecationWarning`` and ``PendingDeprecationWarning`` are treated + differently; see :ref:`ensuring_function_triggers`. + +.. _`recording warnings`: + +.. _recwarn: + +Recording warnings +------------------------ + +You can record raised warnings either using ``pytest.warns`` or with +the ``recwarn`` fixture. + +To record with ``pytest.warns`` without asserting anything about the warnings, +pass ``None`` as the expected warning type:: + + with pytest.warns(None) as record: + warnings.warn("user", UserWarning) + warnings.warn("runtime", RuntimeWarning) + + assert len(record) == 2 + assert str(record[0].message) == "user" + assert str(record[1].message) == "runtime" + +The ``recwarn`` fixture will record warnings for the whole function:: + + import warnings + + def test_hello(recwarn): + warnings.warn("hello", UserWarning) + assert len(recwarn) == 1 + w = recwarn.pop(UserWarning) + assert issubclass(w.category, UserWarning) + assert str(w.message) == "hello" + assert w.filename + assert w.lineno + +Both ``recwarn`` and ``pytest.warns`` return the same interface for recorded +warnings: a WarningsRecorder instance. To view the recorded warnings, you can +iterate over this instance, call ``len`` on it to get the number of recorded +warnings, or index into it to get a particular recorded warning. It also +provides these methods: + +.. autoclass:: _pytest.recwarn.WarningsRecorder() + :members: + +Each recorded warning has the attributes ``message``, ``category``, +``filename``, ``lineno``, ``file``, and ``line``. The ``category`` is the +class of the warning. The ``message`` is the warning itself; calling +``str(message)`` will return the actual message of the warning. + +.. note:: + :class:`RecordedWarning` was changed from a plain class to a namedtuple in pytest 3.1 + +.. note:: + ``DeprecationWarning`` and ``PendingDeprecationWarning`` are treated + differently; see :ref:`ensuring_function_triggers`. + +.. _`ensuring a function triggers a deprecation warning`: + +.. _ensuring_function_triggers: + +Ensuring a function triggers a deprecation warning +------------------------------------------------------- + +You can also call a global helper for checking +that a certain function call triggers a ``DeprecationWarning`` or +``PendingDeprecationWarning``:: + + import pytest + + def test_global(): + pytest.deprecated_call(myfunction, 17) + +By default, ``DeprecationWarning`` and ``PendingDeprecationWarning`` will not be +caught when using ``pytest.warns`` or ``recwarn`` because default Python warnings filters hide +them. If you wish to record them in your own code, use the +command ``warnings.simplefilter('always')``:: + + import warnings + import pytest + + def test_deprecation(recwarn): + warnings.simplefilter('always') + warnings.warn("deprecated", DeprecationWarning) + assert len(recwarn) == 1 + assert recwarn.pop(DeprecationWarning) + +You can also use it as a contextmanager:: + + def test_global(): + with pytest.deprecated_call(): + myobject.deprecated_method() diff --git a/doc/en/writing_plugins.rst b/doc/en/writing_plugins.rst index cc346aaa85c768d..eb52558300002f9 100644 --- a/doc/en/writing_plugins.rst +++ b/doc/en/writing_plugins.rst @@ -23,7 +23,7 @@ reporting by calling `well specified hooks`_ of the following plugins: In principle, each hook call is a ``1:N`` Python function call where ``N`` is the number of registered implementation functions for a given specification. -All specifications and implementations following the ``pytest_`` prefix +All specifications and implementations follow the ``pytest_`` prefix naming convention, making them easy to distinguish and find. .. _`pluginorder`: @@ -49,7 +49,7 @@ Plugin discovery order at tool startup Note that pytest does not find ``conftest.py`` files in deeper nested sub directories at tool startup. It is usually a good idea to keep - your conftest.py file in the top level test or project root directory. + your ``conftest.py`` file in the top level test or project root directory. * by recursively loading all plugins specified by the ``pytest_plugins`` variable in ``conftest.py`` files @@ -57,9 +57,7 @@ Plugin discovery order at tool startup .. _`pytest/plugin`: http://bitbucket.org/pytest-dev/pytest/src/tip/pytest/plugin/ .. _`conftest.py plugins`: -.. _`conftest.py`: .. _`localplugin`: -.. _`conftest`: .. _`local conftest plugins`: conftest.py: local per-directory plugins @@ -87,17 +85,19 @@ sub directory but not for other directories:: Here is how you might run it:: - py.test test_flat.py # will not show "setting up" - py.test a/test_sub.py # will show "setting up" + pytest test_flat.py # will not show "setting up" + pytest a/test_sub.py # will show "setting up" -.. Note:: +.. note:: If you have ``conftest.py`` files which do not reside in a python package directory (i.e. one containing an ``__init__.py``) then "import conftest" can be ambiguous because there might be other - ``conftest.py`` files as well on your PYTHONPATH or ``sys.path``. + ``conftest.py`` files as well on your ``PYTHONPATH`` or ``sys.path``. It is thus good practice for projects to either put ``conftest.py`` under a package scope or to never import anything from a - conftest.py file. + ``conftest.py`` file. + + See also: :ref:`pythonpath`. Writing your own plugin @@ -122,8 +122,8 @@ to extend and add functionality. for authoring plugins. The template provides an excellent starting point with a working plugin, - tests running with tox, comprehensive README and - entry-pointy already pre-configured. + tests running with tox, a comprehensive README file as well as a + pre-configured entry-point. Also consider :ref:`contributing your plugin to pytest-dev` once it has some happy users other than yourself. @@ -158,28 +158,115 @@ it in your setuptools-invocation: 'name_of_plugin = myproject.pluginmodule', ] }, + + # custom PyPI classifier for pytest plugins + classifiers=[ + "Framework :: Pytest", + ], ) If a package is installed this way, ``pytest`` will load ``myproject.pluginmodule`` as a plugin which can define `well specified hooks`_. +.. note:: + + Make sure to include ``Framework :: Pytest`` in your list of + `PyPI classifiers `_ + to make it easy for users to find your plugin. + + +Assertion Rewriting +------------------- + +One of the main features of ``pytest`` is the use of plain assert +statements and the detailed introspection of expressions upon +assertion failures. This is provided by "assertion rewriting" which +modifies the parsed AST before it gets compiled to bytecode. This is +done via a :pep:`302` import hook which gets installed early on when +``pytest`` starts up and will perform this rewriting when modules get +imported. However since we do not want to test different bytecode +then you will run in production this hook only rewrites test modules +themselves as well as any modules which are part of plugins. Any +other imported module will not be rewritten and normal assertion +behaviour will happen. + +If you have assertion helpers in other modules where you would need +assertion rewriting to be enabled you need to ask ``pytest`` +explicitly to rewrite this module before it gets imported. + +.. autofunction:: pytest.register_assert_rewrite + +This is especially important when you write a pytest plugin which is +created using a package. The import hook only treats ``conftest.py`` +files and any modules which are listed in the ``pytest11`` entrypoint +as plugins. As an example consider the following package:: + + pytest_foo/__init__.py + pytest_foo/plugin.py + pytest_foo/helper.py + +With the following typical ``setup.py`` extract: + +.. code-block:: python + + setup( + ... + entry_points={'pytest11': ['foo = pytest_foo.plugin']}, + ... + ) + +In this case only ``pytest_foo/plugin.py`` will be rewritten. If the +helper module also contains assert statements which need to be +rewritten it needs to be marked as such, before it gets imported. +This is easiest by marking it for rewriting inside the +``__init__.py`` module, which will always be imported first when a +module inside a package is imported. This way ``plugin.py`` can still +import ``helper.py`` normally. The contents of +``pytest_foo/__init__.py`` will then need to look like this: + +.. code-block:: python + + import pytest + + pytest.register_assert_rewrite('pytest_foo.helper') Requiring/Loading plugins in a test module or conftest file ----------------------------------------------------------- -You can require plugins in a test module or a conftest file like this:: +You can require plugins in a test module or a ``conftest.py`` file like this: + +.. code-block:: python - pytest_plugins = "name1", "name2", + pytest_plugins = ["name1", "name2"] When the test module or conftest plugin is loaded the specified plugins -will be loaded as well. You can also use dotted path like this:: +will be loaded as well. Any module can be blessed as a plugin, including internal +application modules: + +.. code-block:: python pytest_plugins = "myapp.testsupport.myplugin" -which will import the specified module as a ``pytest`` plugin. +``pytest_plugins`` variables are processed recursively, so note that in the example above +if ``myapp.testsupport.myplugin`` also declares ``pytest_plugins``, the contents +of the variable will also be loaded as plugins, and so on. + +This mechanism makes it easy to share fixtures within applications or even +external applications without the need to create external plugins using +the ``setuptools``'s entry point technique. + +Plugins imported by ``pytest_plugins`` will also automatically be marked +for assertion rewriting (see :func:`pytest.register_assert_rewrite`). +However for this to have any effect the module must not be +imported already; if it was already imported at the time the +``pytest_plugins`` statement is processed, a warning will result and +assertions inside the plugin will not be rewritten. To fix this you +can either call :func:`pytest.register_assert_rewrite` yourself before +the module is imported, or you can arrange the code to delay the +importing until after the plugin is registered. Accessing another plugin by name @@ -194,39 +281,106 @@ the plugin manager like this: plugin = config.pluginmanager.getplugin("name_of_plugin") If you want to look at the names of existing plugins, use -the ``--traceconfig`` option. +the ``--trace-config`` option. Testing plugins --------------- -pytest comes with some facilities that you can enable for testing your -plugin. Given that you have an installed plugin you can enable the -:py:class:`testdir <_pytest.pytester.Testdir>` fixture via specifying a -command line option to include the pytester plugin (``-p pytester``) or -by putting ``pytest_plugins = "pytester"`` into your test or -``conftest.py`` file. You then will have a ``testdir`` fixture which you -can use like this:: +pytest comes with a plugin named ``pytester`` that helps you write tests for +your plugin code. The plugin is disabled by default, so you will have to enable +it before you can use it. - # content of test_myplugin.py +You can do so by adding the following line to a ``conftest.py`` file in your +testing directory: - pytest_plugins = "pytester" # to get testdir fixture +.. code-block:: python - def test_myplugin(testdir): - testdir.makepyfile(""" - def test_example(): - pass + # content of conftest.py + + pytest_plugins = ["pytester"] + +Alternatively you can invoke pytest with the ``-p pytester`` command line +option. + +This will allow you to use the :py:class:`testdir <_pytest.pytester.Testdir>` +fixture for testing your plugin code. + +Let's demonstrate what you can do with the plugin with an example. Imagine we +developed a plugin that provides a fixture ``hello`` which yields a function +and we can invoke this function with one optional parameter. It will return a +string value of ``Hello World!`` if we do not supply a value or ``Hello +{value}!`` if we do supply a string value. + +.. code-block:: python + + # -*- coding: utf-8 -*- + + import pytest + + def pytest_addoption(parser): + group = parser.getgroup('helloworld') + group.addoption( + '--name', + action='store', + dest='name', + default='World', + help='Default "name" for hello().' + ) + + @pytest.fixture + def hello(request): + name = request.config.getoption('name') + + def _hello(name=None): + if not name: + name = request.config.getoption('name') + return "Hello {name}!".format(name=name) + + return _hello + + +Now the ``testdir`` fixture provides a convenient API for creating temporary +``conftest.py`` files and test files. It also allows us to run the tests and +return a result object, with which we can assert the tests' outcomes. + +.. code-block:: python + + def test_hello(testdir): + """Make sure that our plugin works.""" + + # create a temporary conftest.py file + testdir.makeconftest(""" + import pytest + + @pytest.fixture(params=[ + "Brianna", + "Andreas", + "Floris", + ]) + def name(request): + return request.param """) - result = testdir.runpytest("--verbose") - result.fnmatch_lines(""" - test_example* + + # create a temporary pytest test file + testdir.makepyfile(""" + def test_hello_default(hello): + assert hello() == "Hello World!" + + def test_hello_name(hello, name): + assert hello(name) == "Hello {0}!".format(name) """) -Note that by default ``testdir.runpytest()`` will perform a pytest -in-process. You can pass the command line option ``--runpytest=subprocess`` -to have it happen in a subprocess. + # run all tests with pytest + result = testdir.runpytest() + + # check that all 4 tests passed + result.assert_outcomes(passed=4) + + +For more information about the result object that ``runpytest()`` returns, and +the methods that it provides please check out the :py:class:`RunResult +<_pytest.pytester.RunResult>` documentation. -Also see the :py:class:`RunResult <_pytest.pytester.RunResult>` for more -methods of the result object that you get from a call to ``runpytest``. .. _`writinghooks`: @@ -270,6 +424,8 @@ allowed to raise exceptions. Doing so will break the pytest run. +.. _firstresult: + firstresult: stop at first non-None result ------------------------------------------- @@ -287,7 +443,7 @@ hookwrapper: executing around other hooks .. currentmodule:: _pytest.core -.. versionadded:: 2.7 (experimental) +.. versionadded:: 2.7 pytest plugins can implement hook wrappers which wrap the execution of other hook implementations. A hook wrapper is a generator function @@ -296,7 +452,7 @@ hook wrappers and passes the same arguments as to the regular hooks. At the yield point of the hook wrapper pytest will execute the next hook implementations and return their result to the yield point in the form of -a :py:class:`CallOutcome` instance which encapsulates a result or +a :py:class:`Result ` instance which encapsulates a result or exception info. The yield point itself will thus typically not raise exceptions (unless there are bugs). @@ -361,7 +517,7 @@ Here is the order of execution: Plugin1). 4. Plugin3's pytest_collection_modifyitems then executing the code after the yield - point. The yield receives a :py:class:`CallOutcome` instance which encapsulates + point. The yield receives a :py:class:`Result ` instance which encapsulates the result from calling the non-wrappers. Wrappers shall not modify the result. It's possible to use ``tryfirst`` and ``trylast`` also in conjunction with @@ -384,7 +540,7 @@ Hooks are usually declared as do-nothing functions that contain only documentation describing when the hook will be called and what return values are expected. -For an example, see `newhooks.py`_ from :ref:`xdist`. +For an example, see `newhooks.py`_ from `xdist `_. .. _`newhooks.py`: https://github.com/pytest-dev/pytest-xdist/blob/974bd566c599dc6a9ea291838c6f226197208b46/xdist/newhooks.py @@ -430,7 +586,6 @@ Initialization, command line and configuration hooks .. autofunction:: pytest_load_initial_conftests .. autofunction:: pytest_cmdline_preparse .. autofunction:: pytest_cmdline_parse -.. autofunction:: pytest_namespace .. autofunction:: pytest_addoption .. autofunction:: pytest_cmdline_main .. autofunction:: pytest_configure @@ -439,7 +594,7 @@ Initialization, command line and configuration hooks Generic "runtest" hooks ----------------------- -All runtest related hooks receive a :py:class:`pytest.Item` object. +All runtest related hooks receive a :py:class:`pytest.Item <_pytest.main.Item>` object. .. autofunction:: pytest_runtest_protocol .. autofunction:: pytest_runtest_setup @@ -470,6 +625,7 @@ you can use the following hook: .. autofunction:: pytest_pycollect_makeitem .. autofunction:: pytest_generate_tests +.. autofunction:: pytest_make_parametrize_id After collection is complete, you can modify the order of items, delete or otherwise amend the test items: @@ -486,8 +642,11 @@ Session related reporting hooks: .. autofunction:: pytest_collectreport .. autofunction:: pytest_deselected .. autofunction:: pytest_report_header +.. autofunction:: pytest_report_collectionfinish .. autofunction:: pytest_report_teststatus .. autofunction:: pytest_terminal_summary +.. autofunction:: pytest_fixture_setup +.. autofunction:: pytest_fixture_post_finalizer And here is the central hook for reporting about test execution: @@ -544,13 +703,18 @@ Reference of objects involved in hooks :members: :show-inheritance: +.. autoclass:: _pytest.fixtures.FixtureDef() + :members: + :show-inheritance: + .. autoclass:: _pytest.runner.CallInfo() :members: .. autoclass:: _pytest.runner.TestReport() :members: + :inherited-members: -.. autoclass:: _pytest.vendored_packages.pluggy._CallOutcome() +.. autoclass:: pluggy._Result :members: .. autofunction:: _pytest.config.get_plugin_manager() diff --git a/doc/en/xdist.rst b/doc/en/xdist.rst deleted file mode 100644 index ee1bd6032833c9d..000000000000000 --- a/doc/en/xdist.rst +++ /dev/null @@ -1,197 +0,0 @@ - -.. _`xdist`: - -xdist: pytest distributed testing plugin -=============================================================== - -The `pytest-xdist`_ plugin extends ``pytest`` with some unique -test execution modes: - -* Looponfail: run your tests repeatedly in a subprocess. After each - run, ``pytest`` waits until a file in your project changes and then - re-runs the previously failing tests. This is repeated until all - tests pass. At this point a full run is again performed. - -* multiprocess Load-balancing: if you have multiple CPUs or hosts you can use - them for a combined test run. This allows to speed up - development or to use special resources of remote machines. - -* Multi-Platform coverage: you can specify different Python interpreters - or different platforms and run tests in parallel on all of them. - -Before running tests remotely, ``pytest`` efficiently "rsyncs" your -program source code to the remote place. All test results -are reported back and displayed to your local terminal. -You may specify different Python versions and interpreters. - - -Installation of xdist plugin ------------------------------- - -Install the plugin with:: - - easy_install pytest-xdist - - # or - - pip install pytest-xdist - -or use the package in develop/in-place mode with -a checkout of the `pytest-xdist repository`_ :: - - python setup.py develop - - -Usage examples ---------------------- - -.. _`xdistcpu`: - -Speed up test runs by sending tests to multiple CPUs -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -To send tests to multiple CPUs, type:: - - py.test -n NUM - -Especially for longer running tests or tests requiring -a lot of I/O this can lead to considerable speed ups. - - -Running tests in a Python subprocess -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -To instantiate a Python-2.7 subprocess and send tests to it, you may type:: - - py.test -d --tx popen//python=python2.7 - -This will start a subprocess which is run with the "python2.7" -Python interpreter, found in your system binary lookup path. - -If you prefix the --tx option value like this:: - - py.test -d --tx 3*popen//python=python2.7 - -then three subprocesses would be created and the tests -will be distributed to three subprocesses and run simultanously. - -.. _looponfailing: - - -Running tests in looponfailing mode -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -For refactoring a project with a medium or large test suite -you can use the looponfailing mode. Simply add the ``--f`` option:: - - py.test -f - -and ``pytest`` will run your tests. Assuming you have failures it will then -wait for file changes and re-run the failing test set. File changes are detected by looking at ``looponfailingroots`` root directories and all of their contents (recursively). If the default for this value does not work for you you -can change it in your project by setting a configuration option:: - - # content of a pytest.ini, setup.cfg or tox.ini file - [pytest] - looponfailroots = mypkg testdir - -This would lead to only looking for file changes in the respective directories, specified relatively to the ini-file's directory. - -Sending tests to remote SSH accounts -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -Suppose you have a package ``mypkg`` which contains some -tests that you can successfully run locally. And you also -have a ssh-reachable machine ``myhost``. Then -you can ad-hoc distribute your tests by typing:: - - py.test -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg - -This will synchronize your ``mypkg`` package directory -with a remote ssh account and then collect and run your -tests at the remote side. - -You can specify multiple ``--rsyncdir`` directories -to be sent to the remote side. - -.. XXX CHECK - - **NOTE:** For ``pytest`` to collect and send tests correctly - you not only need to make sure all code and tests - directories are rsynced, but that any test (sub) directory - also has an ``__init__.py`` file because internally - ``pytest`` references tests as a fully qualified python - module path. **You will otherwise get strange errors** - during setup of the remote side. - -Sending tests to remote Socket Servers -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -Download the single-module `socketserver.py`_ Python program -and run it like this:: - - python socketserver.py - -It will tell you that it starts listening on the default -port. You can now on your home machine specify this -new socket host with something like this:: - - py.test -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg - - -.. _`atonce`: - -Running tests on many platforms at once -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -The basic command to run tests on multiple platforms is:: - - py.test --dist=each --tx=spec1 --tx=spec2 - -If you specify a windows host, an OSX host and a Linux -environment this command will send each tests to all -platforms - and report back failures from all platforms -at once. The specifications strings use the `xspec syntax`_. - -.. _`xspec syntax`: http://codespeak.net/execnet/basics.html#xspec - -.. _`socketserver.py`: http://bitbucket.org/hpk42/execnet/raw/2af991418160/execnet/script/socketserver.py - -.. _`execnet`: http://codespeak.net/execnet - -Specifying test exec environments in an ini file -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -pytest (since version 2.0) supports ini-style configuration. -For example, you could make running with three subprocesses your default:: - - [pytest] - addopts = -n3 - -You can also add default environments like this:: - - [pytest] - addopts = --tx ssh=myhost//python=python2.7 --tx ssh=myhost//python=python2.6 - -and then just type:: - - py.test --dist=each - -to run tests in each of the environments. - -Specifying "rsync" dirs in an ini-file -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -In a ``tox.ini`` or ``setup.cfg`` file in your root project directory -you may specify directories to include or to exclude in synchronisation:: - - [pytest] - rsyncdirs = . mypkg helperpkg - rsyncignore = .hg - -These directory specifications are relative to the directory -where the configuration file was found. - -.. _`pytest-xdist`: http://pypi.python.org/pypi/pytest-xdist -.. _`pytest-xdist repository`: http://bitbucket.org/pytest-dev/pytest-xdist -.. _`pytest`: http://pytest.org - diff --git a/doc/en/xunit_setup.rst b/doc/en/xunit_setup.rst index 7a80f1299660a4e..148fb1209ea393a 100644 --- a/doc/en/xunit_setup.rst +++ b/doc/en/xunit_setup.rst @@ -7,21 +7,20 @@ classic xunit-style setup This section describes a classic and popular way how you can implement fixtures (setup and teardown test state) on a per-module/class/function basis. -pytest started supporting these methods around 2005 and subsequently -nose and the standard library introduced them (under slightly different -names). While these setup/teardown methods are and will remain fully -supported you may also use pytest's more powerful :ref:`fixture mechanism -` which leverages the concept of dependency injection, allowing -for a more modular and more scalable approach for managing test state, -especially for larger projects and for functional testing. You can -mix both fixture mechanisms in the same file but unittest-based -test methods cannot receive fixture arguments. + .. note:: - As of pytest-2.4, teardownX functions are not called if - setupX existed and failed/was skipped. This harmonizes - behaviour across all major python testing tools. + While these setup/teardown methods are simple and familiar to those + coming from a ``unittest`` or nose ``background``, you may also consider + using pytest's more powerful :ref:`fixture mechanism + ` which leverages the concept of dependency injection, allowing + for a more modular and more scalable approach for managing test state, + especially for larger projects and for functional testing. You can + mix both fixture mechanisms in the same file but + test methods of ``unittest.TestCase`` subclasses + cannot receive fixture arguments. + Module level setup/teardown -------------------------------------- @@ -38,6 +37,8 @@ which will usually be called once for all the functions:: method. """ +As of pytest-3.0, the ``module`` parameter is optional. + Class level setup/teardown ---------------------------------- @@ -71,6 +72,8 @@ Similarly, the following methods are called around each method invocation:: call. """ +As of pytest-3.0, the ``method`` parameter is optional. + If you would rather define test functions directly at module level you can also use the following functions to implement fixtures:: @@ -84,7 +87,13 @@ you can also use the following functions to implement fixtures:: call. """ -Note that it is possible for setup/teardown pairs to be invoked multiple times -per testing process. +As of pytest-3.0, the ``function`` parameter is optional. + +Remarks: + +* It is possible for setup/teardown pairs to be invoked multiple times + per testing process. +* teardown functions are not called if the corresponding setup function existed + and failed/was skipped. .. _`unittest.py module`: http://docs.python.org/library/unittest.html diff --git a/doc/en/yieldfixture.rst b/doc/en/yieldfixture.rst index ee88a27df597d1c..6fd1edac2f5e004 100644 --- a/doc/en/yieldfixture.rst +++ b/doc/en/yieldfixture.rst @@ -1,100 +1,18 @@ +:orphan: + .. _yieldfixture: -Fixture functions using "yield" / context manager integration +"yield_fixture" functions --------------------------------------------------------------- -.. versionadded:: 2.4 - -.. regendoc:wipe - -pytest-2.4 allows fixture functions to seamlessly use a ``yield`` instead -of a ``return`` statement to provide a fixture value while otherwise -fully supporting all other fixture features. - -Let's look at a simple standalone-example using the ``yield`` syntax:: - - # content of test_yield.py - - import pytest - - @pytest.yield_fixture - def passwd(): - print ("\nsetup before yield") - f = open("/etc/passwd") - yield f.readlines() - print ("teardown after yield") - f.close() - - def test_has_lines(passwd): - print ("test called") - assert passwd - -In contrast to :ref:`finalization through registering callbacks -`, our fixture function used a ``yield`` -statement to provide the lines of the ``/etc/passwd`` file. -The code after the ``yield`` statement serves as the teardown code, -avoiding the indirection of registering a teardown callback function. - -Let's run it with output capturing disabled:: - - $ py.test -q -s test_yield.py - - setup before yield - test called - .teardown after yield - - 1 passed in 0.12 seconds - -We can also seamlessly use the new syntax with ``with`` statements. -Let's simplify the above ``passwd`` fixture:: +.. deprecated:: 3.0 - # content of test_yield2.py - - import pytest - - @pytest.yield_fixture - def passwd(): - with open("/etc/passwd") as f: - yield f.readlines() - - def test_has_lines(passwd): - assert len(passwd) >= 1 - -The file ``f`` will be closed after the test finished execution -because the Python ``file`` object supports finalization when -the ``with`` statement ends. - -Note that the yield fixture form supports all other fixture -features such as ``scope``, ``params``, etc., thus changing existing -fixture functions to use ``yield`` is straightforward. - -.. note:: - - While the ``yield`` syntax is similar to what - :py:func:`contextlib.contextmanager` decorated functions - provide, with pytest fixture functions the part after the - "yield" will always be invoked, independently from the - exception status of the test function which uses the fixture. - This behaviour makes sense if you consider that many different - test functions might use a module or session scoped fixture. - - -Discussion and future considerations / feedback -++++++++++++++++++++++++++++++++++++++++++++++++++++ - -There are some topics that are worth mentioning: - -- usually ``yield`` is used for producing multiple values. - But fixture functions can only yield exactly one value. - Yielding a second fixture value will get you an error. - It's possible we can evolve pytest to allow for producing - multiple values as an alternative to current parametrization. - For now, you can just use the normal - :ref:`fixture parametrization ` - mechanisms together with ``yield``-style fixtures. +.. versionadded:: 2.4 -- lastly ``yield`` introduces more than one way to write - fixture functions, so what's the obvious way to a newcomer? +.. important:: + Since pytest-3.0, fixtures using the normal ``fixture`` decorator can use a ``yield`` + statement to provide fixture values and execute teardown code, exactly like ``yield_fixture`` + in previous versions. -If you want to feedback or participate in discussion of the above -topics, please join our :ref:`contact channels`, you are most welcome. + Marking functions as ``yield_fixture`` is still supported, but deprecated and should not + be used in new code. diff --git a/extra/get_issues.py b/extra/get_issues.py index 6437ba4c34e6069..2a8f8c316069444 100644 --- a/extra/get_issues.py +++ b/extra/get_issues.py @@ -2,30 +2,34 @@ import py import textwrap -issues_url = "http://bitbucket.org/api/1.0/repositories/pytest-dev/pytest/issues" +issues_url = "https://api.github.com/repos/pytest-dev/pytest/issues" import requests + def get_issues(): - chunksize = 50 - start = 0 issues = [] + url = issues_url while 1: - post_data = {"accountname": "pytest-dev", - "repo_slug": "pytest", - "start": start, - "limit": chunksize} - print ("getting from", start) - r = requests.get(issues_url, params=post_data) + get_data = {"state": "all"} + r = requests.get(url, params=get_data) data = r.json() - issues.extend(data["issues"]) - if start + chunksize >= data["count"]: - return issues - start += chunksize + if r.status_code == 403: + # API request limit exceeded + print(data['message']) + exit(1) + issues.extend(data) -kind2num = "bug enhancement task proposal".split() + # Look for next page + links = requests.utils.parse_header_links(r.headers['Link']) + another_page = False + for link in links: + if link['rel'] == 'next': + url = link['url'] + another_page = True + if not another_page: + return issues -status2num = "new open resolved duplicate invalid wontfix".split() def main(args): cachefile = py.path.local(args.cache) @@ -35,33 +39,38 @@ def main(args): else: issues = json.loads(cachefile.read()) - open_issues = [x for x in issues - if x["status"] in ("new", "open")] + open_issues = [x for x in issues if x["state"] == "open"] - def kind_and_id(x): - kind = x["metadata"]["kind"] - return kind2num.index(kind), len(issues)-int(x["local_id"]) - open_issues.sort(key=kind_and_id) + open_issues.sort(key=lambda x: x["number"]) report(open_issues) + +def _get_kind(issue): + labels = [l['name'] for l in issue['labels']] + for key in ('bug', 'enhancement', 'proposal'): + if key in labels: + return key + return 'issue' + + def report(issues): for issue in issues: - metadata = issue["metadata"] - priority = issue["priority"] title = issue["title"] - content = issue["content"] - kind = metadata["kind"] - status = issue["status"] - id = issue["local_id"] - link = "https://bitbucket.org/pytest-dev/pytest/issue/%s/" % id + body = issue["body"] + kind = _get_kind(issue) + status = issue["state"] + number = issue["number"] + link = "https://github.com/pytest-dev/pytest/issues/%s/" % number print("----") print(status, kind, link) print(title) #print() - #lines = content.split("\n") + #lines = body.split("\n") #print ("\n".join(lines[:3])) - #if len(lines) > 3 or len(content) > 240: + #if len(lines) > 3 or len(body) > 240: # print ("...") + print("\n\nFound %s open issues" % len(issues)) + if __name__ == "__main__": import argparse @@ -72,3 +81,4 @@ def report(issues): help="cache file") args = parser.parse_args() main(args) + diff --git a/plugin-test.sh b/plugin-test.sh deleted file mode 100644 index 9c61b50536e0be1..000000000000000 --- a/plugin-test.sh +++ /dev/null @@ -1,20 +0,0 @@ -#!/bin/bash - -# this assumes plugins are installed as sister directories - -set -e -cd ../pytest-pep8 -py.test -cd ../pytest-instafail -py.test -cd ../pytest-cache -py.test -cd ../pytest-xprocess -py.test -#cd ../pytest-cov -#py.test -cd ../pytest-capturelog -py.test -cd ../pytest-xdist -py.test - diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 000000000000000..88571e208b9a387 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,35 @@ +[tool.towncrier] +package = "pytest" +filename = "CHANGELOG.rst" +directory = "changelog/" +template = "changelog/_template.rst" + + [[tool.towncrier.type]] + directory = "removal" + name = "Deprecations and Removals" + showcontent = true + + [[tool.towncrier.type]] + directory = "feature" + name = "Features" + showcontent = true + + [[tool.towncrier.type]] + directory = "bugfix" + name = "Bug Fixes" + showcontent = true + + [[tool.towncrier.type]] + directory = "vendor" + name = "Vendored Libraries" + showcontent = true + + [[tool.towncrier.type]] + directory = "doc" + name = "Improved Documentation" + showcontent = true + + [[tool.towncrier.type]] + directory = "trivial" + name = "Trivial/Internal Changes" + showcontent = true diff --git a/pytest.py b/pytest.py index e376e417e8a3353..2b681b64b4c76b5 100644 --- a/pytest.py +++ b/pytest.py @@ -2,6 +2,32 @@ """ pytest: unit and functional testing with Python. """ + + +# else we are imported + +from _pytest.config import ( + main, UsageError, cmdline, + hookspec, hookimpl +) +from _pytest.fixtures import fixture, yield_fixture +from _pytest.assertion import register_assert_rewrite +from _pytest.freeze_support import freeze_includes +from _pytest import __version__ +from _pytest.debugging import pytestPDB as __pytestPDB +from _pytest.recwarn import warns, deprecated_call +from _pytest.outcomes import fail, skip, importorskip, exit, xfail +from _pytest.mark import MARK_GEN as mark, param +from _pytest.main import Item, Collector, File, Session +from _pytest.fixtures import fillfixtures as _fillfuncargs +from _pytest.python import ( + Module, Class, Instance, Function, Generator, +) + +from _pytest.python_api import approx, raises + +set_trace = __pytestPDB.set_trace + __all__ = [ 'main', 'UsageError', @@ -9,20 +35,43 @@ 'hookspec', 'hookimpl', '__version__', + 'register_assert_rewrite', + 'freeze_includes', + 'set_trace', + 'warns', + 'deprecated_call', + 'fixture', + 'yield_fixture', + 'fail', + 'skip', + 'xfail', + 'importorskip', + 'exit', + 'mark', + 'param', + 'approx', + '_fillfuncargs', + + 'Item', + 'File', + 'Collector', + 'Session', + 'Module', + 'Class', + 'Instance', + 'Function', + 'Generator', + 'raises', + + ] -if __name__ == '__main__': # if run as a script or by 'python -m pytest' +if __name__ == '__main__': + # if run as a script or by 'python -m pytest' # we trigger the below "else" condition by the following import import pytest raise SystemExit(pytest.main()) +else: -# else we are imported - -from _pytest.config import ( - main, UsageError, _preloadplugins, cmdline, - hookspec, hookimpl -) -from _pytest import __version__ - -_preloadplugins() # to populate pytest.* namespace so help(pytest) works - + from _pytest.compat import _setup_collect_fakemodule + _setup_collect_fakemodule() diff --git a/requirements-docs.txt b/requirements-docs.txt deleted file mode 100644 index be3a232e57b86b3..000000000000000 --- a/requirements-docs.txt +++ /dev/null @@ -1,3 +0,0 @@ -sphinx==1.2.3 -regendoc -pyyaml diff --git a/runtox.py b/runtox.py deleted file mode 100644 index 8c13c53e1c4739c..000000000000000 --- a/runtox.py +++ /dev/null @@ -1,8 +0,0 @@ -#!/usr/bin/env python - -if __name__ == "__main__": - import subprocess - import sys - subprocess.call([sys.executable, "-m", "tox", - "-i", "ALL=https://devpi.net/hpk/dev/", - "--develop"] + sys.argv[1:]) diff --git a/scripts/call-tox.bat b/scripts/call-tox.bat new file mode 100644 index 000000000000000..86fb25c1df1d101 --- /dev/null +++ b/scripts/call-tox.bat @@ -0,0 +1,8 @@ +REM skip "coveralls" run in PRs or forks +if "%TOXENV%" == "coveralls" ( + if not defined COVERALLS_REPO_TOKEN ( + echo skipping coveralls run because COVERALLS_REPO_TOKEN is not defined + exit /b 0 + ) +) +C:\Python36\python -m tox diff --git a/scripts/check-rst.py b/scripts/check-rst.py new file mode 100644 index 000000000000000..57f717501af47ee --- /dev/null +++ b/scripts/check-rst.py @@ -0,0 +1,11 @@ + +from __future__ import print_function + +import subprocess +import glob +import sys + +sys.exit(subprocess.call([ + 'rst-lint', '--encoding', 'utf-8', + 'CHANGELOG.rst', 'HOWTORELEASE.rst', 'README.rst', +] + glob.glob('changelog/[0-9]*.*'))) diff --git a/scripts/install-pypy.bat b/scripts/install-pypy.bat new file mode 100644 index 000000000000000..8012ea46aca45ff --- /dev/null +++ b/scripts/install-pypy.bat @@ -0,0 +1,6 @@ +REM install pypy using choco +REM redirect to a file because choco install python.pypy is too noisy. If the command fails, write output to console +choco install python.pypy > pypy-inst.log 2>&1 || (type pypy-inst.log & exit /b 1) +set PATH=C:\tools\pypy\pypy;%PATH% # so tox can find pypy +echo PyPy installed +pypy --version diff --git a/setup.cfg b/setup.cfg index 1ab4fd059b2544f..816539e2ec6fb8d 100644 --- a/setup.cfg +++ b/setup.cfg @@ -9,5 +9,12 @@ upload-dir = doc/en/build/html [bdist_wheel] universal = 1 +[check-manifest] +ignore = + _pytest/_version.py + +[metadata] +license_file = LICENSE + [devpi:upload] formats = sdist.tgz,bdist_wheel diff --git a/setup.py b/setup.py index 6660f21604d0129..3eb38efe6551f91 100644 --- a/setup.py +++ b/setup.py @@ -1,32 +1,27 @@ -import os, sys +import os +import sys import setuptools import pkg_resources from setuptools import setup, Command -classifiers = ['Development Status :: 6 - Mature', - 'Intended Audience :: Developers', - 'License :: OSI Approved :: MIT License', - 'Operating System :: POSIX', - 'Operating System :: Microsoft :: Windows', - 'Operating System :: MacOS :: MacOS X', - 'Topic :: Software Development :: Testing', - 'Topic :: Software Development :: Libraries', - 'Topic :: Utilities'] + [ - ('Programming Language :: Python :: %s' % x) for x in - '2 2.6 2.7 3 3.2 3.3 3.4 3.5'.split()] +classifiers = [ + 'Development Status :: 6 - Mature', + 'Intended Audience :: Developers', + 'License :: OSI Approved :: MIT License', + 'Operating System :: POSIX', + 'Operating System :: Microsoft :: Windows', + 'Operating System :: MacOS :: MacOS X', + 'Topic :: Software Development :: Testing', + 'Topic :: Software Development :: Libraries', + 'Topic :: Utilities', +] + [ + ('Programming Language :: Python :: %s' % x) + for x in '2 2.7 3 3.4 3.5 3.6'.split() +] with open('README.rst') as fd: long_description = fd.read() -def get_version(): - p = os.path.join(os.path.dirname( - os.path.abspath(__file__)), "_pytest", "__init__.py") - with open(p) as f: - for line in f.readlines(): - if "__version__" in line: - return line.strip().split("=")[-1].strip(" '") - raise ValueError("could not read version") - def has_environment_marker_support(): """ @@ -37,7 +32,7 @@ def has_environment_marker_support(): References: - * https://wheel.readthedocs.org/en/latest/index.html#defining-conditional-dependencies + * https://wheel.readthedocs.io/en/latest/index.html#defining-conditional-dependencies * https://www.python.org/dev/peps/pep-0426/#environment-markers """ try: @@ -48,67 +43,64 @@ def has_environment_marker_support(): def main(): - install_requires = ['py>=1.4.29'] # pluggy is vendored in _pytest.vendored_packages extras_require = {} + install_requires = [ + 'py>=1.5.0', + 'six>=1.10.0', + 'setuptools', + 'attrs>=17.2.0', + ] + # if _PYTEST_SETUP_SKIP_PLUGGY_DEP is set, skip installing pluggy; + # used by tox.ini to test with pluggy master + if '_PYTEST_SETUP_SKIP_PLUGGY_DEP' not in os.environ: + install_requires.append('pluggy>=0.5,<0.7') if has_environment_marker_support(): - extras_require[':python_version=="2.6" or python_version=="3.0" or python_version=="3.1"'] = ['argparse'] + extras_require[':python_version<"3.0"'] = ['funcsigs'] extras_require[':sys_platform=="win32"'] = ['colorama'] else: - if sys.version_info < (2, 7) or (3,) <= sys.version_info < (3, 2): - install_requires.append('argparse') if sys.platform == 'win32': install_requires.append('colorama') + if sys.version_info < (3, 0): + install_requires.append('funcsigs') setup( name='pytest', description='pytest: simple powerful testing with Python', long_description=long_description, - version=get_version(), + use_scm_version={ + 'write_to': '_pytest/_version.py', + }, url='http://pytest.org', license='MIT license', platforms=['unix', 'linux', 'osx', 'cygwin', 'win32'], - author='Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others', - author_email='holger at merlinux.eu', - entry_points=make_entry_points(), + author=( + 'Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, ' + 'Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others'), + entry_points={'console_scripts': [ + 'pytest=pytest:main', 'py.test=pytest:main']}, classifiers=classifiers, + keywords="test unittest", cmdclass={'test': PyTest}, # the following should be enabled for release + setup_requires=['setuptools-scm'], + python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*', install_requires=install_requires, extras_require=extras_require, - packages=['_pytest', '_pytest.assertion', '_pytest._code', '_pytest.vendored_packages'], + packages=['_pytest', '_pytest.assertion', '_pytest._code'], py_modules=['pytest'], zip_safe=False, ) -def cmdline_entrypoints(versioninfo, platform, basename): - target = 'pytest:main' - if platform.startswith('java'): - points = {'py.test-jython': target} - else: - if basename.startswith('pypy'): - points = {'py.test-%s' % basename: target} - else: # cpython - points = {'py.test-%s.%s' % versioninfo[:2] : target} - points['py.test'] = target - return points - - -def make_entry_points(): - basename = os.path.basename(sys.executable) - points = cmdline_entrypoints(sys.version_info, sys.platform, basename) - keys = list(points.keys()) - keys.sort() - l = ['%s = %s' % (x, points[x]) for x in keys] - return {'console_scripts': l} - - class PyTest(Command): user_options = [] + def initialize_options(self): pass + def finalize_options(self): pass + def run(self): import subprocess PPATH = [x for x in os.environ.get('PYTHONPATH', '').split(':') if x] diff --git a/tasks/__init__.py b/tasks/__init__.py new file mode 100644 index 000000000000000..8ea038f0af036f6 --- /dev/null +++ b/tasks/__init__.py @@ -0,0 +1,12 @@ +""" +Invoke tasks to help with pytest development and release process. +""" + +import invoke + +from . import generate + + +ns = invoke.Collection( + generate, +) diff --git a/tasks/generate.py b/tasks/generate.py new file mode 100644 index 000000000000000..fa8ee6557dfd7b6 --- /dev/null +++ b/tasks/generate.py @@ -0,0 +1,162 @@ +import os +from pathlib import Path +from subprocess import check_output, check_call + +import invoke + + +@invoke.task(help={ + 'version': 'version being released', +}) +def announce(ctx, version): + """Generates a new release announcement entry in the docs.""" + # Get our list of authors + stdout = check_output(["git", "describe", "--abbrev=0", '--tags']) + stdout = stdout.decode('utf-8') + last_version = stdout.strip() + + stdout = check_output(["git", "log", "{}..HEAD".format(last_version), "--format=%aN"]) + stdout = stdout.decode('utf-8') + + contributors = set(stdout.splitlines()) + + template_name = 'release.minor.rst' if version.endswith('.0') else 'release.patch.rst' + template_text = Path(__file__).parent.joinpath(template_name).read_text(encoding='UTF-8') + + contributors_text = '\n'.join('* {}'.format(name) for name in sorted(contributors)) + '\n' + text = template_text.format(version=version, contributors=contributors_text) + + target = Path(__file__).parent.joinpath('../doc/en/announce/release-{}.rst'.format(version)) + target.write_text(text, encoding='UTF-8') + print("[generate.announce] Generated {}".format(target.name)) + + # Update index with the new release entry + index_path = Path(__file__).parent.joinpath('../doc/en/announce/index.rst') + lines = index_path.read_text(encoding='UTF-8').splitlines() + indent = ' ' + for index, line in enumerate(lines): + if line.startswith('{}release-'.format(indent)): + new_line = indent + target.stem + if line != new_line: + lines.insert(index, new_line) + index_path.write_text('\n'.join(lines) + '\n', encoding='UTF-8') + print("[generate.announce] Updated {}".format(index_path.name)) + else: + print("[generate.announce] Skip {} (already contains release)".format(index_path.name)) + break + + check_call(['git', 'add', str(target)]) + + +@invoke.task() +def regen(ctx): + """Call regendoc tool to update examples and pytest output in the docs.""" + print("[generate.regen] Updating docs") + check_call(['tox', '-e', 'regen']) + + +@invoke.task() +def make_tag(ctx, version): + """Create a new (local) tag for the release, only if the repository is clean.""" + from git import Repo + + repo = Repo('.') + if repo.is_dirty(): + print('Current repository is dirty. Please commit any changes and try again.') + raise invoke.Exit(code=2) + + tag_names = [x.name for x in repo.tags] + if version in tag_names: + print("[generate.make_tag] Delete existing tag {}".format(version)) + repo.delete_tag(version) + + print("[generate.make_tag] Create tag {}".format(version)) + repo.create_tag(version) + + +@invoke.task() +def devpi_upload(ctx, version, user, password=None): + """Creates and uploads a package to devpi for testing.""" + if password: + print("[generate.devpi_upload] devpi login {}".format(user)) + check_call(['devpi', 'login', user, '--password', password]) + + check_call(['devpi', 'use', 'https://devpi.net/{}/dev'.format(user)]) + + env = os.environ.copy() + env['SETUPTOOLS_SCM_PRETEND_VERSION'] = version + check_call(['devpi', 'upload', '--formats', 'sdist,bdist_wheel'], env=env) + print("[generate.devpi_upload] package uploaded") + + +@invoke.task(help={ + 'version': 'version being released', + 'user': 'name of the user on devpi to stage the generated package', + 'password': 'user password on devpi to stage the generated package ' + '(if not given assumed logged in)', +}) +def pre_release(ctx, version, user, password=None): + """Generates new docs, release announcements and uploads a new release to devpi for testing.""" + announce(ctx, version) + regen(ctx) + changelog(ctx, version, write_out=True) + + msg = 'Preparing release version {}'.format(version) + check_call(['git', 'commit', '-a', '-m', msg]) + + make_tag(ctx, version) + + devpi_upload(ctx, version=version, user=user, password=password) + + print() + print('[generate.pre_release] Please push your branch and open a PR.') + + +@invoke.task(help={ + 'version': 'version being released', + 'user': 'name of the user on devpi to stage the generated package', + 'pypi_name': 'name of the pypi configuration section in your ~/.pypirc', +}) +def publish_release(ctx, version, user, pypi_name): + """Publishes a package previously created by the 'pre_release' command.""" + from git import Repo + repo = Repo('.') + tag_names = [x.name for x in repo.tags] + if version not in tag_names: + print('Could not find tag for version {}, exiting...'.format(version)) + raise invoke.Exit(code=2) + + check_call(['devpi', 'use', 'https://devpi.net/{}/dev'.format(user)]) + check_call(['devpi', 'push', 'pytest=={}'.format(version), 'pypi:{}'.format(pypi_name)]) + check_call(['git', 'push', 'git@github.com:pytest-dev/pytest.git', version]) + + emails = [ + 'pytest-dev@python.org', + 'python-announce-list@python.org' + ] + if version.endswith('.0'): + emails.append('testing-in-python@lists.idyll.org') + print('Version {} has been published to PyPI!'.format(version)) + print() + print('Please send an email announcement with the contents from:') + print() + print(' doc/en/announce/release-{}.rst'.format(version)) + print() + print('To the following mail lists:') + print() + print(' ', ','.join(emails)) + print() + print('And announce it on twitter adding the #pytest hash tag.') + + +@invoke.task(help={ + 'version': 'version being released', + 'write_out': 'write changes to the actial changelog' +}) +def changelog(ctx, version, write_out=False): + if write_out: + addopts = [] + else: + addopts = ['--draft'] + check_call(['towncrier', '--version', version] + addopts) + diff --git a/tasks/release.minor.rst b/tasks/release.minor.rst new file mode 100644 index 000000000000000..3c0b7d718a1ce38 --- /dev/null +++ b/tasks/release.minor.rst @@ -0,0 +1,27 @@ +pytest-{version} +======================================= + +The pytest team is proud to announce the {version} release! + +pytest is a mature Python testing tool with more than a 1600 tests +against itself, passing on many different interpreters and platforms. + +This release contains a number of bugs fixes and improvements, so users are encouraged +to take a look at the CHANGELOG: + + http://doc.pytest.org/en/latest/changelog.html + +For complete documentation, please visit: + + http://docs.pytest.org + +As usual, you can upgrade from pypi via: + + pip install -U pytest + +Thanks to all who contributed to this release, among them: + +{contributors} + +Happy testing, +The Pytest Development Team diff --git a/tasks/release.patch.rst b/tasks/release.patch.rst new file mode 100644 index 000000000000000..56764b9130794dc --- /dev/null +++ b/tasks/release.patch.rst @@ -0,0 +1,17 @@ +pytest-{version} +======================================= + +pytest {version} has just been released to PyPI. + +This is a bug-fix release, being a drop-in replacement. To upgrade:: + + pip install --upgrade pytest + +The full changelog is available at http://doc.pytest.org/en/latest/changelog.html. + +Thanks to all who contributed to this release, among them: + +{contributors} + +Happy testing, +The pytest Development Team diff --git a/tasks/requirements.txt b/tasks/requirements.txt new file mode 100644 index 000000000000000..6392de0cc0a5aa6 --- /dev/null +++ b/tasks/requirements.txt @@ -0,0 +1,5 @@ +invoke +tox +gitpython +towncrier +wheel diff --git a/testing/acceptance_test.py b/testing/acceptance_test.py index 9bc3a191a537d9a..a7838545b0793fe 100644 --- a/testing/acceptance_test.py +++ b/testing/acceptance_test.py @@ -1,3 +1,6 @@ +# -*- coding: utf-8 -*- +from __future__ import absolute_import, division, print_function +import os import sys import _pytest._code @@ -6,7 +9,7 @@ from _pytest.main import EXIT_NOTESTSCOLLECTED, EXIT_USAGEERROR -class TestGeneralUsage: +class TestGeneralUsage(object): def test_config_error(self, testdir): testdir.makeconftest(""" def pytest_configure(config): @@ -71,14 +74,13 @@ def pytest_unconfigure(): print("---unconfigure") """) result = testdir.runpytest("-s", "asd") - assert result.ret == 4 # EXIT_USAGEERROR + assert result.ret == 4 # EXIT_USAGEERROR result.stderr.fnmatch_lines(["ERROR: file not found*asd"]) result.stdout.fnmatch_lines([ "*---configure", "*---unconfigure", ]) - def test_config_preparse_plugin_option(self, testdir): testdir.makepyfile(pytest_xyz=""" def pytest_addoption(parser): @@ -116,10 +118,11 @@ def test_this(): testdir.makepyfile(import_fails="import does_not_work") result = testdir.runpytest(p) result.stdout.fnmatch_lines([ - #XXX on jython this fails: "> import import_fails", - "E ImportError: No module named *does_not_work*", + # XXX on jython this fails: "> import import_fails", + "ImportError while importing test module*", + "*No module named *does_not_work*", ]) - assert result.ret == 1 + assert result.ret == 2 def test_not_collectable_arguments(self, testdir): p1 = testdir.makepyfile("") @@ -127,7 +130,7 @@ def test_not_collectable_arguments(self, testdir): result = testdir.runpytest(p1, p2) assert result.ret result.stderr.fnmatch_lines([ - "*ERROR: not found:*%s" %(p2.basename,) + "*ERROR: not found:*%s" % (p2.basename,) ]) def test_issue486_better_reporting_on_conftest_load_failure(self, testdir): @@ -143,7 +146,6 @@ def test_issue486_better_reporting_on_conftest_load_failure(self, testdir): *ERROR*could not load*conftest.py* """) - def test_early_skip(self, testdir): testdir.mkdir("xyz") testdir.makeconftest(""" @@ -251,7 +253,7 @@ def pytest_collect_file(path, parent): if path.basename.startswith("conftest"): return MyCollector(path, parent) """) - result = testdir.runpytest(c.basename+"::"+"xyz") + result = testdir.runpytest(c.basename + "::" + "xyz") assert result.ret == 0 result.stdout.fnmatch_lines([ "*1 pass*", @@ -306,15 +308,15 @@ def pytest_configure(): x """) result = testdir.runpytest() - assert result.ret == 3 # internal error + assert result.ret == 3 # internal error result.stderr.fnmatch_lines([ "INTERNAL*pytest_configure*", "INTERNAL*x*", ]) assert 'sessionstarttime' not in result.stderr.str() - @pytest.mark.parametrize('lookfor', ['test_fun.py', 'test_fun.py::test_a']) - def test_issue134_report_syntaxerror_when_collecting_member(self, testdir, lookfor): + @pytest.mark.parametrize('lookfor', ['test_fun.py::test_a']) + def test_issue134_report_error_when_collecting_member(self, testdir, lookfor): testdir.makepyfile(test_fun=""" def test_a(): pass @@ -335,10 +337,16 @@ def test_report_all_failed_collections_initargs(self, testdir): "*ERROR*test_b.py::b*", ]) + @pytest.mark.usefixtures('recwarn') def test_namespace_import_doesnt_confuse_import_hook(self, testdir): - # Ref #383. Python 3.3's namespace package messed with our import hooks - # Importing a module that didn't exist, even if the ImportError was - # gracefully handled, would make our test crash. + """ + Ref #383. Python 3.3's namespace package messed with our import hooks + Importing a module that didn't exist, even if the ImportError was + gracefully handled, would make our test crash. + + Use recwarn here to silence this warning in Python 2.7: + ImportWarning: Not importing directory '...\not_a_package': missing __init__.py + """ testdir.mkdir('not_a_package') p = testdir.makepyfile(""" try: @@ -374,7 +382,7 @@ def test_foo(invalid_fixture): res = testdir.runpytest(p) res.stdout.fnmatch_lines([ "*source code not available*", - "*fixture 'invalid_fixture' not found", + "E*fixture 'invalid_fixture' not found", ]) def test_plugins_given_as_strings(self, tmpdir, monkeypatch): @@ -392,7 +400,7 @@ def test_plugins_given_as_strings(self, tmpdir, monkeypatch): monkeypatch.setitem(sys.modules, 'myplugin', mod) assert pytest.main(args=[str(tmpdir)], plugins=['myplugin']) == 0 - def test_parameterized_with_bytes_regex(self, testdir): + def test_parametrized_with_bytes_regex(self, testdir): p = testdir.makepyfile(""" import re import pytest @@ -400,14 +408,27 @@ def test_parameterized_with_bytes_regex(self, testdir): def test_stuff(r): pass """ - ) + ) res = testdir.runpytest(p) res.stdout.fnmatch_lines([ '*1 passed*' ]) + def test_parametrized_with_null_bytes(self, testdir): + """Test parametrization with values that contain null bytes and unicode characters (#2644, #2957)""" + p = testdir.makepyfile(u""" + # encoding: UTF-8 + import pytest + + @pytest.mark.parametrize("data", [b"\\x00", "\\x00", u'ação']) + def test_foo(data): + assert data + """) + res = testdir.runpytest(p) + res.assert_outcomes(passed=3) + -class TestInvocationVariants: +class TestInvocationVariants(object): def test_earlyinit(self, testdir): p = testdir.makepyfile(""" import pytest @@ -430,8 +451,8 @@ def test_import_star_py_dot_test(self, testdir): #collect #cmdline #Item - #assert collect.Item is Item - #assert collect.Collector is Collector + # assert collect.Item is Item + # assert collect.Collector is Collector main skip xfail @@ -499,7 +520,7 @@ def test_invoke_with_path(self, tmpdir, capsys): out, err = capsys.readouterr() def test_invoke_plugin_api(self, testdir, capsys): - class MyPlugin: + class MyPlugin(object): def pytest_addoption(self, parser): parser.addoption("--myopt") @@ -512,15 +533,15 @@ def test_pyargs_importerror(self, testdir, monkeypatch): path = testdir.mkpydir("tpkg") path.join("test_hello.py").write('raise ImportError') - result = testdir.runpytest("--pyargs", "tpkg.test_hello") + result = testdir.runpytest_subprocess("--pyargs", "tpkg.test_hello") assert result.ret != 0 - # FIXME: It would be more natural to match NOT - # "ERROR*file*or*package*not*found*". + result.stdout.fnmatch_lines([ - "*collected 0 items*" + "collected*0*items*/*1*errors" ]) def test_cmdline_python_package(self, testdir, monkeypatch): + import warnings monkeypatch.delenv('PYTHONDONTWRITEBYTECODE', False) path = testdir.mkpydir("tpkg") path.join("test_hello.py").write("def test_hello(): pass") @@ -539,22 +560,89 @@ def test_cmdline_python_package(self, testdir, monkeypatch): def join_pythonpath(what): cur = py.std.os.environ.get('PYTHONPATH') if cur: - return str(what) + ':' + cur + return str(what) + os.pathsep + cur return what empty_package = testdir.mkpydir("empty_package") monkeypatch.setenv('PYTHONPATH', join_pythonpath(empty_package)) - result = testdir.runpytest("--pyargs", ".") + # the path which is not a package raises a warning on pypy; + # no idea why only pypy and not normal python warn about it here + with warnings.catch_warnings(): + warnings.simplefilter('ignore', ImportWarning) + result = testdir.runpytest("--pyargs", ".") assert result.ret == 0 result.stdout.fnmatch_lines([ "*2 passed*" ]) monkeypatch.setenv('PYTHONPATH', join_pythonpath(testdir)) - path.join('test_hello.py').remove() - result = testdir.runpytest("--pyargs", "tpkg.test_hello") + result = testdir.runpytest("--pyargs", "tpkg.test_missing") assert result.ret != 0 result.stderr.fnmatch_lines([ - "*not*found*test_hello*", + "*not*found*test_missing*", + ]) + + def test_cmdline_python_namespace_package(self, testdir, monkeypatch): + """ + test --pyargs option with namespace packages (#1567) + """ + monkeypatch.delenv('PYTHONDONTWRITEBYTECODE', raising=False) + + search_path = [] + for dirname in "hello", "world": + d = testdir.mkdir(dirname) + search_path.append(d) + ns = d.mkdir("ns_pkg") + ns.join("__init__.py").write( + "__import__('pkg_resources').declare_namespace(__name__)") + lib = ns.mkdir(dirname) + lib.ensure("__init__.py") + lib.join("test_{0}.py".format(dirname)). \ + write("def test_{0}(): pass\n" + "def test_other():pass".format(dirname)) + + # The structure of the test directory is now: + # . + # ├── hello + # │   └── ns_pkg + # │   ├── __init__.py + # │   └── hello + # │   ├── __init__.py + # │   └── test_hello.py + # └── world + # └── ns_pkg + # ├── __init__.py + # └── world + # ├── __init__.py + # └── test_world.py + + def join_pythonpath(*dirs): + cur = py.std.os.environ.get('PYTHONPATH') + if cur: + dirs += (cur,) + return os.pathsep.join(str(p) for p in dirs) + monkeypatch.setenv('PYTHONPATH', join_pythonpath(*search_path)) + for p in search_path: + monkeypatch.syspath_prepend(p) + + os.chdir('world') + # mixed module and filenames: + result = testdir.runpytest("--pyargs", "-v", "ns_pkg.hello", "ns_pkg/world") + testdir.chdir() + assert result.ret == 0 + result.stdout.fnmatch_lines([ + "*test_hello.py::test_hello*PASSED*", + "*test_hello.py::test_other*PASSED*", + "*test_world.py::test_world*PASSED*", + "*test_world.py::test_other*PASSED*", + "*4 passed*" + ]) + + # specify tests within a module + result = testdir.runpytest("--pyargs", "-v", "ns_pkg.world.test_world::test_other") + assert result.ret == 0 + result.stdout.fnmatch_lines([ + "*test_world.py::test_other*PASSED*", + "*1 passed*" ]) def test_cmdline_python_package_not_exists(self, testdir): @@ -601,13 +689,12 @@ def test_core_backward_compatibility(self): import _pytest.config assert type(_pytest.config.get_plugin_manager()) is _pytest.config.PytestPluginManager - def test_has_plugin(self, request): """Test hasplugin function of the plugin manager (#932).""" assert request.config.pluginmanager.hasplugin('python') -class TestDurations: +class TestDurations(object): source = """ import time frag = 0.002 @@ -644,12 +731,12 @@ def test_calls_showall(self, testdir): result = testdir.runpytest("--durations=0") assert result.ret == 0 for x in "123": - for y in 'call',: #'setup', 'call', 'teardown': + for y in 'call', : # 'setup', 'call', 'teardown': for line in result.stdout.lines: if ("test_%s" % x) in line and y in line: break else: - raise AssertionError("not found %s %s" % (x,y)) + raise AssertionError("not found %s %s" % (x, y)) def test_with_deselected(self, testdir): testdir.makepyfile(self.source) @@ -664,14 +751,21 @@ def test_with_failing_collection(self, testdir): testdir.makepyfile(self.source) testdir.makepyfile(test_collecterror="""xyz""") result = testdir.runpytest("--durations=2", "-k test_1") - assert result.ret != 0 + assert result.ret == 2 result.stdout.fnmatch_lines([ - "*durations*", - "*call*test_1*", + "*Interrupted: 1 errors during collection*", ]) + # Collection errors abort test execution, therefore no duration is + # output + assert "duration" not in result.stdout.str() + + def test_with_not(self, testdir): + testdir.makepyfile(self.source) + result = testdir.runpytest("-k not 1") + assert result.ret == 0 -class TestDurationWithFixture: +class TestDurationWithFixture(object): source = """ import time frag = 0.001 @@ -682,6 +776,7 @@ def test_1(): def test_2(): time.sleep(frag) """ + def test_setup_function(self, testdir): testdir.makepyfile(self.source) result = testdir.runpytest("--durations=10") @@ -693,3 +788,63 @@ def test_setup_function(self, testdir): * call *test_1* """) + +def test_zipimport_hook(testdir, tmpdir): + """Test package loader is being used correctly (see #1837).""" + zipapp = pytest.importorskip('zipapp') + testdir.tmpdir.join('app').ensure(dir=1) + testdir.makepyfile(**{ + 'app/foo.py': """ + import pytest + def main(): + pytest.main(['--pyarg', 'foo']) + """, + }) + target = tmpdir.join('foo.zip') + zipapp.create_archive(str(testdir.tmpdir.join('app')), str(target), main='foo:main') + result = testdir.runpython(target) + assert result.ret == 0 + result.stderr.fnmatch_lines(['*not found*foo*']) + assert 'INTERNALERROR>' not in result.stdout.str() + + +def test_import_plugin_unicode_name(testdir): + testdir.makepyfile( + myplugin='', + ) + testdir.makepyfile(""" + def test(): pass + """) + testdir.makeconftest(""" + pytest_plugins = [u'myplugin'] + """) + r = testdir.runpytest() + assert r.ret == 0 + + +def test_deferred_hook_checking(testdir): + """ + Check hooks as late as possible (#1821). + """ + testdir.syspathinsert() + testdir.makepyfile(**{ + 'plugin.py': """ + class Hooks: + def pytest_my_hook(self, config): + pass + + def pytest_configure(config): + config.pluginmanager.add_hookspecs(Hooks) + """, + 'conftest.py': """ + pytest_plugins = ['plugin'] + def pytest_my_hook(config): + return 40 + """, + 'test_foo.py': """ + def test(request): + assert request.config.hook.pytest_my_hook(config=request.config) == [40] + """ + }) + result = testdir.runpytest() + result.stdout.fnmatch_lines(['* 1 passed *']) diff --git a/testing/code/test_code.py b/testing/code/test_code.py index 0db4ad2ab4c2ea7..209a8ef19a023b7 100644 --- a/testing/code/test_code.py +++ b/testing/code/test_code.py @@ -1,8 +1,11 @@ +# coding: utf-8 +from __future__ import absolute_import, division, print_function import sys import _pytest._code import py import pytest +from test_excinfo import TWMock def test_ne(): @@ -11,6 +14,7 @@ def test_ne(): code2 = _pytest._code.Code(compile('foo = "baz"', '', 'exec')) assert code2 != code1 + def test_code_gives_back_name_for_not_existing_file(): name = 'abc-123' co_code = compile("pass\n", name, 'exec') @@ -19,20 +23,24 @@ def test_code_gives_back_name_for_not_existing_file(): assert str(code.path) == name assert code.fullsource is None + def test_code_with_class(): - class A: + class A(object): pass pytest.raises(TypeError, "_pytest._code.Code(A)") + if True: def x(): pass + def test_code_fullsource(): code = _pytest._code.Code(x) full = code.fullsource assert 'test_code_fullsource()' in str(full) + def test_code_source(): code = _pytest._code.Code(x) src = code.source() @@ -40,6 +48,7 @@ def test_code_source(): pass""" assert str(src) == expected + def test_frame_getsourcelineno_myself(): def func(): return sys._getframe(0) @@ -48,6 +57,7 @@ def func(): source, lineno = f.code.fullsource, f.lineno assert source[lineno].startswith(" return sys._getframe(0)") + def test_getstatement_empty_fullsource(): def func(): return sys._getframe(0) @@ -60,34 +70,19 @@ def func(): finally: f.code.__class__.fullsource = prop + def test_code_from_func(): co = _pytest._code.Code(test_frame_getsourcelineno_myself) assert co.firstlineno assert co.path - -def test_builtin_patch_unpatch(monkeypatch): - cpy_builtin = py.builtin.builtins - comp = cpy_builtin.compile - def mycompile(*args, **kwargs): - return comp(*args, **kwargs) - class Sub(AssertionError): - pass - monkeypatch.setattr(cpy_builtin, 'AssertionError', Sub) - monkeypatch.setattr(cpy_builtin, 'compile', mycompile) - _pytest._code.patch_builtins() - assert cpy_builtin.AssertionError != Sub - assert cpy_builtin.compile != mycompile - _pytest._code.unpatch_builtins() - assert cpy_builtin.AssertionError is Sub - assert cpy_builtin.compile == mycompile - - def test_unicode_handling(): value = py.builtin._totext('\xc4\x85\xc4\x87\n', 'utf-8').encode('utf8') + def f(): raise Exception(value) + excinfo = pytest.raises(Exception, f) str(excinfo) if sys.version_info[0] < 3: @@ -97,13 +92,16 @@ def f(): @pytest.mark.skipif(sys.version_info[0] >= 3, reason='python 2 only issue') def test_unicode_handling_syntax_error(): value = py.builtin._totext('\xc4\x85\xc4\x87\n', 'utf-8').encode('utf8') + def f(): raise SyntaxError('invalid syntax', (None, 1, 3, value)) + excinfo = pytest.raises(Exception, f) str(excinfo) if sys.version_info[0] < 3: unicode(excinfo) + def test_code_getargs(): def f1(x): pass @@ -149,26 +147,50 @@ def f4(x, *y, **z): ('z', {'c': 'd'})] -class TestExceptionInfo: +class TestExceptionInfo(object): def test_bad_getsource(self): try: - if False: pass - else: assert False + if False: + pass + else: + assert False except AssertionError: exci = _pytest._code.ExceptionInfo() assert exci.getrepr() -class TestTracebackEntry: +class TestTracebackEntry(object): def test_getsource(self): try: - if False: pass - else: assert False + if False: + pass + else: + assert False except AssertionError: exci = _pytest._code.ExceptionInfo() entry = exci.traceback[0] source = entry.getsource() - assert len(source) == 4 - assert 'else: assert False' in source[3] + assert len(source) == 6 + assert 'assert False' in source[5] + + +class TestReprFuncArgs(object): + + def test_not_raise_exception_with_mixed_encoding(self): + from _pytest._code.code import ReprFuncArgs + + tw = TWMock() + + args = [ + ('unicode_string', u"São Paulo"), + ('utf8_string', 'S\xc3\xa3o Paulo'), + ] + + r = ReprFuncArgs(args) + r.toterminal(tw) + if sys.version_info[0] >= 3: + assert tw.lines[0] == 'unicode_string = São Paulo, utf8_string = São Paulo' + else: + assert tw.lines[0] == 'unicode_string = São Paulo, utf8_string = São Paulo' diff --git a/testing/code/test_excinfo.py b/testing/code/test_excinfo.py index 2defa31035b836d..34db8ffa18983ac 100644 --- a/testing/code/test_excinfo.py +++ b/testing/code/test_excinfo.py @@ -1,13 +1,16 @@ # -*- coding: utf-8 -*- +from __future__ import absolute_import, division, print_function +import operator import _pytest import py import pytest -from _pytest._code.code import FormattedExcinfo, ReprExceptionInfo +from _pytest._code.code import ( + ExceptionInfo, + FormattedExcinfo, + ReprExceptionInfo, + ExceptionChainRepr) -queue = py.builtin._tryimport('queue', 'Queue') - -failsonjython = pytest.mark.xfail("sys.platform.startswith('java')") from test_source import astonly try: @@ -17,21 +20,40 @@ else: invalidate_import_caches = getattr(importlib, "invalidate_caches", None) -import pytest +queue = py.builtin._tryimport('queue', 'Queue') + +failsonjython = pytest.mark.xfail("sys.platform.startswith('java')") + pytest_version_info = tuple(map(int, pytest.__version__.split(".")[:3])) -class TWMock: + +class TWMock(object): + WRITE = object() + def __init__(self): self.lines = [] + self.is_writing = False + def sep(self, sep, line=None): self.lines.append((sep, line)) + + def write(self, msg, **kw): + self.lines.append((TWMock.WRITE, msg)) + def line(self, line, **kw): self.lines.append(line) + def markup(self, text, **kw): return text + def get_write_msg(self, idx): + flag, msg = self.lines[idx] + assert flag == TWMock.WRITE + return msg + fullwidth = 80 + def test_excinfo_simple(): try: raise ValueError @@ -39,41 +61,51 @@ def test_excinfo_simple(): info = _pytest._code.ExceptionInfo() assert info.type == ValueError + def test_excinfo_getstatement(): def g(): raise ValueError + def f(): g() + try: f() except ValueError: excinfo = _pytest._code.ExceptionInfo() - linenumbers = [_pytest._code.getrawcode(f).co_firstlineno - 1 + 3, + linenumbers = [_pytest._code.getrawcode(f).co_firstlineno - 1 + 4, _pytest._code.getrawcode(f).co_firstlineno - 1 + 1, _pytest._code.getrawcode(g).co_firstlineno - 1 + 1, ] - l = list(excinfo.traceback) - foundlinenumbers = [x.lineno for x in l] + values = list(excinfo.traceback) + foundlinenumbers = [x.lineno for x in values] assert foundlinenumbers == linenumbers - #for x in info: + # for x in info: # print "%s:%d %s" %(x.path.relto(root), x.lineno, x.statement) - #xxx + # xxx # testchain for getentries test below + + def f(): # raise ValueError # + + def g(): # __tracebackhide__ = True f() # + + def h(): # g() # -class TestTraceback_f_g_h: + +class TestTraceback_f_g_h(object): def setup_method(self, method): try: h() @@ -83,8 +115,8 @@ def setup_method(self, method): def test_traceback_entries(self): tb = self.excinfo.traceback entries = list(tb) - assert len(tb) == 4 # maybe fragile test - assert len(entries) == 4 # maybe fragile test + assert len(tb) == 4 # maybe fragile test + assert len(entries) == 4 # maybe fragile test names = ['f', 'g', 'h'] for entry in entries: try: @@ -95,7 +127,7 @@ def test_traceback_entries(self): def test_traceback_entry_getsource(self): tb = self.excinfo.traceback - s = str(tb[-1].getsource() ) + s = str(tb[-1].getsource()) assert s.startswith("def f():") assert s.endswith("raise ValueError") @@ -111,10 +143,10 @@ def xyz(): xyz() """) try: - exec (source.compile()) + exec(source.compile()) except NameError: tb = _pytest._code.ExceptionInfo().traceback - print (tb[-1].getsource()) + print(tb[-1].getsource()) s = str(tb[-1].getsource()) assert s.startswith("def xyz():\n try:") assert s.strip().endswith("except somenoname:") @@ -125,7 +157,7 @@ def test_traceback_cut(self): traceback = self.excinfo.traceback newtraceback = traceback.cut(path=path, firstlineno=firstlineno) assert len(newtraceback) == 1 - newtraceback = traceback.cut(path=path, lineno=firstlineno+2) + newtraceback = traceback.cut(path=path, lineno=firstlineno + 2) assert len(newtraceback) == 1 def test_traceback_cut_excludepath(self, testdir): @@ -143,6 +175,41 @@ def test_traceback_filter(self): ntraceback = traceback.filter() assert len(ntraceback) == len(traceback) - 1 + @pytest.mark.parametrize('tracebackhide, matching', [ + (lambda info: True, True), + (lambda info: False, False), + (operator.methodcaller('errisinstance', ValueError), True), + (operator.methodcaller('errisinstance', IndexError), False), + ]) + def test_traceback_filter_selective(self, tracebackhide, matching): + def f(): + # + raise ValueError + # + + def g(): + # + __tracebackhide__ = tracebackhide + f() + # + + def h(): + # + g() + # + + excinfo = pytest.raises(ValueError, h) + traceback = excinfo.traceback + ntraceback = traceback.filter() + print('old: {0!r}'.format(traceback)) + print('new: {0!r}'.format(ntraceback)) + + if matching: + assert len(ntraceback) == len(traceback) - 2 + else: + # -1 because of the __tracebackhide__ in pytest.raises + assert len(ntraceback) == len(traceback) - 1 + def test_traceback_recursion_index(self): def f(n): if n < 10: @@ -157,7 +224,7 @@ def test_traceback_only_specific_recursion_errors(self, monkeypatch): def f(n): if n == 0: raise RuntimeError("hello") - f(n-1) + f(n - 1) excinfo = pytest.raises(RuntimeError, f, 100) monkeypatch.delattr(excinfo.traceback.__class__, "recursionindex") @@ -167,22 +234,25 @@ def f(n): def test_traceback_no_recursion_index(self): def do_stuff(): raise RuntimeError + def reraise_me(): import sys exc, val, tb = sys.exc_info() py.builtin._reraise(exc, val, tb) + def f(n): try: do_stuff() - except: + except: # noqa reraise_me() + excinfo = pytest.raises(RuntimeError, f, 8) traceback = excinfo.traceback recindex = traceback.recursionindex() assert recindex is None def test_traceback_messy_recursion(self): - #XXX: simplified locally testable version + # XXX: simplified locally testable version decorator = pytest.importorskip('decorator').decorator def log(f, *k, **kw): @@ -198,17 +268,18 @@ def fail(): excinfo = pytest.raises(ValueError, fail) assert excinfo.traceback.recursionindex() is None - - def test_traceback_getcrashentry(self): def i(): __tracebackhide__ = True raise ValueError + def h(): i() + def g(): __tracebackhide__ = True h() + def f(): g() @@ -224,6 +295,7 @@ def test_traceback_getcrashentry_empty(self): def g(): __tracebackhide__ = True raise ValueError + def f(): __tracebackhide__ = True g() @@ -236,56 +308,47 @@ def f(): assert entry.lineno == co.firstlineno + 2 assert entry.frame.code.name == 'g' -def hello(x): - x + 5 - -def test_tbentry_reinterpret(): - try: - hello("hello") - except TypeError: - excinfo = _pytest._code.ExceptionInfo() - tbentry = excinfo.traceback[-1] - msg = tbentry.reinterpret() - assert msg.startswith("TypeError: ('hello' + 5)") def test_excinfo_exconly(): excinfo = pytest.raises(ValueError, h) assert excinfo.exconly().startswith('ValueError') excinfo = pytest.raises(ValueError, - "raise ValueError('hello\\nworld')") + "raise ValueError('hello\\nworld')") msg = excinfo.exconly(tryshort=True) assert msg.startswith('ValueError') assert msg.endswith("world") + def test_excinfo_repr(): excinfo = pytest.raises(ValueError, h) s = repr(excinfo) assert s == "" + def test_excinfo_str(): excinfo = pytest.raises(ValueError, h) s = str(excinfo) - assert s.startswith(__file__[:-9]) # pyc file and $py.class + assert s.startswith(__file__[:-9]) # pyc file and $py.class assert s.endswith("ValueError") - assert len(s.split(":")) >= 3 # on windows it's 4 + assert len(s.split(":")) >= 3 # on windows it's 4 + def test_excinfo_errisinstance(): excinfo = pytest.raises(ValueError, h) assert excinfo.errisinstance(ValueError) + def test_excinfo_no_sourcecode(): try: - exec ("raise ValueError()") + exec("raise ValueError()") except ValueError: excinfo = _pytest._code.ExceptionInfo() s = str(excinfo.traceback[-1]) - if py.std.sys.version_info < (2,5): - assert s == " File '':1 in ?\n ???\n" - else: - assert s == " File '':1 in \n ???\n" + assert s == " File '':1 in \n ???\n" + def test_excinfo_no_python_sourcecode(tmpdir): - #XXX: simplified locally testable version + # XXX: simplified locally testable version tmpdir.join('test.txt').write("{{ h()}}:") jinja2 = pytest.importorskip('jinja2') @@ -293,10 +356,10 @@ def test_excinfo_no_python_sourcecode(tmpdir): env = jinja2.Environment(loader=loader) template = env.get_template('test.txt') excinfo = pytest.raises(ValueError, - template.render, h=h) + template.render, h=h) for item in excinfo.traceback: - print(item) #XXX: for some reason jinja.Template.render is printed in full - item.source # shouldnt fail + print(item) # XXX: for some reason jinja.Template.render is printed in full + item.source # shouldnt fail if item.path.basename == 'test.txt': assert str(item.source) == '{{ h()}}:' @@ -312,6 +375,7 @@ def test_entrysource_Queue_example(): s = str(source).strip() assert s.startswith("def get") + def test_codepath_Queue_example(): try: queue.Queue().get(timeout=0.001) @@ -323,11 +387,35 @@ def test_codepath_Queue_example(): assert path.basename.lower() == "queue.py" assert path.check() -class TestFormattedExcinfo: - def pytest_funcarg__importasmod(self, request): + +def test_match_succeeds(): + with pytest.raises(ZeroDivisionError) as excinfo: + 0 // 0 + excinfo.match(r'.*zero.*') + + +def test_match_raises_error(testdir): + testdir.makepyfile(""" + import pytest + def test_division_zero(): + with pytest.raises(ZeroDivisionError) as excinfo: + 0 / 0 + excinfo.match(r'[123]+') + """) + result = testdir.runpytest() + assert result.ret != 0 + result.stdout.fnmatch_lines([ + "*AssertionError*Pattern*[123]*not found*", + ]) + + +class TestFormattedExcinfo(object): + + @pytest.fixture + def importasmod(self, request): def importasmod(source): source = _pytest._code.Source(source) - tmpdir = request.getfuncargvalue("tmpdir") + tmpdir = request.getfixturevalue("tmpdir") modpath = tmpdir.join("mod.py") tmpdir.ensure("__init__.py") modpath.write(source) @@ -339,10 +427,10 @@ def importasmod(source): def excinfo_from_exec(self, source): source = _pytest._code.Source(source).strip() try: - exec (source.compile()) + exec(source.compile()) except KeyboardInterrupt: raise - except: + except: # noqa return _pytest._code.ExceptionInfo() assert 0, "did not raise" @@ -372,19 +460,20 @@ def f(): assert lines == [ ' def f():', '> assert 0', - 'E assert 0' + 'E AssertionError' ] - def test_repr_source_not_existing(self): pr = FormattedExcinfo() co = compile("raise ValueError()", "", "exec") try: - exec (co) + exec(co) except ValueError: excinfo = _pytest._code.ExceptionInfo() repr = pr.repr_excinfo(excinfo) assert repr.reprtraceback.reprentries[1].lines[0] == "> ???" + if py.std.sys.version_info[0] >= 3: + assert repr.chain[0][0].reprentries[1].lines[0] == "> ???" def test_repr_many_line_source_not_existing(self): pr = FormattedExcinfo() @@ -393,23 +482,27 @@ def test_repr_many_line_source_not_existing(self): raise ValueError() """, "", "exec") try: - exec (co) + exec(co) except ValueError: excinfo = _pytest._code.ExceptionInfo() repr = pr.repr_excinfo(excinfo) assert repr.reprtraceback.reprentries[1].lines[0] == "> ???" + if py.std.sys.version_info[0] >= 3: + assert repr.chain[0][0].reprentries[1].lines[0] == "> ???" def test_repr_source_failing_fullsource(self): pr = FormattedExcinfo() class FakeCode(object): - class raw: + class raw(object): co_filename = '?' + path = '?' firstlineno = 5 def fullsource(self): return None + fullsource = property(fullsource) class FakeFrame(object): @@ -418,8 +511,8 @@ class FakeFrame(object): f_globals = {} class FakeTracebackEntry(_pytest._code.Traceback.Entry): - def __init__(self, tb): - self.lineno = 5+3 + def __init__(self, tb, excinfo=None): + self.lineno = 5 + 3 @property def frame(self): @@ -430,28 +523,36 @@ class Traceback(_pytest._code.Traceback): class FakeExcinfo(_pytest._code.ExceptionInfo): typename = "Foo" + value = Exception() + def __init__(self): pass def exconly(self, tryshort): return "EXC" + def errisinstance(self, cls): return False excinfo = FakeExcinfo() + class FakeRawTB(object): tb_next = None + tb = FakeRawTB() excinfo.traceback = Traceback(tb) fail = IOError() # noqa repr = pr.repr_excinfo(excinfo) assert repr.reprtraceback.reprentries[0].lines[0] == "> ???" + if py.std.sys.version_info[0] >= 3: + assert repr.chain[0][0].reprentries[0].lines[0] == "> ???" fail = py.error.ENOENT # noqa repr = pr.repr_excinfo(excinfo) assert repr.reprtraceback.reprentries[0].lines[0] == "> ???" - + if py.std.sys.version_info[0] >= 3: + assert repr.chain[0][0].reprentries[0].lines[0] == "> ???" def test_repr_local(self): p = FormattedExcinfo(showlocals=True) @@ -492,19 +593,19 @@ def func1(): loc = repr_entry.reprfileloc assert loc.path == mod.__file__ assert loc.lineno == 3 - #assert loc.message == "ValueError: hello" + # assert loc.message == "ValueError: hello" def test_repr_tracebackentry_lines2(self, importasmod): mod = importasmod(""" def func1(m, x, y, z): raise ValueError("hello\\nworld") """) - excinfo = pytest.raises(ValueError, mod.func1, "m"*90, 5, 13, "z"*120) + excinfo = pytest.raises(ValueError, mod.func1, "m" * 90, 5, 13, "z" * 120) excinfo.traceback = excinfo.traceback.filter() entry = excinfo.traceback[-1] p = FormattedExcinfo(funcargs=True) reprfuncargs = p.repr_args(entry) - assert reprfuncargs.args[0] == ('m', repr("m"*90)) + assert reprfuncargs.args[0] == ('m', repr("m" * 90)) assert reprfuncargs.args[1] == ('x', '5') assert reprfuncargs.args[2] == ('y', '13') assert reprfuncargs.args[3] == ('z', repr("z" * 120)) @@ -637,6 +738,9 @@ def entry(): repr = p.repr_excinfo(excinfo) assert repr.reprtraceback assert len(repr.reprtraceback.reprentries) == len(reprtb.reprentries) + if py.std.sys.version_info[0] >= 3: + assert repr.chain[0][0] + assert len(repr.chain[0][0].reprentries) == len(reprtb.reprentries) assert repr.reprcrash.path.endswith("mod.py") assert repr.reprcrash.message == "ValueError: 0" @@ -650,8 +754,10 @@ def entry(): excinfo = pytest.raises(ValueError, mod.entry) p = FormattedExcinfo() + def raiseos(): raise OSError(2) + monkeypatch.setattr(py.std.os, 'getcwd', raiseos) assert p._makepath(__file__) == __file__ p.repr_traceback(excinfo) @@ -698,23 +804,6 @@ def entry(): assert reprtb.extraline == "!!! Recursion detected (same locals & position)" assert str(reprtb) - def test_tb_entry_AssertionError(self, importasmod): - # probably this test is a bit redundant - # as py/magic/testing/test_assertion.py - # already tests correctness of - # assertion-reinterpretation logic - mod = importasmod(""" - def somefunc(): - x = 1 - assert x == 2 - """) - excinfo = pytest.raises(AssertionError, mod.somefunc) - - p = FormattedExcinfo() - reprentry = p.repr_traceback_entry(excinfo.traceback[-1], excinfo) - lines = reprentry.lines - assert lines[-1] == "E assert 1 == 2" - def test_reprexcinfo_getrepr(self, importasmod): mod = importasmod(""" def f(x): @@ -727,14 +816,21 @@ def entry(): for style in ("short", "long", "no"): for showlocals in (True, False): repr = excinfo.getrepr(style=style, showlocals=showlocals) - assert isinstance(repr, ReprExceptionInfo) + if py.std.sys.version_info[0] < 3: + assert isinstance(repr, ReprExceptionInfo) assert repr.reprtraceback.style == style + if py.std.sys.version_info[0] >= 3: + assert isinstance(repr, ExceptionChainRepr) + for repr in repr.chain: + assert repr[0].style == style def test_reprexcinfo_unicode(self): from _pytest._code.code import TerminalRepr + class MyRepr(TerminalRepr): def toterminal(self, tw): tw.line(py.builtin._totext("я", "utf-8")) + x = py.builtin._totext(MyRepr()) assert x == py.builtin._totext("я", "utf-8") @@ -755,14 +851,18 @@ def f(): assert tw.lines[0] == " def f():" assert tw.lines[1] == "> g(3)" assert tw.lines[2] == "" - assert tw.lines[3].endswith("mod.py:5: ") - assert tw.lines[4] == ("_ ", None) - assert tw.lines[5] == "" - assert tw.lines[6] == " def g(x):" - assert tw.lines[7] == "> raise ValueError(x)" - assert tw.lines[8] == "E ValueError: 3" - assert tw.lines[9] == "" - assert tw.lines[10].endswith("mod.py:3: ValueError") + line = tw.get_write_msg(3) + assert line.endswith("mod.py") + assert tw.lines[4] == (":5: ") + assert tw.lines[5] == ("_ ", None) + assert tw.lines[6] == "" + assert tw.lines[7] == " def g(x):" + assert tw.lines[8] == "> raise ValueError(x)" + assert tw.lines[9] == "E ValueError: 3" + assert tw.lines[10] == "" + line = tw.get_write_msg(11) + assert line.endswith("mod.py") + assert tw.lines[12] == ":3: ValueError" def test_toterminal_long_missing_source(self, importasmod, tmpdir): mod = importasmod(""" @@ -781,13 +881,17 @@ def f(): tw.lines.pop(0) assert tw.lines[0] == "> ???" assert tw.lines[1] == "" - assert tw.lines[2].endswith("mod.py:5: ") - assert tw.lines[3] == ("_ ", None) - assert tw.lines[4] == "" - assert tw.lines[5] == "> ???" - assert tw.lines[6] == "E ValueError: 3" - assert tw.lines[7] == "" - assert tw.lines[8].endswith("mod.py:3: ValueError") + line = tw.get_write_msg(2) + assert line.endswith("mod.py") + assert tw.lines[3] == ":5: " + assert tw.lines[4] == ("_ ", None) + assert tw.lines[5] == "" + assert tw.lines[6] == "> ???" + assert tw.lines[7] == "E ValueError: 3" + assert tw.lines[8] == "" + line = tw.get_write_msg(9) + assert line.endswith("mod.py") + assert tw.lines[10] == ":3: ValueError" def test_toterminal_long_incomplete_source(self, importasmod, tmpdir): mod = importasmod(""" @@ -806,13 +910,17 @@ def f(): tw.lines.pop(0) assert tw.lines[0] == "> ???" assert tw.lines[1] == "" - assert tw.lines[2].endswith("mod.py:5: ") - assert tw.lines[3] == ("_ ", None) - assert tw.lines[4] == "" - assert tw.lines[5] == "> ???" - assert tw.lines[6] == "E ValueError: 3" - assert tw.lines[7] == "" - assert tw.lines[8].endswith("mod.py:3: ValueError") + line = tw.get_write_msg(2) + assert line.endswith("mod.py") + assert tw.lines[3] == ":5: " + assert tw.lines[4] == ("_ ", None) + assert tw.lines[5] == "" + assert tw.lines[6] == "> ???" + assert tw.lines[7] == "E ValueError: 3" + assert tw.lines[8] == "" + line = tw.get_write_msg(9) + assert line.endswith("mod.py") + assert tw.lines[10] == ":3: ValueError" def test_toterminal_long_filenames(self, importasmod): mod = importasmod(""" @@ -826,25 +934,28 @@ def f(): try: repr = excinfo.getrepr(abspath=False) repr.toterminal(tw) - line = tw.lines[-1] x = py.path.local().bestrelpath(path) if len(x) < len(str(path)): - assert line == "mod.py:3: ValueError" + msg = tw.get_write_msg(-2) + assert msg == "mod.py" + assert tw.lines[-1] == ":3: ValueError" repr = excinfo.getrepr(abspath=True) repr.toterminal(tw) + msg = tw.get_write_msg(-2) + assert msg == path line = tw.lines[-1] - assert line == "%s:3: ValueError" %(path,) + assert line == ":3: ValueError" finally: old.chdir() @pytest.mark.parametrize('reproptions', [ {'style': style, 'showlocals': showlocals, 'funcargs': funcargs, 'tbfilter': tbfilter - } for style in ("long", "short", "no") - for showlocals in (True, False) - for tbfilter in (True, False) - for funcargs in (True, False)]) + } for style in ("long", "short", "no") + for showlocals in (True, False) + for tbfilter in (True, False) + for funcargs in (True, False)]) def test_format_excinfo(self, importasmod, reproptions): mod = importasmod(""" def g(x): @@ -858,21 +969,6 @@ def f(): repr.toterminal(tw) assert tw.stringio.getvalue() - - def test_native_style(self): - excinfo = self.excinfo_from_exec(""" - assert 0 - """) - repr = excinfo.getrepr(style='native') - assert "assert 0" in str(repr.reprcrash) - s = str(repr) - assert s.startswith('Traceback (most recent call last):\n File') - assert s.endswith('\nAssertionError: assert 0') - assert 'exec (source.compile())' in s - # python 2.4 fails to get the source line for the assert - if py.std.sys.version_info >= (2, 5): - assert s.count('assert 0') == 2 - def test_traceback_repr_style(self, importasmod): mod = importasmod(""" def f(): @@ -891,21 +987,259 @@ def i(): r = excinfo.getrepr(style="long") tw = TWMock() r.toterminal(tw) - for line in tw.lines: print (line) + for line in tw.lines: + print(line) assert tw.lines[0] == "" assert tw.lines[1] == " def f():" assert tw.lines[2] == "> g()" assert tw.lines[3] == "" - assert tw.lines[4].endswith("mod.py:3: ") - assert tw.lines[5] == ("_ ", None) - assert tw.lines[6].endswith("in g") - assert tw.lines[7] == " h()" - assert tw.lines[8].endswith("in h") - assert tw.lines[9] == " i()" - assert tw.lines[10] == ("_ ", None) - assert tw.lines[11] == "" - assert tw.lines[12] == " def i():" - assert tw.lines[13] == "> raise ValueError()" - assert tw.lines[14] == "E ValueError" + msg = tw.get_write_msg(4) + assert msg.endswith("mod.py") + assert tw.lines[5] == ":3: " + assert tw.lines[6] == ("_ ", None) + tw.get_write_msg(7) + assert tw.lines[8].endswith("in g") + assert tw.lines[9] == " h()" + tw.get_write_msg(10) + assert tw.lines[11].endswith("in h") + assert tw.lines[12] == " i()" + assert tw.lines[13] == ("_ ", None) + assert tw.lines[14] == "" + assert tw.lines[15] == " def i():" + assert tw.lines[16] == "> raise ValueError()" + assert tw.lines[17] == "E ValueError" + assert tw.lines[18] == "" + msg = tw.get_write_msg(19) + msg.endswith("mod.py") + assert tw.lines[20] == ":9: ValueError" + + @pytest.mark.skipif("sys.version_info[0] < 3") + def test_exc_chain_repr(self, importasmod): + mod = importasmod(""" + class Err(Exception): + pass + def f(): + try: + g() + except Exception as e: + raise Err() from e + finally: + h() + def g(): + raise ValueError() + + def h(): + raise AttributeError() + """) + excinfo = pytest.raises(AttributeError, mod.f) + r = excinfo.getrepr(style="long") + tw = TWMock() + r.toterminal(tw) + for line in tw.lines: + print(line) + assert tw.lines[0] == "" + assert tw.lines[1] == " def f():" + assert tw.lines[2] == " try:" + assert tw.lines[3] == "> g()" + assert tw.lines[4] == "" + line = tw.get_write_msg(5) + assert line.endswith('mod.py') + assert tw.lines[6] == ':6: ' + assert tw.lines[7] == ("_ ", None) + assert tw.lines[8] == "" + assert tw.lines[9] == " def g():" + assert tw.lines[10] == "> raise ValueError()" + assert tw.lines[11] == "E ValueError" + assert tw.lines[12] == "" + line = tw.get_write_msg(13) + assert line.endswith('mod.py') + assert tw.lines[14] == ':12: ValueError' assert tw.lines[15] == "" - assert tw.lines[16].endswith("mod.py:9: ValueError") + assert tw.lines[16] == "The above exception was the direct cause of the following exception:" + assert tw.lines[17] == "" + assert tw.lines[18] == " def f():" + assert tw.lines[19] == " try:" + assert tw.lines[20] == " g()" + assert tw.lines[21] == " except Exception as e:" + assert tw.lines[22] == "> raise Err() from e" + assert tw.lines[23] == "E test_exc_chain_repr0.mod.Err" + assert tw.lines[24] == "" + line = tw.get_write_msg(25) + assert line.endswith('mod.py') + assert tw.lines[26] == ":8: Err" + assert tw.lines[27] == "" + assert tw.lines[28] == "During handling of the above exception, another exception occurred:" + assert tw.lines[29] == "" + assert tw.lines[30] == " def f():" + assert tw.lines[31] == " try:" + assert tw.lines[32] == " g()" + assert tw.lines[33] == " except Exception as e:" + assert tw.lines[34] == " raise Err() from e" + assert tw.lines[35] == " finally:" + assert tw.lines[36] == "> h()" + assert tw.lines[37] == "" + line = tw.get_write_msg(38) + assert line.endswith('mod.py') + assert tw.lines[39] == ":10: " + assert tw.lines[40] == ('_ ', None) + assert tw.lines[41] == "" + assert tw.lines[42] == " def h():" + assert tw.lines[43] == "> raise AttributeError()" + assert tw.lines[44] == "E AttributeError" + assert tw.lines[45] == "" + line = tw.get_write_msg(46) + assert line.endswith('mod.py') + assert tw.lines[47] == ":15: AttributeError" + + @pytest.mark.skipif("sys.version_info[0] < 3") + def test_exc_repr_with_raise_from_none_chain_suppression(self, importasmod): + mod = importasmod(""" + def f(): + try: + g() + except Exception: + raise AttributeError() from None + def g(): + raise ValueError() + """) + excinfo = pytest.raises(AttributeError, mod.f) + r = excinfo.getrepr(style="long") + tw = TWMock() + r.toterminal(tw) + for line in tw.lines: + print(line) + assert tw.lines[0] == "" + assert tw.lines[1] == " def f():" + assert tw.lines[2] == " try:" + assert tw.lines[3] == " g()" + assert tw.lines[4] == " except Exception:" + assert tw.lines[5] == "> raise AttributeError() from None" + assert tw.lines[6] == "E AttributeError" + assert tw.lines[7] == "" + line = tw.get_write_msg(8) + assert line.endswith('mod.py') + assert tw.lines[9] == ":6: AttributeError" + assert len(tw.lines) == 10 + + @pytest.mark.skipif("sys.version_info[0] < 3") + @pytest.mark.parametrize('reason, description', [ + ('cause', 'The above exception was the direct cause of the following exception:'), + ('context', 'During handling of the above exception, another exception occurred:'), + ]) + def test_exc_chain_repr_without_traceback(self, importasmod, reason, description): + """ + Handle representation of exception chains where one of the exceptions doesn't have a + real traceback, such as those raised in a subprocess submitted by the multiprocessing + module (#1984). + """ + from _pytest.pytester import LineMatcher + exc_handling_code = ' from e' if reason == 'cause' else '' + mod = importasmod(""" + def f(): + try: + g() + except Exception as e: + raise RuntimeError('runtime problem'){exc_handling_code} + def g(): + raise ValueError('invalid value') + """.format(exc_handling_code=exc_handling_code)) + + with pytest.raises(RuntimeError) as excinfo: + mod.f() + + # emulate the issue described in #1984 + attr = '__%s__' % reason + getattr(excinfo.value, attr).__traceback__ = None + + r = excinfo.getrepr() + tw = py.io.TerminalWriter(stringio=True) + tw.hasmarkup = False + r.toterminal(tw) + + matcher = LineMatcher(tw.stringio.getvalue().splitlines()) + matcher.fnmatch_lines([ + "ValueError: invalid value", + description, + "* except Exception as e:", + "> * raise RuntimeError('runtime problem')" + exc_handling_code, + "E *RuntimeError: runtime problem", + ]) + + +@pytest.mark.parametrize("style", ["short", "long"]) +@pytest.mark.parametrize("encoding", [None, "utf8", "utf16"]) +def test_repr_traceback_with_unicode(style, encoding): + msg = u'☹' + if encoding is not None: + msg = msg.encode(encoding) + try: + raise RuntimeError(msg) + except RuntimeError: + e_info = ExceptionInfo() + formatter = FormattedExcinfo(style=style) + repr_traceback = formatter.repr_traceback(e_info) + assert repr_traceback is not None + + +def test_cwd_deleted(testdir): + testdir.makepyfile(""" + def test(tmpdir): + tmpdir.chdir() + tmpdir.remove() + assert False + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines(['* 1 failed in *']) + assert 'INTERNALERROR' not in result.stdout.str() + result.stderr.str() + + +def test_exception_repr_extraction_error_on_recursion(): + """ + Ensure we can properly detect a recursion error even + if some locals raise error on comparision (#2459). + """ + class numpy_like(object): + + def __eq__(self, other): + if type(other) is numpy_like: + raise ValueError('The truth value of an array ' + 'with more than one element is ambiguous.') + + def a(x): + return b(numpy_like()) + + def b(x): + return a(numpy_like()) + + try: + a(numpy_like()) + except: # noqa + from _pytest._code.code import ExceptionInfo + from _pytest.pytester import LineMatcher + exc_info = ExceptionInfo() + + matcher = LineMatcher(str(exc_info.getrepr()).splitlines()) + matcher.fnmatch_lines([ + '!!! Recursion error detected, but an error occurred locating the origin of recursion.', + '*The following exception happened*', + '*ValueError: The truth value of an array*', + ]) + + +def test_no_recursion_index_on_recursion_error(): + """ + Ensure that we don't break in case we can't find the recursion index + during a recursion error (#2486). + """ + try: + class RecursionDepthError(object): + def __getattr__(self, attr): + return getattr(self, '_' + attr) + + RecursionDepthError().trigger + except: # noqa + from _pytest._code.code import ExceptionInfo + exc_info = ExceptionInfo() + assert 'maximum recursion' in str(exc_info.getrepr()) + else: + assert 0 diff --git a/testing/code/test_source.py b/testing/code/test_source.py index 007ad1433aad9cd..8eda68a6e2f8954 100644 --- a/testing/code/test_source.py +++ b/testing/code/test_source.py @@ -1,6 +1,7 @@ # flake8: noqa # disable flake check on this file because some constructs are strange # or redundant on purpose and can't be disable on a line-by-line basis +from __future__ import absolute_import, division, print_function import sys import _pytest._code @@ -16,6 +17,7 @@ failsonjython = pytest.mark.xfail("sys.platform.startswith('java')") + def test_source_str_function(): x = Source("3") assert str(x) == "3" @@ -33,6 +35,7 @@ def test_source_str_function(): """, rstrip=True) assert str(x) == "\n3" + def test_unicode(): try: unicode @@ -44,23 +47,27 @@ def test_unicode(): val = eval(co) assert isinstance(val, unicode) + def test_source_from_function(): source = _pytest._code.Source(test_source_str_function) assert str(source).startswith('def test_source_str_function():') + def test_source_from_method(): - class TestClass: + class TestClass(object): def test_method(self): pass source = _pytest._code.Source(TestClass().test_method) assert source.lines == ["def test_method(self):", " pass"] + def test_source_from_lines(): lines = ["a \n", "b\n", "c"] source = _pytest._code.Source(lines) assert source.lines == ['a ', 'b', 'c'] + def test_source_from_inner_function(): def f(): pass @@ -69,6 +76,7 @@ def f(): source = _pytest._code.Source(f) assert str(source).startswith('def f():') + def test_source_putaround_simple(): source = Source("raise ValueError") source = source.putaround( @@ -77,7 +85,7 @@ def test_source_putaround_simple(): x = 42 else: x = 23""") - assert str(source)=="""\ + assert str(source) == """\ try: raise ValueError except ValueError: @@ -85,6 +93,7 @@ def test_source_putaround_simple(): else: x = 23""" + def test_source_putaround(): source = Source() source = source.putaround(""" @@ -93,24 +102,28 @@ def test_source_putaround(): """) assert str(source).strip() == "if 1:\n x=1" + def test_source_strips(): source = Source("") assert source == Source() assert str(source) == '' assert source.strip() == source + def test_source_strip_multiline(): source = Source() source.lines = ["", " hello", " "] source2 = source.strip() assert source2.lines == [" hello"] + def test_syntaxerror_rerepresentation(): ex = pytest.raises(SyntaxError, _pytest._code.compile, 'xyz xyz') assert ex.value.lineno == 1 - assert ex.value.offset in (4,7) # XXX pypy/jython versus cpython? + assert ex.value.offset in (4, 7) # XXX pypy/jython versus cpython? assert ex.value.text.strip(), 'x x' + def test_isparseable(): assert Source("hello").isparseable() assert Source("if 1:\n pass").isparseable() @@ -119,13 +132,15 @@ def test_isparseable(): assert not Source(" \nif 1:\npass").isparseable() assert not Source(chr(0)).isparseable() -class TestAccesses: + +class TestAccesses(object): source = Source("""\ def f(x): pass def g(x): pass """) + def test_getrange(self): x = self.source[0:2] assert x.isparseable() @@ -140,10 +155,11 @@ def test_len(self): assert len(self.source) == 4 def test_iter(self): - l = [x for x in self.source] - assert len(l) == 4 + values = [x for x in self.source] + assert len(values) == 4 + -class TestSourceParsingAndCompiling: +class TestSourceParsingAndCompiling(object): source = Source("""\ def f(x): assert (x == @@ -154,12 +170,12 @@ def f(x): def test_compile(self): co = _pytest._code.compile("x=3") d = {} - exec (co, d) + exec(co, d) assert d['x'] == 3 def test_compile_and_getsource_simple(self): co = _pytest._code.compile("x=3") - exec (co) + exec(co) source = _pytest._code.Source(co) assert str(source) == "x=3" @@ -180,16 +196,16 @@ def f(): assert 'ValueError' in source2 def test_getstatement(self): - #print str(self.source) + # print str(self.source) ass = str(self.source[1:]) for i in range(1, 4): - #print "trying start in line %r" % self.source[i] + # print "trying start in line %r" % self.source[i] s = self.source.getstatement(i) #x = s.deindent() assert str(s) == ass def test_getstatementrange_triple_quoted(self): - #print str(self.source) + # print str(self.source) source = Source("""hello(''' ''')""") s = source.getstatement(0) @@ -210,12 +226,12 @@ def test_getstatementrange_within_constructs(self): """) assert len(source) == 7 # check all lineno's that could occur in a traceback - #assert source.getstatementrange(0) == (0, 7) - #assert source.getstatementrange(1) == (1, 5) + # assert source.getstatementrange(0) == (0, 7) + # assert source.getstatementrange(1) == (1, 5) assert source.getstatementrange(2) == (2, 3) assert source.getstatementrange(3) == (3, 4) assert source.getstatementrange(4) == (4, 5) - #assert source.getstatementrange(5) == (0, 7) + # assert source.getstatementrange(5) == (0, 7) assert source.getstatementrange(6) == (6, 7) def test_getstatementrange_bug(self): @@ -257,17 +273,15 @@ def test_some(): assert getstatement(2, source).lines == source.lines[2:3] assert getstatement(3, source).lines == source.lines[3:4] - @pytest.mark.skipif("sys.version_info < (2,6)") def test_getstatementrange_out_of_bounds_py3(self): source = Source("if xxx:\n from .collections import something") r = source.getstatementrange(1) - assert r == (1,2) + assert r == (1, 2) def test_getstatementrange_with_syntaxerror_issue7(self): source = Source(":") pytest.raises(SyntaxError, lambda: source.getstatementrange(0)) - @pytest.mark.skipif("sys.version_info < (2,6)") def test_compile_to_ast(self): import ast source = Source("x = 4") @@ -282,16 +296,17 @@ def test_compile_and_getsource(self): excinfo = pytest.raises(AssertionError, "f(6)") frame = excinfo.traceback[-1].frame stmt = frame.code.fullsource.getstatement(frame.lineno) - #print "block", str(block) + # print "block", str(block) assert str(stmt).strip().startswith('assert') - def test_compilefuncs_and_path_sanity(self): + @pytest.mark.parametrize('name', ['', None, 'my']) + def test_compilefuncs_and_path_sanity(self, name): def check(comp, name): co = comp(self.source, name) if not name: - expected = "codegen %s:%d>" %(mypath, mylineno+2+1) + expected = "codegen %s:%d>" % (mypath, mylineno + 2 + 2) else: - expected = "codegen %r %s:%d>" % (name, mypath, mylineno+2+1) + expected = "codegen %r %s:%d>" % (name, mypath, mylineno + 2 + 2) fn = co.co_filename assert fn.endswith(expected) @@ -300,36 +315,23 @@ def check(comp, name): mypath = mycode.path for comp in _pytest._code.compile, _pytest._code.Source.compile: - for name in '', None, 'my': - yield check, comp, name + check(comp, name) def test_offsetless_synerr(self): pytest.raises(SyntaxError, _pytest._code.compile, "lambda a,a: 0", mode='eval') + def test_getstartingblock_singleline(): - class A: + class A(object): def __init__(self, *args): frame = sys._getframe(1) self.source = _pytest._code.Frame(frame).statement x = A('x', 'y') - l = [i for i in x.source.lines if i.strip()] - assert len(l) == 1 - -def test_getstartingblock_multiline(): - class A: - def __init__(self, *args): - frame = sys._getframe(1) - self.source = _pytest._code.Frame(frame).statement + values = [i for i in x.source.lines if i.strip()] + assert len(values) == 1 - x = A('x', - 'y' \ - , - 'z') - - l = [i for i in x.source.lines if i.strip()] - assert len(l) == 4 def test_getline_finally(): def c(): pass @@ -344,6 +346,7 @@ def c(): pass source = excinfo.traceback[-1].statement assert str(source).strip() == 'c(1)' + def test_getfuncsource_dynamic(): source = """ def f(): @@ -385,8 +388,7 @@ def g(): lines = deindent(source.splitlines()) assert lines == ['', 'def f():', ' def g():', ' pass', ' '] -@pytest.mark.xfail("sys.version_info[:3] < (2,7,0) or " - "((3,0) <= sys.version_info[:2] < (3,2))") + def test_source_of_class_at_eof_without_newline(tmpdir): # this test fails because the implicit inspect.getsource(A) below # does not return the "x = 1" last line. @@ -400,10 +402,12 @@ def method(self): s2 = _pytest._code.Source(tmpdir.join("a.py").pyimport().A) assert str(source).strip() == str(s2).strip() + if True: def x(): pass + def test_getsource_fallback(): from _pytest._code.source import getsource expected = """def x(): @@ -411,6 +415,7 @@ def test_getsource_fallback(): src = getsource(x) assert src == expected + def test_idem_compile_and_getsource(): from _pytest._code.source import getsource expected = "def x(): pass" @@ -418,12 +423,14 @@ def test_idem_compile_and_getsource(): src = getsource(co) assert src == expected + def test_findsource_fallback(): from _pytest._code.source import findsource src, lineno = findsource(x) assert 'test_findsource_simple' in str(src) assert src[lineno] == ' def x():' + def test_findsource(): from _pytest._code.source import findsource co = _pytest._code.compile("""if 1: @@ -450,7 +457,7 @@ def f(x): fspath, lineno = getfslineno(f) assert fspath.basename == "test_source.py" - assert lineno == _pytest._code.getrawcode(f).co_firstlineno - 1 # see findsource + assert lineno == _pytest._code.getrawcode(f).co_firstlineno - 1 # see findsource class A(object): pass @@ -462,16 +469,19 @@ class A(object): assert lineno == A_lineno assert getfslineno(3) == ("", -1) - class B: + + class B(object): pass B.__name__ = "B2" assert getfslineno(B)[1] == -1 + def test_code_of_object_instance_with_call(): - class A: + class A(object): pass pytest.raises(TypeError, lambda: _pytest._code.Source(A())) - class WithCall: + + class WithCall(object): def __call__(self): pass @@ -490,10 +500,12 @@ def getstatement(lineno, source): ast, start, end = getstatementrange_ast(lineno, source) return source[start:end] + def test_oneline(): source = getstatement(0, "raise ValueError") assert str(source) == "raise ValueError" + def test_comment_and_no_newline_at_end(): from _pytest._code.source import getstatementrange_ast source = Source(['def test_basic_complex():', @@ -502,10 +514,12 @@ def test_comment_and_no_newline_at_end(): ast, start, end = getstatementrange_ast(1, source) assert end == 2 + def test_oneline_and_comment(): source = getstatement(0, "raise ValueError\n#hello") assert str(source) == "raise ValueError" + @pytest.mark.xfail(hasattr(sys, "pypy_version_info"), reason='does not work on pypy') def test_comments(): @@ -521,29 +535,33 @@ def test_comments(): comment 4 """ ''' - for line in range(2,6): + for line in range(2, 6): assert str(getstatement(line, source)) == ' x = 1' - for line in range(6,10): + for line in range(6, 10): assert str(getstatement(line, source)) == ' assert False' assert str(getstatement(10, source)) == '"""' + def test_comment_in_statement(): source = '''test(foo=1, # comment 1 bar=2) ''' - for line in range(1,3): + for line in range(1, 3): assert str(getstatement(line, source)) == \ - 'test(foo=1,\n # comment 1\n bar=2)' + 'test(foo=1,\n # comment 1\n bar=2)' + def test_single_line_else(): source = getstatement(1, "if False: 2\nelse: 3") assert str(source) == "else: 3" + def test_single_line_finally(): source = getstatement(1, "try: 1\nfinally: 3") assert str(source) == "finally: 3" + def test_issue55(): source = ('def round_trip(dinp):\n assert 1 == dinp\n' 'def test_rt():\n round_trip("""\n""")\n') @@ -560,7 +578,8 @@ def XXXtest_multiline(): """) assert str(source) == "raise ValueError(\n 23\n)" -class TestTry: + +class TestTry(object): pytestmark = astonly source = """\ try: @@ -587,7 +606,8 @@ def test_else(self): source = getstatement(5, self.source) assert str(source) == " raise KeyError()" -class TestTryFinally: + +class TestTryFinally(object): source = """\ try: raise ValueError @@ -604,8 +624,7 @@ def test_finally(self): assert str(source) == " raise IndexError(1)" - -class TestIf: +class TestIf(object): pytestmark = astonly source = """\ if 1: @@ -632,6 +651,7 @@ def test_else(self): source = getstatement(5, self.source) assert str(source) == " y = 7" + def test_semicolon(): s = """\ hello ; pytest.skip() @@ -639,6 +659,7 @@ def test_semicolon(): source = getstatement(0, s) assert str(source) == s.strip() + def test_def_online(): s = """\ def func(): raise ValueError(42) @@ -649,6 +670,7 @@ def something(): source = getstatement(0, s) assert str(source) == "def func(): raise ValueError(42)" + def XXX_test_expression_multiline(): source = """\ something @@ -656,4 +678,3 @@ def XXX_test_expression_multiline(): '''""" result = getstatement(1, source) assert str(result) == "'''\n'''" - diff --git a/testing/code/test_source_multiline_block.py b/testing/code/test_source_multiline_block.py new file mode 100644 index 000000000000000..b356d191f0b04b3 --- /dev/null +++ b/testing/code/test_source_multiline_block.py @@ -0,0 +1,26 @@ +# flake8: noqa +import sys + +import _pytest._code + + +def test_getstartingblock_multiline(): + """ + This test was originally found in test_source.py, but it depends on the weird + formatting of the ``x = A`` construct seen here and our autopep8 tool can only exclude entire + files (it does not support excluding lines/blocks using the traditional #noqa comment yet, + see hhatto/autopep8#307). It was considered better to just move this single test to its own + file and exclude it from autopep8 than try to complicate things. + """ + class A(object): + def __init__(self, *args): + frame = sys._getframe(1) + self.source = _pytest._code.Frame(frame).statement + + x = A('x', + 'y' + , + 'z') + + values = [i for i in x.source.lines if i.strip()] + assert len(values) == 4 diff --git a/testing/cx_freeze/install_cx_freeze.py b/testing/cx_freeze/install_cx_freeze.py deleted file mode 100644 index 83dce87aa56d61b..000000000000000 --- a/testing/cx_freeze/install_cx_freeze.py +++ /dev/null @@ -1,64 +0,0 @@ -""" -Installs cx_freeze from source, but first patching -setup.py as described here: - -http://stackoverflow.com/questions/25107697/compiling-cx-freeze-under-ubuntu -""" -import glob -import tarfile -import os -import sys -import platform -import py - -if __name__ == '__main__': - if 'ubuntu' not in platform.version().lower(): - - print('Not Ubuntu, installing using pip. (platform.version() is %r)' % - platform.version()) - res = os.system('pip install cx_freeze') - if res != 0: - sys.exit(res) - sys.exit(0) - - rootdir = py.path.local.make_numbered_dir(prefix='cx_freeze') - - res = os.system('pip install --download %s --no-use-wheel ' - 'cx_freeze' % rootdir) - if res != 0: - sys.exit(res) - - packages = glob.glob('%s/*.tar.gz' % rootdir) - assert len(packages) == 1 - tar_filename = packages[0] - - tar_file = tarfile.open(tar_filename) - try: - tar_file.extractall(path=str(rootdir)) - finally: - tar_file.close() - - basename = os.path.basename(tar_filename).replace('.tar.gz', '') - setup_py_filename = '%s/%s/setup.py' % (rootdir, basename) - with open(setup_py_filename) as f: - lines = f.readlines() - - line_to_patch = 'if not vars.get("Py_ENABLE_SHARED", 0):' - for index, line in enumerate(lines): - if line_to_patch in line: - indent = line[:line.index(line_to_patch)] - lines[index] = indent + 'if True:\n' - print('Patched line %d' % (index + 1)) - break - else: - sys.exit('Could not find line in setup.py to patch!') - - with open(setup_py_filename, 'w') as f: - f.writelines(lines) - - os.chdir('%s/%s' % (rootdir, basename)) - res = os.system('python setup.py install') - if res != 0: - sys.exit(res) - - sys.exit(0) diff --git a/testing/cx_freeze/runtests_setup.py b/testing/cx_freeze/runtests_setup.py deleted file mode 100644 index a2874a655eb489e..000000000000000 --- a/testing/cx_freeze/runtests_setup.py +++ /dev/null @@ -1,15 +0,0 @@ -""" -Sample setup.py script that generates an executable with pytest runner embedded. -""" -if __name__ == '__main__': - from cx_Freeze import setup, Executable - import pytest - - setup( - name="runtests", - version="0.1", - description="exemple of how embedding py.test into an executable using cx_freeze", - executables=[Executable("runtests_script.py")], - options={"build_exe": {'includes': pytest.freeze_includes()}}, - ) - diff --git a/testing/cx_freeze/tox_run.py b/testing/cx_freeze/tox_run.py deleted file mode 100644 index e8df2684bb1b14d..000000000000000 --- a/testing/cx_freeze/tox_run.py +++ /dev/null @@ -1,15 +0,0 @@ -""" -Called by tox.ini: uses the generated executable to run the tests in ./tests/ -directory. - -.. note:: somehow calling "build/runtests_script" directly from tox doesn't - seem to work (at least on Windows). -""" -if __name__ == '__main__': - import os - import sys - - executable = os.path.join(os.getcwd(), 'build', 'runtests_script') - if sys.platform.startswith('win'): - executable += '.exe' - sys.exit(os.system('%s tests' % executable)) \ No newline at end of file diff --git a/testing/deprecated_test.py b/testing/deprecated_test.py new file mode 100644 index 000000000000000..f3c40cb3dbba399 --- /dev/null +++ b/testing/deprecated_test.py @@ -0,0 +1,114 @@ +from __future__ import absolute_import, division, print_function +import pytest + + +def test_yield_tests_deprecation(testdir): + testdir.makepyfile(""" + def func1(arg, arg2): + assert arg == arg2 + def test_gen(): + yield "m1", func1, 15, 3*5 + yield "m2", func1, 42, 6*7 + def test_gen2(): + for k in range(10): + yield func1, 1, 1 + """) + result = testdir.runpytest('-ra') + result.stdout.fnmatch_lines([ + '*yield tests are deprecated, and scheduled to be removed in pytest 4.0*', + '*2 passed*', + ]) + assert result.stdout.str().count('yield tests are deprecated') == 2 + + +def test_funcarg_prefix_deprecation(testdir): + testdir.makepyfile(""" + def pytest_funcarg__value(): + return 10 + + def test_funcarg_prefix(value): + assert value == 10 + """) + result = testdir.runpytest('-ra') + result.stdout.fnmatch_lines([ + ('*pytest_funcarg__value: ' + 'declaring fixtures using "pytest_funcarg__" prefix is deprecated ' + 'and scheduled to be removed in pytest 4.0. ' + 'Please remove the prefix and use the @pytest.fixture decorator instead.'), + '*1 passed*', + ]) + + +def test_pytest_setup_cfg_deprecated(testdir): + testdir.makefile('.cfg', setup=''' + [pytest] + addopts = --verbose + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['*pytest*section in setup.cfg files is deprecated*use*tool:pytest*instead*']) + + +def test_str_args_deprecated(tmpdir, testdir): + """Deprecate passing strings to pytest.main(). Scheduled for removal in pytest-4.0.""" + from _pytest.main import EXIT_NOTESTSCOLLECTED + warnings = [] + + class Collect(object): + def pytest_logwarning(self, message): + warnings.append(message) + + ret = pytest.main("%s -x" % tmpdir, plugins=[Collect()]) + testdir.delete_loaded_modules() + msg = ('passing a string to pytest.main() is deprecated, ' + 'pass a list of arguments instead.') + assert msg in warnings + assert ret == EXIT_NOTESTSCOLLECTED + + +def test_getfuncargvalue_is_deprecated(request): + pytest.deprecated_call(request.getfuncargvalue, 'tmpdir') + + +def test_resultlog_is_deprecated(testdir): + result = testdir.runpytest('--help') + result.stdout.fnmatch_lines(['*DEPRECATED path for machine-readable result log*']) + + testdir.makepyfile(''' + def test(): + pass + ''') + result = testdir.runpytest('--result-log=%s' % testdir.tmpdir.join('result.log')) + result.stdout.fnmatch_lines([ + '*--result-log is deprecated and scheduled for removal in pytest 4.0*', + '*See https://docs.pytest.org/*/usage.html#creating-resultlog-format-files for more information*', + ]) + + +@pytest.mark.filterwarnings('always:Metafunc.addcall is deprecated') +def test_metafunc_addcall_deprecated(testdir): + testdir.makepyfile(""" + def pytest_generate_tests(metafunc): + metafunc.addcall({'i': 1}) + metafunc.addcall({'i': 2}) + def test_func(i): + pass + """) + res = testdir.runpytest('-s') + assert res.ret == 0 + res.stdout.fnmatch_lines([ + "*Metafunc.addcall is deprecated*", + "*2 passed, 2 warnings*", + ]) + + +def test_pytest_catchlog_deprecated(testdir): + testdir.makepyfile(""" + def test_func(pytestconfig): + pytestconfig.pluginmanager.register(None, 'pytest_catchlog') + """) + res = testdir.runpytest() + assert res.ret == 0 + res.stdout.fnmatch_lines([ + "*pytest-catchlog plugin has been merged into the core*", + "*1 passed, 1 warnings*", + ]) diff --git a/testing/freeze/.gitignore b/testing/freeze/.gitignore new file mode 100644 index 000000000000000..490310b6c116cff --- /dev/null +++ b/testing/freeze/.gitignore @@ -0,0 +1,3 @@ +build/ +dist/ +*.spec \ No newline at end of file diff --git a/testing/freeze/create_executable.py b/testing/freeze/create_executable.py new file mode 100644 index 000000000000000..f4f6088ef7d0afe --- /dev/null +++ b/testing/freeze/create_executable.py @@ -0,0 +1,12 @@ +""" +Generates an executable with pytest runner embedded using PyInstaller. +""" +if __name__ == '__main__': + import pytest + import subprocess + + hidden = [] + for x in pytest.freeze_includes(): + hidden.extend(['--hidden-import', x]) + args = ['pyinstaller', '--noconfirm'] + hidden + ['runtests_script.py'] + subprocess.check_call(' '.join(args), shell=True) diff --git a/testing/cx_freeze/runtests_script.py b/testing/freeze/runtests_script.py similarity index 82% rename from testing/cx_freeze/runtests_script.py rename to testing/freeze/runtests_script.py index f2b032d76555c32..d281601c0684939 100644 --- a/testing/cx_freeze/runtests_script.py +++ b/testing/freeze/runtests_script.py @@ -1,9 +1,9 @@ -""" -This is the script that is actually frozen into an executable: simply executes -py.test main(). -""" - -if __name__ == '__main__': - import sys - import pytest - sys.exit(pytest.main()) \ No newline at end of file +""" +This is the script that is actually frozen into an executable: simply executes +py.test main(). +""" + +if __name__ == '__main__': + import sys + import pytest + sys.exit(pytest.main()) diff --git a/testing/cx_freeze/tests/test_doctest.txt b/testing/freeze/tests/test_doctest.txt similarity index 100% rename from testing/cx_freeze/tests/test_doctest.txt rename to testing/freeze/tests/test_doctest.txt diff --git a/testing/cx_freeze/tests/test_trivial.py b/testing/freeze/tests/test_trivial.py similarity index 65% rename from testing/cx_freeze/tests/test_trivial.py rename to testing/freeze/tests/test_trivial.py index d8a572baaed1159..45622b850bb5c13 100644 --- a/testing/cx_freeze/tests/test_trivial.py +++ b/testing/freeze/tests/test_trivial.py @@ -1,6 +1,7 @@ - -def test_upper(): - assert 'foo'.upper() == 'FOO' - -def test_lower(): - assert 'FOO'.lower() == 'foo' \ No newline at end of file + +def test_upper(): + assert 'foo'.upper() == 'FOO' + + +def test_lower(): + assert 'FOO'.lower() == 'foo' diff --git a/testing/freeze/tox_run.py b/testing/freeze/tox_run.py new file mode 100644 index 000000000000000..3fc388040957b62 --- /dev/null +++ b/testing/freeze/tox_run.py @@ -0,0 +1,12 @@ +""" +Called by tox.ini: uses the generated executable to run the tests in ./tests/ +directory. +""" +if __name__ == '__main__': + import os + import sys + + executable = os.path.join(os.getcwd(), 'dist', 'runtests_script', 'runtests_script') + if sys.platform.startswith('win'): + executable += '.exe' + sys.exit(os.system('%s tests' % executable)) diff --git a/testing/logging/test_fixture.py b/testing/logging/test_fixture.py new file mode 100644 index 000000000000000..c27b31137ffcaef --- /dev/null +++ b/testing/logging/test_fixture.py @@ -0,0 +1,70 @@ +# -*- coding: utf-8 -*- +import logging + + +logger = logging.getLogger(__name__) +sublogger = logging.getLogger(__name__ + '.baz') + + +def test_fixture_help(testdir): + result = testdir.runpytest('--fixtures') + result.stdout.fnmatch_lines(['*caplog*']) + + +def test_change_level(caplog): + caplog.set_level(logging.INFO) + logger.debug('handler DEBUG level') + logger.info('handler INFO level') + + caplog.set_level(logging.CRITICAL, logger=sublogger.name) + sublogger.warning('logger WARNING level') + sublogger.critical('logger CRITICAL level') + + assert 'DEBUG' not in caplog.text + assert 'INFO' in caplog.text + assert 'WARNING' not in caplog.text + assert 'CRITICAL' in caplog.text + + +def test_with_statement(caplog): + with caplog.at_level(logging.INFO): + logger.debug('handler DEBUG level') + logger.info('handler INFO level') + + with caplog.at_level(logging.CRITICAL, logger=sublogger.name): + sublogger.warning('logger WARNING level') + sublogger.critical('logger CRITICAL level') + + assert 'DEBUG' not in caplog.text + assert 'INFO' in caplog.text + assert 'WARNING' not in caplog.text + assert 'CRITICAL' in caplog.text + + +def test_log_access(caplog): + logger.info('boo %s', 'arg') + assert caplog.records[0].levelname == 'INFO' + assert caplog.records[0].msg == 'boo %s' + assert 'boo arg' in caplog.text + + +def test_record_tuples(caplog): + logger.info('boo %s', 'arg') + + assert caplog.record_tuples == [ + (__name__, logging.INFO, 'boo arg'), + ] + + +def test_unicode(caplog): + logger.info(u'bū') + assert caplog.records[0].levelname == 'INFO' + assert caplog.records[0].msg == u'bū' + assert u'bū' in caplog.text + + +def test_clear(caplog): + logger.info(u'bū') + assert len(caplog.records) + caplog.clear() + assert not len(caplog.records) diff --git a/testing/logging/test_reporting.py b/testing/logging/test_reporting.py new file mode 100644 index 000000000000000..c02ee21722793c4 --- /dev/null +++ b/testing/logging/test_reporting.py @@ -0,0 +1,398 @@ +# -*- coding: utf-8 -*- +import os +import pytest + + +def test_nothing_logged(testdir): + testdir.makepyfile(''' + import sys + + def test_foo(): + sys.stdout.write('text going to stdout') + sys.stderr.write('text going to stderr') + assert False + ''') + result = testdir.runpytest() + assert result.ret == 1 + result.stdout.fnmatch_lines(['*- Captured stdout call -*', + 'text going to stdout']) + result.stdout.fnmatch_lines(['*- Captured stderr call -*', + 'text going to stderr']) + with pytest.raises(pytest.fail.Exception): + result.stdout.fnmatch_lines(['*- Captured *log call -*']) + + +def test_messages_logged(testdir): + testdir.makepyfile(''' + import sys + import logging + + logger = logging.getLogger(__name__) + + def test_foo(): + sys.stdout.write('text going to stdout') + sys.stderr.write('text going to stderr') + logger.info('text going to logger') + assert False + ''') + result = testdir.runpytest() + assert result.ret == 1 + result.stdout.fnmatch_lines(['*- Captured *log call -*', + '*text going to logger*']) + result.stdout.fnmatch_lines(['*- Captured stdout call -*', + 'text going to stdout']) + result.stdout.fnmatch_lines(['*- Captured stderr call -*', + 'text going to stderr']) + + +def test_setup_logging(testdir): + testdir.makepyfile(''' + import logging + + logger = logging.getLogger(__name__) + + def setup_function(function): + logger.info('text going to logger from setup') + + def test_foo(): + logger.info('text going to logger from call') + assert False + ''') + result = testdir.runpytest() + assert result.ret == 1 + result.stdout.fnmatch_lines(['*- Captured *log setup -*', + '*text going to logger from setup*', + '*- Captured *log call -*', + '*text going to logger from call*']) + + +def test_teardown_logging(testdir): + testdir.makepyfile(''' + import logging + + logger = logging.getLogger(__name__) + + def test_foo(): + logger.info('text going to logger from call') + + def teardown_function(function): + logger.info('text going to logger from teardown') + assert False + ''') + result = testdir.runpytest() + assert result.ret == 1 + result.stdout.fnmatch_lines(['*- Captured *log call -*', + '*text going to logger from call*', + '*- Captured *log teardown -*', + '*text going to logger from teardown*']) + + +def test_disable_log_capturing(testdir): + testdir.makepyfile(''' + import sys + import logging + + logger = logging.getLogger(__name__) + + def test_foo(): + sys.stdout.write('text going to stdout') + logger.warning('catch me if you can!') + sys.stderr.write('text going to stderr') + assert False + ''') + result = testdir.runpytest('--no-print-logs') + print(result.stdout) + assert result.ret == 1 + result.stdout.fnmatch_lines(['*- Captured stdout call -*', + 'text going to stdout']) + result.stdout.fnmatch_lines(['*- Captured stderr call -*', + 'text going to stderr']) + with pytest.raises(pytest.fail.Exception): + result.stdout.fnmatch_lines(['*- Captured *log call -*']) + + +def test_disable_log_capturing_ini(testdir): + testdir.makeini( + ''' + [pytest] + log_print=False + ''' + ) + testdir.makepyfile(''' + import sys + import logging + + logger = logging.getLogger(__name__) + + def test_foo(): + sys.stdout.write('text going to stdout') + logger.warning('catch me if you can!') + sys.stderr.write('text going to stderr') + assert False + ''') + result = testdir.runpytest() + print(result.stdout) + assert result.ret == 1 + result.stdout.fnmatch_lines(['*- Captured stdout call -*', + 'text going to stdout']) + result.stdout.fnmatch_lines(['*- Captured stderr call -*', + 'text going to stderr']) + with pytest.raises(pytest.fail.Exception): + result.stdout.fnmatch_lines(['*- Captured *log call -*']) + + +def test_log_cli_default_level(testdir): + # Default log file level + testdir.makepyfile(''' + import pytest + import logging + def test_log_cli(request): + plugin = request.config.pluginmanager.getplugin('logging-plugin') + assert plugin.log_cli_handler.level == logging.WARNING + logging.getLogger('catchlog').info("This log message won't be shown") + logging.getLogger('catchlog').warning("This log message will be shown") + print('PASSED') + ''') + + result = testdir.runpytest('-s') + + # fnmatch_lines does an assertion internally + result.stdout.fnmatch_lines([ + 'test_log_cli_default_level.py PASSED', + ]) + result.stderr.fnmatch_lines([ + "* This log message will be shown" + ]) + for line in result.errlines: + try: + assert "This log message won't be shown" in line + pytest.fail("A log message was shown and it shouldn't have been") + except AssertionError: + continue + + # make sure that that we get a '0' exit code for the testsuite + assert result.ret == 0 + + +def test_log_cli_level(testdir): + # Default log file level + testdir.makepyfile(''' + import pytest + import logging + def test_log_cli(request): + plugin = request.config.pluginmanager.getplugin('logging-plugin') + assert plugin.log_cli_handler.level == logging.INFO + logging.getLogger('catchlog').debug("This log message won't be shown") + logging.getLogger('catchlog').info("This log message will be shown") + print('PASSED') + ''') + + result = testdir.runpytest('-s', '--log-cli-level=INFO') + + # fnmatch_lines does an assertion internally + result.stdout.fnmatch_lines([ + 'test_log_cli_level.py PASSED', + ]) + result.stderr.fnmatch_lines([ + "* This log message will be shown" + ]) + for line in result.errlines: + try: + assert "This log message won't be shown" in line + pytest.fail("A log message was shown and it shouldn't have been") + except AssertionError: + continue + + # make sure that that we get a '0' exit code for the testsuite + assert result.ret == 0 + + result = testdir.runpytest('-s', '--log-level=INFO') + + # fnmatch_lines does an assertion internally + result.stdout.fnmatch_lines([ + 'test_log_cli_level.py PASSED', + ]) + result.stderr.fnmatch_lines([ + "* This log message will be shown" + ]) + for line in result.errlines: + try: + assert "This log message won't be shown" in line + pytest.fail("A log message was shown and it shouldn't have been") + except AssertionError: + continue + + # make sure that that we get a '0' exit code for the testsuite + assert result.ret == 0 + + +def test_log_cli_ini_level(testdir): + testdir.makeini( + """ + [pytest] + log_cli_level = INFO + """) + testdir.makepyfile(''' + import pytest + import logging + def test_log_cli(request): + plugin = request.config.pluginmanager.getplugin('logging-plugin') + assert plugin.log_cli_handler.level == logging.INFO + logging.getLogger('catchlog').debug("This log message won't be shown") + logging.getLogger('catchlog').info("This log message will be shown") + print('PASSED') + ''') + + result = testdir.runpytest('-s') + + # fnmatch_lines does an assertion internally + result.stdout.fnmatch_lines([ + 'test_log_cli_ini_level.py PASSED', + ]) + result.stderr.fnmatch_lines([ + "* This log message will be shown" + ]) + for line in result.errlines: + try: + assert "This log message won't be shown" in line + pytest.fail("A log message was shown and it shouldn't have been") + except AssertionError: + continue + + # make sure that that we get a '0' exit code for the testsuite + assert result.ret == 0 + + +def test_log_file_cli(testdir): + # Default log file level + testdir.makepyfile(''' + import pytest + import logging + def test_log_file(request): + plugin = request.config.pluginmanager.getplugin('logging-plugin') + assert plugin.log_file_handler.level == logging.WARNING + logging.getLogger('catchlog').info("This log message won't be shown") + logging.getLogger('catchlog').warning("This log message will be shown") + print('PASSED') + ''') + + log_file = testdir.tmpdir.join('pytest.log').strpath + + result = testdir.runpytest('-s', '--log-file={0}'.format(log_file)) + + # fnmatch_lines does an assertion internally + result.stdout.fnmatch_lines([ + 'test_log_file_cli.py PASSED', + ]) + + # make sure that that we get a '0' exit code for the testsuite + assert result.ret == 0 + assert os.path.isfile(log_file) + with open(log_file) as rfh: + contents = rfh.read() + assert "This log message will be shown" in contents + assert "This log message won't be shown" not in contents + + +def test_log_file_cli_level(testdir): + # Default log file level + testdir.makepyfile(''' + import pytest + import logging + def test_log_file(request): + plugin = request.config.pluginmanager.getplugin('logging-plugin') + assert plugin.log_file_handler.level == logging.INFO + logging.getLogger('catchlog').debug("This log message won't be shown") + logging.getLogger('catchlog').info("This log message will be shown") + print('PASSED') + ''') + + log_file = testdir.tmpdir.join('pytest.log').strpath + + result = testdir.runpytest('-s', + '--log-file={0}'.format(log_file), + '--log-file-level=INFO') + + # fnmatch_lines does an assertion internally + result.stdout.fnmatch_lines([ + 'test_log_file_cli_level.py PASSED', + ]) + + # make sure that that we get a '0' exit code for the testsuite + assert result.ret == 0 + assert os.path.isfile(log_file) + with open(log_file) as rfh: + contents = rfh.read() + assert "This log message will be shown" in contents + assert "This log message won't be shown" not in contents + + +def test_log_file_ini(testdir): + log_file = testdir.tmpdir.join('pytest.log').strpath + + testdir.makeini( + """ + [pytest] + log_file={0} + """.format(log_file)) + testdir.makepyfile(''' + import pytest + import logging + def test_log_file(request): + plugin = request.config.pluginmanager.getplugin('logging-plugin') + assert plugin.log_file_handler.level == logging.WARNING + logging.getLogger('catchlog').info("This log message won't be shown") + logging.getLogger('catchlog').warning("This log message will be shown") + print('PASSED') + ''') + + result = testdir.runpytest('-s') + + # fnmatch_lines does an assertion internally + result.stdout.fnmatch_lines([ + 'test_log_file_ini.py PASSED', + ]) + + # make sure that that we get a '0' exit code for the testsuite + assert result.ret == 0 + assert os.path.isfile(log_file) + with open(log_file) as rfh: + contents = rfh.read() + assert "This log message will be shown" in contents + assert "This log message won't be shown" not in contents + + +def test_log_file_ini_level(testdir): + log_file = testdir.tmpdir.join('pytest.log').strpath + + testdir.makeini( + """ + [pytest] + log_file={0} + log_file_level = INFO + """.format(log_file)) + testdir.makepyfile(''' + import pytest + import logging + def test_log_file(request): + plugin = request.config.pluginmanager.getplugin('logging-plugin') + assert plugin.log_file_handler.level == logging.INFO + logging.getLogger('catchlog').debug("This log message won't be shown") + logging.getLogger('catchlog').info("This log message will be shown") + print('PASSED') + ''') + + result = testdir.runpytest('-s') + + # fnmatch_lines does an assertion internally + result.stdout.fnmatch_lines([ + 'test_log_file_ini_level.py PASSED', + ]) + + # make sure that that we get a '0' exit code for the testsuite + assert result.ret == 0 + assert os.path.isfile(log_file) + with open(log_file) as rfh: + contents = rfh.read() + assert "This log message will be shown" in contents + assert "This log message won't be shown" not in contents diff --git a/testing/python/approx.py b/testing/python/approx.py new file mode 100644 index 000000000000000..300e1ce86f783fa --- /dev/null +++ b/testing/python/approx.py @@ -0,0 +1,392 @@ +# encoding: utf-8 +import operator +import sys +import pytest +import doctest + +from pytest import approx +from operator import eq, ne +from decimal import Decimal +from fractions import Fraction +inf, nan = float('inf'), float('nan') + + +class MyDocTestRunner(doctest.DocTestRunner): + + def __init__(self): + doctest.DocTestRunner.__init__(self) + + def report_failure(self, out, test, example, got): + raise AssertionError("'{}' evaluates to '{}', not '{}'".format( + example.source.strip(), got.strip(), example.want.strip())) + + +class TestApprox(object): + + def test_repr_string(self): + plus_minus = u'\u00b1' if sys.version_info[0] > 2 else u'+-' + tol1, tol2, infr = '1.0e-06', '2.0e-06', 'inf' + assert repr(approx(1.0)) == '1.0 {pm} {tol1}'.format(pm=plus_minus, tol1=tol1) + assert repr(approx([1.0, 2.0])) == 'approx([1.0 {pm} {tol1}, 2.0 {pm} {tol2}])'.format( + pm=plus_minus, tol1=tol1, tol2=tol2) + assert repr(approx((1.0, 2.0))) == 'approx((1.0 {pm} {tol1}, 2.0 {pm} {tol2}))'.format( + pm=plus_minus, tol1=tol1, tol2=tol2) + assert repr(approx(inf)) == 'inf' + assert repr(approx(1.0, rel=nan)) == '1.0 {pm} ???'.format(pm=plus_minus) + assert repr(approx(1.0, rel=inf)) == '1.0 {pm} {infr}'.format(pm=plus_minus, infr=infr) + assert repr(approx(1.0j, rel=inf)) == '1j' + + # Dictionaries aren't ordered, so we need to check both orders. + assert repr(approx({'a': 1.0, 'b': 2.0})) in ( + "approx({{'a': 1.0 {pm} {tol1}, 'b': 2.0 {pm} {tol2}}})".format(pm=plus_minus, tol1=tol1, tol2=tol2), + "approx({{'b': 2.0 {pm} {tol2}, 'a': 1.0 {pm} {tol1}}})".format(pm=plus_minus, tol1=tol1, tol2=tol2), + ) + + def test_operator_overloading(self): + assert 1 == approx(1, rel=1e-6, abs=1e-12) + assert not (1 != approx(1, rel=1e-6, abs=1e-12)) + assert 10 != approx(1, rel=1e-6, abs=1e-12) + assert not (10 == approx(1, rel=1e-6, abs=1e-12)) + + def test_exactly_equal(self): + examples = [ + (2.0, 2.0), + (0.1e200, 0.1e200), + (1.123e-300, 1.123e-300), + (12345, 12345.0), + (0.0, -0.0), + (345678, 345678), + (Decimal('1.0001'), Decimal('1.0001')), + (Fraction(1, 3), Fraction(-1, -3)), + ] + for a, x in examples: + assert a == approx(x) + + def test_opposite_sign(self): + examples = [ + (eq, 1e-100, -1e-100), + (ne, 1e100, -1e100), + ] + for op, a, x in examples: + assert op(a, approx(x)) + + def test_zero_tolerance(self): + within_1e10 = [ + (1.1e-100, 1e-100), + (-1.1e-100, -1e-100), + ] + for a, x in within_1e10: + assert x == approx(x, rel=0.0, abs=0.0) + assert a != approx(x, rel=0.0, abs=0.0) + assert a == approx(x, rel=0.0, abs=5e-101) + assert a != approx(x, rel=0.0, abs=5e-102) + assert a == approx(x, rel=5e-1, abs=0.0) + assert a != approx(x, rel=5e-2, abs=0.0) + + def test_negative_tolerance(self): + # Negative tolerances are not allowed. + illegal_kwargs = [ + dict(rel=-1e100), + dict(abs=-1e100), + dict(rel=1e100, abs=-1e100), + dict(rel=-1e100, abs=1e100), + dict(rel=-1e100, abs=-1e100), + ] + for kwargs in illegal_kwargs: + with pytest.raises(ValueError): + 1.1 == approx(1, **kwargs) + + def test_inf_tolerance(self): + # Everything should be equal if the tolerance is infinite. + large_diffs = [ + (1, 1000), + (1e-50, 1e50), + (-1.0, -1e300), + (0.0, 10), + ] + for a, x in large_diffs: + assert a != approx(x, rel=0.0, abs=0.0) + assert a == approx(x, rel=inf, abs=0.0) + assert a == approx(x, rel=0.0, abs=inf) + assert a == approx(x, rel=inf, abs=inf) + + def test_inf_tolerance_expecting_zero(self): + # If the relative tolerance is zero but the expected value is infinite, + # the actual tolerance is a NaN, which should be an error. + illegal_kwargs = [ + dict(rel=inf, abs=0.0), + dict(rel=inf, abs=inf), + ] + for kwargs in illegal_kwargs: + with pytest.raises(ValueError): + 1 == approx(0, **kwargs) + + def test_nan_tolerance(self): + illegal_kwargs = [ + dict(rel=nan), + dict(abs=nan), + dict(rel=nan, abs=nan), + ] + for kwargs in illegal_kwargs: + with pytest.raises(ValueError): + 1.1 == approx(1, **kwargs) + + def test_reasonable_defaults(self): + # Whatever the defaults are, they should work for numbers close to 1 + # than have a small amount of floating-point error. + assert 0.1 + 0.2 == approx(0.3) + + def test_default_tolerances(self): + # This tests the defaults as they are currently set. If you change the + # defaults, this test will fail but you should feel free to change it. + # None of the other tests (except the doctests) should be affected by + # the choice of defaults. + examples = [ + # Relative tolerance used. + (eq, 1e100 + 1e94, 1e100), + (ne, 1e100 + 2e94, 1e100), + (eq, 1e0 + 1e-6, 1e0), + (ne, 1e0 + 2e-6, 1e0), + # Absolute tolerance used. + (eq, 1e-100, + 1e-106), + (eq, 1e-100, + 2e-106), + (eq, 1e-100, 0), + ] + for op, a, x in examples: + assert op(a, approx(x)) + + def test_custom_tolerances(self): + assert 1e8 + 1e0 == approx(1e8, rel=5e-8, abs=5e0) + assert 1e8 + 1e0 == approx(1e8, rel=5e-9, abs=5e0) + assert 1e8 + 1e0 == approx(1e8, rel=5e-8, abs=5e-1) + assert 1e8 + 1e0 != approx(1e8, rel=5e-9, abs=5e-1) + + assert 1e0 + 1e-8 == approx(1e0, rel=5e-8, abs=5e-8) + assert 1e0 + 1e-8 == approx(1e0, rel=5e-9, abs=5e-8) + assert 1e0 + 1e-8 == approx(1e0, rel=5e-8, abs=5e-9) + assert 1e0 + 1e-8 != approx(1e0, rel=5e-9, abs=5e-9) + + assert 1e-8 + 1e-16 == approx(1e-8, rel=5e-8, abs=5e-16) + assert 1e-8 + 1e-16 == approx(1e-8, rel=5e-9, abs=5e-16) + assert 1e-8 + 1e-16 == approx(1e-8, rel=5e-8, abs=5e-17) + assert 1e-8 + 1e-16 != approx(1e-8, rel=5e-9, abs=5e-17) + + def test_relative_tolerance(self): + within_1e8_rel = [ + (1e8 + 1e0, 1e8), + (1e0 + 1e-8, 1e0), + (1e-8 + 1e-16, 1e-8), + ] + for a, x in within_1e8_rel: + assert a == approx(x, rel=5e-8, abs=0.0) + assert a != approx(x, rel=5e-9, abs=0.0) + + def test_absolute_tolerance(self): + within_1e8_abs = [ + (1e8 + 9e-9, 1e8), + (1e0 + 9e-9, 1e0), + (1e-8 + 9e-9, 1e-8), + ] + for a, x in within_1e8_abs: + assert a == approx(x, rel=0, abs=5e-8) + assert a != approx(x, rel=0, abs=5e-9) + + def test_expecting_zero(self): + examples = [ + (ne, 1e-6, 0.0), + (ne, -1e-6, 0.0), + (eq, 1e-12, 0.0), + (eq, -1e-12, 0.0), + (ne, 2e-12, 0.0), + (ne, -2e-12, 0.0), + (ne, inf, 0.0), + (ne, nan, 0.0), + ] + for op, a, x in examples: + assert op(a, approx(x, rel=0.0, abs=1e-12)) + assert op(a, approx(x, rel=1e-6, abs=1e-12)) + + def test_expecting_inf(self): + examples = [ + (eq, inf, inf), + (eq, -inf, -inf), + (ne, inf, -inf), + (ne, 0.0, inf), + (ne, nan, inf), + ] + for op, a, x in examples: + assert op(a, approx(x)) + + def test_expecting_nan(self): + examples = [ + (eq, nan, nan), + (eq, -nan, -nan), + (eq, nan, -nan), + (ne, 0.0, nan), + (ne, inf, nan), + ] + for op, a, x in examples: + # Nothing is equal to NaN by default. + assert a != approx(x) + + # If ``nan_ok=True``, then NaN is equal to NaN. + assert op(a, approx(x, nan_ok=True)) + + def test_int(self): + within_1e6 = [ + (1000001, 1000000), + (-1000001, -1000000), + ] + for a, x in within_1e6: + assert a == approx(x, rel=5e-6, abs=0) + assert a != approx(x, rel=5e-7, abs=0) + assert approx(x, rel=5e-6, abs=0) == a + assert approx(x, rel=5e-7, abs=0) != a + + def test_decimal(self): + within_1e6 = [ + (Decimal('1.000001'), Decimal('1.0')), + (Decimal('-1.000001'), Decimal('-1.0')), + ] + for a, x in within_1e6: + assert a == approx(x, rel=Decimal('5e-6'), abs=0) + assert a != approx(x, rel=Decimal('5e-7'), abs=0) + assert approx(x, rel=Decimal('5e-6'), abs=0) == a + assert approx(x, rel=Decimal('5e-7'), abs=0) != a + + def test_fraction(self): + within_1e6 = [ + (1 + Fraction(1, 1000000), Fraction(1)), + (-1 - Fraction(-1, 1000000), Fraction(-1)), + ] + for a, x in within_1e6: + assert a == approx(x, rel=5e-6, abs=0) + assert a != approx(x, rel=5e-7, abs=0) + assert approx(x, rel=5e-6, abs=0) == a + assert approx(x, rel=5e-7, abs=0) != a + + def test_complex(self): + within_1e6 = [ + (1.000001 + 1.0j, 1.0 + 1.0j), + (1.0 + 1.000001j, 1.0 + 1.0j), + (-1.000001 + 1.0j, -1.0 + 1.0j), + (1.0 - 1.000001j, 1.0 - 1.0j), + ] + for a, x in within_1e6: + assert a == approx(x, rel=5e-6, abs=0) + assert a != approx(x, rel=5e-7, abs=0) + assert approx(x, rel=5e-6, abs=0) == a + assert approx(x, rel=5e-7, abs=0) != a + + def test_list(self): + actual = [1 + 1e-7, 2 + 1e-8] + expected = [1, 2] + + # Return false if any element is outside the tolerance. + assert actual == approx(expected, rel=5e-7, abs=0) + assert actual != approx(expected, rel=5e-8, abs=0) + assert approx(expected, rel=5e-7, abs=0) == actual + assert approx(expected, rel=5e-8, abs=0) != actual + + def test_list_wrong_len(self): + assert [1, 2] != approx([1]) + assert [1, 2] != approx([1, 2, 3]) + + def test_tuple(self): + actual = (1 + 1e-7, 2 + 1e-8) + expected = (1, 2) + + # Return false if any element is outside the tolerance. + assert actual == approx(expected, rel=5e-7, abs=0) + assert actual != approx(expected, rel=5e-8, abs=0) + assert approx(expected, rel=5e-7, abs=0) == actual + assert approx(expected, rel=5e-8, abs=0) != actual + + def test_tuple_wrong_len(self): + assert (1, 2) != approx((1,)) + assert (1, 2) != approx((1, 2, 3)) + + def test_dict(self): + actual = {'a': 1 + 1e-7, 'b': 2 + 1e-8} + # Dictionaries became ordered in python3.6, so switch up the order here + # to make sure it doesn't matter. + expected = {'b': 2, 'a': 1} + + # Return false if any element is outside the tolerance. + assert actual == approx(expected, rel=5e-7, abs=0) + assert actual != approx(expected, rel=5e-8, abs=0) + assert approx(expected, rel=5e-7, abs=0) == actual + assert approx(expected, rel=5e-8, abs=0) != actual + + def test_dict_wrong_len(self): + assert {'a': 1, 'b': 2} != approx({'a': 1}) + assert {'a': 1, 'b': 2} != approx({'a': 1, 'c': 2}) + assert {'a': 1, 'b': 2} != approx({'a': 1, 'b': 2, 'c': 3}) + + def test_numpy_array(self): + np = pytest.importorskip('numpy') + + actual = np.array([1 + 1e-7, 2 + 1e-8]) + expected = np.array([1, 2]) + + # Return false if any element is outside the tolerance. + assert actual == approx(expected, rel=5e-7, abs=0) + assert actual != approx(expected, rel=5e-8, abs=0) + assert approx(expected, rel=5e-7, abs=0) == expected + assert approx(expected, rel=5e-8, abs=0) != actual + + # Should be able to compare lists with numpy arrays. + assert list(actual) == approx(expected, rel=5e-7, abs=0) + assert list(actual) != approx(expected, rel=5e-8, abs=0) + assert actual == approx(list(expected), rel=5e-7, abs=0) + assert actual != approx(list(expected), rel=5e-8, abs=0) + + def test_numpy_array_wrong_shape(self): + np = pytest.importorskip('numpy') + + a12 = np.array([[1, 2]]) + a21 = np.array([[1], [2]]) + + assert a12 != approx(a21) + assert a21 != approx(a12) + + def test_doctests(self): + parser = doctest.DocTestParser() + test = parser.get_doctest( + approx.__doc__, + {'approx': approx}, + approx.__name__, + None, None, + ) + runner = MyDocTestRunner() + runner.run(test) + + def test_unicode_plus_minus(self, testdir): + """ + Comparing approx instances inside lists should not produce an error in the detailed diff. + Integration test for issue #2111. + """ + testdir.makepyfile(""" + import pytest + def test_foo(): + assert [3] == [pytest.approx(4)] + """) + expected = '4.0e-06' + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*At index 0 diff: 3 != 4 * {0}'.format(expected), + '=* 1 failed in *=', + ]) + + @pytest.mark.parametrize('op', [ + pytest.param(operator.le, id='<='), + pytest.param(operator.lt, id='<'), + pytest.param(operator.ge, id='>='), + pytest.param(operator.gt, id='>'), + ]) + def test_comparison_operator_type_error(self, op): + """ + pytest.approx should raise TypeError for operators other than == and != (#2003). + """ + with pytest.raises(TypeError): + op(1, approx(1, rel=1e-6, abs=1e-12)) diff --git a/testing/python/collect.py b/testing/python/collect.py index 22433da7716157d..16c2154b8c1e98d 100644 --- a/testing/python/collect.py +++ b/testing/python/collect.py @@ -1,18 +1,24 @@ # -*- coding: utf-8 -*- +import os import sys from textwrap import dedent import _pytest._code import py import pytest -from _pytest.main import EXIT_NOTESTSCOLLECTED +from _pytest.main import ( + Collector, + EXIT_NOTESTSCOLLECTED +) -class TestModule: +ignore_parametrized_marks = pytest.mark.filterwarnings('ignore:Applying marks directly to parameters') + + +class TestModule(object): def test_failing_import(self, testdir): modcol = testdir.getmodulecol("import alksdjalskdjalkjals") - pytest.raises(ImportError, modcol.collect) - pytest.raises(ImportError, modcol.collect) + pytest.raises(Collector.CollectError, modcol.collect) def test_import_duplicate(self, testdir): a = testdir.mkdir("a") @@ -60,17 +66,75 @@ def test_module_considers_pluginmanager_at_import(self, testdir): modcol = testdir.getmodulecol("pytest_plugins='xasdlkj',") pytest.raises(ImportError, lambda: modcol.obj) -class TestClass: + def test_invalid_test_module_name(self, testdir): + a = testdir.mkdir('a') + a.ensure('test_one.part1.py') + result = testdir.runpytest("-rw") + result.stdout.fnmatch_lines([ + "ImportError while importing test module*test_one.part1*", + "Hint: make sure your test modules/packages have valid Python names.", + ]) + + @pytest.mark.parametrize('verbose', [0, 1, 2]) + def test_show_traceback_import_error(self, testdir, verbose): + """Import errors when collecting modules should display the traceback (#1976). + + With low verbosity we omit pytest and internal modules, otherwise show all traceback entries. + """ + testdir.makepyfile( + foo_traceback_import_error=""" + from bar_traceback_import_error import NOT_AVAILABLE + """, + bar_traceback_import_error="", + ) + testdir.makepyfile(""" + import foo_traceback_import_error + """) + args = ('-v',) * verbose + result = testdir.runpytest(*args) + result.stdout.fnmatch_lines([ + "ImportError while importing test module*", + "Traceback:", + "*from bar_traceback_import_error import NOT_AVAILABLE", + "*cannot import name *NOT_AVAILABLE*", + ]) + assert result.ret == 2 + + stdout = result.stdout.str() + for name in ('_pytest', os.path.join('py', '_path')): + if verbose == 2: + assert name in stdout + else: + assert name not in stdout + + def test_show_traceback_import_error_unicode(self, testdir): + """Check test modules collected which raise ImportError with unicode messages + are handled properly (#2336). + """ + testdir.makepyfile(u""" + # -*- coding: utf-8 -*- + raise ImportError(u'Something bad happened ☺') + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + "ImportError while importing test module*", + "Traceback:", + "*raise ImportError*Something bad happened*", + ]) + assert result.ret == 2 + + +class TestClass(object): def test_class_with_init_warning(self, testdir): testdir.makepyfile(""" - class TestClass1: + class TestClass1(object): def __init__(self): pass """) result = testdir.runpytest("-rw") - result.stdout.fnmatch_lines_random(""" - WC1*test_class_with_init_warning.py*__init__* - """) + result.stdout.fnmatch_lines([ + "*cannot collect test class 'TestClass1' because it has a __init__ constructor", + ]) def test_class_subclassobject(self, testdir): testdir.getmodulecol(""" @@ -82,9 +146,32 @@ class test(object): "*collected 0*", ]) + def test_static_method(self, testdir): + """Support for collecting staticmethod tests (#2528, #2699)""" + testdir.getmodulecol(""" + import pytest + class Test(object): + @staticmethod + def test_something(): + pass + + @pytest.fixture + def fix(self): + return 1 + + @staticmethod + def test_fix(fix): + assert fix == 1 + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + "*collected 2 items*", + "*2 passed in*", + ]) + def test_setup_teardown_class_as_classmethod(self, testdir): testdir.makepyfile(test_mod1=""" - class TestClassMethod: + class TestClassMethod(object): @classmethod def setup_class(cls): pass @@ -109,8 +196,30 @@ def __getattr__(self, name): colitems = modcol.collect() assert len(colitems) == 0 + def test_issue1579_namedtuple(self, testdir): + testdir.makepyfile(""" + import collections + + TestCase = collections.namedtuple('TestCase', ['a']) + """) + result = testdir.runpytest('-rw') + result.stdout.fnmatch_lines( + "*cannot collect test class 'TestCase' " + "because it has a __new__ constructor*" + ) -class TestGenerator: + def test_issue2234_property(self, testdir): + testdir.makepyfile(""" + class TestCase(object): + @property + def prop(self): + raise NotImplementedError() + """) + result = testdir.runpytest() + assert result.ret == EXIT_NOTESTSCOLLECTED + + +class TestGenerator(object): def test_generative_functions(self, testdir): modcol = testdir.getmodulecol(""" def func1(arg, arg2): @@ -135,7 +244,7 @@ def test_generative_methods(self, testdir): modcol = testdir.getmodulecol(""" def func1(arg, arg2): assert arg == arg2 - class TestGenMethods: + class TestGenMethods(object): def test_gen(self): yield func1, 17, 3*5 yield func1, 42, 6*7 @@ -189,7 +298,7 @@ def test_generative_methods_with_explicit_names(self, testdir): modcol = testdir.getmodulecol(""" def func1(arg, arg2): assert arg == arg2 - class TestGenMethods: + class TestGenMethods(object): def test_gen(self): yield "m1", func1, 17, 3*5 yield "m2", func1, 42, 6*7 @@ -207,6 +316,7 @@ def test_gen(self): def test_order_of_execution_generator_same_codeline(self, testdir, tmpdir): o = testdir.makepyfile(""" + from __future__ import print_function def test_generative_order_of_execution(): import py, pytest test_list = [] @@ -216,8 +326,8 @@ def list_append(item): test_list.append(item) def assert_order_of_execution(): - py.builtin.print_('expected order', expected_list) - py.builtin.print_('but got ', test_list) + print('expected order', expected_list) + print('but got ', test_list) assert test_list == expected_list for i in expected_list: @@ -231,6 +341,7 @@ def assert_order_of_execution(): def test_order_of_execution_generator_different_codeline(self, testdir): o = testdir.makepyfile(""" + from __future__ import print_function def test_generative_tests_different_codeline(): import py, pytest test_list = [] @@ -246,8 +357,8 @@ def list_append_0(): test_list.append(0) def assert_order_of_execution(): - py.builtin.print_('expected order', expected_list) - py.builtin.print_('but got ', test_list) + print('expected order', expected_list) + print('but got ', test_list) assert test_list == expected_list yield list_append_0 @@ -269,7 +380,7 @@ def test_setupstate_is_preserved_134(self, testdir): # has been used during collection. o = testdir.makepyfile(""" setuplist = [] - class TestClass: + class TestClass(object): def setup_method(self, func): #print "setup_method", self, func setuplist.append(self) @@ -303,7 +414,7 @@ def test_setuplist(): assert not skipped and not failed -class TestFunction: +class TestFunction(object): def test_getmodulecollector(self, testdir): item = testdir.getitem("def test_func(): pass") modcol = item.getparent(pytest.Module) @@ -312,7 +423,7 @@ def test_getmodulecollector(self, testdir): def test_function_as_object_instance_ignored(self, testdir): testdir.makepyfile(""" - class A: + class A(object): def __call__(self, tmpdir): 0/0 @@ -322,19 +433,22 @@ def __call__(self, tmpdir): reprec.assertoutcome() def test_function_equality(self, testdir, tmpdir): - from _pytest.python import FixtureManager + from _pytest.fixtures import FixtureManager config = testdir.parseconfigure() session = testdir.Session(config) session._fixturemanager = FixtureManager(session) + def func1(): pass + def func2(): pass + f1 = pytest.Function(name="name", parent=session, config=config, - args=(1,), callobj=func1) + args=(1,), callobj=func1) assert f1 == f1 - f2 = pytest.Function(name="name",config=config, - callobj=func2, parent=session) + f2 = pytest.Function(name="name", config=config, + callobj=func2, parent=session) assert f1 != f2 def test_issue197_parametrize_emptyset(self, testdir): @@ -360,7 +474,7 @@ def test_function(arg): def test_issue213_parametrize_value_no_equal(self, testdir): testdir.makepyfile(""" import pytest - class A: + class A(object): def __eq__(self, other): raise ValueError("not possible") @pytest.mark.parametrize('arg', [A()]) @@ -388,7 +502,6 @@ def test_archival_to_version(key, value): rec = testdir.inline_run() rec.assertoutcome(passed=2) - def test_parametrize_with_non_hashable_values_indirect(self, testdir): """Test parametrization with non-hashable values with indirect parametrization.""" testdir.makepyfile(""" @@ -416,7 +529,6 @@ def test_archival_to_version(key, value): rec = testdir.inline_run() rec.assertoutcome(passed=2) - def test_parametrize_overrides_fixture(self, testdir): """Test parametrization when parameter overrides existing fixture with same name.""" testdir.makepyfile(""" @@ -444,7 +556,6 @@ def test_overridden_via_multiparam(other, value): rec = testdir.inline_run() rec.assertoutcome(passed=3) - def test_parametrize_overrides_parametrized_fixture(self, testdir): """Test parametrization when parameter overrides existing parametrized fixture with same name.""" testdir.makepyfile(""" @@ -462,7 +573,8 @@ def test_overridden_via_param(value): rec = testdir.inline_run() rec.assertoutcome(passed=1) - def test_parametrize_with_mark(selfself, testdir): + @ignore_parametrized_marks + def test_parametrize_with_mark(self, testdir): items = testdir.getitems(""" import pytest @pytest.mark.foo @@ -490,12 +602,15 @@ def test_function(arg): def test_pyfunc_call(self, testdir): item = testdir.getitem("def test_func(): raise ValueError") config = item.config - class MyPlugin1: + + class MyPlugin1(object): def pytest_pyfunc_call(self, pyfuncitem): raise ValueError - class MyPlugin2: + + class MyPlugin2(object): def pytest_pyfunc_call(self, pyfuncitem): return True + config.pluginmanager.register(MyPlugin1()) config.pluginmanager.register(MyPlugin2()) config.hook.pytest_runtest_setup(item=item) @@ -532,6 +647,7 @@ def test2(self, x, y): assert colitems[2].name == 'test2[a-c]' assert colitems[3].name == 'test2[b-c]' + @ignore_parametrized_marks def test_parametrize_skipif(self, testdir): testdir.makepyfile(""" import pytest @@ -545,6 +661,7 @@ def test_skip_if(x): result = testdir.runpytest() result.stdout.fnmatch_lines('* 2 passed, 1 skipped in *') + @ignore_parametrized_marks def test_parametrize_skip(self, testdir): testdir.makepyfile(""" import pytest @@ -558,6 +675,7 @@ def test_skip(x): result = testdir.runpytest() result.stdout.fnmatch_lines('* 2 passed, 1 skipped in *') + @ignore_parametrized_marks def test_parametrize_skipif_no_skip(self, testdir): testdir.makepyfile(""" import pytest @@ -571,6 +689,7 @@ def test_skipif_no_skip(x): result = testdir.runpytest() result.stdout.fnmatch_lines('* 1 failed, 2 passed in *') + @ignore_parametrized_marks def test_parametrize_xfail(self, testdir): testdir.makepyfile(""" import pytest @@ -584,6 +703,7 @@ def test_xfail(x): result = testdir.runpytest() result.stdout.fnmatch_lines('* 2 passed, 1 xfailed in *') + @ignore_parametrized_marks def test_parametrize_passed(self, testdir): testdir.makepyfile(""" import pytest @@ -597,6 +717,7 @@ def test_xfail(x): result = testdir.runpytest() result.stdout.fnmatch_lines('* 2 passed, 1 xpassed in *') + @ignore_parametrized_marks def test_parametrize_xfail_passed(self, testdir): testdir.makepyfile(""" import pytest @@ -610,8 +731,17 @@ def test_passed(x): result = testdir.runpytest() result.stdout.fnmatch_lines('* 3 passed in *') + def test_function_original_name(self, testdir): + items = testdir.getitems(""" + import pytest + @pytest.mark.parametrize('arg', [1,2]) + def test_func(arg): + pass + """) + assert [x.originalname for x in items] == ['test_func', 'test_func'] + -class TestSorting: +class TestSorting(object): def test_check_equality(self, testdir): modcol = testdir.getmodulecol(""" def test_pass(): pass @@ -633,11 +763,11 @@ def test_fail(): assert 0 assert not (fn1 == fn3) assert fn1 != fn3 - for fn in fn1,fn2,fn3: + for fn in fn1, fn2, fn3: assert fn != 3 assert fn != modcol - assert fn != [1,2,3] - assert [1,2,3] != fn + assert fn != [1, 2, 3] + assert [1, 2, 3] != fn assert modcol != fn def test_allow_sane_sorting_for_decorators(self, testdir): @@ -661,7 +791,7 @@ def test_a(y): assert [item.name for item in colitems] == ['test_b', 'test_a'] -class TestConftestCustomization: +class TestConftestCustomization(object): def test_pytest_pycollect_module(self, testdir): testdir.makeconftest(""" import pytest @@ -682,10 +812,12 @@ def pytest_pycollect_makemodule(path, parent): def test_customized_pymakemodule_issue205_subdir(self, testdir): b = testdir.mkdir("a").mkdir("b") b.join("conftest.py").write(_pytest._code.Source(""" - def pytest_pycollect_makemodule(__multicall__): - mod = __multicall__.execute() + import pytest + @pytest.hookimpl(hookwrapper=True) + def pytest_pycollect_makemodule(): + outcome = yield + mod = outcome.get_result() mod.obj.hello = "world" - return mod """)) b.join("test_module.py").write(_pytest._code.Source(""" def test_hello(): @@ -702,7 +834,7 @@ def test_customized_pymakeitem(self, testdir): def pytest_pycollect_makeitem(): outcome = yield if outcome.excinfo is None: - result = outcome.result + result = outcome.get_result() if result: for func in result: func._some123 = "world" @@ -736,11 +868,40 @@ def pytest_pycollect_makeitem(collector, name, obj): def test_makeitem_non_underscore(self, testdir, monkeypatch): modcol = testdir.getmodulecol("def _hello(): pass") - l = [] + values = [] monkeypatch.setattr(pytest.Module, 'makeitem', - lambda self, name, obj: l.append(name)) - l = modcol.collect() - assert '_hello' not in l + lambda self, name, obj: values.append(name)) + values = modcol.collect() + assert '_hello' not in values + + def test_issue2369_collect_module_fileext(self, testdir): + """Ensure we can collect files with weird file extensions as Python + modules (#2369)""" + # We'll implement a little finder and loader to import files containing + # Python source code whose file extension is ".narf". + testdir.makeconftest(""" + import sys, os, imp + from _pytest.python import Module + + class Loader: + def load_module(self, name): + return imp.load_source(name, name + ".narf") + class Finder: + def find_module(self, name, path=None): + if os.path.exists(name + ".narf"): + return Loader() + sys.meta_path.append(Finder()) + + def pytest_collect_file(path, parent): + if path.ext == ".narf": + return Module(path, parent)""") + testdir.makefile(".narf", """ + def test_something(): + assert 1 + 1 == 2""") + # Use runpytest_subprocess, since we're futzing with sys.meta_path. + result = testdir.runpytest_subprocess() + result.stdout.fnmatch_lines('*1 passed*') + def test_setup_only_available_in_subdir(testdir): sub1 = testdir.mkpydir("sub1") @@ -768,6 +929,7 @@ def pytest_runtest_teardown(item): result = testdir.runpytest("-v", "-s") result.assert_outcomes(passed=2) + def test_modulecol_roundtrip(testdir): modcol = testdir.getmodulecol("pass", withinit=True) trail = modcol.nodeid @@ -775,7 +937,7 @@ def test_modulecol_roundtrip(testdir): assert modcol.name == newcol.name -class TestTracebackCutting: +class TestTracebackCutting(object): def test_skip_simple(self): excinfo = pytest.raises(pytest.skip.Exception, 'pytest.skip("xxx")') assert excinfo.traceback[-1].frame.code.name == "skip" @@ -783,22 +945,25 @@ def test_skip_simple(self): def test_traceback_argsetup(self, testdir): testdir.makeconftest(""" - def pytest_funcarg__hello(request): + import pytest + + @pytest.fixture + def hello(request): raise ValueError("xyz") """) p = testdir.makepyfile("def test(hello): pass") result = testdir.runpytest(p) assert result.ret != 0 out = result.stdout.str() - assert out.find("xyz") != -1 - assert out.find("conftest.py:2: ValueError") != -1 - numentries = out.count("_ _ _") # separator for traceback entries + assert "xyz" in out + assert "conftest.py:5: ValueError" in out + numentries = out.count("_ _ _") # separator for traceback entries assert numentries == 0 result = testdir.runpytest("--fulltrace", p) out = result.stdout.str() - assert out.find("conftest.py:2: ValueError") != -1 - numentries = out.count("_ _ _ _") # separator for traceback entries + assert "conftest.py:5: ValueError" in out + numentries = out.count("_ _ _ _") # separator for traceback entries assert numentries > 3 def test_traceback_error_during_import(self, testdir): @@ -898,7 +1063,7 @@ def foo(): assert filter_traceback(tb[-1]) -class TestReportInfo: +class TestReportInfo(object): def test_itemreport_reportinfo(self, testdir, linecomp): testdir.makeconftest(""" import pytest @@ -923,7 +1088,7 @@ def test_func_reportinfo(self, testdir): def test_class_reportinfo(self, testdir): modcol = testdir.getmodulecol(""" # lineno 0 - class TestClass: + class TestClass(object): def test_hello(self): pass """) classcol = testdir.collect_by_name(modcol, "TestClass") @@ -958,7 +1123,7 @@ def test_genfunc(): def check(x): pass yield check, 3 - class TestClass: + class TestClass(object): def test_method(self): pass """ @@ -967,7 +1132,7 @@ def test_reportinfo_with_nasty_getattr(self, testdir): # https://github.com/pytest-dev/pytest/issues/1204 modcol = testdir.getmodulecol(""" # lineno 0 - class TestClass: + class TestClass(object): def __getattr__(self, name): return "this is not an int" @@ -989,7 +1154,7 @@ def test_customized_python_discovery(testdir): p = testdir.makepyfile(""" def check_simple(): pass - class CheckMyApp: + class CheckMyApp(object): def check_meth(self): pass """) @@ -1049,6 +1214,7 @@ def test_hello(): "*1 passed*", ]) + def test_customize_through_attributes(testdir): testdir.makeconftest(""" import pytest @@ -1064,7 +1230,7 @@ def pytest_pycollect_makeitem(collector, name, obj): return MyClass(name, parent=collector) """) testdir.makepyfile(""" - class MyTestClass: + class MyTestClass(object): def test_hello(self): pass """) @@ -1078,11 +1244,11 @@ def test_hello(self): def test_unorderable_types(testdir): testdir.makepyfile(""" - class TestJoinEmpty: + class TestJoinEmpty(object): pass def make_test(): - class Test: + class Test(object): pass Test.__name__ = "TestFoo" return Test @@ -1156,8 +1322,8 @@ def test_real(): result = testdir.runpytest('-rw') result.stdout.fnmatch_lines([ '*collected 1 item*', - 'WC2 *', - '*1 passed, 1 pytest-warnings in *', + "*cannot collect 'test_a' because it is not a function*", + '*1 passed, 1 warnings in *', ]) @@ -1198,3 +1364,39 @@ def test_syntax_error_with_non_ascii_chars(testdir): '*SyntaxError*', '*1 error in*', ]) + + +def test_skip_duplicates_by_default(testdir): + """Test for issue https://github.com/pytest-dev/pytest/issues/1609 (#1609) + + Ignore duplicate directories. + """ + a = testdir.mkdir("a") + fh = a.join("test_a.py") + fh.write(_pytest._code.Source(""" + import pytest + def test_real(): + pass + """)) + result = testdir.runpytest(a.strpath, a.strpath) + result.stdout.fnmatch_lines([ + '*collected 1 item*', + ]) + + +def test_keep_duplicates(testdir): + """Test for issue https://github.com/pytest-dev/pytest/issues/1609 (#1609) + + Use --keep-duplicates to collect tests from duplicate directories. + """ + a = testdir.mkdir("a") + fh = a.join("test_a.py") + fh.write(_pytest._code.Source(""" + import pytest + def test_real(): + pass + """)) + result = testdir.runpytest("--keep-duplicates", a.strpath, a.strpath) + result.stdout.fnmatch_lines([ + '*collected 2 item*', + ]) diff --git a/testing/python/fixture.py b/testing/python/fixture.py index 506d8426e3c3518..b159e8ebb8e1138 100644 --- a/testing/python/fixture.py +++ b/testing/python/fixture.py @@ -2,42 +2,57 @@ import _pytest._code import pytest -import sys -from _pytest import python as funcargs from _pytest.pytester import get_public_names -from _pytest.python import FixtureLookupError +from _pytest.fixtures import FixtureLookupError +from _pytest import fixtures def test_getfuncargnames(): - def f(): pass - assert not funcargs.getfuncargnames(f) - def g(arg): pass - assert funcargs.getfuncargnames(g) == ('arg',) - def h(arg1, arg2="hello"): pass - assert funcargs.getfuncargnames(h) == ('arg1',) - def h(arg1, arg2, arg3="hello"): pass - assert funcargs.getfuncargnames(h) == ('arg1', 'arg2') - class A: + def f(): + pass + assert not fixtures.getfuncargnames(f) + + def g(arg): + pass + assert fixtures.getfuncargnames(g) == ('arg',) + + def h(arg1, arg2="hello"): + pass + assert fixtures.getfuncargnames(h) == ('arg1',) + + def h(arg1, arg2, arg3="hello"): + pass + assert fixtures.getfuncargnames(h) == ('arg1', 'arg2') + + class A(object): def f(self, arg1, arg2="hello"): pass - assert funcargs.getfuncargnames(A().f) == ('arg1',) - if sys.version_info < (3,0): - assert funcargs.getfuncargnames(A.f) == ('arg1',) -class TestFillFixtures: + @staticmethod + def static(arg1, arg2): + pass + + assert fixtures.getfuncargnames(A().f) == ('arg1',) + assert fixtures.getfuncargnames(A.static, cls=A) == ('arg1', 'arg2') + + +class TestFillFixtures(object): def test_fillfuncargs_exposed(self): # used by oejskit, kept for compatibility - assert pytest._fillfuncargs == funcargs.fillfixtures + assert pytest._fillfuncargs == fixtures.fillfixtures def test_funcarg_lookupfails(self, testdir): testdir.makepyfile(""" - def pytest_funcarg__xyzsomething(request): + import pytest + + @pytest.fixture + def xyzsomething(request): return 42 def test_func(some): pass """) - result = testdir.runpytest() # "--collect-only") + result = testdir.runpytest() # "--collect-only") assert result.ret != 0 result.stdout.fnmatch_lines([ "*def test_func(some)*", @@ -47,14 +62,18 @@ def test_func(some): def test_funcarg_basic(self, testdir): item = testdir.getitem(""" - def pytest_funcarg__some(request): + import pytest + + @pytest.fixture + def some(request): return request.function.__name__ - def pytest_funcarg__other(request): + @pytest.fixture + def other(request): return 42 def test_func(some, other): pass """) - funcargs.fillfixtures(item) + fixtures.fillfixtures(item) del item.funcargs["request"] assert len(get_public_names(item.funcargs)) == 2 assert item.funcargs['some'] == "test_func" @@ -62,10 +81,13 @@ def test_func(some, other): def test_funcarg_lookup_modulelevel(self, testdir): testdir.makepyfile(""" - def pytest_funcarg__something(request): + import pytest + + @pytest.fixture + def something(request): return request.function.__name__ - class TestClass: + class TestClass(object): def test_method(self, something): assert something == "test_method" def test_func(something): @@ -76,9 +98,13 @@ def test_func(something): def test_funcarg_lookup_classlevel(self, testdir): p = testdir.makepyfile(""" - class TestClass: - def pytest_funcarg__something(self, request): + import pytest + class TestClass(object): + + @pytest.fixture + def something(self, request): return request.instance + def test_method(self, something): assert something is self """) @@ -92,13 +118,15 @@ def test_conftest_funcargs_only_available_in_subdir(self, testdir): sub2 = testdir.mkpydir("sub2") sub1.join("conftest.py").write(_pytest._code.Source(""" import pytest - def pytest_funcarg__arg1(request): - pytest.raises(Exception, "request.getfuncargvalue('arg2')") + @pytest.fixture + def arg1(request): + pytest.raises(Exception, "request.getfixturevalue('arg2')") """)) sub2.join("conftest.py").write(_pytest._code.Source(""" import pytest - def pytest_funcarg__arg2(request): - pytest.raises(Exception, "request.getfuncargvalue('arg1')") + @pytest.fixture + def arg2(request): + pytest.raises(Exception, "request.getfixturevalue('arg1')") """)) sub1.join("test_in_sub1.py").write("def test_1(arg1): pass") @@ -114,7 +142,7 @@ def test_extend_fixture_module_class(self, testdir): def spam(): return 'spam' - class TestSpam: + class TestSpam(object): @pytest.fixture def spam(self, spam): @@ -336,6 +364,38 @@ def test_spam(spam): result = testdir.runpytest(testfile) result.stdout.fnmatch_lines(["*3 passed*"]) + def test_override_autouse_fixture_with_parametrized_fixture_conftest_conftest(self, testdir): + """Test override of the autouse fixture with parametrized one on the conftest level. + This test covers the issue explained in issue 1601 + """ + testdir.makeconftest(""" + import pytest + + @pytest.fixture(autouse=True) + def spam(): + return 'spam' + """) + subdir = testdir.mkpydir('subdir') + subdir.join("conftest.py").write(_pytest._code.Source(""" + import pytest + + @pytest.fixture(params=[1, 2, 3]) + def spam(request): + return request.param + """)) + testfile = subdir.join("test_spam.py") + testfile.write(_pytest._code.Source(""" + params = {'spam': 1} + + def test_spam(spam): + assert spam == params['spam'] + params['spam'] += 1 + """)) + result = testdir.runpytest() + result.stdout.fnmatch_lines(["*3 passed*"]) + result = testdir.runpytest(testfile) + result.stdout.fnmatch_lines(["*3 passed*"]) + def test_autouse_fixture_plugin(self, testdir): # A fixture from a plugin has no baseid set, which screwed up # the autouse fixture handling. @@ -357,16 +417,32 @@ def test_foo(request): assert result.ret == 0 def test_funcarg_lookup_error(self, testdir): + testdir.makeconftest(""" + import pytest + + @pytest.fixture + def a_fixture(): pass + + @pytest.fixture + def b_fixture(): pass + + @pytest.fixture + def c_fixture(): pass + + @pytest.fixture + def d_fixture(): pass + """) testdir.makepyfile(""" def test_lookup_error(unknown): pass """) result = testdir.runpytest() result.stdout.fnmatch_lines([ - "*ERROR*test_lookup_error*", - "*def test_lookup_error(unknown):*", - "*fixture*unknown*not found*", - "*available fixtures*", + "*ERROR at setup of test_lookup_error*", + " def test_lookup_error(unknown):*", + "E fixture 'unknown' not found", + "> available fixtures:*a_fixture,*b_fixture,*c_fixture,*d_fixture*monkeypatch,*", # sorted + "> use 'py*test --fixtures *' for help on them.", "*1 error*", ]) assert "INTERNAL" not in result.stdout.str() @@ -394,13 +470,16 @@ def test_leak(leak): assert result.ret == 0 -class TestRequestBasic: +class TestRequestBasic(object): def test_request_attributes(self, testdir): item = testdir.getitem(""" - def pytest_funcarg__something(request): pass + import pytest + + @pytest.fixture + def something(request): pass def test_func(something): pass """) - req = funcargs.FixtureRequest(item) + req = fixtures.FixtureRequest(item) assert req.function == item.obj assert req.keywords == item.keywords assert hasattr(req.module, 'test_func') @@ -411,8 +490,11 @@ def test_func(something): pass def test_request_attributes_method(self, testdir): item, = testdir.getitems(""" - class TestB: - def pytest_funcarg__something(self, request): + import pytest + class TestB(object): + + @pytest.fixture + def something(self, request): return 1 def test_func(self, something): pass @@ -421,62 +503,90 @@ def test_func(self, something): assert req.cls.__name__ == "TestB" assert req.instance.__class__ == req.cls - def XXXtest_request_contains_funcarg_arg2fixturedefs(self, testdir): + def test_request_contains_funcarg_arg2fixturedefs(self, testdir): modcol = testdir.getmodulecol(""" - def pytest_funcarg__something(request): + import pytest + @pytest.fixture + def something(request): pass - class TestClass: + class TestClass(object): def test_method(self, something): pass """) item1, = testdir.genitems([modcol]) assert item1.name == "test_method" - arg2fixturedefs = funcargs.FixtureRequest(item1)._arg2fixturedefs + arg2fixturedefs = fixtures.FixtureRequest(item1)._arg2fixturedefs assert len(arg2fixturedefs) == 1 - assert arg2fixturedefs[0].__name__ == "pytest_funcarg__something" + assert arg2fixturedefs['something'][0].argname == "something" - def test_getfuncargvalue_recursive(self, testdir): + def test_getfixturevalue_recursive(self, testdir): testdir.makeconftest(""" - def pytest_funcarg__something(request): + import pytest + + @pytest.fixture + def something(request): return 1 """) testdir.makepyfile(""" - def pytest_funcarg__something(request): - return request.getfuncargvalue("something") + 1 + import pytest + + @pytest.fixture + def something(request): + return request.getfixturevalue("something") + 1 def test_func(something): assert something == 2 """) reprec = testdir.inline_run() reprec.assertoutcome(passed=1) - def test_getfuncargvalue(self, testdir): + @pytest.mark.parametrize( + 'getfixmethod', ('getfixturevalue', 'getfuncargvalue')) + def test_getfixturevalue(self, testdir, getfixmethod): item = testdir.getitem(""" - l = [2] - def pytest_funcarg__something(request): return 1 - def pytest_funcarg__other(request): - return l.pop() + import pytest + values = [2] + @pytest.fixture + def something(request): return 1 + @pytest.fixture + def other(request): + return values.pop() def test_func(something): pass """) + import contextlib + if getfixmethod == 'getfuncargvalue': + warning_expectation = pytest.warns(DeprecationWarning) + else: + # see #1830 for a cleaner way to accomplish this + @contextlib.contextmanager + def expecting_no_warning(): + yield + + warning_expectation = expecting_no_warning() + req = item._request - pytest.raises(FixtureLookupError, req.getfuncargvalue, "notexists") - val = req.getfuncargvalue("something") - assert val == 1 - val = req.getfuncargvalue("something") - assert val == 1 - val2 = req.getfuncargvalue("other") - assert val2 == 2 - val2 = req.getfuncargvalue("other") # see about caching - assert val2 == 2 - pytest._fillfuncargs(item) - assert item.funcargs["something"] == 1 - assert len(get_public_names(item.funcargs)) == 2 - assert "request" in item.funcargs - #assert item.funcargs == {'something': 1, "other": 2} + with warning_expectation: + fixture_fetcher = getattr(req, getfixmethod) + with pytest.raises(FixtureLookupError): + fixture_fetcher("notexists") + val = fixture_fetcher("something") + assert val == 1 + val = fixture_fetcher("something") + assert val == 1 + val2 = fixture_fetcher("other") + assert val2 == 2 + val2 = fixture_fetcher("other") # see about caching + assert val2 == 2 + pytest._fillfuncargs(item) + assert item.funcargs["something"] == 1 + assert len(get_public_names(item.funcargs)) == 2 + assert "request" in item.funcargs def test_request_addfinalizer(self, testdir): item = testdir.getitem(""" + import pytest teardownlist = [] - def pytest_funcarg__something(request): + @pytest.fixture + def something(request): request.addfinalizer(lambda: teardownlist.append(1)) def test_func(something): pass """) @@ -490,18 +600,33 @@ def test_func(something): pass print(ss.stack) assert teardownlist == [1] + def test_mark_as_fixture_with_prefix_and_decorator_fails(self, testdir): + testdir.makeconftest(""" + import pytest + + @pytest.fixture + def pytest_funcarg__marked_with_prefix_and_decorator(): + pass + """) + result = testdir.runpytest_subprocess() + assert result.ret != 0 + result.stdout.fnmatch_lines([ + "*AssertionError: fixtures cannot have*@pytest.fixture*", + "*pytest_funcarg__marked_with_prefix_and_decorator*" + ]) + def test_request_addfinalizer_failing_setup(self, testdir): testdir.makepyfile(""" import pytest - l = [1] + values = [1] @pytest.fixture def myfix(request): - request.addfinalizer(l.pop) + request.addfinalizer(values.pop) assert 0 def test_fix(myfix): pass def test_finalizer_ran(): - assert not l + assert not values """) reprec = testdir.inline_run("-s") reprec.assertoutcome(failed=1, passed=1) @@ -509,39 +634,73 @@ def test_finalizer_ran(): def test_request_addfinalizer_failing_setup_module(self, testdir): testdir.makepyfile(""" import pytest - l = [1, 2] + values = [1, 2] @pytest.fixture(scope="module") def myfix(request): - request.addfinalizer(l.pop) - request.addfinalizer(l.pop) + request.addfinalizer(values.pop) + request.addfinalizer(values.pop) assert 0 def test_fix(myfix): pass """) reprec = testdir.inline_run("-s") mod = reprec.getcalls("pytest_runtest_setup")[0].item.module - assert not mod.l - + assert not mod.values def test_request_addfinalizer_partial_setup_failure(self, testdir): p = testdir.makepyfile(""" - l = [] - def pytest_funcarg__something(request): - request.addfinalizer(lambda: l.append(None)) + import pytest + values = [] + @pytest.fixture + def something(request): + request.addfinalizer(lambda: values.append(None)) def test_func(something, missingarg): pass def test_second(): - assert len(l) == 1 + assert len(values) == 1 """) result = testdir.runpytest(p) result.stdout.fnmatch_lines([ "*1 error*" # XXX the whole module collection fails - ]) + ]) + + def test_request_subrequest_addfinalizer_exceptions(self, testdir): + """ + Ensure exceptions raised during teardown by a finalizer are suppressed + until all finalizers are called, re-raising the first exception (#2440) + """ + testdir.makepyfile(""" + import pytest + values = [] + def _excepts(where): + raise Exception('Error in %s fixture' % where) + @pytest.fixture + def subrequest(request): + return request + @pytest.fixture + def something(subrequest): + subrequest.addfinalizer(lambda: values.append(1)) + subrequest.addfinalizer(lambda: values.append(2)) + subrequest.addfinalizer(lambda: _excepts('something')) + @pytest.fixture + def excepts(subrequest): + subrequest.addfinalizer(lambda: _excepts('excepts')) + subrequest.addfinalizer(lambda: values.append(3)) + def test_first(something, excepts): + pass + def test_second(): + assert values == [3, 2, 1] + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*Exception: Error in excepts fixture', + '* 2 passed, 1 error in *', + ]) def test_request_getmodulepath(self, testdir): modcol = testdir.getmodulecol("def test_somefunc(): pass") item, = testdir.genitems([modcol]) - req = funcargs.FixtureRequest(item) + req = fixtures.FixtureRequest(item) assert req.fspath == modcol.fspath def test_request_fixturenames(self, testdir): @@ -567,9 +726,11 @@ def test_function(request, farg): def test_funcargnames_compatattr(self, testdir): testdir.makepyfile(""" + import pytest def pytest_generate_tests(metafunc): assert metafunc.funcargnames == metafunc.fixturenames - def pytest_funcarg__fn(request): + @pytest.fixture + def fn(request): assert request._pyfuncitem.funcargnames == \ request._pyfuncitem.fixturenames return request.funcargnames, request.fixturenames @@ -583,28 +744,28 @@ def test_hello(fn): def test_setupdecorator_and_xunit(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope='module', autouse=True) def setup_module(): - l.append("module") + values.append("module") @pytest.fixture(autouse=True) def setup_function(): - l.append("function") + values.append("function") def test_func(): pass - class TestClass: + class TestClass(object): @pytest.fixture(scope="class", autouse=True) def setup_class(self): - l.append("class") + values.append("class") @pytest.fixture(autouse=True) def setup_method(self): - l.append("method") + values.append("method") def test_method(self): pass def test_all(): - assert l == ["module", "function", "class", + assert values == ["module", "function", "class", "function", "method", "function"] """) reprec = testdir.inline_run("-v") @@ -614,7 +775,9 @@ def test_fixtures_sub_subdir_normalize_sep(self, testdir): # this tests that normalization of nodeids takes place b = testdir.mkdir("tests").mkdir("unit") b.join("conftest.py").write(_pytest._code.Source(""" - def pytest_funcarg__arg1(): + import pytest + @pytest.fixture + def arg1(): pass """)) p = b.join("test_module.py") @@ -659,18 +822,22 @@ def test_1(arg): reprec = testdir.inline_run() reprec.assertoutcome(passed=2) -class TestRequestMarking: + +class TestRequestMarking(object): def test_applymarker(self, testdir): - item1,item2 = testdir.getitems(""" - def pytest_funcarg__something(request): + item1, item2 = testdir.getitems(""" + import pytest + + @pytest.fixture + def something(request): pass - class TestClass: + class TestClass(object): def test_func1(self, something): pass def test_func2(self, something): pass """) - req1 = funcargs.FixtureRequest(item1) + req1 = fixtures.FixtureRequest(item1) assert 'xfail' not in item1.keywords req1.applymarker(pytest.mark.xfail) assert 'xfail' in item1.keywords @@ -716,17 +883,21 @@ def test_fun2(keywords): reprec = testdir.inline_run() reprec.assertoutcome(passed=2) -class TestRequestCachedSetup: + +class TestRequestCachedSetup(object): def test_request_cachedsetup_defaultmodule(self, testdir): reprec = testdir.inline_runsource(""" mysetup = ["hello",].pop - def pytest_funcarg__something(request): + import pytest + + @pytest.fixture + def something(request): return request.cached_setup(mysetup, scope="module") def test_func1(something): assert something == "hello" - class TestClass: + class TestClass(object): def test_func1a(self, something): assert something == "hello" """) @@ -736,13 +907,15 @@ def test_request_cachedsetup_class(self, testdir): reprec = testdir.inline_runsource(""" mysetup = ["hello", "hello2", "hello3"].pop - def pytest_funcarg__something(request): + import pytest + @pytest.fixture + def something(request): return request.cached_setup(mysetup, scope="class") def test_func1(something): assert something == "hello3" def test_func2(something): assert something == "hello2" - class TestClass: + class TestClass(object): def test_func1a(self, something): assert something == "hello" def test_func2b(self, something): @@ -752,10 +925,12 @@ def test_func2b(self, something): def test_request_cachedsetup_extrakey(self, testdir): item1 = testdir.getitem("def test_func(): pass") - req1 = funcargs.FixtureRequest(item1) - l = ["hello", "world"] + req1 = fixtures.FixtureRequest(item1) + values = ["hello", "world"] + def setup(): - return l.pop() + return values.pop() + ret1 = req1.cached_setup(setup, extrakey=1) ret2 = req1.cached_setup(setup, extrakey=2) assert ret2 == "hello" @@ -767,28 +942,35 @@ def setup(): def test_request_cachedsetup_cache_deletion(self, testdir): item1 = testdir.getitem("def test_func(): pass") - req1 = funcargs.FixtureRequest(item1) - l = [] + req1 = fixtures.FixtureRequest(item1) + values = [] + def setup(): - l.append("setup") + values.append("setup") + def teardown(val): - l.append("teardown") + values.append("teardown") + req1.cached_setup(setup, teardown, scope="function") - assert l == ['setup'] + assert values == ['setup'] # artificial call of finalizer setupstate = req1._pyfuncitem.session._setupstate setupstate._callfinalizers(item1) - assert l == ["setup", "teardown"] + assert values == ["setup", "teardown"] req1.cached_setup(setup, teardown, scope="function") - assert l == ["setup", "teardown", "setup"] + assert values == ["setup", "teardown", "setup"] setupstate._callfinalizers(item1) - assert l == ["setup", "teardown", "setup", "teardown"] + assert values == ["setup", "teardown", "setup", "teardown"] def test_request_cached_setup_two_args(self, testdir): testdir.makepyfile(""" - def pytest_funcarg__arg1(request): + import pytest + + @pytest.fixture + def arg1(request): return request.cached_setup(lambda: 42) - def pytest_funcarg__arg2(request): + @pytest.fixture + def arg2(request): return request.cached_setup(lambda: 17) def test_two_different_setups(arg1, arg2): assert arg1 != arg2 @@ -798,12 +980,16 @@ def test_two_different_setups(arg1, arg2): "*1 passed*" ]) - def test_request_cached_setup_getfuncargvalue(self, testdir): + def test_request_cached_setup_getfixturevalue(self, testdir): testdir.makepyfile(""" - def pytest_funcarg__arg1(request): - arg1 = request.getfuncargvalue("arg2") + import pytest + + @pytest.fixture + def arg1(request): + arg1 = request.getfixturevalue("arg2") return request.cached_setup(lambda: arg1 + 1) - def pytest_funcarg__arg2(request): + @pytest.fixture + def arg2(request): return request.cached_setup(lambda: 10) def test_two_funcarg(arg1): assert arg1 == 11 @@ -815,16 +1001,18 @@ def test_two_funcarg(arg1): def test_request_cached_setup_functional(self, testdir): testdir.makepyfile(test_0=""" - l = [] - def pytest_funcarg__something(request): + import pytest + values = [] + @pytest.fixture + def something(request): val = request.cached_setup(fsetup, fteardown) return val def fsetup(mycache=[1]): - l.append(mycache.pop()) - return l + values.append(mycache.pop()) + return values def fteardown(something): - l.remove(something[0]) - l.append(2) + values.remove(something[0]) + values.append(2) def test_list_once(something): assert something == [1] def test_list_twice(something): @@ -833,7 +1021,7 @@ def test_list_twice(something): testdir.makepyfile(test_1=""" import test_0 # should have run already def test_check_test0_has_teardown_correct(): - assert test_0.l == [2] + assert test_0.values == [2] """) result = testdir.runpytest("-v") result.stdout.fnmatch_lines([ @@ -842,7 +1030,10 @@ def test_check_test0_has_teardown_correct(): def test_issue117_sessionscopeteardown(self, testdir): testdir.makepyfile(""" - def pytest_funcarg__app(request): + import pytest + + @pytest.fixture + def app(request): app = request.cached_setup( scope='session', setup=lambda: 0, @@ -858,7 +1049,8 @@ def test_func(app): "*ZeroDivisionError*", ]) -class TestFixtureUsages: + +class TestFixtureUsages(object): def test_noargfixturedec(self, testdir): testdir.makepyfile(""" import pytest @@ -935,13 +1127,29 @@ def test_add(arg1, arg2): "*1 error*" ]) + def test_invalid_scope(self, testdir): + testdir.makepyfile(""" + import pytest + @pytest.fixture(scope="functions") + def badscope(): + pass + + def test_nothing(badscope): + pass + """) + result = testdir.runpytest_inprocess() + result.stdout.fnmatch_lines( + ("*ValueError: fixture badscope from test_invalid_scope.py has an unsupported" + " scope value 'functions'") + ) + def test_funcarg_parametrized_and_used_twice(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(params=[1,2]) def arg1(request): - l.append(1) + values.append(1) return request.param @pytest.fixture() @@ -950,7 +1158,7 @@ def arg2(arg1): def test_add(arg1, arg2): assert arg2 == arg1 + 1 - assert len(l) == arg1 + assert len(values) == arg1 """) result = testdir.runpytest() result.stdout.fnmatch_lines([ @@ -984,15 +1192,15 @@ def test_missing(call_fail): def test_factory_setup_as_classes_fails(self, testdir): testdir.makepyfile(""" import pytest - class arg1: + class arg1(object): def __init__(self, request): self.x = 1 arg1 = pytest.fixture()(arg1) """) reprec = testdir.inline_run() - l = reprec.getfailedcollections() - assert len(l) == 1 + values = reprec.getfailedcollections() + assert len(values) == 1 def test_request_can_be_overridden(self, testdir): testdir.makepyfile(""" @@ -1011,20 +1219,20 @@ def test_usefixtures_marker(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope="class") def myfix(request): request.cls.hello = "world" - l.append(1) + values.append(1) - class TestClass: + class TestClass(object): def test_one(self): assert self.hello == "world" - assert len(l) == 1 + assert len(values) == 1 def test_two(self): assert self.hello == "world" - assert len(l) == 1 + assert len(values) == 1 pytest.mark.usefixtures("myfix")(TestClass) """) reprec = testdir.inline_run() @@ -1044,7 +1252,7 @@ def myfix(request): """) testdir.makepyfile(""" - class TestClass: + class TestClass(object): def test_one(self): assert self.hello == "world" def test_two(self): @@ -1063,7 +1271,7 @@ def test_request_instance_issue203(self, testdir): testdir.makepyfile(""" import pytest - class TestClass: + class TestClass(object): @pytest.fixture def setup1(self, request): assert self == request.instance @@ -1078,7 +1286,7 @@ def test_fixture_parametrized_with_iterator(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] def f(): yield 1 yield 2 @@ -1092,34 +1300,41 @@ def arg2(request): return request.param def test_1(arg): - l.append(arg) + values.append(arg) def test_2(arg2): - l.append(arg2*10) + values.append(arg2*10) """) reprec = testdir.inline_run("-v") reprec.assertoutcome(passed=4) - l = reprec.getcalls("pytest_runtest_call")[0].item.module.l - assert l == [1,2, 10,20] + values = reprec.getcalls("pytest_runtest_call")[0].item.module.values + assert values == [1, 2, 10, 20] + +class TestFixtureManagerParseFactories(object): -class TestFixtureManagerParseFactories: - def pytest_funcarg__testdir(self, request): - testdir = request.getfuncargvalue("testdir") + @pytest.fixture + def testdir(self, request): + testdir = request.getfixturevalue("testdir") testdir.makeconftest(""" - def pytest_funcarg__hello(request): + import pytest + + @pytest.fixture + def hello(request): return "conftest" - def pytest_funcarg__fm(request): + @pytest.fixture + def fm(request): return request._fixturemanager - def pytest_funcarg__item(request): + @pytest.fixture + def item(request): return request._pyfuncitem """) return testdir def test_parsefactories_evil_objects_issue214(self, testdir): testdir.makepyfile(""" - class A: + class A(object): def __call__(self): pass def __getattr__(self, name): @@ -1138,17 +1353,21 @@ def test_hello(item, fm): faclist = fm.getfixturedefs(name, item.nodeid) assert len(faclist) == 1 fac = faclist[0] - assert fac.func.__name__ == "pytest_funcarg__" + name + assert fac.func.__name__ == name """) reprec = testdir.inline_run("-s") reprec.assertoutcome(passed=1) def test_parsefactories_conftest_and_module_and_class(self, testdir): testdir.makepyfile(""" - def pytest_funcarg__hello(request): + import pytest + + @pytest.fixture + def hello(request): return "module" - class TestClass: - def pytest_funcarg__hello(self, request): + class TestClass(object): + @pytest.fixture + def hello(self, request): return "class" def test_hello(self, item, fm): faclist = fm.getfixturedefs("hello", item.nodeid) @@ -1195,8 +1414,10 @@ def test_x(one): reprec.assertoutcome(passed=2) -class TestAutouseDiscovery: - def pytest_funcarg__testdir(self, testdir): +class TestAutouseDiscovery(object): + + @pytest.fixture + def testdir(self, testdir): testdir.makeconftest(""" import pytest @pytest.fixture(autouse=True) @@ -1210,10 +1431,12 @@ def arg1(tmpdir): def perfunction2(arg1): pass - def pytest_funcarg__fm(request): + @pytest.fixture + def fm(request): return request._fixturemanager - def pytest_funcarg__item(request): + @pytest.fixture + def item(request): return request._pyfuncitem """) return testdir @@ -1233,20 +1456,20 @@ def test_check_setup(item, fm): def test_two_classes_separated_autouse(self, testdir): testdir.makepyfile(""" import pytest - class TestA: - l = [] + class TestA(object): + values = [] @pytest.fixture(autouse=True) def setup1(self): - self.l.append(1) + self.values.append(1) def test_setup1(self): - assert self.l == [1] - class TestB: - l = [] + assert self.values == [1] + class TestB(object): + values = [] @pytest.fixture(autouse=True) def setup2(self): - self.l.append(1) + self.values.append(1) def test_setup2(self): - assert self.l == [1] + assert self.values == [1] """) reprec = testdir.inline_run() reprec.assertoutcome(passed=2) @@ -1254,7 +1477,7 @@ def test_setup2(self): def test_setup_at_classlevel(self, testdir): testdir.makepyfile(""" import pytest - class TestClass: + class TestClass(object): @pytest.fixture(autouse=True) def permethod(self, request): request.instance.funcname = request.function.__name__ @@ -1329,28 +1552,28 @@ def hello(): def test_autouse_in_module_and_two_classes(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(autouse=True) def append1(): - l.append("module") + values.append("module") def test_x(): - assert l == ["module"] + assert values == ["module"] - class TestA: + class TestA(object): @pytest.fixture(autouse=True) def append2(self): - l.append("A") + values.append("A") def test_hello(self): - assert l == ["module", "module", "A"], l - class TestA2: + assert values == ["module", "module", "A"], values + class TestA2(object): def test_world(self): - assert l == ["module", "module", "A", "module"], l + assert values == ["module", "module", "A", "module"], values """) reprec = testdir.inline_run() reprec.assertoutcome(passed=3) -class TestAutouseManagement: +class TestAutouseManagement(object): def test_autouse_conftest_mid_directory(self, testdir): pkgdir = testdir.mkpydir("xyz123") pkgdir.join("conftest.py").write(_pytest._code.Source(""" @@ -1385,28 +1608,26 @@ def f(hello): reprec = testdir.inline_run() reprec.assertoutcome(passed=2) - - def test_funcarg_and_setup(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope="module") def arg(): - l.append(1) + values.append(1) return 0 @pytest.fixture(scope="module", autouse=True) def something(arg): - l.append(2) + values.append(2) def test_hello(arg): - assert len(l) == 2 - assert l == [1,2] + assert len(values) == 2 + assert values == [1,2] assert arg == 0 def test_hello2(arg): - assert len(l) == 2 - assert l == [1,2] + assert len(values) == 2 + assert values == [1,2] assert arg == 0 """) reprec = testdir.inline_run() @@ -1415,20 +1636,20 @@ def test_hello2(arg): def test_uses_parametrized_resource(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(params=[1,2]) def arg(request): return request.param @pytest.fixture(autouse=True) def something(arg): - l.append(arg) + values.append(arg) def test_hello(): - if len(l) == 1: - assert l == [1] - elif len(l) == 2: - assert l == [1, 2] + if len(values) == 1: + assert values == [1] + elif len(values) == 2: + assert values == [1, 2] else: 0/0 @@ -1440,7 +1661,7 @@ def test_session_parametrized_function(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope="session", params=[1,2]) def arg(request): @@ -1449,14 +1670,14 @@ def arg(request): @pytest.fixture(scope="function", autouse=True) def append(request, arg): if request.function.__name__ == "test_some": - l.append(arg) + values.append(arg) def test_some(): pass def test_result(arg): - assert len(l) == arg - assert l[:arg] == [1,2][:arg] + assert len(values) == arg + assert values[:arg] == [1,2][:arg] """) reprec = testdir.inline_run("-v", "-s") reprec.assertoutcome(passed=4) @@ -1466,7 +1687,7 @@ def test_class_function_parametrization_finalization(self, testdir): import pytest import pprint - l = [] + values = [] @pytest.fixture(scope="function", params=[1,2]) def farg(request): @@ -1479,42 +1700,43 @@ def carg(request): @pytest.fixture(scope="function", autouse=True) def append(request, farg, carg): def fin(): - l.append("fin_%s%s" % (carg, farg)) + values.append("fin_%s%s" % (carg, farg)) request.addfinalizer(fin) """) testdir.makepyfile(""" import pytest - class TestClass: + class TestClass(object): def test_1(self): pass - class TestClass2: + class TestClass2(object): def test_2(self): pass """) - reprec = testdir.inline_run("-v","-s") + confcut = "--confcutdir={0}".format(testdir.tmpdir) + reprec = testdir.inline_run("-v", "-s", confcut) reprec.assertoutcome(passed=8) config = reprec.getcalls("pytest_unconfigure")[0].config - l = config.pluginmanager._getconftestmodules(p)[0].l - assert l == ["fin_a1", "fin_a2", "fin_b1", "fin_b2"] * 2 + values = config.pluginmanager._getconftestmodules(p)[0].values + assert values == ["fin_a1", "fin_a2", "fin_b1", "fin_b2"] * 2 def test_scope_ordering(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope="function", autouse=True) def fappend2(): - l.append(2) + values.append(2) @pytest.fixture(scope="class", autouse=True) def classappend3(): - l.append(3) + values.append(3) @pytest.fixture(scope="module", autouse=True) def mappend(): - l.append(1) + values.append(1) - class TestHallo: + class TestHallo(object): def test_method(self): - assert l == [1,3,2] + assert values == [1,3,2] """) reprec = testdir.inline_run() reprec.assertoutcome(passed=1) @@ -1522,23 +1744,23 @@ def test_method(self): def test_parametrization_setup_teardown_ordering(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] def pytest_generate_tests(metafunc): if metafunc.cls is not None: metafunc.parametrize("item", [1,2], scope="class") - class TestClass: + class TestClass(object): @pytest.fixture(scope="class", autouse=True) def addteardown(self, item, request): - l.append("setup-%d" % item) - request.addfinalizer(lambda: l.append("teardown-%d" % item)) + values.append("setup-%d" % item) + request.addfinalizer(lambda: values.append("teardown-%d" % item)) def test_step1(self, item): - l.append("step1-%d" % item) + values.append("step1-%d" % item) def test_step2(self, item): - l.append("step2-%d" % item) + values.append("step2-%d" % item) def test_finish(): - print (l) - assert l == ["setup-1", "step1-1", "step2-1", "teardown-1", + print (values) + assert values == ["setup-1", "step1-1", "step2-1", "teardown-1", "setup-2", "step1-2", "step2-2", "teardown-2",] """) reprec = testdir.inline_run() @@ -1548,56 +1770,56 @@ def test_ordering_autouse_before_explicit(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(autouse=True) def fix1(): - l.append(1) + values.append(1) @pytest.fixture() def arg1(): - l.append(2) + values.append(2) def test_hello(arg1): - assert l == [1,2] + assert values == [1,2] """) reprec = testdir.inline_run() reprec.assertoutcome(passed=1) @pytest.mark.issue226 - @pytest.mark.parametrize("param1", ["", "params=[1]"], ids=["p00","p01"]) - @pytest.mark.parametrize("param2", ["", "params=[1]"], ids=["p10","p11"]) + @pytest.mark.parametrize("param1", ["", "params=[1]"], ids=["p00", "p01"]) + @pytest.mark.parametrize("param2", ["", "params=[1]"], ids=["p10", "p11"]) def test_ordering_dependencies_torndown_first(self, testdir, param1, param2): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(%(param1)s) def arg1(request): - request.addfinalizer(lambda: l.append("fin1")) - l.append("new1") + request.addfinalizer(lambda: values.append("fin1")) + values.append("new1") @pytest.fixture(%(param2)s) def arg2(request, arg1): - request.addfinalizer(lambda: l.append("fin2")) - l.append("new2") + request.addfinalizer(lambda: values.append("fin2")) + values.append("new2") def test_arg(arg2): pass def test_check(): - assert l == ["new1", "new2", "fin2", "fin1"] + assert values == ["new1", "new2", "fin2", "fin1"] """ % locals()) reprec = testdir.inline_run("-s") reprec.assertoutcome(passed=2) -class TestFixtureMarker: +class TestFixtureMarker(object): def test_parametrize(self, testdir): testdir.makepyfile(""" import pytest @pytest.fixture(params=["a", "b", "c"]) def arg(request): return request.param - l = [] + values = [] def test_param(arg): - l.append(arg) + values.append(arg) def test_result(): - assert l == list("abc") + assert values == list("abc") """) reprec = testdir.inline_run() reprec.assertoutcome(passed=4) @@ -1641,21 +1863,21 @@ def test_foo(fixt, val): def test_scope_session(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope="module") def arg(): - l.append(1) + values.append(1) return 1 def test_1(arg): assert arg == 1 def test_2(arg): assert arg == 1 - assert len(l) == 1 - class TestClass: + assert len(values) == 1 + class TestClass(object): def test3(self, arg): assert arg == 1 - assert len(l) == 1 + assert len(values) == 1 """) reprec = testdir.inline_run() reprec.assertoutcome(passed=3) @@ -1663,10 +1885,10 @@ def test3(self, arg): def test_scope_session_exc(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope="session") def fix(): - l.append(1) + values.append(1) pytest.skip('skipping') def test_1(fix): @@ -1674,7 +1896,7 @@ def test_1(fix): def test_2(fix): pass def test_last(): - assert l == [1] + assert values == [1] """) reprec = testdir.inline_run() reprec.assertoutcome(skipped=2, passed=1) @@ -1682,11 +1904,11 @@ def test_last(): def test_scope_session_exc_two_fix(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] m = [] @pytest.fixture(scope="session") def a(): - l.append(1) + values.append(1) pytest.skip('skipping') @pytest.fixture(scope="session") def b(a): @@ -1697,7 +1919,7 @@ def test_1(b): def test_2(b): pass def test_last(): - assert l == [1] + assert values == [1] assert m == [] """) reprec = testdir.inline_run() @@ -1735,21 +1957,21 @@ def test_last(req_list): def test_scope_module_uses_session(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope="module") def arg(): - l.append(1) + values.append(1) return 1 def test_1(arg): assert arg == 1 def test_2(arg): assert arg == 1 - assert len(l) == 1 - class TestClass: + assert len(values) == 1 + class TestClass(object): def test3(self, arg): assert arg == 1 - assert len(l) == 1 + assert len(values) == 1 """) reprec = testdir.inline_run() reprec.assertoutcome(passed=3) @@ -1757,17 +1979,19 @@ def test3(self, arg): def test_scope_module_and_finalizer(self, testdir): testdir.makeconftest(""" import pytest - finalized = [] - created = [] + finalized_list = [] + created_list = [] @pytest.fixture(scope="module") def arg(request): - created.append(1) + created_list.append(1) assert request.scope == "module" - request.addfinalizer(lambda: finalized.append(1)) - def pytest_funcarg__created(request): - return len(created) - def pytest_funcarg__finalized(request): - return len(finalized) + request.addfinalizer(lambda: finalized_list.append(1)) + @pytest.fixture + def created(request): + return len(created_list) + @pytest.fixture + def finalized(request): + return len(finalized_list) """) testdir.makepyfile( test_mod1=""" @@ -1790,9 +2014,9 @@ def test_4(arg, created, finalized): reprec.assertoutcome(passed=4) @pytest.mark.parametrize("method", [ - 'request.getfuncargvalue("arg")', + 'request.getfixturevalue("arg")', 'request.cached_setup(lambda: None, scope="function")', - ], ids=["getfuncargvalue", "cached_setup"]) + ], ids=["getfixturevalue", "cached_setup"]) def test_scope_mismatch_various(self, testdir, method): testdir.makeconftest(""" import pytest @@ -1842,17 +2066,17 @@ def test_parametrize_and_scope(self, testdir): @pytest.fixture(scope="module", params=["a", "b", "c"]) def arg(request): return request.param - l = [] + values = [] def test_param(arg): - l.append(arg) + values.append(arg) """) reprec = testdir.inline_run("-v") reprec.assertoutcome(passed=3) - l = reprec.getcalls("pytest_runtest_call")[0].item.module.l - assert len(l) == 3 - assert "a" in l - assert "b" in l - assert "c" in l + values = reprec.getcalls("pytest_runtest_call")[0].item.module.values + assert len(values) == 3 + assert "a" in values + assert "b" in values + assert "c" in values def test_scope_mismatch(self, testdir): testdir.makeconftest(""" @@ -1883,18 +2107,22 @@ def test_parametrize_separated_order(self, testdir): def arg(request): return request.param - l = [] + values = [] def test_1(arg): - l.append(arg) + values.append(arg) def test_2(arg): - l.append(arg) + values.append(arg) """) reprec = testdir.inline_run("-v") reprec.assertoutcome(passed=4) - l = reprec.getcalls("pytest_runtest_call")[0].item.module.l - assert l == [1,1,2,2] + values = reprec.getcalls("pytest_runtest_call")[0].item.module.values + assert values == [1, 1, 2, 2] def test_module_parametrized_ordering(self, testdir): + testdir.makeini(""" + [pytest] + console_output_style=classic + """) testdir.makeconftest(""" import pytest @@ -1941,10 +2169,14 @@ def test_func4(marg): """) def test_class_ordering(self, testdir): + testdir.makeini(""" + [pytest] + console_output_style=classic + """) testdir.makeconftest(""" import pytest - l = [] + values = [] @pytest.fixture(scope="function", params=[1,2]) def farg(request): @@ -1957,18 +2189,18 @@ def carg(request): @pytest.fixture(scope="function", autouse=True) def append(request, farg, carg): def fin(): - l.append("fin_%s%s" % (carg, farg)) + values.append("fin_%s%s" % (carg, farg)) request.addfinalizer(fin) """) testdir.makepyfile(""" import pytest - class TestClass2: + class TestClass2(object): def test_1(self): pass def test_2(self): pass - class TestClass: + class TestClass(object): def test_3(self): pass """) @@ -1995,30 +2227,30 @@ def test_parametrize_separated_order_higher_scope_first(self, testdir): @pytest.fixture(scope="function", params=[1, 2]) def arg(request): param = request.param - request.addfinalizer(lambda: l.append("fin:%s" % param)) - l.append("create:%s" % param) + request.addfinalizer(lambda: values.append("fin:%s" % param)) + values.append("create:%s" % param) return request.param @pytest.fixture(scope="module", params=["mod1", "mod2"]) def modarg(request): param = request.param - request.addfinalizer(lambda: l.append("fin:%s" % param)) - l.append("create:%s" % param) + request.addfinalizer(lambda: values.append("fin:%s" % param)) + values.append("create:%s" % param) return request.param - l = [] + values = [] def test_1(arg): - l.append("test1") + values.append("test1") def test_2(modarg): - l.append("test2") + values.append("test2") def test_3(arg, modarg): - l.append("test3") + values.append("test3") def test_4(modarg, arg): - l.append("test4") + values.append("test4") """) reprec = testdir.inline_run("-v") reprec.assertoutcome(passed=12) - l = reprec.getcalls("pytest_runtest_call")[0].item.module.l + values = reprec.getcalls("pytest_runtest_call")[0].item.module.values expected = [ 'create:1', 'test1', 'fin:1', 'create:2', 'test1', 'fin:2', 'create:mod1', 'test2', 'create:1', 'test3', @@ -2027,10 +2259,10 @@ def test_4(modarg, arg): 'fin:mod1', 'create:mod2', 'test2', 'create:1', 'test3', 'fin:1', 'create:2', 'test3', 'fin:2', 'create:1', 'test4', 'fin:1', 'create:2', 'test4', 'fin:2', - 'fin:mod2'] + 'fin:mod2'] import pprint - pprint.pprint(list(zip(l, expected))) - assert l == expected + pprint.pprint(list(zip(values, expected))) + assert values == expected def test_parametrized_fixture_teardown_order(self, testdir): testdir.makepyfile(""" @@ -2039,29 +2271,29 @@ def test_parametrized_fixture_teardown_order(self, testdir): def param1(request): return request.param - l = [] + values = [] - class TestClass: + class TestClass(object): @classmethod @pytest.fixture(scope="class", autouse=True) def setup1(self, request, param1): - l.append(1) + values.append(1) request.addfinalizer(self.teardown1) @classmethod def teardown1(self): - assert l.pop() == 1 + assert values.pop() == 1 @pytest.fixture(scope="class", autouse=True) def setup2(self, request, param1): - l.append(2) + values.append(2) request.addfinalizer(self.teardown2) @classmethod def teardown2(self): - assert l.pop() == 2 + assert values.pop() == 2 def test(self): pass def test_finish(): - assert not l + assert not values """) result = testdir.runpytest("-v") result.stdout.fnmatch_lines(""" @@ -2083,7 +2315,7 @@ def finalize(): return {} """) b = testdir.mkdir("subdir") - b.join("test_overriden_fixture_finalizer.py").write(dedent(""" + b.join("test_overridden_fixture_finalizer.py").write(dedent(""" import pytest @pytest.fixture def browser(browser): @@ -2101,7 +2333,7 @@ def test_class_scope_with_normal_tests(self, testdir): testpath = testdir.makepyfile(""" import pytest - class Box: + class Box(object): value = 0 @pytest.fixture(scope='class') @@ -2112,11 +2344,11 @@ def a(request): def test_a(a): assert a == 1 - class Test1: + class Test1(object): def test_b(self, a): assert a == 2 - class Test2: + class Test2(object): def test_c(self, a): assert a == 3""") reprec = testdir.inline_run(testpath) @@ -2126,42 +2358,42 @@ def test_c(self, a): def test_request_is_clean(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(params=[1, 2]) def fix(request): - request.addfinalizer(lambda: l.append(request.param)) + request.addfinalizer(lambda: values.append(request.param)) def test_fix(fix): pass """) reprec = testdir.inline_run("-s") - l = reprec.getcalls("pytest_runtest_call")[0].item.module.l - assert l == [1,2] + values = reprec.getcalls("pytest_runtest_call")[0].item.module.values + assert values == [1, 2] def test_parametrize_separated_lifecycle(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope="module", params=[1, 2]) def arg(request): x = request.param - request.addfinalizer(lambda: l.append("fin%s" % x)) + request.addfinalizer(lambda: values.append("fin%s" % x)) return request.param def test_1(arg): - l.append(arg) + values.append(arg) def test_2(arg): - l.append(arg) + values.append(arg) """) reprec = testdir.inline_run("-vs") reprec.assertoutcome(passed=4) - l = reprec.getcalls("pytest_runtest_call")[0].item.module.l + values = reprec.getcalls("pytest_runtest_call")[0].item.module.values import pprint - pprint.pprint(l) - #assert len(l) == 6 - assert l[0] == l[1] == 1 - assert l[2] == "fin1" - assert l[3] == l[4] == 2 - assert l[5] == "fin2" + pprint.pprint(values) + # assert len(values) == 6 + assert values[0] == values[1] == 1 + assert values[2] == "fin1" + assert values[3] == values[4] == 2 + assert values[5] == "fin2" def test_parametrize_function_scoped_finalizers_called(self, testdir): testdir.makepyfile(""" @@ -2170,28 +2402,27 @@ def test_parametrize_function_scoped_finalizers_called(self, testdir): @pytest.fixture(scope="function", params=[1, 2]) def arg(request): x = request.param - request.addfinalizer(lambda: l.append("fin%s" % x)) + request.addfinalizer(lambda: values.append("fin%s" % x)) return request.param - l = [] + values = [] def test_1(arg): - l.append(arg) + values.append(arg) def test_2(arg): - l.append(arg) + values.append(arg) def test_3(): - assert len(l) == 8 - assert l == [1, "fin1", 2, "fin2", 1, "fin1", 2, "fin2"] + assert len(values) == 8 + assert values == [1, "fin1", 2, "fin2", 1, "fin1", 2, "fin2"] """) reprec = testdir.inline_run("-v") reprec.assertoutcome(passed=5) - @pytest.mark.issue246 @pytest.mark.parametrize("scope", ["session", "function", "module"]) def test_finalizer_order_on_parametrization(self, scope, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(scope=%(scope)r, params=["1"]) def fix1(request): @@ -2200,13 +2431,13 @@ def fix1(request): @pytest.fixture(scope=%(scope)r) def fix2(request, base): def cleanup_fix2(): - assert not l, "base should not have been finalized" + assert not values, "base should not have been finalized" request.addfinalizer(cleanup_fix2) @pytest.fixture(scope=%(scope)r) def base(request, fix1): def cleanup_base(): - l.append("fin_base") + values.append("fin_base") print ("finalizing base") request.addfinalizer(cleanup_base) @@ -2224,29 +2455,29 @@ def test_other(): def test_class_scope_parametrization_ordering(self, testdir): testdir.makepyfile(""" import pytest - l = [] + values = [] @pytest.fixture(params=["John", "Doe"], scope="class") def human(request): - request.addfinalizer(lambda: l.append("fin %s" % request.param)) + request.addfinalizer(lambda: values.append("fin %s" % request.param)) return request.param - class TestGreetings: + class TestGreetings(object): def test_hello(self, human): - l.append("test_hello") + values.append("test_hello") - class TestMetrics: + class TestMetrics(object): def test_name(self, human): - l.append("test_name") + values.append("test_name") def test_population(self, human): - l.append("test_population") + values.append("test_population") """) reprec = testdir.inline_run() reprec.assertoutcome(passed=6) - l = reprec.getcalls("pytest_runtest_call")[0].item.module.l - assert l == ["test_hello", "fin John", "test_hello", "fin Doe", - "test_name", "test_population", "fin John", - "test_name", "test_population", "fin Doe"] + values = reprec.getcalls("pytest_runtest_call")[0].item.module.values + assert values == ["test_hello", "fin John", "test_hello", "fin Doe", + "test_name", "test_population", "fin John", + "test_name", "test_population", "fin Doe"] def test_parametrize_setup_function(self, testdir): testdir.makepyfile(""" @@ -2258,21 +2489,21 @@ def arg(request): @pytest.fixture(scope="module", autouse=True) def mysetup(request, arg): - request.addfinalizer(lambda: l.append("fin%s" % arg)) - l.append("setup%s" % arg) + request.addfinalizer(lambda: values.append("fin%s" % arg)) + values.append("setup%s" % arg) - l = [] + values = [] def test_1(arg): - l.append(arg) + values.append(arg) def test_2(arg): - l.append(arg) + values.append(arg) def test_3(): import pprint - pprint.pprint(l) + pprint.pprint(values) if arg == 1: - assert l == ["setup1", 1, 1, ] + assert values == ["setup1", 1, 1, ] elif arg == 2: - assert l == ["setup1", 1, 1, "fin1", + assert values == ["setup1", 1, 1, "fin1", "setup2", 2, 2, ] """) @@ -2326,9 +2557,42 @@ def test_foo(fix): '*test_foo*alpha*', '*test_foo*beta*']) + @pytest.mark.issue920 + def test_deterministic_fixture_collection(self, testdir, monkeypatch): + testdir.makepyfile(""" + import pytest + + @pytest.fixture(scope="module", + params=["A", + "B", + "C"]) + def A(request): + return request.param + + @pytest.fixture(scope="module", + params=["DDDDDDDDD", "EEEEEEEEEEEE", "FFFFFFFFFFF", "banansda"]) + def B(request, A): + return request.param -class TestRequestScopeAccess: - pytestmark = pytest.mark.parametrize(("scope", "ok", "error"),[ + def test_foo(B): + # Something funky is going on here. + # Despite specified seeds, on what is collected, + # sometimes we get unexpected passes. hashing B seems + # to help? + assert hash(B) or True + """) + monkeypatch.setenv("PYTHONHASHSEED", "1") + out1 = testdir.runpytest_subprocess("-v") + monkeypatch.setenv("PYTHONHASHSEED", "2") + out2 = testdir.runpytest_subprocess("-v") + out1 = [line for line in out1.outlines if line.startswith("test_deterministic_fixture_collection.py::test_foo")] + out2 = [line for line in out2.outlines if line.startswith("test_deterministic_fixture_collection.py::test_foo")] + assert len(out1) == 12 + assert out1 == out2 + + +class TestRequestScopeAccess(object): + pytestmark = pytest.mark.parametrize(("scope", "ok", "error"), [ ["session", "", "fspath class function module"], ["module", "module fspath", "cls function"], ["class", "module fspath cls", "function"], @@ -2349,7 +2613,7 @@ def myscoped(request): assert request.config def test_func(): pass - """ %(scope, ok.split(), error.split())) + """ % (scope, ok.split(), error.split())) reprec = testdir.inline_run("-l") reprec.assertoutcome(passed=1) @@ -2367,11 +2631,12 @@ def arg(request): assert request.config def test_func(arg): pass - """ %(scope, ok.split(), error.split())) + """ % (scope, ok.split(), error.split())) reprec = testdir.inline_run() reprec.assertoutcome(passed=1) -class TestErrors: + +class TestErrors(object): def test_subfactory_missing_funcarg(self, testdir): testdir.makepyfile(""" import pytest @@ -2399,13 +2664,13 @@ def f(): request.addfinalizer(f) return object() - l = [] + values = [] def test_1(fix1): - l.append(fix1) + values.append(fix1) def test_2(fix1): - l.append(fix1) + values.append(fix1) def test_3(): - assert l[0] != l[1] + assert values[0] != values[1] """) result = testdir.runpytest() result.stdout.fnmatch_lines(""" @@ -2416,8 +2681,6 @@ def test_3(): *3 pass*2 error* """) - - def test_setupfunc_missing_funcarg(self, testdir): testdir.makepyfile(""" import pytest @@ -2435,7 +2698,8 @@ def test_something(): "*1 error*", ]) -class TestShowFixtures: + +class TestShowFixtures(object): def test_funcarg_compat(self, testdir): config = testdir.parseconfigure("--funcargs") assert config.option.showfixtures @@ -2443,18 +2707,16 @@ def test_funcarg_compat(self, testdir): def test_show_fixtures(self, testdir): result = testdir.runpytest("--fixtures") result.stdout.fnmatch_lines([ - "*tmpdir*", - "*temporary directory*", - ] - ) + "*tmpdir*", + "*temporary directory*", + ]) def test_show_fixtures_verbose(self, testdir): result = testdir.runpytest("--fixtures", "-v") result.stdout.fnmatch_lines([ - "*tmpdir*--*tmpdir.py*", - "*temporary directory*", - ] - ) + "*tmpdir*--*tmpdir.py*", + "*temporary directory*", + ]) def test_show_fixtures_testmodule(self, testdir): p = testdir.makepyfile(''' @@ -2497,7 +2759,7 @@ def test_hello(): """) def test_show_fixtures_trimmed_doc(self, testdir): - p = testdir.makepyfile(''' + p = testdir.makepyfile(dedent(''' import pytest @pytest.fixture def arg1(): @@ -2513,9 +2775,9 @@ def arg2(): line2 """ - ''') + ''')) result = testdir.runpytest("--fixtures", p) - result.stdout.fnmatch_lines(""" + result.stdout.fnmatch_lines(dedent(""" * fixtures defined from test_show_fixtures_trimmed_doc * arg2 line1 @@ -2524,8 +2786,64 @@ def arg2(): line1 line2 - """) + """)) + + def test_show_fixtures_indented_doc(self, testdir): + p = testdir.makepyfile(dedent(''' + import pytest + @pytest.fixture + def fixture1(): + """ + line1 + indented line + """ + ''')) + result = testdir.runpytest("--fixtures", p) + result.stdout.fnmatch_lines(dedent(""" + * fixtures defined from test_show_fixtures_indented_doc * + fixture1 + line1 + indented line + """)) + def test_show_fixtures_indented_doc_first_line_unindented(self, testdir): + p = testdir.makepyfile(dedent(''' + import pytest + @pytest.fixture + def fixture1(): + """line1 + line2 + indented line + """ + ''')) + result = testdir.runpytest("--fixtures", p) + result.stdout.fnmatch_lines(dedent(""" + * fixtures defined from test_show_fixtures_indented_doc_first_line_unindented * + fixture1 + line1 + line2 + indented line + """)) + + def test_show_fixtures_indented_in_class(self, testdir): + p = testdir.makepyfile(dedent(''' + import pytest + class TestClass: + @pytest.fixture + def fixture1(self): + """line1 + line2 + indented line + """ + ''')) + result = testdir.runpytest("--fixtures", p) + result.stdout.fnmatch_lines(dedent(""" + * fixtures defined from test_show_fixtures_indented_in_class * + fixture1 + line1 + line2 + indented line + """)) def test_show_fixtures_different_files(self, testdir): """ @@ -2564,12 +2882,46 @@ def test_b(fix_b): Fixture B """) + def test_show_fixtures_with_same_name(self, testdir): + testdir.makeconftest(''' + import pytest + @pytest.fixture + def arg1(): + """Hello World in conftest.py""" + return "Hello World" + ''') + testdir.makepyfile(''' + def test_foo(arg1): + assert arg1 == "Hello World" + ''') + testdir.makepyfile(''' + import pytest + @pytest.fixture + def arg1(): + """Hi from test module""" + return "Hi" + def test_bar(arg1): + assert arg1 == "Hi" + ''') + result = testdir.runpytest("--fixtures") + result.stdout.fnmatch_lines(''' + * fixtures defined from conftest * + arg1 + Hello World in conftest.py + + * fixtures defined from test_show_fixtures_with_same_name * + arg1 + Hi from test module + ''') + + +@pytest.mark.parametrize('flavor', ['fixture', 'yield_fixture']) +class TestContextManagerFixtureFuncs(object): -class TestContextManagerFixtureFuncs: - def test_simple(self, testdir): + def test_simple(self, testdir, flavor): testdir.makepyfile(""" import pytest - @pytest.yield_fixture + @pytest.{flavor} def arg1(): print ("setup") yield 1 @@ -2579,7 +2931,7 @@ def test_1(arg1): def test_2(arg1): print ("test2 %s" % arg1) assert 0 - """) + """.format(flavor=flavor)) result = testdir.runpytest("-s") result.stdout.fnmatch_lines(""" *setup* @@ -2590,10 +2942,10 @@ def test_2(arg1): *teardown* """) - def test_scoped(self, testdir): + def test_scoped(self, testdir, flavor): testdir.makepyfile(""" import pytest - @pytest.yield_fixture(scope="module") + @pytest.{flavor}(scope="module") def arg1(): print ("setup") yield 1 @@ -2602,7 +2954,7 @@ def test_1(arg1): print ("test1 %s" % arg1) def test_2(arg1): print ("test2 %s" % arg1) - """) + """.format(flavor=flavor)) result = testdir.runpytest("-s") result.stdout.fnmatch_lines(""" *setup* @@ -2611,83 +2963,209 @@ def test_2(arg1): *teardown* """) - def test_setup_exception(self, testdir): + def test_setup_exception(self, testdir, flavor): testdir.makepyfile(""" import pytest - @pytest.yield_fixture(scope="module") + @pytest.{flavor}(scope="module") def arg1(): pytest.fail("setup") yield 1 def test_1(arg1): pass - """) + """.format(flavor=flavor)) result = testdir.runpytest("-s") result.stdout.fnmatch_lines(""" *pytest.fail*setup* *1 error* """) - def test_teardown_exception(self, testdir): + def test_teardown_exception(self, testdir, flavor): testdir.makepyfile(""" import pytest - @pytest.yield_fixture(scope="module") + @pytest.{flavor}(scope="module") def arg1(): yield 1 pytest.fail("teardown") def test_1(arg1): pass - """) + """.format(flavor=flavor)) result = testdir.runpytest("-s") result.stdout.fnmatch_lines(""" *pytest.fail*teardown* *1 passed*1 error* """) - def test_yields_more_than_one(self, testdir): + def test_yields_more_than_one(self, testdir, flavor): testdir.makepyfile(""" import pytest - @pytest.yield_fixture(scope="module") + @pytest.{flavor}(scope="module") def arg1(): yield 1 yield 2 def test_1(arg1): pass - """) + """.format(flavor=flavor)) result = testdir.runpytest("-s") result.stdout.fnmatch_lines(""" *fixture function* *test_yields*:2* """) - - def test_no_yield(self, testdir): + def test_custom_name(self, testdir, flavor): testdir.makepyfile(""" import pytest - @pytest.yield_fixture(scope="module") + @pytest.{flavor}(name='meow') def arg1(): - return 1 - def test_1(arg1): - pass - """) + return 'mew' + def test_1(meow): + print(meow) + """.format(flavor=flavor)) result = testdir.runpytest("-s") - result.stdout.fnmatch_lines(""" - *yield_fixture*requires*yield* - *yield_fixture* - *def arg1* - """) + result.stdout.fnmatch_lines("*mew*") - def test_yield_not_allowed_in_non_yield(self, testdir): - testdir.makepyfile(""" + +class TestParameterizedSubRequest(object): + def test_call_from_fixture(self, testdir): + testfile = testdir.makepyfile(""" import pytest - @pytest.fixture(scope="module") - def arg1(): - yield 1 - def test_1(arg1): + + @pytest.fixture(params=[0, 1, 2]) + def fix_with_param(request): + return request.param + + @pytest.fixture + def get_named_fixture(request): + return request.getfixturevalue('fix_with_param') + + def test_foo(request, get_named_fixture): pass - """) - result = testdir.runpytest("-s") + """) + result = testdir.runpytest() result.stdout.fnmatch_lines(""" - *fixture*cannot use*yield* - *def arg1* - """) + E*Failed: The requested fixture has no parameter defined for the current test. + E* + E*Requested fixture 'fix_with_param' defined in: + E*{0}:4 + E*Requested here: + E*{1}:9 + *1 error* + """.format(testfile.basename, testfile.basename)) + def test_call_from_test(self, testdir): + testfile = testdir.makepyfile(""" + import pytest + + @pytest.fixture(params=[0, 1, 2]) + def fix_with_param(request): + return request.param + + def test_foo(request): + request.getfixturevalue('fix_with_param') + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines(""" + E*Failed: The requested fixture has no parameter defined for the current test. + E* + E*Requested fixture 'fix_with_param' defined in: + E*{0}:4 + E*Requested here: + E*{1}:8 + *1 failed* + """.format(testfile.basename, testfile.basename)) + + def test_external_fixture(self, testdir): + conffile = testdir.makeconftest(""" + import pytest + + @pytest.fixture(params=[0, 1, 2]) + def fix_with_param(request): + return request.param + """) + + testfile = testdir.makepyfile(""" + def test_foo(request): + request.getfixturevalue('fix_with_param') + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines(""" + E*Failed: The requested fixture has no parameter defined for the current test. + E* + E*Requested fixture 'fix_with_param' defined in: + E*{0}:4 + E*Requested here: + E*{1}:2 + *1 failed* + """.format(conffile.basename, testfile.basename)) + + def test_non_relative_path(self, testdir): + tests_dir = testdir.mkdir('tests') + fixdir = testdir.mkdir('fixtures') + fixfile = fixdir.join("fix.py") + fixfile.write(_pytest._code.Source(""" + import pytest + + @pytest.fixture(params=[0, 1, 2]) + def fix_with_param(request): + return request.param + """)) + + testfile = tests_dir.join("test_foos.py") + testfile.write(_pytest._code.Source(""" + from fix import fix_with_param + + def test_foo(request): + request.getfixturevalue('fix_with_param') + """)) + + tests_dir.chdir() + testdir.syspathinsert(fixdir) + result = testdir.runpytest() + result.stdout.fnmatch_lines(""" + E*Failed: The requested fixture has no parameter defined for the current test. + E* + E*Requested fixture 'fix_with_param' defined in: + E*{0}:5 + E*Requested here: + E*{1}:5 + *1 failed* + """.format(fixfile.strpath, testfile.basename)) + + +def test_pytest_fixture_setup_and_post_finalizer_hook(testdir): + testdir.makeconftest(""" + from __future__ import print_function + def pytest_fixture_setup(fixturedef, request): + print('ROOT setup hook called for {0} from {1}'.format(fixturedef.argname, request.node.name)) + def pytest_fixture_post_finalizer(fixturedef, request): + print('ROOT finalizer hook called for {0} from {1}'.format(fixturedef.argname, request.node.name)) + """) + testdir.makepyfile(**{ + 'tests/conftest.py': """ + from __future__ import print_function + def pytest_fixture_setup(fixturedef, request): + print('TESTS setup hook called for {0} from {1}'.format(fixturedef.argname, request.node.name)) + def pytest_fixture_post_finalizer(fixturedef, request): + print('TESTS finalizer hook called for {0} from {1}'.format(fixturedef.argname, request.node.name)) + """, + 'tests/test_hooks.py': """ + from __future__ import print_function + import pytest + + @pytest.fixture() + def my_fixture(): + return 'some' + + def test_func(my_fixture): + print('TEST test_func') + assert my_fixture == 'some' + """ + }) + result = testdir.runpytest("-s") + assert result.ret == 0 + result.stdout.fnmatch_lines([ + "*TESTS setup hook called for my_fixture from test_func*", + "*ROOT setup hook called for my_fixture from test_func*", + "*TEST test_func*", + "*TESTS finalizer hook called for my_fixture from test_func*", + "*ROOT finalizer hook called for my_fixture from test_func*", + ]) diff --git a/testing/python/integration.py b/testing/python/integration.py index dea86f942fdba0f..6ea29fa98b9da60 100644 --- a/testing/python/integration.py +++ b/testing/python/integration.py @@ -3,8 +3,8 @@ from _pytest import runner -class TestOEJSKITSpecials: - def test_funcarg_non_pycollectobj(self, testdir): # rough jstests usage +class TestOEJSKITSpecials(object): + def test_funcarg_non_pycollectobj(self, testdir): # rough jstests usage testdir.makeconftest(""" import pytest def pytest_pycollect_makeitem(collector, name, obj): @@ -15,9 +15,11 @@ def reportinfo(self): return self.fspath, 3, "xyz" """) modcol = testdir.getmodulecol(""" - def pytest_funcarg__arg1(request): + import pytest + @pytest.fixture + def arg1(request): return 42 - class MyClass: + class MyClass(object): pass """) # this hook finds funcarg factories @@ -28,7 +30,7 @@ class MyClass: pytest._fillfuncargs(clscol) assert clscol.funcargs['arg1'] == 42 - def test_autouse_fixture(self, testdir): # rough jstests usage + def test_autouse_fixture(self, testdir): # rough jstests usage testdir.makeconftest(""" import pytest def pytest_pycollect_makeitem(collector, name, obj): @@ -43,9 +45,10 @@ def reportinfo(self): @pytest.fixture(autouse=True) def hello(): pass - def pytest_funcarg__arg1(request): + @pytest.fixture + def arg1(request): return 42 - class MyClass: + class MyClass(object): pass """) # this hook finds funcarg factories @@ -60,10 +63,12 @@ class MyClass: def test_wrapped_getfslineno(): def func(): pass + def wrap(f): func.__wrapped__ = f func.patchings = ["qwe"] return func + @wrap def wrapped_func(x, y, z): pass @@ -71,33 +76,42 @@ def wrapped_func(x, y, z): fs2, lineno2 = python.getfslineno(wrap) assert lineno > lineno2, "getfslineno does not unwrap correctly" -class TestMockDecoration: + +class TestMockDecoration(object): def test_wrapped_getfuncargnames(self): - from _pytest.python import getfuncargnames + from _pytest.compat import getfuncargnames + def wrap(f): + def func(): pass + func.__wrapped__ = f return func + @wrap def f(x): pass - l = getfuncargnames(f) - assert l == ("x",) + + values = getfuncargnames(f) + assert values == ("x",) def test_wrapped_getfuncargnames_patching(self): - from _pytest.python import getfuncargnames + from _pytest.compat import getfuncargnames + def wrap(f): def func(): pass func.__wrapped__ = f func.patchings = ["qwe"] return func + @wrap def f(x, y, z): pass - l = getfuncargnames(f) - assert l == ("y", "z") + + values = getfuncargnames(f) + assert values == ("y", "z") def test_unittest_mock(self, testdir): pytest.importorskip("unittest.mock") @@ -160,7 +174,7 @@ def test_someting(normpath, abspath, tmpdir): reprec.assertoutcome(passed=2) calls = reprec.getcalls("pytest_runtest_logreport") funcnames = [call.report.location[2] for call in calls - if call.report.when == "call"] + if call.report.when == "call"] assert funcnames == ["T.test_hello", "test_someting"] def test_mock_sorting(self, testdir): @@ -194,7 +208,7 @@ def test_mock_double_patch_issue473(self, testdir): @patch('os.getcwd') @patch('os.path') @mark.slow - class TestSimple: + class TestSimple(object): def test_simple_thing(self, mock_path, mock_getcwd): pass """) @@ -202,7 +216,7 @@ def test_simple_thing(self, mock_path, mock_getcwd): reprec.assertoutcome(passed=1) -class TestReRunTests: +class TestReRunTests(object): def test_rerun(self, testdir): testdir.makeconftest(""" from _pytest.runner import runtestprotocol @@ -233,12 +247,13 @@ def test_fix(fix): *2 passed* """) + def test_pytestconfig_is_session_scoped(): - from _pytest.python import pytestconfig + from _pytest.fixtures import pytestconfig assert pytestconfig._pytestfixturefunction.scope == "session" -class TestNoselikeTestAttribute: +class TestNoselikeTestAttribute(object): def test_module_with_global_test(self, testdir): testdir.makepyfile(""" __test__ = False @@ -257,7 +272,7 @@ def test_func(): pass test_func.__test__ = False - class TestSome: + class TestSome(object): __test__ = False def test_method(self): pass @@ -315,7 +330,7 @@ def test_blah(self): @pytest.mark.issue351 -class TestParameterize: +class TestParameterize(object): def test_idfn_marker(self, testdir): testdir.makepyfile(""" diff --git a/testing/python/metafunc.py b/testing/python/metafunc.py index faa687f40565e3c..2ffb7bb5da2a1c3 100644 --- a/testing/python/metafunc.py +++ b/testing/python/metafunc.py @@ -1,32 +1,43 @@ # -*- coding: utf-8 -*- import re +import sys import _pytest._code import py import pytest -from _pytest import python as funcargs +from _pytest import python, fixtures -class TestMetafunc: +import hypothesis +from hypothesis import strategies + +PY3 = sys.version_info >= (3, 0) + + +class TestMetafunc(object): def Metafunc(self, func): # the unit tests of this class check if things work correctly # on the funcarg level, so we don't need a full blown # initiliazation - class FixtureInfo: + class FixtureInfo(object): name2fixturedefs = None + def __init__(self, names): self.names_closure = names - names = funcargs.getfuncargnames(func) + + names = fixtures.getfuncargnames(func) fixtureinfo = FixtureInfo(names) - return funcargs.Metafunc(func, fixtureinfo, None) + return python.Metafunc(func, fixtureinfo, None) def test_no_funcargs(self, testdir): - def function(): pass + def function(): + pass metafunc = self.Metafunc(function) assert not metafunc.fixturenames repr(metafunc._calls) def test_function_basic(self): - def func(arg1, arg2="qwe"): pass + def func(arg1, arg2="qwe"): + pass metafunc = self.Metafunc(func) assert len(metafunc.fixturenames) == 1 assert 'arg1' in metafunc.fixturenames @@ -34,7 +45,8 @@ def func(arg1, arg2="qwe"): pass assert metafunc.cls is None def test_addcall_no_args(self): - def func(arg1): pass + def func(arg1): + pass metafunc = self.Metafunc(func) metafunc.addcall() assert len(metafunc._calls) == 1 @@ -43,7 +55,8 @@ def func(arg1): pass assert not hasattr(call, 'param') def test_addcall_id(self): - def func(arg1): pass + def func(arg1): + pass metafunc = self.Metafunc(func) pytest.raises(ValueError, "metafunc.addcall(id=None)") @@ -56,9 +69,13 @@ def func(arg1): pass assert metafunc._calls[1].id == "2" def test_addcall_param(self): - def func(arg1): pass + def func(arg1): + pass metafunc = self.Metafunc(func) - class obj: pass + + class obj(object): + pass + metafunc.addcall(param=obj) metafunc.addcall(param=obj) metafunc.addcall(param=1) @@ -68,9 +85,14 @@ class obj: pass assert metafunc._calls[2].getparam("arg1") == 1 def test_addcall_funcargs(self): - def func(x): pass + def func(x): + pass + metafunc = self.Metafunc(func) - class obj: pass + + class obj(object): + pass + metafunc.addcall(funcargs={"x": 2}) metafunc.addcall(funcargs={"x": 3}) pytest.raises(pytest.fail.Exception, "metafunc.addcall({'xyz': 0})") @@ -80,40 +102,72 @@ class obj: pass assert not hasattr(metafunc._calls[1], 'param') def test_parametrize_error(self): - def func(x, y): pass + def func(x, y): + pass + metafunc = self.Metafunc(func) + metafunc.parametrize("x", [1, 2]) + pytest.raises(ValueError, lambda: metafunc.parametrize("x", [5, 6])) + pytest.raises(ValueError, lambda: metafunc.parametrize("x", [5, 6])) + metafunc.parametrize("y", [1, 2]) + pytest.raises(ValueError, lambda: metafunc.parametrize("y", [5, 6])) + pytest.raises(ValueError, lambda: metafunc.parametrize("y", [5, 6])) + + def test_parametrize_bad_scope(self, testdir): + def func(x): + pass metafunc = self.Metafunc(func) - metafunc.parametrize("x", [1,2]) - pytest.raises(ValueError, lambda: metafunc.parametrize("x", [5,6])) - pytest.raises(ValueError, lambda: metafunc.parametrize("x", [5,6])) - metafunc.parametrize("y", [1,2]) - pytest.raises(ValueError, lambda: metafunc.parametrize("y", [5,6])) - pytest.raises(ValueError, lambda: metafunc.parametrize("y", [5,6])) + try: + metafunc.parametrize("x", [1], scope='doggy') + except ValueError as ve: + assert "has an unsupported scope value 'doggy'" in str(ve) def test_parametrize_and_id(self): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) - metafunc.parametrize("x", [1,2], ids=['basic', 'advanced']) + metafunc.parametrize("x", [1, 2], ids=['basic', 'advanced']) metafunc.parametrize("y", ["abc", "def"]) ids = [x.id for x in metafunc._calls] assert ids == ["basic-abc", "basic-def", "advanced-abc", "advanced-def"] + def test_parametrize_and_id_unicode(self): + """Allow unicode strings for "ids" parameter in Python 2 (##1905)""" + def func(x): + pass + metafunc = self.Metafunc(func) + metafunc.parametrize("x", [1, 2], ids=[u'basic', u'advanced']) + ids = [x.id for x in metafunc._calls] + assert ids == [u"basic", u"advanced"] + def test_parametrize_with_wrong_number_of_ids(self, testdir): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) pytest.raises(ValueError, lambda: - metafunc.parametrize("x", [1,2], ids=['basic'])) + metafunc.parametrize("x", [1, 2], ids=['basic'])) pytest.raises(ValueError, lambda: - metafunc.parametrize(("x","y"), [("abc", "def"), - ("ghi", "jkl")], ids=["one"])) + metafunc.parametrize(("x", "y"), [("abc", "def"), + ("ghi", "jkl")], ids=["one"])) + + @pytest.mark.issue510 + def test_parametrize_empty_list(self): + def func(y): + pass + metafunc = self.Metafunc(func) + metafunc.parametrize("y", []) + assert 'skip' == metafunc._calls[0].marks[0].name def test_parametrize_with_userobjects(self): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) - class A: + + class A(object): pass + metafunc.parametrize("x", [A(), A()]) metafunc.parametrize("y", list("ab")) assert metafunc._calls[0].id == "x0-a" @@ -121,20 +175,45 @@ class A: assert metafunc._calls[2].id == "x1-a" assert metafunc._calls[3].id == "x1-b" - @pytest.mark.skipif('sys.version_info[0] >= 3') - def test_unicode_idval_python2(self): - """unittest for the expected behavior to obtain ids for parametrized - unicode values in Python 2: if convertible to ascii, they should appear - as ascii values, otherwise fallback to hide the value behind the name - of the parametrized variable name. #1086 + @hypothesis.given(strategies.text() | strategies.binary()) + def test_idval_hypothesis(self, value): + from _pytest.python import _idval + escaped = _idval(value, 'a', 6, None) + assert isinstance(escaped, str) + if PY3: + escaped.encode('ascii') + else: + escaped.decode('ascii') + + def test_unicode_idval(self): + """This tests that Unicode strings outside the ASCII character set get + escaped, using byte escapes if they're in that range or unicode + escapes if they're not. + """ from _pytest.python import _idval values = [ - (u'', ''), - (u'ascii', 'ascii'), - (u'ação', 'a6'), - (u'josé@blah.com', 'a6'), - (u'δοκ.ιμή@παράδειγμα.δοκιμή', 'a6'), + ( + u'', + '' + ), + ( + u'ascii', + 'ascii' + ), + ( + u'ação', + 'a\\xe7\\xe3o' + ), + ( + u'josé@blah.com', + 'jos\\xe9@blah.com' + ), + ( + u'δοκ.ιμή@παράδειγμα.δοκιμή', + '\\u03b4\\u03bf\\u03ba.\\u03b9\\u03bc\\u03ae@\\u03c0\\u03b1\\u03c1\\u03ac\\u03b4\\u03b5\\u03b9\\u03b3' + '\\u03bc\\u03b1.\\u03b4\\u03bf\\u03ba\\u03b9\\u03bc\\u03ae' + ), ] for val, expected in values: assert _idval(val, 'a', 6, None) == expected @@ -159,37 +238,40 @@ def test_bytes_idval(self): @pytest.mark.issue250 def test_idmaker_autoname(self): from _pytest.python import idmaker - result = idmaker(("a", "b"), [("string", 1.0), - ("st-ring", 2.0)]) + result = idmaker(("a", "b"), [pytest.param("string", 1.0), + pytest.param("st-ring", 2.0)]) assert result == ["string-1.0", "st-ring-2.0"] - result = idmaker(("a", "b"), [(object(), 1.0), - (object(), object())]) + result = idmaker(("a", "b"), [pytest.param(object(), 1.0), + pytest.param(object(), object())]) assert result == ["a0-1.0", "a1-b1"] # unicode mixing, issue250 - result = idmaker((py.builtin._totext("a"), "b"), [({}, b'\xc3\xb4')]) + result = idmaker( + (py.builtin._totext("a"), "b"), + [pytest.param({}, b'\xc3\xb4')]) assert result == ['a0-\\xc3\\xb4'] def test_idmaker_with_bytes_regex(self): from _pytest.python import idmaker - result = idmaker(("a"), [(re.compile(b'foo'), 1.0)]) + result = idmaker(("a"), [pytest.param(re.compile(b'foo'), 1.0)]) assert result == ["foo"] def test_idmaker_native_strings(self): from _pytest.python import idmaker totext = py.builtin._totext - result = idmaker(("a", "b"), [(1.0, -1.1), - (2, -202), - ("three", "three hundred"), - (True, False), - (None, None), - (re.compile('foo'), re.compile('bar')), - (str, int), - (list("six"), [66, 66]), - (set([7]), set("seven")), - (tuple("eight"), (8, -8, 8)), - (b'\xc3\xb4', b"name"), - (b'\xc3\xb4', totext("other")), + result = idmaker(("a", "b"), [ + pytest.param(1.0, -1.1), + pytest.param(2, -202), + pytest.param("three", "three hundred"), + pytest.param(True, False), + pytest.param(None, None), + pytest.param(re.compile('foo'), re.compile('bar')), + pytest.param(str, int), + pytest.param(list("six"), [66, 66]), + pytest.param(set([7]), set("seven")), + pytest.param(tuple("eight"), (8, -8, 8)), + pytest.param(b'\xc3\xb4', b"name"), + pytest.param(b'\xc3\xb4', totext("other")), ]) assert result == ["1.0--1.1", "2--202", @@ -209,60 +291,126 @@ def test_idmaker_enum(self): from _pytest.python import idmaker enum = pytest.importorskip("enum") e = enum.Enum("Foo", "one, two") - result = idmaker(("a", "b"), [(e.one, e.two)]) + result = idmaker(("a", "b"), [pytest.param(e.one, e.two)]) assert result == ["Foo.one-Foo.two"] @pytest.mark.issue351 def test_idmaker_idfn(self): from _pytest.python import idmaker + def ids(val): if isinstance(val, Exception): return repr(val) - result = idmaker(("a", "b"), [(10.0, IndexError()), - (20, KeyError()), - ("three", [1, 2, 3]), + result = idmaker(("a", "b"), [ + pytest.param(10.0, IndexError()), + pytest.param(20, KeyError()), + pytest.param("three", [1, 2, 3]), ], idfn=ids) assert result == ["10.0-IndexError()", "20-KeyError()", "three-b2", - ] + ] @pytest.mark.issue351 def test_idmaker_idfn_unique_names(self): from _pytest.python import idmaker + def ids(val): return 'a' - result = idmaker(("a", "b"), [(10.0, IndexError()), - (20, KeyError()), - ("three", [1, 2, 3]), - ], idfn=ids) - assert result == ["0a-a", - "1a-a", - "2a-a", - ] + result = idmaker(("a", "b"), [pytest.param(10.0, IndexError()), + pytest.param(20, KeyError()), + pytest.param("three", [1, 2, 3]), + ], idfn=ids) + assert result == ["a-a0", + "a-a1", + "a-a2", + ] @pytest.mark.issue351 def test_idmaker_idfn_exception(self): from _pytest.python import idmaker + from _pytest.recwarn import WarningsRecorder + + class BadIdsException(Exception): + pass + def ids(val): - raise Exception("bad code") + raise BadIdsException("ids raised") + + rec = WarningsRecorder() + with rec: + idmaker(("a", "b"), [ + pytest.param(10.0, IndexError()), + pytest.param(20, KeyError()), + pytest.param("three", [1, 2, 3]), + ], idfn=ids) + + assert [str(i.message) for i in rec.list] == [ + "Raised while trying to determine id of parameter a at position 0." + "\nUpdate your code as this will raise an error in pytest-4.0.", + "Raised while trying to determine id of parameter b at position 0." + "\nUpdate your code as this will raise an error in pytest-4.0.", + "Raised while trying to determine id of parameter a at position 1." + "\nUpdate your code as this will raise an error in pytest-4.0.", + "Raised while trying to determine id of parameter b at position 1." + "\nUpdate your code as this will raise an error in pytest-4.0.", + "Raised while trying to determine id of parameter a at position 2." + "\nUpdate your code as this will raise an error in pytest-4.0.", + "Raised while trying to determine id of parameter b at position 2." + "\nUpdate your code as this will raise an error in pytest-4.0.", + ] - result = idmaker(("a", "b"), [(10.0, IndexError()), - (20, KeyError()), - ("three", [1, 2, 3]), - ], idfn=ids) - assert result == ["10.0-b0", - "20-b1", - "three-b2", - ] + def test_parametrize_ids_exception(self, testdir): + """ + :param testdir: the instance of Testdir class, a temporary + test directory. + """ + testdir.makepyfile(""" + import pytest + + def ids(arg): + raise Exception("bad ids") + + @pytest.mark.parametrize("arg", ["a", "b"], ids=ids) + def test_foo(arg): + pass + """) + with pytest.warns(DeprecationWarning): + result = testdir.runpytest("--collect-only") + result.stdout.fnmatch_lines([ + "", + " ", + " ", + ]) + + def test_idmaker_with_ids(self): + from _pytest.python import idmaker + result = idmaker(("a", "b"), [pytest.param(1, 2), + pytest.param(3, 4)], + ids=["a", None]) + assert result == ["a", "3-4"] + + def test_idmaker_with_paramset_id(self): + from _pytest.python import idmaker + result = idmaker(("a", "b"), [pytest.param(1, 2, id="me"), + pytest.param(3, 4, id="you")], + ids=["a", None]) + assert result == ["me", "you"] + + def test_idmaker_with_ids_unique_names(self): + from _pytest.python import idmaker + result = idmaker(("a"), map(pytest.param, [1, 2, 3, 4, 5]), + ids=["a", "a", "b", "c", "b"]) + assert result == ["a0", "a1", "b0", "c", "b1"] def test_addcall_and_parametrize(self): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) metafunc.addcall({'x': 1}) - metafunc.parametrize('y', [2,3]) + metafunc.parametrize('y', [2, 3]) assert len(metafunc._calls) == 2 assert metafunc._calls[0].funcargs == {'x': 1, 'y': 2} assert metafunc._calls[1].funcargs == {'x': 1, 'y': 3} @@ -271,19 +419,21 @@ def func(x, y): pass @pytest.mark.issue714 def test_parametrize_indirect(self): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) metafunc.parametrize('x', [1], indirect=True) - metafunc.parametrize('y', [2,3], indirect=True) + metafunc.parametrize('y', [2, 3], indirect=True) assert len(metafunc._calls) == 2 assert metafunc._calls[0].funcargs == {} assert metafunc._calls[1].funcargs == {} - assert metafunc._calls[0].params == dict(x=1,y=2) - assert metafunc._calls[1].params == dict(x=1,y=3) + assert metafunc._calls[0].params == dict(x=1, y=2) + assert metafunc._calls[1].params == dict(x=1, y=3) @pytest.mark.issue714 def test_parametrize_indirect_list(self): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) metafunc.parametrize('x, y', [('a', 'b')], indirect=['x']) assert metafunc._calls[0].funcargs == dict(y='b') @@ -291,7 +441,8 @@ def func(x, y): pass @pytest.mark.issue714 def test_parametrize_indirect_list_all(self): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) metafunc.parametrize('x, y', [('a', 'b')], indirect=['x', 'y']) assert metafunc._calls[0].funcargs == {} @@ -299,7 +450,8 @@ def func(x, y): pass @pytest.mark.issue714 def test_parametrize_indirect_list_empty(self): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) metafunc.parametrize('x, y', [('a', 'b')], indirect=[]) assert metafunc._calls[0].funcargs == dict(x='a', y='b') @@ -337,7 +489,8 @@ def test_simple(x,y): @pytest.mark.issue714 def test_parametrize_indirect_list_error(self, testdir): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) with pytest.raises(ValueError): metafunc.parametrize('x, y', [('a', 'b')], indirect=['x', 'z']) @@ -358,7 +511,7 @@ def test_simple(x): """) result = testdir.runpytest("--collect-only") result.stdout.fnmatch_lines([ - "*uses no fixture 'y'*", + "*uses no argument 'y'*", ]) @pytest.mark.issue714 @@ -381,6 +534,23 @@ def test_simple(x): "*uses no fixture 'y'*", ]) + @pytest.mark.issue714 + def test_parametrize_indirect_uses_no_fixture_error_indirect_string(self, testdir): + testdir.makepyfile(""" + import pytest + @pytest.fixture(scope='function') + def x(request): + return request.param * 3 + + @pytest.mark.parametrize('x, y', [('a', 'b')], indirect='y') + def test_simple(x): + assert len(x) == 3 + """) + result = testdir.runpytest("--collect-only") + result.stdout.fnmatch_lines([ + "*uses no fixture 'y'*", + ]) + @pytest.mark.issue714 def test_parametrize_indirect_uses_no_fixture_error_indirect_list(self, testdir): testdir.makepyfile(""" @@ -389,7 +559,7 @@ def test_parametrize_indirect_uses_no_fixture_error_indirect_list(self, testdir) def x(request): return request.param * 3 - @pytest.mark.parametrize('x, y', [('a', 'b')], indirect=['x']) + @pytest.mark.parametrize('x, y', [('a', 'b')], indirect=['y']) def test_simple(x): assert len(x) == 3 """) @@ -398,27 +568,45 @@ def test_simple(x): "*uses no fixture 'y'*", ]) + @pytest.mark.issue714 + def test_parametrize_argument_not_in_indirect_list(self, testdir): + testdir.makepyfile(""" + import pytest + @pytest.fixture(scope='function') + def x(request): + return request.param * 3 + + @pytest.mark.parametrize('x, y', [('a', 'b')], indirect=['x']) + def test_simple(x): + assert len(x) == 3 + """) + result = testdir.runpytest("--collect-only") + result.stdout.fnmatch_lines([ + "*uses no argument 'y'*", + ]) + def test_addcalls_and_parametrize_indirect(self): - def func(x, y): pass + def func(x, y): + pass metafunc = self.Metafunc(func) metafunc.addcall(param="123") metafunc.parametrize('x', [1], indirect=True) - metafunc.parametrize('y', [2,3], indirect=True) + metafunc.parametrize('y', [2, 3], indirect=True) assert len(metafunc._calls) == 2 assert metafunc._calls[0].funcargs == {} assert metafunc._calls[1].funcargs == {} - assert metafunc._calls[0].params == dict(x=1,y=2) - assert metafunc._calls[1].params == dict(x=1,y=3) + assert metafunc._calls[0].params == dict(x=1, y=2) + assert metafunc._calls[1].params == dict(x=1, y=3) def test_parametrize_functional(self, testdir): testdir.makepyfile(""" + import pytest def pytest_generate_tests(metafunc): metafunc.parametrize('x', [1,2], indirect=True) metafunc.parametrize('y', [2]) - def pytest_funcarg__x(request): + @pytest.fixture + def x(request): return request.param * 10 - #def pytest_funcarg__y(request): - # return request.param def test_simple(x,y): assert x in (10,20) @@ -433,7 +621,7 @@ def test_simple(x,y): def test_parametrize_onearg(self): metafunc = self.Metafunc(lambda x: None) - metafunc.parametrize("x", [1,2]) + metafunc.parametrize("x", [1, 2]) assert len(metafunc._calls) == 2 assert metafunc._calls[0].funcargs == dict(x=1) assert metafunc._calls[0].id == "1" @@ -442,15 +630,15 @@ def test_parametrize_onearg(self): def test_parametrize_onearg_indirect(self): metafunc = self.Metafunc(lambda x: None) - metafunc.parametrize("x", [1,2], indirect=True) + metafunc.parametrize("x", [1, 2], indirect=True) assert metafunc._calls[0].params == dict(x=1) assert metafunc._calls[0].id == "1" assert metafunc._calls[1].params == dict(x=2) assert metafunc._calls[1].id == "2" def test_parametrize_twoargs(self): - metafunc = self.Metafunc(lambda x,y: None) - metafunc.parametrize(("x", "y"), [(1,2), (3,4)]) + metafunc = self.Metafunc(lambda x, y: None) + metafunc.parametrize(("x", "y"), [(1, 2), (3, 4)]) assert len(metafunc._calls) == 2 assert metafunc._calls[0].funcargs == dict(x=1, y=2) assert metafunc._calls[0].id == "1-2" @@ -463,7 +651,7 @@ def test_parametrize_multiple_times(self, testdir): pytestmark = pytest.mark.parametrize("x", [1,2]) def test_func(x): assert 0, x - class TestClass: + class TestClass(object): pytestmark = pytest.mark.parametrize("y", [3,4]) def test_meth(self, x, y): assert 0, x @@ -521,20 +709,24 @@ def test_3(self, arg, arg2): """) def test_format_args(self): - def function1(): pass - assert funcargs._format_args(function1) == '()' + def function1(): + pass + assert fixtures._format_args(function1) == '()' - def function2(arg1): pass - assert funcargs._format_args(function2) == "(arg1)" + def function2(arg1): + pass + assert fixtures._format_args(function2) == "(arg1)" - def function3(arg1, arg2="qwe"): pass - assert funcargs._format_args(function3) == "(arg1, arg2='qwe')" + def function3(arg1, arg2="qwe"): + pass + assert fixtures._format_args(function3) == "(arg1, arg2='qwe')" - def function4(arg1, *args, **kwargs): pass - assert funcargs._format_args(function4) == "(arg1, *args, **kwargs)" + def function4(arg1, *args, **kwargs): + pass + assert fixtures._format_args(function4) == "(arg1, *args, **kwargs)" -class TestMetafuncFunctional: +class TestMetafuncFunctional(object): def test_attributes(self, testdir): p = testdir.makepyfile(""" # assumes that generate/provide runs in the same process @@ -542,7 +734,8 @@ def test_attributes(self, testdir): def pytest_generate_tests(metafunc): metafunc.addcall(param=metafunc) - def pytest_funcarg__metafunc(request): + @pytest.fixture + def metafunc(request): assert request._pyfuncitem._genid == "0" return request.param @@ -552,7 +745,7 @@ def test_function(metafunc, pytestconfig): assert metafunc.function == test_function assert metafunc.cls is None - class TestClass: + class TestClass(object): def test_method(self, metafunc, pytestconfig): assert metafunc.config == pytestconfig assert metafunc.module.__name__ == __name__ @@ -577,7 +770,7 @@ def pytest_generate_tests(metafunc): def pytest_generate_tests(metafunc): metafunc.addcall(funcargs=dict(arg1=1, arg2=1)) - class TestClass: + class TestClass(object): def test_myfunc(self, arg1, arg2): assert arg1 == arg2 """) @@ -594,7 +787,9 @@ def pytest_generate_tests(metafunc): metafunc.addcall(param=10) metafunc.addcall(param=20) - def pytest_funcarg__arg1(request): + import pytest + @pytest.fixture + def arg1(request): return request.param def test_func1(arg1): @@ -615,14 +810,13 @@ def test_noself_in_method(self, testdir): def pytest_generate_tests(metafunc): assert 'xyz' not in metafunc.fixturenames - class TestHello: + class TestHello(object): def test_hello(xyz): pass """) result = testdir.runpytest(p) result.assert_outcomes(passed=1) - def test_generate_plugin_and_module(self, testdir): testdir.makeconftest(""" def pytest_generate_tests(metafunc): @@ -633,12 +827,15 @@ def pytest_generate_tests(metafunc): def pytest_generate_tests(metafunc): metafunc.addcall(param=(1,1), id="hello") - def pytest_funcarg__arg1(request): + import pytest + @pytest.fixture + def arg1(request): return request.param[0] - def pytest_funcarg__arg2(request): + @pytest.fixture + def arg2(request): return request.param[1] - class TestClass: + class TestClass(object): def test_myfunc(self, arg1, arg2): assert arg1 == arg2 """) @@ -651,7 +848,7 @@ def test_myfunc(self, arg1, arg2): def test_generate_tests_in_class(self, testdir): p = testdir.makepyfile(""" - class TestClass: + class TestClass(object): def pytest_generate_tests(self, metafunc): metafunc.addcall(funcargs={'hello': 'world'}, id="hello") @@ -670,7 +867,7 @@ def pytest_generate_tests(metafunc): metafunc.addcall({'arg1': 10}) metafunc.addcall({'arg1': 20}) - class TestClass: + class TestClass(object): def test_func(self, arg1): assert not hasattr(self, 'x') self.x = 1 @@ -687,7 +884,7 @@ def test_issue28_setup_method_in_generate_tests(self, testdir): def pytest_generate_tests(metafunc): metafunc.addcall({'arg1': 1}) - class TestClass: + class TestClass(object): def test_method(self, arg1): assert arg1 == self.val def setup_method(self, func): @@ -713,17 +910,20 @@ def test_hello(arg1, arg2): "*4 failed*", ]) - def test_parametrize_and_inner_getfuncargvalue(self, testdir): + def test_parametrize_and_inner_getfixturevalue(self, testdir): p = testdir.makepyfile(""" def pytest_generate_tests(metafunc): metafunc.parametrize("arg1", [1], indirect=True) metafunc.parametrize("arg2", [10], indirect=True) - def pytest_funcarg__arg1(request): - x = request.getfuncargvalue("arg2") + import pytest + @pytest.fixture + def arg1(request): + x = request.getfixturevalue("arg2") return x + request.param - def pytest_funcarg__arg2(request): + @pytest.fixture + def arg2(request): return request.param def test_func1(arg1, arg2): @@ -741,10 +941,13 @@ def pytest_generate_tests(metafunc): assert "arg1" in metafunc.fixturenames metafunc.parametrize("arg1", [1], indirect=True) - def pytest_funcarg__arg1(request): + import pytest + @pytest.fixture + def arg1(request): return request.param - def pytest_funcarg__arg2(request, arg1): + @pytest.fixture + def arg2(request, arg1): return 10 * arg1 def test_func(arg2): @@ -757,6 +960,10 @@ def test_func(arg2): ]) def test_parametrize_with_ids(self, testdir): + testdir.makeini(""" + [pytest] + console_output_style=classic + """) testdir.makepyfile(""" import pytest def pytest_generate_tests(metafunc): @@ -789,25 +996,100 @@ def test_function(a, b): *test_function*1.3-b1* """) + def test_parametrize_with_None_in_ids(self, testdir): + testdir.makepyfile(""" + import pytest + def pytest_generate_tests(metafunc): + metafunc.parametrize(("a", "b"), [(1,1), (1,1), (1,2)], + ids=["basic", None, "advanced"]) + + def test_function(a, b): + assert a == b + """) + result = testdir.runpytest("-v") + assert result.ret == 1 + result.stdout.fnmatch_lines_random([ + "*test_function*basic*PASSED*", + "*test_function*1-1*PASSED*", + "*test_function*advanced*FAILED*", + ]) + + def test_fixture_parametrized_empty_ids(self, testdir): + """Fixtures parametrized with empty ids cause an internal error (#1849).""" + testdir.makepyfile(''' + import pytest + + @pytest.fixture(scope="module", ids=[], params=[]) + def temp(request): + return request.param + + def test_temp(temp): + pass + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['* 1 skipped *']) + + def test_parametrized_empty_ids(self, testdir): + """Tests parametrized with empty ids cause an internal error (#1849).""" + testdir.makepyfile(''' + import pytest + + @pytest.mark.parametrize('temp', [], ids=list()) + def test_temp(temp): + pass + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['* 1 skipped *']) + + def test_parametrized_ids_invalid_type(self, testdir): + """Tests parametrized with ids as non-strings (#1857).""" + testdir.makepyfile(''' + import pytest + + @pytest.mark.parametrize("x, expected", [(10, 20), (40, 80)], ids=(None, 2)) + def test_ids_numbers(x,expected): + assert x * 2 == expected + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['*ids must be list of strings, found: 2 (type: int)*']) + + def test_parametrize_with_identical_ids_get_unique_names(self, testdir): + testdir.makepyfile(""" + import pytest + def pytest_generate_tests(metafunc): + metafunc.parametrize(("a", "b"), [(1,1), (1,2)], + ids=["a", "a"]) + + def test_function(a, b): + assert a == b + """) + result = testdir.runpytest("-v") + assert result.ret == 1 + result.stdout.fnmatch_lines_random([ + "*test_function*a0*PASSED*", + "*test_function*a1*FAILED*" + ]) + @pytest.mark.parametrize(("scope", "length"), [("module", 2), ("function", 4)]) def test_parametrize_scope_overrides(self, testdir, scope, length): testdir.makepyfile(""" import pytest - l = [] + values = [] def pytest_generate_tests(metafunc): if "arg" in metafunc.funcargnames: metafunc.parametrize("arg", [1,2], indirect=True, scope=%r) - def pytest_funcarg__arg(request): - l.append(request.param) + @pytest.fixture + def arg(request): + values.append(request.param) return request.param def test_hello(arg): assert arg in (1,2) def test_world(arg): assert arg in (1,2) def test_checklength(): - assert len(l) == %d + assert len(values) == %d """ % (scope, length)) reprec = testdir.inline_run() reprec.assertoutcome(passed=5) @@ -855,7 +1137,7 @@ def pytest_generate_tests(metafunc): """)) sub1.join("test_in_sub1.py").write("def test_1(): pass") sub2.join("test_in_sub2.py").write("def test_2(): pass") - result = testdir.runpytest("-v", "-s", sub1, sub2, sub1) + result = testdir.runpytest("--keep-duplicates", "-v", "-s", sub1, sub2, sub1) result.assert_outcomes(passed=3) def test_generate_same_function_names_issue403(self, testdir): @@ -876,7 +1158,7 @@ def test_foo(x): @pytest.mark.issue463 @pytest.mark.parametrize('attr', ['parametrise', 'parameterize', - 'parameterise']) + 'parameterise']) def test_parametrize_misspelling(self, testdir, attr): testdir.makepyfile(""" import pytest @@ -892,8 +1174,129 @@ def test_foo(x): assert expectederror in failures[0].longrepr.reprcrash.message -class TestMarkersWithParametrization: - pytestmark = pytest.mark.issue308 +class TestMetafuncFunctionalAuto(object): + """ + Tests related to automatically find out the correct scope for parametrized tests (#1832). + """ + + def test_parametrize_auto_scope(self, testdir): + testdir.makepyfile(''' + import pytest + + @pytest.fixture(scope='session', autouse=True) + def fixture(): + return 1 + + @pytest.mark.parametrize('animal', ["dog", "cat"]) + def test_1(animal): + assert animal in ('dog', 'cat') + + @pytest.mark.parametrize('animal', ['fish']) + def test_2(animal): + assert animal == 'fish' + + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['* 3 passed *']) + + def test_parametrize_auto_scope_indirect(self, testdir): + testdir.makepyfile(''' + import pytest + + @pytest.fixture(scope='session') + def echo(request): + return request.param + + @pytest.mark.parametrize('animal, echo', [("dog", 1), ("cat", 2)], indirect=['echo']) + def test_1(animal, echo): + assert animal in ('dog', 'cat') + assert echo in (1, 2, 3) + + @pytest.mark.parametrize('animal, echo', [('fish', 3)], indirect=['echo']) + def test_2(animal, echo): + assert animal == 'fish' + assert echo in (1, 2, 3) + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['* 3 passed *']) + + def test_parametrize_auto_scope_override_fixture(self, testdir): + testdir.makepyfile(''' + import pytest + + @pytest.fixture(scope='session', autouse=True) + def animal(): + return 'fox' + + @pytest.mark.parametrize('animal', ["dog", "cat"]) + def test_1(animal): + assert animal in ('dog', 'cat') + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['* 2 passed *']) + + def test_parametrize_all_indirects(self, testdir): + testdir.makepyfile(''' + import pytest + + @pytest.fixture() + def animal(request): + return request.param + + @pytest.fixture(scope='session') + def echo(request): + return request.param + + @pytest.mark.parametrize('animal, echo', [("dog", 1), ("cat", 2)], indirect=True) + def test_1(animal, echo): + assert animal in ('dog', 'cat') + assert echo in (1, 2, 3) + + @pytest.mark.parametrize('animal, echo', [("fish", 3)], indirect=True) + def test_2(animal, echo): + assert animal == 'fish' + assert echo in (1, 2, 3) + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['* 3 passed *']) + + def test_parametrize_issue634(self, testdir): + testdir.makepyfile(''' + import pytest + + @pytest.fixture(scope='module') + def foo(request): + print('preparing foo-%d' % request.param) + return 'foo-%d' % request.param + + def test_one(foo): + pass + + def test_two(foo): + pass + + test_two.test_with = (2, 3) + + def pytest_generate_tests(metafunc): + params = (1, 2, 3, 4) + if not 'foo' in metafunc.fixturenames: + return + + test_with = getattr(metafunc.function, 'test_with', None) + if test_with: + params = test_with + metafunc.parametrize('foo', params, indirect=True) + ''') + result = testdir.runpytest("-s") + output = result.stdout.str() + assert output.count('preparing foo-2') == 1 + assert output.count('preparing foo-3') == 1 + + +@pytest.mark.filterwarnings('ignore:Applying marks directly to parameters') +@pytest.mark.issue308 +class TestMarkersWithParametrization(object): + def test_simple_mark(self, testdir): s = """ import pytest @@ -1036,22 +1439,23 @@ def test_increment(n, expected): reprec = testdir.inline_run() reprec.assertoutcome(passed=2, skipped=1) - def test_xfail_passing_is_xpass(self, testdir): + @pytest.mark.parametrize('strict', [True, False]) + def test_xfail_passing_is_xpass(self, testdir, strict): s = """ import pytest @pytest.mark.parametrize(("n", "expected"), [ (1, 2), - pytest.mark.xfail("sys.version > 0", reason="some bug")((2, 3)), + pytest.mark.xfail("sys.version_info > (0, 0, 0)", reason="some bug", strict={strict})((2, 3)), (3, 4), ]) def test_increment(n, expected): assert n + 1 == expected - """ + """.format(strict=strict) testdir.makepyfile(s) reprec = testdir.inline_run() - # xpass is fail, obviously :) - reprec.assertoutcome(passed=2, failed=1) + passed, failed = (2, 1) if strict else (3, 0) + reprec.assertoutcome(passed=passed, failed=failed) def test_parametrize_called_in_generate_tests(self, testdir): s = """ @@ -1076,7 +1480,6 @@ def test_increment(n, expected): reprec = testdir.inline_run() reprec.assertoutcome(passed=2, skipped=2) - @pytest.mark.issue290 def test_parametrize_ID_generation_string_int_works(self, testdir): testdir.makepyfile(""" @@ -1092,3 +1495,67 @@ def test_limit(limit, myfixture): """) reprec = testdir.inline_run() reprec.assertoutcome(passed=2) + + @pytest.mark.parametrize('strict', [True, False]) + def test_parametrize_marked_value(self, testdir, strict): + s = """ + import pytest + + @pytest.mark.parametrize(("n", "expected"), [ + pytest.param( + 2,3, + marks=pytest.mark.xfail("sys.version_info > (0, 0, 0)", reason="some bug", strict={strict}), + ), + pytest.param( + 2,3, + marks=[pytest.mark.xfail("sys.version_info > (0, 0, 0)", reason="some bug", strict={strict})], + ), + ]) + def test_increment(n, expected): + assert n + 1 == expected + """.format(strict=strict) + testdir.makepyfile(s) + reprec = testdir.inline_run() + passed, failed = (0, 2) if strict else (2, 0) + reprec.assertoutcome(passed=passed, failed=failed) + + def test_pytest_make_parametrize_id(self, testdir): + testdir.makeconftest(""" + def pytest_make_parametrize_id(config, val): + return str(val * 2) + """) + testdir.makepyfile(""" + import pytest + + @pytest.mark.parametrize("x", range(2)) + def test_func(x): + pass + """) + result = testdir.runpytest("-v") + result.stdout.fnmatch_lines([ + "*test_func*0*PASS*", + "*test_func*2*PASS*", + ]) + + def test_pytest_make_parametrize_id_with_argname(self, testdir): + testdir.makeconftest(""" + def pytest_make_parametrize_id(config, val, argname): + return str(val * 2 if argname == 'x' else val * 10) + """) + testdir.makepyfile(""" + import pytest + + @pytest.mark.parametrize("x", range(2)) + def test_func_a(x): + pass + + @pytest.mark.parametrize("y", [1]) + def test_func_b(y): + pass + """) + result = testdir.runpytest("-v") + result.stdout.fnmatch_lines([ + "*test_func_a*0*PASS*", + "*test_func_a*2*PASS*", + "*test_func_b*10*PASS*", + ]) diff --git a/testing/python/raises.py b/testing/python/raises.py index 0ea7f9bee3ebbf8..321ee349ee60bdc 100644 --- a/testing/python/raises.py +++ b/testing/python/raises.py @@ -1,6 +1,8 @@ import pytest +import sys -class TestRaises: + +class TestRaises(object): def test_raises(self): source = "int('qwe')" excinfo = pytest.raises(ValueError, source) @@ -18,7 +20,7 @@ def test_raises_function(self): pytest.raises(ValueError, int, 'hello') def test_raises_callable_no_exception(self): - class A: + class A(object): def __call__(self): pass try: @@ -26,14 +28,6 @@ def __call__(self): except pytest.raises.Exception: pass - def test_raises_flip_builtin_AssertionError(self): - # we replace AssertionError on python level - # however c code might still raise the builtin one - from _pytest.assertion.util import BuiltinAssertionError # noqa - pytest.raises(AssertionError,""" - raise BuiltinAssertionError - """) - def test_raises_as_contextmanager(self, testdir): testdir.makepyfile(""" from __future__ import with_statement @@ -76,3 +70,65 @@ def test_no_raise_message(self): pytest.raises(ValueError, int, '0') except pytest.raises.Exception as e: assert e.msg == "DID NOT RAISE {0}".format(repr(ValueError)) + else: + assert False, "Expected pytest.raises.Exception" + + try: + with pytest.raises(ValueError): + pass + except pytest.raises.Exception as e: + assert e.msg == "DID NOT RAISE {0}".format(repr(ValueError)) + else: + assert False, "Expected pytest.raises.Exception" + + def test_custom_raise_message(self): + message = "TEST_MESSAGE" + try: + with pytest.raises(ValueError, message=message): + pass + except pytest.raises.Exception as e: + assert e.msg == message + else: + assert False, "Expected pytest.raises.Exception" + + @pytest.mark.parametrize('method', ['function', 'with']) + def test_raises_cyclic_reference(self, method): + """ + Ensure pytest.raises does not leave a reference cycle (#1965). + """ + import gc + + class T(object): + def __call__(self): + raise ValueError + + t = T() + if method == 'function': + pytest.raises(ValueError, t) + else: + with pytest.raises(ValueError): + t() + + # ensure both forms of pytest.raises don't leave exceptions in sys.exc_info() + assert sys.exc_info() == (None, None, None) + + del t + + # ensure the t instance is not stuck in a cyclic reference + for o in gc.get_objects(): + assert type(o) is not T + + def test_raises_match(self): + msg = r"with base \d+" + with pytest.raises(ValueError, match=msg): + int('asdf') + + msg = "with base 10" + with pytest.raises(ValueError, match=msg): + int('asdf') + + msg = "with base 16" + expr = r"Pattern '{0}' not found in 'invalid literal for int\(\) with base 10: 'asdf''".format(msg) + with pytest.raises(AssertionError, match=expr): + with pytest.raises(ValueError, match=msg): + int('asdf', base=10) diff --git a/testing/python/setup_only.py b/testing/python/setup_only.py new file mode 100644 index 000000000000000..ab34312fcc8ed88 --- /dev/null +++ b/testing/python/setup_only.py @@ -0,0 +1,243 @@ +import pytest + + +@pytest.fixture(params=['--setup-only', '--setup-plan', '--setup-show'], + scope='module') +def mode(request): + return request.param + + +def test_show_only_active_fixtures(testdir, mode): + p = testdir.makepyfile(''' + import pytest + @pytest.fixture + def _arg0(): + """hidden arg0 fixture""" + @pytest.fixture + def arg1(): + """arg1 docstring""" + def test_arg1(arg1): + pass + ''') + + result = testdir.runpytest(mode, p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*SETUP F arg1*', + '*test_arg1 (fixtures used: arg1)*', + '*TEARDOWN F arg1*', + ]) + assert "_arg0" not in result.stdout.str() + + +def test_show_different_scopes(testdir, mode): + p = testdir.makepyfile(''' + import pytest + @pytest.fixture + def arg_function(): + """function scoped fixture""" + @pytest.fixture(scope='session') + def arg_session(): + """session scoped fixture""" + def test_arg1(arg_session, arg_function): + pass + ''') + + result = testdir.runpytest(mode, p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + 'SETUP S arg_session*', + '*SETUP F arg_function*', + '*test_arg1 (fixtures used: arg_function, arg_session)*', + '*TEARDOWN F arg_function*', + 'TEARDOWN S arg_session*', + ]) + + +def test_show_nested_fixtures(testdir, mode): + testdir.makeconftest(''' + import pytest + @pytest.fixture(scope='session') + def arg_same(): + """session scoped fixture""" + ''') + p = testdir.makepyfile(''' + import pytest + @pytest.fixture(scope='function') + def arg_same(arg_same): + """function scoped fixture""" + def test_arg1(arg_same): + pass + ''') + + result = testdir.runpytest(mode, p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + 'SETUP S arg_same*', + '*SETUP F arg_same (fixtures used: arg_same)*', + '*test_arg1 (fixtures used: arg_same)*', + '*TEARDOWN F arg_same*', + 'TEARDOWN S arg_same*', + ]) + + +def test_show_fixtures_with_autouse(testdir, mode): + p = testdir.makepyfile(''' + import pytest + @pytest.fixture + def arg_function(): + """function scoped fixture""" + @pytest.fixture(scope='session', autouse=True) + def arg_session(): + """session scoped fixture""" + def test_arg1(arg_function): + pass + ''') + + result = testdir.runpytest(mode, p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + 'SETUP S arg_session*', + '*SETUP F arg_function*', + '*test_arg1 (fixtures used: arg_function, arg_session)*', + ]) + + +def test_show_fixtures_with_parameters(testdir, mode): + testdir.makeconftest(''' + import pytest + @pytest.fixture(scope='session', params=['foo', 'bar']) + def arg_same(): + """session scoped fixture""" + ''') + p = testdir.makepyfile(''' + import pytest + @pytest.fixture(scope='function') + def arg_other(arg_same): + """function scoped fixture""" + def test_arg1(arg_other): + pass + ''') + + result = testdir.runpytest(mode, p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + 'SETUP S arg_same?foo?', + 'TEARDOWN S arg_same?foo?', + 'SETUP S arg_same?bar?', + 'TEARDOWN S arg_same?bar?', + ]) + + +def test_show_fixtures_with_parameter_ids(testdir, mode): + testdir.makeconftest(''' + import pytest + @pytest.fixture( + scope='session', params=['foo', 'bar'], ids=['spam', 'ham']) + def arg_same(): + """session scoped fixture""" + ''') + p = testdir.makepyfile(''' + import pytest + @pytest.fixture(scope='function') + def arg_other(arg_same): + """function scoped fixture""" + def test_arg1(arg_other): + pass + ''') + + result = testdir.runpytest(mode, p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + 'SETUP S arg_same?spam?', + 'SETUP S arg_same?ham?', + ]) + + +def test_show_fixtures_with_parameter_ids_function(testdir, mode): + p = testdir.makepyfile(''' + import pytest + @pytest.fixture(params=['foo', 'bar'], ids=lambda p: p.upper()) + def foobar(): + pass + def test_foobar(foobar): + pass + ''') + + result = testdir.runpytest(mode, p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*SETUP F foobar?FOO?', + '*SETUP F foobar?BAR?', + ]) + + +def test_dynamic_fixture_request(testdir): + p = testdir.makepyfile(''' + import pytest + @pytest.fixture() + def dynamically_requested_fixture(): + pass + @pytest.fixture() + def dependent_fixture(request): + request.getfixturevalue('dynamically_requested_fixture') + def test_dyn(dependent_fixture): + pass + ''') + + result = testdir.runpytest('--setup-only', p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*SETUP F dynamically_requested_fixture', + '*TEARDOWN F dynamically_requested_fixture' + ]) + + +def test_capturing(testdir): + p = testdir.makepyfile(''' + import pytest, sys + @pytest.fixture() + def one(): + sys.stdout.write('this should be captured') + sys.stderr.write('this should also be captured') + @pytest.fixture() + def two(one): + assert 0 + def test_capturing(two): + pass + ''') + + result = testdir.runpytest('--setup-only', p) + result.stdout.fnmatch_lines([ + 'this should be captured', + 'this should also be captured' + ]) + + +def test_show_fixtures_and_execute_test(testdir): + """ Verifies that setups are shown and tests are executed. """ + p = testdir.makepyfile(''' + import pytest + @pytest.fixture + def arg(): + assert True + def test_arg(arg): + assert False + ''') + + result = testdir.runpytest("--setup-show", p) + assert result.ret == 1 + + result.stdout.fnmatch_lines([ + '*SETUP F arg*', + '*test_arg (fixtures used: arg)F*', + '*TEARDOWN F arg*', + ]) diff --git a/testing/python/setup_plan.py b/testing/python/setup_plan.py new file mode 100644 index 000000000000000..8c98224692a8b90 --- /dev/null +++ b/testing/python/setup_plan.py @@ -0,0 +1,19 @@ +def test_show_fixtures_and_test(testdir): + """ Verifies that fixtures are not executed. """ + p = testdir.makepyfile(''' + import pytest + @pytest.fixture + def arg(): + assert False + def test_arg(arg): + assert False + ''') + + result = testdir.runpytest("--setup-plan", p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*SETUP F arg*', + '*test_arg (fixtures used: arg)', + '*TEARDOWN F arg*', + ]) diff --git a/testing/python/show_fixtures_per_test.py b/testing/python/show_fixtures_per_test.py new file mode 100644 index 000000000000000..741f33946a0ee3c --- /dev/null +++ b/testing/python/show_fixtures_per_test.py @@ -0,0 +1,158 @@ +# -*- coding: utf-8 -*- + + +def test_no_items_should_not_show_output(testdir): + result = testdir.runpytest('--fixtures-per-test') + assert 'fixtures used by' not in result.stdout.str() + assert result.ret == 0 + + +def test_fixtures_in_module(testdir): + p = testdir.makepyfile(''' + import pytest + @pytest.fixture + def _arg0(): + """hidden arg0 fixture""" + @pytest.fixture + def arg1(): + """arg1 docstring""" + def test_arg1(arg1): + pass + ''') + + result = testdir.runpytest("--fixtures-per-test", p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*fixtures used by test_arg1*', + '*(test_fixtures_in_module.py:9)*', + 'arg1', + ' arg1 docstring', + ]) + assert "_arg0" not in result.stdout.str() + + +def test_fixtures_in_conftest(testdir): + testdir.makeconftest(''' + import pytest + @pytest.fixture + def arg1(): + """arg1 docstring""" + @pytest.fixture + def arg2(): + """arg2 docstring""" + @pytest.fixture + def arg3(arg1, arg2): + """arg3 + docstring + """ + ''') + p = testdir.makepyfile(''' + def test_arg2(arg2): + pass + def test_arg3(arg3): + pass + ''') + result = testdir.runpytest("--fixtures-per-test", p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*fixtures used by test_arg2*', + '*(test_fixtures_in_conftest.py:2)*', + 'arg2', + ' arg2 docstring', + '*fixtures used by test_arg3*', + '*(test_fixtures_in_conftest.py:4)*', + 'arg1', + ' arg1 docstring', + 'arg2', + ' arg2 docstring', + 'arg3', + ' arg3', + ' docstring', + ]) + + +def test_should_show_fixtures_used_by_test(testdir): + testdir.makeconftest(''' + import pytest + @pytest.fixture + def arg1(): + """arg1 from conftest""" + @pytest.fixture + def arg2(): + """arg2 from conftest""" + ''') + p = testdir.makepyfile(''' + import pytest + @pytest.fixture + def arg1(): + """arg1 from testmodule""" + def test_args(arg1, arg2): + pass + ''') + result = testdir.runpytest("--fixtures-per-test", p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*fixtures used by test_args*', + '*(test_should_show_fixtures_used_by_test.py:6)*', + 'arg1', + ' arg1 from testmodule', + 'arg2', + ' arg2 from conftest', + ]) + + +def test_verbose_include_private_fixtures_and_loc(testdir): + testdir.makeconftest(''' + import pytest + @pytest.fixture + def _arg1(): + """_arg1 from conftest""" + @pytest.fixture + def arg2(_arg1): + """arg2 from conftest""" + ''') + p = testdir.makepyfile(''' + import pytest + @pytest.fixture + def arg3(): + """arg3 from testmodule""" + def test_args(arg2, arg3): + pass + ''') + result = testdir.runpytest("--fixtures-per-test", "-v", p) + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*fixtures used by test_args*', + '*(test_verbose_include_private_fixtures_and_loc.py:6)*', + '_arg1 -- conftest.py:3', + ' _arg1 from conftest', + 'arg2 -- conftest.py:6', + ' arg2 from conftest', + 'arg3 -- test_verbose_include_private_fixtures_and_loc.py:3', + ' arg3 from testmodule', + ]) + + +def test_doctest_items(testdir): + testdir.makepyfile(''' + def foo(): + """ + >>> 1 + 1 + 2 + """ + ''') + testdir.maketxtfile(''' + >>> 1 + 1 + 2 + ''') + result = testdir.runpytest("--fixtures-per-test", "--doctest-modules", + "--doctest-glob=*.txt", "-v") + assert result.ret == 0 + + result.stdout.fnmatch_lines([ + '*collected 2 items*', + ]) diff --git a/testing/python/test_deprecations.py b/testing/python/test_deprecations.py new file mode 100644 index 000000000000000..5001f765f6c588b --- /dev/null +++ b/testing/python/test_deprecations.py @@ -0,0 +1,22 @@ +import pytest + +from _pytest.python import PyCollector + + +class PyCollectorMock(PyCollector): + """evil hack""" + + def __init__(self): + self.called = False + + def _makeitem(self, *k): + """hack to disable the actual behaviour""" + self.called = True + + +def test_pycollector_makeitem_is_deprecated(): + + collector = PyCollectorMock() + with pytest.deprecated_call(): + collector.makeitem('foo', 'bar') + assert collector.called diff --git a/testing/test_argcomplete.py b/testing/test_argcomplete.py index ace7d8ceb432f82..c9261257743f2b8 100644 --- a/testing/test_argcomplete.py +++ b/testing/test_argcomplete.py @@ -1,8 +1,10 @@ -from __future__ import with_statement -import py, pytest +from __future__ import absolute_import, division, print_function +import py +import pytest # test for _argcomplete but not specific for any application + def equal_with_bash(prefix, ffc, fc, out=None): res = ffc(prefix) res_bash = set(fc(prefix)) @@ -17,10 +19,12 @@ def equal_with_bash(prefix, ffc, fc, out=None): # copied from argcomplete.completers as import from there # also pulls in argcomplete.__init__ which opens filedescriptor 9 # this gives an IOError at the end of testrun + + def _wrapcall(*args, **kargs): try: - if py.std.sys.version_info > (2,7): - return py.std.subprocess.check_output(*args,**kargs).decode().splitlines() + if py.std.sys.version_info > (2, 7): + return py.std.subprocess.check_output(*args, **kargs).decode().splitlines() if 'stdout' in kargs: raise ValueError('stdout argument not allowed, it will be overridden.') process = py.std.subprocess.Popen( @@ -36,9 +40,11 @@ def _wrapcall(*args, **kargs): except py.std.subprocess.CalledProcessError: return [] + class FilesCompleter(object): 'File completer class, optionally takes a list of allowed extensions' - def __init__(self,allowednames=(),directories=True): + + def __init__(self, allowednames=(), directories=True): # Fix if someone passes in a string instead of a list if type(allowednames) is str: allowednames = [allowednames] @@ -50,32 +56,33 @@ def __call__(self, prefix, **kwargs): completion = [] if self.allowednames: if self.directories: - files = _wrapcall(['bash','-c', - "compgen -A directory -- '{p}'".format(p=prefix)]) - completion += [ f + '/' for f in files] + files = _wrapcall(['bash', '-c', + "compgen -A directory -- '{p}'".format(p=prefix)]) + completion += [f + '/' for f in files] for x in self.allowednames: completion += _wrapcall(['bash', '-c', - "compgen -A file -X '!*.{0}' -- '{p}'".format(x,p=prefix)]) + "compgen -A file -X '!*.{0}' -- '{p}'".format(x, p=prefix)]) else: completion += _wrapcall(['bash', '-c', - "compgen -A file -- '{p}'".format(p=prefix)]) + "compgen -A file -- '{p}'".format(p=prefix)]) anticomp = _wrapcall(['bash', '-c', - "compgen -A directory -- '{p}'".format(p=prefix)]) + "compgen -A directory -- '{p}'".format(p=prefix)]) - completion = list( set(completion) - set(anticomp)) + completion = list(set(completion) - set(anticomp)) if self.directories: completion += [f + '/' for f in anticomp] return completion -class TestArgComplete: + +class TestArgComplete(object): @pytest.mark.skipif("sys.platform in ('win32', 'darwin')") def test_compare_with_compgen(self): from _pytest._argcomplete import FastFilesCompleter ffc = FastFilesCompleter() fc = FilesCompleter() - for x in '/ /d /data qqq'.split(): + for x in ['/', '/d', '/data', 'qqq', '']: assert equal_with_bash(x, ffc, fc, out=py.std.sys.stdout) @pytest.mark.skipif("sys.platform in ('win32', 'darwin')") diff --git a/testing/test_assertinterpret.py b/testing/test_assertinterpret.py deleted file mode 100644 index 67a352ce7f5efee..000000000000000 --- a/testing/test_assertinterpret.py +++ /dev/null @@ -1,274 +0,0 @@ -"PYTEST_DONT_REWRITE" -import py -import pytest -from _pytest.assertion import util - - -def exvalue(): - return py.std.sys.exc_info()[1] - -def f(): - return 2 - -def test_not_being_rewritten(): - assert "@py_builtins" not in globals() - -def test_assert(): - try: - assert f() == 3 - except AssertionError: - e = exvalue() - s = str(e) - assert s.startswith('assert 2 == 3\n') - -def test_assert_with_explicit_message(): - try: - assert f() == 3, "hello" - except AssertionError: - e = exvalue() - assert e.msg == 'hello' - -def test_assert_within_finally(): - excinfo = pytest.raises(ZeroDivisionError, """ - try: - 1/0 - finally: - i = 42 - """) - s = excinfo.exconly() - assert py.std.re.search("division.+by zero", s) is not None - - #def g(): - # A.f() - #excinfo = getexcinfo(TypeError, g) - #msg = getmsg(excinfo) - #assert msg.find("must be called with A") != -1 - - -def test_assert_multiline_1(): - try: - assert (f() == - 3) - except AssertionError: - e = exvalue() - s = str(e) - assert s.startswith('assert 2 == 3\n') - -def test_assert_multiline_2(): - try: - assert (f() == (4, - 3)[-1]) - except AssertionError: - e = exvalue() - s = str(e) - assert s.startswith('assert 2 ==') - -def test_in(): - try: - assert "hi" in [1, 2] - except AssertionError: - e = exvalue() - s = str(e) - assert s.startswith("assert 'hi' in") - -def test_is(): - try: - assert 1 is 2 - except AssertionError: - e = exvalue() - s = str(e) - assert s.startswith("assert 1 is 2") - - -def test_attrib(): - class Foo(object): - b = 1 - i = Foo() - try: - assert i.b == 2 - except AssertionError: - e = exvalue() - s = str(e) - assert s.startswith("assert 1 == 2") - -def test_attrib_inst(): - class Foo(object): - b = 1 - try: - assert Foo().b == 2 - except AssertionError: - e = exvalue() - s = str(e) - assert s.startswith("assert 1 == 2") - -def test_len(): - l = list(range(42)) - try: - assert len(l) == 100 - except AssertionError: - e = exvalue() - s = str(e) - assert s.startswith("assert 42 == 100") - assert "where 42 = len([" in s - -def test_assert_non_string_message(): - class A: - def __str__(self): - return "hello" - try: - assert 0 == 1, A() - except AssertionError: - e = exvalue() - assert e.msg == "hello" - -def test_assert_keyword_arg(): - def f(x=3): - return False - try: - assert f(x=5) - except AssertionError: - e = exvalue() - assert "x=5" in e.msg - -def test_private_class_variable(): - class X: - def __init__(self): - self.__v = 41 - def m(self): - assert self.__v == 42 - try: - X().m() - except AssertionError: - e = exvalue() - assert "== 42" in e.msg - -# These tests should both fail, but should fail nicely... -class WeirdRepr: - def __repr__(self): - return '' - -def bug_test_assert_repr(): - v = WeirdRepr() - try: - assert v == 1 - except AssertionError: - e = exvalue() - assert e.msg.find('WeirdRepr') != -1 - assert e.msg.find('second line') != -1 - assert 0 - -def test_assert_non_string(): - try: - assert 0, ['list'] - except AssertionError: - e = exvalue() - assert e.msg.find("list") != -1 - -def test_assert_implicit_multiline(): - try: - x = [1,2,3] - assert x != [1, - 2, 3] - except AssertionError: - e = exvalue() - assert e.msg.find('assert [1, 2, 3] !=') != -1 - - -def test_assert_with_brokenrepr_arg(): - class BrokenRepr: - def __repr__(self): 0 / 0 - e = AssertionError(BrokenRepr()) - if e.msg.find("broken __repr__") == -1: - pytest.fail("broken __repr__ not handle correctly") - -def test_multiple_statements_per_line(): - try: - a = 1; assert a == 2 - except AssertionError: - e = exvalue() - assert "assert 1 == 2" in e.msg - -def test_power(): - try: - assert 2**3 == 7 - except AssertionError: - e = exvalue() - assert "assert (2 ** 3) == 7" in e.msg - - -def test_assert_customizable_reprcompare(monkeypatch): - monkeypatch.setattr(util, '_reprcompare', lambda *args: 'hello') - try: - assert 3 == 4 - except AssertionError: - e = exvalue() - s = str(e) - assert "hello" in s - -def test_assert_long_source_1(): - try: - assert len == [ - (None, ['somet text', 'more text']), - ] - except AssertionError: - e = exvalue() - s = str(e) - assert 're-run' not in s - assert 'somet text' in s - -def test_assert_long_source_2(): - try: - assert(len == [ - (None, ['somet text', 'more text']), - ]) - except AssertionError: - e = exvalue() - s = str(e) - assert 're-run' not in s - assert 'somet text' in s - -def test_assert_raise_alias(testdir): - testdir.makepyfile(""" - "PYTEST_DONT_REWRITE" - import sys - EX = AssertionError - def test_hello(): - raise EX("hello" - "multi" - "line") - """) - result = testdir.runpytest() - result.stdout.fnmatch_lines([ - "*def test_hello*", - "*raise EX*", - "*1 failed*", - ]) - - -def test_assert_raise_subclass(): - class SomeEx(AssertionError): - def __init__(self, *args): - super(SomeEx, self).__init__() - try: - raise SomeEx("hello") - except AssertionError: - s = str(exvalue()) - assert 're-run' not in s - assert 'could not determine' in s - -def test_assert_raises_in_nonzero_of_object_pytest_issue10(): - class A(object): - def __nonzero__(self): - raise ValueError(42) - def __lt__(self, other): - return A() - def __repr__(self): - return "" - def myany(x): - return True - try: - assert not(myany(A() < 0)) - except AssertionError: - e = exvalue() - s = str(e) - assert " < 0" in s diff --git a/testing/test_assertion.py b/testing/test_assertion.py index 347278e1904f73a..328fe7fa918e4d7 100644 --- a/testing/test_assertion.py +++ b/testing/test_assertion.py @@ -1,46 +1,299 @@ # -*- coding: utf-8 -*- +from __future__ import absolute_import, division, print_function import sys import textwrap import _pytest.assertion as plugin -import _pytest._code import py import pytest -from _pytest.assertion import reinterpret from _pytest.assertion import util +from _pytest.assertion import truncate PY3 = sys.version_info >= (3, 0) @pytest.fixture def mock_config(): + class Config(object): verbose = False + def getoption(self, name): if name == 'verbose': return self.verbose raise KeyError('Not mocked out: %s' % name) + return Config() -def interpret(expr): - return reinterpret.reinterpret(expr, _pytest._code.Frame(sys._getframe(1))) +class TestImportHookInstallation(object): + + @pytest.mark.parametrize('initial_conftest', [True, False]) + @pytest.mark.parametrize('mode', ['plain', 'rewrite']) + def test_conftest_assertion_rewrite(self, testdir, initial_conftest, mode): + """Test that conftest files are using assertion rewrite on import. + (#1619) + """ + testdir.tmpdir.join('foo/tests').ensure(dir=1) + conftest_path = 'conftest.py' if initial_conftest else 'foo/conftest.py' + contents = { + conftest_path: """ + import pytest + @pytest.fixture + def check_first(): + def check(values, value): + assert values.pop(0) == value + return check + """, + 'foo/tests/test_foo.py': """ + def test(check_first): + check_first([10, 30], 30) + """ + } + testdir.makepyfile(**contents) + result = testdir.runpytest_subprocess('--assert=%s' % mode) + if mode == 'plain': + expected = 'E AssertionError' + elif mode == 'rewrite': + expected = '*assert 10 == 30*' + else: + assert 0 + result.stdout.fnmatch_lines([expected]) + + def test_rewrite_assertions_pytester_plugin(self, testdir): + """ + Assertions in the pytester plugin must also benefit from assertion + rewriting (#1920). + """ + testdir.makepyfile(""" + pytest_plugins = ['pytester'] + def test_dummy_failure(testdir): # how meta! + testdir.makepyfile('def test(): assert 0') + r = testdir.inline_run() + r.assertoutcome(passed=1) + """) + result = testdir.runpytest_subprocess() + result.stdout.fnmatch_lines([ + '*assert 1 == 0*', + ]) -class TestBinReprIntegration: + @pytest.mark.parametrize('mode', ['plain', 'rewrite']) + def test_pytest_plugins_rewrite(self, testdir, mode): + contents = { + 'conftest.py': """ + pytest_plugins = ['ham'] + """, + 'ham.py': """ + import pytest + @pytest.fixture + def check_first(): + def check(values, value): + assert values.pop(0) == value + return check + """, + 'test_foo.py': """ + def test_foo(check_first): + check_first([10, 30], 30) + """, + } + testdir.makepyfile(**contents) + result = testdir.runpytest_subprocess('--assert=%s' % mode) + if mode == 'plain': + expected = 'E AssertionError' + elif mode == 'rewrite': + expected = '*assert 10 == 30*' + else: + assert 0 + result.stdout.fnmatch_lines([expected]) + + @pytest.mark.parametrize('mode', ['str', 'list']) + def test_pytest_plugins_rewrite_module_names(self, testdir, mode): + """Test that pluginmanager correct marks pytest_plugins variables + for assertion rewriting if they are defined as plain strings or + list of strings (#1888). + """ + plugins = '"ham"' if mode == 'str' else '["ham"]' + contents = { + 'conftest.py': """ + pytest_plugins = {plugins} + """.format(plugins=plugins), + 'ham.py': """ + import pytest + """, + 'test_foo.py': """ + def test_foo(pytestconfig): + assert 'ham' in pytestconfig.pluginmanager.rewrite_hook._must_rewrite + """, + } + testdir.makepyfile(**contents) + result = testdir.runpytest_subprocess('--assert=rewrite') + assert result.ret == 0 + + def test_pytest_plugins_rewrite_module_names_correctly(self, testdir): + """Test that we match files correctly when they are marked for rewriting (#2939).""" + contents = { + 'conftest.py': """ + pytest_plugins = "ham" + """, + 'ham.py': "", + 'hamster.py': "", + 'test_foo.py': """ + def test_foo(pytestconfig): + assert pytestconfig.pluginmanager.rewrite_hook.find_module('ham') is not None + assert pytestconfig.pluginmanager.rewrite_hook.find_module('hamster') is None + """, + } + testdir.makepyfile(**contents) + result = testdir.runpytest_subprocess('--assert=rewrite') + assert result.ret == 0 + + @pytest.mark.parametrize('mode', ['plain', 'rewrite']) + @pytest.mark.parametrize('plugin_state', ['development', 'installed']) + def test_installed_plugin_rewrite(self, testdir, mode, plugin_state): + # Make sure the hook is installed early enough so that plugins + # installed via setuptools are rewritten. + testdir.tmpdir.join('hampkg').ensure(dir=1) + contents = { + 'hampkg/__init__.py': """ + import pytest + + @pytest.fixture + def check_first2(): + def check(values, value): + assert values.pop(0) == value + return check + """, + 'spamplugin.py': """ + import pytest + from hampkg import check_first2 + + @pytest.fixture + def check_first(): + def check(values, value): + assert values.pop(0) == value + return check + """, + 'mainwrapper.py': """ + import pytest, pkg_resources + + plugin_state = "{plugin_state}" + + class DummyDistInfo(object): + project_name = 'spam' + version = '1.0' + + def _get_metadata(self, name): + # 'RECORD' meta-data only available in installed plugins + if name == 'RECORD' and plugin_state == "installed": + return ['spamplugin.py,sha256=abc,123', + 'hampkg/__init__.py,sha256=abc,123'] + # 'SOURCES.txt' meta-data only available for plugins in development mode + elif name == 'SOURCES.txt' and plugin_state == "development": + return ['spamplugin.py', + 'hampkg/__init__.py'] + return [] + + class DummyEntryPoint(object): + name = 'spam' + module_name = 'spam.py' + attrs = () + extras = None + dist = DummyDistInfo() + + def load(self, require=True, *args, **kwargs): + import spamplugin + return spamplugin + + def iter_entry_points(name): + yield DummyEntryPoint() + + pkg_resources.iter_entry_points = iter_entry_points + pytest.main() + """.format(plugin_state=plugin_state), + 'test_foo.py': """ + def test(check_first): + check_first([10, 30], 30) + + def test2(check_first2): + check_first([10, 30], 30) + """, + } + testdir.makepyfile(**contents) + result = testdir.run(sys.executable, 'mainwrapper.py', '-s', '--assert=%s' % mode) + if mode == 'plain': + expected = 'E AssertionError' + elif mode == 'rewrite': + expected = '*assert 10 == 30*' + else: + assert 0 + result.stdout.fnmatch_lines([expected]) + + def test_rewrite_ast(self, testdir): + testdir.tmpdir.join('pkg').ensure(dir=1) + contents = { + 'pkg/__init__.py': """ + import pytest + pytest.register_assert_rewrite('pkg.helper') + """, + 'pkg/helper.py': """ + def tool(): + a, b = 2, 3 + assert a == b + """, + 'pkg/plugin.py': """ + import pytest, pkg.helper + @pytest.fixture + def tool(): + return pkg.helper.tool + """, + 'pkg/other.py': """ + values = [3, 2] + def tool(): + assert values.pop() == 3 + """, + 'conftest.py': """ + pytest_plugins = ['pkg.plugin'] + """, + 'test_pkg.py': """ + import pkg.other + def test_tool(tool): + tool() + def test_other(): + pkg.other.tool() + """, + } + testdir.makepyfile(**contents) + result = testdir.runpytest_subprocess('--assert=rewrite') + result.stdout.fnmatch_lines(['>*assert a == b*', + 'E*assert 2 == 3*', + '>*assert values.pop() == 3*', + 'E*AssertionError']) + + def test_register_assert_rewrite_checks_types(self): + with pytest.raises(TypeError): + pytest.register_assert_rewrite(['pytest_tests_internal_non_existing']) + pytest.register_assert_rewrite('pytest_tests_internal_non_existing', + 'pytest_tests_internal_non_existing2') + + +class TestBinReprIntegration(object): def test_pytest_assertrepr_compare_called(self, testdir): testdir.makeconftest(""" - l = [] + import pytest + values = [] def pytest_assertrepr_compare(op, left, right): - l.append((op, left, right)) - def pytest_funcarg__l(request): - return l + values.append((op, left, right)) + + @pytest.fixture + def list(request): + return values """) testdir.makepyfile(""" def test_hello(): assert 0 == 1 - def test_check(l): - assert l == [("==", 0, 1)] + def test_check(list): + assert list == [("==", 0, 1)] """) result = testdir.runpytest("-v") result.stdout.fnmatch_lines([ @@ -48,13 +301,14 @@ def test_check(l): "*test_check*PASS*", ]) + def callequal(left, right, verbose=False): config = mock_config() config.verbose = verbose return plugin.pytest_assertrepr_compare(config, '==', left, right) -class TestAssert_reprcompare: +class TestAssert_reprcompare(object): def test_different_types(self): assert callequal([0, 1], 'foo') is None @@ -68,15 +322,15 @@ def test_text_diff(self): assert '+ eggs' in diff def test_text_skipping(self): - lines = callequal('a'*50 + 'spam', 'a'*50 + 'eggs') + lines = callequal('a' * 50 + 'spam', 'a' * 50 + 'eggs') assert 'Skipping' in lines[1] for line in lines: - assert 'a'*50 not in line + assert 'a' * 50 not in line def test_text_skipping_verbose(self): - lines = callequal('a'*50 + 'spam', 'a'*50 + 'eggs', verbose=True) - assert '- ' + 'a'*50 + 'spam' in lines - assert '+ ' + 'a'*50 + 'eggs' in lines + lines = callequal('a' * 50 + 'spam', 'a' * 50 + 'eggs', verbose=True) + assert '- ' + 'a' * 50 + 'spam' in lines + assert '+ ' + 'a' * 50 + 'eggs' in lines def test_multiline_text_diff(self): left = 'foo\nspam\nbar' @@ -131,7 +385,7 @@ def test_iterable_full_diff(self, left, right, expected): expl = '\n'.join(callequal(left, right, verbose=True)) assert expl.endswith(textwrap.dedent(expected).strip()) - def test_list_different_lenghts(self): + def test_list_different_lengths(self): expl = callequal([0, 1], [0, 1, 2]) assert len(expl) > 1 expl = callequal([0, 1, 2], [0, 1]) @@ -148,8 +402,16 @@ def test_dict_omitting(self): for line in lines[1:]: assert 'b' not in line - def test_dict_omitting_verbose(self): - lines = callequal({'a': 0, 'b': 1}, {'a': 1, 'b': 1}, verbose=True) + def test_dict_omitting_with_verbosity_1(self): + """ Ensure differing items are visible for verbosity=1 (#1512) """ + lines = callequal({'a': 0, 'b': 1}, {'a': 1, 'b': 1}, verbose=1) + assert lines[1].startswith('Omitting 1 identical item') + assert lines[2].startswith('Differing items') + assert lines[3] == "{'a': 0} != {'a': 1}" + assert 'Common items' not in lines + + def test_dict_omitting_with_verbosity_2(self): + lines = callequal({'a': 0, 'b': 1}, {'a': 1, 'b': 1}, verbose=2) assert lines[1].startswith('Common items:') assert 'Omitting' not in lines[1] assert lines[2] == "{'b': 1}" @@ -194,13 +456,13 @@ def insert(self, item, index): assert len(expl) > 1 def test_list_tuples(self): - expl = callequal([], [(1,2)]) + expl = callequal([], [(1, 2)]) assert len(expl) > 1 - expl = callequal([(1,2)], []) + expl = callequal([(1, 2)], []) assert len(expl) > 1 def test_list_bad_repr(self): - class A: + class A(object): def __repr__(self): raise ValueError(42) expl = callequal([], [A()]) @@ -259,7 +521,7 @@ def test_mojibake(self): assert msg -class TestFormatExplanation: +class TestFormatExplanation(object): def test_special_chars_full(self, testdir): # Issue 453, for the bug this would raise IndexError @@ -351,6 +613,111 @@ def test_fmt_multi_newline_before_where(self): assert util.format_explanation(expl) == res +class TestTruncateExplanation(object): + + """ Confirm assertion output is truncated as expected """ + + # The number of lines in the truncation explanation message. Used + # to calculate that results have the expected length. + LINES_IN_TRUNCATION_MSG = 2 + + def test_doesnt_truncate_when_input_is_empty_list(self): + expl = [] + result = truncate._truncate_explanation(expl, max_lines=8, max_chars=100) + assert result == expl + + def test_doesnt_truncate_at_when_input_is_5_lines_and_LT_max_chars(self): + expl = ['a' * 100 for x in range(5)] + result = truncate._truncate_explanation(expl, max_lines=8, max_chars=8 * 80) + assert result == expl + + def test_truncates_at_8_lines_when_given_list_of_empty_strings(self): + expl = ['' for x in range(50)] + result = truncate._truncate_explanation(expl, max_lines=8, max_chars=100) + assert result != expl + assert len(result) == 8 + self.LINES_IN_TRUNCATION_MSG + assert "Full output truncated" in result[-1] + assert "43 lines hidden" in result[-1] + last_line_before_trunc_msg = result[- self.LINES_IN_TRUNCATION_MSG - 1] + assert last_line_before_trunc_msg.endswith("...") + + def test_truncates_at_8_lines_when_first_8_lines_are_LT_max_chars(self): + expl = ['a' for x in range(100)] + result = truncate._truncate_explanation(expl, max_lines=8, max_chars=8 * 80) + assert result != expl + assert len(result) == 8 + self.LINES_IN_TRUNCATION_MSG + assert "Full output truncated" in result[-1] + assert "93 lines hidden" in result[-1] + last_line_before_trunc_msg = result[- self.LINES_IN_TRUNCATION_MSG - 1] + assert last_line_before_trunc_msg.endswith("...") + + def test_truncates_at_8_lines_when_first_8_lines_are_EQ_max_chars(self): + expl = ['a' * 80 for x in range(16)] + result = truncate._truncate_explanation(expl, max_lines=8, max_chars=8 * 80) + assert result != expl + assert len(result) == 8 + self.LINES_IN_TRUNCATION_MSG + assert "Full output truncated" in result[-1] + assert "9 lines hidden" in result[-1] + last_line_before_trunc_msg = result[- self.LINES_IN_TRUNCATION_MSG - 1] + assert last_line_before_trunc_msg.endswith("...") + + def test_truncates_at_4_lines_when_first_4_lines_are_GT_max_chars(self): + expl = ['a' * 250 for x in range(10)] + result = truncate._truncate_explanation(expl, max_lines=8, max_chars=999) + assert result != expl + assert len(result) == 4 + self.LINES_IN_TRUNCATION_MSG + assert "Full output truncated" in result[-1] + assert "7 lines hidden" in result[-1] + last_line_before_trunc_msg = result[- self.LINES_IN_TRUNCATION_MSG - 1] + assert last_line_before_trunc_msg.endswith("...") + + def test_truncates_at_1_line_when_first_line_is_GT_max_chars(self): + expl = ['a' * 250 for x in range(1000)] + result = truncate._truncate_explanation(expl, max_lines=8, max_chars=100) + assert result != expl + assert len(result) == 1 + self.LINES_IN_TRUNCATION_MSG + assert "Full output truncated" in result[-1] + assert "1000 lines hidden" in result[-1] + last_line_before_trunc_msg = result[- self.LINES_IN_TRUNCATION_MSG - 1] + assert last_line_before_trunc_msg.endswith("...") + + def test_full_output_truncated(self, monkeypatch, testdir): + """ Test against full runpytest() output. """ + + line_count = 7 + line_len = 100 + expected_truncated_lines = 2 + testdir.makepyfile(r""" + def test_many_lines(): + a = list([str(i)[0] * %d for i in range(%d)]) + b = a[::2] + a = '\n'.join(map(str, a)) + b = '\n'.join(map(str, b)) + assert a == b + """ % (line_len, line_count)) + monkeypatch.delenv('CI', raising=False) + + result = testdir.runpytest() + # without -vv, truncate the message showing a few diff lines only + result.stdout.fnmatch_lines([ + "*- 1*", + "*- 3*", + "*- 5*", + "*truncated (%d lines hidden)*use*-vv*" % expected_truncated_lines, + ]) + + result = testdir.runpytest('-vv') + result.stdout.fnmatch_lines([ + "* 6*", + ]) + + monkeypatch.setenv('CI', '1') + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + "* 6*", + ]) + + def test_python25_compile_issue257(testdir): testdir.makepyfile(""" def test_rewritten(): @@ -364,6 +731,7 @@ def test_rewritten(): *1 failed* """) + def test_rewritten(testdir): testdir.makepyfile(""" def test_rewritten(): @@ -371,11 +739,13 @@ def test_rewritten(): """) assert testdir.runpytest().ret == 0 + def test_reprcompare_notin(mock_config): detail = plugin.pytest_assertrepr_compare( mock_config, 'not in', 'foo', 'aaafoobbb')[1:] assert detail == ["'foo' is contained here:", ' aaafoobbb', '? +++'] + def test_pytest_assertrepr_compare_integration(testdir): testdir.makepyfile(""" def test_hello(): @@ -392,6 +762,7 @@ def test_hello(): "*E*50*", ]) + def test_sequence_comparison_uses_repr(testdir): testdir.makepyfile(""" def test_hello(): @@ -410,40 +781,6 @@ def test_hello(): ]) -def test_assert_compare_truncate_longmessage(monkeypatch, testdir): - testdir.makepyfile(r""" - def test_long(): - a = list(range(200)) - b = a[::2] - a = '\n'.join(map(str, a)) - b = '\n'.join(map(str, b)) - assert a == b - """) - monkeypatch.delenv('CI', raising=False) - - result = testdir.runpytest() - # without -vv, truncate the message showing a few diff lines only - result.stdout.fnmatch_lines([ - "*- 1", - "*- 3", - "*- 5", - "*- 7", - "*truncated (191 more lines)*use*-vv*", - ]) - - - result = testdir.runpytest('-vv') - result.stdout.fnmatch_lines([ - "*- 197", - ]) - - monkeypatch.setenv('CI', '1') - result = testdir.runpytest() - result.stdout.fnmatch_lines([ - "*- 197", - ]) - - def test_assertrepr_loaded_per_dir(testdir): testdir.makepyfile(test_base=['def test_base(): assert 1 == 2']) a = testdir.mkdir('a') @@ -458,12 +795,12 @@ def test_assertrepr_loaded_per_dir(testdir): b_conftest.write('def pytest_assertrepr_compare(): return ["summary b"]') result = testdir.runpytest() result.stdout.fnmatch_lines([ - '*def test_base():*', - '*E*assert 1 == 2*', - '*def test_a():*', - '*E*assert summary a*', - '*def test_b():*', - '*E*assert summary b*']) + '*def test_base():*', + '*E*assert 1 == 2*', + '*def test_a():*', + '*E*assert summary a*', + '*def test_b():*', + '*E*assert summary b*']) def test_assertion_options(testdir): @@ -474,24 +811,9 @@ def test_hello(): """) result = testdir.runpytest() assert "3 == 4" in result.stdout.str() - off_options = (("--no-assert",), - ("--nomagic",), - ("--no-assert", "--nomagic"), - ("--assert=plain",), - ("--assert=plain", "--no-assert"), - ("--assert=plain", "--nomagic"), - ("--assert=plain", "--no-assert", "--nomagic")) - for opt in off_options: - result = testdir.runpytest_subprocess(*opt) - assert "3 == 4" not in result.stdout.str() - -def test_old_assert_mode(testdir): - testdir.makepyfile(""" - def test_in_old_mode(): - assert "@py_builtins" not in globals() - """) - result = testdir.runpytest_subprocess("--assert=reinterp") - assert result.ret == 0 + result = testdir.runpytest_subprocess("--assert=plain") + assert "3 == 4" not in result.stdout.str() + def test_triple_quoted_string_issue113(testdir): testdir.makepyfile(""" @@ -504,6 +826,7 @@ def test_hello(): ]) assert 'SyntaxError' not in result.stdout.str() + def test_traceback_failure(testdir): p1 = testdir.makepyfile(""" def g(): @@ -515,7 +838,7 @@ def test_onefails(): """) result = testdir.runpytest(p1, "--tb=long") result.stdout.fnmatch_lines([ - "*test_traceback_failure.py F", + "*test_traceback_failure.py F*", "====* FAILURES *====", "____*____", "", @@ -524,7 +847,7 @@ def test_onefails(): "", "*test_*.py:6: ", "_ _ _ *", - #"", + # "", " def f(x):", "> assert x == g()", "E assert 3 == 2", @@ -533,9 +856,9 @@ def test_onefails(): "*test_traceback_failure.py:4: AssertionError" ]) - result = testdir.runpytest(p1) # "auto" + result = testdir.runpytest(p1) # "auto" result.stdout.fnmatch_lines([ - "*test_traceback_failure.py F", + "*test_traceback_failure.py F*", "====* FAILURES *====", "____*____", "", @@ -552,18 +875,50 @@ def test_onefails(): "*test_traceback_failure.py:4: AssertionError" ]) -@pytest.mark.skipif("'__pypy__' in sys.builtin_module_names or sys.platform.startswith('java')" ) + +@pytest.mark.skipif(sys.version_info[:2] <= (3, 3), reason='Python 3.4+ shows chained exceptions on multiprocess') +def test_exception_handling_no_traceback(testdir): + """ + Handle chain exceptions in tasks submitted by the multiprocess module (#1984). + """ + p1 = testdir.makepyfile(""" + from multiprocessing import Pool + + def process_task(n): + assert n == 10 + + def multitask_job(): + tasks = [1] + with Pool(processes=1) as pool: + pool.map(process_task, tasks) + + def test_multitask_job(): + multitask_job() + """) + result = testdir.runpytest(p1, "--tb=long") + result.stdout.fnmatch_lines([ + "====* FAILURES *====", + "*multiprocessing.pool.RemoteTraceback:*", + "Traceback (most recent call last):", + "*assert n == 10", + "The above exception was the direct cause of the following exception:", + "> * multitask_job()", + ]) + + +@pytest.mark.skipif("'__pypy__' in sys.builtin_module_names or sys.platform.startswith('java')") def test_warn_missing(testdir): testdir.makepyfile("") result = testdir.run(sys.executable, "-OO", "-m", "pytest", "-h") result.stderr.fnmatch_lines([ "*WARNING*assert statements are not executed*", ]) - result = testdir.run(sys.executable, "-OO", "-m", "pytest", "--no-assert") + result = testdir.run(sys.executable, "-OO", "-m", "pytest") result.stderr.fnmatch_lines([ "*WARNING*assert statements are not executed*", ]) + def test_recursion_source_decode(testdir): testdir.makepyfile(""" def test_something(): @@ -578,6 +933,7 @@ def test_something(): """) + def test_AssertionError_message(testdir): testdir.makepyfile(""" def test_hello(): @@ -591,6 +947,7 @@ def test_hello(): *AssertionError: (1, 2)* """) + @pytest.mark.skipif(PY3, reason='This bug does not exist on PY3') def test_set_with_unsortable_elements(): # issue #718 @@ -626,3 +983,84 @@ def __hash__(self): + repr(3) """).strip() assert '\n'.join(expl) == dedent + + +def test_diff_newline_at_end(monkeypatch, testdir): + testdir.makepyfile(r""" + def test_diff(): + assert 'asdf' == 'asdf\n' + """) + + result = testdir.runpytest() + result.stdout.fnmatch_lines(r""" + *assert 'asdf' == 'asdf\n' + * - asdf + * + asdf + * ? + + """) + + +def test_assert_tuple_warning(testdir): + testdir.makepyfile(""" + def test_tuple(): + assert(False, 'you shall not pass') + """) + result = testdir.runpytest('-rw') + result.stdout.fnmatch_lines([ + '*test_assert_tuple_warning.py:2', + '*assertion is always true*', + ]) + + +def test_assert_indirect_tuple_no_warning(testdir): + testdir.makepyfile(""" + def test_tuple(): + tpl = ('foo', 'bar') + assert tpl + """) + result = testdir.runpytest('-rw') + output = '\n'.join(result.stdout.lines) + assert 'WR1' not in output + + +def test_assert_with_unicode(monkeypatch, testdir): + testdir.makepyfile(u""" + # -*- coding: utf-8 -*- + def test_unicode(): + assert u'유니코드' == u'Unicode' + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines(['*AssertionError*']) + + +def test_raise_unprintable_assertion_error(testdir): + testdir.makepyfile(r""" + def test_raise_assertion_error(): + raise AssertionError('\xff') + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([r"> raise AssertionError('\xff')", 'E AssertionError: *']) + + +def test_raise_assertion_error_raisin_repr(testdir): + testdir.makepyfile(u""" + class RaisingRepr(object): + def __repr__(self): + raise Exception() + def test_raising_repr(): + raise AssertionError(RaisingRepr()) + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines(['E AssertionError: ']) + + +def test_issue_1944(testdir): + testdir.makepyfile(""" + def f(): + return + + assert f() == 10 + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines(["*1 error*"]) + assert "AttributeError: 'Module' object has no attribute '_obj'" not in result.stdout.str() diff --git a/testing/test_assertrewrite.py b/testing/test_assertrewrite.py index f43c424ca9432ba..0e22c6dac47ad39 100644 --- a/testing/test_assertrewrite.py +++ b/testing/test_assertrewrite.py @@ -1,25 +1,30 @@ +from __future__ import absolute_import, division, print_function + +import glob import os +import py_compile import stat import sys import zipfile import py import pytest +import _pytest._code +from _pytest.assertion import util +from _pytest.assertion.rewrite import rewrite_asserts, PYTEST_TAG, AssertionRewritingHook +from _pytest.main import EXIT_NOTESTSCOLLECTED + ast = pytest.importorskip("ast") if sys.platform.startswith("java"): # XXX should be xfail pytest.skip("assert rewrite does currently not work on jython") -import _pytest._code -from _pytest.assertion import util -from _pytest.assertion.rewrite import rewrite_asserts, PYTEST_TAG -from _pytest.main import EXIT_NOTESTSCOLLECTED - def setup_module(mod): mod._old_reprcompare = util._reprcompare _pytest._code._reprcompare = None + def teardown_module(mod): util._reprcompare = mod._old_reprcompare del mod._old_reprcompare @@ -30,6 +35,7 @@ def rewrite(src): rewrite_asserts(tree) return tree + def getmsg(f, extra_ns=None, must_pass=False): """Rewrite the assertions in f, run it, and get the failure message.""" src = '\n'.join(_pytest._code.Code(f).source().lines) @@ -54,18 +60,23 @@ def getmsg(f, extra_ns=None, must_pass=False): pytest.fail("function didn't raise at all") -class TestAssertionRewrite: +class TestAssertionRewrite(object): def test_place_initial_imports(self): s = """'Doc string'\nother = stuff""" m = rewrite(s) - assert isinstance(m.body[0], ast.Expr) - assert isinstance(m.body[0].value, ast.Str) - for imp in m.body[1:3]: + # Module docstrings in 3.7 are part of Module node, it's not in the body + # so we remove it so the following body items have the same indexes on + # all Python versions + if sys.version_info < (3, 7): + assert isinstance(m.body[0], ast.Expr) + assert isinstance(m.body[0].value, ast.Str) + del m.body[0] + for imp in m.body[0:2]: assert isinstance(imp, ast.Import) assert imp.lineno == 2 assert imp.col_offset == 0 - assert isinstance(m.body[3], ast.Assign) + assert isinstance(m.body[2], ast.Assign) s = """from __future__ import with_statement\nother_stuff""" m = rewrite(s) assert isinstance(m.body[0], ast.ImportFrom) @@ -74,16 +85,29 @@ def test_place_initial_imports(self): assert imp.lineno == 2 assert imp.col_offset == 0 assert isinstance(m.body[3], ast.Expr) + s = """'doc string'\nfrom __future__ import with_statement""" + m = rewrite(s) + if sys.version_info < (3, 7): + assert isinstance(m.body[0], ast.Expr) + assert isinstance(m.body[0].value, ast.Str) + del m.body[0] + assert isinstance(m.body[0], ast.ImportFrom) + for imp in m.body[1:3]: + assert isinstance(imp, ast.Import) + assert imp.lineno == 2 + assert imp.col_offset == 0 s = """'doc string'\nfrom __future__ import with_statement\nother""" m = rewrite(s) - assert isinstance(m.body[0], ast.Expr) - assert isinstance(m.body[0].value, ast.Str) - assert isinstance(m.body[1], ast.ImportFrom) - for imp in m.body[2:4]: + if sys.version_info < (3, 7): + assert isinstance(m.body[0], ast.Expr) + assert isinstance(m.body[0].value, ast.Str) + del m.body[0] + assert isinstance(m.body[0], ast.ImportFrom) + for imp in m.body[1:3]: assert isinstance(imp, ast.Import) assert imp.lineno == 3 assert imp.col_offset == 0 - assert isinstance(m.body[4], ast.Expr) + assert isinstance(m.body[3], ast.Expr) s = """from . import relative\nother_stuff""" m = rewrite(s) for imp in m.body[0:2]: @@ -95,30 +119,43 @@ def test_place_initial_imports(self): def test_dont_rewrite(self): s = """'PYTEST_DONT_REWRITE'\nassert 14""" m = rewrite(s) - assert len(m.body) == 2 - assert isinstance(m.body[0].value, ast.Str) - assert isinstance(m.body[1], ast.Assert) - assert m.body[1].msg is None + if sys.version_info < (3, 7): + assert len(m.body) == 2 + assert isinstance(m.body[0], ast.Expr) + assert isinstance(m.body[0].value, ast.Str) + del m.body[0] + else: + assert len(m.body) == 1 + assert m.body[0].msg is None def test_name(self): def f(): assert False assert getmsg(f) == "assert False" + def f(): f = False assert f + assert getmsg(f) == "assert False" + def f(): assert a_global # noqa - assert getmsg(f, {"a_global" : False}) == "assert False" + + assert getmsg(f, {"a_global": False}) == "assert False" + def f(): assert sys == 42 - assert getmsg(f, {"sys" : sys}) == "assert sys == 42" + + assert getmsg(f, {"sys": sys}) == "assert sys == 42" + def f(): assert cls == 42 # noqa + class X(object): pass - assert getmsg(f, {"cls" : X}) == "assert cls == 42" + + assert getmsg(f, {"cls": X}) == "assert cls == 42" def test_assert_already_has_message(self): def f(): @@ -190,76 +227,110 @@ def test_boolop(self): def f(): f = g = False assert f and g + assert getmsg(f) == "assert (False)" + def f(): f = True g = False assert f and g + assert getmsg(f) == "assert (True and False)" + def f(): f = False g = True assert f and g + assert getmsg(f) == "assert (False)" + def f(): f = g = False assert f or g + assert getmsg(f) == "assert (False or False)" + def f(): f = g = False assert not f and not g + getmsg(f, must_pass=True) + def x(): return False + def f(): assert x() and x() - assert getmsg(f, {"x" : x}) == "assert (x())" + + assert getmsg(f, {"x": x}) == """assert (False) + + where False = x()""" + def f(): assert False or x() - assert getmsg(f, {"x" : x}) == "assert (False or x())" + + assert getmsg(f, {"x": x}) == """assert (False or False) + + where False = x()""" + def f(): assert 1 in {} and 2 in {} + assert getmsg(f) == "assert (1 in {})" + def f(): x = 1 y = 2 - assert x in {1 : None} and y in {} + assert x in {1: None} and y in {} + assert getmsg(f) == "assert (1 in {1: None} and 2 in {})" + def f(): f = True g = False assert f or g + getmsg(f, must_pass=True) + def f(): f = g = h = lambda: True assert f() and g() and h() + getmsg(f, must_pass=True) - def test_short_circut_evaluation(self): + def test_short_circuit_evaluation(self): def f(): assert True or explode # noqa + getmsg(f, must_pass=True) + def f(): x = 1 assert x == 1 or x == 2 + getmsg(f, must_pass=True) def test_unary_op(self): def f(): x = True assert not x + assert getmsg(f) == "assert not True" + def f(): x = 0 assert ~x + 1 + assert getmsg(f) == "assert (~0 + 1)" + def f(): x = 3 assert -x + x + assert getmsg(f) == "assert (-3 + 3)" + def f(): x = 0 assert +x + x + assert getmsg(f) == "assert (+0 + 0)" def test_binary_op(self): @@ -267,7 +338,9 @@ def f(): x = 1 y = -1 assert x + y + assert getmsg(f) == "assert (1 + -1)" + def f(): assert not 5 % 4 assert getmsg(f) == "assert not (5 % 4)" @@ -275,7 +348,9 @@ def f(): def test_boolop_percent(self): def f(): assert 3 % 2 and False + assert getmsg(f) == "assert ((3 % 2) and False)" + def f(): assert False or 4 % 2 assert getmsg(f) == "assert (False or (4 % 2))" @@ -283,7 +358,7 @@ def f(): @pytest.mark.skipif("sys.version_info < (3,5)") def test_at_operator_issue1290(self, testdir): testdir.makepyfile(""" - class Matrix: + class Matrix(object): def __init__(self, num): self.num = num def __matmul__(self, other): @@ -296,105 +371,159 @@ def test_multmat_operator(): def test_call(self): def g(a=42, *args, **kwargs): return False - ns = {"g" : g} + + ns = {"g": g} + def f(): assert g() - assert getmsg(f, ns) == """assert g()""" + + assert getmsg(f, ns) == """assert False + + where False = g()""" + def f(): assert g(1) - assert getmsg(f, ns) == """assert g(1)""" + + assert getmsg(f, ns) == """assert False + + where False = g(1)""" + def f(): assert g(1, 2) - assert getmsg(f, ns) == """assert g(1, 2)""" + + assert getmsg(f, ns) == """assert False + + where False = g(1, 2)""" + def f(): assert g(1, g=42) - assert getmsg(f, ns) == """assert g(1, g=42)""" + + assert getmsg(f, ns) == """assert False + + where False = g(1, g=42)""" + def f(): assert g(1, 3, g=23) - assert getmsg(f, ns) == """assert g(1, 3, g=23)""" + + assert getmsg(f, ns) == """assert False + + where False = g(1, 3, g=23)""" + def f(): seq = [1, 2, 3] assert g(*seq) - assert getmsg(f, ns) == """assert g(*[1, 2, 3])""" + + assert getmsg(f, ns) == """assert False + + where False = g(*[1, 2, 3])""" + def f(): x = "a" - assert g(**{x : 2}) - assert getmsg(f, ns) == """assert g(**{'a': 2})""" + assert g(**{x: 2}) + + assert getmsg(f, ns) == """assert False + + where False = g(**{'a': 2})""" def test_attribute(self): class X(object): g = 3 - ns = {"x" : X} + + ns = {"x": X} + def f(): - assert not x.g # noqa + assert not x.g # noqa + assert getmsg(f, ns) == """assert not 3 + where 3 = x.g""" + def f(): x.a = False # noqa assert x.a # noqa - assert getmsg(f, ns) == """assert x.a""" + + assert getmsg(f, ns) == """assert False + + where False = x.a""" def test_comparisons(self): + def f(): a, b = range(2) assert b < a + assert getmsg(f) == """assert 1 < 0""" + def f(): a, b, c = range(3) assert a > b > c + assert getmsg(f) == """assert 0 > 1""" + def f(): a, b, c = range(3) assert a < b > c + assert getmsg(f) == """assert 1 > 2""" + def f(): a, b, c = range(3) assert a < b <= c + getmsg(f, must_pass=True) + def f(): a, b, c = range(3) assert a < b assert b < c + getmsg(f, must_pass=True) def test_len(self): + def f(): - l = list(range(10)) - assert len(l) == 11 + values = list(range(10)) + assert len(values) == 11 + assert getmsg(f).startswith("""assert 10 == 11 + where 10 = len([""") def test_custom_reprcompare(self, monkeypatch): def my_reprcompare(op, left, right): return "42" + monkeypatch.setattr(util, "_reprcompare", my_reprcompare) + def f(): assert 42 < 3 + assert getmsg(f) == "assert 42" + def my_reprcompare(op, left, right): return "%s %s %s" % (left, op, right) + monkeypatch.setattr(util, "_reprcompare", my_reprcompare) + def f(): assert 1 < 3 < 5 <= 4 < 7 + assert getmsg(f) == "assert 5 <= 4" def test_assert_raising_nonzero_in_comparison(self): def f(): class A(object): + def __nonzero__(self): raise ValueError(42) + def __lt__(self, other): return A() + def __repr__(self): return "" + def myany(x): return False + assert myany(A() < 0) + assert " < 0" in getmsg(f) def test_formatchar(self): def f(): assert "%test" == "test" + assert getmsg(f).startswith("assert '%test' == 'test'") def test_custom_repr(self): @@ -404,12 +533,14 @@ class Foo(object): def __repr__(self): return "\n{ \n~ \n}" + f = Foo() assert 0 == f.a + assert r"where 1 = \n{ \n~ \n}.a" in util._format_lines([getmsg(f)])[0] -class TestRewriteOnImport: +class TestRewriteOnImport(object): def test_pycache_is_a_file(self, testdir): testdir.tmpdir.join("__pycache__").write("Hello") @@ -449,7 +580,7 @@ def test_zipfile(self, testdir): def test_readonly(self, testdir): sub = testdir.mkdir("testing") sub.join("test_readonly.py").write( - py.builtin._totext(""" + py.builtin._totext(""" def test_rewritten(): assert "@py_builtins" in globals() """).encode("utf-8"), "wb") @@ -470,6 +601,31 @@ def test_no_bytecode(): monkeypatch.setenv("PYTHONDONTWRITEBYTECODE", "1") assert testdir.runpytest_subprocess().ret == 0 + def test_orphaned_pyc_file(self, testdir): + if sys.version_info < (3, 0) and hasattr(sys, 'pypy_version_info'): + pytest.skip("pypy2 doesn't run orphaned pyc files") + + testdir.makepyfile(""" + import orphan + def test_it(): + assert orphan.value == 17 + """) + testdir.makepyfile(orphan=""" + value = 17 + """) + py_compile.compile("orphan.py") + os.remove("orphan.py") + + # Python 3 puts the .pyc files in a __pycache__ directory, and will + # not import from there without source. It will import a .pyc from + # the source location though. + if not os.path.exists("orphan.pyc"): + pycs = glob.glob("__pycache__/orphan.*.pyc") + assert len(pycs) == 1 + os.rename(pycs[0], "orphan.pyc") + + assert testdir.runpytest().ret == 0 + @pytest.mark.skipif('"__pypy__" in sys.modules') def test_pyc_vs_pyo(self, testdir, monkeypatch): testdir.makepyfile(""" @@ -477,7 +633,7 @@ def test_pyc_vs_pyo(self, testdir, monkeypatch): def test_optimized(): "hello" assert test_optimized.__doc__ is None""" - ) + ) p = py.path.local.make_numbered_dir(prefix="runpytest-", keep=None, rootdir=testdir.tmpdir) tmp = "--basetemp=%s" % p @@ -506,14 +662,99 @@ def test_translate_newlines(self, testdir): testdir.tmpdir.join("test_newlines.py").write(b, "wb") assert testdir.runpytest().ret == 0 - @pytest.mark.skipif(sys.version_info < (3,3), - reason='packages without __init__.py not supported on python 2') + @pytest.mark.skipif(sys.version_info < (3, 4), + reason='packages without __init__.py not supported on python 2') def test_package_without__init__py(self, testdir): pkg = testdir.mkdir('a_package_without_init_py') pkg.join('module.py').ensure() testdir.makepyfile("import a_package_without_init_py.module") assert testdir.runpytest().ret == EXIT_NOTESTSCOLLECTED + def test_rewrite_warning(self, pytestconfig, monkeypatch): + hook = AssertionRewritingHook(pytestconfig) + warnings = [] + + def mywarn(code, msg): + warnings.append((code, msg)) + + monkeypatch.setattr(hook.config, 'warn', mywarn) + hook.mark_rewrite('_pytest') + assert '_pytest' in warnings[0][1] + + def test_rewrite_module_imported_from_conftest(self, testdir): + testdir.makeconftest(''' + import test_rewrite_module_imported + ''') + testdir.makepyfile(test_rewrite_module_imported=''' + def test_rewritten(): + assert "@py_builtins" in globals() + ''') + assert testdir.runpytest_subprocess().ret == 0 + + def test_remember_rewritten_modules(self, pytestconfig, testdir, monkeypatch): + """ + AssertionRewriteHook should remember rewritten modules so it + doesn't give false positives (#2005). + """ + monkeypatch.syspath_prepend(testdir.tmpdir) + testdir.makepyfile(test_remember_rewritten_modules='') + warnings = [] + hook = AssertionRewritingHook(pytestconfig) + monkeypatch.setattr(hook.config, 'warn', lambda code, msg: warnings.append(msg)) + hook.find_module('test_remember_rewritten_modules') + hook.load_module('test_remember_rewritten_modules') + hook.mark_rewrite('test_remember_rewritten_modules') + hook.mark_rewrite('test_remember_rewritten_modules') + assert warnings == [] + + def test_rewrite_warning_using_pytest_plugins(self, testdir): + testdir.makepyfile(**{ + 'conftest.py': "pytest_plugins = ['core', 'gui', 'sci']", + 'core.py': "", + 'gui.py': "pytest_plugins = ['core', 'sci']", + 'sci.py': "pytest_plugins = ['core']", + 'test_rewrite_warning_pytest_plugins.py': "def test(): pass", + }) + testdir.chdir() + result = testdir.runpytest_subprocess() + result.stdout.fnmatch_lines(['*= 1 passed in *=*']) + assert 'pytest-warning summary' not in result.stdout.str() + + def test_rewrite_warning_using_pytest_plugins_env_var(self, testdir, monkeypatch): + monkeypatch.setenv('PYTEST_PLUGINS', 'plugin') + testdir.makepyfile(**{ + 'plugin.py': "", + 'test_rewrite_warning_using_pytest_plugins_env_var.py': """ + import plugin + pytest_plugins = ['plugin'] + def test(): + pass + """, + }) + testdir.chdir() + result = testdir.runpytest_subprocess() + result.stdout.fnmatch_lines(['*= 1 passed in *=*']) + assert 'pytest-warning summary' not in result.stdout.str() + + @pytest.mark.skipif(sys.version_info[0] > 2, reason='python 2 only') + def test_rewrite_future_imports(self, testdir): + """Test that rewritten modules don't inherit the __future__ flags + from the assertrewrite module. + + assertion.rewrite imports __future__.division (and others), so + ensure rewritten modules don't inherit those flags. + + The test below will fail if __future__.division is enabled + """ + testdir.makepyfile(''' + def test(): + x = 1 / 2 + assert type(x) is int + ''') + result = testdir.runpytest() + assert result.ret == 0 + + class TestAssertionRewriteHookDetails(object): def test_loader_is_package_false_for_module(self, testdir): testdir.makepyfile(test_fun=""" @@ -596,10 +837,12 @@ def test_write_pyc(self, testdir, tmpdir, monkeypatch): source_path = tmpdir.ensure("source.py") pycpath = tmpdir.join("pyc").strpath assert _write_pyc(state, [1], source_path.stat(), pycpath) + def open(*args): e = IOError() e.errno = 10 raise e + monkeypatch.setattr(b, "open", open) assert not _write_pyc(state, [1], source_path.stat(), pycpath) @@ -684,7 +927,7 @@ def test_get_data_support(self, testdir): """ path = testdir.mkpydir("foo") path.join("test_foo.py").write(_pytest._code.Source(""" - class Test: + class Test(object): def test_foo(self): import pkgutil data = pkgutil.get_data('foo.test_foo', 'data.txt') @@ -712,5 +955,42 @@ def test_long_repr(): assert 'unbalanced braces' not in result.stdout.str() -def test_collapse_false_unbalanced_braces(): - util._collapse_false('some text{ False\n{False = some more text\n}') +class TestIssue925(object): + def test_simple_case(self, testdir): + testdir.makepyfile(""" + def test_ternary_display(): + assert (False == False) == False + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines('*E*assert (False == False) == False') + + def test_long_case(self, testdir): + testdir.makepyfile(""" + def test_ternary_display(): + assert False == (False == True) == True + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines('*E*assert (False == True) == True') + + def test_many_brackets(self, testdir): + testdir.makepyfile(""" + def test_ternary_display(): + assert True == ((False == True) == True) + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines('*E*assert True == ((False == True) == True)') + + +class TestIssue2121(): + def test_simple(self, testdir): + testdir.tmpdir.join("tests/file.py").ensure().write(""" +def test_simple_failure(): + assert 1 + 1 == 3 +""") + testdir.tmpdir.join("pytest.ini").write(py.std.textwrap.dedent(""" + [pytest] + python_files = tests/**.py + """)) + + result = testdir.runpytest() + result.stdout.fnmatch_lines('*E*assert (1 + 1) == 3') diff --git a/testing/test_cache.py b/testing/test_cache.py index 98053f86947c1da..a37170cdd2b16ab 100755 --- a/testing/test_cache.py +++ b/testing/test_cache.py @@ -1,5 +1,6 @@ +from __future__ import absolute_import, division, print_function import sys - +import py import _pytest import pytest import os @@ -7,7 +8,8 @@ pytest_plugins = "pytester", -class TestNewAPI: + +class TestNewAPI(object): def test_config_cache_makedir(self, testdir): testdir.makeini("[pytest]") config = testdir.parseconfigure() @@ -54,7 +56,7 @@ def test_error(): assert result.ret == 1 result.stdout.fnmatch_lines([ "*could not create cache path*", - "*1 pytest-warnings*", + "*1 warnings*", ]) def test_config_cache(self, testdir): @@ -86,6 +88,36 @@ def test_cachefuncarg(cache): assert result.ret == 0 result.stdout.fnmatch_lines(["*1 passed*"]) + def test_custom_rel_cache_dir(self, testdir): + rel_cache_dir = os.path.join('custom_cache_dir', 'subdir') + testdir.makeini(""" + [pytest] + cache_dir = {cache_dir} + """.format(cache_dir=rel_cache_dir)) + testdir.makepyfile(test_errored='def test_error():\n assert False') + testdir.runpytest() + assert testdir.tmpdir.join(rel_cache_dir).isdir() + + def test_custom_abs_cache_dir(self, testdir, tmpdir_factory): + tmp = str(tmpdir_factory.mktemp('tmp')) + abs_cache_dir = os.path.join(tmp, 'custom_cache_dir') + testdir.makeini(""" + [pytest] + cache_dir = {cache_dir} + """.format(cache_dir=abs_cache_dir)) + testdir.makepyfile(test_errored='def test_error():\n assert False') + testdir.runpytest() + assert py.path.local(abs_cache_dir).isdir() + + def test_custom_cache_dir_with_env_var(self, testdir, monkeypatch): + monkeypatch.setenv('env_var', 'custom_cache_dir') + testdir.makeini(""" + [pytest] + cache_dir = {cache_dir} + """.format(cache_dir='$env_var')) + testdir.makepyfile(test_errored='def test_error():\n assert False') + testdir.runpytest() + assert testdir.tmpdir.join('custom_cache_dir').isdir() def test_cache_reportheader(testdir): @@ -129,7 +161,7 @@ def pytest_configure(config): ]) -class TestLastFailed: +class TestLastFailed(object): def test_lastfailed_usecase(self, testdir, monkeypatch): monkeypatch.setenv("PYTHONDONTWRITEBYTECODE", 1) @@ -191,13 +223,37 @@ def test_always_fails(): "test_a.py*", "test_b.py*", ]) - result = testdir.runpytest("--lf", "--ff") + result = testdir.runpytest("--ff") # Test order will be failing tests firs result.stdout.fnmatch_lines([ "test_b.py*", "test_a.py*", ]) + def test_lastfailed_failedfirst_order(self, testdir): + testdir.makepyfile(**{ + 'test_a.py': """ + def test_always_passes(): + assert 1 + """, + 'test_b.py': """ + def test_always_fails(): + assert 0 + """, + }) + result = testdir.runpytest() + # Test order will be collection order; alphabetical + result.stdout.fnmatch_lines([ + "test_a.py*", + "test_b.py*", + ]) + result = testdir.runpytest("--lf", "--ff") + # Test order will be failing tests firs + result.stdout.fnmatch_lines([ + "test_b.py*", + ]) + assert 'test_a.py' not in result.stdout.str() + def test_lastfailed_difference_invocations(self, testdir, monkeypatch): monkeypatch.setenv("PYTHONDONTWRITEBYTECODE", 1) testdir.makepyfile(test_a=""" @@ -284,6 +340,73 @@ def test_fail(val): result = testdir.runpytest() result.stdout.fnmatch_lines('*1 failed in*') + def test_terminal_report_lastfailed(self, testdir): + test_a = testdir.makepyfile(test_a=""" + def test_a1(): + pass + def test_a2(): + pass + """) + test_b = testdir.makepyfile(test_b=""" + def test_b1(): + assert 0 + def test_b2(): + assert 0 + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + 'collected 4 items', + '*2 failed, 2 passed in*', + ]) + + result = testdir.runpytest('--lf') + result.stdout.fnmatch_lines([ + 'collected 4 items', + 'run-last-failure: rerun previous 2 failures', + '*2 failed, 2 deselected in*', + ]) + + result = testdir.runpytest(test_a, '--lf') + result.stdout.fnmatch_lines([ + 'collected 2 items', + 'run-last-failure: run all (no recorded failures)', + '*2 passed in*', + ]) + + result = testdir.runpytest(test_b, '--lf') + result.stdout.fnmatch_lines([ + 'collected 2 items', + 'run-last-failure: rerun previous 2 failures', + '*2 failed in*', + ]) + + result = testdir.runpytest('test_b.py::test_b1', '--lf') + result.stdout.fnmatch_lines([ + 'collected 1 item', + 'run-last-failure: rerun previous 1 failure', + '*1 failed in*', + ]) + + def test_terminal_report_failedfirst(self, testdir): + testdir.makepyfile(test_a=""" + def test_a1(): + assert 0 + def test_a2(): + pass + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + 'collected 2 items', + '*1 failed, 1 passed in*', + ]) + + result = testdir.runpytest('--ff') + result.stdout.fnmatch_lines([ + 'collected 2 items', + 'run-last-failure: rerun previous 1 failure first', + '*1 failed, 1 passed in*', + ]) + def test_lastfailed_collectfailure(self, testdir, monkeypatch): testdir.makepyfile(test_maybe=""" @@ -313,7 +436,6 @@ def rlf(fail_import, fail_run): lastfailed = rlf(fail_import=0, fail_run=1) assert list(lastfailed) == ['test_maybe.py::test_hello'] - def test_lastfailed_failure_subset(self, testdir, monkeypatch): testdir.makepyfile(test_maybe=""" @@ -355,12 +477,10 @@ def rlf(fail_import, fail_run, args=()): result, lastfailed = rlf(fail_import=1, fail_run=0) assert sorted(list(lastfailed)) == ['test_maybe.py', 'test_maybe2.py'] - result, lastfailed = rlf(fail_import=0, fail_run=0, args=('test_maybe2.py',)) assert list(lastfailed) == ['test_maybe.py'] - # edge case of test selection - even if we remember failures # from other tests we still need to run all tests if no test # matches the failures @@ -384,3 +504,102 @@ def test_lastfailed_creates_cache_when_needed(self, testdir): testdir.makepyfile(test_errored='def test_error():\n assert False') testdir.runpytest('-q', '--lf') assert os.path.exists('.cache') + + def test_xfail_not_considered_failure(self, testdir): + testdir.makepyfile(''' + import pytest + @pytest.mark.xfail + def test(): + assert 0 + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines('*1 xfailed*') + assert self.get_cached_last_failed(testdir) == [] + + def test_xfail_strict_considered_failure(self, testdir): + testdir.makepyfile(''' + import pytest + @pytest.mark.xfail(strict=True) + def test(): + pass + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines('*1 failed*') + assert self.get_cached_last_failed(testdir) == ['test_xfail_strict_considered_failure.py::test'] + + @pytest.mark.parametrize('mark', ['mark.xfail', 'mark.skip']) + def test_failed_changed_to_xfail_or_skip(self, testdir, mark): + testdir.makepyfile(''' + import pytest + def test(): + assert 0 + ''') + result = testdir.runpytest() + assert self.get_cached_last_failed(testdir) == ['test_failed_changed_to_xfail_or_skip.py::test'] + assert result.ret == 1 + + testdir.makepyfile(''' + import pytest + @pytest.{mark} + def test(): + assert 0 + '''.format(mark=mark)) + result = testdir.runpytest() + assert result.ret == 0 + assert self.get_cached_last_failed(testdir) == [] + assert result.ret == 0 + + def get_cached_last_failed(self, testdir): + config = testdir.parseconfigure() + return sorted(config.cache.get("cache/lastfailed", {})) + + def test_cache_cumulative(self, testdir): + """ + Test workflow where user fixes errors gradually file by file using --lf. + """ + # 1. initial run + test_bar = testdir.makepyfile(test_bar=""" + def test_bar_1(): + pass + def test_bar_2(): + assert 0 + """) + test_foo = testdir.makepyfile(test_foo=""" + def test_foo_3(): + pass + def test_foo_4(): + assert 0 + """) + testdir.runpytest() + assert self.get_cached_last_failed(testdir) == ['test_bar.py::test_bar_2', 'test_foo.py::test_foo_4'] + + # 2. fix test_bar_2, run only test_bar.py + testdir.makepyfile(test_bar=""" + def test_bar_1(): + pass + def test_bar_2(): + pass + """) + result = testdir.runpytest(test_bar) + result.stdout.fnmatch_lines('*2 passed*') + # ensure cache does not forget that test_foo_4 failed once before + assert self.get_cached_last_failed(testdir) == ['test_foo.py::test_foo_4'] + + result = testdir.runpytest('--last-failed') + result.stdout.fnmatch_lines('*1 failed, 3 deselected*') + assert self.get_cached_last_failed(testdir) == ['test_foo.py::test_foo_4'] + + # 3. fix test_foo_4, run only test_foo.py + test_foo = testdir.makepyfile(test_foo=""" + def test_foo_3(): + pass + def test_foo_4(): + pass + """) + result = testdir.runpytest(test_foo, '--last-failed') + result.stdout.fnmatch_lines('*1 passed, 1 deselected*') + assert self.get_cached_last_failed(testdir) == [] + + result = testdir.runpytest('--last-failed') + result.stdout.fnmatch_lines('*4 passed*') + assert self.get_cached_last_failed(testdir) == [] diff --git a/testing/test_capture.py b/testing/test_capture.py index 73660692b8e9f8b..f769a725dc4b5fd 100644 --- a/testing/test_capture.py +++ b/testing/test_capture.py @@ -1,9 +1,11 @@ +from __future__ import absolute_import, division, print_function # note: py.io capture tests where copied from # pylib 1.4.20.dev2 (rev 13d9af95547e) from __future__ import with_statement import pickle import os import sys +from io import UnsupportedOperation import _pytest._code import py @@ -13,7 +15,7 @@ from _pytest import capture from _pytest.capture import CaptureManager from _pytest.main import EXIT_NOTESTSCOLLECTED -from py.builtin import print_ + needsosdup = pytest.mark.xfail("not hasattr(os, 'dup')") @@ -47,15 +49,15 @@ def oswritebytes(fd, obj): os.write(fd, tobytes(obj)) - def StdCaptureFD(out=True, err=True, in_=True): return capture.MultiCapture(out, err, in_, Capture=capture.FDCapture) + def StdCapture(out=True, err=True, in_=True): return capture.MultiCapture(out, err, in_, Capture=capture.SysCapture) -class TestCaptureManager: +class TestCaptureManager(object): def test_getmethod_default_no_fd(self, monkeypatch): from _pytest.capture import pytest_addoption from _pytest.config import Parser @@ -70,29 +72,29 @@ def test_getmethod_default_no_fd(self, monkeypatch): @needsosdup @pytest.mark.parametrize("method", - ['no', 'sys', pytest.mark.skipif('not hasattr(os, "dup")', 'fd')]) + ['no', 'sys', pytest.mark.skipif('not hasattr(os, "dup")', 'fd')]) def test_capturing_basic_api(self, method): capouter = StdCaptureFD() old = sys.stdout, sys.stderr, sys.stdin try: capman = CaptureManager(method) - capman.init_capturings() - outerr = capman.suspendcapture() + capman.start_global_capturing() + outerr = capman.suspend_global_capture() assert outerr == ("", "") - outerr = capman.suspendcapture() + outerr = capman.suspend_global_capture() assert outerr == ("", "") - print ("hello") - out, err = capman.suspendcapture() + print("hello") + out, err = capman.suspend_global_capture() if method == "no": assert old == (sys.stdout, sys.stderr, sys.stdin) else: assert not out - capman.resumecapture() - print ("hello") - out, err = capman.suspendcapture() + capman.resume_global_capture() + print("hello") + out, err = capman.suspend_global_capture() if method != "no": assert out == "hello\n" - capman.reset_capturings() + capman.stop_global_capturing() finally: capouter.stop_capturing() @@ -101,16 +103,16 @@ def test_init_capturing(self): capouter = StdCaptureFD() try: capman = CaptureManager("fd") - capman.init_capturings() - pytest.raises(AssertionError, "capman.init_capturings()") - capman.reset_capturings() + capman.start_global_capturing() + pytest.raises(AssertionError, "capman.start_global_capturing()") + capman.stop_global_capturing() finally: capouter.stop_capturing() @pytest.mark.parametrize("method", ['fd', 'sys']) def test_capturing_unicode(testdir, method): - if hasattr(sys, "pypy_version_info") and sys.pypy_version_info < (2,2): + if hasattr(sys, "pypy_version_info") and sys.pypy_version_info < (2, 2): pytest.xfail("does not work on pypy < 2.2") if sys.version_info >= (3, 0): obj = "'b\u00f6y'" @@ -154,7 +156,7 @@ def test_collect_capturing(testdir): ]) -class TestPerTestCapturing: +class TestPerTestCapturing(object): def test_capture_and_fixtures(self, testdir): p = testdir.makepyfile(""" def setup_module(mod): @@ -232,7 +234,7 @@ def test_func1(): "setup func1*", "in func1*", "teardown func1*", - #"*1 fixture failure*" + # "*1 fixture failure*" ]) def test_teardown_capturing_final(self, testdir): @@ -264,7 +266,7 @@ def test_capturing_error(): """) result = testdir.runpytest(p1) result.stdout.fnmatch_lines([ - "*test_capturing_outerr.py .F", + "*test_capturing_outerr.py .F*", "====* FAILURES *====", "____*____", "*test_capturing_outerr.py:8: ValueError", @@ -275,18 +277,18 @@ def test_capturing_error(): ]) -class TestLoggingInteraction: +class TestLoggingInteraction(object): def test_logging_stream_ownership(self, testdir): p = testdir.makepyfile(""" def test_logging(): import logging import pytest - stream = capture.TextIO() + stream = capture.CaptureIO() logging.basicConfig(stream=stream) stream.close() # to free memory/release resources """) result = testdir.runpytest_subprocess(p) - result.stderr.str().find("atexit") == -1 + assert result.stderr.str().find("atexit") == -1 def test_logging_and_immediate_setupteardown(self, testdir): p = testdir.makepyfile(""" @@ -303,7 +305,7 @@ def teardown_function(function): assert 0 """) for optargs in (('--capture=sys',), ('--capture=fd',)): - print (optargs) + print(optargs) result = testdir.runpytest_subprocess(p, *optargs) s = result.stdout.str() result.stdout.fnmatch_lines([ @@ -329,7 +331,7 @@ def teardown_module(function): assert 0 """) for optargs in (('--capture=sys',), ('--capture=fd',)): - print (optargs) + print(optargs) result = testdir.runpytest_subprocess(p, *optargs) s = result.stdout.str() result.stdout.fnmatch_lines([ @@ -340,26 +342,6 @@ def teardown_module(function): # verify proper termination assert "closed" not in s - def test_logging_initialized_in_test(self, testdir): - p = testdir.makepyfile(""" - import sys - def test_something(): - # pytest does not import logging - assert 'logging' not in sys.modules - import logging - logging.basicConfig() - logging.warn("hello432") - assert 0 - """) - result = testdir.runpytest_subprocess( - p, "--traceconfig", - "-p", "no:capturelog") - assert result.ret != 0 - result.stdout.fnmatch_lines([ - "*hello432*", - ]) - assert 'operation on closed file' not in result.stderr.str() - def test_conftestlogging_is_shown(self, testdir): testdir.makeconftest(""" import logging @@ -395,7 +377,7 @@ def test_hello(): assert 'operation on closed file' not in result.stderr.str() -class TestCaptureFixture: +class TestCaptureFixture(object): @pytest.mark.parametrize("opt", [[], ["-s"]]) def test_std_functional(self, testdir, opt): reprec = testdir.inline_runsource(""" @@ -416,11 +398,41 @@ def test_two(capfd, capsys): result = testdir.runpytest(p) result.stdout.fnmatch_lines([ "*ERROR*setup*test_one*", - "*capsys*capfd*same*time*", + "E*capfd*capsys*same*time*", "*ERROR*setup*test_two*", - "*capsys*capfd*same*time*", + "E*capsys*capfd*same*time*", "*2 error*"]) + def test_capturing_getfixturevalue(self, testdir): + """Test that asking for "capfd" and "capsys" using request.getfixturevalue + in the same test is an error. + """ + testdir.makepyfile(""" + def test_one(capsys, request): + request.getfixturevalue("capfd") + def test_two(capfd, request): + request.getfixturevalue("capsys") + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + "*test_one*", + "*capsys*capfd*same*time*", + "*test_two*", + "*capfd*capsys*same*time*", + "*2 failed in*", + ]) + + def test_capsyscapfdbinary(self, testdir): + p = testdir.makepyfile(""" + def test_one(capsys, capfdbinary): + pass + """) + result = testdir.runpytest(p) + result.stdout.fnmatch_lines([ + "*ERROR*setup*test_one*", + "E*capfdbinary*capsys*same*time*", + "*1 error*"]) + @pytest.mark.parametrize("method", ["sys", "fd"]) def test_capture_is_represented_on_failure_issue128(self, testdir, method): p = testdir.makepyfile(""" @@ -445,6 +457,51 @@ def test_hello(capfd): """) reprec.assertoutcome(passed=1) + @needsosdup + def test_capfdbinary(self, testdir): + reprec = testdir.inline_runsource(""" + def test_hello(capfdbinary): + import os + # some likely un-decodable bytes + os.write(1, b'\\xfe\\x98\\x20') + out, err = capfdbinary.readouterr() + assert out == b'\\xfe\\x98\\x20' + assert err == b'' + """) + reprec.assertoutcome(passed=1) + + @pytest.mark.skipif( + sys.version_info < (3,), + reason='only have capsysbinary in python 3', + ) + def test_capsysbinary(self, testdir): + reprec = testdir.inline_runsource(""" + def test_hello(capsysbinary): + import sys + # some likely un-decodable bytes + sys.stdout.buffer.write(b'\\xfe\\x98\\x20') + out, err = capsysbinary.readouterr() + assert out == b'\\xfe\\x98\\x20' + assert err == b'' + """) + reprec.assertoutcome(passed=1) + + @pytest.mark.skipif( + sys.version_info >= (3,), + reason='only have capsysbinary in python 3', + ) + def test_capsysbinary_forbidden_in_python2(self, testdir): + testdir.makepyfile(""" + def test_hello(capsysbinary): + pass + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + "*test_hello*", + "*capsysbinary is only supported on python 3*", + "*1 error in*", + ]) + def test_partial_setup_failure(self, testdir): p = testdir.makepyfile(""" def test_hello(capsys, missingarg): @@ -480,6 +537,66 @@ def test_log(capsys): result = testdir.runpytest_subprocess(p) assert 'closed' not in result.stderr.str() + @pytest.mark.parametrize('fixture', ['capsys', 'capfd']) + @pytest.mark.parametrize('no_capture', [True, False]) + def test_disabled_capture_fixture(self, testdir, fixture, no_capture): + testdir.makepyfile(""" + def test_disabled({fixture}): + print('captured before') + with {fixture}.disabled(): + print('while capture is disabled') + print('captured after') + assert {fixture}.readouterr() == ('captured before\\ncaptured after\\n', '') + + def test_normal(): + print('test_normal executed') + """.format(fixture=fixture)) + args = ('-s',) if no_capture else () + result = testdir.runpytest_subprocess(*args) + result.stdout.fnmatch_lines(""" + *while capture is disabled* + """) + assert 'captured before' not in result.stdout.str() + assert 'captured after' not in result.stdout.str() + if no_capture: + assert 'test_normal executed' in result.stdout.str() + else: + assert 'test_normal executed' not in result.stdout.str() + + @pytest.mark.parametrize('fixture', ['capsys', 'capfd']) + def test_fixture_use_by_other_fixtures(self, testdir, fixture): + """ + Ensure that capsys and capfd can be used by other fixtures during setup and teardown. + """ + testdir.makepyfile(""" + from __future__ import print_function + import sys + import pytest + + @pytest.fixture + def captured_print({fixture}): + print('stdout contents begin') + print('stderr contents begin', file=sys.stderr) + out, err = {fixture}.readouterr() + + yield out, err + + print('stdout contents end') + print('stderr contents end', file=sys.stderr) + out, err = {fixture}.readouterr() + assert out == 'stdout contents end\\n' + assert err == 'stderr contents end\\n' + + def test_captured_print(captured_print): + out, err = captured_print + assert out == 'stdout contents begin\\n' + assert err == 'stderr contents begin\\n' + """.format(fixture=fixture)) + result = testdir.runpytest_subprocess() + result.stdout.fnmatch_lines("*1 passed*") + assert 'stdout contents begin' not in result.stdout.str() + assert 'stderr contents begin' not in result.stdout.str() + def test_setup_failure_does_not_kill_capturing(testdir): sub1 = testdir.mkpydir("sub1") @@ -569,7 +686,7 @@ def test_foo(): def test_error_during_readouterr(testdir): - """Make sure we suspend capturing if errors occurr during readouterr""" + """Make sure we suspend capturing if errors occur during readouterr""" testdir.makepyfile(pytest_xyz=""" from _pytest.capture import FDCapture def bad_snap(self): @@ -587,16 +704,16 @@ def bad_snap(self): ]) -class TestTextIO: +class TestCaptureIO(object): def test_text(self): - f = capture.TextIO() + f = capture.CaptureIO() f.write("hello") s = f.getvalue() assert s == "hello" f.close() def test_unicode_and_str_mixture(self): - f = capture.TextIO() + f = capture.CaptureIO() if sys.version_info >= (3, 0): f.write("\u00f6") pytest.raises(TypeError, "f.write(bytes('hello', 'UTF-8'))") @@ -607,6 +724,18 @@ def test_unicode_and_str_mixture(self): f.close() assert isinstance(s, unicode) + @pytest.mark.skipif( + sys.version_info[0] == 2, + reason='python 3 only behaviour', + ) + def test_write_bytes_to_buffer(self): + """In python3, stdout / stderr are text io wrappers (exposing a buffer + property of the underlying bytestream). See issue #1407 + """ + f = capture.CaptureIO() + f.buffer.write(b'foo\r\n') + assert f.getvalue() == 'foo\r\n' + def test_bytes_io(): f = py.io.BytesIO() @@ -623,7 +752,29 @@ def test_dontreadfrominput(): pytest.raises(IOError, f.read) pytest.raises(IOError, f.readlines) pytest.raises(IOError, iter, f) - pytest.raises(ValueError, f.fileno) + pytest.raises(UnsupportedOperation, f.fileno) + f.close() # just for completeness + + +@pytest.mark.skipif('sys.version_info < (3,)', reason='python2 has no buffer') +def test_dontreadfrominput_buffer_python3(): + from _pytest.capture import DontReadFromInput + f = DontReadFromInput() + fb = f.buffer + assert not fb.isatty() + pytest.raises(IOError, fb.read) + pytest.raises(IOError, fb.readlines) + pytest.raises(IOError, iter, fb) + pytest.raises(ValueError, fb.fileno) + f.close() # just for completeness + + +@pytest.mark.skipif('sys.version_info >= (3,)', reason='python2 has no buffer') +def test_dontreadfrominput_buffer_python2(): + from _pytest.capture import DontReadFromInput + f = DontReadFromInput() + with pytest.raises(AttributeError): + f.buffer f.close() # just for completeness @@ -634,6 +785,7 @@ def tmpfile(testdir): if not f.closed: f.close() + @needsosdup def test_dupfile(tmpfile): flist = [] @@ -642,27 +794,39 @@ def test_dupfile(tmpfile): assert nf != tmpfile assert nf.fileno() != tmpfile.fileno() assert nf not in flist - print_(i, end="", file=nf) + print(i, end="", file=nf) flist.append(nf) + + fname_open = flist[0].name + assert fname_open == repr(flist[0].buffer) + for i in range(5): f = flist[i] f.close() + fname_closed = flist[0].name + assert fname_closed == repr(flist[0].buffer) + assert fname_closed != fname_open tmpfile.seek(0) s = tmpfile.read() assert "01234" in repr(s) tmpfile.close() + assert fname_closed == repr(flist[0].buffer) + def test_dupfile_on_bytesio(): io = py.io.BytesIO() f = capture.safe_text_dupfile(io, "wb") f.write("hello") assert io.getvalue() == b"hello" + assert 'BytesIO object' in f.name + def test_dupfile_on_textio(): io = py.io.TextIO() f = capture.safe_text_dupfile(io, "wb") f.write("hello") assert io.getvalue() == "hello" + assert not hasattr(f, 'name') @contextlib.contextmanager @@ -680,7 +844,7 @@ def lsof_check(): assert len2 < len1 + 3, out2 -class TestFDCapture: +class TestFDCapture(object): pytestmark = needsosdup def test_simple(self, tmpfile): @@ -716,7 +880,7 @@ def test_simple_fail_second_start(self, tmpfile): def test_stderr(self): cap = capture.FDCapture(2) cap.start() - print_("hello", file=sys.stderr) + print("hello", file=sys.stderr) s = cap.snap() cap.done() assert s == "hello\n" @@ -775,7 +939,7 @@ def saved_fd(fd): os.close(new_fd) -class TestStdCapture: +class TestStdCapture(object): captureclass = staticmethod(StdCapture) @contextlib.contextmanager @@ -805,7 +969,7 @@ def test_capturing_reset_simple(self): def test_capturing_readouterr(self): with self.getcapture() as cap: - print ("hello world") + print("hello world") sys.stderr.write("hello error\n") out, err = cap.readouterr() assert out == "hello world\n" @@ -814,9 +978,17 @@ def test_capturing_readouterr(self): out, err = cap.readouterr() assert err == "error2" + def test_capture_results_accessible_by_attribute(self): + with self.getcapture() as cap: + sys.stdout.write("hello") + sys.stderr.write("world") + capture_result = cap.readouterr() + assert capture_result.out == "hello" + assert capture_result.err == "world" + def test_capturing_readouterr_unicode(self): with self.getcapture() as cap: - print ("hx\xc4\x85\xc4\x87") + print("hx\xc4\x85\xc4\x87") out, err = cap.readouterr() assert out == py.builtin._totext("hx\xc4\x85\xc4\x87\n", "utf8") @@ -831,7 +1003,7 @@ def test_capturing_readouterr_decode_error_handling(self): def test_reset_twice_error(self): with self.getcapture() as cap: - print ("hello") + print("hello") out, err = cap.readouterr() pytest.raises(ValueError, cap.stop_capturing) assert out == "hello\n" @@ -843,9 +1015,9 @@ def test_capturing_modify_sysouterr_in_between(self): with self.getcapture() as cap: sys.stdout.write("hello") sys.stderr.write("world") - sys.stdout = capture.TextIO() - sys.stderr = capture.TextIO() - print ("not seen") + sys.stdout = capture.CaptureIO() + sys.stderr = capture.CaptureIO() + print("not seen") sys.stderr.write("not seen\n") out, err = cap.readouterr() assert out == "hello" @@ -855,9 +1027,9 @@ def test_capturing_modify_sysouterr_in_between(self): def test_capturing_error_recursive(self): with self.getcapture() as cap1: - print ("cap1") + print("cap1") with self.getcapture() as cap2: - print ("cap2") + print("cap2") out2, err2 = cap2.readouterr() out1, err1 = cap1.readouterr() assert out1 == "cap1\n" @@ -887,9 +1059,9 @@ def test_stdin_restored(self): assert sys.stdin is old def test_stdin_nulled_by_default(self): - print ("XXX this test may well hang instead of crashing") - print ("XXX which indicates an error in the underlying capturing") - print ("XXX mechanisms") + print("XXX this test may well hang instead of crashing") + print("XXX which indicates an error in the underlying capturing") + print("XXX mechanisms") with self.getcapture(): pytest.raises(IOError, "sys.stdin.read()") @@ -933,7 +1105,7 @@ def test_many(self, capfd): cap.stop_capturing() -class TestStdCaptureFDinvalidFD: +class TestStdCaptureFDinvalidFD(object): pytestmark = needsosdup def test_stdcapture_fd_invalid_fd(self, testdir): @@ -966,6 +1138,23 @@ def test_capture_not_started_but_reset(): capsys.stop_capturing() +def test_using_capsys_fixture_works_with_sys_stdout_encoding(capsys): + test_text = 'test text' + + print(test_text.encode(sys.stdout.encoding, 'replace')) + (out, err) = capsys.readouterr() + assert out + assert err == '' + + +def test_capsys_results_accessible_by_attribute(capsys): + sys.stdout.write("spam") + sys.stderr.write("eggs") + capture_result = capsys.readouterr() + assert capture_result.out == "spam" + assert capture_result.err == "eggs" + + @needsosdup @pytest.mark.parametrize('use', [True, False]) def test_fdcapture_tmpfile_remains_the_same(tmpfile, use): @@ -981,6 +1170,7 @@ def test_fdcapture_tmpfile_remains_the_same(tmpfile, use): capfile2 = cap.err.tmpfile assert capfile2 == capfile + @needsosdup def test_close_and_capture_again(testdir): testdir.makepyfile(""" @@ -1000,7 +1190,6 @@ def test_capture_again(): """) - @pytest.mark.parametrize('method', ['SysCapture', 'FDCapture']) def test_capturing_and_logging_fundamentals(testdir, method): if method == "StdCaptureFD" and not hasattr(os, 'dup'): @@ -1047,6 +1236,23 @@ def test_capattr(): reprec.assertoutcome(passed=1) +@pytest.mark.skipif(not sys.platform.startswith('win') and sys.version_info[:2] >= (3, 6), + reason='only py3.6+ on windows') +def test_py36_windowsconsoleio_workaround_non_standard_streams(): + """ + Ensure _py36_windowsconsoleio_workaround function works with objects that + do not implement the full ``io``-based stream protocol, for example execnet channels (#2666). + """ + from _pytest.capture import _py36_windowsconsoleio_workaround + + class DummyStream: + def write(self, s): + pass + + stream = DummyStream() + _py36_windowsconsoleio_workaround(stream) + + def test_dontreadfrominput_has_encoding(testdir): testdir.makepyfile(""" import sys @@ -1059,7 +1265,7 @@ def test_capattr(): reprec.assertoutcome(passed=1) -def test_pickling_and_unpickling_enocded_file(): +def test_pickling_and_unpickling_encoded_file(): # See https://bitbucket.org/pytest-dev/pytest/pull-request/194 # pickle.loads() raises infinite recursion if # EncodedFile.__getattr__ is not implemented properly diff --git a/testing/test_collection.py b/testing/test_collection.py index 749c5b7ce452e71..563ed0439c0ce87 100644 --- a/testing/test_collection.py +++ b/testing/test_collection.py @@ -1,8 +1,12 @@ -import pytest, py +from __future__ import absolute_import, division, print_function +import pytest +import py -from _pytest.main import Session, EXIT_NOTESTSCOLLECTED +import _pytest._code +from _pytest.main import Session, EXIT_NOTESTSCOLLECTED, _in_venv -class TestCollector: + +class TestCollector(object): def test_collect_versus_item(self): from pytest import Collector, Item assert not issubclass(Collector, Item) @@ -41,16 +45,16 @@ def test_fail(): assert 0 assert not (fn1 == fn3) assert fn1 != fn3 - for fn in fn1,fn2,fn3: + for fn in fn1, fn2, fn3: assert fn != 3 assert fn != modcol - assert fn != [1,2,3] - assert [1,2,3] != fn + assert fn != [1, 2, 3] + assert [1, 2, 3] != fn assert modcol != fn def test_getparent(self, testdir): modcol = testdir.getmodulecol(""" - class TestClass: + class TestClass(object): def test_foo(): pass """) @@ -67,7 +71,6 @@ def test_foo(): parent = fn.getparent(pytest.Class) assert parent is cls - def test_getcustomfile_roundtrip(self, testdir): hello = testdir.makefile(".xxx", hello="world") testdir.makepyfile(conftest=""" @@ -85,9 +88,28 @@ def pytest_collect_file(path, parent): assert len(nodes) == 1 assert isinstance(nodes[0], pytest.File) -class TestCollectFS: + def test_can_skip_class_with_test_attr(self, testdir): + """Assure test class is skipped when using `__test__=False` (See #2007).""" + testdir.makepyfile(""" + class TestFoo(object): + __test__ = False + def __init__(self): + pass + def test_foo(): + assert True + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + 'collected 0 items', + '*no tests ran in*', + ]) + + +class TestCollectFS(object): def test_ignored_certain_directories(self, testdir): tmpdir = testdir.tmpdir + tmpdir.ensure("build", 'test_notfound.py') + tmpdir.ensure("dist", 'test_notfound.py') tmpdir.ensure("_darcs", 'test_notfound.py') tmpdir.ensure("CVS", 'test_notfound.py') tmpdir.ensure("{arch}", 'test_notfound.py') @@ -102,6 +124,53 @@ def test_ignored_certain_directories(self, testdir): assert "test_notfound" not in s assert "test_found" in s + @pytest.mark.parametrize('fname', + ("activate", "activate.csh", "activate.fish", + "Activate", "Activate.bat", "Activate.ps1")) + def test_ignored_virtualenvs(self, testdir, fname): + bindir = "Scripts" if py.std.sys.platform.startswith("win") else "bin" + testdir.tmpdir.ensure("virtual", bindir, fname) + testfile = testdir.tmpdir.ensure("virtual", "test_invenv.py") + testfile.write("def test_hello(): pass") + + # by default, ignore tests inside a virtualenv + result = testdir.runpytest() + assert "test_invenv" not in result.stdout.str() + # allow test collection if user insists + result = testdir.runpytest("--collect-in-virtualenv") + assert "test_invenv" in result.stdout.str() + # allow test collection if user directly passes in the directory + result = testdir.runpytest("virtual") + assert "test_invenv" in result.stdout.str() + + @pytest.mark.parametrize('fname', + ("activate", "activate.csh", "activate.fish", + "Activate", "Activate.bat", "Activate.ps1")) + def test_ignored_virtualenvs_norecursedirs_precedence(self, testdir, fname): + bindir = "Scripts" if py.std.sys.platform.startswith("win") else "bin" + # norecursedirs takes priority + testdir.tmpdir.ensure(".virtual", bindir, fname) + testfile = testdir.tmpdir.ensure(".virtual", "test_invenv.py") + testfile.write("def test_hello(): pass") + result = testdir.runpytest("--collect-in-virtualenv") + assert "test_invenv" not in result.stdout.str() + # ...unless the virtualenv is explicitly given on the CLI + result = testdir.runpytest("--collect-in-virtualenv", ".virtual") + assert "test_invenv" in result.stdout.str() + + @pytest.mark.parametrize('fname', + ("activate", "activate.csh", "activate.fish", + "Activate", "Activate.bat", "Activate.ps1")) + def test__in_venv(self, testdir, fname): + """Directly test the virtual env detection function""" + bindir = "Scripts" if py.std.sys.platform.startswith("win") else "bin" + # no bin/activate, not a virtualenv + base_path = testdir.tmpdir.mkdir('venv') + assert _in_venv(base_path) is False + # with bin/activate, totally a virtualenv + base_path.ensure(bindir, fname) + assert _in_venv(base_path) is True + def test_custom_norecursedirs(self, testdir): testdir.makeini(""" [pytest] @@ -145,12 +214,16 @@ def test_testpaths_ini(self, testdir, monkeypatch): assert [x.name for x in items] == ['test_%s' % dirname] -class TestCollectPluginHookRelay: +class TestCollectPluginHookRelay(object): def test_pytest_collect_file(self, testdir): wascalled = [] - class Plugin: + + class Plugin(object): def pytest_collect_file(self, path, parent): - wascalled.append(path) + if not path.basename.startswith("."): + # Ignore hidden files, e.g. .testmondata. + wascalled.append(path) + testdir.makefile(".abc", "xyz") pytest.main([testdir.tmpdir], plugins=[Plugin()]) assert len(wascalled) == 1 @@ -158,26 +231,19 @@ def pytest_collect_file(self, path, parent): def test_pytest_collect_directory(self, testdir): wascalled = [] - class Plugin: + + class Plugin(object): def pytest_collect_directory(self, path, parent): wascalled.append(path.basename) + testdir.mkdir("hello") testdir.mkdir("world") pytest.main(testdir.tmpdir, plugins=[Plugin()]) assert "hello" in wascalled assert "world" in wascalled -class TestPrunetraceback: - def test_collection_error(self, testdir): - p = testdir.makepyfile(""" - import not_exists - """) - result = testdir.runpytest(p) - assert "__import__" not in result.stdout.str(), "too long traceback" - result.stdout.fnmatch_lines([ - "*ERROR collecting*", - "*mport*not_exists*" - ]) + +class TestPrunetraceback(object): def test_custom_repr_failure(self, testdir): p = testdir.makepyfile(""" @@ -211,10 +277,12 @@ def test_collect_report_postprocessing(self, testdir): """) testdir.makeconftest(""" import pytest - def pytest_make_collect_report(__multicall__): - rep = __multicall__.execute() + @pytest.hookimpl(hookwrapper=True) + def pytest_make_collect_report(): + outcome = yield + rep = outcome.get_result() rep.headerlines += ["header1"] - return rep + outcome.force_result(rep) """) result = testdir.runpytest(p) result.stdout.fnmatch_lines([ @@ -223,7 +291,7 @@ def pytest_make_collect_report(__multicall__): ]) -class TestCustomConftests: +class TestCustomConftests(object): def test_ignore_collect_path(self, testdir): testdir.makeconftest(""" def pytest_ignore_collect(path, config): @@ -318,7 +386,8 @@ def pytest_collect_file(path, parent): "*test_x*" ]) -class TestSession: + +class TestSession(object): def test_parsearg(self, testdir): p = testdir.makepyfile("def test_func(): pass") subdir = testdir.mkdir("sub") @@ -331,11 +400,11 @@ def test_parsearg(self, testdir): assert rcol.fspath == subdir parts = rcol._parsearg(p.basename) - assert parts[0] == target + assert parts[0] == target assert len(parts) == 1 parts = rcol._parsearg(p.basename + "::test_func") - assert parts[0] == target - assert parts[1] == "test_func" + assert parts[0] == target + assert parts[1] == "test_func" assert len(parts) == 2 def test_collect_topdir(self, testdir): @@ -346,13 +415,18 @@ def test_collect_topdir(self, testdir): topdir = testdir.tmpdir rcol = Session(config) assert topdir == rcol.fspath - #rootid = rcol.nodeid - #root2 = rcol.perform_collect([rcol.nodeid], genitems=False)[0] - #assert root2 == rcol, rootid + # rootid = rcol.nodeid + # root2 = rcol.perform_collect([rcol.nodeid], genitems=False)[0] + # assert root2 == rcol, rootid colitems = rcol.perform_collect([rcol.nodeid], genitems=False) assert len(colitems) == 1 assert colitems[0].fspath == p + def get_reported_items(self, hookrec): + """Return pytest.Item instances reported by the pytest_collectreport hook""" + calls = hookrec.getcalls('pytest_collectreport') + return [x for call in calls for x in call.report.result + if isinstance(x, pytest.Item)] def test_collect_protocol_single_function(self, testdir): p = testdir.makepyfile("def test_func(): pass") @@ -370,13 +444,14 @@ def test_collect_protocol_single_function(self, testdir): ("pytest_collectstart", "collector.fspath == p"), ("pytest_make_collect_report", "collector.fspath == p"), ("pytest_pycollect_makeitem", "name == 'test_func'"), - ("pytest_collectreport", "report.nodeid.startswith(p.basename)"), - ("pytest_collectreport", "report.nodeid == ''") + ("pytest_collectreport", "report.result[0].name == 'test_func'"), ]) + # ensure we are reporting the collection of the single test item (#2464) + assert [x.name for x in self.get_reported_items(hookrec)] == ['test_func'] def test_collect_protocol_method(self, testdir): p = testdir.makepyfile(""" - class TestClass: + class TestClass(object): def test_method(self): pass """) @@ -391,6 +466,8 @@ def test_method(self): assert items[0].name == "test_method" newid = items[0].nodeid assert newid == normid + # ensure we are reporting the collection of the single test item (#2464) + assert [x.name for x in self.get_reported_items(hookrec)] == ['test_method'] def test_collect_custom_nodes_multi_id(self, testdir): p = testdir.makepyfile("def test_func(): pass") @@ -420,9 +497,8 @@ def pytest_collect_file(path, parent): "collector.__class__.__name__ == 'Module'"), ("pytest_pycollect_makeitem", "name == 'test_func'"), ("pytest_collectreport", "report.nodeid.startswith(p.basename)"), - #("pytest_collectreport", - # "report.fspath == %r" % str(rcol.fspath)), ]) + assert len(self.get_reported_items(hookrec)) == 2 def test_collect_subdir_event_ordering(self, testdir): p = testdir.makepyfile("def test_func(): pass") @@ -437,7 +513,7 @@ def test_collect_subdir_event_ordering(self, testdir): ("pytest_collectstart", "collector.fspath == test_aaa"), ("pytest_pycollect_makeitem", "name == 'test_func'"), ("pytest_collectreport", - "report.nodeid.startswith('aaa/test_aaa.py')"), + "report.nodeid.startswith('aaa/test_aaa.py')"), ]) def test_collect_two_commandline_args(self, testdir): @@ -475,24 +551,27 @@ def test_serialization_byid(self, testdir): def test_find_byid_without_instance_parents(self, testdir): p = testdir.makepyfile(""" - class TestClass: + class TestClass(object): def test_method(self): pass """) - arg = p.basename + ("::TestClass::test_method") + arg = p.basename + "::TestClass::test_method" items, hookrec = testdir.inline_genitems(arg) assert len(items) == 1 item, = items assert item.nodeid.endswith("TestClass::()::test_method") + # ensure we are reporting the collection of the single test item (#2464) + assert [x.name for x in self.get_reported_items(hookrec)] == ['test_method'] + -class Test_getinitialnodes: +class Test_getinitialnodes(object): def test_global_file(self, testdir, tmpdir): x = tmpdir.ensure("x.py") - config = testdir.parseconfigure(x) + with tmpdir.as_cwd(): + config = testdir.parseconfigure(x) col = testdir.getnode(config, x) assert isinstance(col, pytest.Module) assert col.name == 'x.py' - assert col.parent.name == testdir.tmpdir.basename assert col.parent.parent is None for col in col.listchain(): assert col.config is config @@ -502,7 +581,8 @@ def test_pkgfile(self, testdir): subdir = tmpdir.join("subdir") x = subdir.ensure("x.py") subdir.ensure("__init__.py") - config = testdir.parseconfigure(x) + with subdir.as_cwd(): + config = testdir.parseconfigure(x) col = testdir.getnode(config, x) assert isinstance(col, pytest.Module) assert col.name == 'x.py' @@ -510,7 +590,8 @@ def test_pkgfile(self, testdir): for col in col.listchain(): assert col.config is config -class Test_genitems: + +class Test_genitems(object): def test_check_collect_hashes(self, testdir): p = testdir.makepyfile(""" def test_1(): @@ -533,7 +614,7 @@ def test_example_items1(self, testdir): def testone(): pass - class TestX: + class TestX(object): def testmethod_one(self): pass @@ -566,11 +647,11 @@ def test_class_and_functions_discovery_using_glob(self, testdir): python_functions = *_test test """) p = testdir.makepyfile(''' - class MyTestSuite: + class MyTestSuite(object): def x_test(self): pass - class TestCase: + class TestCase(object): def test_y(self): pass ''') @@ -585,7 +666,7 @@ def test_matchnodes_two_collections_same_file(testdir): def pytest_configure(config): config.pluginmanager.register(Plugin2()) - class Plugin2: + class Plugin2(object): def pytest_collect_file(self, path, parent): if path.ext == ".abc": return MyFile2(path, parent) @@ -617,15 +698,15 @@ def runtest(self): ]) -class TestNodekeywords: +class TestNodekeywords(object): def test_no_under(self, testdir): modcol = testdir.getmodulecol(""" def test_pass(): pass def test_fail(): assert 0 """) - l = list(modcol.keywords) - assert modcol.name in l - for x in l: + values = list(modcol.keywords) + assert modcol.name in values + for x in values: assert not x.startswith("_") assert modcol.name in repr(modcol.keywords) @@ -639,3 +720,138 @@ def test___repr__(): """) reprec = testdir.inline_run("-k repr") reprec.assertoutcome(passed=1, failed=0) + + +COLLECTION_ERROR_PY_FILES = dict( + test_01_failure=""" + def test_1(): + assert False + """, + test_02_import_error=""" + import asdfasdfasdf + def test_2(): + assert True + """, + test_03_import_error=""" + import asdfasdfasdf + def test_3(): + assert True + """, + test_04_success=""" + def test_4(): + assert True + """, +) + + +def test_exit_on_collection_error(testdir): + """Verify that all collection errors are collected and no tests executed""" + testdir.makepyfile(**COLLECTION_ERROR_PY_FILES) + + res = testdir.runpytest() + assert res.ret == 2 + + res.stdout.fnmatch_lines([ + "collected 2 items / 2 errors", + "*ERROR collecting test_02_import_error.py*", + "*No module named *asdfa*", + "*ERROR collecting test_03_import_error.py*", + "*No module named *asdfa*", + ]) + + +def test_exit_on_collection_with_maxfail_smaller_than_n_errors(testdir): + """ + Verify collection is aborted once maxfail errors are encountered ignoring + further modules which would cause more collection errors. + """ + testdir.makepyfile(**COLLECTION_ERROR_PY_FILES) + + res = testdir.runpytest("--maxfail=1") + assert res.ret == 1 + + res.stdout.fnmatch_lines([ + "*ERROR collecting test_02_import_error.py*", + "*No module named *asdfa*", + ]) + + assert 'test_03' not in res.stdout.str() + + +def test_exit_on_collection_with_maxfail_bigger_than_n_errors(testdir): + """ + Verify the test run aborts due to collection errors even if maxfail count of + errors was not reached. + """ + testdir.makepyfile(**COLLECTION_ERROR_PY_FILES) + + res = testdir.runpytest("--maxfail=4") + assert res.ret == 2 + + res.stdout.fnmatch_lines([ + "collected 2 items / 2 errors", + "*ERROR collecting test_02_import_error.py*", + "*No module named *asdfa*", + "*ERROR collecting test_03_import_error.py*", + "*No module named *asdfa*", + ]) + + +def test_continue_on_collection_errors(testdir): + """ + Verify tests are executed even when collection errors occur when the + --continue-on-collection-errors flag is set + """ + testdir.makepyfile(**COLLECTION_ERROR_PY_FILES) + + res = testdir.runpytest("--continue-on-collection-errors") + assert res.ret == 1 + + res.stdout.fnmatch_lines([ + "collected 2 items / 2 errors", + "*1 failed, 1 passed, 2 error*", + ]) + + +def test_continue_on_collection_errors_maxfail(testdir): + """ + Verify tests are executed even when collection errors occur and that maxfail + is honoured (including the collection error count). + 4 tests: 2 collection errors + 1 failure + 1 success + test_4 is never executed because the test run is with --maxfail=3 which + means it is interrupted after the 2 collection errors + 1 failure. + """ + testdir.makepyfile(**COLLECTION_ERROR_PY_FILES) + + res = testdir.runpytest("--continue-on-collection-errors", "--maxfail=3") + assert res.ret == 1 + + res.stdout.fnmatch_lines([ + "collected 2 items / 2 errors", + "*1 failed, 2 error*", + ]) + + +def test_fixture_scope_sibling_conftests(testdir): + """Regression test case for https://github.com/pytest-dev/pytest/issues/2836""" + foo_path = testdir.mkpydir("foo") + foo_path.join("conftest.py").write(_pytest._code.Source(""" + import pytest + @pytest.fixture + def fix(): + return 1 + """)) + foo_path.join("test_foo.py").write("def test_foo(fix): assert fix == 1") + + # Tests in `food/` should not see the conftest fixture from `foo/` + food_path = testdir.mkpydir("food") + food_path.join("test_food.py").write("def test_food(fix): assert fix == 1") + + res = testdir.runpytest() + assert res.ret == 1 + + res.stdout.fnmatch_lines([ + "*ERROR at setup of test_food*", + "E*fixture 'fix' not found", + "*1 passed, 1 error*", + ]) diff --git a/testing/test_compat.py b/testing/test_compat.py new file mode 100644 index 000000000000000..c74801c6c85485c --- /dev/null +++ b/testing/test_compat.py @@ -0,0 +1,101 @@ +from __future__ import absolute_import, division, print_function +import sys + +import pytest +from _pytest.compat import is_generator, get_real_func, safe_getattr +from _pytest.outcomes import OutcomeException + + +def test_is_generator(): + def zap(): + yield + + def foo(): + pass + + assert is_generator(zap) + assert not is_generator(foo) + + +def test_real_func_loop_limit(): + + class Evil(object): + def __init__(self): + self.left = 1000 + + def __repr__(self): + return "".format(left=self.left) + + def __getattr__(self, attr): + if not self.left: + raise RuntimeError('its over') + self.left -= 1 + return self + + evil = Evil() + + with pytest.raises(ValueError): + res = get_real_func(evil) + print(res) + + +@pytest.mark.skipif(sys.version_info < (3, 4), + reason='asyncio available in Python 3.4+') +def test_is_generator_asyncio(testdir): + testdir.makepyfile(""" + from _pytest.compat import is_generator + import asyncio + @asyncio.coroutine + def baz(): + yield from [1,2,3] + + def test_is_generator_asyncio(): + assert not is_generator(baz) + """) + # avoid importing asyncio into pytest's own process, + # which in turn imports logging (#8) + result = testdir.runpytest_subprocess() + result.stdout.fnmatch_lines(['*1 passed*']) + + +@pytest.mark.skipif(sys.version_info < (3, 5), + reason='async syntax available in Python 3.5+') +def test_is_generator_async_syntax(testdir): + testdir.makepyfile(""" + from _pytest.compat import is_generator + def test_is_generator_py35(): + async def foo(): + await foo() + + async def bar(): + pass + + assert not is_generator(foo) + assert not is_generator(bar) + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines(['*1 passed*']) + + +class ErrorsHelper(object): + @property + def raise_exception(self): + raise Exception('exception should be catched') + + @property + def raise_fail(self): + pytest.fail('fail should be catched') + + +def test_helper_failures(): + helper = ErrorsHelper() + with pytest.raises(Exception): + helper.raise_exception + with pytest.raises(OutcomeException): + helper.raise_fail + + +def test_safe_getattr(): + helper = ErrorsHelper() + assert safe_getattr(helper, 'raise_exception', 'default') == 'default' + assert safe_getattr(helper, 'raise_fail', 'default') == 'default' diff --git a/testing/test_config.py b/testing/test_config.py index 92c9bdb8b087549..44b8c317a285735 100644 --- a/testing/test_config.py +++ b/testing/test_config.py @@ -1,28 +1,36 @@ -import py, pytest +from __future__ import absolute_import, division, print_function +import sys +import py +import pytest import _pytest._code -from _pytest.config import getcfg, get_common_ancestor, determine_setup +from _pytest.config import getcfg, get_common_ancestor, determine_setup, _iter_rewritable_modules from _pytest.main import EXIT_NOTESTSCOLLECTED -class TestParseIni: - def test_getcfg_and_config(self, testdir, tmpdir): + +class TestParseIni(object): + + @pytest.mark.parametrize('section, filename', + [('pytest', 'pytest.ini'), ('tool:pytest', 'setup.cfg')]) + def test_getcfg_and_config(self, testdir, tmpdir, section, filename): sub = tmpdir.mkdir("sub") sub.chdir() - tmpdir.join("setup.cfg").write(_pytest._code.Source(""" - [pytest] + tmpdir.join(filename).write(_pytest._code.Source(""" + [{section}] name = value - """)) - rootdir, inifile, cfg = getcfg([sub], ["setup.cfg"]) + """.format(section=section))) + rootdir, inifile, cfg = getcfg([sub]) assert cfg['name'] == "value" config = testdir.parseconfigure(sub) assert config.inicfg['name'] == 'value' - def test_getcfg_empty_path(self, tmpdir): - getcfg([''], ['setup.cfg']) #happens on py.test "" + def test_getcfg_empty_path(self): + """correctly handle zero length arguments (a la pytest '')""" + getcfg(['']) def test_append_parse_args(self, testdir, tmpdir, monkeypatch): monkeypatch.setenv('PYTEST_ADDOPTS', '--color no -rs --tb="short"') - tmpdir.join("setup.cfg").write(_pytest._code.Source(""" + tmpdir.join("pytest.ini").write(_pytest._code.Source(""" [pytest] addopts = --verbose """)) @@ -31,10 +39,6 @@ def test_append_parse_args(self, testdir, tmpdir, monkeypatch): assert config.option.reportchars == 's' assert config.option.tbstyle == 'short' assert config.option.verbose - #config = testdir.Config() - #args = [tmpdir,] - #config._preparse(args, addopts=False) - #assert len(args) == 1 def test_tox_ini_wrong_version(self, testdir): testdir.makefile('.ini', tox=""" @@ -47,12 +51,16 @@ def test_tox_ini_wrong_version(self, testdir): "*tox.ini:2*requires*9.0*actual*" ]) - @pytest.mark.parametrize("name", "setup.cfg tox.ini pytest.ini".split()) - def test_ini_names(self, testdir, name): + @pytest.mark.parametrize("section, name", [ + ('tool:pytest', 'setup.cfg'), + ('pytest', 'tox.ini'), + ('pytest', 'pytest.ini')], + ) + def test_ini_names(self, testdir, name, section): testdir.tmpdir.join(name).write(py.std.textwrap.dedent(""" - [pytest] + [{section}] minversion = 1.0 - """)) + """.format(section=section))) config = testdir.parseconfig() assert config.getini("minversion") == "1.0" @@ -80,7 +88,8 @@ def test_confcutdir(self, testdir): result = testdir.inline_run("--confcutdir=.") assert result.ret == 0 -class TestConfigCmdlineParsing: + +class TestConfigCmdlineParsing(object): def test_parsing_again_fails(self, testdir): config = testdir.parseconfig() pytest.raises(AssertionError, lambda: config.parse([])) @@ -94,21 +103,32 @@ def pytest_addoption(parser): [pytest] custom = 0 """) - testdir.makefile(".cfg", custom = """ + testdir.makefile(".cfg", custom=""" [pytest] custom = 1 """) config = testdir.parseconfig("-c", "custom.cfg") assert config.getini("custom") == "1" -class TestConfigAPI: + def test_absolute_win32_path(self, testdir): + temp_cfg_file = testdir.makefile(".cfg", custom=""" + [pytest] + addopts = --version + """) + from os.path import normpath + temp_cfg_file = normpath(str(temp_cfg_file)) + ret = pytest.main("-c " + temp_cfg_file) + assert ret == _pytest.main.EXIT_OK + + +class TestConfigAPI(object): def test_config_trace(self, testdir): config = testdir.parseconfig() - l = [] - config.trace.root.setwriter(l.append) + values = [] + config.trace.root.setwriter(values.append) config.trace("hello") - assert len(l) == 1 - assert l[0] == "hello [config]\n" + assert len(values) == 1 + assert values[0] == "hello [config]\n" def test_config_getoption(self, testdir): testdir.makeconftest(""" @@ -120,13 +140,13 @@ def pytest_addoption(parser): assert config.getoption(x) == "this" pytest.raises(ValueError, "config.getoption('qweqwe')") - @pytest.mark.skipif('sys.version_info[:2] not in [(2, 6), (2, 7)]') + @pytest.mark.skipif('sys.version_info[0] < 3') def test_config_getoption_unicode(self, testdir): testdir.makeconftest(""" from __future__ import unicode_literals def pytest_addoption(parser): - parser.addoption('--hello', type='string') + parser.addoption('--hello', type=str) """) config = testdir.parseconfig('--hello=this') assert config.getoption('hello') == 'this' @@ -134,7 +154,7 @@ def pytest_addoption(parser): def test_config_getvalueorskip(self, testdir): config = testdir.parseconfig() pytest.raises(pytest.skip.Exception, - "config.getvalueorskip('hello')") + "config.getvalueorskip('hello')") verbose = config.getvalueorskip("verbose") assert verbose == config.option.verbose @@ -190,10 +210,10 @@ def pytest_addoption(parser): paths=hello world/sub.py """) config = testdir.parseconfig() - l = config.getini("paths") - assert len(l) == 2 - assert l[0] == p.dirpath('hello') - assert l[1] == p.dirpath('world/sub.py') + values = config.getini("paths") + assert len(values) == 2 + assert values[0] == p.dirpath('hello') + assert values[1] == p.dirpath('world/sub.py') pytest.raises(ValueError, config.getini, 'other') def test_addini_args(self, testdir): @@ -207,11 +227,11 @@ def pytest_addoption(parser): args=123 "123 hello" "this" """) config = testdir.parseconfig() - l = config.getini("args") - assert len(l) == 3 - assert l == ["123", "123 hello", "this"] - l = config.getini("a2") - assert l == list("123") + values = config.getini("args") + assert len(values) == 3 + assert values == ["123", "123 hello", "this"] + values = config.getini("a2") + assert values == list("123") def test_addini_linelist(self, testdir): testdir.makeconftest(""" @@ -225,11 +245,11 @@ def pytest_addoption(parser): second line """) config = testdir.parseconfig() - l = config.getini("xy") - assert len(l) == 2 - assert l == ["123 345", "second line"] - l = config.getini("a2") - assert l == [] + values = config.getini("xy") + assert len(values) == 2 + assert values == ["123 345", "second line"] + values = config.getini("a2") + assert values == [] @pytest.mark.parametrize('str_val, bool_val', [('True', True), ('no', False), ('no-ini', True)]) @@ -256,13 +276,13 @@ def pytest_addoption(parser): xy= 123 """) config = testdir.parseconfig() - l = config.getini("xy") - assert len(l) == 1 - assert l == ["123"] + values = config.getini("xy") + assert len(values) == 1 + assert values == ["123"] config.addinivalue_line("xy", "456") - l = config.getini("xy") - assert len(l) == 2 - assert l == ["123", "456"] + values = config.getini("xy") + assert len(values) == 2 + assert values == ["123", "456"] def test_addinivalue_line_new(self, testdir): testdir.makeconftest(""" @@ -272,16 +292,35 @@ def pytest_addoption(parser): config = testdir.parseconfig() assert not config.getini("xy") config.addinivalue_line("xy", "456") - l = config.getini("xy") - assert len(l) == 1 - assert l == ["456"] + values = config.getini("xy") + assert len(values) == 1 + assert values == ["456"] config.addinivalue_line("xy", "123") - l = config.getini("xy") - assert len(l) == 2 - assert l == ["456", "123"] + values = config.getini("xy") + assert len(values) == 2 + assert values == ["456", "123"] + + def test_confcutdir_check_isdir(self, testdir): + """Give an error if --confcutdir is not a valid directory (#2078)""" + with pytest.raises(pytest.UsageError): + testdir.parseconfig('--confcutdir', testdir.tmpdir.join('file').ensure(file=1)) + with pytest.raises(pytest.UsageError): + testdir.parseconfig('--confcutdir', testdir.tmpdir.join('inexistant')) + config = testdir.parseconfig('--confcutdir', testdir.tmpdir.join('dir').ensure(dir=1)) + assert config.getoption('confcutdir') == str(testdir.tmpdir.join('dir')) + + @pytest.mark.parametrize('names, expected', [ + (['bar.py'], ['bar']), + (['foo', 'bar.py'], []), + (['foo', 'bar.pyc'], []), + (['foo', '__init__.py'], ['foo']), + (['foo', 'bar', '__init__.py'], []), + ]) + def test_iter_rewritable_modules(self, names, expected): + assert list(_iter_rewritable_modules(['/'.join(names)])) == expected -class TestConfigFromdictargs: +class TestConfigFromdictargs(object): def test_basic_behavior(self): from _pytest.config import Config option_dict = { @@ -355,23 +394,35 @@ def test_f2(): assert 0 """) for opts in ([], ['-l'], ['-s'], ['--tb=no'], ['--tb=short'], - ['--tb=long'], ['--fulltrace'], ['--nomagic'], + ['--tb=long'], ['--fulltrace'], ['--traceconfig'], ['-v'], ['-v', '-v']): runfiletest(opts + [path]) + def test_preparse_ordering_with_setuptools(testdir, monkeypatch): pkg_resources = pytest.importorskip("pkg_resources") + def my_iter(name): assert name == "pytest11" - class EntryPoint: + + class Dist(object): + project_name = 'spam' + version = '1.0' + + def _get_metadata(self, name): + return ['foo.txt,sha256=abc,123'] + + class EntryPoint(object): name = "mytestplugin" - class dist: - pass + dist = Dist() + def load(self): - class PseudoPlugin: + class PseudoPlugin(object): x = 42 return PseudoPlugin() + return iter([EntryPoint()]) + monkeypatch.setattr(pkg_resources, 'iter_entry_points', my_iter) testdir.makeconftest(""" pytest_plugins = "mytestplugin", @@ -381,19 +432,69 @@ class PseudoPlugin: plugin = config.pluginmanager.getplugin("mytestplugin") assert plugin.x == 42 -def test_plugin_preparse_prevents_setuptools_loading(testdir, monkeypatch): + +def test_setuptools_importerror_issue1479(testdir, monkeypatch): + pkg_resources = pytest.importorskip("pkg_resources") + + def my_iter(name): + assert name == "pytest11" + + class Dist(object): + project_name = 'spam' + version = '1.0' + + def _get_metadata(self, name): + return ['foo.txt,sha256=abc,123'] + + class EntryPoint(object): + name = "mytestplugin" + dist = Dist() + + def load(self): + raise ImportError("Don't hide me!") + + return iter([EntryPoint()]) + + monkeypatch.setattr(pkg_resources, 'iter_entry_points', my_iter) + with pytest.raises(ImportError): + testdir.parseconfig() + + +@pytest.mark.parametrize('block_it', [True, False]) +def test_plugin_preparse_prevents_setuptools_loading(testdir, monkeypatch, block_it): pkg_resources = pytest.importorskip("pkg_resources") + + plugin_module_placeholder = object() + def my_iter(name): assert name == "pytest11" - class EntryPoint: + + class Dist(object): + project_name = 'spam' + version = '1.0' + + def _get_metadata(self, name): + return ['foo.txt,sha256=abc,123'] + + class EntryPoint(object): name = "mytestplugin" + dist = Dist() + def load(self): - assert 0, "should not arrive here" + return plugin_module_placeholder + return iter([EntryPoint()]) + monkeypatch.setattr(pkg_resources, 'iter_entry_points', my_iter) - config = testdir.parseconfig("-p", "no:mytestplugin") - plugin = config.pluginmanager.getplugin("mytestplugin") - assert plugin is None + args = ("-p", "no:mytestplugin") if block_it else () + config = testdir.parseconfig(*args) + config.pluginmanager.import_plugin("mytestplugin") + if block_it: + assert "mytestplugin" not in sys.modules + assert config.pluginmanager.get_plugin('mytestplugin') is None + else: + assert config.pluginmanager.get_plugin('mytestplugin') is plugin_module_placeholder + def test_cmdline_processargs_simple(testdir): testdir.makeconftest(""" @@ -406,6 +507,7 @@ def pytest_cmdline_preparse(args): "*-h*", ]) + def test_invalid_options_show_extra_information(testdir): """display extra information when pytest exits due to unrecognized options in the command-line""" @@ -441,8 +543,9 @@ def test_consider_args_after_options_for_rootdir_and_inifile(testdir, args): args[i] = d1 elif arg == 'dir2': args[i] = d2 - result = testdir.runpytest(*args) - result.stdout.fnmatch_lines(['*rootdir: *myroot, inifile: ']) + with root.as_cwd(): + result = testdir.runpytest(*args) + result.stdout.fnmatch_lines(['*rootdir: *myroot, inifile:']) @pytest.mark.skipif("sys.platform == 'win32'") @@ -450,15 +553,41 @@ def test_toolongargs_issue224(testdir): result = testdir.runpytest("-m", "hello" * 500) assert result.ret == EXIT_NOTESTSCOLLECTED + +def test_config_in_subdirectory_colon_command_line_issue2148(testdir): + conftest_source = ''' + def pytest_addoption(parser): + parser.addini('foo', 'foo') + ''' + + testdir.makefile('.ini', **{ + 'pytest': '[pytest]\nfoo = root', + 'subdir/pytest': '[pytest]\nfoo = subdir', + }) + + testdir.makepyfile(**{ + 'conftest': conftest_source, + 'subdir/conftest': conftest_source, + 'subdir/test_foo': ''' + def test_foo(pytestconfig): + assert pytestconfig.getini('foo') == 'subdir' + '''}) + + result = testdir.runpytest('subdir/test_foo.py::test_foo') + assert result.ret == 0 + + def test_notify_exception(testdir, capfd): config = testdir.parseconfig() excinfo = pytest.raises(ValueError, "raise ValueError(1)") config.notify_exception(excinfo) out, err = capfd.readouterr() assert "ValueError" in err - class A: + + class A(object): def pytest_internalerror(self, excrepr): return True + config.pluginmanager.register(A()) config.notify_exception(excinfo) out, err = capfd.readouterr() @@ -466,64 +595,92 @@ def pytest_internalerror(self, excrepr): def test_load_initial_conftest_last_ordering(testdir): - from _pytest.config import get_config + from _pytest.config import get_config pm = get_config().pluginmanager - class My: + + class My(object): def pytest_load_initial_conftests(self): pass + m = My() pm.register(m) hc = pm.hook.pytest_load_initial_conftests - l = hc._nonwrappers + hc._wrappers - assert l[-1].function.__module__ == "_pytest.capture" - assert l[-2].function == m.pytest_load_initial_conftests - assert l[-3].function.__module__ == "_pytest.config" - -class TestWarning: + values = hc._nonwrappers + hc._wrappers + expected = [ + "_pytest.config", + 'test_config', + '_pytest.capture', + ] + assert [x.function.__module__ for x in values] == expected + + +def test_get_plugin_specs_as_list(): + from _pytest.config import _get_plugin_specs_as_list + with pytest.raises(pytest.UsageError): + _get_plugin_specs_as_list(set(['foo'])) + with pytest.raises(pytest.UsageError): + _get_plugin_specs_as_list(dict()) + + assert _get_plugin_specs_as_list(None) == [] + assert _get_plugin_specs_as_list('') == [] + assert _get_plugin_specs_as_list('foo') == ['foo'] + assert _get_plugin_specs_as_list('foo,bar') == ['foo', 'bar'] + assert _get_plugin_specs_as_list(['foo', 'bar']) == ['foo', 'bar'] + assert _get_plugin_specs_as_list(('foo', 'bar')) == ['foo', 'bar'] + + +class TestWarning(object): def test_warn_config(self, testdir): testdir.makeconftest(""" - l = [] + values = [] def pytest_configure(config): config.warn("C1", "hello") def pytest_logwarning(code, message): if message == "hello" and code == "C1": - l.append(1) + values.append(1) """) testdir.makepyfile(""" def test_proper(pytestconfig): import conftest - assert conftest.l == [1] + assert conftest.values == [1] """) reprec = testdir.inline_run() reprec.assertoutcome(passed=1) - def test_warn_on_test_item_from_request(self, testdir): + def test_warn_on_test_item_from_request(self, testdir, request): testdir.makepyfile(""" import pytest @pytest.fixture def fix(request): request.node.warn("T1", "hello") + def test_hello(fix): pass """) - result = testdir.runpytest() - assert result.parseoutcomes()["pytest-warnings"] > 0 + result = testdir.runpytest("--disable-pytest-warnings") + assert result.parseoutcomes()["warnings"] > 0 assert "hello" not in result.stdout.str() - result = testdir.runpytest("-rw") + result = testdir.runpytest() result.stdout.fnmatch_lines(""" - ===*pytest-warning summary*=== - *WT1*test_warn_on_test_item*:5*hello* + ===*warnings summary*=== + *test_warn_on_test_item_from_request.py::test_hello* + *hello* """) -class TestRootdir: + +class TestRootdir(object): def test_simple_noini(self, tmpdir): assert get_common_ancestor([tmpdir]) == tmpdir - assert get_common_ancestor([tmpdir.mkdir("a"), tmpdir]) == tmpdir - assert get_common_ancestor([tmpdir, tmpdir.join("a")]) == tmpdir + a = tmpdir.mkdir("a") + assert get_common_ancestor([a, tmpdir]) == tmpdir + assert get_common_ancestor([tmpdir, a]) == tmpdir with tmpdir.as_cwd(): assert get_common_ancestor([]) == tmpdir + no_path = tmpdir.join('does-not-exist') + assert get_common_ancestor([no_path]) == tmpdir + assert get_common_ancestor([no_path.join('a')]) == tmpdir @pytest.mark.parametrize("name", "setup.cfg tox.ini pytest.ini".split()) def test_with_ini(self, tmpdir, name): @@ -536,7 +693,7 @@ def test_with_ini(self, tmpdir, name): rootdir, inifile, inicfg = determine_setup(None, args) assert rootdir == tmpdir assert inifile == inifile - rootdir, inifile, inicfg = determine_setup(None, [b,a]) + rootdir, inifile, inicfg = determine_setup(None, [b, a]) assert rootdir == tmpdir assert inifile == inifile @@ -558,7 +715,8 @@ def test_setuppy_fallback(self, tmpdir): assert inifile is None assert inicfg == {} - def test_nothing(self, tmpdir): + def test_nothing(self, tmpdir, monkeypatch): + monkeypatch.chdir(str(tmpdir)) rootdir, inifile, inicfg = determine_setup(None, [tmpdir]) assert rootdir == tmpdir assert inifile is None @@ -568,3 +726,137 @@ def test_with_specific_inifile(self, tmpdir): inifile = tmpdir.ensure("pytest.ini") rootdir, inifile, inicfg = determine_setup(inifile, [tmpdir]) assert rootdir == tmpdir + + +class TestOverrideIniArgs(object): + @pytest.mark.parametrize("name", "setup.cfg tox.ini pytest.ini".split()) + def test_override_ini_names(self, testdir, name): + testdir.tmpdir.join(name).write(py.std.textwrap.dedent(""" + [pytest] + custom = 1.0""")) + testdir.makeconftest(""" + def pytest_addoption(parser): + parser.addini("custom", "")""") + testdir.makepyfile(""" + def test_pass(pytestconfig): + ini_val = pytestconfig.getini("custom") + print('\\ncustom_option:%s\\n' % ini_val)""") + + result = testdir.runpytest("--override-ini", "custom=2.0", "-s") + assert result.ret == 0 + result.stdout.fnmatch_lines(["custom_option:2.0"]) + + result = testdir.runpytest("--override-ini", "custom=2.0", + "--override-ini=custom=3.0", "-s") + assert result.ret == 0 + result.stdout.fnmatch_lines(["custom_option:3.0"]) + + def test_override_ini_pathlist(self, testdir): + testdir.makeconftest(""" + def pytest_addoption(parser): + parser.addini("paths", "my new ini value", type="pathlist")""") + testdir.makeini(""" + [pytest] + paths=blah.py""") + testdir.makepyfile(""" + import py.path + def test_pathlist(pytestconfig): + config_paths = pytestconfig.getini("paths") + print(config_paths) + for cpf in config_paths: + print('\\nuser_path:%s' % cpf.basename)""") + result = testdir.runpytest("--override-ini", + 'paths=foo/bar1.py foo/bar2.py', "-s") + result.stdout.fnmatch_lines(["user_path:bar1.py", + "user_path:bar2.py"]) + + def test_override_multiple_and_default(self, testdir): + testdir.makeconftest(""" + def pytest_addoption(parser): + addini = parser.addini + addini("custom_option_1", "", default="o1") + addini("custom_option_2", "", default="o2") + addini("custom_option_3", "", default=False, type="bool") + addini("custom_option_4", "", default=True, type="bool")""") + testdir.makeini(""" + [pytest] + custom_option_1=custom_option_1 + custom_option_2=custom_option_2""") + testdir.makepyfile(""" + def test_multiple_options(pytestconfig): + prefix = "custom_option" + for x in range(1, 5): + ini_value=pytestconfig.getini("%s_%d" % (prefix, x)) + print('\\nini%d:%s' % (x, ini_value))""") + result = testdir.runpytest( + "--override-ini", 'custom_option_1=fulldir=/tmp/user1', + 'custom_option_2=url=/tmp/user2?a=b&d=e', + "-o", 'custom_option_3=True', + "-o", 'custom_option_4=no', "-s") + result.stdout.fnmatch_lines(["ini1:fulldir=/tmp/user1", + "ini2:url=/tmp/user2?a=b&d=e", + "ini3:True", + "ini4:False"]) + + def test_override_ini_usage_error_bad_style(self, testdir): + testdir.makeini(""" + [pytest] + xdist_strict=False + """) + result = testdir.runpytest("--override-ini", 'xdist_strict True', "-s") + result.stderr.fnmatch_lines(["*ERROR* *expects option=value*"]) + + @pytest.mark.parametrize('with_ini', [True, False]) + def test_override_ini_handled_asap(self, testdir, with_ini): + """-o should be handled as soon as possible and always override what's in ini files (#2238)""" + if with_ini: + testdir.makeini(""" + [pytest] + python_files=test_*.py + """) + testdir.makepyfile(unittest_ini_handle=""" + def test(): + pass + """) + result = testdir.runpytest("--override-ini", 'python_files=unittest_*.py') + result.stdout.fnmatch_lines(["*1 passed in*"]) + + def test_with_arg_outside_cwd_without_inifile(self, tmpdir, monkeypatch): + monkeypatch.chdir(str(tmpdir)) + a = tmpdir.mkdir("a") + b = tmpdir.mkdir("b") + rootdir, inifile, inicfg = determine_setup(None, [a, b]) + assert rootdir == tmpdir + assert inifile is None + + def test_with_arg_outside_cwd_with_inifile(self, tmpdir): + a = tmpdir.mkdir("a") + b = tmpdir.mkdir("b") + inifile = a.ensure("pytest.ini") + rootdir, parsed_inifile, inicfg = determine_setup(None, [a, b]) + assert rootdir == a + assert inifile == parsed_inifile + + @pytest.mark.parametrize('dirs', ([], ['does-not-exist'], + ['a/does-not-exist'])) + def test_with_non_dir_arg(self, dirs, tmpdir): + with tmpdir.ensure(dir=True).as_cwd(): + rootdir, inifile, inicfg = determine_setup(None, dirs) + assert rootdir == tmpdir + assert inifile is None + + def test_with_existing_file_in_subdir(self, tmpdir): + a = tmpdir.mkdir("a") + a.ensure("exist") + with tmpdir.as_cwd(): + rootdir, inifile, inicfg = determine_setup(None, ['a/exist']) + assert rootdir == tmpdir + assert inifile is None + + def test_addopts_before_initini(self, testdir, tmpdir, monkeypatch): + cache_dir = '.custom_cache' + monkeypatch.setenv('PYTEST_ADDOPTS', '-o cache_dir=%s' % cache_dir) + from _pytest.config import get_config + config = get_config() + config._preparse([], addopts=True) + assert config._override_ini == [['cache_dir=%s' % cache_dir]] diff --git a/testing/test_conftest.py b/testing/test_conftest.py index 6f5e77f6d4973b4..c0411b723211474 100644 --- a/testing/test_conftest.py +++ b/testing/test_conftest.py @@ -1,3 +1,4 @@ +from __future__ import absolute_import, division, print_function from textwrap import dedent import _pytest._code @@ -18,20 +19,23 @@ def basedir(request, tmpdir_factory): tmpdir.ensure("adir/b/__init__.py") return tmpdir + def ConftestWithSetinitial(path): conftest = PytestPluginManager() conftest_setinitial(conftest, [path]) return conftest + def conftest_setinitial(conftest, args, confcutdir=None): - class Namespace: + class Namespace(object): def __init__(self): self.file_or_dir = args self.confcutdir = str(confcutdir) self.noconftest = False conftest._set_initial_conftests(Namespace()) -class TestConftestValueAccessGlobal: + +class TestConftestValueAccessGlobal(object): def test_basic_init(self, basedir): conftest = PytestPluginManager() p = basedir.join("adir") @@ -42,7 +46,7 @@ def test_immediate_initialiation_and_incremental_are_the_same(self, basedir): len(conftest._path2confmods) conftest._getconftestmodules(basedir) snap1 = len(conftest._path2confmods) - #assert len(conftest._path2confmods) == snap1 + 1 + # assert len(conftest._path2confmods) == snap1 + 1 conftest._getconftestmodules(basedir.join('adir')) assert len(conftest._path2confmods) == snap1 + 1 conftest._getconftestmodules(basedir.join('b')) @@ -64,11 +68,12 @@ def test_value_access_with_confmod(self, basedir): startdir.ensure("xx", dir=True) conftest = ConftestWithSetinitial(startdir) mod, value = conftest._rget_with_confmod("a", startdir) - assert value == 1.5 + assert value == 1.5 path = py.path.local(mod.__file__) assert path.dirpath() == basedir.join("adir", "b") assert path.purebasename.startswith("conftest") + def test_conftest_in_nonpkg_with_init(tmpdir): tmpdir.ensure("adir-1.0/conftest.py").write("a=1 ; Directory = 3") tmpdir.ensure("adir-1.0/b/conftest.py").write("b=2 ; a = 1.5") @@ -76,13 +81,15 @@ def test_conftest_in_nonpkg_with_init(tmpdir): tmpdir.ensure("adir-1.0/__init__.py") ConftestWithSetinitial(tmpdir.join("adir-1.0", "b")) + def test_doubledash_considered(testdir): conf = testdir.mkdir("--option") conf.join("conftest.py").ensure() conftest = PytestPluginManager() conftest_setinitial(conftest, [conf.basename, conf.basename]) - l = conftest._getconftestmodules(conf) - assert len(l) == 1 + values = conftest._getconftestmodules(conf) + assert len(values) == 1 + def test_issue151_load_all_conftests(testdir): names = "code proj src".split() @@ -95,6 +102,7 @@ def test_issue151_load_all_conftests(testdir): d = list(conftest._conftestpath2mod.values()) assert len(d) == len(names) + def test_conftest_global_import(testdir): testdir.makeconftest("x=3") p = testdir.makepyfile(""" @@ -116,32 +124,35 @@ def test_conftest_global_import(testdir): res = testdir.runpython(p) assert res.ret == 0 + def test_conftestcutdir(testdir): conf = testdir.makeconftest("") p = testdir.mkdir("x") conftest = PytestPluginManager() conftest_setinitial(conftest, [testdir.tmpdir], confcutdir=p) - l = conftest._getconftestmodules(p) - assert len(l) == 0 - l = conftest._getconftestmodules(conf.dirpath()) - assert len(l) == 0 + values = conftest._getconftestmodules(p) + assert len(values) == 0 + values = conftest._getconftestmodules(conf.dirpath()) + assert len(values) == 0 assert conf not in conftest._conftestpath2mod # but we can still import a conftest directly conftest._importconftest(conf) - l = conftest._getconftestmodules(conf.dirpath()) - assert l[0].__file__.startswith(str(conf)) + values = conftest._getconftestmodules(conf.dirpath()) + assert values[0].__file__.startswith(str(conf)) # and all sub paths get updated properly - l = conftest._getconftestmodules(p) - assert len(l) == 1 - assert l[0].__file__.startswith(str(conf)) + values = conftest._getconftestmodules(p) + assert len(values) == 1 + assert values[0].__file__.startswith(str(conf)) + def test_conftestcutdir_inplace_considered(testdir): conf = testdir.makeconftest("") conftest = PytestPluginManager() conftest_setinitial(conftest, [conf.dirpath()], confcutdir=conf.dirpath()) - l = conftest._getconftestmodules(conf.dirpath()) - assert len(l) == 1 - assert l[0].__file__.startswith(str(conf)) + values = conftest._getconftestmodules(conf.dirpath()) + assert len(values) == 1 + assert values[0].__file__.startswith(str(conf)) + @pytest.mark.parametrize("name", 'test tests whatever .dotdir'.split()) def test_setinitial_conftest_subdirs(testdir, name): @@ -150,12 +161,13 @@ def test_setinitial_conftest_subdirs(testdir, name): conftest = PytestPluginManager() conftest_setinitial(conftest, [sub.dirpath()], confcutdir=testdir.tmpdir) if name not in ('whatever', '.dotdir'): - assert subconftest in conftest._conftestpath2mod + assert subconftest in conftest._conftestpath2mod assert len(conftest._conftestpath2mod) == 1 else: - assert subconftest not in conftest._conftestpath2mod + assert subconftest not in conftest._conftestpath2mod assert len(conftest._conftestpath2mod) == 0 + def test_conftest_confcutdir(testdir): testdir.makeconftest("assert 0") x = testdir.mkdir("x") @@ -167,6 +179,7 @@ def pytest_addoption(parser): result.stdout.fnmatch_lines(["*--xyz*"]) assert 'warning: could not load initial' not in result.stdout.str() + def test_no_conftest(testdir): testdir.makeconftest("assert 0") result = testdir.runpytest("--noconftest") @@ -175,6 +188,7 @@ def test_no_conftest(testdir): result = testdir.runpytest() assert result.ret == EXIT_USAGEERROR + def test_conftest_existing_resultlog(testdir): x = testdir.mkdir("tests") x.join("conftest.py").write(_pytest._code.Source(""" @@ -185,6 +199,7 @@ def pytest_addoption(parser): result = testdir.runpytest("-h", "--resultlog", "result.log") result.stdout.fnmatch_lines(["*--xyz*"]) + def test_conftest_existing_junitxml(testdir): x = testdir.mkdir("tests") x.join("conftest.py").write(_pytest._code.Source(""" @@ -195,14 +210,18 @@ def pytest_addoption(parser): result = testdir.runpytest("-h", "--junitxml", "junit.xml") result.stdout.fnmatch_lines(["*--xyz*"]) + def test_conftest_import_order(testdir, monkeypatch): ct1 = testdir.makeconftest("") sub = testdir.mkdir("sub") ct2 = sub.join("conftest.py") ct2.write("") + def impct(p): return p + conftest = PytestPluginManager() + conftest._confcutdir = testdir.tmpdir monkeypatch.setattr(conftest, '_importconftest', impct) assert conftest._getconftestmodules(sub) == [ct1, ct2] @@ -262,7 +281,7 @@ def test_hello(found): """) -class TestConftestVisibility: +class TestConftestVisibility(object): def _setup_tree(self, testdir): # for issue616 # example mostly taken from: # https://mail.python.org/pipermail/pytest-dev/2014-September/002617.html @@ -302,9 +321,9 @@ def test_no_conftest(fxtr): # use value from parent dir's """)) - print ("created directory structure:") + print("created directory structure:") for x in testdir.tmpdir.visit(): - print (" " + x.relto(testdir.tmpdir)) + print(" " + x.relto(testdir.tmpdir)) return { "runner": runner, @@ -315,38 +334,38 @@ def test_no_conftest(fxtr): # N.B.: "swc" stands for "subdir with conftest.py" # "snc" stands for "subdir no [i.e. without] conftest.py" @pytest.mark.parametrize("chdir,testarg,expect_ntests_passed", [ - # Effective target: package/.. - ("runner", "..", 3), - ("package", "..", 3), - ("swc", "../..", 3), - ("snc", "../..", 3), - - # Effective target: package - ("runner", "../package", 3), - ("package", ".", 3), - ("swc", "..", 3), - ("snc", "..", 3), - - # Effective target: package/swc - ("runner", "../package/swc", 1), - ("package", "./swc", 1), - ("swc", ".", 1), - ("snc", "../swc", 1), - - # Effective target: package/snc - ("runner", "../package/snc", 1), - ("package", "./snc", 1), - ("swc", "../snc", 1), - ("snc", ".", 1), + # Effective target: package/.. + ("runner", "..", 3), + ("package", "..", 3), + ("swc", "../..", 3), + ("snc", "../..", 3), + + # Effective target: package + ("runner", "../package", 3), + ("package", ".", 3), + ("swc", "..", 3), + ("snc", "..", 3), + + # Effective target: package/swc + ("runner", "../package/swc", 1), + ("package", "./swc", 1), + ("swc", ".", 1), + ("snc", "../swc", 1), + + # Effective target: package/snc + ("runner", "../package/snc", 1), + ("package", "./snc", 1), + ("swc", "../snc", 1), + ("snc", ".", 1), ]) @pytest.mark.issue616 def test_parsefactories_relative_node_ids( - self, testdir, chdir,testarg, expect_ntests_passed): + self, testdir, chdir, testarg, expect_ntests_passed): dirs = self._setup_tree(testdir) - print("pytest run in cwd: %s" %( + print("pytest run in cwd: %s" % ( dirs[chdir].relto(testdir.tmpdir))) - print("pytestarg : %s" %(testarg)) - print("expected pass : %s" %(expect_ntests_passed)) + print("pytestarg : %s" % (testarg)) + print("expected pass : %s" % (expect_ntests_passed)) with dirs[chdir].as_cwd(): reprec = testdir.inline_run(testarg, "-q", "--traceconfig") reprec.assertoutcome(passed=expect_ntests_passed) @@ -395,7 +414,7 @@ def out_of_reach(): pass def test_issue1073_conftest_special_objects(testdir): testdir.makeconftest(""" - class DontTouchMe: + class DontTouchMe(object): def __getattr__(self, x): raise Exception('cant touch me') @@ -407,3 +426,53 @@ def test_some(): """) res = testdir.runpytest() assert res.ret == 0 + + +def test_conftest_exception_handling(testdir): + testdir.makeconftest(''' + raise ValueError() + ''') + testdir.makepyfile(""" + def test_some(): + pass + """) + res = testdir.runpytest() + assert res.ret == 4 + assert 'raise ValueError()' in [line.strip() for line in res.errlines] + + +def test_hook_proxy(testdir): + """Session's gethookproxy() would cache conftests incorrectly (#2016). + It was decided to remove the cache altogether. + """ + testdir.makepyfile(**{ + 'root/demo-0/test_foo1.py': "def test1(): pass", + + 'root/demo-a/test_foo2.py': "def test1(): pass", + 'root/demo-a/conftest.py': """ + def pytest_ignore_collect(path, config): + return True + """, + + 'root/demo-b/test_foo3.py': "def test1(): pass", + 'root/demo-c/test_foo4.py': "def test1(): pass", + }) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*test_foo1.py*', + '*test_foo3.py*', + '*test_foo4.py*', + '*3 passed*', + ]) + + +def test_required_option_help(testdir): + testdir.makeconftest("assert 0") + x = testdir.mkdir("x") + x.join("conftest.py").write(_pytest._code.Source(""" + def pytest_addoption(parser): + parser.addoption("--xyz", action="store_true", required=True) + """)) + result = testdir.runpytest("-h", x) + assert 'argument --xyz is required' not in result.stdout.str() + assert 'general:' in result.stdout.str() diff --git a/testing/test_doctest.py b/testing/test_doctest.py index a4821ee4c9ecade..b15067f15e9b299 100644 --- a/testing/test_doctest.py +++ b/testing/test_doctest.py @@ -1,10 +1,13 @@ # encoding: utf-8 +from __future__ import absolute_import, division, print_function import sys import _pytest._code +from _pytest.compat import MODULE_NOT_FOUND_ERROR from _pytest.doctest import DoctestItem, DoctestModule, DoctestTextfile import pytest -class TestDoctests: + +class TestDoctests(object): def test_collect_testtextfile(self, testdir): w = testdir.maketxtfile(whatever="") @@ -14,26 +17,29 @@ def test_collect_testtextfile(self, testdir): >>> i-1 4 """) + for x in (testdir.tmpdir, checkfile): - #print "checking that %s returns custom items" % (x,) + # print "checking that %s returns custom items" % (x,) items, reprec = testdir.inline_genitems(x) assert len(items) == 1 - assert isinstance(items[0], DoctestTextfile) + assert isinstance(items[0], DoctestItem) + assert isinstance(items[0].parent, DoctestTextfile) + # Empty file has no items. items, reprec = testdir.inline_genitems(w) - assert len(items) == 1 + assert len(items) == 0 def test_collect_module_empty(self, testdir): path = testdir.makepyfile(whatever="#") for p in (path, testdir.tmpdir): items, reprec = testdir.inline_genitems(p, - '--doctest-modules') + '--doctest-modules') assert len(items) == 0 def test_collect_module_single_modulelevel_doctest(self, testdir): path = testdir.makepyfile(whatever='""">>> pass"""') for p in (path, testdir.tmpdir): items, reprec = testdir.inline_genitems(p, - '--doctest-modules') + '--doctest-modules') assert len(items) == 1 assert isinstance(items[0], DoctestItem) assert isinstance(items[0].parent, DoctestModule) @@ -46,7 +52,7 @@ def my_func(): """) for p in (path, testdir.tmpdir): items, reprec = testdir.inline_genitems(p, - '--doctest-modules') + '--doctest-modules') assert len(items) == 2 assert isinstance(items[0], DoctestItem) assert isinstance(items[1], DoctestItem) @@ -71,7 +77,7 @@ def another(): """) for p in (path, testdir.tmpdir): items, reprec = testdir.inline_genitems(p, - '--doctest-modules') + '--doctest-modules') assert len(items) == 2 assert isinstance(items[0], DoctestItem) assert isinstance(items[1], DoctestItem) @@ -126,6 +132,33 @@ def test_multiple_patterns(self, testdir): '*1 passed*', ]) + @pytest.mark.parametrize( + ' test_string, encoding', + [ + (u'foo', 'ascii'), + (u'öäü', 'latin1'), + (u'öäü', 'utf-8') + ] + ) + def test_encoding(self, testdir, test_string, encoding): + """Test support for doctest_encoding ini option. + """ + testdir.makeini(""" + [pytest] + doctest_encoding={0} + """.format(encoding)) + doctest = u""" + >>> u"{0}" + {1} + """.format(test_string, repr(test_string)) + testdir._makefile(".txt", [doctest], {}, encoding=encoding) + + result = testdir.runpytest() + + result.stdout.fnmatch_lines([ + '*1 passed*', + ]) + def test_doctest_unexpected_exception(self, testdir): testdir.maketxtfile(""" >>> i = 0 @@ -140,7 +173,7 @@ def test_doctest_unexpected_exception(self, testdir): "*UNEXPECTED*ZeroDivision*", ]) - def test_docstring_context_around_error(self, testdir): + def test_docstring_partial_context_around_error(self, testdir): """Test that we show some context before the actual line of a failing doctest. """ @@ -166,7 +199,7 @@ def foo(): ''') result = testdir.runpytest('--doctest-modules') result.stdout.fnmatch_lines([ - '*docstring_context_around_error*', + '*docstring_partial_context_around_error*', '005*text-line-3', '006*text-line-4', '013*text-line-11', @@ -180,6 +213,32 @@ def foo(): assert 'text-line-2' not in result.stdout.str() assert 'text-line-after' not in result.stdout.str() + def test_docstring_full_context_around_error(self, testdir): + """Test that we show the whole context before the actual line of a failing + doctest, provided that the context is up to 10 lines long. + """ + testdir.makepyfile(''' + def foo(): + """ + text-line-1 + text-line-2 + + >>> 1 + 1 + 3 + """ + ''') + result = testdir.runpytest('--doctest-modules') + result.stdout.fnmatch_lines([ + '*docstring_full_context_around_error*', + '003*text-line-1', + '004*text-line-2', + '006*>>> 1 + 1', + 'Expected:', + ' 3', + 'Got:', + ' 2', + ]) + def test_doctest_linedata_missing(self, testdir): testdir.tmpdir.join('hello.py').write(_pytest._code.Source(""" class Fun(object): @@ -199,8 +258,20 @@ def test(self): "*1 failed*", ]) + def test_doctest_unex_importerror_only_txt(self, testdir): + testdir.maketxtfile(""" + >>> import asdalsdkjaslkdjasd + >>> + """) + result = testdir.runpytest() + # doctest is never executed because of error during hello.py collection + result.stdout.fnmatch_lines([ + "*>>> import asdals*", + "*UNEXPECTED*{e}*".format(e=MODULE_NOT_FOUND_ERROR), + "{e}: No module named *asdal*".format(e=MODULE_NOT_FOUND_ERROR), + ]) - def test_doctest_unex_importerror(self, testdir): + def test_doctest_unex_importerror_with_module(self, testdir): testdir.tmpdir.join("hello.py").write(_pytest._code.Source(""" import asdalsdkjaslkdjasd """)) @@ -209,10 +280,11 @@ def test_doctest_unex_importerror(self, testdir): >>> """) result = testdir.runpytest("--doctest-modules") + # doctest is never executed because of error during hello.py collection result.stdout.fnmatch_lines([ - "*>>> import hello", - "*UNEXPECTED*ImportError*", - "*import asdals*", + "*ERROR collecting hello.py*", + "*{e}: No module named *asdals*".format(e=MODULE_NOT_FOUND_ERROR), + "*Interrupted: 1 errors during collection*", ]) def test_doctestmodule(self, testdir): @@ -248,7 +320,6 @@ def somefunc(): "*:5: DocTestFailure" ]) - def test_txtfile_failing(self, testdir): p = testdir.maketxtfile(""" >>> i = 0 @@ -333,7 +404,7 @@ def another(): def test_doctestmodule_two_tests_one_fail(self, testdir): p = testdir.makepyfile(""" - class MyClass: + class MyClass(object): def bad_meth(self): ''' >>> magic = 42 @@ -356,7 +427,7 @@ def test_ignored_whitespace(self, testdir): doctest_optionflags = ELLIPSIS NORMALIZE_WHITESPACE """) p = testdir.makepyfile(""" - class MyClass: + class MyClass(object): ''' >>> a = "foo " >>> print(a) @@ -373,7 +444,7 @@ def test_non_ignored_whitespace(self, testdir): doctest_optionflags = ELLIPSIS """) p = testdir.makepyfile(""" - class MyClass: + class MyClass(object): ''' >>> a = "foo " >>> print(a) @@ -459,8 +530,93 @@ def foo(): "--junit-xml=junit.xml") reprec.assertoutcome(failed=1) + def test_unicode_doctest(self, testdir): + """ + Test case for issue 2434: DecodeError on Python 2 when doctest contains non-ascii + characters. + """ + p = testdir.maketxtfile(test_unicode_doctest=""" + .. doctest:: + + >>> print( + ... "Hi\\n\\nByé") + Hi + ... + Byé + >>> 1/0 # Byé + 1 + """) + result = testdir.runpytest(p) + result.stdout.fnmatch_lines([ + '*UNEXPECTED EXCEPTION: ZeroDivisionError*', + '*1 failed*', + ]) + + def test_unicode_doctest_module(self, testdir): + """ + Test case for issue 2434: DecodeError on Python 2 when doctest docstring + contains non-ascii characters. + """ + p = testdir.makepyfile(test_unicode_doctest_module=""" + # -*- encoding: utf-8 -*- + from __future__ import unicode_literals + + def fix_bad_unicode(text): + ''' + >>> print(fix_bad_unicode('único')) + único + ''' + return "único" + """) + result = testdir.runpytest(p, '--doctest-modules') + result.stdout.fnmatch_lines(['* 1 passed *']) + + def test_reportinfo(self, testdir): + ''' + Test case to make sure that DoctestItem.reportinfo() returns lineno. + ''' + p = testdir.makepyfile(test_reportinfo=""" + def foo(x): + ''' + >>> foo('a') + 'b' + ''' + return 'c' + """) + items, reprec = testdir.inline_genitems(p, '--doctest-modules') + reportinfo = items[0].reportinfo() + assert reportinfo[1] == 1 + + def test_valid_setup_py(self, testdir): + ''' + Test to make sure that pytest ignores valid setup.py files when ran + with --doctest-modules + ''' + p = testdir.makepyfile(setup=""" + from setuptools import setup, find_packages + setup(name='sample', + version='0.0', + description='description', + packages=find_packages() + ) + """) + result = testdir.runpytest(p, '--doctest-modules') + result.stdout.fnmatch_lines(['*collected 0 items*']) + + def test_invalid_setup_py(self, testdir): + ''' + Test to make sure that pytest reads setup.py files that are not used + for python packages when ran with --doctest-modules + ''' + p = testdir.makepyfile(setup=""" + def test_foo(): + return 'bar' + """) + result = testdir.runpytest(p, '--doctest-modules') + result.stdout.fnmatch_lines(['*collected 1 item*']) + -class TestLiterals: +class TestLiterals(object): @pytest.mark.parametrize('config_mode', ['ini', 'comment']) def test_allow_unicode(self, testdir, config_mode): @@ -547,7 +703,7 @@ def test_bytes_literal(self, testdir): reprec.assertoutcome(passed=passed, failed=int(not passed)) -class TestDoctestSkips: +class TestDoctestSkips(object): """ If all examples in a doctest are skipped due to the SKIP option, then the tests should be SKIPPED rather than PASSED. (#957) @@ -595,8 +751,13 @@ def test_all_skipped(self, testdir, makedoctest): reprec = testdir.inline_run("--doctest-modules") reprec.assertoutcome(skipped=1) + def test_vacuous_all_skipped(self, testdir, makedoctest): + makedoctest('') + reprec = testdir.inline_run("--doctest-modules") + reprec.assertoutcome(passed=0, skipped=0) + -class TestDoctestAutoUseFixtures: +class TestDoctestAutoUseFixtures(object): SCOPES = ['module', 'session', 'class', 'function'] @@ -713,3 +874,130 @@ def auto(request): result = testdir.runpytest('--doctest-modules') assert 'FAILURES' not in str(result.stdout.str()) result.stdout.fnmatch_lines(['*=== 1 passed in *']) + + +class TestDoctestNamespaceFixture(object): + + SCOPES = ['module', 'session', 'class', 'function'] + + @pytest.mark.parametrize('scope', SCOPES) + def test_namespace_doctestfile(self, testdir, scope): + """ + Check that inserting something into the namespace works in a + simple text file doctest + """ + testdir.makeconftest(""" + import pytest + import contextlib + + @pytest.fixture(autouse=True, scope="{scope}") + def add_contextlib(doctest_namespace): + doctest_namespace['cl'] = contextlib + """.format(scope=scope)) + p = testdir.maketxtfile(""" + >>> print(cl.__name__) + contextlib + """) + reprec = testdir.inline_run(p) + reprec.assertoutcome(passed=1) + + @pytest.mark.parametrize('scope', SCOPES) + def test_namespace_pyfile(self, testdir, scope): + """ + Check that inserting something into the namespace works in a + simple Python file docstring doctest + """ + testdir.makeconftest(""" + import pytest + import contextlib + + @pytest.fixture(autouse=True, scope="{scope}") + def add_contextlib(doctest_namespace): + doctest_namespace['cl'] = contextlib + """.format(scope=scope)) + p = testdir.makepyfile(""" + def foo(): + ''' + >>> print(cl.__name__) + contextlib + ''' + """) + reprec = testdir.inline_run(p, "--doctest-modules") + reprec.assertoutcome(passed=1) + + +class TestDoctestReportingOption(object): + def _run_doctest_report(self, testdir, format): + testdir.makepyfile(""" + def foo(): + ''' + >>> foo() + a b + 0 1 4 + 1 2 4 + 2 3 6 + ''' + print(' a b\\n' + '0 1 4\\n' + '1 2 5\\n' + '2 3 6') + """) + return testdir.runpytest("--doctest-modules", "--doctest-report", format) + + @pytest.mark.parametrize('format', ['udiff', 'UDIFF', 'uDiFf']) + def test_doctest_report_udiff(self, testdir, format): + result = self._run_doctest_report(testdir, format) + result.stdout.fnmatch_lines([ + ' 0 1 4', + ' -1 2 4', + ' +1 2 5', + ' 2 3 6', + ]) + + def test_doctest_report_cdiff(self, testdir): + result = self._run_doctest_report(testdir, 'cdiff') + result.stdout.fnmatch_lines([ + ' a b', + ' 0 1 4', + ' ! 1 2 4', + ' 2 3 6', + ' --- 1,4 ----', + ' a b', + ' 0 1 4', + ' ! 1 2 5', + ' 2 3 6', + ]) + + def test_doctest_report_ndiff(self, testdir): + result = self._run_doctest_report(testdir, 'ndiff') + result.stdout.fnmatch_lines([ + ' a b', + ' 0 1 4', + ' - 1 2 4', + ' ? ^', + ' + 1 2 5', + ' ? ^', + ' 2 3 6', + ]) + + @pytest.mark.parametrize('format', ['none', 'only_first_failure']) + def test_doctest_report_none_or_only_first_failure(self, testdir, format): + result = self._run_doctest_report(testdir, format) + result.stdout.fnmatch_lines([ + 'Expected:', + ' a b', + ' 0 1 4', + ' 1 2 4', + ' 2 3 6', + 'Got:', + ' a b', + ' 0 1 4', + ' 1 2 5', + ' 2 3 6', + ]) + + def test_doctest_report_invalid(self, testdir): + result = self._run_doctest_report(testdir, 'obviously_invalid_format') + result.stderr.fnmatch_lines([ + "*error: argument --doctest-report: invalid choice: 'obviously_invalid_format' (choose from*" + ]) diff --git a/testing/test_entry_points.py b/testing/test_entry_points.py new file mode 100644 index 000000000000000..6ca68b481fa1f9c --- /dev/null +++ b/testing/test_entry_points.py @@ -0,0 +1,14 @@ +from __future__ import absolute_import, division, print_function +import pkg_resources + +import pytest + + +@pytest.mark.parametrize("entrypoint", ['py.test', 'pytest']) +def test_entry_point_exist(entrypoint): + assert entrypoint in pkg_resources.get_entry_map('pytest')['console_scripts'] + + +def test_pytest_entry_points_are_identical(): + entryMap = pkg_resources.get_entry_map('pytest')['console_scripts'] + assert entryMap['pytest'].module_name == entryMap['py.test'].module_name diff --git a/testing/test_genscript.py b/testing/test_genscript.py deleted file mode 100644 index 1260a5a6b268dd9..000000000000000 --- a/testing/test_genscript.py +++ /dev/null @@ -1,51 +0,0 @@ -import pytest -import sys - - -@pytest.fixture(scope="module") -def standalone(request): - return Standalone(request) - -class Standalone: - def __init__(self, request): - self.testdir = request.getfuncargvalue("testdir") - script = "mypytest" - result = self.testdir.runpytest("--genscript=%s" % script) - assert result.ret == 0 - self.script = self.testdir.tmpdir.join(script) - assert self.script.check() - - def run(self, anypython, testdir, *args): - return testdir._run(anypython, self.script, *args) - -def test_gen(testdir, anypython, standalone): - if sys.version_info >= (2,7): - result = testdir._run(anypython, "-c", - "import sys;print (sys.version_info >=(2,7))") - assert result.ret == 0 - if result.stdout.str() == "False": - pytest.skip("genscript called from python2.7 cannot work " - "earlier python versions") - result = standalone.run(anypython, testdir, '--version') - if result.ret == 2: - result.stderr.fnmatch_lines(["*ERROR: setuptools not installed*"]) - elif result.ret == 0: - result.stderr.fnmatch_lines([ - "*imported from*mypytest*" - ]) - p = testdir.makepyfile("def test_func(): assert 0") - result = standalone.run(anypython, testdir, p) - assert result.ret != 0 - else: - pytest.fail("Unexpected return code") - - -def test_freeze_includes(): - """ - Smoke test for freeze_includes(), to ensure that it works across all - supported python versions. - """ - includes = pytest.freeze_includes() - assert len(includes) > 1 - assert '_pytest.genscript' in includes - diff --git a/testing/test_helpconfig.py b/testing/test_helpconfig.py index 9f8d87b7cb54ff2..845005a0575a436 100644 --- a/testing/test_helpconfig.py +++ b/testing/test_helpconfig.py @@ -1,10 +1,12 @@ +from __future__ import absolute_import, division, print_function from _pytest.main import EXIT_NOTESTSCOLLECTED import pytest + def test_version(testdir, pytestconfig): result = testdir.runpytest("--version") assert result.ret == 0 - #p = py.path.local(py.__file__).dirpath() + # p = py.path.local(py.__file__).dirpath() result.stderr.fnmatch_lines([ '*pytest*%s*imported from*' % (pytest.__version__, ) ]) @@ -14,6 +16,7 @@ def test_version(testdir, pytestconfig): "*at*", ]) + def test_help(testdir): result = testdir.runpytest("--help") assert result.ret == 0 @@ -21,10 +24,11 @@ def test_help(testdir): *-v*verbose* *setup.cfg* *minversion* - *to see*markers*py.test --markers* - *to see*fixtures*py.test --fixtures* + *to see*markers*pytest --markers* + *to see*fixtures*pytest --fixtures* """) + def test_hookvalidation_unknown(testdir): testdir.makeconftest(""" def pytest_hello(xyz): @@ -32,10 +36,11 @@ def pytest_hello(xyz): """) result = testdir.runpytest() assert result.ret != 0 - result.stderr.fnmatch_lines([ + result.stdout.fnmatch_lines([ '*unknown hook*pytest_hello*' ]) + def test_hookvalidation_optional(testdir): testdir.makeconftest(""" import pytest @@ -46,6 +51,7 @@ def pytest_hello(xyz): result = testdir.runpytest() assert result.ret == EXIT_NOTESTSCOLLECTED + def test_traceconfig(testdir): result = testdir.runpytest("--traceconfig") result.stdout.fnmatch_lines([ @@ -53,12 +59,14 @@ def test_traceconfig(testdir): "*active plugins*", ]) + def test_debug(testdir, monkeypatch): result = testdir.runpytest_subprocess("--debug") assert result.ret == EXIT_NOTESTSCOLLECTED p = testdir.tmpdir.join("pytestdebug.log") assert "pytest_sessionstart" in p.read() + def test_PYTEST_DEBUG(testdir, monkeypatch): monkeypatch.setenv("PYTEST_DEBUG", "1") result = testdir.runpytest_subprocess() diff --git a/testing/test_junitxml.py b/testing/test_junitxml.py index 5960f88253543c9..b604c02a3dea9f4 100644 --- a/testing/test_junitxml.py +++ b/testing/test_junitxml.py @@ -1,7 +1,6 @@ # -*- coding: utf-8 -*- - +from __future__ import absolute_import, division, print_function from xml.dom import minidom -from _pytest.main import EXIT_NOTESTSCOLLECTED import py import sys import os @@ -80,7 +79,7 @@ def next_siebling(self): return type(self)(self.__node.nextSibling) -class TestPython: +class TestPython(object): def test_summing_simple(self, testdir): testdir.makepyfile(""" import pytest @@ -100,7 +99,31 @@ def test_xpass(): result, dom = runandparse(testdir) assert result.ret node = dom.find_first_by_tag("testsuite") - node.assert_attr(name="pytest", errors=0, failures=1, skips=3, tests=2) + node.assert_attr(name="pytest", errors=0, failures=1, skips=2, tests=5) + + def test_summing_simple_with_errors(self, testdir): + testdir.makepyfile(""" + import pytest + @pytest.fixture + def fixture(): + raise Exception() + def test_pass(): + pass + def test_fail(): + assert 0 + def test_error(fixture): + pass + @pytest.mark.xfail + def test_xfail(): + assert False + @pytest.mark.xfail(strict=True) + def test_xpass(): + assert True + """) + result, dom = runandparse(testdir) + assert result.ret + node = dom.find_first_by_tag("testsuite") + node.assert_attr(name="pytest", errors=1, failures=2, skips=1, tests=5) def test_timing_function(self, testdir): testdir.makepyfile(""" @@ -120,7 +143,10 @@ def test_sleep(): def test_setup_error(self, testdir): testdir.makepyfile(""" - def pytest_funcarg__arg(request): + import pytest + + @pytest.fixture + def arg(request): raise ValueError() def test_function(arg): pass @@ -128,17 +154,64 @@ def test_function(arg): result, dom = runandparse(testdir) assert result.ret node = dom.find_first_by_tag("testsuite") - node.assert_attr(errors=1, tests=0) + node.assert_attr(errors=1, tests=1) tnode = node.find_first_by_tag("testcase") tnode.assert_attr( file="test_setup_error.py", - line="2", + line="5", classname="test_setup_error", name="test_function") fnode = tnode.find_first_by_tag("error") fnode.assert_attr(message="test setup failure") assert "ValueError" in fnode.toxml() + def test_teardown_error(self, testdir): + testdir.makepyfile(""" + import pytest + + @pytest.fixture + def arg(): + yield + raise ValueError() + def test_function(arg): + pass + """) + result, dom = runandparse(testdir) + assert result.ret + node = dom.find_first_by_tag("testsuite") + tnode = node.find_first_by_tag("testcase") + tnode.assert_attr( + file="test_teardown_error.py", + line="6", + classname="test_teardown_error", + name="test_function") + fnode = tnode.find_first_by_tag("error") + fnode.assert_attr(message="test teardown failure") + assert "ValueError" in fnode.toxml() + + def test_call_failure_teardown_error(self, testdir): + testdir.makepyfile(""" + import pytest + + @pytest.fixture + def arg(): + yield + raise Exception("Teardown Exception") + def test_function(arg): + raise Exception("Call Exception") + """) + result, dom = runandparse(testdir) + assert result.ret + node = dom.find_first_by_tag("testsuite") + node.assert_attr(errors=1, failures=1, tests=1) + first, second = dom.find_by_tag("testcase") + if not first or not second or first == second: + assert 0 + fnode = first.find_first_by_tag("failure") + fnode.assert_attr(message="Exception: Call Exception") + snode = second.find_first_by_tag("error") + snode.assert_attr(message="test teardown failure") + def test_skip_contains_name_reason(self, testdir): testdir.makepyfile(""" import pytest @@ -158,9 +231,62 @@ def test_skip(): snode = tnode.find_first_by_tag("skipped") snode.assert_attr(type="pytest.skip", message="hello23", ) + def test_mark_skip_contains_name_reason(self, testdir): + testdir.makepyfile(""" + import pytest + @pytest.mark.skip(reason="hello24") + def test_skip(): + assert True + """) + result, dom = runandparse(testdir) + assert result.ret == 0 + node = dom.find_first_by_tag("testsuite") + node.assert_attr(skips=1) + tnode = node.find_first_by_tag("testcase") + tnode.assert_attr( + file="test_mark_skip_contains_name_reason.py", + line="1", + classname="test_mark_skip_contains_name_reason", + name="test_skip") + snode = tnode.find_first_by_tag("skipped") + snode.assert_attr(type="pytest.skip", message="hello24", ) + + def test_mark_skipif_contains_name_reason(self, testdir): + testdir.makepyfile(""" + import pytest + GLOBAL_CONDITION = True + @pytest.mark.skipif(GLOBAL_CONDITION, reason="hello25") + def test_skip(): + assert True + """) + result, dom = runandparse(testdir) + assert result.ret == 0 + node = dom.find_first_by_tag("testsuite") + node.assert_attr(skips=1) + tnode = node.find_first_by_tag("testcase") + tnode.assert_attr( + file="test_mark_skipif_contains_name_reason.py", + line="2", + classname="test_mark_skipif_contains_name_reason", + name="test_skip") + snode = tnode.find_first_by_tag("skipped") + snode.assert_attr(type="pytest.skip", message="hello25", ) + + def test_mark_skip_doesnt_capture_output(self, testdir): + testdir.makepyfile(""" + import pytest + @pytest.mark.skip(reason="foo") + def test_skip(): + print("bar!") + """) + result, dom = runandparse(testdir) + assert result.ret == 0 + node_xml = dom.find_first_by_tag("testsuite").toxml() + assert "bar!" not in node_xml + def test_classname_instance(self, testdir): testdir.makepyfile(""" - class TestClass: + class TestClass(object): def test_method(self): assert 0 """) @@ -195,7 +321,7 @@ def test_internal_error(self, testdir): result, dom = runandparse(testdir) assert result.ret node = dom.find_first_by_tag("testsuite") - node.assert_attr(errors=1, tests=0) + node.assert_attr(errors=1, tests=1) tnode = node.find_first_by_tag("testcase") tnode.assert_attr(classname="pytest", name="internal") fnode = tnode.find_first_by_tag("error") @@ -273,7 +399,7 @@ def test_junit_prefixing(self, testdir): testdir.makepyfile(""" def test_func(): assert 0 - class TestHello: + class TestHello(object): def test_hello(self): pass """) @@ -304,7 +430,7 @@ def test_xfail(): result, dom = runandparse(testdir) assert not result.ret node = dom.find_first_by_tag("testsuite") - node.assert_attr(skips=1, tests=0) + node.assert_attr(skips=1, tests=1) tnode = node.find_first_by_tag("testcase") tnode.assert_attr( file="test_xfailure_function.py", @@ -325,23 +451,40 @@ def test_xpass(): result, dom = runandparse(testdir) # assert result.ret node = dom.find_first_by_tag("testsuite") - node.assert_attr(skips=1, tests=0) + node.assert_attr(skips=0, tests=1) tnode = node.find_first_by_tag("testcase") tnode.assert_attr( file="test_xfailure_xpass.py", line="1", classname="test_xfailure_xpass", name="test_xpass") - fnode = tnode.find_first_by_tag("skipped") - fnode.assert_attr(message="xfail-marked test passes unexpectedly") - # assert "ValueError" in fnode.toxml() + + def test_xfailure_xpass_strict(self, testdir): + testdir.makepyfile(""" + import pytest + @pytest.mark.xfail(strict=True, reason="This needs to fail!") + def test_xpass(): + pass + """) + result, dom = runandparse(testdir) + # assert result.ret + node = dom.find_first_by_tag("testsuite") + node.assert_attr(skips=0, tests=1) + tnode = node.find_first_by_tag("testcase") + tnode.assert_attr( + file="test_xfailure_xpass_strict.py", + line="1", + classname="test_xfailure_xpass_strict", + name="test_xpass") + fnode = tnode.find_first_by_tag("failure") + fnode.assert_attr(message="[XPASS(strict)] This needs to fail!") def test_collect_error(self, testdir): testdir.makepyfile("syntax error") result, dom = runandparse(testdir) assert result.ret node = dom.find_first_by_tag("testsuite") - node.assert_attr(errors=1, tests=0) + node.assert_attr(errors=1, tests=1) tnode = node.find_first_by_tag("testcase") tnode.assert_attr( file="test_collect_error.py", @@ -351,23 +494,6 @@ def test_collect_error(self, testdir): fnode.assert_attr(message="collection failure") assert "SyntaxError" in fnode.toxml() - def test_collect_skipped(self, testdir): - testdir.makepyfile("import pytest; pytest.skip('xyz')") - result, dom = runandparse(testdir) - assert result.ret == EXIT_NOTESTSCOLLECTED - node = dom.find_first_by_tag("testsuite") - node.assert_attr(skips=1, tests=0) - tnode = node.find_first_by_tag("testcase") - tnode.assert_attr( - file="test_collect_skipped.py", - name="test_collect_skipped") - - # py.test doesn't give us a line here. - assert tnode["line"] is None - - fnode = tnode.find_first_by_tag("skipped") - fnode.assert_attr(message="collection skipped") - def test_unicode(self, testdir): value = 'hx\xc4\x85\xc4\x87\n' testdir.makepyfile(""" @@ -421,7 +547,10 @@ def test_pass(): def test_setup_error_captures_stdout(self, testdir): testdir.makepyfile(""" - def pytest_funcarg__arg(request): + import pytest + + @pytest.fixture + def arg(request): print('hello-stdout') raise ValueError() def test_function(arg): @@ -436,7 +565,10 @@ def test_function(arg): def test_setup_error_captures_stderr(self, testdir): testdir.makepyfile(""" import sys - def pytest_funcarg__arg(request): + import pytest + + @pytest.fixture + def arg(request): sys.stderr.write('hello-stderr') raise ValueError() def test_function(arg): @@ -448,6 +580,26 @@ def test_function(arg): systemout = pnode.find_first_by_tag("system-err") assert "hello-stderr" in systemout.toxml() + def test_avoid_double_stdout(self, testdir): + testdir.makepyfile(""" + import sys + import pytest + + @pytest.fixture + def arg(request): + yield + sys.stdout.write('hello-stdout teardown') + raise ValueError() + def test_function(arg): + sys.stdout.write('hello-stdout call') + """) + result, dom = runandparse(testdir) + node = dom.find_first_by_tag("testsuite") + pnode = node.find_first_by_tag("testcase") + systemout = pnode.find_first_by_tag("system-out") + assert "hello-stdout call" in systemout.toxml() + assert "hello-stdout teardown" in systemout.toxml() + def test_mangle_test_address(): from _pytest.junitxml import mangle_test_address @@ -460,11 +612,14 @@ def test_mangle_test_address(): def test_dont_configure_on_slaves(tmpdir): gotten = [] - class FakeConfig: + class FakeConfig(object): def __init__(self): self.pluginmanager = self self.option = self + def getini(self, name): + return "pytest" + junitprefix = None # XXX: shouldnt need tmpdir ? xmlpath = str(tmpdir.join('junix.xml')) @@ -479,7 +634,7 @@ def __init__(self): assert len(gotten) == 1 -class TestNonPython: +class TestNonPython(object): def test_summing_simple(self, testdir): testdir.makeconftest(""" import pytest @@ -607,17 +762,23 @@ def test_pass(): assert testdir.tmpdir.join("path/to/results.xml").check() +def test_logxml_check_isdir(testdir): + """Give an error if --junit-xml is a directory (#2089)""" + result = testdir.runpytest("--junit-xml=.") + result.stderr.fnmatch_lines(["*--junitxml must be a filename*"]) + + def test_escaped_parametrized_names_xml(testdir): testdir.makepyfile(""" import pytest - @pytest.mark.parametrize('char', ["\\x00"]) + @pytest.mark.parametrize('char', [u"\\x00"]) def test_func(char): assert char """) result, dom = runandparse(testdir) assert result.ret == 0 node = dom.find_first_by_tag("testcase") - node.assert_attr(name="test_func[#x00]") + node.assert_attr(name="test_func[\\x00]") def test_double_colon_split_function_issue469(testdir): @@ -637,7 +798,7 @@ def test_func(param): def test_double_colon_split_method_issue469(testdir): testdir.makepyfile(""" import pytest - class TestClass: + class TestClass(object): @pytest.mark.parametrize('param', ["double::colon"]) def test_func(self, param): pass @@ -697,7 +858,10 @@ def test_record(record_xml_property, other): pnodes = psnode.find_by_tag('property') pnodes[0].assert_attr(name="bar", value="1") pnodes[1].assert_attr(name="foo", value="<1") - result.stdout.fnmatch_lines('*C3*test_record_property.py*experimental*') + result.stdout.fnmatch_lines([ + 'test_record_property.py::test_record', + '*record_xml_property*experimental*', + ]) def test_record_property_same_name(testdir): @@ -814,3 +978,85 @@ def test_pass(): u'test_fancy_items_regression test_pass' u' test_fancy_items_regression.py', ] + + +def test_global_properties(testdir): + path = testdir.tmpdir.join("test_global_properties.xml") + log = LogXML(str(path), None) + from _pytest.runner import BaseReport + + class Report(BaseReport): + sections = [] + nodeid = "test_node_id" + + log.pytest_sessionstart() + log.add_global_property('foo', 1) + log.add_global_property('bar', 2) + log.pytest_sessionfinish() + + dom = minidom.parse(str(path)) + + properties = dom.getElementsByTagName('properties') + + assert (properties.length == 1), "There must be one node" + + property_list = dom.getElementsByTagName('property') + + assert (property_list.length == 2), "There most be only 2 property nodes" + + expected = {'foo': '1', 'bar': '2'} + actual = {} + + for p in property_list: + k = str(p.getAttribute('name')) + v = str(p.getAttribute('value')) + actual[k] = v + + assert actual == expected + + +def test_url_property(testdir): + test_url = "http://www.github.com/pytest-dev" + path = testdir.tmpdir.join("test_url_property.xml") + log = LogXML(str(path), None) + from _pytest.runner import BaseReport + + class Report(BaseReport): + longrepr = "FooBarBaz" + sections = [] + nodeid = "something" + location = 'tests/filename.py', 42, 'TestClass.method' + url = test_url + + test_report = Report() + + log.pytest_sessionstart() + node_reporter = log._opentestcase(test_report) + node_reporter.append_failure(test_report) + log.pytest_sessionfinish() + + test_case = minidom.parse(str(path)).getElementsByTagName('testcase')[0] + + assert (test_case.getAttribute('url') == test_url), "The URL did not get written to the xml" + + +@pytest.mark.parametrize('suite_name', ['my_suite', '']) +def test_set_suite_name(testdir, suite_name): + if suite_name: + testdir.makeini(""" + [pytest] + junit_suite_name={0} + """.format(suite_name)) + expected = suite_name + else: + expected = 'pytest' + testdir.makepyfile(""" + import pytest + + def test_func(): + pass + """) + result, dom = runandparse(testdir) + assert result.ret == 0 + node = dom.find_first_by_tag("testsuite") + node.assert_attr(name=expected) diff --git a/testing/test_mark.py b/testing/test_mark.py index aa1be6f7c66ba1b..46bf0b0e7787b7d 100644 --- a/testing/test_mark.py +++ b/testing/test_mark.py @@ -1,37 +1,59 @@ +from __future__ import absolute_import, division, print_function import os +import sys -import py, pytest -from _pytest.mark import MarkGenerator as Mark +import pytest +from _pytest.mark import MarkGenerator as Mark, ParameterSet, transfer_markers -class TestMark: + +class TestMark(object): def test_markinfo_repr(self): - from _pytest.mark import MarkInfo - m = MarkInfo("hello", (1,2), {}) + from _pytest.mark import MarkInfo, Mark + m = MarkInfo(Mark("hello", (1, 2), {})) repr(m) - def test_pytest_exists_in_namespace_all(self): - assert 'mark' in py.test.__all__ - assert 'mark' in pytest.__all__ + @pytest.mark.parametrize('attr', ['mark', 'param']) + @pytest.mark.parametrize('modulename', ['py.test', 'pytest']) + def test_pytest_exists_in_namespace_all(self, attr, modulename): + module = sys.modules[modulename] + assert attr in module.__all__ def test_pytest_mark_notcallable(self): mark = Mark() pytest.raises((AttributeError, TypeError), mark) + def test_mark_with_param(self): + def some_function(abc): + pass + + class SomeClass(object): + pass + + assert pytest.mark.fun(some_function) is some_function + assert pytest.mark.fun.with_args(some_function) is not some_function + + assert pytest.mark.fun(SomeClass) is SomeClass + assert pytest.mark.fun.with_args(SomeClass) is not SomeClass + def test_pytest_mark_name_starts_with_underscore(self): mark = Mark() pytest.raises(AttributeError, getattr, mark, '_some_name') def test_pytest_mark_bare(self): mark = Mark() + def f(): pass + mark.hello(f) assert f.hello def test_pytest_mark_keywords(self): mark = Mark() + def f(): pass + mark.world(x=3, y=4)(f) assert f.world assert f.world.kwargs['x'] == 3 @@ -39,8 +61,10 @@ def f(): def test_apply_multiple_and_merge(self): mark = Mark() + def f(): pass + mark.world mark.world(x=3)(f) assert f.world.kwargs['x'] == 3 @@ -53,33 +77,43 @@ def f(): def test_pytest_mark_positional(self): mark = Mark() + def f(): pass + mark.world("hello")(f) assert f.world.args[0] == "hello" mark.world("world")(f) def test_pytest_mark_positional_func_and_keyword(self): mark = Mark() + def f(): raise Exception + m = mark.world(f, omega="hello") + def g(): pass + assert m(g) == g assert g.world.args[0] is f assert g.world.kwargs["omega"] == "hello" def test_pytest_mark_reuse(self): mark = Mark() + def f(): pass + w = mark.some w("hello", reason="123")(f) assert f.some.args[0] == "hello" assert f.some.kwargs['reason'] == "123" + def g(): pass + w("world", reason2="456")(g) assert g.some.args[0] == "world" assert 'reason' not in g.some.kwargs @@ -120,19 +154,55 @@ def test_markers(pytestconfig): rec = testdir.inline_run() rec.assertoutcome(passed=1) + def test_markers_option(testdir): testdir.makeini(""" [pytest] markers = a1: this is a webtest marker a1some: another marker + nodescription """) result = testdir.runpytest("--markers", ) result.stdout.fnmatch_lines([ "*a1*this is a webtest*", "*a1some*another marker", + "*nodescription*", ]) + +def test_ini_markers_whitespace(testdir): + testdir.makeini(""" + [pytest] + markers = + a1 : this is a whitespace marker + """) + testdir.makepyfile(""" + import pytest + + @pytest.mark.a1 + def test_markers(): + assert True + """) + rec = testdir.inline_run("--strict", "-m", "a1") + rec.assertoutcome(passed=1) + + +def test_marker_without_description(testdir): + testdir.makefile(".cfg", setup=""" + [tool:pytest] + markers=slow + """) + testdir.makeconftest(""" + import pytest + pytest.mark.xfail('FAIL') + """) + ftdir = testdir.mkdir("ft1_dummy") + testdir.tmpdir.join("conftest.py").move(ftdir.join("conftest.py")) + rec = testdir.runpytest_subprocess("--strict") + rec.assert_outcomes() + + def test_markers_option_with_plugin_in_current_dir(testdir): testdir.makeconftest('pytest_plugins = "flip_flop"') testdir.makepyfile(flip_flop="""\ @@ -166,6 +236,7 @@ def test_hello(): reprec = testdir.inline_run() reprec.assertoutcome(passed=1) + def test_strict_prohibits_unregistered_markers(testdir): testdir.makepyfile(""" import pytest @@ -179,11 +250,12 @@ def test_hello(): "*unregisteredmark*not*registered*", ]) + @pytest.mark.parametrize("spec", [ - ("xyz", ("test_one",)), - ("xyz and xyz2", ()), - ("xyz2", ("test_two",)), - ("xyz or xyz2", ("test_one", "test_two"),) + ("xyz", ("test_one",)), + ("xyz and xyz2", ()), + ("xyz2", ("test_two",)), + ("xyz or xyz2", ("test_one", "test_two"),) ]) def test_mark_option(spec, testdir): testdir.makepyfile(""" @@ -202,9 +274,10 @@ def test_two(): assert len(passed) == len(passed_result) assert list(passed) == list(passed_result) + @pytest.mark.parametrize("spec", [ - ("interface", ("test_interface",)), - ("not interface", ("test_nointer",)), + ("interface", ("test_interface",)), + ("not interface", ("test_nointer",)), ]) def test_mark_option_custom(spec, testdir): testdir.makeconftest(""" @@ -227,11 +300,12 @@ def test_nointer(): assert len(passed) == len(passed_result) assert list(passed) == list(passed_result) + @pytest.mark.parametrize("spec", [ - ("interface", ("test_interface",)), - ("not interface", ("test_nointer", "test_pass")), - ("pass", ("test_pass",)), - ("not pass", ("test_interface", "test_nointer")), + ("interface", ("test_interface",)), + ("not interface", ("test_nointer", "test_pass")), + ("pass", ("test_pass",)), + ("not pass", ("test_interface", "test_nointer")), ]) def test_keyword_option_custom(spec, testdir): testdir.makepyfile(""" @@ -251,9 +325,9 @@ def test_pass(): @pytest.mark.parametrize("spec", [ - ("None", ("test_func[None]",)), - ("1.3", ("test_func[1.3]",)), - ("2-3", ("test_func[2-3]",)) + ("None", ("test_func[None]",)), + ("1.3", ("test_func[1.3]",)), + ("2-3", ("test_func[2-3]",)) ]) def test_keyword_option_parametrize(spec, testdir): testdir.makepyfile(""" @@ -285,7 +359,42 @@ def test_func(arg): rec.assertoutcome(passed=3) -class TestFunctional: +def test_parametrized_collect_with_wrong_args(testdir): + """Test collect parametrized func with wrong number of args.""" + py_file = testdir.makepyfile(""" + import pytest + + @pytest.mark.parametrize('foo, bar', [(1, 2, 3)]) + def test_func(foo, bar): + pass + """) + + result = testdir.runpytest(py_file) + result.stdout.fnmatch_lines([ + 'E ValueError: In "parametrize" the number of values ((1, 2, 3)) ' + 'must be equal to the number of names ([\'foo\', \'bar\'])' + ]) + + +def test_parametrized_with_kwargs(testdir): + """Test collect parametrized func with wrong number of args.""" + py_file = testdir.makepyfile(""" + import pytest + + @pytest.fixture(params=[1,2]) + def a(request): + return request.param + + @pytest.mark.parametrize(argnames='b', argvalues=[1, 2]) + def test_func(a, b): + pass + """) + + result = testdir.runpytest(py_file) + assert(result.ret == 0) + + +class TestFunctional(object): def test_mark_per_function(self, testdir): p = testdir.makepyfile(""" @@ -310,7 +419,7 @@ def test_func(): def test_marklist_per_class(self, testdir): item = testdir.getitem(""" import pytest - class TestClass: + class TestClass(object): pytestmark = [pytest.mark.hello, pytest.mark.world] def test_func(self): assert TestClass.test_func.hello @@ -323,7 +432,7 @@ def test_marklist_per_module(self, testdir): item = testdir.getitem(""" import pytest pytestmark = [pytest.mark.hello, pytest.mark.world] - class TestClass: + class TestClass(object): def test_func(self): assert TestClass.test_func.hello assert TestClass.test_func.world @@ -336,7 +445,7 @@ def test_mark_per_class_decorator(self, testdir): item = testdir.getitem(""" import pytest @pytest.mark.hello - class TestClass: + class TestClass(object): def test_func(self): assert TestClass.test_func.hello """) @@ -347,7 +456,7 @@ def test_mark_per_class_decorator_plus_existing_dec(self, testdir): item = testdir.getitem(""" import pytest @pytest.mark.hello - class TestClass: + class TestClass(object): pytestmark = pytest.mark.world def test_func(self): assert TestClass.test_func.hello @@ -361,7 +470,7 @@ def test_merging_markers(self, testdir): p = testdir.makepyfile(""" import pytest pytestmark = pytest.mark.hello("pos1", x=1, y=2) - class TestClass: + class TestClass(object): # classlevel overrides module level pytestmark = pytest.mark.hello(x=3) @pytest.mark.hello("pos0", z=4) @@ -376,29 +485,29 @@ def test_func(self): assert marker.kwargs == {'x': 1, 'y': 2, 'z': 4} # test the new __iter__ interface - l = list(marker) - assert len(l) == 3 - assert l[0].args == ("pos0",) - assert l[1].args == () - assert l[2].args == ("pos1", ) + values = list(marker) + assert len(values) == 3 + assert values[0].args == ("pos0",) + assert values[1].args == () + assert values[2].args == ("pos1", ) @pytest.mark.xfail(reason='unfixed') def test_merging_markers_deep(self, testdir): # issue 199 - propagate markers into nested classes p = testdir.makepyfile(""" import pytest - class TestA: + class TestA(object): pytestmark = pytest.mark.a def test_b(self): assert True - class TestC: + class TestC(object): # this one didnt get marked def test_d(self): assert True """) items, rec = testdir.inline_genitems(p) for item in items: - print (item, item.keywords) + print(item, item.keywords) assert 'a' in item.keywords def test_mark_decorator_subclass_does_not_propagate_to_base(self, testdir): @@ -406,7 +515,7 @@ def test_mark_decorator_subclass_does_not_propagate_to_base(self, testdir): import pytest @pytest.mark.a - class Base: pass + class Base(object): pass @pytest.mark.b class Test1(Base): @@ -418,12 +527,36 @@ def test_bar(self): pass items, rec = testdir.inline_genitems(p) self.assert_markers(items, test_foo=('a', 'b'), test_bar=('a',)) + @pytest.mark.issue568 + @pytest.mark.xfail(reason="markers smear on methods of base classes") + def test_mark_should_not_pass_to_siebling_class(self, testdir): + p = testdir.makepyfile(""" + import pytest + + class TestBase(object): + def test_foo(self): + pass + + @pytest.mark.b + class TestSub(TestBase): + pass + + + class TestOtherSub(TestBase): + pass + + """) + items, rec = testdir.inline_genitems(p) + base_item, sub_item, sub_item_other = items + assert not hasattr(base_item.obj, 'b') + assert not hasattr(sub_item_other.obj, 'b') + def test_mark_decorator_baseclasses_merged(self, testdir): p = testdir.makepyfile(""" import pytest @pytest.mark.a - class Base: pass + class Base(object): pass @pytest.mark.b class Base2(Base): pass @@ -443,23 +576,24 @@ def test_bar(self): pass def test_mark_with_wrong_marker(self, testdir): reprec = testdir.inline_runsource(""" import pytest - class pytestmark: + class pytestmark(object): pass def test_func(): pass """) - l = reprec.getfailedcollections() - assert len(l) == 1 - assert "TypeError" in str(l[0].longrepr) + values = reprec.getfailedcollections() + assert len(values) == 1 + assert "TypeError" in str(values[0].longrepr) def test_mark_dynamically_in_funcarg(self, testdir): testdir.makeconftest(""" import pytest - def pytest_funcarg__arg(request): + @pytest.fixture + def arg(request): request.applymarker(pytest.mark.hello) def pytest_terminal_summary(terminalreporter): - l = terminalreporter.stats['passed'] - terminalreporter.writer.line("keyword: %s" % l[0].keywords) + values = terminalreporter.stats['passed'] + terminalreporter._tw.line("keyword: %s" % values[0].keywords) """) testdir.makepyfile(""" def test_func(arg): @@ -482,10 +616,10 @@ def test_func(): item, = items keywords = item.keywords marker = keywords['hello'] - l = list(marker) - assert len(l) == 2 - assert l[0].args == ("pos0",) - assert l[1].args == ("pos1",) + values = list(marker) + assert len(values) == 2 + assert values[0].args == ("pos0",) + assert values[1].args == ("pos1",) def test_no_marker_match_on_unmarked_names(self, testdir): p = testdir.makepyfile(""" @@ -563,8 +697,32 @@ def assert_markers(self, items, **expected): if isinstance(v, MarkInfo)]) assert marker_names == set(expected_markers) + @pytest.mark.xfail(reason='callspec2.setmulti misuses keywords') + @pytest.mark.issue1540 + def test_mark_from_parameters(self, testdir): + testdir.makepyfile(""" + import pytest + + pytestmark = pytest.mark.skipif(True, reason='skip all') + + # skipifs inside fixture params + params = [pytest.mark.skipif(False, reason='dont skip')('parameter')] + + + @pytest.fixture(params=params) + def parameter(request): + return request.param + + + def test_1(parameter): + assert True + """) + reprec = testdir.inline_run() + reprec.assertoutcome(skipped=1) + + +class TestKeywordSelection(object): -class TestKeywordSelection: def test_select_simple(self, testdir): file_test = testdir.makepyfile(""" def test_one(): @@ -573,6 +731,7 @@ class TestClass(object): def test_method_one(self): assert 42 == 43 """) + def check(keyword, name): reprec = testdir.inline_run("-s", "-k", keyword, file_test) passed, skipped, failed = reprec.listoutcomes() @@ -591,7 +750,7 @@ def test_select_extra_keywords(self, testdir, keyword): p = testdir.makepyfile(test_select=""" def test_1(): pass - class TestClass: + class TestClass(object): def test_2(self): pass """) @@ -605,7 +764,7 @@ def pytest_pycollect_makeitem(name): item.extra_keyword_matches.add("xxx") """) reprec = testdir.inline_run(p.dirpath(), '-s', '-k', keyword) - py.builtin.print_("keyword", repr(keyword)) + print("keyword", repr(keyword)) passed, skipped, failed = reprec.listoutcomes() assert len(passed) == 1 assert passed[0].nodeid.endswith("test_2") @@ -659,6 +818,7 @@ def test_no_magic_values(self, testdir): p = testdir.makepyfile(""" def test_one(): assert 1 """) + def assert_test_is_not_selected(keyword): reprec = testdir.inline_run("-k", keyword, p) passed, skipped, failed = reprec.countoutcomes() @@ -670,3 +830,49 @@ def assert_test_is_not_selected(keyword): assert_test_is_not_selected("__") assert_test_is_not_selected("()") + +@pytest.mark.parametrize('argval, expected', [ + (pytest.mark.skip()((1, 2)), + ParameterSet(values=(1, 2), marks=[pytest.mark.skip], id=None)), + (pytest.mark.xfail(pytest.mark.skip()((1, 2))), + ParameterSet(values=(1, 2), + marks=[pytest.mark.xfail, pytest.mark.skip], id=None)), + +]) +@pytest.mark.filterwarnings('ignore') +def test_parameterset_extractfrom(argval, expected): + extracted = ParameterSet.extract_from(argval) + assert extracted == expected + + +def test_legacy_transfer(): + + class FakeModule(object): + pytestmark = [] + + class FakeClass(object): + pytestmark = pytest.mark.nofun + + @pytest.mark.fun + def fake_method(self): + pass + + transfer_markers(fake_method, FakeClass, FakeModule) + + # legacy marks transfer smeared + assert fake_method.nofun + assert fake_method.fun + # pristine marks dont transfer + assert fake_method.pytestmark == [pytest.mark.fun.mark] + + +class TestMarkDecorator(object): + + @pytest.mark.parametrize('lhs, rhs, expected', [ + (pytest.mark.foo(), pytest.mark.foo(), True), + (pytest.mark.foo(), pytest.mark.bar(), False), + (pytest.mark.foo(), 'bar', False), + ('foo', pytest.mark.bar(), False) + ]) + def test__eq__(self, lhs, rhs, expected): + assert (lhs == rhs) == expected diff --git a/testing/test_modimport.py b/testing/test_modimport.py new file mode 100644 index 000000000000000..2ab86bf7af10664 --- /dev/null +++ b/testing/test_modimport.py @@ -0,0 +1,25 @@ +import py +import subprocess +import sys +import pytest +import _pytest + +MODSET = [ + x for x in py.path.local(_pytest.__file__).dirpath().visit('*.py') + if x.purebasename != '__init__' +] + + +@pytest.mark.parametrize('modfile', MODSET, ids=lambda x: x.purebasename) +def test_fileimport(modfile): + # this test ensures all internal packages can import + # without needing the pytest namespace being set + # this is critical for the initialization of xdist + + res = subprocess.call([ + sys.executable, + '-c', 'import sys, py; py.path.local(sys.argv[1]).pyimport()', + modfile.strpath, + ]) + if res: + pytest.fail("command result %s" % res) diff --git a/testing/test_monkeypatch.py b/testing/test_monkeypatch.py index 048c942c867c8fd..4427908ab3bcfce 100644 --- a/testing/test_monkeypatch.py +++ b/testing/test_monkeypatch.py @@ -1,25 +1,23 @@ +from __future__ import absolute_import, division, print_function import os import sys import textwrap import pytest -from _pytest.monkeypatch import monkeypatch as MonkeyPatch +from _pytest.monkeypatch import MonkeyPatch -def pytest_funcarg__mp(request): +@pytest.fixture +def mp(): cwd = os.getcwd() sys_path = list(sys.path) - - def cleanup(): - sys.path[:] = sys_path - os.chdir(cwd) - - request.addfinalizer(cleanup) - return MonkeyPatch() + yield MonkeyPatch() + sys.path[:] = sys_path + os.chdir(cwd) def test_setattr(): - class A: + class A(object): x = 1 monkeypatch = MonkeyPatch() @@ -42,7 +40,7 @@ class A: assert A.x == 5 -class TestSetattrWithImportPath: +class TestSetattrWithImportPath(object): def test_string_expression(self, monkeypatch): monkeypatch.setattr("os.path.abspath", lambda x: "hello2") assert os.path.abspath("123") == "hello2" @@ -82,7 +80,7 @@ def test_delattr(self, monkeypatch): def test_delattr(): - class A: + class A(object): x = 1 monkeypatch = MonkeyPatch() @@ -205,7 +203,7 @@ def test_setenv_prepend(): def test_monkeypatch_plugin(testdir): reprec = testdir.inline_runsource(""" def test_method(monkeypatch): - assert monkeypatch.__class__.__name__ == "monkeypatch" + assert monkeypatch.__class__.__name__ == "MonkeyPatch" """) res = reprec.countoutcomes() assert tuple(res) == (1, 0, 0), res @@ -297,7 +295,7 @@ class SampleNewInherit(SampleNew): pass -class SampleOld: +class SampleOld(object): # oldstyle on python2 @staticmethod def hello(): @@ -321,10 +319,11 @@ def test_issue156_undo_staticmethod(Sample): monkeypatch.undo() assert Sample.hello() + def test_issue1338_name_resolving(): pytest.importorskip('requests') monkeypatch = MonkeyPatch() try: - monkeypatch.delattr('requests.sessions.Session.request') + monkeypatch.delattr('requests.sessions.Session.request') finally: - monkeypatch.undo() \ No newline at end of file + monkeypatch.undo() diff --git a/testing/test_nodes.py b/testing/test_nodes.py new file mode 100644 index 000000000000000..6f4540f99b96451 --- /dev/null +++ b/testing/test_nodes.py @@ -0,0 +1,18 @@ +import pytest + +from _pytest import nodes + + +@pytest.mark.parametrize("baseid, nodeid, expected", ( + ('', '', True), + ('', 'foo', True), + ('', 'foo/bar', True), + ('', 'foo/bar::TestBaz::()', True), + ('foo', 'food', False), + ('foo/bar::TestBaz::()', 'foo/bar', False), + ('foo/bar::TestBaz::()', 'foo/bar::TestBop::()', False), + ('foo/bar', 'foo/bar::TestBop::()', True), +)) +def test_ischildnode(baseid, nodeid, expected): + result = nodes.ischildnode(baseid, nodeid) + assert result is expected diff --git a/testing/test_nose.py b/testing/test_nose.py index a5162381e47079e..df3e1a94b05bfcc 100644 --- a/testing/test_nose.py +++ b/testing/test_nose.py @@ -1,22 +1,25 @@ +from __future__ import absolute_import, division, print_function import pytest + def setup_module(mod): mod.nose = pytest.importorskip("nose") + def test_nose_setup(testdir): p = testdir.makepyfile(""" - l = [] + values = [] from nose.tools import with_setup - @with_setup(lambda: l.append(1), lambda: l.append(2)) + @with_setup(lambda: values.append(1), lambda: values.append(2)) def test_hello(): - assert l == [1] + assert values == [1] def test_world(): - assert l == [1,2] + assert values == [1,2] - test_hello.setup = lambda: l.append(1) - test_hello.teardown = lambda: l.append(2) + test_hello.setup = lambda: values.append(1) + test_hello.teardown = lambda: values.append(2) """) result = testdir.runpytest(p, '-p', 'nose') result.assert_outcomes(passed=2) @@ -24,43 +27,48 @@ def test_world(): def test_setup_func_with_setup_decorator(): from _pytest.nose import call_optional - l = [] - class A: + values = [] + + class A(object): @pytest.fixture(autouse=True) def f(self): - l.append(1) + values.append(1) + call_optional(A(), "f") - assert not l + assert not values def test_setup_func_not_callable(): from _pytest.nose import call_optional - class A: + + class A(object): f = 1 + call_optional(A(), "f") + def test_nose_setup_func(testdir): p = testdir.makepyfile(""" from nose.tools import with_setup - l = [] + values = [] def my_setup(): a = 1 - l.append(a) + values.append(a) def my_teardown(): b = 2 - l.append(b) + values.append(b) @with_setup(my_setup, my_teardown) def test_hello(): - print (l) - assert l == [1] + print (values) + assert values == [1] def test_world(): - print (l) - assert l == [1,2] + print (values) + assert values == [1,2] """) result = testdir.runpytest(p, '-p', 'nose') @@ -71,18 +79,18 @@ def test_nose_setup_func_failure(testdir): p = testdir.makepyfile(""" from nose.tools import with_setup - l = [] + values = [] my_setup = lambda x: 1 my_teardown = lambda x: 2 @with_setup(my_setup, my_teardown) def test_hello(): - print (l) - assert l == [1] + print (values) + assert values == [1] def test_world(): - print (l) - assert l == [1,2] + print (values) + assert values == [1,2] """) result = testdir.runpytest(p, '-p', 'nose') @@ -93,13 +101,13 @@ def test_world(): def test_nose_setup_func_failure_2(testdir): testdir.makepyfile(""" - l = [] + values = [] my_setup = 1 my_teardown = 2 def test_hello(): - assert l == [] + assert values == [] test_hello.setup = my_setup test_hello.teardown = my_teardown @@ -107,31 +115,32 @@ def test_hello(): reprec = testdir.inline_run() reprec.assertoutcome(passed=1) + def test_nose_setup_partial(testdir): pytest.importorskip("functools") p = testdir.makepyfile(""" from functools import partial - l = [] + values = [] def my_setup(x): a = x - l.append(a) + values.append(a) def my_teardown(x): b = x - l.append(b) + values.append(b) my_setup_partial = partial(my_setup, 1) my_teardown_partial = partial(my_teardown, 2) def test_hello(): - print (l) - assert l == [1] + print (values) + assert values == [1] def test_world(): - print (l) - assert l == [1,2] + print (values) + assert values == [1,2] test_hello.setup = my_setup_partial test_hello.teardown = my_teardown_partial @@ -242,31 +251,32 @@ def test_local_setup(): def test_nose_style_setup_teardown(testdir): testdir.makepyfile(""" - l = [] + values = [] def setup_module(): - l.append(1) + values.append(1) def teardown_module(): - del l[0] + del values[0] def test_hello(): - assert l == [1] + assert values == [1] def test_world(): - assert l == [1] + assert values == [1] """) result = testdir.runpytest('-p', 'nose') result.stdout.fnmatch_lines([ "*2 passed*", ]) + def test_nose_setup_ordering(testdir): testdir.makepyfile(""" def setup_module(mod): mod.visited = True - class TestClass: + class TestClass(object): def setup(self): assert visited def test_first(self): @@ -300,6 +310,7 @@ def test_fun(self): result = testdir.runpytest() result.assert_outcomes(passed=1) + def test_setup_teardown_linking_issue265(testdir): # we accidentally didnt integrate nose setupstate with normal setupstate # this test ensures that won't happen again @@ -347,6 +358,7 @@ def test_skipping(): reprec = testdir.inline_run() reprec.assertoutcome(skipped=1) + def test_istest_function_decorator(testdir): p = testdir.makepyfile(""" import nose.tools @@ -357,6 +369,7 @@ def not_test_prefix(): result = testdir.runpytest(p) result.assert_outcomes(passed=1) + def test_nottest_function_decorator(testdir): testdir.makepyfile(""" import nose.tools @@ -369,22 +382,24 @@ def test_prefix(): calls = reprec.getreports("pytest_runtest_logreport") assert not calls + def test_istest_class_decorator(testdir): p = testdir.makepyfile(""" import nose.tools @nose.tools.istest - class NotTestPrefix: + class NotTestPrefix(object): def test_method(self): pass """) result = testdir.runpytest(p) result.assert_outcomes(passed=1) + def test_nottest_class_decorator(testdir): testdir.makepyfile(""" import nose.tools @nose.tools.nottest - class TestPrefix: + class TestPrefix(object): def test_method(self): pass """) diff --git a/testing/test_parseopt.py b/testing/test_parseopt.py index e45ee285409988e..921592570197aa0 100644 --- a/testing/test_parseopt.py +++ b/testing/test_parseopt.py @@ -1,14 +1,17 @@ -from __future__ import with_statement +from __future__ import absolute_import, division, print_function import sys import os -import py, pytest +import py +import pytest from _pytest import config as parseopt + @pytest.fixture def parser(): return parseopt.Parser() -class TestParser: + +class TestParser(object): def test_no_help_by_default(self, capsys): parser = parseopt.Parser(usage="xyz") pytest.raises(SystemExit, lambda: parser.parse(["-h"])) @@ -29,17 +32,21 @@ def test_argument(self): assert argument.dest == 'test' argument = parseopt.Argument('-t', '--test', dest='abc') assert argument.dest == 'abc' + assert str(argument) == ( + "Argument(_short_opts: ['-t'], _long_opts: ['--test'], dest: 'abc')" + ) def test_argument_type(self): - argument = parseopt.Argument('-t', dest='abc', type='int') + argument = parseopt.Argument('-t', dest='abc', type=int) assert argument.type is int - argument = parseopt.Argument('-t', dest='abc', type='string') + argument = parseopt.Argument('-t', dest='abc', type=str) assert argument.type is str argument = parseopt.Argument('-t', dest='abc', type=float) assert argument.type is float - with pytest.raises(KeyError): - argument = parseopt.Argument('-t', dest='abc', type='choice') - argument = parseopt.Argument('-t', dest='abc', type='choice', + with pytest.warns(DeprecationWarning): + with pytest.raises(KeyError): + argument = parseopt.Argument('-t', dest='abc', type='choice') + argument = parseopt.Argument('-t', dest='abc', type=str, choices=['red', 'blue']) assert argument.type is str @@ -77,6 +84,13 @@ def test_group_addoption(self): assert len(group.options) == 1 assert isinstance(group.options[0], parseopt.Argument) + def test_group_addoption_conflict(self): + group = parseopt.OptionGroup("hello again") + group.addoption("--option1", "--option-1", action="store_true") + with pytest.raises(ValueError) as err: + group.addoption("--option1", "--option-one", action="store_true") + assert str(set(["--option1"])) in str(err.value) + def test_group_shortopt_lowercase(self, parser): group = parser.getgroup("hello") pytest.raises(ValueError, """ @@ -128,7 +142,10 @@ def test_parse_will_set_default(self, parser): def test_parse_setoption(self, parser): parser.addoption("--hello", dest="hello", action="store") parser.addoption("--world", dest="world", default=42) - class A: pass + + class A(object): + pass + option = A() args = parser.parse_setoption(['--hello', 'world'], option) assert option.hello == "world" @@ -147,12 +164,12 @@ def test_parse_split_positional_arguments(self, parser): assert getattr(args, parseopt.FILE_OR_DIR) == ['4', '2'] args = parser.parse(['-R', '-S', '4', '2', '-R']) assert getattr(args, parseopt.FILE_OR_DIR) == ['4', '2'] - assert args.R == True - assert args.S == False + assert args.R is True + assert args.S is False args = parser.parse(['-R', '4', '-S', '2']) assert getattr(args, parseopt.FILE_OR_DIR) == ['4', '2'] - assert args.R == True - assert args.S == False + assert args.R is True + assert args.S is False def test_parse_defaultgetter(self): def defaultget(option): @@ -163,8 +180,8 @@ def defaultget(option): elif option.type is str: option.default = "world" parser = parseopt.Parser(processopt=defaultget) - parser.addoption("--this", dest="this", type="int", action="store") - parser.addoption("--hello", dest="hello", type="string", action="store") + parser.addoption("--this", dest="this", type=int, action="store") + parser.addoption("--hello", dest="hello", type=str, action="store") parser.addoption("--no", dest="no", action="store_true") option = parser.parse([]) assert option.hello == "world" @@ -174,7 +191,7 @@ def defaultget(option): def test_drop_short_helper(self): parser = py.std.argparse.ArgumentParser(formatter_class=parseopt.DropShorterLongHelpFormatter) parser.add_argument('-t', '--twoword', '--duo', '--two-word', '--two', - help='foo').map_long_option = {'two': 'two-word'} + help='foo').map_long_option = {'two': 'two-word'} # throws error on --deux only! parser.add_argument('-d', '--deuxmots', '--deux-mots', action='store_true', help='foo').map_long_option = {'deux': 'deux-mots'} @@ -224,21 +241,33 @@ def test_drop_short_3(self, parser): assert args.file_or_dir == ['abcd'] def test_drop_short_help0(self, parser, capsys): - parser.addoption('--func-args', '--doit', help = 'foo', + parser.addoption('--func-args', '--doit', help='foo', action='store_true') parser.parse([]) help = parser.optparser.format_help() - assert '--func-args, --doit foo' in help + assert '--func-args, --doit foo' in help # testing would be more helpful with all help generated def test_drop_short_help1(self, parser, capsys): group = parser.getgroup("general") group.addoption('--doit', '--func-args', action='store_true', help='foo') group._addoption("-h", "--help", action="store_true", dest="help", - help="show help message and configuration info") + help="show help message and configuration info") parser.parse(['-h']) help = parser.optparser.format_help() - assert '-doit, --func-args foo' in help + assert '-doit, --func-args foo' in help + + def test_multiple_metavar_help(self, parser): + """ + Help text for options with a metavar tuple should display help + in the form "--preferences=value1 value2 value3" (#2004). + """ + group = parser.getgroup("general") + group.addoption('--preferences', metavar=('value1', 'value2', 'value3'), nargs=3) + group._addoption("-h", "--help", action="store_true", dest="help") + parser.parse(['-h']) + help = parser.optparser.format_help() + assert '--preferences=value1 value2 value3' in help def test_argcomplete(testdir, monkeypatch): @@ -246,8 +275,8 @@ def test_argcomplete(testdir, monkeypatch): pytest.skip("bash not available") script = str(testdir.tmpdir.join("test_argcomplete")) pytest_bin = sys.argv[0] - if "py.test" not in os.path.basename(pytest_bin): - pytest.skip("need to be run with py.test executable, not %s" %(pytest_bin,)) + if "pytest" not in os.path.basename(pytest_bin): + pytest.skip("need to be run with pytest executable, not %s" % (pytest_bin,)) with open(str(script), 'w') as fp: # redirect output from argcomplete to stdin and stderr is not trivial @@ -258,12 +287,12 @@ def test_argcomplete(testdir, monkeypatch): # to handle a keyword argument env that replaces os.environ in popen or # extends the copy, advantage: could not forget to restore monkeypatch.setenv('_ARGCOMPLETE', "1") - monkeypatch.setenv('_ARGCOMPLETE_IFS',"\x0b") + monkeypatch.setenv('_ARGCOMPLETE_IFS', "\x0b") monkeypatch.setenv('COMP_WORDBREAKS', ' \\t\\n"\\\'><=;|&(:') arg = '--fu' - monkeypatch.setenv('COMP_LINE', "py.test " + arg) - monkeypatch.setenv('COMP_POINT', str(len("py.test " + arg))) + monkeypatch.setenv('COMP_LINE', "pytest " + arg) + monkeypatch.setenv('COMP_POINT', str(len("pytest " + arg))) result = testdir.run('bash', str(script), arg) if result.ret == 255: # argcomplete not found @@ -271,17 +300,10 @@ def test_argcomplete(testdir, monkeypatch): elif not result.stdout.str(): pytest.skip("bash provided no output, argcomplete not available?") else: - if py.std.sys.version_info < (2,7): - result.stdout.lines = result.stdout.lines[0].split('\x0b') - result.stdout.fnmatch_lines(["--funcargs", "--fulltrace"]) - else: - result.stdout.fnmatch_lines(["--funcargs", "--fulltrace"]) - if py.std.sys.version_info < (2,7): - return + result.stdout.fnmatch_lines(["--funcargs", "--fulltrace"]) os.mkdir('test_argcomplete.d') arg = 'test_argc' - monkeypatch.setenv('COMP_LINE', "py.test " + arg) - monkeypatch.setenv('COMP_POINT', str(len('py.test ' + arg))) + monkeypatch.setenv('COMP_LINE', "pytest " + arg) + monkeypatch.setenv('COMP_POINT', str(len('pytest ' + arg))) result = testdir.run('bash', str(script), arg) result.stdout.fnmatch_lines(["test_argcomplete", "test_argcomplete.d/"]) - diff --git a/testing/test_pastebin.py b/testing/test_pastebin.py index 03570a5c70b3520..6b1742d1415555c 100644 --- a/testing/test_pastebin.py +++ b/testing/test_pastebin.py @@ -1,8 +1,10 @@ # encoding: utf-8 +from __future__ import absolute_import, division, print_function import sys import pytest -class TestPasteCapture: + +class TestPasteCapture(object): @pytest.fixture def pastebinlist(self, monkeypatch, request): @@ -25,7 +27,7 @@ def test_skip(): assert len(pastebinlist) == 1 s = pastebinlist[0] assert s.find("def test_fail") != -1 - assert reprec.countoutcomes() == [1,1,1] + assert reprec.countoutcomes() == [1, 1, 1] def test_all(self, testdir, pastebinlist): from _pytest.pytester import LineMatcher @@ -39,7 +41,7 @@ def test_skip(): pytest.skip("") """) reprec = testdir.inline_run(testpath, "--pastebin=all", '-v') - assert reprec.countoutcomes() == [1,1,1] + assert reprec.countoutcomes() == [1, 1, 1] assert len(pastebinlist) == 1 contents = pastebinlist[0].decode('utf-8') matcher = LineMatcher(contents.splitlines()) @@ -71,7 +73,7 @@ def test(): ]) -class TestPaste: +class TestPaste(object): @pytest.fixture def pastebin(self, request): @@ -84,9 +86,11 @@ def mocked_urlopen(self, monkeypatch): function that connects to bpaste service. """ calls = [] + def mocked(url, data): calls.append((url, data)) - class DummyFile: + + class DummyFile(object): def read(self): # part of html of a normal response return b'View raw.' @@ -111,5 +115,3 @@ def test_create_new_paste(self, pastebin, mocked_urlopen): assert 'lexer=%s' % lexer in data.decode() assert 'code=full-paste-contents' in data.decode() assert 'expiry=1week' in data.decode() - - diff --git a/testing/test_pdb.py b/testing/test_pdb.py index eeddcf0ae80c517..70a5c3c5bdbac2b 100644 --- a/testing/test_pdb.py +++ b/testing/test_pdb.py @@ -1,23 +1,49 @@ +from __future__ import absolute_import, division, print_function import sys +import platform import _pytest._code +import pytest def runpdb_and_get_report(testdir, source): p = testdir.makepyfile(source) result = testdir.runpytest_inprocess("--pdb", p) reports = result.reprec.getreports("pytest_runtest_logreport") - assert len(reports) == 3, reports # setup/call/teardown + assert len(reports) == 3, reports # setup/call/teardown return reports[1] -class TestPDB: - def pytest_funcarg__pdblist(self, request): - monkeypatch = request.getfuncargvalue("monkeypatch") +@pytest.fixture +def custom_pdb_calls(): + called = [] + + # install dummy debugger class and track which methods were called on it + class _CustomPdb(object): + def __init__(self, *args, **kwargs): + called.append("init") + + def reset(self): + called.append("reset") + + def interaction(self, *args): + called.append("interaction") + + _pytest._CustomPdb = _CustomPdb + return called + + +class TestPDB(object): + + @pytest.fixture + def pdblist(self, request): + monkeypatch = request.getfixturevalue("monkeypatch") pdblist = [] + def mypdb(*args): pdblist.append(args) - plugin = request.config.pluginmanager.getplugin('pdb') + + plugin = request.config.pluginmanager.getplugin('debugging') monkeypatch.setattr(plugin, 'post_mortem', mypdb) return pdblist @@ -73,9 +99,48 @@ def test_1(): rest = child.read().decode("utf8") assert "1 failed" in rest assert "def test_1" not in rest + self.flush(child) + + @staticmethod + def flush(child): + if platform.system() == 'Darwin': + return if child.isalive(): child.wait() + def test_pdb_unittest_postmortem(self, testdir): + p1 = testdir.makepyfile(""" + import unittest + class Blub(unittest.TestCase): + def tearDown(self): + self.filename = None + def test_false(self): + self.filename = 'debug' + '.me' + assert 0 + """) + child = testdir.spawn_pytest("--pdb %s" % p1) + child.expect('(Pdb)') + child.sendline('p self.filename') + child.sendeof() + rest = child.read().decode("utf8") + assert 'debug.me' in rest + self.flush(child) + + def test_pdb_unittest_skip(self, testdir): + """Test for issue #2137""" + p1 = testdir.makepyfile(""" + import unittest + @unittest.skipIf(True, 'Skipping also with pdb active') + class MyTestCase(unittest.TestCase): + def test_one(self): + assert 0 + """) + child = testdir.spawn_pytest("-rs --pdb %s" % p1) + child.expect('Skipping also with pdb active') + child.expect('1 skipped in') + child.sendeof() + self.flush(child) + def test_pdb_interaction_capture(self, testdir): p1 = testdir.makepyfile(""" def test_1(): @@ -89,8 +154,7 @@ def test_1(): rest = child.read().decode("utf8") assert "1 failed" in rest assert "getrekt" not in rest - if child.isalive(): - child.wait() + self.flush(child) def test_pdb_interaction_exception(self, testdir): p1 = testdir.makepyfile(""" @@ -108,8 +172,7 @@ def test_1(): child.expect(".*function") child.sendeof() child.expect("1 failed") - if child.isalive(): - child.wait() + self.flush(child) def test_pdb_interaction_on_collection_issue181(self, testdir): p1 = testdir.makepyfile(""" @@ -117,12 +180,11 @@ def test_pdb_interaction_on_collection_issue181(self, testdir): xxx """) child = testdir.spawn_pytest("--pdb %s" % p1) - #child.expect(".*import pytest.*") + # child.expect(".*import pytest.*") child.expect("(Pdb)") child.sendeof() child.expect("1 error") - if child.isalive(): - child.wait() + self.flush(child) def test_pdb_interaction_on_internal_error(self, testdir): testdir.makeconftest(""" @@ -131,11 +193,10 @@ def pytest_runtest_protocol(): """) p1 = testdir.makepyfile("def test_func(): pass") child = testdir.spawn_pytest("--pdb %s" % p1) - #child.expect(".*import pytest.*") + # child.expect(".*import pytest.*") child.expect("(Pdb)") child.sendeof() - if child.isalive(): - child.wait() + self.flush(child) def test_pdb_interaction_capturing_simple(self, testdir): p1 = testdir.makepyfile(""" @@ -154,9 +215,8 @@ def test_1(): rest = child.read().decode("utf-8") assert "1 failed" in rest assert "def test_1" in rest - assert "hello17" in rest # out is captured - if child.isalive(): - child.wait() + assert "hello17" in rest # out is captured + self.flush(child) def test_pdb_set_trace_interception(self, testdir): p1 = testdir.makepyfile(""" @@ -171,8 +231,7 @@ def test_1(): rest = child.read().decode("utf8") assert "1 failed" in rest assert "reading from stdin while output" not in rest - if child.isalive(): - child.wait() + self.flush(child) def test_pdb_and_capsys(self, testdir): p1 = testdir.makepyfile(""" @@ -187,8 +246,7 @@ def test_1(capsys): child.expect("hello1") child.sendeof() child.read() - if child.isalive(): - child.wait() + self.flush(child) def test_set_trace_capturing_afterwards(self, testdir): p1 = testdir.makepyfile(""" @@ -207,8 +265,7 @@ def test_2(): child.expect("hello") child.sendeof() child.read() - if child.isalive(): - child.wait() + self.flush(child) def test_pdb_interaction_doctest(self, testdir): p1 = testdir.makepyfile(""" @@ -227,8 +284,7 @@ def function_1(): child.sendeof() rest = child.read().decode("utf8") assert "1 failed" in rest - if child.isalive(): - child.wait() + self.flush(child) def test_pdb_interaction_capturing_twice(self, testdir): p1 = testdir.makepyfile(""" @@ -252,10 +308,9 @@ def test_1(): rest = child.read().decode("utf8") assert "1 failed" in rest assert "def test_1" in rest - assert "hello17" in rest # out is captured - assert "hello18" in rest # out is captured - if child.isalive(): - child.wait() + assert "hello17" in rest # out is captured + assert "hello18" in rest # out is captured + self.flush(child) def test_pdb_used_outside_test(self, testdir): p1 = testdir.makepyfile(""" @@ -263,10 +318,10 @@ def test_pdb_used_outside_test(self, testdir): pytest.set_trace() x = 5 """) - child = testdir.spawn("%s %s" %(sys.executable, p1)) + child = testdir.spawn("%s %s" % (sys.executable, p1)) child.expect("x = 5") child.sendeof() - child.wait() + self.flush(child) def test_pdb_used_in_generate_tests(self, testdir): p1 = testdir.makepyfile(""" @@ -280,7 +335,7 @@ def test_foo(a): child = testdir.spawn_pytest(str(p1)) child.expect("x = 5") child.sendeof() - child.wait() + self.flush(child) def test_pdb_collection_failure_is_shown(self, testdir): p1 = testdir.makepyfile("""xxx """) @@ -309,5 +364,43 @@ def test_foo(): child.expect("enter_pdb_hook") child.send('c\n') child.sendeof() + self.flush(child) + + def test_pdb_custom_cls(self, testdir, custom_pdb_calls): + p1 = testdir.makepyfile("""xxx """) + result = testdir.runpytest_inprocess( + "--pdb", "--pdbcls=_pytest:_CustomPdb", p1) + result.stdout.fnmatch_lines([ + "*NameError*xxx*", + "*1 error*", + ]) + assert custom_pdb_calls == ["init", "reset", "interaction"] + + def test_pdb_custom_cls_without_pdb(self, testdir, custom_pdb_calls): + p1 = testdir.makepyfile("""xxx """) + result = testdir.runpytest_inprocess( + "--pdbcls=_pytest:_CustomPdb", p1) + result.stdout.fnmatch_lines([ + "*NameError*xxx*", + "*1 error*", + ]) + assert custom_pdb_calls == [] + + def test_pdb_custom_cls_with_settrace(self, testdir, monkeypatch): + testdir.makepyfile(custom_pdb=""" + class CustomPdb(object): + def set_trace(*args, **kwargs): + print 'custom set_trace>' + """) + p1 = testdir.makepyfile(""" + import pytest + + def test_foo(): + pytest.set_trace() + """) + monkeypatch.setenv('PYTHONPATH', str(testdir.tmpdir)) + child = testdir.spawn_pytest("--pdbcls=custom_pdb:CustomPdb %s" % str(p1)) + + child.expect('custom set_trace>') if child.isalive(): child.wait() diff --git a/testing/test_pluginmanager.py b/testing/test_pluginmanager.py index 36847638d486df1..6192176e8a8d2f7 100644 --- a/testing/test_pluginmanager.py +++ b/testing/test_pluginmanager.py @@ -1,15 +1,19 @@ +# encoding: UTF-8 +from __future__ import absolute_import, division, print_function import pytest import py import os from _pytest.config import get_config, PytestPluginManager -from _pytest.main import EXIT_NOTESTSCOLLECTED +from _pytest.main import EXIT_NOTESTSCOLLECTED, Session + @pytest.fixture def pytestpm(): return PytestPluginManager() -class TestPytestPluginInteractions: + +class TestPytestPluginInteractions(object): def test_addhooks_conftestplugin(self, testdir): testdir.makepyfile(newhooks=""" def pytest_myhook(xyz): @@ -26,9 +30,9 @@ def pytest_myhook(xyz): config = get_config() pm = config.pluginmanager pm.hook.pytest_addhooks.call_historic( - kwargs=dict(pluginmanager=config.pluginmanager)) + kwargs=dict(pluginmanager=config.pluginmanager)) config.pluginmanager._importconftest(conf) - #print(config.pluginmanager.get_plugins()) + # print(config.pluginmanager.get_plugins()) res = config.hook.pytest_myhook(xyz=10) assert res == [11] @@ -81,46 +85,50 @@ def pytest_addoption(parser): def test_configure(self, testdir): config = testdir.parseconfig() - l = [] - class A: + values = [] + + class A(object): def pytest_configure(self, config): - l.append(self) + values.append(self) config.pluginmanager.register(A()) - assert len(l) == 0 + assert len(values) == 0 config._do_configure() - assert len(l) == 1 + assert len(values) == 1 config.pluginmanager.register(A()) # leads to a configured() plugin - assert len(l) == 2 - assert l[0] != l[1] + assert len(values) == 2 + assert values[0] != values[1] config._ensure_unconfigure() config.pluginmanager.register(A()) - assert len(l) == 2 + assert len(values) == 2 def test_hook_tracing(self): pytestpm = get_config().pluginmanager # fully initialized with plugins saveindent = [] - class api1: + + class api1(object): def pytest_plugin_registered(self): saveindent.append(pytestpm.trace.root.indent) - class api2: + + class api2(object): def pytest_plugin_registered(self): saveindent.append(pytestpm.trace.root.indent) raise ValueError() - l = [] - pytestpm.trace.root.setwriter(l.append) + + values = [] + pytestpm.trace.root.setwriter(values.append) undo = pytestpm.enable_tracing() try: indent = pytestpm.trace.root.indent p = api1() pytestpm.register(p) assert pytestpm.trace.root.indent == indent - assert len(l) >= 2 - assert 'pytest_plugin_registered' in l[0] - assert 'finish' in l[1] + assert len(values) >= 2 + assert 'pytest_plugin_registered' in values[0] + assert 'finish' in values[1] - l[:] = [] + values[:] = [] with pytest.raises(ValueError): pytestpm.register(api2()) assert pytestpm.trace.root.indent == indent @@ -128,31 +136,33 @@ def pytest_plugin_registered(self): finally: undo() - def test_warn_on_deprecated_multicall(self, pytestpm): - warnings = [] - - class get_warnings: - def pytest_logwarning(self, message): - warnings.append(message) - - class Plugin: - def pytest_configure(self, __multicall__): - pass - - pytestpm.register(get_warnings()) - before = list(warnings) - pytestpm.register(Plugin()) - assert len(warnings) == len(before) + 1 - assert "deprecated" in warnings[-1] + def test_hook_proxy(self, testdir): + """Test the gethookproxy function(#2016)""" + config = testdir.parseconfig() + session = Session(config) + testdir.makepyfile(**{ + 'tests/conftest.py': '', + 'tests/subdir/conftest.py': '', + }) + + conftest1 = testdir.tmpdir.join('tests/conftest.py') + conftest2 = testdir.tmpdir.join('tests/subdir/conftest.py') + + config.pluginmanager._importconftest(conftest1) + ihook_a = session.gethookproxy(testdir.tmpdir.join('tests')) + assert ihook_a is not None + config.pluginmanager._importconftest(conftest2) + ihook_b = session.gethookproxy(testdir.tmpdir.join('tests')) + assert ihook_a is not ihook_b def test_warn_on_deprecated_addhooks(self, pytestpm): warnings = [] - class get_warnings: + class get_warnings(object): def pytest_logwarning(self, code, fslocation, message, nodeid): warnings.append(message) - class Plugin: + class Plugin(object): def pytest_testhook(): pass @@ -171,6 +181,7 @@ def test_namespace_has_default_and_env_plugins(testdir): result = testdir.runpython(p) assert result.ret == 0 + def test_default_markers(testdir): result = testdir.runpytest("--markers") result.stdout.fnmatch_lines([ @@ -179,30 +190,40 @@ def test_default_markers(testdir): ]) -def test_importplugin_issue375(testdir, pytestpm): +def test_importplugin_error_message(testdir, pytestpm): """Don't hide import errors when importing plugins and provide an easy to debug message. + + See #375 and #1998. """ testdir.syspathinsert(testdir.tmpdir) - testdir.makepyfile(qwe="import aaaa") + testdir.makepyfile(qwe=""" + # encoding: UTF-8 + def test_traceback(): + raise ImportError(u'Not possible to import: ☺') + test_traceback() + """) with pytest.raises(ImportError) as excinfo: pytestpm.import_plugin("qwe") - expected = '.*Error importing plugin "qwe": No module named \'?aaaa\'?' - assert py.std.re.match(expected, str(excinfo.value)) + + expected_message = '.*Error importing plugin "qwe": Not possible to import: .' + expected_traceback = ".*in test_traceback" + assert py.std.re.match(expected_message, str(excinfo.value)) + assert py.std.re.match(expected_traceback, str(excinfo.traceback[-1])) -class TestPytestPluginManager: +class TestPytestPluginManager(object): def test_register_imported_modules(self): pm = PytestPluginManager() mod = py.std.types.ModuleType("x.y.pytest_hello") pm.register(mod) assert pm.is_registered(mod) - l = pm.get_plugins() - assert mod in l + values = pm.get_plugins() + assert mod in values pytest.raises(ValueError, "pm.register(mod)") pytest.raises(ValueError, lambda: pm.register(mod)) - #assert not pm.is_registered(mod2) - assert pm.get_plugins() == l + # assert not pm.is_registered(mod2) + assert pm.get_plugins() == values def test_canonical_import(self, monkeypatch): mod = py.std.types.ModuleType("pytest_xyz") @@ -228,7 +249,7 @@ def test_consider_module_import_module(self, testdir): mod.pytest_plugins = "pytest_a" aplugin = testdir.makepyfile(pytest_a="#") reprec = testdir.make_hook_recorder(pytestpm) - #syspath.prepend(aplugin.dirpath()) + # syspath.prepend(aplugin.dirpath()) py.std.sys.path.insert(0, str(aplugin.dirpath())) pytestpm.consider_module(mod) call = reprec.getcall(pytestpm.hook.pytest_plugin_registered.name) @@ -236,8 +257,8 @@ def test_consider_module_import_module(self, testdir): # check that it is not registered twice pytestpm.consider_module(mod) - l = reprec.getcalls("pytest_plugin_registered") - assert len(l) == 1 + values = reprec.getcalls("pytest_plugin_registered") + assert len(values) == 1 def test_consider_env_fails_to_import(self, monkeypatch, pytestpm): monkeypatch.setenv('PYTEST_PLUGINS', 'nonexisting', prepend=",") @@ -254,8 +275,8 @@ def test_plugin_skip(self, testdir, monkeypatch): result = testdir.runpytest("-rw", "-p", "skipping1", syspathinsert=True) assert result.ret == EXIT_NOTESTSCOLLECTED result.stdout.fnmatch_lines([ - "WI1*skipped plugin*skipping1*hello*", - "WI1*skipped plugin*skipping2*hello*", + "*skipped plugin*skipping1*hello*", + "*skipped plugin*skipping2*hello*", ]) def test_consider_env_plugin_instantiation(self, testdir, monkeypatch, pytestpm): @@ -318,10 +339,10 @@ def test_consider_conftest_deps(self, testdir, pytestpm): pytestpm.consider_conftest(mod) -class TestPytestPluginManagerBootstrapming: +class TestPytestPluginManagerBootstrapming(object): def test_preparse_args(self, pytestpm): pytest.raises(ImportError, lambda: - pytestpm.consider_preparse(["xyz", "-p", "hello123"])) + pytestpm.consider_preparse(["xyz", "-p", "hello123"])) def test_plugin_prevent_register(self, pytestpm): pytestpm.consider_preparse(["xyz", "-p", "no:abc"]) diff --git a/testing/test_pytester.py b/testing/test_pytester.py index 65660afdfe36e33..9508c2954e8d1b9 100644 --- a/testing/test_pytester.py +++ b/testing/test_pytester.py @@ -1,3 +1,5 @@ +# -*- coding: utf-8 -*- +from __future__ import absolute_import, division, print_function import pytest import os from _pytest.pytester import HookRecorder @@ -11,7 +13,8 @@ def test_make_hook_recorder(testdir): assert not recorder.getfailures() pytest.xfail("internal reportrecorder tests need refactoring") - class rep: + + class rep(object): excinfo = None passed = False failed = True @@ -24,7 +27,7 @@ class rep: failures = recorder.getfailures() assert failures == [rep] - class rep: + class rep(object): excinfo = None passed = False failed = False @@ -62,6 +65,7 @@ def test_parseconfig(testdir): assert config2 != config1 assert config1 != pytest.config + def test_testdir_runs_with_plugin(testdir): testdir.makepyfile(""" pytest_plugins = "pytester" @@ -73,17 +77,21 @@ def test_hello(testdir): def make_holder(): - class apiclass: + class apiclass(object): def pytest_xyz(self, arg): "x" + def pytest_xyz_noarg(self): "x" apimod = type(os)('api') + def pytest_xyz(arg): "x" + def pytest_xyz_noarg(): "x" + apimod.pytest_xyz = pytest_xyz apimod.pytest_xyz_noarg = pytest_xyz_noarg return apiclass, apimod @@ -112,6 +120,17 @@ def test_makepyfile_unicode(testdir): unichr = chr testdir.makepyfile(unichr(0xfffd)) + +def test_makepyfile_utf8(testdir): + """Ensure makepyfile accepts utf-8 bytes as input (#2738)""" + utf8_contents = u""" + def setup_function(function): + mixed_encoding = u'São Paulo' + """.encode('utf-8') + p = testdir.makepyfile(utf8_contents) + assert u"mixed_encoding = u'São Paulo'".encode('utf-8') in p.read('rb') + + def test_inline_run_clean_modules(testdir): test_mod = testdir.makepyfile("def test_foo(): assert True") result = testdir.inline_run(str(test_mod)) @@ -120,3 +139,11 @@ def test_inline_run_clean_modules(testdir): test_mod.write("def test_foo(): assert False") result2 = testdir.inline_run(str(test_mod)) assert result2.ret == EXIT_TESTSFAILED + + +def test_assert_outcomes_after_pytest_erro(testdir): + testdir.makepyfile("def test_foo(): assert True") + + result = testdir.runpytest('--unexpected-argument') + with pytest.raises(ValueError, message="Pytest terminal report not found"): + result.assert_outcomes(passed=0) diff --git a/testing/test_recwarn.py b/testing/test_recwarn.py index 87e5846c2b0bc3d..31e70460fe466ed 100644 --- a/testing/test_recwarn.py +++ b/testing/test_recwarn.py @@ -1,5 +1,8 @@ +from __future__ import absolute_import, division, print_function import warnings +import re import py + import pytest from _pytest.recwarn import WarningsRecorder @@ -7,25 +10,19 @@ def test_recwarn_functional(testdir): reprec = testdir.inline_runsource(""" import warnings - oldwarn = warnings.showwarning def test_method(recwarn): - assert warnings.showwarning != oldwarn warnings.warn("hello") warn = recwarn.pop() assert isinstance(warn.message, UserWarning) - def test_finalized(): - assert warnings.showwarning == oldwarn """) res = reprec.countoutcomes() - assert tuple(res) == (2, 0, 0), res + assert tuple(res) == (1, 0, 0), res class TestWarningsRecorderChecker(object): - def test_recording(self, recwarn): - showwarning = py.std.warnings.showwarning + def test_recording(self): rec = WarningsRecorder() with rec: - assert py.std.warnings.showwarning != showwarning assert not rec.list py.std.warnings.warn_explicit("hello", UserWarning, "xyz", 13) assert len(rec.list) == 1 @@ -33,14 +30,12 @@ def test_recording(self, recwarn): assert len(rec.list) == 2 warn = rec.pop() assert str(warn.message) == "hello" - l = rec.list + values = rec.list rec.clear() assert len(rec.list) == 0 - assert l is rec.list + assert values is rec.list pytest.raises(AssertionError, "rec.pop()") - assert showwarning == py.std.warnings.showwarning - def test_typechecking(self): from _pytest.recwarn import WarningsChecker with pytest.raises(TypeError): @@ -81,7 +76,7 @@ def dep_explicit(self, i): def test_deprecated_call_raises(self): with pytest.raises(AssertionError) as excinfo: pytest.deprecated_call(self.dep, 3, 5) - assert str(excinfo).find("did not produce") != -1 + assert 'Did not produce' in str(excinfo) def test_deprecated_call(self): pytest.deprecated_call(self.dep, 0, 5) @@ -110,50 +105,70 @@ def test_deprecated_explicit_call(self): pytest.deprecated_call(self.dep_explicit, 0) pytest.deprecated_call(self.dep_explicit, 0) - def test_deprecated_call_as_context_manager_no_warning(self): - with pytest.raises(pytest.fail.Exception) as ex: - with pytest.deprecated_call(): - self.dep(1) - assert str(ex.value) == "DID NOT WARN" + @pytest.mark.parametrize('mode', ['context_manager', 'call']) + def test_deprecated_call_no_warning(self, mode): + """Ensure deprecated_call() raises the expected failure when its block/function does + not raise a deprecation warning. + """ + def f(): + pass - def test_deprecated_call_as_context_manager(self): - with pytest.deprecated_call(): - self.dep(0) + msg = 'Did not produce DeprecationWarning or PendingDeprecationWarning' + with pytest.raises(AssertionError, matches=msg): + if mode == 'call': + pytest.deprecated_call(f) + else: + with pytest.deprecated_call(): + f() + + @pytest.mark.parametrize('warning_type', [PendingDeprecationWarning, DeprecationWarning]) + @pytest.mark.parametrize('mode', ['context_manager', 'call']) + @pytest.mark.parametrize('call_f_first', [True, False]) + @pytest.mark.filterwarnings('ignore') + def test_deprecated_call_modes(self, warning_type, mode, call_f_first): + """Ensure deprecated_call() captures a deprecation warning as expected inside its + block/function. + """ + def f(): + warnings.warn(warning_type("hi")) + return 10 + + # ensure deprecated_call() can capture the warning even if it has already been triggered + if call_f_first: + assert f() == 10 + if mode == 'call': + assert pytest.deprecated_call(f) == 10 + else: + with pytest.deprecated_call(): + assert f() == 10 - def test_deprecated_call_pending(self): + @pytest.mark.parametrize('mode', ['context_manager', 'call']) + def test_deprecated_call_exception_is_raised(self, mode): + """If the block of the code being tested by deprecated_call() raises an exception, + it must raise the exception undisturbed. + """ def f(): - py.std.warnings.warn(PendingDeprecationWarning("hi")) - pytest.deprecated_call(f) + raise ValueError('some exception') + + with pytest.raises(ValueError, match='some exception'): + if mode == 'call': + pytest.deprecated_call(f) + else: + with pytest.deprecated_call(): + f() def test_deprecated_call_specificity(self): other_warnings = [Warning, UserWarning, SyntaxWarning, RuntimeWarning, FutureWarning, ImportWarning, UnicodeWarning] for warning in other_warnings: def f(): - py.std.warnings.warn(warning("hi")) + warnings.warn(warning("hi")) + with pytest.raises(AssertionError): pytest.deprecated_call(f) - - def test_deprecated_function_already_called(self, testdir): - """deprecated_call should be able to catch a call to a deprecated - function even if that function has already been called in the same - module. See #1190. - """ - testdir.makepyfile(""" - import warnings - import pytest - - def deprecated_function(): - warnings.warn("deprecated", DeprecationWarning) - - def test_one(): - deprecated_function() - - def test_two(): - pytest.deprecated_call(deprecated_function) - """) - result = testdir.runpytest() - result.stdout.fnmatch_lines('*=== 2 passed in *===') + with pytest.raises(AssertionError): + with pytest.deprecated_call(): + f() class TestWarns(object): @@ -185,16 +200,37 @@ def test_as_contextmanager(self): with pytest.warns(RuntimeWarning): warnings.warn("runtime", RuntimeWarning) - with pytest.raises(pytest.fail.Exception): + with pytest.warns(UserWarning): + warnings.warn("user", UserWarning) + + with pytest.raises(pytest.fail.Exception) as excinfo: with pytest.warns(RuntimeWarning): warnings.warn("user", UserWarning) + excinfo.match(r"DID NOT WARN. No warnings of type \(.+RuntimeWarning.+,\) was emitted. " + r"The list of emitted warnings is: \[UserWarning\('user',\)\].") - with pytest.raises(pytest.fail.Exception): + with pytest.raises(pytest.fail.Exception) as excinfo: with pytest.warns(UserWarning): warnings.warn("runtime", RuntimeWarning) + excinfo.match(r"DID NOT WARN. No warnings of type \(.+UserWarning.+,\) was emitted. " + r"The list of emitted warnings is: \[RuntimeWarning\('runtime',\)\].") - with pytest.warns(UserWarning): - warnings.warn("user", UserWarning) + with pytest.raises(pytest.fail.Exception) as excinfo: + with pytest.warns(UserWarning): + pass + excinfo.match(r"DID NOT WARN. No warnings of type \(.+UserWarning.+,\) was emitted. " + r"The list of emitted warnings is: \[\].") + + warning_classes = (UserWarning, FutureWarning) + with pytest.raises(pytest.fail.Exception) as excinfo: + with pytest.warns(warning_classes) as warninfo: + warnings.warn("runtime", RuntimeWarning) + warnings.warn("import", ImportWarning) + + message_template = ("DID NOT WARN. No warnings of type {0} was emitted. " + "The list of emitted warnings is: {1}.") + excinfo.match(re.escape(message_template.format(warning_classes, + [each.message for each in warninfo]))) def test_record(self): with pytest.warns(UserWarning) as record: @@ -212,6 +248,29 @@ def test_record_only(self): assert str(record[0].message) == "user" assert str(record[1].message) == "runtime" + def test_record_by_subclass(self): + with pytest.warns(Warning) as record: + warnings.warn("user", UserWarning) + warnings.warn("runtime", RuntimeWarning) + + assert len(record) == 2 + assert str(record[0].message) == "user" + assert str(record[1].message) == "runtime" + + class MyUserWarning(UserWarning): + pass + + class MyRuntimeWarning(RuntimeWarning): + pass + + with pytest.warns((UserWarning, RuntimeWarning)) as record: + warnings.warn("user", MyUserWarning) + warnings.warn("runtime", MyRuntimeWarning) + + assert len(record) == 2 + assert str(record[0].message) == "user" + assert str(record[1].message) == "runtime" + def test_double_test(self, testdir): """If a test is run again, the warning should still be raised""" testdir.makepyfile(''' @@ -225,3 +284,27 @@ def test(run): ''') result = testdir.runpytest() result.stdout.fnmatch_lines(['*2 passed in*']) + + def test_match_regex(self): + with pytest.warns(UserWarning, match=r'must be \d+$'): + warnings.warn("value must be 42", UserWarning) + + with pytest.raises(pytest.fail.Exception): + with pytest.warns(UserWarning, match=r'must be \d+$'): + warnings.warn("this is not here", UserWarning) + + with pytest.raises(pytest.fail.Exception): + with pytest.warns(FutureWarning, match=r'must be \d+$'): + warnings.warn("value must be 42", UserWarning) + + def test_one_from_multiple_warns(self): + with pytest.warns(UserWarning, match=r'aaa'): + warnings.warn("cccccccccc", UserWarning) + warnings.warn("bbbbbbbbbb", UserWarning) + warnings.warn("aaaaaaaaaa", UserWarning) + + def test_none_of_multiple_warns(self): + with pytest.raises(pytest.fail.Exception): + with pytest.warns(UserWarning, match=r'aaa'): + warnings.warn("bbbbbbbbbb", UserWarning) + warnings.warn("cccccccccc", UserWarning) diff --git a/testing/test_resultlog.py b/testing/test_resultlog.py index 74d13f643306913..b7dd2687cdfda9e 100644 --- a/testing/test_resultlog.py +++ b/testing/test_resultlog.py @@ -1,3 +1,4 @@ +from __future__ import absolute_import, division, print_function import os import _pytest._code @@ -5,7 +6,7 @@ import pytest from _pytest.main import Node, Item, FSCollector from _pytest.resultlog import generic_path, ResultLog, \ - pytest_configure, pytest_unconfigure + pytest_configure, pytest_unconfigure def test_generic_path(testdir): @@ -13,10 +14,10 @@ def test_generic_path(testdir): config = testdir.parseconfig() session = Session(config) p1 = Node('a', config=config, session=session) - #assert p1.fspath is None + # assert p1.fspath is None p2 = Node('B', parent=p1) - p3 = Node('()', parent = p2) - item = Item('c', parent = p3) + p3 = Node('()', parent=p2) + item = Item('c', parent=p3) res = generic_path(item) assert res == 'a.B().c' @@ -24,13 +25,14 @@ def test_generic_path(testdir): p0 = FSCollector('proj/test', config=config, session=session) p1 = FSCollector('proj/test/a', parent=p0) p2 = Node('B', parent=p1) - p3 = Node('()', parent = p2) + p3 = Node('()', parent=p2) p4 = Node('c', parent=p3) - item = Item('[1]', parent = p4) + item = Item('[1]', parent=p4) res = generic_path(item) assert res == 'test/a:B().c[1]' + def test_write_log_entry(): reslog = ResultLog(None, None) reslog.logfile = py.io.TextIO() @@ -67,10 +69,10 @@ def test_write_log_entry(): entry_lines = entry.splitlines() assert len(entry_lines) == 5 assert entry_lines[0] == 'F name' - assert entry_lines[1:] == [' '+line for line in longrepr.splitlines()] + assert entry_lines[1:] == [' ' + line for line in longrepr.splitlines()] -class TestWithFunctionIntegration: +class TestWithFunctionIntegration(object): # XXX (hpk) i think that the resultlog plugin should # provide a Parser object so that one can remain # ignorant regarding formatting details. @@ -83,19 +85,10 @@ def getresultlog(self, testdir, arg): def test_collection_report(self, testdir): ok = testdir.makepyfile(test_collection_ok="") - skip = testdir.makepyfile(test_collection_skip= - "import pytest ; pytest.skip('hello')") fail = testdir.makepyfile(test_collection_fail="XXX") lines = self.getresultlog(testdir, ok) assert not lines - lines = self.getresultlog(testdir, skip) - assert len(lines) == 2 - assert lines[0].startswith("S ") - assert lines[0].endswith("test_collection_skip.py") - assert lines[1].startswith(" ") - assert lines[1].endswith("test_collection_skip.py:1: Skipped: hello") - lines = self.getresultlog(testdir, fail) assert lines assert lines[0].startswith("F ") @@ -152,7 +145,7 @@ def test_internal_exception(self, style): assert entry_lines[0].startswith('! ') if style != "native": - assert os.path.basename(__file__)[:-9] in entry_lines[0] #.pyc/class + assert os.path.basename(__file__)[:-9] in entry_lines[0] # .pyc/class assert entry_lines[-1][0] == ' ' assert 'ValueError' in entry @@ -184,6 +177,7 @@ def test_xfail_norun(): "x *:test_xfail_norun", ]) + def test_makedir_for_resultlog(testdir, LineMatcher): """--resultlog should automatically create directories for the log file""" testdir.plugins.append("resultlog") @@ -231,6 +225,4 @@ def test_func(): pass """) result = testdir.runpytest("--resultlog=log") - assert result.ret == 1 - - + assert result.ret == 2 diff --git a/testing/test_runner.py b/testing/test_runner.py index 4421c5d0d29e82a..c8e2a6463a0223f 100644 --- a/testing/test_runner.py +++ b/testing/test_runner.py @@ -1,23 +1,24 @@ # -*- coding: utf-8 -*- -from __future__ import with_statement +from __future__ import absolute_import, division, print_function import _pytest._code import os import py import pytest import sys -from _pytest import runner, main +from _pytest import runner, main, outcomes -class TestSetupState: + +class TestSetupState(object): def test_setup(self, testdir): ss = runner.SetupState() item = testdir.getitem("def test_func(): pass") - l = [1] + values = [1] ss.prepare(item) - ss.addfinalizer(l.pop, colitem=item) - assert l + ss.addfinalizer(values.pop, colitem=item) + assert values ss._pop_and_teardown() - assert not l + assert not values def test_teardown_exact_stack_empty(self, testdir): item = testdir.getitem("def test_func(): pass") @@ -31,16 +32,23 @@ def test_setup_fails_and_failure_is_cached(self, testdir): def setup_module(mod): raise ValueError(42) def test_func(): pass - """) # noqa + """) # noqa ss = runner.SetupState() pytest.raises(ValueError, lambda: ss.prepare(item)) pytest.raises(ValueError, lambda: ss.prepare(item)) def test_teardown_multiple_one_fails(self, testdir): r = [] - def fin1(): r.append('fin1') - def fin2(): raise Exception('oops') - def fin3(): r.append('fin3') + + def fin1(): + r.append('fin1') + + def fin2(): + raise Exception('oops') + + def fin3(): + r.append('fin3') + item = testdir.getitem("def test_func(): pass") ss = runner.SetupState() ss.addfinalizer(fin1, item) @@ -54,8 +62,12 @@ def fin3(): r.append('fin3') def test_teardown_multiple_fail(self, testdir): # Ensure the first exception is the one which is re-raised. # Ideally both would be reported however. - def fin1(): raise Exception('oops1') - def fin2(): raise Exception('oops2') + def fin1(): + raise Exception('oops1') + + def fin2(): + raise Exception('oops2') + item = testdir.getitem("def test_func(): pass") ss = runner.SetupState() ss.addfinalizer(fin1, item) @@ -65,7 +77,7 @@ def fin2(): raise Exception('oops2') assert err.value.args == ('oops2',) -class BaseFunctionalTests: +class BaseFunctionalTests(object): def test_passfunction(self, testdir): reports = testdir.runitem(""" def test_func(): @@ -88,7 +100,7 @@ def test_func(): assert rep.failed assert rep.when == "call" assert rep.outcome == "failed" - #assert isinstance(rep.longrepr, ReprExceptionInfo) + # assert isinstance(rep.longrepr, ReprExceptionInfo) def test_skipfunction(self, testdir): reports = testdir.runitem(""" @@ -101,12 +113,12 @@ def test_func(): assert not rep.passed assert rep.skipped assert rep.outcome == "skipped" - #assert rep.skipped.when == "call" - #assert rep.skipped.when == "call" - #assert rep.skipped == "%sreason == "hello" - #assert rep.skipped.location.lineno == 3 - #assert rep.skipped.location.path - #assert not rep.skipped.failurerepr + # assert rep.skipped.when == "call" + # assert rep.skipped.when == "call" + # assert rep.skipped == "%sreason == "hello" + # assert rep.skipped.location.lineno == 3 + # assert rep.skipped.location.path + # assert not rep.skipped.failurerepr def test_skip_in_setup_function(self, testdir): reports = testdir.runitem(""" @@ -121,11 +133,11 @@ def test_func(): assert not rep.failed assert not rep.passed assert rep.skipped - #assert rep.skipped.reason == "hello" - #assert rep.skipped.location.lineno == 3 - #assert rep.skipped.location.lineno == 3 + # assert rep.skipped.reason == "hello" + # assert rep.skipped.location.lineno == 3 + # assert rep.skipped.location.lineno == 3 assert len(reports) == 2 - assert reports[1].passed # teardown + assert reports[1].passed # teardown def test_failure_in_setup_function(self, testdir): reports = testdir.runitem(""" @@ -157,8 +169,8 @@ def test_func(): assert not rep.passed assert rep.failed assert rep.when == "teardown" - #assert rep.longrepr.reprcrash.lineno == 3 - #assert rep.longrepr.reprtraceback.reprentries + # assert rep.longrepr.reprcrash.lineno == 3 + # assert rep.longrepr.reprtraceback.reprentries def test_custom_failure_repr(self, testdir): testdir.makepyfile(conftest=""" @@ -176,10 +188,10 @@ def test_func(): assert not rep.skipped assert not rep.passed assert rep.failed - #assert rep.outcome.when == "call" - #assert rep.failed.where.lineno == 3 - #assert rep.failed.where.path.basename == "test_func.py" - #assert rep.failed.failurerepr == "hello" + # assert rep.outcome.when == "call" + # assert rep.failed.where.lineno == 3 + # assert rep.failed.where.path.basename == "test_func.py" + # assert rep.failed.failurerepr == "hello" def test_teardown_final_returncode(self, testdir): rec = testdir.inline_runsource(""" @@ -194,7 +206,7 @@ def test_exact_teardown_issue90(self, testdir): rec = testdir.inline_runsource(""" import pytest - class TestClass: + class TestClass(object): def test_method(self): pass def teardown_class(cls): @@ -214,20 +226,54 @@ def teardown_function(func): raise ValueError(42) """) reps = rec.getreports("pytest_runtest_logreport") - print (reps) + print(reps) for i in range(2): assert reps[i].nodeid.endswith("test_method") assert reps[i].passed assert reps[2].when == "teardown" assert reps[2].failed assert len(reps) == 6 - for i in range(3,5): + for i in range(3, 5): assert reps[i].nodeid.endswith("test_func") assert reps[i].passed assert reps[5].when == "teardown" assert reps[5].nodeid.endswith("test_func") assert reps[5].failed + def test_exact_teardown_issue1206(self, testdir): + """issue shadowing error with wrong number of arguments on teardown_method.""" + rec = testdir.inline_runsource(""" + import pytest + + class TestClass(object): + def teardown_method(self, x, y, z): + pass + + def test_method(self): + assert True + """) + reps = rec.getreports("pytest_runtest_logreport") + print(reps) + assert len(reps) == 3 + # + assert reps[0].nodeid.endswith("test_method") + assert reps[0].passed + assert reps[0].when == 'setup' + # + assert reps[1].nodeid.endswith("test_method") + assert reps[1].passed + assert reps[1].when == 'call' + # + assert reps[2].nodeid.endswith("test_method") + assert reps[2].failed + assert reps[2].when == "teardown" + assert reps[2].longrepr.reprcrash.message in ( + # python3 error + "TypeError: teardown_method() missing 2 required positional arguments: 'y' and 'z'", + # python2 error + 'TypeError: teardown_method() takes exactly 4 arguments (2 given)' + ) + def test_failure_in_setup_function_ignores_custom_repr(self, testdir): testdir.makepyfile(conftest=""" import pytest @@ -247,10 +293,10 @@ def test_func(): assert not rep.skipped assert not rep.passed assert rep.failed - #assert rep.outcome.when == "setup" - #assert rep.outcome.where.lineno == 3 - #assert rep.outcome.where.path.basename == "test_func.py" - #assert instanace(rep.failed.failurerepr, PythonFailureRepr) + # assert rep.outcome.when == "setup" + # assert rep.outcome.where.lineno == 3 + # assert rep.outcome.where.path.basename == "test_func.py" + # assert instanace(rep.failed.failurerepr, PythonFailureRepr) def test_systemexit_does_not_bail_out(self, testdir): try: @@ -276,6 +322,7 @@ def test_func(): else: pytest.fail("did not raise") + class TestExecutionNonForked(BaseFunctionalTests): def getrunner(self): def f(item): @@ -293,6 +340,7 @@ def test_func(): else: pytest.fail("did not raise") + class TestExecutionForked(BaseFunctionalTests): pytestmark = pytest.mark.skipif("not hasattr(os, 'fork')") @@ -311,12 +359,13 @@ def test_func(): assert rep.failed assert rep.when == "???" -class TestSessionReports: + +class TestSessionReports(object): def test_collect_result(self, testdir): col = testdir.getmodulecol(""" def test_func1(): pass - class TestClass: + class TestClass(object): pass """) rep = runner.collect_one_node(col) @@ -332,18 +381,6 @@ class TestClass: assert res[0].name == "test_func1" assert res[1].name == "TestClass" - def test_skip_at_module_scope(self, testdir): - col = testdir.getmodulecol(""" - import pytest - pytest.skip("hello") - def test_func(): - pass - """) - rep = main.collect_one_node(col) - assert not rep.failed - assert not rep.passed - assert rep.skipped - reporttypes = [ runner.BaseReport, @@ -352,6 +389,7 @@ def test_func(): runner.CollectReport, ] + @pytest.mark.parametrize('reporttype', reporttypes, ids=[x.__name__ for x in reporttypes]) def test_report_extra_parameters(reporttype): if hasattr(py.std.inspect, 'signature'): @@ -362,12 +400,13 @@ def test_report_extra_parameters(reporttype): report = reporttype(newthing=1, **basekw) assert report.newthing == 1 + def test_callinfo(): ci = runner.CallInfo(lambda: 0, '123') assert ci.when == "123" assert ci.result == 0 assert "result" in repr(ci) - ci = runner.CallInfo(lambda: 0/0, '123') + ci = runner.CallInfo(lambda: 0 / 0, '123') assert ci.when == "123" assert not hasattr(ci, 'result') assert ci.excinfo @@ -375,16 +414,20 @@ def test_callinfo(): # design question: do we want general hooks in python files? # then something like the following functional tests makes sense + + @pytest.mark.xfail def test_runtest_in_module_ordering(testdir): p1 = testdir.makepyfile(""" + import pytest def pytest_runtest_setup(item): # runs after class-level! item.function.mylist.append("module") - class TestClass: + class TestClass(object): def pytest_runtest_setup(self, item): assert not hasattr(item.function, 'mylist') item.function.mylist = ['class'] - def pytest_funcarg__mylist(self, request): + @pytest.fixture + def mylist(self, request): return request.function.mylist def pytest_runtest_call(self, item, __multicall__): try: @@ -406,9 +449,18 @@ def pytest_runtest_teardown(item): def test_outcomeexception_exceptionattributes(): - outcome = runner.OutcomeException('test') + outcome = outcomes.OutcomeException('test') assert outcome.args[0] == outcome.msg + +def test_outcomeexception_passes_except_Exception(): + with pytest.raises(outcomes.OutcomeException): + try: + raise outcomes.OutcomeException('test') + except Exception: + pass + + def test_pytest_exit(): try: pytest.exit("hello") @@ -416,6 +468,7 @@ def test_pytest_exit(): excinfo = _pytest._code.ExceptionInfo() assert excinfo.errisinstance(KeyboardInterrupt) + def test_pytest_fail(): try: pytest.fail("hello") @@ -424,6 +477,20 @@ def test_pytest_fail(): s = excinfo.exconly(tryshort=True) assert s.startswith("Failed") + +def test_pytest_exit_msg(testdir): + testdir.makeconftest(""" + import pytest + + def pytest_configure(config): + pytest.exit('oh noes') + """) + result = testdir.runpytest() + result.stderr.fnmatch_lines([ + "Exit: oh noes", + ]) + + def test_pytest_fail_notrace(testdir): testdir.makepyfile(""" import pytest @@ -471,12 +538,12 @@ def test_foo(): assert 1 """) result = testdir.runpytest() - result.stdout.fnmatch_lines('*collected 1 items*') + result.stdout.fnmatch_lines('*collected 1 item*') result.stdout.fnmatch_lines('*1 passed*') assert result.ret == main.EXIT_OK result = testdir.runpytest('-k nonmatch') - result.stdout.fnmatch_lines('*collected 1 items*') + result.stdout.fnmatch_lines('*collected 1 item*') result.stdout.fnmatch_lines('*1 deselected*') assert result.ret == main.EXIT_NOTESTSCOLLECTED @@ -489,15 +556,18 @@ def test_exception_printing_skip(): s = excinfo.exconly(tryshort=True) assert s.startswith("Skipped") + def test_importorskip(monkeypatch): importorskip = pytest.importorskip + def f(): importorskip("asdlkj") + try: sys = importorskip("sys") # noqa assert sys == py.std.sys - #path = pytest.importorskip("os.path") - #assert path == py.std.os.path + # path = pytest.importorskip("os.path") + # assert path == py.std.os.path excinfo = pytest.raises(pytest.skip.Exception, f) path = py.path.local(excinfo.getrepr().reprcrash.path) # check that importorskip reports the actual call @@ -517,10 +587,12 @@ def f(): print(_pytest._code.ExceptionInfo()) pytest.fail("spurious skip") + def test_importorskip_imports_last_module_part(): ospath = pytest.importorskip("os.path") assert os.path == ospath + def test_importorskip_dev_module(monkeypatch): try: mod = py.std.types.ModuleType("mockmodule") @@ -535,6 +607,19 @@ def test_importorskip_dev_module(monkeypatch): pytest.fail("spurious skip") +def test_importorskip_module_level(testdir): + """importorskip must be able to skip entire modules when used at module level""" + testdir.makepyfile(''' + import pytest + foobarbaz = pytest.importorskip("foobarbaz") + + def test_foo(): + pass + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines(['*collected 0 items / 1 skipped*']) + + def test_pytest_cmdline_main(testdir): p = testdir.makepyfile(""" import pytest @@ -552,12 +637,14 @@ def test_hello(): def test_unicode_in_longrepr(testdir): testdir.makeconftest(""" - import py - def pytest_runtest_makereport(__multicall__): - rep = __multicall__.execute() + # -*- coding: utf-8 -*- + import pytest + @pytest.hookimpl(hookwrapper=True) + def pytest_runtest_makereport(): + outcome = yield + rep = outcome.get_result() if rep.when == "call": - rep.longrepr = py.builtin._totext("\\xc3\\xa4", "utf8") - return rep + rep.longrepr = u'ä' """) testdir.makepyfile(""" def test_out(): @@ -594,11 +681,13 @@ def test_makereport_getsource_dynamic_code(testdir, monkeypatch): """Test that exception in dynamically generated code doesn't break getting the source line.""" import inspect original_findsource = inspect.findsource + def findsource(obj, *args, **kwargs): # Can be triggered by dynamically created functions if obj.__name__ == 'foo': raise IndexError() return original_findsource(obj, *args, **kwargs) + monkeypatch.setattr(inspect, 'findsource', findsource) testdir.makepyfile(""" @@ -621,7 +710,9 @@ def test_store_except_info_on_eror(): sys.last_traceback and friends. """ # Simulate item that raises a specific exception - class ItemThatRaises: + class ItemThatRaises(object): + nodeid = 'item_that_raises' + def runtest(self): raise IndexError('TEST') try: @@ -632,3 +723,91 @@ def runtest(self): assert sys.last_type is IndexError assert sys.last_value.args[0] == 'TEST' assert sys.last_traceback + + +def test_current_test_env_var(testdir, monkeypatch): + pytest_current_test_vars = [] + monkeypatch.setattr(sys, 'pytest_current_test_vars', pytest_current_test_vars, raising=False) + testdir.makepyfile(''' + import pytest + import sys + import os + + @pytest.fixture + def fix(): + sys.pytest_current_test_vars.append(('setup', os.environ['PYTEST_CURRENT_TEST'])) + yield + sys.pytest_current_test_vars.append(('teardown', os.environ['PYTEST_CURRENT_TEST'])) + + def test(fix): + sys.pytest_current_test_vars.append(('call', os.environ['PYTEST_CURRENT_TEST'])) + ''') + result = testdir.runpytest_inprocess() + assert result.ret == 0 + test_id = 'test_current_test_env_var.py::test' + assert pytest_current_test_vars == [ + ('setup', test_id + ' (setup)'), ('call', test_id + ' (call)'), ('teardown', test_id + ' (teardown)')] + assert 'PYTEST_CURRENT_TEST' not in os.environ + + +class TestReportContents(object): + """ + Test user-level API of ``TestReport`` objects. + """ + + def getrunner(self): + return lambda item: runner.runtestprotocol(item, log=False) + + def test_longreprtext_pass(self, testdir): + reports = testdir.runitem(""" + def test_func(): + pass + """) + rep = reports[1] + assert rep.longreprtext == '' + + def test_longreprtext_failure(self, testdir): + reports = testdir.runitem(""" + def test_func(): + x = 1 + assert x == 4 + """) + rep = reports[1] + assert 'assert 1 == 4' in rep.longreprtext + + def test_captured_text(self, testdir): + reports = testdir.runitem(""" + import pytest + import sys + + @pytest.fixture + def fix(): + sys.stdout.write('setup: stdout\\n') + sys.stderr.write('setup: stderr\\n') + yield + sys.stdout.write('teardown: stdout\\n') + sys.stderr.write('teardown: stderr\\n') + assert 0 + + def test_func(fix): + sys.stdout.write('call: stdout\\n') + sys.stderr.write('call: stderr\\n') + assert 0 + """) + setup, call, teardown = reports + assert setup.capstdout == 'setup: stdout\n' + assert call.capstdout == 'setup: stdout\ncall: stdout\n' + assert teardown.capstdout == 'setup: stdout\ncall: stdout\nteardown: stdout\n' + + assert setup.capstderr == 'setup: stderr\n' + assert call.capstderr == 'setup: stderr\ncall: stderr\n' + assert teardown.capstderr == 'setup: stderr\ncall: stderr\nteardown: stderr\n' + + def test_no_captured_text(self, testdir): + reports = testdir.runitem(""" + def test_func(): + pass + """) + rep = reports[1] + assert rep.capstdout == '' + assert rep.capstderr == '' diff --git a/testing/test_runner_xunit.py b/testing/test_runner_xunit.py index f32a1311ba8e61e..fc931f86720ac39 100644 --- a/testing/test_runner_xunit.py +++ b/testing/test_runner_xunit.py @@ -1,6 +1,10 @@ -# -# test correct setup/teardowns at -# module, class, and instance level +""" + test correct setup/teardowns at + module, class, and instance level +""" +from __future__ import absolute_import, division, print_function +import pytest + def test_module_and_function_setup(testdir): reprec = testdir.inline_runsource(""" @@ -22,7 +26,7 @@ def test_modlevel(): assert modlevel[0] == 42 assert test_modlevel.answer == 17 - class TestFromClass: + class TestFromClass(object): def test_module(self): assert modlevel[0] == 42 assert not hasattr(test_modlevel, 'answer') @@ -32,22 +36,24 @@ def test_module(self): rep = reprec.matchreport("test_module") assert rep.passed + def test_module_setup_failure_no_teardown(testdir): reprec = testdir.inline_runsource(""" - l = [] + values = [] def setup_module(module): - l.append(1) + values.append(1) 0/0 def test_nothing(): pass def teardown_module(module): - l.append(2) + values.append(2) """) reprec.assertoutcome(failed=1) calls = reprec.getcalls("pytest_runtest_setup") - assert calls[0].item.module.l == [1] + assert calls[0].item.module.values == [1] + def test_setup_function_failure_no_teardown(testdir): reprec = testdir.inline_runsource(""" @@ -65,9 +71,10 @@ def test_func(): calls = reprec.getcalls("pytest_runtest_setup") assert calls[0].item.module.modlevel == [1] + def test_class_setup(testdir): reprec = testdir.inline_runsource(""" - class TestSimpleClassSetup: + class TestSimpleClassSetup(object): clslevel = [] def setup_class(cls): cls.clslevel.append(23) @@ -86,11 +93,12 @@ def test_cleanup(): assert not TestSimpleClassSetup.clslevel assert not TestInheritedClassSetupStillWorks.clslevel """) - reprec.assertoutcome(passed=1+2+1) + reprec.assertoutcome(passed=1 + 2 + 1) + def test_class_setup_failure_no_teardown(testdir): reprec = testdir.inline_runsource(""" - class TestSimpleClassSetup: + class TestSimpleClassSetup(object): clslevel = [] def setup_class(cls): 0/0 @@ -106,9 +114,10 @@ def test_cleanup(): """) reprec.assertoutcome(failed=1, passed=1) + def test_method_setup(testdir): reprec = testdir.inline_runsource(""" - class TestSetupMethod: + class TestSetupMethod(object): def setup_method(self, meth): self.methsetup = meth def teardown_method(self, meth): @@ -122,9 +131,10 @@ def test_other(self): """) reprec.assertoutcome(passed=2) + def test_method_setup_failure_no_teardown(testdir): reprec = testdir.inline_runsource(""" - class TestMethodSetup: + class TestMethodSetup(object): clslevel = [] def setup_method(self, method): self.clslevel.append(1) @@ -141,9 +151,10 @@ def test_cleanup(): """) reprec.assertoutcome(failed=1, passed=1) + def test_method_generator_setup(testdir): reprec = testdir.inline_runsource(""" - class TestSetupTeardownOnInstance: + class TestSetupTeardownOnInstance(object): def setup_class(cls): cls.classsetup = True @@ -163,6 +174,7 @@ def generated(self, value): """) reprec.assertoutcome(passed=1, failed=1) + def test_func_generator_setup(testdir): reprec = testdir.inline_runsource(""" import sys @@ -191,9 +203,10 @@ def check(): rep = reprec.matchreport("test_one", names="pytest_runtest_logreport") assert rep.passed + def test_method_setup_uses_fresh_instances(testdir): reprec = testdir.inline_runsource(""" - class TestSelfState1: + class TestSelfState1(object): memory = [] def test_hello(self): self.memory.append(self) @@ -203,6 +216,7 @@ def test_afterhello(self): """) reprec.assertoutcome(passed=2, failed=0) + def test_setup_that_skips_calledagain(testdir): p = testdir.makepyfile(""" import pytest @@ -216,6 +230,7 @@ def test_function2(): reprec = testdir.inline_run(p) reprec.assertoutcome(skipped=2) + def test_setup_fails_again_on_all_tests(testdir): p = testdir.makepyfile(""" import pytest @@ -229,12 +244,14 @@ def test_function2(): reprec = testdir.inline_run(p) reprec.assertoutcome(failed=2) + def test_setup_funcarg_setup_when_outer_scope_fails(testdir): p = testdir.makepyfile(""" import pytest def setup_module(mod): raise ValueError(42) - def pytest_funcarg__hello(request): + @pytest.fixture + def hello(request): raise ValueError("xyz43") def test_function1(hello): pass @@ -250,3 +267,53 @@ def test_function2(hello): "*2 error*" ]) assert "xyz43" not in result.stdout.str() + + +@pytest.mark.parametrize('arg', ['', 'arg']) +def test_setup_teardown_function_level_with_optional_argument(testdir, monkeypatch, arg): + """parameter to setup/teardown xunit-style functions parameter is now optional (#1728).""" + import sys + trace_setups_teardowns = [] + monkeypatch.setattr(sys, 'trace_setups_teardowns', trace_setups_teardowns, raising=False) + p = testdir.makepyfile(""" + import pytest + import sys + + trace = sys.trace_setups_teardowns.append + + def setup_module({arg}): trace('setup_module') + def teardown_module({arg}): trace('teardown_module') + + def setup_function({arg}): trace('setup_function') + def teardown_function({arg}): trace('teardown_function') + + def test_function_1(): pass + def test_function_2(): pass + + class Test(object): + def setup_method(self, {arg}): trace('setup_method') + def teardown_method(self, {arg}): trace('teardown_method') + + def test_method_1(self): pass + def test_method_2(self): pass + """.format(arg=arg)) + result = testdir.inline_run(p) + result.assertoutcome(passed=4) + + expected = [ + 'setup_module', + + 'setup_function', + 'teardown_function', + 'setup_function', + 'teardown_function', + + 'setup_method', + 'teardown_method', + + 'setup_method', + 'teardown_method', + + 'teardown_module', + ] + assert trace_setups_teardowns == expected diff --git a/testing/test_session.py b/testing/test_session.py index 76f804b4f999eb8..9ec13f523e65a0a 100644 --- a/testing/test_session.py +++ b/testing/test_session.py @@ -1,8 +1,10 @@ +from __future__ import absolute_import, division, print_function import pytest from _pytest.main import EXIT_NOTESTSCOLLECTED -class SessionTests: + +class SessionTests(object): def test_basic_testitem_events(self, testdir): tfile = testdir.makepyfile(""" def test_one(): @@ -11,7 +13,7 @@ def test_one_one(): assert 0 def test_other(): raise ValueError(23) - class TestClass: + class TestClass(object): def test_two(self, someargs): pass """) @@ -20,15 +22,18 @@ def test_two(self, someargs): assert len(skipped) == 0 assert len(passed) == 1 assert len(failed) == 3 - end = lambda x: x.nodeid.split("::")[-1] + + def end(x): + return x.nodeid.split("::")[-1] + assert end(failed[0]) == "test_one_one" assert end(failed[1]) == "test_other" itemstarted = reprec.getcalls("pytest_itemcollected") assert len(itemstarted) == 4 # XXX check for failing funcarg setup - #colreports = reprec.getcalls("pytest_collectreport") - #assert len(colreports) == 4 - #assert colreports[1].report.failed + # colreports = reprec.getcalls("pytest_collectreport") + # assert len(colreports) == 4 + # assert colreports[1].report.failed def test_nested_import_error(self, testdir): tfile = testdir.makepyfile(""" @@ -40,9 +45,9 @@ def test_this(): a = 1 """) reprec = testdir.inline_run(tfile) - l = reprec.getfailedcollections() - assert len(l) == 1 - out = l[0].longrepr.reprcrash.message + values = reprec.getfailedcollections() + assert len(values) == 1 + out = str(values[0].longrepr) assert out.find('does_not_work') != -1 def test_raises_output(self, testdir): @@ -70,9 +75,9 @@ def test_1(): def test_syntax_error_module(self, testdir): reprec = testdir.inline_runsource("this is really not python") - l = reprec.getfailedcollections() - assert len(l) == 1 - out = str(l[0].longrepr) + values = reprec.getfailedcollections() + assert len(values) == 1 + out = str(values[0].longrepr) assert out.find(str('not python')) != -1 def test_exit_first_problem(self, testdir): @@ -97,12 +102,12 @@ def test_three(): assert 0 def test_broken_repr(self, testdir): p = testdir.makepyfile(""" import pytest - class BrokenRepr1: + class BrokenRepr1(object): foo=0 def __repr__(self): raise Exception("Ha Ha fooled you, I'm a broken repr().") - class TestBrokenClass: + class TestBrokenClass(object): def test_explicit_bad_repr(self): t = BrokenRepr1() pytest.raises(Exception, 'repr(t)') @@ -116,7 +121,7 @@ def test_implicit_bad_repr1(self): passed, skipped, failed = reprec.listoutcomes() assert len(failed) == 1 out = failed[0].longrepr.reprcrash.message - assert out.find("""[Exception("Ha Ha fooled you, I'm a broken repr().") raised in repr()]""") != -1 #' + assert out.find("""[Exception("Ha Ha fooled you, I'm a broken repr().") raised in repr()]""") != -1 # ' def test_skip_file_by_conftest(self, testdir): testdir.makepyfile(conftest=""" @@ -134,19 +139,20 @@ def test_one(): pass assert len(reports) == 1 assert reports[0].skipped + class TestNewSession(SessionTests): def test_order_of_execution(self, testdir): reprec = testdir.inline_runsource(""" - l = [] + values = [] def test_1(): - l.append(1) + values.append(1) def test_2(): - l.append(2) + values.append(2) def test_3(): - assert l == [1,2] - class Testmygroup: - reslist = l + assert values == [1,2] + class Testmygroup(object): + reslist = values def test_1(self): self.reslist.append(1) def test_2(self): @@ -167,17 +173,13 @@ def test_collect_only_with_various_situations(self, testdir): def test_one(): raise ValueError() - class TestX: + class TestX(object): def test_method_one(self): pass class TestY(TestX): pass """, - test_two=""" - import pytest - pytest.skip('xxx') - """, test_three="xxxdsadsadsadsa", __init__="" ) @@ -189,11 +191,9 @@ class TestY(TestX): started = reprec.getcalls("pytest_collectstart") finished = reprec.getreports("pytest_collectreport") assert len(started) == len(finished) - assert len(started) == 8 # XXX extra TopCollector + assert len(started) == 7 # XXX extra TopCollector colfail = [x for x in finished if x.failed] - colskipped = [x for x in finished if x.skipped] assert len(colfail) == 1 - assert len(colskipped) == 1 def test_minus_x_import_error(self, testdir): testdir.makepyfile(__init__="") @@ -203,14 +203,23 @@ def test_minus_x_import_error(self, testdir): colfail = [x for x in finished if x.failed] assert len(colfail) == 1 + def test_minus_x_overridden_by_maxfail(self, testdir): + testdir.makepyfile(__init__="") + testdir.makepyfile(test_one="xxxx", test_two="yyyy", test_third="zzz") + reprec = testdir.inline_run("-x", "--maxfail=2", testdir.tmpdir) + finished = reprec.getreports("pytest_collectreport") + colfail = [x for x in finished if x.failed] + assert len(colfail) == 2 + def test_plugin_specify(testdir): pytest.raises(ImportError, """ testdir.parseconfig("-p", "nqweotexistent") """) - #pytest.raises(ImportError, + # pytest.raises(ImportError, # "config.do_configure(config)" - #) + # ) + def test_plugin_already_exists(testdir): config = testdir.parseconfig("-p", "terminal") @@ -218,6 +227,7 @@ def test_plugin_already_exists(testdir): config._do_configure() config._ensure_unconfigure() + def test_exclude(testdir): hellodir = testdir.mkdir("hello") hellodir.join("test_hello.py").write("x y syntaxerror") @@ -228,16 +238,17 @@ def test_exclude(testdir): assert result.ret == 0 result.stdout.fnmatch_lines(["*1 passed*"]) + def test_sessionfinish_with_start(testdir): testdir.makeconftest(""" import os - l = [] + values = [] def pytest_sessionstart(): - l.append(os.getcwd()) + values.append(os.getcwd()) os.chdir("..") def pytest_sessionfinish(): - assert l[0] == os.getcwd() + assert values[0] == os.getcwd() """) res = testdir.runpytest("--collect-only") diff --git a/testing/test_skipping.py b/testing/test_skipping.py index 3464974e0c27f64..978944876f35a35 100644 --- a/testing/test_skipping.py +++ b/testing/test_skipping.py @@ -1,3 +1,4 @@ +from __future__ import absolute_import, division, print_function import pytest import sys @@ -5,7 +6,7 @@ from _pytest.runner import runtestprotocol -class TestEvaluator: +class TestEvaluator(object): def test_no_marker(self, testdir): item = testdir.getitem("def test_func(): pass") evalskipif = MarkEvaluator(item, 'skipif') @@ -79,7 +80,7 @@ def test_marked_one_arg_twice(self, testdir): %s def test_func(): pass - """ % (lines[i], lines[(i+1) %2])) + """ % (lines[i], lines[(i + 1) % 2])) ev = MarkEvaluator(item, 'skipif') assert ev assert ev.istrue() @@ -114,7 +115,7 @@ def test_func(): def test_skipif_class(self, testdir): item, = testdir.getitems(""" import pytest - class TestClass: + class TestClass(object): pytestmark = pytest.mark.skipif("config._hackxyz") def test_func(self): pass @@ -126,7 +127,7 @@ def test_func(self): assert expl == "condition: config._hackxyz" -class TestXFail: +class TestXFail(object): @pytest.mark.parametrize('strict', [True, False]) def test_xfail_simple(self, testdir, strict): @@ -145,7 +146,20 @@ def test_func(): def test_xfail_xpassed(self, testdir): item = testdir.getitem(""" import pytest - @pytest.mark.xfail + @pytest.mark.xfail(reason="this is an xfail") + def test_func(): + assert 1 + """) + reports = runtestprotocol(item, log=False) + assert len(reports) == 3 + callreport = reports[1] + assert callreport.passed + assert callreport.wasxfail == "this is an xfail" + + def test_xfail_xpassed_strict(self, testdir): + item = testdir.getitem(""" + import pytest + @pytest.mark.xfail(strict=True, reason="nope") def test_func(): assert 1 """) @@ -153,7 +167,8 @@ def test_func(): assert len(reports) == 3 callreport = reports[1] assert callreport.failed - assert callreport.wasxfail == "" + assert callreport.longrepr == "[XPASS(strict)] nope" + assert not hasattr(callreport, "wasxfail") def test_xfail_run_anyway(self, testdir): testdir.makepyfile(""" @@ -192,9 +207,9 @@ def test_this(): assert 0 """) testdir.runpytest(p, '-v') - #result.stdout.fnmatch_lines([ + # result.stdout.fnmatch_lines([ # "*HINT*use*-r*" - #]) + # ]) def test_xfail_not_run_xfail_reporting(self, testdir): p = testdir.makepyfile(test_one=""" @@ -209,7 +224,7 @@ def test_this_true(): def test_this_false(): assert 1 """) - result = testdir.runpytest(p, '--report=xfailed', ) + result = testdir.runpytest(p, '-rx', ) result.stdout.fnmatch_lines([ "*test_one*test_this*", "*NOTRUN*noway", @@ -227,7 +242,7 @@ def test_this(): def setup_module(mod): raise ValueError(42) """) - result = testdir.runpytest(p, '--report=xfailed', ) + result = testdir.runpytest(p, '-rx', ) result.stdout.fnmatch_lines([ "*test_one*test_this*", "*NOTRUN*hello", @@ -309,7 +324,8 @@ def test_that(): def test_dynamic_xfail_no_run(self, testdir): p = testdir.makepyfile(""" import pytest - def pytest_funcarg__arg(request): + @pytest.fixture + def arg(request): request.applymarker(pytest.mark.xfail(run=False)) def test_this(arg): assert 0 @@ -323,7 +339,8 @@ def test_this(arg): def test_dynamic_xfail_set_during_funcarg_setup(self, testdir): p = testdir.makepyfile(""" import pytest - def pytest_funcarg__arg(request): + @pytest.fixture + def arg(request): request.applymarker(pytest.mark.xfail) def test_this2(arg): assert 0 @@ -333,7 +350,6 @@ def test_this2(arg): "*1 xfailed*", ]) - @pytest.mark.parametrize('expected, actual, matchline', [('TypeError', 'TypeError', "*1 xfailed*"), ('(AttributeError, TypeError)', 'TypeError', "*1 xfailed*"), @@ -405,6 +421,19 @@ def test_foo(): result.stdout.fnmatch_lines('*1 passed*') assert result.ret == 0 + @pytest.mark.parametrize('strict', [True, False]) + def test_xfail_condition_keyword(self, testdir, strict): + p = testdir.makepyfile(""" + import pytest + + @pytest.mark.xfail(condition=False, reason='unsupported feature', strict=%s) + def test_foo(): + pass + """ % strict) + result = testdir.runpytest(p, '-rxX') + result.stdout.fnmatch_lines('*1 passed*') + assert result.ret == 0 + @pytest.mark.parametrize('strict_val', ['true', 'false']) def test_strict_xfail_default_from_file(self, testdir, strict_val): testdir.makeini(''' @@ -423,7 +452,7 @@ def test_foo(): assert result.ret == (1 if strict else 0) -class TestXFailwithSetupTeardown: +class TestXFailwithSetupTeardown(object): def test_failing_setup_issue9(self, testdir): testdir.makepyfile(""" import pytest @@ -455,7 +484,7 @@ def test_func(): ]) -class TestSkip: +class TestSkip(object): def test_skip_class(self, testdir): testdir.makepyfile(""" import pytest @@ -539,14 +568,28 @@ def test_baz(): "*1 passed*2 skipped*", ]) -class TestSkipif: + def test_strict_and_skip(self, testdir): + testdir.makepyfile(""" + import pytest + @pytest.mark.skip + def test_hello(): + pass + """) + result = testdir.runpytest("-rs") + result.stdout.fnmatch_lines([ + "*unconditional skip*", + "*1 skipped*", + ]) + + +class TestSkipif(object): def test_skipif_conditional(self, testdir): item = testdir.getitem(""" import pytest @pytest.mark.skipif("hasattr(os, 'sep')") def test_func(): pass - """) # noqa + """) # noqa x = pytest.raises(pytest.skip.Exception, lambda: pytest_runtest_setup(item)) assert x.value.msg == "condition: hasattr(os, 'sep')" @@ -597,7 +640,7 @@ def test_this(): """) result = testdir.runpytest(p, '-v') result.stdout.fnmatch_lines([ - #"*HINT*use*-r*", + # "*HINT*use*-r*", "*1 skipped*", ]) @@ -606,7 +649,7 @@ def test_skipif_class(testdir): p = testdir.makepyfile(""" import pytest - class TestClass: + class TestClass(object): pytestmark = pytest.mark.skipif("True") def test_that(self): assert 0 @@ -620,12 +663,12 @@ def test_though(self): def test_skip_reasons_folding(): - path = 'xyz' + path = "xyz" lineno = 3 message = "justso" longrepr = (path, lineno, message) - class X: + class X(object): pass ev1 = X() ev1.when = "execute" @@ -633,17 +676,24 @@ class X: ev1.longrepr = longrepr ev2 = X() + ev2.when = "execute" ev2.longrepr = longrepr ev2.skipped = True - l = folded_skips([ev1, ev2]) - assert len(l) == 1 - num, fspath, lineno, reason = l[0] - assert num == 2 + # ev3 might be a collection report + ev3 = X() + ev3.longrepr = longrepr + ev3.skipped = True + + values = folded_skips([ev1, ev2, ev3]) + assert len(values) == 1 + num, fspath, lineno, reason = values[0] + assert num == 3 assert fspath == path assert lineno == lineno assert reason == message + def test_skipped_reasons_functional(testdir): testdir.makepyfile( test_one=""" @@ -652,26 +702,44 @@ def setup_function(func): doskip() def test_func(): pass - class TestClass: + class TestClass(object): def test_method(self): doskip() """, - test_two = """ - from conftest import doskip - doskip() - """, - conftest = """ + conftest=""" import pytest def doskip(): pytest.skip('test') """ ) - result = testdir.runpytest('--report=skipped') + result = testdir.runpytest('-rs') result.stdout.fnmatch_lines([ - "*SKIP*3*conftest.py:3: test", + "*SKIP*2*conftest.py:4: test", ]) assert result.ret == 0 + +def test_skipped_folding(testdir): + testdir.makepyfile( + test_one=""" + import pytest + pytestmark = pytest.mark.skip("Folding") + def setup_function(func): + pass + def test_func(): + pass + class TestClass(object): + def test_method(self): + pass + """, + ) + result = testdir.runpytest('-rs') + result.stdout.fnmatch_lines([ + "*SKIP*2*test_one.py: Folding" + ]) + assert result.ret == 0 + + def test_reportchars(testdir): testdir.makepyfile(""" import pytest @@ -694,6 +762,7 @@ def test_4(): "SKIP*four*", ]) + def test_reportchars_error(testdir): testdir.makepyfile( conftest=""" @@ -709,6 +778,7 @@ def test_foo(): 'ERROR*test_foo*', ]) + def test_reportchars_all(testdir): testdir.makepyfile(""" import pytest @@ -731,6 +801,7 @@ def test_4(): "XPASS*test_3*", ]) + def test_reportchars_all_error(testdir): testdir.makepyfile( conftest=""" @@ -746,6 +817,7 @@ def test_foo(): 'ERROR*test_foo*', ]) + @pytest.mark.xfail("hasattr(sys, 'pypy_version_info')") def test_errors_in_xfail_skip_expressions(testdir): testdir.makepyfile(""" @@ -777,6 +849,7 @@ def test_func(): "*1 pass*2 error*", ]) + def test_xfail_skipif_with_globals(testdir): testdir.makepyfile(""" import pytest @@ -795,6 +868,7 @@ def test_boolean(): "*x == 3*", ]) + def test_direct_gives_error(testdir): testdir.makepyfile(""" import pytest @@ -812,9 +886,10 @@ def test_default_markers(testdir): result = testdir.runpytest("--markers") result.stdout.fnmatch_lines([ "*skipif(*condition)*skip*", - "*xfail(*condition, reason=None, run=True, raises=None)*expected failure*", + "*xfail(*condition, reason=None, run=True, raises=None, strict=False)*expected failure*", ]) + def test_xfail_test_setup_exception(testdir): testdir.makeconftest(""" def pytest_runtest_setup(): @@ -831,6 +906,7 @@ def test_func(): assert 'xfailed' in result.stdout.str() assert 'xpassed' not in result.stdout.str() + def test_imperativeskip_on_xfail_test(testdir): testdir.makepyfile(""" import pytest @@ -854,7 +930,8 @@ def pytest_runtest_setup(item): *2 skipped* """) -class TestBooleanCondition: + +class TestBooleanCondition(object): def test_skipif(self, testdir): testdir.makepyfile(""" import pytest @@ -915,3 +992,76 @@ def pytest_collect_file(path, parent): assert not failed xfailed = [r for r in skipped if hasattr(r, 'wasxfail')] assert xfailed + + +def test_module_level_skip_error(testdir): + """ + Verify that using pytest.skip at module level causes a collection error + """ + testdir.makepyfile(""" + import pytest + @pytest.skip + def test_func(): + assert True + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines( + "*Using pytest.skip outside of a test is not allowed*" + ) + + +def test_module_level_skip_with_allow_module_level(testdir): + """ + Verify that using pytest.skip(allow_module_level=True) is allowed + """ + testdir.makepyfile(""" + import pytest + pytest.skip("skip_module_level", allow_module_level=True) + + def test_func(): + assert 0 + """) + result = testdir.runpytest("-rxs") + result.stdout.fnmatch_lines( + "*SKIP*skip_module_level" + ) + + +def test_invalid_skip_keyword_parameter(testdir): + """ + Verify that using pytest.skip() with unknown parameter raises an error + """ + testdir.makepyfile(""" + import pytest + pytest.skip("skip_module_level", unknown=1) + + def test_func(): + assert 0 + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines( + "*TypeError:*['unknown']*" + ) + + +def test_mark_xfail_item(testdir): + # Ensure pytest.mark.xfail works with non-Python Item + testdir.makeconftest(""" + import pytest + + class MyItem(pytest.Item): + nodeid = 'foo' + def setup(self): + marker = pytest.mark.xfail(True, reason="Expected failure") + self.add_marker(marker) + def runtest(self): + assert False + + def pytest_collect_file(path, parent): + return MyItem("foo", parent) + """) + result = testdir.inline_run() + passed, skipped, failed = result.listoutcomes() + assert not failed + xfailed = [r for r in skipped if hasattr(r, 'wasxfail')] + assert xfailed diff --git a/testing/test_terminal.py b/testing/test_terminal.py index b43d6b37923cd37..97c2f71fb66a27a 100644 --- a/testing/test_terminal.py +++ b/testing/test_terminal.py @@ -1,39 +1,36 @@ """ terminal reporting of the full testing process. """ +from __future__ import absolute_import, division, print_function import collections import sys -import _pytest._pluggy as pluggy +import pluggy import _pytest._code import py import pytest -from _pytest import runner from _pytest.main import EXIT_NOTESTSCOLLECTED from _pytest.terminal import TerminalReporter, repr_pythonversion, getreportopt from _pytest.terminal import build_summary_stats_line, _plugin_nameversions -def basic_run_report(item): - runner.call_and_report(item, "setup", log=False) - return runner.call_and_report(item, "call", log=False) - DistInfo = collections.namedtuple('DistInfo', ['project_name', 'version']) -class Option: +class Option(object): def __init__(self, verbose=False, fulltrace=False): self.verbose = verbose self.fulltrace = fulltrace @property def args(self): - l = [] + values = [] if self.verbose: - l.append('-v') + values.append('-v') if self.fulltrace: - l.append('--fulltrace') - return l + values.append('--fulltrace') + return values + def pytest_generate_tests(metafunc): if "option" in metafunc.fixturenames: @@ -42,7 +39,7 @@ def pytest_generate_tests(metafunc): metafunc.addcall(id="verbose", funcargs={'option': Option(verbose=True)}) metafunc.addcall(id="quiet", - funcargs={'option': Option(verbose= -1)}) + funcargs={'option': Option(verbose=-1)}) metafunc.addcall(id="fulltrace", funcargs={'option': Option(fulltrace=True)}) @@ -61,7 +58,7 @@ def test_plugin_nameversion(input, expected): assert result == expected -class TestTerminal: +class TestTerminal(object): def test_pass_skip_fail(self, testdir, option): testdir.makepyfile(""" import pytest @@ -81,8 +78,8 @@ def test_func(): ]) else: result.stdout.fnmatch_lines([ - "*test_pass_skip_fail.py .sF" - ]) + "*test_pass_skip_fail.py .sF*" + ]) result.stdout.fnmatch_lines([ " def test_func():", "> assert 0", @@ -114,7 +111,7 @@ def test_show_runtest_logstart(self, testdir, linecomp): item.config.pluginmanager.register(tr) location = item.reportinfo() tr.config.hook.pytest_runtest_logstart(nodeid=item.nodeid, - location=location, fspath=str(item.fspath)) + location=location, fspath=str(item.fspath)) linecomp.assert_contains_lines([ "*test_show_runtest_logstart.py*" ]) @@ -132,7 +129,7 @@ def test_1(): def test_itemreport_subclasses_show_subclassed_file(self, testdir): testdir.makepyfile(test_p1=""" - class BaseTests: + class BaseTests(object): def test_p1(self): pass class TestClass(BaseTests): @@ -145,18 +142,18 @@ class TestMore(BaseTests): """) result = testdir.runpytest(p2) result.stdout.fnmatch_lines([ - "*test_p2.py .", + "*test_p2.py .*", "*1 passed*", ]) result = testdir.runpytest("-v", p2) result.stdout.fnmatch_lines([ - "*test_p2.py::TestMore::test_p1* <- *test_p1.py*PASSED", + "*test_p2.py::TestMore::test_p1* <- *test_p1.py*PASSED*", ]) def test_itemreport_directclasses_not_shown_as_subclasses(self, testdir): a = testdir.mkpydir("a123") a.join("test_hello123.py").write(_pytest._code.Source(""" - class TestClass: + class TestClass(object): def test_method(self): pass """)) @@ -208,8 +205,27 @@ def test_foobar(): assert result.ret == 2 result.stdout.fnmatch_lines(['*KeyboardInterrupt*']) + def test_collect_single_item(self, testdir): + """Use singular 'item' when reporting a single test item""" + testdir.makepyfile(""" + def test_foobar(): + pass + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines(['collected 1 item']) + + def test_rewrite(self, testdir, monkeypatch): + config = testdir.parseconfig() + f = py.io.TextIO() + monkeypatch.setattr(f, 'isatty', lambda *args: True) + tr = TerminalReporter(config, f) + tr._tw.fullwidth = 10 + tr.write('hello') + tr.rewrite('hey', erase=True) + assert f.getvalue() == 'hello' + '\r' + 'hey' + (6 * ' ') + -class TestCollectonly: +class TestCollectonly(object): def test_collectonly_basic(self, testdir): testdir.makepyfile(""" def test_func(): @@ -217,8 +233,8 @@ def test_func(): """) result = testdir.runpytest("--collect-only",) result.stdout.fnmatch_lines([ - "", - " ", + "", + " ", ]) def test_collectonly_skipped_module(self, testdir): @@ -228,8 +244,7 @@ def test_collectonly_skipped_module(self, testdir): """) result = testdir.runpytest("--collect-only", "-rs") result.stdout.fnmatch_lines([ - "SKIP*hello*", - "*1 skip*", + "*ERROR collecting*", ]) def test_collectonly_failed_module(self, testdir): @@ -255,29 +270,29 @@ def test_collectonly_simple(self, testdir): p = testdir.makepyfile(""" def test_func1(): pass - class TestClass: + class TestClass(object): def test_method(self): pass """) result = testdir.runpytest("--collect-only", p) - #assert stderr.startswith("inserting into sys.path") + # assert stderr.startswith("inserting into sys.path") assert result.ret == 0 result.stdout.fnmatch_lines([ "*", "* ", "* ", - #"* ", + # "* ", "* ", ]) def test_collectonly_error(self, testdir): p = testdir.makepyfile("import Errlkjqweqwe") result = testdir.runpytest("--collect-only", p) - assert result.ret == 1 + assert result.ret == 2 result.stdout.fnmatch_lines(_pytest._code.Source(""" *ERROR* - *import Errlk* *ImportError* + *No module named *Errlk* *1 error* """).strip()) @@ -314,9 +329,10 @@ def test_repr_python_version(monkeypatch): py.std.sys.version_info = x = (2, 3) assert repr_pythonversion() == str(x) finally: - monkeypatch.undo() # do this early as pytest can get confused + monkeypatch.undo() # do this early as pytest can get confused + -class TestFixtureReporting: +class TestFixtureReporting(object): def test_setup_fixture_error(self, testdir): testdir.makepyfile(""" def setup_function(function): @@ -374,9 +390,35 @@ def teardown_function(function): "*def test_fail():", "*failingfunc*", "*1 failed*1 error*", - ]) + ]) -class TestTerminalFunctional: + def test_setup_teardown_output_and_test_failure(self, testdir): + """ Test for issue #442 """ + testdir.makepyfile(""" + def setup_function(function): + print ("setup func") + + def test_fail(): + assert 0, "failingfunc" + + def teardown_function(function): + print ("teardown func") + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + "*test_fail*", + "*def test_fail():", + "*failingfunc*", + "*Captured stdout setup*", + "*setup func*", + "*Captured stdout teardown*", + "*teardown func*", + + "*1 failed*", + ]) + + +class TestTerminalFunctional(object): def test_deselected(self, testdir): testpath = testdir.makepyfile(""" def test_one(): @@ -386,11 +428,11 @@ def test_two(): def test_three(): pass """ - ) + ) result = testdir.runpytest("-k", "test_two:", testpath) result.stdout.fnmatch_lines([ - "*test_deselected.py ..", - "=* 1 test*deselected by*test_two:*=", + "*test_deselected.py ..*", + "=* 1 test*deselected *=", ]) assert result.ret == 0 @@ -412,7 +454,7 @@ def test_passes(self, testdir): p1 = testdir.makepyfile(""" def test_passes(): pass - class TestClass: + class TestClass(object): def test_method(self): pass """) @@ -422,7 +464,7 @@ def test_method(self): finally: old.chdir() result.stdout.fnmatch_lines([ - "test_passes.py ..", + "test_passes.py ..*", "* 2 pass*", ]) assert result.ret == 0 @@ -439,7 +481,7 @@ def test_passes(): "platform %s -- Python %s*pytest-%s*py-%s*pluggy-%s" % ( py.std.sys.platform, verinfo, pytest.__version__, py.__version__, pluggy.__version__), - "*test_header_trailer_info.py .", + "*test_header_trailer_info.py .*", "=* 1 passed*in *.[0-9][0-9] seconds *=", ]) if pytest.config.pluginmanager.list_plugin_distinfo(): @@ -456,7 +498,7 @@ def test_showlocals(): """) result = testdir.runpytest(p1, '-l') result.stdout.fnmatch_lines([ - #"_ _ * Locals *", + # "_ _ * Locals *", "x* = 3", "y* = 'xxxxxx*" ]) @@ -468,7 +510,7 @@ def test_fail(): raise ValueError() def test_pass(): pass - class TestClass: + class TestClass(object): def test_skip(self): pytest.skip("hello") def test_gen(): @@ -512,6 +554,23 @@ def test_more_quiet_reporting(self, testdir): assert "===" not in s assert "passed" not in s + def test_report_collectionfinish_hook(self, testdir): + testdir.makeconftest(""" + def pytest_report_collectionfinish(config, startdir, items): + return ['hello from hook: {0} items'.format(len(items))] + """) + testdir.makepyfile(""" + import pytest + @pytest.mark.parametrize('i', range(3)) + def test(i): + pass + """) + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + "collected 3 items", + "hello from hook: 3 items", + ]) + def test_fail_extra_reporting(testdir): testdir.makepyfile("def test_this(): assert 0") @@ -523,11 +582,13 @@ def test_fail_extra_reporting(testdir): "FAIL*test_fail_extra_reporting*", ]) + def test_fail_reporting_on_pass(testdir): testdir.makepyfile("def test_this(): assert 1") result = testdir.runpytest('-rf') assert 'short test summary' not in result.stdout.str() + def test_pass_extra_reporting(testdir): testdir.makepyfile("def test_this(): assert 1") result = testdir.runpytest() @@ -538,11 +599,13 @@ def test_pass_extra_reporting(testdir): "PASS*test_pass_extra_reporting*", ]) + def test_pass_reporting_on_fail(testdir): testdir.makepyfile("def test_this(): assert 0") result = testdir.runpytest('-rp') assert 'short test summary' not in result.stdout.str() + def test_pass_output_reporting(testdir): testdir.makepyfile(""" def test_pass_output(): @@ -555,6 +618,7 @@ def test_pass_output(): "Four score and seven years ago...", ]) + def test_color_yes(testdir): testdir.makepyfile("def test_this(): assert 1") result = testdir.runpytest('--color=yes') @@ -593,29 +657,33 @@ def test_this(i): def test_getreportopt(): - class config: - class option: + class config(object): + class option(object): reportchars = "" - config.option.report = "xfailed" - assert getreportopt(config) == "x" - - config.option.report = "xfailed,skipped" - assert getreportopt(config) == "xs" - - config.option.report = "skipped,xfailed" - assert getreportopt(config) == "sx" + disable_warnings = True - config.option.report = "skipped" config.option.reportchars = "sf" assert getreportopt(config) == "sf" - config.option.reportchars = "sfx" + config.option.reportchars = "sfxw" assert getreportopt(config) == "sfx" + config.option.reportchars = "sfx" + config.option.disable_warnings = False + assert getreportopt(config) == "sfxw" + + config.option.reportchars = "sfxw" + config.option.disable_warnings = False + assert getreportopt(config) == "sfxw" + + def test_terminalreporter_reportopt_addopts(testdir): testdir.makeini("[pytest]\naddopts=-rs") testdir.makepyfile(""" - def pytest_funcarg__tr(request): + import pytest + + @pytest.fixture + def tr(request): tr = request.config.pluginmanager.getplugin("terminalreporter") return tr def test_opt(tr): @@ -627,9 +695,13 @@ def test_opt(tr): "*1 passed*" ]) + def test_tbstyle_short(testdir): p = testdir.makepyfile(""" - def pytest_funcarg__arg(request): + import pytest + + @pytest.fixture + def arg(request): return 42 def test_opt(arg): x = 0 @@ -640,7 +712,7 @@ def test_opt(arg): assert 'arg = 42' not in s assert 'x = 0' not in s result.stdout.fnmatch_lines([ - "*%s:5*" % p.basename, + "*%s:8*" % p.basename, " assert x", "E assert*", ]) @@ -649,6 +721,7 @@ def test_opt(arg): assert 'x = 0' in s assert 'assert x' in s + def test_traceconfig(testdir, monkeypatch): result = testdir.runpytest("--traceconfig") result.stdout.fnmatch_lines([ @@ -657,16 +730,17 @@ def test_traceconfig(testdir, monkeypatch): assert result.ret == EXIT_NOTESTSCOLLECTED -class TestGenericReporting: +class TestGenericReporting(object): """ this test class can be subclassed with a different option provider to run e.g. distributed tests. """ + def test_collect_fail(self, testdir, option): testdir.makepyfile("import xyz\n") result = testdir.runpytest(*option.args) result.stdout.fnmatch_lines([ - "? import xyz", - "E ImportError: No module named *xyz*", + "ImportError while importing*", + "*No module named *xyz*", "*1 error*", ]) @@ -683,11 +757,9 @@ def test_3(): result.stdout.fnmatch_lines([ "*def test_1():*", "*def test_2():*", - "*!! Interrupted: stopping after 2 failures*!!*", "*2 failed*", ]) - def test_tb_option(self, testdir, option): testdir.makepyfile(""" import pytest @@ -751,6 +823,7 @@ def pytest_report_header(config, startdir): str(testdir.tmpdir), ]) + @pytest.mark.xfail("not hasattr(os, 'dup')") def test_fdopen_kept_alive_issue124(testdir): testdir.makepyfile(""" @@ -769,6 +842,7 @@ def test_close_kept_alive_file(): "*2 passed*" ]) + def test_tbstyle_native_setup_error(testdir): testdir.makepyfile(""" import pytest @@ -781,20 +855,23 @@ def test_error_fixture(setup_error_fixture): """) result = testdir.runpytest("--tb=native") result.stdout.fnmatch_lines([ - '*File *test_tbstyle_native_setup_error.py", line *, in setup_error_fixture*' - ]) + '*File *test_tbstyle_native_setup_error.py", line *, in setup_error_fixture*' + ]) + def test_terminal_summary(testdir): testdir.makeconftest(""" - def pytest_terminal_summary(terminalreporter): + def pytest_terminal_summary(terminalreporter, exitstatus): w = terminalreporter w.section("hello") w.line("world") + w.line("exitstatus: {0}".format(exitstatus)) """) result = testdir.runpytest() result.stdout.fnmatch_lines(""" *==== hello ====* world + exitstatus: 5 """) @@ -809,8 +886,8 @@ def pytest_terminal_summary(terminalreporter): """) result = testdir.runpytest('-rw') result.stdout.fnmatch_lines([ - '*C1*internal warning', - '*== 1 pytest-warnings in *', + '*internal warning', + '*== 1 warnings in *', ]) @@ -830,11 +907,11 @@ def pytest_terminal_summary(terminalreporter): ("yellow", "1 weird", {"weird": (1,)}), ("yellow", "1 passed, 1 weird", {"weird": (1,), "passed": (1,)}), - ("yellow", "1 pytest-warnings", {"warnings": (1,)}), - ("yellow", "1 passed, 1 pytest-warnings", {"warnings": (1,), - "passed": (1,)}), + ("yellow", "1 warnings", {"warnings": (1,)}), + ("yellow", "1 passed, 1 warnings", {"warnings": (1,), + "passed": (1,)}), - ("green", "5 passed", {"passed": (1,2,3,4,5)}), + ("green", "5 passed", {"passed": (1, 2, 3, 4, 5)}), # "Boring" statuses. These have no effect on the color of the summary @@ -863,13 +940,13 @@ def pytest_terminal_summary(terminalreporter): # A couple more complex combinations ("red", "1 failed, 2 passed, 3 xfailed", - {"passed": (1,2), "failed": (1,), "xfailed": (1,2,3)}), + {"passed": (1, 2), "failed": (1,), "xfailed": (1, 2, 3)}), ("green", "1 passed, 2 skipped, 3 deselected, 2 xfailed", {"passed": (1,), - "skipped": (1,2), - "deselected": (1,2,3), - "xfailed": (1,2)}), + "skipped": (1, 2), + "deselected": (1, 2, 3), + "xfailed": (1, 2)}), ]) def test_summary_stats(exp_line, exp_color, stats_arg): print("Based on stats: %s" % stats_arg) @@ -878,3 +955,85 @@ def test_summary_stats(exp_line, exp_color, stats_arg): print("Actually got: \"%s\"; with color \"%s\"" % (line, color)) assert line == exp_line assert color == exp_color + + +def test_no_trailing_whitespace_after_inifile_word(testdir): + result = testdir.runpytest('') + assert 'inifile:\n' in result.stdout.str() + + testdir.makeini('[pytest]') + result = testdir.runpytest('') + assert 'inifile: tox.ini\n' in result.stdout.str() + + +class TestProgress: + + @pytest.fixture + def many_tests_file(self, testdir): + testdir.makepyfile( + test_bar=""" + import pytest + @pytest.mark.parametrize('i', range(10)) + def test_bar(i): pass + """, + test_foo=""" + import pytest + @pytest.mark.parametrize('i', range(5)) + def test_foo(i): pass + """, + test_foobar=""" + import pytest + @pytest.mark.parametrize('i', range(5)) + def test_foobar(i): pass + """, + ) + + def test_zero_tests_collected(self, testdir): + """Some plugins (testmon for example) might issue pytest_runtest_logreport without any tests being + actually collected (#2971).""" + testdir.makeconftest(""" + def pytest_collection_modifyitems(items, config): + from _pytest.runner import CollectReport + for node_id in ('nodeid1', 'nodeid2'): + rep = CollectReport(node_id, 'passed', None, None) + rep.when = 'passed' + rep.duration = 0.1 + config.hook.pytest_runtest_logreport(report=rep) + """) + output = testdir.runpytest() + assert 'ZeroDivisionError' not in output.stdout.str() + output.stdout.fnmatch_lines([ + '=* 2 passed in *=', + ]) + + def test_normal(self, many_tests_file, testdir): + output = testdir.runpytest() + output.stdout.re_match_lines([ + r'test_bar.py \.{10} \s+ \[ 50%\]', + r'test_foo.py \.{5} \s+ \[ 75%\]', + r'test_foobar.py \.{5} \s+ \[100%\]', + ]) + + def test_verbose(self, many_tests_file, testdir): + output = testdir.runpytest('-v') + output.stdout.re_match_lines([ + r'test_bar.py::test_bar\[0\] PASSED \s+ \[ 5%\]', + r'test_foo.py::test_foo\[4\] PASSED \s+ \[ 75%\]', + r'test_foobar.py::test_foobar\[4\] PASSED \s+ \[100%\]', + ]) + + def test_xdist_normal(self, many_tests_file, testdir): + pytest.importorskip('xdist') + output = testdir.runpytest('-n2') + output.stdout.re_match_lines([ + r'\.{20} \s+ \[100%\]', + ]) + + def test_xdist_verbose(self, many_tests_file, testdir): + pytest.importorskip('xdist') + output = testdir.runpytest('-n2', '-v') + output.stdout.re_match_lines_random([ + r'\[gw\d\] \[\s*\d+%\] PASSED test_bar.py::test_bar\[1\]', + r'\[gw\d\] \[\s*\d+%\] PASSED test_foo.py::test_foo\[1\]', + r'\[gw\d\] \[\s*\d+%\] PASSED test_foobar.py::test_foobar\[1\]', + ]) diff --git a/testing/test_tmpdir.py b/testing/test_tmpdir.py index d514e722e1f3dfa..467e77252e781e9 100644 --- a/testing/test_tmpdir.py +++ b/testing/test_tmpdir.py @@ -1,9 +1,11 @@ +from __future__ import absolute_import, division, print_function import sys import py import pytest from _pytest.tmpdir import tmpdir + def test_funcarg(testdir): testdir.makepyfile(""" def pytest_generate_tests(metafunc): @@ -28,14 +30,15 @@ def test_func(tmpdir): pass bn = p.basename.strip("0123456789") assert bn == "qwe__abc" + def test_ensuretemp(recwarn): - #pytest.deprecated_call(pytest.ensuretemp, 'hello') d1 = pytest.ensuretemp('hello') d2 = pytest.ensuretemp('hello') assert d1 == d2 assert d1.check(dir=1) -class TestTempdirHandler: + +class TestTempdirHandler(object): def test_mktemp(self, testdir): from _pytest.tmpdir import TempdirFactory config = testdir.parseconfig() @@ -49,7 +52,8 @@ def test_mktemp(self, testdir): assert tmp2.relto(t.getbasetemp()).startswith("this") assert tmp2 != tmp -class TestConfigTmpdir: + +class TestConfigTmpdir(object): def test_getbasetemp_custom_removes_old(self, testdir): mytemp = testdir.tmpdir.join("xyz") p = testdir.makepyfile(""" @@ -76,6 +80,7 @@ def test_1(): assert result.ret == 0 assert mytemp.join('hello').check() + @pytest.mark.skipif(not hasattr(py.path.local, 'mksymlinkto'), reason="symlink not available on this platform") def test_tmpdir_always_is_realpath(testdir): diff --git a/testing/test_unittest.py b/testing/test_unittest.py index 144aad79bf46339..e197735871fc53f 100644 --- a/testing/test_unittest.py +++ b/testing/test_unittest.py @@ -1,28 +1,32 @@ +from __future__ import absolute_import, division, print_function from _pytest.main import EXIT_NOTESTSCOLLECTED import pytest +import gc + def test_simple_unittest(testdir): testpath = testdir.makepyfile(""" import unittest class MyTestCase(unittest.TestCase): def testpassing(self): - self.assertEquals('foo', 'foo') + self.assertEqual('foo', 'foo') def test_failing(self): - self.assertEquals('foo', 'bar') + self.assertEqual('foo', 'bar') """) reprec = testdir.inline_run(testpath) assert reprec.matchreport("testpassing").passed assert reprec.matchreport("test_failing").failed + def test_runTest_method(testdir): testdir.makepyfile(""" import unittest class MyTestCaseWithRunTest(unittest.TestCase): def runTest(self): - self.assertEquals('foo', 'foo') + self.assertEqual('foo', 'foo') class MyTestCaseWithoutRunTest(unittest.TestCase): def runTest(self): - self.assertEquals('foo', 'foo') + self.assertEqual('foo', 'foo') def test_something(self): pass """) @@ -33,6 +37,7 @@ def test_something(self): *2 passed* """) + def test_isclasscheck_issue53(testdir): testpath = testdir.makepyfile(""" import unittest @@ -44,6 +49,7 @@ def __getattr__(self, tag): result = testdir.runpytest(testpath) assert result.ret == EXIT_NOTESTSCOLLECTED + def test_setup(testdir): testpath = testdir.makepyfile(""" import unittest @@ -53,7 +59,7 @@ def setUp(self): def setup_method(self, method): self.foo2 = 1 def test_both(self): - self.assertEquals(1, self.foo) + self.assertEqual(1, self.foo) assert self.foo2 == 1 def teardown_method(self, method): assert 0, "42" @@ -64,36 +70,38 @@ def teardown_method(self, method): rep = reprec.matchreport("test_both", when="teardown") assert rep.failed and '42' in str(rep.longrepr) + def test_setUpModule(testdir): testpath = testdir.makepyfile(""" - l = [] + values = [] def setUpModule(): - l.append(1) + values.append(1) def tearDownModule(): - del l[0] + del values[0] def test_hello(): - assert l == [1] + assert values == [1] def test_world(): - assert l == [1] + assert values == [1] """) result = testdir.runpytest(testpath) result.stdout.fnmatch_lines([ "*2 passed*", ]) + def test_setUpModule_failing_no_teardown(testdir): testpath = testdir.makepyfile(""" - l = [] + values = [] def setUpModule(): 0/0 def tearDownModule(): - l.append(1) + values.append(1) def test_hello(): pass @@ -101,7 +109,8 @@ def test_hello(): reprec = testdir.inline_run(testpath) reprec.assertoutcome(passed=0, failed=1) call = reprec.getcalls("pytest_runtest_setup")[0] - assert not call.item.module.l + assert not call.item.module.values + def test_new_instances(testdir): testpath = testdir.makepyfile(""" @@ -115,18 +124,19 @@ def test_func2(self): reprec = testdir.inline_run(testpath) reprec.assertoutcome(passed=2) + def test_teardown(testdir): testpath = testdir.makepyfile(""" import unittest class MyTestCase(unittest.TestCase): - l = [] + values = [] def test_one(self): pass def tearDown(self): - self.l.append(None) + self.values.append(None) class Second(unittest.TestCase): def test_check(self): - self.assertEquals(MyTestCase.l, [None]) + self.assertEqual(MyTestCase.values, [None]) """) reprec = testdir.inline_run(testpath) passed, skipped, failed = reprec.countoutcomes() @@ -134,7 +144,30 @@ def test_check(self): assert passed == 2 assert passed + skipped + failed == 2 -@pytest.mark.skipif("sys.version_info < (2,7)") + +def test_teardown_issue1649(testdir): + """ + Are TestCase objects cleaned up? Often unittest TestCase objects set + attributes that are large and expensive during setUp. + + The TestCase will not be cleaned up if the test fails, because it + would then exist in the stackframe. + """ + testpath = testdir.makepyfile(""" + import unittest + class TestCaseObjectsShouldBeCleanedUp(unittest.TestCase): + def setUp(self): + self.an_expensive_object = 1 + def test_demo(self): + pass + + """) + testdir.inline_run("-s", testpath) + gc.collect() + for obj in gc.get_objects(): + assert type(obj).__name__ != 'TestCaseObjectsShouldBeCleanedUp' + + def test_unittest_skip_issue148(testdir): testpath = testdir.makepyfile(""" import unittest @@ -153,6 +186,7 @@ def tearDownClass(self): reprec = testdir.inline_run(testpath) reprec.assertoutcome(skipped=1) + def test_method_and_teardown_failing_reporting(testdir): testdir.makepyfile(""" import unittest, pytest @@ -172,6 +206,7 @@ def test_method(self): "*1 failed*1 error*", ]) + def test_setup_failure_is_shown(testdir): testdir.makepyfile(""" import unittest @@ -192,6 +227,7 @@ def test_method(self): ]) assert 'never42' not in result.stdout.str() + def test_setup_setUpClass(testdir): testpath = testdir.makepyfile(""" import unittest @@ -214,6 +250,7 @@ def test_teareddown(): reprec = testdir.inline_run(testpath) reprec.assertoutcome(passed=3) + def test_setup_class(testdir): testpath = testdir.makepyfile(""" import unittest @@ -255,6 +292,7 @@ def test_hello(self): result = testdir.runpytest() assert 'should not raise' not in result.stdout.str() + @pytest.mark.parametrize("type", ['Error', 'Failure']) def test_testcase_custom_exception_info(testdir, type): testdir.makepyfile(""" @@ -265,8 +303,8 @@ class MyTestCase(TestCase): def run(self, result): excinfo = pytest.raises(ZeroDivisionError, lambda: 0/0) # we fake an incompatible exception info - from _pytest.monkeypatch import monkeypatch - mp = monkeypatch() + from _pytest.monkeypatch import MonkeyPatch + mp = MonkeyPatch() def t(*args): mp.undo() raise TypeError() @@ -286,6 +324,7 @@ def test_hello(self): "*1 failed*", ]) + def test_testcase_totally_incompatible_exception_info(testdir): item, = testdir.getitems(""" from unittest import TestCase @@ -297,6 +336,7 @@ def test_hello(self): excinfo = item._excinfo.pop(0) assert 'ERROR: Unknown Incompatible' in str(excinfo.getrepr()) + def test_module_level_pytestmark(testdir): testpath = testdir.makepyfile(""" import unittest @@ -310,61 +350,12 @@ def test_func1(self): reprec.assertoutcome(skipped=1) -def test_trial_testcase_skip_property(testdir): - pytest.importorskip('twisted.trial.unittest') - testpath = testdir.makepyfile(""" - from twisted.trial import unittest - class MyTestCase(unittest.TestCase): - skip = 'dont run' - def test_func(self): - pass - """) - reprec = testdir.inline_run(testpath, "-s") - reprec.assertoutcome(skipped=1) - - -def test_trial_testfunction_skip_property(testdir): - pytest.importorskip('twisted.trial.unittest') - testpath = testdir.makepyfile(""" - from twisted.trial import unittest - class MyTestCase(unittest.TestCase): - def test_func(self): - pass - test_func.skip = 'dont run' - """) - reprec = testdir.inline_run(testpath, "-s") - reprec.assertoutcome(skipped=1) - - -def test_trial_testcase_todo_property(testdir): - pytest.importorskip('twisted.trial.unittest') - testpath = testdir.makepyfile(""" - from twisted.trial import unittest - class MyTestCase(unittest.TestCase): - todo = 'dont run' - def test_func(self): - assert 0 - """) - reprec = testdir.inline_run(testpath, "-s") - reprec.assertoutcome(skipped=1) - - -def test_trial_testfunction_todo_property(testdir): - pytest.importorskip('twisted.trial.unittest') - testpath = testdir.makepyfile(""" - from twisted.trial import unittest - class MyTestCase(unittest.TestCase): - def test_func(self): - assert 0 - test_func.todo = 'dont run' - """) - reprec = testdir.inline_run(testpath, "-s") - reprec.assertoutcome(skipped=1) - - -class TestTrialUnittest: +class TestTrialUnittest(object): def setup_class(cls): cls.ut = pytest.importorskip("twisted.trial.unittest") + # on windows trial uses a socket for a reactor and apparently doesn't close it properly + # https://twistedmatrix.com/trac/ticket/9227 + cls.ignore_unclosed_socket_warning = ('-W', 'always') def test_trial_testcase_runtest_not_collected(self, testdir): testdir.makepyfile(""" @@ -374,7 +365,7 @@ class TC(TestCase): def test_hello(self): pass """) - reprec = testdir.inline_run() + reprec = testdir.inline_run(*self.ignore_unclosed_socket_warning) reprec.assertoutcome(passed=1) testdir.makepyfile(""" from twisted.trial.unittest import TestCase @@ -383,7 +374,7 @@ class TC(TestCase): def runTest(self): pass """) - reprec = testdir.inline_run() + reprec = testdir.inline_run(*self.ignore_unclosed_socket_warning) reprec.assertoutcome(passed=1) def test_trial_exceptions_with_skips(self, testdir): @@ -419,8 +410,9 @@ def setup_class(cls): def test_method(self): pass """) - result = testdir.runpytest("-rxs") - assert result.ret == 0 + from _pytest.compat import _is_unittest_unexpected_success_a_failure + should_fail = _is_unittest_unexpected_success_a_failure() + result = testdir.runpytest("-rxs", *self.ignore_unclosed_socket_warning) result.stdout.fnmatch_lines_random([ "*XFAIL*test_trial_todo*", "*trialselfskip*", @@ -429,8 +421,9 @@ def test_method(self): "*i2wanto*", "*sys.version_info*", "*skip_in_method*", - "*4 skipped*3 xfail*1 xpass*", + "*1 failed*4 skipped*3 xfailed*" if should_fail else "*4 skipped*3 xfail*1 xpass*", ]) + assert result.ret == (1 if should_fail else 0) def test_trial_error(self, testdir): testdir.makepyfile(""" @@ -494,6 +487,51 @@ def test_hello(self): child.expect("hellopdb") child.sendeof() + def test_trial_testcase_skip_property(self, testdir): + testpath = testdir.makepyfile(""" + from twisted.trial import unittest + class MyTestCase(unittest.TestCase): + skip = 'dont run' + def test_func(self): + pass + """) + reprec = testdir.inline_run(testpath, "-s") + reprec.assertoutcome(skipped=1) + + def test_trial_testfunction_skip_property(self, testdir): + testpath = testdir.makepyfile(""" + from twisted.trial import unittest + class MyTestCase(unittest.TestCase): + def test_func(self): + pass + test_func.skip = 'dont run' + """) + reprec = testdir.inline_run(testpath, "-s") + reprec.assertoutcome(skipped=1) + + def test_trial_testcase_todo_property(self, testdir): + testpath = testdir.makepyfile(""" + from twisted.trial import unittest + class MyTestCase(unittest.TestCase): + todo = 'dont run' + def test_func(self): + assert 0 + """) + reprec = testdir.inline_run(testpath, "-s") + reprec.assertoutcome(skipped=1) + + def test_trial_testfunction_todo_property(self, testdir): + testpath = testdir.makepyfile(""" + from twisted.trial import unittest + class MyTestCase(unittest.TestCase): + def test_func(self): + assert 0 + test_func.todo = 'dont run' + """) + reprec = testdir.inline_run(testpath, "-s", *self.ignore_unclosed_socket_warning) + reprec.assertoutcome(skipped=1) + + def test_djangolike_testcase(testdir): # contributed from Morten Breekevold testdir.makepyfile(""" @@ -554,11 +592,12 @@ def test_unittest_not_shown_in_traceback(testdir): class t(unittest.TestCase): def test_hello(self): x = 3 - self.assertEquals(x, 4) + self.assertEqual(x, 4) """) res = testdir.runpytest() assert "failUnlessEqual" not in res.stdout.str() + def test_unorderable_types(testdir): testdir.makepyfile(""" import unittest @@ -576,6 +615,7 @@ class Test(unittest.TestCase): assert "TypeError" not in result.stdout.str() assert result.ret == EXIT_NOTESTSCOLLECTED + def test_unittest_typerror_traceback(testdir): testdir.makepyfile(""" import unittest @@ -587,24 +627,60 @@ def test_hello(self, arg1): assert "TypeError" in result.stdout.str() assert result.ret == 1 -@pytest.mark.skipif("sys.version_info < (2,7)") -def test_unittest_unexpected_failure(testdir): - testdir.makepyfile(""" + +@pytest.mark.parametrize('runner', ['pytest', 'unittest']) +def test_unittest_expected_failure_for_failing_test_is_xfail(testdir, runner): + script = testdir.makepyfile(""" import unittest class MyTestCase(unittest.TestCase): @unittest.expectedFailure - def test_func1(self): - assert 0 + def test_failing_test_is_xfail(self): + assert False + if __name__ == '__main__': + unittest.main() + """) + if runner == 'pytest': + result = testdir.runpytest("-rxX") + result.stdout.fnmatch_lines([ + "*XFAIL*MyTestCase*test_failing_test_is_xfail*", + "*1 xfailed*", + ]) + else: + result = testdir.runpython(script) + result.stderr.fnmatch_lines([ + "*1 test in*", + "*OK*(expected failures=1)*", + ]) + assert result.ret == 0 + + +@pytest.mark.parametrize('runner', ['pytest', 'unittest']) +def test_unittest_expected_failure_for_passing_test_is_fail(testdir, runner): + script = testdir.makepyfile(""" + import unittest + class MyTestCase(unittest.TestCase): @unittest.expectedFailure - def test_func2(self): - assert 1 + def test_passing_test_is_fail(self): + assert True + if __name__ == '__main__': + unittest.main() """) - result = testdir.runpytest("-rxX") - result.stdout.fnmatch_lines([ - "*XFAIL*MyTestCase*test_func1*", - "*XPASS*MyTestCase*test_func2*", - "*1 xfailed*1 xpass*", - ]) + from _pytest.compat import _is_unittest_unexpected_success_a_failure + should_fail = _is_unittest_unexpected_success_a_failure() + if runner == 'pytest': + result = testdir.runpytest("-rxX") + result.stdout.fnmatch_lines([ + "*MyTestCase*test_passing_test_is_fail*", + "*1 failed*" if should_fail else "*1 xpassed*", + ]) + else: + result = testdir.runpython(script) + result.stderr.fnmatch_lines([ + "*1 test in*", + "*(unexpected successes=1)*", + ]) + + assert result.ret == (1 if should_fail else 0) @pytest.mark.parametrize('fix_type, stmt', [ @@ -641,7 +717,7 @@ def test_classattr(self): def test_non_unittest_no_setupclass_support(testdir): testpath = testdir.makepyfile(""" - class TestFoo: + class TestFoo(object): x = 0 @classmethod @@ -691,8 +767,10 @@ def test_notTornDown(): def test_issue333_result_clearing(testdir): testdir.makeconftest(""" - def pytest_runtest_call(__multicall__, item): - __multicall__.execute() + import pytest + @pytest.hookimpl(hookwrapper=True) + def pytest_runtest_call(item): + yield assert 0 """) testdir.makepyfile(""" @@ -705,7 +783,7 @@ def test_func(self): reprec = testdir.inline_run() reprec.assertoutcome(failed=1) -@pytest.mark.skipif("sys.version_info < (2,7)") + def test_unittest_raise_skip_issue748(testdir): testdir.makepyfile(test_foo=""" import unittest @@ -720,11 +798,11 @@ def test_one(self): *1 skipped* """) -@pytest.mark.skipif("sys.version_info < (2,7)") + def test_unittest_skip_issue1169(testdir): testdir.makepyfile(test_foo=""" import unittest - + class MyTestCase(unittest.TestCase): @unittest.skip("skipping due to reasons") def test_skip(self): @@ -735,3 +813,18 @@ def test_skip(self): *SKIP*[1]*skipping due to reasons* *1 skipped* """) + + +def test_class_method_containing_test_issue1558(testdir): + testdir.makepyfile(test_foo=""" + import unittest + + class MyTestCase(unittest.TestCase): + def test_should_run(self): + pass + def test_should_not_run(self): + pass + test_should_not_run.__test__ = False + """) + reprec = testdir.inline_run() + reprec.assertoutcome(passed=1) diff --git a/testing/test_warnings.py b/testing/test_warnings.py new file mode 100644 index 000000000000000..02400bd1dedacff --- /dev/null +++ b/testing/test_warnings.py @@ -0,0 +1,258 @@ +# -*- coding: utf8 -*- +from __future__ import unicode_literals + +import sys + +import pytest + + +WARNINGS_SUMMARY_HEADER = 'warnings summary' + + +@pytest.fixture +def pyfile_with_warnings(testdir, request): + """ + Create a test file which calls a function in a module which generates warnings. + """ + testdir.syspathinsert() + test_name = request.function.__name__ + module_name = test_name.lstrip('test_') + '_module' + testdir.makepyfile(**{ + module_name: ''' + import warnings + def foo(): + warnings.warn(UserWarning("user warning")) + warnings.warn(RuntimeWarning("runtime warning")) + return 1 + ''', + test_name: ''' + import {module_name} + def test_func(): + assert {module_name}.foo() == 1 + '''.format(module_name=module_name) + }) + + +@pytest.mark.filterwarnings('always') +def test_normal_flow(testdir, pyfile_with_warnings): + """ + Check that the warnings section is displayed, containing test node ids followed by + all warnings generated by that test node. + """ + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*== %s ==*' % WARNINGS_SUMMARY_HEADER, + + '*test_normal_flow.py::test_func', + + '*normal_flow_module.py:3: UserWarning: user warning', + '* warnings.warn(UserWarning("user warning"))', + + '*normal_flow_module.py:4: RuntimeWarning: runtime warning', + '* warnings.warn(RuntimeWarning("runtime warning"))', + '* 1 passed, 2 warnings*', + ]) + assert result.stdout.str().count('test_normal_flow.py::test_func') == 1 + + +@pytest.mark.filterwarnings('always') +def test_setup_teardown_warnings(testdir, pyfile_with_warnings): + testdir.makepyfile(''' + import warnings + import pytest + + @pytest.fixture + def fix(): + warnings.warn(UserWarning("warning during setup")) + yield + warnings.warn(UserWarning("warning during teardown")) + + def test_func(fix): + pass + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*== %s ==*' % WARNINGS_SUMMARY_HEADER, + + '*test_setup_teardown_warnings.py:6: UserWarning: warning during setup', + '*warnings.warn(UserWarning("warning during setup"))', + + '*test_setup_teardown_warnings.py:8: UserWarning: warning during teardown', + '*warnings.warn(UserWarning("warning during teardown"))', + '* 1 passed, 2 warnings*', + ]) + + +@pytest.mark.parametrize('method', ['cmdline', 'ini']) +def test_as_errors(testdir, pyfile_with_warnings, method): + args = ('-W', 'error') if method == 'cmdline' else () + if method == 'ini': + testdir.makeini(''' + [pytest] + filterwarnings= error + ''') + result = testdir.runpytest(*args) + result.stdout.fnmatch_lines([ + 'E UserWarning: user warning', + 'as_errors_module.py:3: UserWarning', + '* 1 failed in *', + ]) + + +@pytest.mark.parametrize('method', ['cmdline', 'ini']) +def test_ignore(testdir, pyfile_with_warnings, method): + args = ('-W', 'ignore') if method == 'cmdline' else () + if method == 'ini': + testdir.makeini(''' + [pytest] + filterwarnings= ignore + ''') + + result = testdir.runpytest(*args) + result.stdout.fnmatch_lines([ + '* 1 passed in *', + ]) + assert WARNINGS_SUMMARY_HEADER not in result.stdout.str() + + +@pytest.mark.skipif(sys.version_info < (3, 0), + reason='warnings message is unicode is ok in python3') +@pytest.mark.filterwarnings('always') +def test_unicode(testdir, pyfile_with_warnings): + testdir.makepyfile(''' + # -*- coding: utf8 -*- + import warnings + import pytest + + + @pytest.fixture + def fix(): + warnings.warn(u"测试") + yield + + def test_func(fix): + pass + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*== %s ==*' % WARNINGS_SUMMARY_HEADER, + '*test_unicode.py:8: UserWarning: \u6d4b\u8bd5*', + '* 1 passed, 1 warnings*', + ]) + + +@pytest.mark.skipif(sys.version_info >= (3, 0), + reason='warnings message is broken as it is not str instance') +def test_py2_unicode(testdir, pyfile_with_warnings): + if getattr(sys, "pypy_version_info", ())[:2] == (5, 9) and sys.platform.startswith('win'): + pytest.xfail("fails with unicode error on PyPy2 5.9 and Windows (#2905)") + testdir.makepyfile(''' + # -*- coding: utf8 -*- + import warnings + import pytest + + + @pytest.fixture + def fix(): + warnings.warn(u"测试") + yield + + @pytest.mark.filterwarnings('always') + def test_func(fix): + pass + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*== %s ==*' % WARNINGS_SUMMARY_HEADER, + + '*test_py2_unicode.py:8: UserWarning: \\u6d4b\\u8bd5', + '*warnings.warn(u"\u6d4b\u8bd5")', + '*warnings.py:*: UnicodeWarning: Warning is using unicode non*', + '* 1 passed, 2 warnings*', + ]) + + +def test_py2_unicode_ascii(testdir): + """Ensure that our warning about 'unicode warnings containing non-ascii messages' + does not trigger with ascii-convertible messages""" + testdir.makeini('[pytest]') + testdir.makepyfile(''' + import pytest + import warnings + + @pytest.mark.filterwarnings('always') + def test_func(): + warnings.warn(u"hello") + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*== %s ==*' % WARNINGS_SUMMARY_HEADER, + '*warnings.warn(u"hello")', + '* 1 passed, 1 warnings in*' + ]) + + +def test_works_with_filterwarnings(testdir): + """Ensure our warnings capture does not mess with pre-installed filters (#2430).""" + testdir.makepyfile(''' + import warnings + + class MyWarning(Warning): + pass + + warnings.filterwarnings("error", category=MyWarning) + + class TestWarnings(object): + def test_my_warning(self): + try: + warnings.warn(MyWarning("warn!")) + assert False + except MyWarning: + assert True + ''') + result = testdir.runpytest() + result.stdout.fnmatch_lines([ + '*== 1 passed in *', + ]) + + +@pytest.mark.parametrize('default_config', ['ini', 'cmdline']) +def test_filterwarnings_mark(testdir, default_config): + """ + Test ``filterwarnings`` mark works and takes precedence over command line and ini options. + """ + if default_config == 'ini': + testdir.makeini(""" + [pytest] + filterwarnings = always + """) + testdir.makepyfile(""" + import warnings + import pytest + + @pytest.mark.filterwarnings('ignore::RuntimeWarning') + def test_ignore_runtime_warning(): + warnings.warn(RuntimeWarning()) + + @pytest.mark.filterwarnings('error') + def test_warning_error(): + warnings.warn(RuntimeWarning()) + + def test_show_warning(): + warnings.warn(RuntimeWarning()) + """) + result = testdir.runpytest('-W always' if default_config == 'cmdline' else '') + result.stdout.fnmatch_lines(['*= 1 failed, 2 passed, 1 warnings in *']) + + +def test_non_string_warning_argument(testdir): + """Non-str argument passed to warning breaks pytest (#2956)""" + testdir.makepyfile(""" + import warnings + import pytest + + def test(): + warnings.warn(UserWarning(1, u'foo')) + """) + result = testdir.runpytest('-W', 'always') + result.stdout.fnmatch_lines(['*= 1 passed, 1 warnings in *']) diff --git a/tox.ini b/tox.ini index 5f65446e42b6e7e..900b602dc337da1 100644 --- a/tox.ini +++ b/tox.ini @@ -1,160 +1,220 @@ [tox] -minversion=2.0 -distshare={homedir}/.tox/distshare -envlist= - linting,py26,py27,py33,py34,py35,pypy, - {py27,py35}-{pexpect,xdist,trial}, - py27-nobyte,doctesting,py27-cxfreeze +minversion = 2.0 +distshare = {homedir}/.tox/distshare +# make sure to update environment list in travis.yml and appveyor.yml +envlist = + linting + py27 + py34 + py35 + py36 + py37 + pypy + {py27,py36}-{pexpect,xdist,trial,numpy,pluggymaster} + py27-nobyte + doctesting + py35-freeze + docs [testenv] -commands= py.test --lsof -rfsxX {posargs:testing} +commands = pytest --lsof -ra {posargs:testing} passenv = USER USERNAME -deps= +deps = + hypothesis>=3.5.2 nose mock requests -[testenv:py26] -commands= py.test --lsof -rfsxX {posargs:testing} -deps= - nose - mock<1.1 # last supported version for py26 - [testenv:py27-subprocess] -changedir=. -basepython=python2.7 -deps=pytest-xdist>=1.13 +changedir = . +deps = + pytest-xdist>=1.13 mock nose -commands= - py.test -n3 -rfsxX --runpytest=subprocess {posargs:testing} +commands = + pytest -n3 -ra --runpytest=subprocess {posargs:testing} -[testenv:genscript] -commands= py.test --genscript=pytest1 [testenv:linting] +skipsdist = True +usedevelop = True basepython = python2.7 -deps = flake8 +deps = + flake8 + # pygments required by rst-lint + pygments restructuredtext_lint -commands = flake8 pytest.py _pytest testing - rst-lint CHANGELOG.rst +commands = + flake8 pytest.py _pytest testing setup.py pytest.py + {envpython} scripts/check-rst.py [testenv:py27-xdist] -deps=pytest-xdist>=1.13 +deps = + pytest-xdist>=1.13 mock nose -commands= - py.test -n1 -rfsxX {posargs:testing} + hypothesis>=3.5.2 +changedir=testing +commands = + pytest -n1 -ra {posargs:.} -[testenv:py35-xdist] -deps={[testenv:py27-xdist]deps} -commands= - py.test -n3 -rfsxX {posargs:testing} +[testenv:py36-xdist] +deps = {[testenv:py27-xdist]deps} +commands = + pytest -n3 -ra {posargs:testing} [testenv:py27-pexpect] -changedir=testing -platform=linux|darwin -deps=pexpect -commands= - py.test -rfsxX test_pdb.py test_terminal.py test_unittest.py - -[testenv:py35-pexpect] -changedir=testing -platform=linux|darwin -deps={[testenv:py27-pexpect]deps} -commands= - py.test -rfsxX test_pdb.py test_terminal.py test_unittest.py +changedir = testing +platform = linux|darwin +deps = pexpect +commands = + pytest -ra test_pdb.py test_terminal.py test_unittest.py + +[testenv:py36-pexpect] +changedir = testing +platform = linux|darwin +deps = {[testenv:py27-pexpect]deps} +commands = + pytest -ra test_pdb.py test_terminal.py test_unittest.py [testenv:py27-nobyte] -deps=pytest-xdist>=1.13 -distribute=true -setenv= +deps = + pytest-xdist>=1.13 + hypothesis>=3.5.2 +distribute = true +changedir=testing +setenv = PYTHONDONTWRITEBYTECODE=1 -commands= - py.test -n3 -rfsxX {posargs:testing} +commands = + pytest -n3 -ra {posargs:.} [testenv:py27-trial] -deps=twisted +deps = twisted +commands = + pytest -ra {posargs:testing/test_unittest.py} + +[testenv:py36-trial] +deps = {[testenv:py27-trial]deps} +commands = + pytest -ra {posargs:testing/test_unittest.py} + +[testenv:py27-numpy] +deps=numpy commands= - py.test -rsxf {posargs:testing/test_unittest.py} + pytest -ra {posargs:testing/python/approx.py} -[testenv:py35-trial] -platform=linux|darwin -deps={[testenv:py27-trial]deps} +[testenv:py36-numpy] +deps=numpy commands= - py.test -rsxf {posargs:testing/test_unittest.py} + pytest -ra {posargs:testing/python/approx.py} -[testenv:doctest] -commands=py.test --doctest-modules _pytest -deps= +[testenv:py27-pluggymaster] +setenv= + _PYTEST_SETUP_SKIP_PLUGGY_DEP=1 +deps = + {[testenv]deps} + git+https://github.com/pytest-dev/pluggy.git@master -[testenv:doc] -basepython=python -changedir=doc/en -deps=sphinx - PyYAML +[testenv:py35-pluggymaster] +setenv= + _PYTEST_SETUP_SKIP_PLUGGY_DEP=1 +deps = + {[testenv:py27-pluggymaster]deps} + git+https://github.com/pytest-dev/pluggy.git@master + +[testenv:docs] +skipsdist = True +usedevelop = True +basepython = python +changedir = doc/en +deps = + sphinx + PyYAML -commands= - make clean - make html +commands = + sphinx-build -W -b html . _build [testenv:doctesting] -basepython = python3.4 -changedir=doc/en -deps=PyYAML -commands= py.test -rfsxX {posargs} +basepython = python +usedevelop = True +skipsdist = True +# ensure the given pyargs cant mean anytrhing else +changedir = doc/ +deps = + PyYAML +commands = + pytest -ra en + pytest --doctest-modules --pyargs _pytest [testenv:regen] -changedir=doc/en -basepython = python3.4 -deps=sphinx - PyYAML - regendoc>=0.6.1 -whitelist_externals= +changedir = doc/en +skipsdist = True +basepython = python3.5 +deps = + sphinx + PyYAML + regendoc>=0.6.1 +whitelist_externals = rm make -commands= +commands = rm -rf /tmp/doc-exec* make regen -[testenv:jython] -changedir=testing -commands= - {envpython} {envbindir}/py.test-jython -rfsxX {posargs} +[testenv:fix-lint] +skipsdist = True +usedevelop = True +deps = + autopep8 +commands = + autopep8 --in-place -r --max-line-length=120 --exclude=test_source_multiline_block.py _pytest testing setup.py pytest.py -[testenv:py27-cxfreeze] -changedir=testing/cx_freeze -platform=linux|darwin -commands= - {envpython} install_cx_freeze.py - {envpython} runtests_setup.py build --build-exe build +[testenv:jython] +changedir = testing +commands = + {envpython} {envbindir}/py.test-jython -ra {posargs} + +[testenv:py35-freeze] +changedir = testing/freeze +deps = pyinstaller +commands = + {envpython} create_executable.py {envpython} tox_run.py [testenv:coveralls] passenv = TRAVIS TRAVIS_JOB_ID TRAVIS_BRANCH COVERALLS_REPO_TOKEN -usedevelop=True -basepython=python3.4 -changedir=. +usedevelop = True +changedir = . deps = {[testenv]deps} coveralls -commands= +commands = coverage run --source=_pytest -m pytest testing coverage report -m coveralls [pytest] -minversion=2.0 -plugins=pytester +minversion = 2.0 +plugins = pytester #--pyargs --doctest-modules --ignore=.tox -addopts= -rxsX -p pytester --ignore=testing/cx_freeze -rsyncdirs=tox.ini pytest.py _pytest testing -python_files=test_*.py *_test.py testing/*/*.py -python_classes=Test Acceptance -python_functions=test +addopts = -ra -p pytester --ignore=testing/cx_freeze +rsyncdirs = tox.ini pytest.py _pytest testing +python_files = test_*.py *_test.py testing/*/*.py +python_classes = Test Acceptance +python_functions = test norecursedirs = .tox ja .hg cx_freeze_source - +xfail_strict=true +filterwarnings = + error + # produced by path.local + ignore:bad escape.*:DeprecationWarning:re + # produced by path.readlines + ignore:.*U.*mode is deprecated:DeprecationWarning + # produced by pytest-xdist + ignore:.*type argument to addoption.*:DeprecationWarning + # produced by python >=3.5 on execnet (pytest-xdist) + ignore:.*inspect.getargspec.*deprecated, use inspect.signature.*:DeprecationWarning [flake8] -ignore =E401,E225,E261,E128,E124,E301,E302,E121,E303,W391,E501,E231,E126,E701,E265,E241,E251,E226,E101,W191,E131,E203,E122,E123,E271,E712,E222,E127,E125,E221,W292,E111,E113,E293,E262,W293,E129,E702,E201,E272,E202,E704,E731,E402 +max-line-length = 120