Pytest-其二

Pytest-其二

1. 运行多个测试

   pytest会运行当前文件夹下以及其子文件夹下的所有格式为test_*.py或*_test的文件。他遵循标准的测试发现规则

2. 断言异常

   使用raises函数断言程序抛出某个异常

    # content of test_sysexit.py
    import pytest
    def f():
    	raise SystemExit(1)
    def test_mytest():
    	with pytest.raises(SystemExit):
    		f()
   

   抛出某个异常组

    # content of test_exceptiongroup.py
    import pytest
    def f():
    	raise ExceptionGroup(
    		"Group message",
    		[
    		RuntimeError(),
    		],
   	 )
    def test_exception_in_group():
    	with pytest.raises(ExceptionGroup) as excinfo:
    		f()
    	assert excinfo.group_contains(RuntimeError)
    	assert not excinfo.group_contains(TypeError)
   

   对实际引发的异常进行包装。感兴趣的主要属性是 .type、.value 和 .traceback

   def test_recursion_depth():
    	with pytest.raises(RuntimeError) as excinfo:
   		def f():
    			f()
    	f()
       assert "maximum recursion" in str(excinfo.value)
   

    def test_foo_not_implemented():
    	def foo():
    		raise NotImplementedError
    	with pytest.raises(RuntimeError) as excinfo:
    		foo()
    	assert excinfo.type is RuntimeError
   

   pytest.raises（） 调用将成功，即使该函数引发 NotImplementedError，因为 NotImplementedError 是 RuntimeError 的子类;但是，以下 assert 语句将捕获问题

   match异常中的message

    import pytest
    def myfunc():
    	raise ValueError("Exception 123 raised")
    def test_match():
    	with pytest.raises(ValueError, match=r".* 123 .*"):
    		myfunc()
   

   pytest.mark.xfail

   pytest.mark.xfail 指定一个 raises 参数，该参数检查测试是否以更具体的方式失败，而不仅仅是引发任何异常

    def f():
    	raise IndexError()
    @pytest.mark.xfail(raises=IndexError)
    def test_f():
    	f()
   

   使用 pytest.raises（） 可能更适合于测试自己的代码故意引发的异常，这是大多数情况

3. pytest中的quiet报告模式

   可以通过在命令行中使用 -q 或 --quiet 参数来启用安静模式。

   在安静模式下，pytest 通常会只显示测试用例的执行结果摘要，而不会输出过多的详细信息，如测试用例的名称、测试过程中的一些中间信息等。这样可以在快速查看测试结果时非常有用，尤其是当测试用例数量较多时，可以避免信息过多造成的干扰。

4. 启动方式

   Run tests in a module

   pytest test_mod.py
   

   Run tests in a directory

    pytest testing/
   

   Run tests by keyword expressions

    pytest-k 'MyClass and not method'
   

   The example above will run TestMyClass. test_something but not TestMyClass.test_method_simple. Use "" instead of '' in expression when running this on Windows

   Run tests by collection arguments

   Pass the module filename relative to the working directory, followed by specifiers like the class name and function name separated by :: characters, and parameters from parameterization enclosed in [].

    pytest tests/test_mod.py::test_func[x1,y2]
   

   Run tests by marker expressions

   To run all tests which are decorated with the @pytest.mark.slow decorator:

   pytest-m slow 
   

   To run all tests which are decorated with the annotated @pytest.mark.slow(phase=1) decorator, with the phase keyword argument set to 1:

   pytest-m "slow(phase=1)
   

   python code 启动

   这就像您从命令行调用 “pytest” 一样。它不会引发 SystemExit，而是返回退出代码。如果你没有向它传递任何参数，main 会从进程的命令行参数 （sys.argv） 中读取参数，这可能是不需要的。您可以显式传入选项和参数：

   retcode = pytest.main(["-x", "mytestdir"])
   

5. 分析测试执行持续时间

   To get a list of the slowest 10 test durations over 1.0s long:

   pytest--durations=10--durations-min=1.0
   

6. fixtures

   在基本层面上，测试函数通过将 fixture 声明为参数来请求它们需要的 fixtures。当 pytest 去运行测试时，它会查看该测试函数签名中的参数，然后搜索与这些参数同名的 fixtures。一旦 pytest 找到它们，它就会运行这些 fixture，捕获它们返回的内容（如果有的话），并将这些对象作为参数传递给 test 函数。

   At a basic level, test functions request fixtures they require by declaring them as arguments. When pytest goes to run a test, it looks at the parameters in that test function’s signature, and then searches for fixtures that have the same names as those parameters. Once pytest finds them, it runs those fixtures, captures what they returned (if anything), and passes those objects into the test function as arguments

   import pytest
   
   class Fruit:
       def __init__(self, name):
           self.name = name
           self.cubed = False
   
       def cube(self):
           self.cubed = True
   
   
   class FruitSalad:
       def __init__(self, *fruit_bowl):
           self.fruit = fruit_bowl
           self._cube_fruit()
   
       def _cube_fruit(self):
           for fruit in self.fruit:
               fruit.cube()
   
   
   # Arrange
   @pytest.fixture
   def fruit_bowl():
       return [Fruit("apple"), Fruit("banana")]
   
   
   def test_fruit_salad(fruit_bowl):
       # Act
       fruit_salad = FruitSalad(*fruit_bowl)
       # Assert
       assert all(fruit.cubed for fruit in fruit_salad.fruit)
   

7. fixtures使用其他的fixtures

   pytest 最大的优势之一是其极其灵活的 fixture 系统。pytest 中的 Fixture 请求 Fixtures 就像 tests 一样。

   # contents of test_append.py
   import pytest
   
   
   # Arrange
   @pytest.fixture
   def first_entry():
       return "a"
   
   
   # Arrange
   @pytest.fixture
   def order(first_entry):
       return [first_entry]
   
   
   def test_string(order):
       # Act
       order.append("b")
       # Assert
       assert order == ["a", "b"]
   

   如果手工写，那他的示例是：

   def first_entry():
       return "a"
   
   
   def order(first_entry):
       return [first_entry]
   
   
   def test_string(order):
       # Act
       order.append("b")
       # Assert
       assert order == ["a", "b"]
   
   
   entry = first_entry()
   the_list = order(first_entry=entry)
   test_string(order=the_list)
   

8. 同一test中多次调用fixture

   同一test中多次调用fixture时，pytest是不会多次调用fixture的，返回值是被缓存的

   import pytest
   
   
   # Arrange
   @pytest.fixture
   def first_entry():
       return "a"
   
   
   # Arrange
   @pytest.fixture
   def order():
       return []
   
   
   # Act
   @pytest.fixture
   def append_first(order, first_entry):
       return order.append(first_entry)
   
   
   def test_string_only(append_first, order, first_entry):
       # Assert
       assert order == [first_entry]
   
   

   如果fixture是执行2次的，那么test是fail的，因为append_first和test_string_only引用的order都是[]，但是由于order的值在第一次执行后缓存了，append_first和test_string_only引用的order是同一个对象

9. Autouse fixtures (fixtures you don’t have to request)

   使用autouse=True

   import pytest
   
   @pytest.fixture
   def first_entry():
       return "a"
   
   
   @pytest.fixture
   def order(first_entry):
       return []
   
   
   @pytest.fixture(autouse=True)
   def append_first(order, first_entry):
       return order.append(first_entry)
   
   
   def test_string_only(order, first_entry):
       assert order == [first_entry]
   
   
   def test_string_and_int(order, first_entry):
       order.append(2)
       assert order == [first_entry, 2]
   
   

10. scope

    # content of conftest.py
    import smtplib
    import pytest
    
    
    @pytest.fixture(scope="module")
    def smtp_connection():
        return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
    

    # content of test_module.py
    def test_ehlo(smtp_connection):
        response, msg = smtp_connection.ehlo()
        assert response == 250
        assert b"smtp.gmail.com" in msg
        assert 0  # for demo purposes
    
        def test_noop(smtp_connection):
            response, msg = smtp_connection.noop()
            assert response == 250
            assert 0  # for demo purposes
    

    Fixture scopes Fixtures are created when first requested by a test, and are destroyed based on their scope:

    • function: the default scope, the fixture is destroyed at the end of the test.
    • class: the fixture is destroyed during teardown of the last test in the class.
    • module: the fixture is destroyed during teardown of the last test in the module.
    • package: the fixture is destroyed during teardown of the last test in the package where the fixture is defined, including sub-packages and sub-directories within it.
    • session: the fixture is destroyed at the end of the test session

11. 动态scope

    在某些情况下，您可能希望更改 fixture 的范围而不更改代码。为此，请将 callable 传递给 scope。callable 必须返回一个具有有效范围的字符串，并且只会执行一次 - 在 fixture 定义期间。它将使用两个关键字参数调用 - fixture_name 作为字符串和带有配置对象的 config

    def determine_scope(fixture_name, config):
        if config.getoption("--keep-containers", None):
            return "session"
        return "function"
    
    
    @pytest.fixture(scope=determine_scope)
    def docker_container():
        yield spawn_container()
    

12. Teardown/Cleanup

    存在3中方式

    • yield fixtures （推荐）

      • Return 被换成 Yield
      • fixtures的任何teardown都放在 yield 之后。
      • 一旦 pytest 确定了 fixture 的线性顺序，它将运行每个 fixture 直到它返回或yield，然后继续执行列表中的下一个 fixture 以执行相同的操作
      • 测试执行完成后，pytest 将返回 fixture 列表，但以相反的顺序，获取每个 yield 的 fixture，并运行 yield 语句之后的代码
      • 如果在yield之前fixture发生了异常，pytest 不会尝试在该 yield fixture 的 yield 语句之后运行 teardown 代码。但是对于该test的其余成功fixture，pytest依旧执行tear down
      • yield原理是addfinalizer
      • fixtures尽量保持原子性

      class MailAdminClient:
          def create_user(self):
              return MailUser()
      
          def delete_user(self, user):
              # do some cleanup
              pass
      
      
      class MailUser:
          def __init__(self):
              self.inbox = []
      
          def send_email(self, email, other):
              other.inbox.append(email)
      
          def clear_mailbox(self):
              self.inbox.clear()
      
      
      class Email:
          def __init__(self, subject, body):
              self.subject = subject
              self.body = body
      

      @pytest.fixture
      def mail_admin():
          return MailAdminClient()
      
      
      @pytest.fixture
      def sending_user(mail_admin):
          user = mail_admin.create_user()
          yield user
          mail_admin.delete_user(user)
      
      
      @pytest.fixture
      def receiving_user(mail_admin):
          user = mail_admin.create_user()
          yield user
          user.clear_mailbox()
          mail_admin.delete_user(user)
      
      
      def test_email_received(sending_user, receiving_user):
          email = Email(subject="Hey!", body="How's it going?")
          sending_user.send_email(email, receiving_user)
          assert email in receiving_user.inbox
      

    • addfinalizer

      @pytest.fixture
      def receiving_user(mail_admin, request):
          user = mail_admin.create_user()
      
          def delete_user():
              mail_admin.delete_user(user)
      
          request.addfinalizer(delete_user)
          return user
      

13. 实现多断言

    # contents of tests/end_to_end/test_login.py
    from uuid import uuid4
    from urllib.parse import urljoin
    from selenium.webdriver import Chrome
    import pytest
    from src.utils.pages import LoginPage, LandingPage
    from src.utils import AdminApiClient
    from src.utils.data_types import User
    
    
    @pytest.fixture(scope="class")
    def admin_client(base_url, admin_credentials):
        return AdminApiClient(base_url, **admin_credentials)
    
    
    @pytest.fixture(scope="class")
    def user(admin_client):
        _user = User(name="Susan", username=f"testuser-{uuid4()}", password="P4$$word")
        admin_client.create_user(_user)
        yield _user
        admin_client.delete_user(_user)
    
    
    @pytest.fixture(scope="class")
    def driver():
        _driver = Chrome()
        yield _driver
        _driver.quit()
    
    
    @pytest.fixture(scope="class")
    def landing_page(driver, login):
        return LandingPage(driver)
    
    
    class TestLandingPageSuccess:
        @pytest.fixture(scope="class", autouse=True)
        def login(self, driver, base_url, user):
            driver.get(urljoin(base_url, "/login"))
            page = LoginPage(driver)
            page.login(user)
    
        def test_name_in_header(self, landing_page, user):
            assert landing_page.header == f"Welcome, {user.name}!"
    
        def test_sign_out_button(self, landing_page):
            assert landing_page.sign_out_button.is_displayed()
    
        def test_profile_link(self, landing_page, user):
            profile_href = urljoin(base_url, f"/profile?id={user.profile_id}")
            assert landing_page.profile_link.get_attribute("href") == profile_href
    
    

    每种方法都只需要请求它实际需要的 Fixtures，而不必担心顺序。因为act fixture 是 autouse fixture，并且它在执行之前，执行了所有的其余fixture

    为什么login写在了类里面

    因为不是所有的test都需要登录成功，有些需要登录失败

    class TestLandingPageBadCredentials:
        @pytest.fixture(scope="class")
        def faux_user(self, user):
            _user = deepcopy(user)
            _user.password = "badpass"
            return _user
    
        def test_raises_bad_credentials_exception(self, login_page, faux_user):
            with pytest.raises(BadCredentialsException):
                login_page.login(faux_user)
    

14. Factories as fixtures模式

    fixtures除了可以返回固定的data数据，还可以返回生成数据的方法，该方法会在该test中调用多次

    @pytest.fixture
    def make_customer_record():
        def _make_customer_record(name):
            return {"name": name, "orders": []}
    
        return _make_customer_record
    
    
    def test_customer_records(make_customer_record):
        customer_1 = make_customer_record("Lisa")
        customer_2 = make_customer_record("Mike")
        customer_3 = make_customer_record("Meredith")
    

    If the data created by the factory requires managing, the fixture can take care of that:

    注意看，挺妙的

    @pytest.fixture
    def make_customer_record():
        created_records = []
    
        def _make_customer_record(name):
            record = models.Customer(name=name, orders=[])
            created_records.append(record)
            return record
    
        yield _make_customer_record
        for record in created_records:
            record.destroy()
    
    
    def test_customer_records(make_customer_record):
        customer_1 = make_customer_record("Lisa")
        customer_2 = make_customer_record("Mike")
        customer_3 = make_customer_record("Meredith")
    
    

15. pytest.fixture中的id参数和param参数

    • param参数

      param参数用于定义fixture的参数化值。通过提供多个param值，fixture可以在不同的参数取值下多次运行相关的测试。这对于测试具有多种输入情况的函数或类方法非常有用。

      当test多次使用参数fixture时，每次调用都会遍历params

      在多test调用fixture时，每个参数会在teardown执行之后再生成另一个参数的setup，以保证当前存在最少的活动fixture

           import pytest
      
      
           def add(a, b):
               return a + b
      
      
           @pytest.fixture(params=[(1, 2), (3, 4), (5, 6)])
           def input_numbers(request):
               return request.param
      
      
           def test_add(input_numbers):
               a, b = input_numbers
               result = add(a, b)
               assert result == a + b
      

    • id参数

      在pytest中，当使用参数化的fixture时，id参数用于为每个参数化的实例提供一个唯一的标识符。这个标识符在测试报告中会显示出来，使得测试结果更易于理解和区分。

      当不写ids时，pytest也会自动生成

           import pytest
      
           @pytest.fixture(params=[1, 2, 3], ids=['input1', 'input2', 'input3'])
           def input_value(request):
               return request.param
      

16. pytest.param()用法

    # content of test_fixture_marks.py
    import pytest
    
    
    @pytest.fixture(params=[0, 1, pytest.param(2, marks=pytest.mark.skip)])
    def data_set(request):
        return request.param
    
    
    def test_data(data_set):
        pass
    
    

     $ pytest test_fixture_marks.py-v
     =========================== test session starts ============================
     platform linux-- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y-- $PYTHON_PREFIX/bin/
     →python
     cachedir: .pytest_cache
     rootdir: /home/sweet/project
     collecting ... collected 3 items
     test_fixture_marks.py::test_data[0] PASSED
     test_fixture_marks.py::test_data[1] PASSED
     test_fixture_marks.py::test_data[2] SKIPPED (unconditional skip)
     [ 33%]
     [ 66%]
     [100%]
     ======================= 2 passed, 1 skipped in 0.12s =======================
    

17. pytest.mark.usefixtures

    当test不关注fixture返回的对象时，但是确实要执行一些前置操作，可以使用usefixtures来完成

    import os
    import tempfile
    import pytest
    
    
    @pytest.fixture
    def cleandir():
        with tempfile.TemporaryDirectory() as newpath:
            old_cwd = os.getcwd()
            os.chdir(newpath)
            yield
            os.chdir(old_cwd)
    
    

    # content of test_setenv.py
    import os
    import pytest
    
    
    @pytest.mark.usefixtures("cleandir")
    class TestDirectoryInit:
        def test_cwd_starts_empty(self):
            assert os.listdir(os.getcwd()) == []
            with open("myfile", "w", encoding="utf-8") as f:
                f.write("hello")
    
        def test_cwd_again_starts_empty(self):
            assert os.listdir(os.getcwd()) == []
    
    

    也可以使用另外一种实现方式

     pytestmark = pytest.mark.usefixtures("cleandir")
    

    再在pytest.ini中定义

     # content of pytest.ini
     [pytest]
     usefixtures = cleandir
    

18. fixtures可以被重写，test使用与自己层级更近的那个。或可以根据入参的不同进行区分。

19. pytest中的marks只能适用于tests,而不适用于fixtures

20. 使用marks可以在pytest.ini中定义

    [pytest]
     markers =
     	slow: marks tests as slow (deselect with '-m "not slow"')
     	serial
    

    ：后面的为可选描述

21. pytest中的--strict-markers

    对markers进行严格检查

    • 标记定义检查
      • 当使用--strict - markers选项运行pytest时，它会检查所有在测试用例中使用的标记是否已经被正确定义。如果有未定义的标记被使用，pytest将会报错。例如，如果有一个测试用例使用了@pytest.mark.new_feature，但是在整个项目中并没有对new_feature这个标记进行定义，那么在使用--strict - markers选项运行测试时就会触发错误。
    • 标记使用规范检查
      • 除了检查标记的定义，--strict - markers还可以检查标记的使用是否符合规范。这有助于确保标记在整个项目中的一致性使用，避免因标记使用方式的差异而导致的混淆。例如，如果定义了一个标记应该只用于特定类型的测试函数，--strict - markers可以检查是否有不符合此规定的使用情况。

    如果不使用该参数，不会报错，但会警告

22. xfail(expected to fail)

23. 组合参数

    如果想要组合多个参数，可以叠加使用parameter装饰器

    import pytest
    
    
    @pytest.mark.parametrize("x", [0, 1])
    @pytest.mark.parametrize("y", [2, 3])
    def test_foo(x, y):
        pass
    
    

    This will run the test with the arguments set to x=0/y=2, x=1/y=2, x=0/y=3, and x=1/y=3 exhausting parameters in the order of the decorators

24. 重新运行失败用例

    pytest--lf（last-failed）

    第一次运行50个用例，存在两个失败的

    第二次使用--lf再次执行，会重新执行这2个失败的，其余的48个测试用例不会被收集执行

    pytest--ff（full-failed)

    执行全部的50个用例，但失败的会先执行

25. 日志管理

    可以通过pytest.ini管理

    log_format = %(asctime)s %(levelname)s %(message)s
    log_date_format = %Y-%m-%d %H:%M:%S
    

26. pytest_collection_modifyitems(session, config, items)

    hook fuction，pytest收集全部的items后执行