Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue: @test decorator adds to pytest output an extra test as well as a warning #867

Open
mobiware opened this issue Jul 12, 2024 · 2 comments

Comments

@mobiware
Copy link

Issue you'd like to raise.

If we decorate one of our tests with the @test decorator, we notice in pytest output several issues:

  1. there is an extra test. For instance pytest output says collected 10 items even though we only have 9 test cases
    When looking at verbose pytest output (using the -vv flag) we see this extra test:
project/integration_tests/test_chain.py::test <- ../../../../../../Library/Caches/pypoetry/virtualenvs/darwin-ai-langchain-b0bmgt7a-py3.10/lib/python3.10/site-packages/langsmith/_testing.py PASSED [ 10%]
  1. this extra test seems to trigger a pytest warning:
================================================================================================================ warnings summary =================================================================================================================
project/integration_tests/test_chain.py::test
  /Users/name/Library/Caches/pypoetry/virtualenvs/darwin-ai-langchain-b0bmgt7a-py3.10/lib/python3.10/site-packages/_pytest/python.py:166: PytestReturnNotNoneWarning: Expected None, but project/integration_tests/test_chain.py::test returned <function test.<locals>.decorator at 0x101922b00>, which will be an error in a future version of pytest.  Did you mean to use `assert` instead of `return`?
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

Suggestion:

Might be due to the decorator name, since pytest collects test function by matching with the test prefix in their names ?

@mobiware
Copy link
Author

An easy way to remove the warning and extra test is by renaming langsmith.test import so that it doesn't start with test, i.e.:

from langsmith import test as ls_test

@ls_test
def test_chain(input, output):
    assert ...

@hinthornw
Copy link
Collaborator

good catch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants