Skip to content

Feat/create test from report #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Feat/create test from report #73

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4d8a951
implemented option to create test on report
AenEnlil Oct 8, 2025
4d8531c
implemented WORKDIR option
AenEnlil Oct 8, 2025
1c2aad0
updated README.md
AenEnlil Oct 9, 2025
62abd9e
bump: version 2.10.2 -> 2.10.3b3
AenEnlil Oct 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 11 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,19 +186,21 @@ You can use environment variable to control certain features of testomat.io


#### Test Run configuration
| Env variable | What it does | Examples |
|--------------------------|----------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
| TESTOMATIO_TITLE | Name of a test run to create on testomat.io | TESTOMATIO_TITLE="Nightly Smoke Tests" pytest --testomatio report |
| TESTOMATIO_RUN_ID | Id of existing test run to use for sending test results to | TESTOMATIO_RUN_ID=98dfas0 pytest --testomatio report |
| TESTOMATIO_RUNGROUP_TITLE | Create a group (folder) for a test run. If group already exists, attach test run to it | TESTOMATIO_RUNGROUP_TITLE="Release 2.0" pytest --testomatio report |
| TESTOMATIO_ENV | Assign environment to a test run, env variant of **testRunEnv** option. Has a lower precedence than **testRunEnv** option. | TESTOMATIO_ENV="linux,chrome,1920x1080" pytest --testomatio report |
| TESTOMATIO_LABEL | Assign labels to a test run. Labels must exist in project and their scope must be enabled for runs | TESTOMATIO_LABEL="smoke,regression" pytest --testomatio report |
| TESTOMATIO_UPDATE_CODE | Send code of your test to Testomat.io on each run. If not enabled(default) assumes the code is pushed using **sync** command | TESTOMATIO_UPDATE_CODE=True pytest --testomatio report |
| TESTOMATIO_EXCLUDE_SKIPPED | Exclude skipped tests from the report | TESTOMATIO_EXCLUDE_SKIPPED=1 pytest --testomatio report |
| Env variable | What it does | Examples |
|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------|
| TESTOMATIO_TITLE | Name of a test run to create on testomat.io | TESTOMATIO_TITLE="Nightly Smoke Tests" pytest --testomatio report |
| TESTOMATIO_RUN_ID | Id of existing test run to use for sending test results to | TESTOMATIO_RUN_ID=98dfas0 pytest --testomatio report |
| TESTOMATIO_RUNGROUP_TITLE | Create a group (folder) for a test run. If group already exists, attach test run to it | TESTOMATIO_RUNGROUP_TITLE="Release 2.0" pytest --testomatio report |
| TESTOMATIO_ENV | Assign environment to a test run, env variant of **testRunEnv** option. Has a lower precedence than **testRunEnv** option. | TESTOMATIO_ENV="linux,chrome,1920x1080" pytest --testomatio report |
| TESTOMATIO_LABEL | Assign labels to a test run. Labels must exist in project and their scope must be enabled for runs | TESTOMATIO_LABEL="smoke,regression" pytest --testomatio report |
| TESTOMATIO_UPDATE_CODE | Send code of your test to Testomat.io on each run. If not enabled(default) assumes the code is pushed using **sync** command | TESTOMATIO_UPDATE_CODE=True pytest --testomatio report |
| TESTOMATIO_EXCLUDE_SKIPPED | Exclude skipped tests from the report | TESTOMATIO_EXCLUDE_SKIPPED=1 pytest --testomatio report |
| TESTOMATIO_PUBLISH | Publish run after reporting and provide a public URL | TESTOMATIO_PUBLISH=true pytest --testomatio report |
| TESTOMATIO_PROCEED | Do not finalize the run | TESTOMATIO_PROCEED=1 pytest --testomatio report |
| TESTOMATIO_SHARED_RUN | Report parallel execution to the same run matching it by title. If the run was created more than 20 minutes ago, a new run will be created instead. | TESTOMATIO_TITLE="Run1" TESTOMATIO_SHARED_RUN=1 pytest --testomatio report |
| TESTOMATIO_SHARED_RUN_TIMEOUT | Changes timeout of shared run. After timeout, shared run won`t accept other runs with same name, and new runs will be created. Timeout is set in minutes, default is 20 minutes. | TESTOMATIO_TITLE="Run1" TESTOMATIO_SHARED_RUN=1 TESTOMATIO_SHARED_RUN_TIMEOUT=10 pytest --testomatio report |
| TESTOMATIO_CREATE | Create test which are not yet exist in a project | TESTOMATIO_CREATE=1 pytest --testomatio report |
| TESTOMATIO_WORKDIR | Specify a custom working directory for relative file paths in test reports. When tests are created with **TESTOMATIO_CREATE=1**, file paths will be relative to this directory. | TESTOMATIO_WORKDIR=new_dir pytest --testomatio report |


#### S3 Bucket configuration
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ version_provider = "pep621"
update_changelog_on_bump = false
[project]
name = "pytestomatio"
version = "2.10.2"
version = "2.10.3b3"


dependencies = [
Expand Down
4 changes: 4 additions & 0 deletions pytestomatio/connect/connector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@ def update_test_status(self, run_id: str,
rid: str,
status: str,
title: str,
create: bool | None,
suite_title: str,
suite_id: str,
test_id: str,
Expand All @@ -197,12 +198,15 @@ def update_test_status(self, run_id: str,
steps: str,
code: str,
example: dict,
file: str | None,
overwrite: bool | None,
meta: dict) -> None:

request = {
"status": status, # Enum: "passed" "failed" "skipped"
"title": title,
"create": create,
"file": file,
"suite_title": suite_title,
"suite_id": suite_id,
"test_id": test_id,
Expand Down
9 changes: 9 additions & 0 deletions pytestomatio/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import os, pytest, logging, json, time
import warnings
from os.path import join, normpath

from pytest import Parser, Session, Config, Item, CallInfo
from pytestomatio.connect.connector import Connector
Expand Down Expand Up @@ -171,7 +172,9 @@ def pytest_runtest_makereport(item: Item, call: CallInfo):
request = {
'status': None,
'title': test_item.exec_title,
'create': None,
'run_time': call.duration,
'file': None,
'suite_title': test_item.suite_title,
'suite_id': None,
'test_id': test_id,
Expand All @@ -190,6 +193,12 @@ def pytest_runtest_makereport(item: Item, call: CallInfo):
request['code'] = test_item.source_code
request['overwrite'] = True

if pytest.testomatio.test_run_config.create_tests:
request['create'] = True
workdir = pytest.testomatio.test_run_config.workdir
if workdir:
request['file'] = normpath(join(workdir, test_item.file_name))

# TODO: refactor it and use TestItem setter to upate those attributes
if call.when in ['setup', 'call']:
if call.excinfo is not None:
Expand Down
3 changes: 3 additions & 0 deletions pytestomatio/testomatio/testRunConfig.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ class TestRunConfig:
def __init__(self):
run_id = os.environ.get('TESTOMATIO_RUN_ID') or os.environ.get('TESTOMATIO_RUN')
title = os.environ.get('TESTOMATIO_TITLE') if os.environ.get('TESTOMATIO_TITLE') else 'test run at ' + dt.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
create_tests = os.environ.get('TESTOMATIO_CREATE', False) in ['True', 'true', '1']
shared_run = os.environ.get('TESTOMATIO_SHARED_RUN') in ['True', 'true', '1']
update_code = os.environ.get('TESTOMATIO_UPDATE_CODE', False) in ['True', 'true', '1']
exclude_skipped = os.environ.get('TESTOMATIO_EXCLUDE_SKIPPED', False) in ['True', 'true', '1']
Expand All @@ -19,6 +20,7 @@ def __init__(self):
self.title = title
self.environment = safe_string_list(os.environ.get('TESTOMATIO_ENV'))
self.exclude_skipped = exclude_skipped
self.create_tests = create_tests if create_tests else None
self.label = safe_string_list(os.environ.get('TESTOMATIO_LABEL'))
self.group_title = os.environ.get('TESTOMATIO_RUNGROUP_TITLE')
# This allows to report tests to the test run by it's id. https://docs.testomat.io/getting-started/running-automated-tests/#reporting-parallel-tests
Expand All @@ -29,6 +31,7 @@ def __init__(self):
self.proceed = os.getenv('TESTOMATIO_PROCEED', False)
self.status_request = {}
self.update_code = update_code
self.workdir = os.getenv('TESTOMATIO_WORKDIR', None)
self.build_url = self.resolve_build_url()
self.meta = self.update_meta()

Expand Down
12 changes: 10 additions & 2 deletions tests/test_connect/test_connector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,7 @@ def test_update_test_status_success(self, mock_post, connector):
rid="rid123",
status="passed",
title="Test Login",
create=True,
suite_title="Auth Suite",
suite_id="suite_456",
test_id="test_789",
Expand All @@ -268,7 +269,8 @@ def test_update_test_status_success(self, mock_post, connector):
code="def test_login(): pass",
example={"param": "value"},
overwrite=True,
meta={}
meta={},
file='file',
)

assert mock_post.call_count == 1
Expand All @@ -277,6 +279,8 @@ def test_update_test_status_success(self, mock_post, connector):
assert f'{connector.base_url}/api/reporter/run_123/testrun' in call_args[0][0]

payload = call_args[1]['json']
assert payload['create']
assert payload['file']
assert payload['status'] == 'passed'
assert payload['title'] == 'Test Login'
assert payload['run_time'] == 1.5
Expand All @@ -296,6 +300,7 @@ def test_update_test_status_filters_none_values(self, mock_post, connector):
run_id="run_123",
status="passed",
title="Test Login",
create=None,
suite_title="Auth Suite",
suite_id="suite_456",
test_id="test_789",
Expand All @@ -308,7 +313,8 @@ def test_update_test_status_filters_none_values(self, mock_post, connector):
example={"param": "value"},
overwrite=None,
rid=None,
meta=None
meta=None,
file=None,
)

assert mock_post.call_count == 1
Expand All @@ -322,6 +328,8 @@ def test_update_test_status_filters_none_values(self, mock_post, connector):
assert payload['run_time'] == 1.5
assert payload['artifacts'] == ["screenshot.png"]
assert 'message' not in payload
assert 'create' not in payload
assert 'file' not in payload
assert 'code' not in payload
assert 'overwrite' not in payload

Expand Down
Loading
Loading