Coverage Run unit tests first before docker containers are set up#1055
Coverage Run unit tests first before docker containers are set up#1055kevinjqliu merged 1 commit intoapache:mainfrom
Conversation
9f86746 to
349c9b6
Compare
349c9b6 to
244f635
Compare
sungwy
left a comment
There was a problem hiding this comment.
Thank you @Minfante377 for this PR, and @youssef-itanii as well!
kevinjqliu
left a comment
There was a problem hiding this comment.
Thanks for the contribution! Added a comment about testing this method and some nits
Makefile
Outdated
| docker compose -f dev/docker-compose-integration.yml cp ./dev/provision.py spark-iceberg:/opt/spark/provision.py | ||
| docker compose -f dev/docker-compose-integration.yml exec -T spark-iceberg ipython ./provision.py | ||
| poetry run coverage run --source=pyiceberg/ -m pytest tests/ ${PYTEST_ARGS} | ||
| poetry run coverage run --data-file=.coverage.integration --source=pyiceberg/ -m pytest tests/ -m integration ${PYTEST_ARGS} |
There was a problem hiding this comment.
nit: to match the other command, also added -v to match test-integration
| poetry run coverage run --data-file=.coverage.integration --source=pyiceberg/ -m pytest tests/ -m integration ${PYTEST_ARGS} | |
| poetry run coverage run --source=pyiceberg/ --data-file=.coverage.integration -m pytest tests/ -v -m integration ${PYTEST_ARGS} |
| assert len(batches) == entries | ||
|
|
||
|
|
||
| @pytest.mark.integration |
There was a problem hiding this comment.
Does CI fail if we don't add this line? It's what we're testing for in #1051
There was a problem hiding this comment.
It fails indeed. If I don't add that marker the test-coverage-unit command will try to run that test without initializing any docker container
244f635 to
09b0298
Compare
kevinjqliu
left a comment
There was a problem hiding this comment.
LGTM!
Verified that unit test fails
#1060
https://github.com/apache/iceberg-python/actions/runs/10380009878/job/28739139875?pr=1060
=========================== short test summary info ============================
FAILED tests/integration/test_reads.py::test_table_scan_default_to_large_types[session_catalog_hive] - thrift.transport.TTransport.TTransportException: Could not connect to any of [('::1', 9083, 0, 0), ('127.0.0.1', 9083)]
ERROR tests/integration/test_reads.py::test_table_scan_default_to_large_types[session_catalog] - requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8181): Max retries exceeded with url: /v1/config (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2b1351e0d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
=== 1 failed, 2781 passed, 834 deselected, 1113 warnings, 1 error in 54.83s ====
This PR implements:
Should close #1051