Skip to content

Update the pid/ppid in transport metadata if they have changed #825

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jun 5, 2020

Conversation

basepi
Copy link
Contributor

@basepi basepi commented May 14, 2020

Related issues

Closes #792

@basepi basepi requested a review from beniwohli May 14, 2020 19:11
@apmmachine
Copy link
Contributor

apmmachine commented May 14, 2020

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #825 updated]

  • Start Time: 2020-06-05T14:39:36.420+0000

  • Duration: 27 min 40 sec

Test stats 🧪

Test Results
Failed 2
Passed 9255
Skipped 6796
Total 16053

Test errors

Expand to view the tests failures

  • Name: Test / windows-3.8-none / test_send_remote_failover_sync[validating_httpserver0-sending_elasticapm_client0] – tests.client.client_tests

    • Age: 1
    • Duration: 1.05
    • Error Details: assert not True + where True = <bound method TransportState.did_fail of <elasticapm.transport.base.TransportState object at 0x00000255A077CAC0>>() + where <bound method TransportState.did_fail of <elasticapm.transport.base.TransportState object at 0x00000255A077CAC0>> = <elasticapm.transport.base.TransportState object at 0x00000255A077CAC0>.did_fail + where <elasticapm.transport.base.TransportState object at 0x00000255A077CAC0> = <elasticapm.transport.http.Transport object at 0x00000255A077CB50>.state + where <elasticapm.transport.http.Transport object at 0x00000255A077CB50> = <elasticapm.base.Client object at 0x00000255A075F190>._transport
  • Name: Test / windows-3.8-none / test_ssl_cert_pinning_http – tests.transports.test_urllib3

    • Age: 1
    • Duration: 0.57
    • Error Details: elasticapm.transport.base.TransportException: Unable to reach APM Server: ('Connection aborted.', ConnectionAbortedError(10053, 'An established connection was aborted by the software in your host machine', None, 10053, None)) (url: http://127.0.0.1:50145)

Log output

Expand to view the last 100 lines of log output

[2020-06-05T15:05:57.164Z]     + built_wheel=(/tmp/built_wheel/*.whl)
[2020-06-05T15:05:57.164Z]     + rm -rf /tmp/repaired_wheels
[2020-06-05T15:05:57.164Z]     + mkdir /tmp/repaired_wheels
[2020-06-05T15:05:57.164Z]     + [[ /tmp/built_wheel/elastic_apm-5.6.0-py2-none-any.whl == *none-any.whl ]]
[2020-06-05T15:05:57.164Z]     + mv /tmp/built_wheel/elastic_apm-5.6.0-py2-none-any.whl /tmp/repaired_wheels
[2020-06-05T15:05:57.164Z]     + repaired_wheels=(/tmp/repaired_wheels/*.whl)
[2020-06-05T15:05:57.164Z]     + '[' '!' -z '' ']'
[2020-06-05T15:05:57.164Z]     + mv /tmp/repaired_wheels/elastic_apm-5.6.0-py2-none-any.whl /output
[2020-06-05T15:05:57.164Z]     + for repaired_wheel in '"${repaired_wheels[@]}"'
[2020-06-05T15:05:57.164Z]      + basename /tmp/repaired_wheels/elastic_apm-5.6.0-py2-none-any.whl
[2020-06-05T15:05:57.164Z]     + chown 1156:1157 /output/elastic_apm-5.6.0-py2-none-any.whl
[2020-06-05T15:05:57.164Z] + docker exec -i cibuildwheel-0215bd52-36d2-42e2-a4b0-49b577883100 sh -c 'cat > /constraints.txt'
[2020-06-05T15:05:57.740Z] + docker exec -i cibuildwheel-0215bd52-36d2-42e2-a4b0-49b577883100 /bin/bash
[2020-06-05T15:05:58.002Z]     + mkdir -p /output
[2020-06-05T15:05:58.002Z]     + cd /project
[2020-06-05T15:05:58.002Z]     + PYBIN=/opt/python/pp36-pypy36_pp73/bin
[2020-06-05T15:05:58.002Z]     + export PATH=/opt/python/pp36-pypy36_pp73/bin:/opt/rh/devtoolset-8/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[2020-06-05T15:05:58.002Z]     + PATH=/opt/python/pp36-pypy36_pp73/bin:/opt/rh/devtoolset-8/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[2020-06-05T15:05:58.002Z]      + which pip
[2020-06-05T15:05:58.002Z]     + '[' /opt/python/pp36-pypy36_pp73/bin/pip '!=' /opt/python/pp36-pypy36_pp73/bin/pip ']'
[2020-06-05T15:05:58.002Z]      + which python
[2020-06-05T15:05:58.002Z]     + '[' /opt/python/pp36-pypy36_pp73/bin/python '!=' /opt/python/pp36-pypy36_pp73/bin/python ']'
[2020-06-05T15:05:58.002Z]     + '[' '!' -z '' ']'
[2020-06-05T15:05:58.002Z]     + rm -rf /tmp/built_wheel
[2020-06-05T15:05:58.002Z]     + mkdir /tmp/built_wheel
[2020-06-05T15:05:58.002Z]     + pip wheel /project/. -w /tmp/built_wheel --no-deps
[2020-06-05T15:06:00.556Z] Processing /project
[2020-06-05T15:06:01.504Z]   Installing build dependencies: started
[2020-06-05T15:06:05.737Z]   Installing build dependencies: finished with status 'done'
[2020-06-05T15:06:05.737Z]   Getting requirements to build wheel: started
[2020-06-05T15:06:07.172Z]   Getting requirements to build wheel: finished with status 'done'
[2020-06-05T15:06:07.172Z]     Preparing wheel metadata: started
[2020-06-05T15:06:08.624Z]     Preparing wheel metadata: finished with status 'done'
[2020-06-05T15:06:08.624Z] Building wheels for collected packages: elastic-apm
[2020-06-05T15:06:08.624Z]   Building wheel for elastic-apm (PEP 517): started
[2020-06-05T15:06:10.615Z]   Building wheel for elastic-apm (PEP 517): finished with status 'done'
[2020-06-05T15:06:10.615Z]   Created wheel for elastic-apm: filename=elastic_apm-5.6.0-py3-none-any.whl size=447397 sha256=a151a39e4bc2736bc4e189164a66cda8a2237b11bb3c3c8ef3c8146694a2f985
[2020-06-05T15:06:10.615Z]   Stored in directory: /tmp/pip-ephem-wheel-cache-r62angzc/wheels/de/ab/88/a237fdf750e25f63db6cf5343e3feef89c5f93e985be647379
[2020-06-05T15:06:10.615Z] Successfully built elastic-apm
[2020-06-05T15:06:10.924Z] WARNING: You are using pip version 20.0.2; however, version 20.1.1 is available.
[2020-06-05T15:06:10.924Z] You should consider upgrading via the '/opt/pypy/pypy3.6-7.3.1/bin/pypy -m pip install --upgrade pip' command.
[2020-06-05T15:06:10.924Z]     + built_wheel=(/tmp/built_wheel/*.whl)
[2020-06-05T15:06:10.924Z]     + rm -rf /tmp/repaired_wheels
[2020-06-05T15:06:10.924Z]     + mkdir /tmp/repaired_wheels
[2020-06-05T15:06:10.924Z]     + [[ /tmp/built_wheel/elastic_apm-5.6.0-py3-none-any.whl == *none-any.whl ]]
[2020-06-05T15:06:10.924Z]     + mv /tmp/built_wheel/elastic_apm-5.6.0-py3-none-any.whl /tmp/repaired_wheels
[2020-06-05T15:06:10.924Z]     + repaired_wheels=(/tmp/repaired_wheels/*.whl)
[2020-06-05T15:06:10.924Z]     + '[' '!' -z '' ']'
[2020-06-05T15:06:10.924Z]     + mv /tmp/repaired_wheels/elastic_apm-5.6.0-py3-none-any.whl /output
[2020-06-05T15:06:10.924Z]     + for repaired_wheel in '"${repaired_wheels[@]}"'
[2020-06-05T15:06:10.924Z]      + basename /tmp/repaired_wheels/elastic_apm-5.6.0-py3-none-any.whl
[2020-06-05T15:06:10.924Z]     + chown 1156:1157 /output/elastic_apm-5.6.0-py3-none-any.whl
[2020-06-05T15:06:10.924Z] + docker cp cibuildwheel-0215bd52-36d2-42e2-a4b0-49b577883100:/output/. /var/lib/jenkins/workspace/thon_apm-agent-python-mbp_PR-825/src/github.com/elastic/apm-agent-python/wheelhouse
[2020-06-05T15:06:10.924Z] + docker rm --force -v cibuildwheel-0215bd52-36d2-42e2-a4b0-49b577883100
[2020-06-05T15:06:11.530Z] cibuildwheel-0215bd52-36d2-42e2-a4b0-49b577883100
[2020-06-05T15:06:11.530Z] + ls -l wheelhouse
[2020-06-05T15:06:11.530Z] total 12720
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 497013 Jun  5 15:04 elastic_apm-5.6.0-cp27-cp27m-manylinux1_i686.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 502855 Jun  5 15:03 elastic_apm-5.6.0-cp27-cp27m-manylinux1_x86_64.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 497016 Jun  5 15:04 elastic_apm-5.6.0-cp27-cp27m-manylinux2010_i686.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 502857 Jun  5 15:03 elastic_apm-5.6.0-cp27-cp27m-manylinux2010_x86_64.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 497020 Jun  5 15:04 elastic_apm-5.6.0-cp27-cp27mu-manylinux1_i686.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 502845 Jun  5 15:03 elastic_apm-5.6.0-cp27-cp27mu-manylinux1_x86_64.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 497020 Jun  5 15:04 elastic_apm-5.6.0-cp27-cp27mu-manylinux2010_i686.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 502847 Jun  5 15:03 elastic_apm-5.6.0-cp27-cp27mu-manylinux2010_x86_64.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 500526 Jun  5 15:04 elastic_apm-5.6.0-cp35-cp35m-manylinux1_i686.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 506241 Jun  5 15:03 elastic_apm-5.6.0-cp35-cp35m-manylinux1_x86_64.whl
[2020-06-05T15:06:11.530Z] -rw-r--r-- 1 jenkins jenkins 500528 Jun  5 15:04 elastic_apm-5.6.0-cp35-cp35m-manylinux2010_i686.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 506244 Jun  5 15:03 elastic_apm-5.6.0-cp35-cp35m-manylinux2010_x86_64.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 500774 Jun  5 15:05 elastic_apm-5.6.0-cp36-cp36m-manylinux1_i686.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 506483 Jun  5 15:03 elastic_apm-5.6.0-cp36-cp36m-manylinux1_x86_64.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 500777 Jun  5 15:05 elastic_apm-5.6.0-cp36-cp36m-manylinux2010_i686.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 506486 Jun  5 15:03 elastic_apm-5.6.0-cp36-cp36m-manylinux2010_x86_64.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 501703 Jun  5 15:05 elastic_apm-5.6.0-cp37-cp37m-manylinux1_i686.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 507418 Jun  5 15:03 elastic_apm-5.6.0-cp37-cp37m-manylinux1_x86_64.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 501707 Jun  5 15:05 elastic_apm-5.6.0-cp37-cp37m-manylinux2010_i686.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 507421 Jun  5 15:03 elastic_apm-5.6.0-cp37-cp37m-manylinux2010_x86_64.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 506352 Jun  5 15:05 elastic_apm-5.6.0-cp38-cp38-manylinux1_i686.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 511742 Jun  5 15:04 elastic_apm-5.6.0-cp38-cp38-manylinux1_x86_64.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 506355 Jun  5 15:05 elastic_apm-5.6.0-cp38-cp38-manylinux2010_i686.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 511743 Jun  5 15:04 elastic_apm-5.6.0-cp38-cp38-manylinux2010_x86_64.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 447389 Jun  5 15:05 elastic_apm-5.6.0-py2-none-any.whl
[2020-06-05T15:06:11.531Z] -rw-r--r-- 1 jenkins jenkins 447397 Jun  5 15:06 elastic_apm-5.6.0-py3-none-any.whl
[2020-06-05T15:06:12.818Z] Stashed 26 file(s)
[2020-06-05T15:06:14.011Z] Scheduling project: APM Integration Test MBP Selector » master
[2020-06-05T15:06:14.029Z] [INFO] Can not determine redirect link!!!
[2020-06-05T15:06:15.088Z] Stage "Benchmarks" skipped due to when conditional
[2020-06-05T15:06:15.129Z] Timeout set to expire in 12 hr
[2020-06-05T15:06:15.180Z] Stage "Prepare Release" skipped due to when conditional
[2020-06-05T15:06:15.198Z] Stage "Prepare Release" skipped due to when conditional
[2020-06-05T15:06:15.232Z] Stage "Prepare Release" skipped due to when conditional
[2020-06-05T15:06:15.267Z] Stage "Prepare Release" skipped due to when conditional
[2020-06-05T15:06:15.497Z] Running on Jenkins in /var/lib/jenkins/workspace/thon_apm-agent-python-mbp_PR-825
[2020-06-05T15:06:15.597Z] [INFO] getVaultSecret: Getting secrets
[2020-06-05T15:06:15.670Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-06-05T15:06:16.328Z] + chmod 755 generate-build-data.sh
[2020-06-05T15:06:16.328Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-825/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-825/runs/20 UNSTABLE 1599647
[2020-06-05T15:06:16.878Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-825/runs/20/steps/?limit=10000 -o steps-info.json
[2020-06-05T15:06:19.361Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-825/runs/20/tests/?status=FAILED -o tests-errors.json
[2020-06-05T15:06:20.705Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-825/runs/20/log/ -o pipeline-log.txt

@basepi
Copy link
Contributor Author

basepi commented May 26, 2020

@beniwohli this could still use a review when you have a moment.

Copy link
Contributor

@beniwohli beniwohli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but maybe we can simplify a bit

A test for this would also be great :)

basepi added 2 commits June 3, 2020 14:20
Also made build_metadata a public method in the client
@basepi basepi requested a review from beniwohli June 3, 2020 20:21
@basepi
Copy link
Contributor Author

basepi commented Jun 3, 2020

@beniwohli I realized when writing the test that if we generate metadata at start_thread time, we would lose any metadata explicitly set in the Transport kwargs. This was problematic for testing, but also begged the question: why would we bother generating it at all before we create the Transport, when it's going to be immediately overwritten when we start the thread?

So now we store any metadata that is sent into the Transport as self._metadata_kwarg, and any time we build metadata, we update the result with that dictionary to retain those overrides.

This is a much larger-scale change now, and I'm interested in your opinion.

Copy link
Contributor

@beniwohli beniwohli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, I like the idea, but I'm struggling to find a use case for overriding/modifying the metadata per transport. Did you have something in mind?

Copy link
Contributor

@beniwohli beniwohli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@basepi
Copy link
Contributor Author

basepi commented Jun 5, 2020

Just flakey tests failing. Merging.

@basepi basepi merged commit c9c290c into elastic:master Jun 5, 2020
romulorosa pushed a commit to romulorosa/apm-agent-python that referenced this pull request Oct 15, 2020
…ic#825)

* Update the pid/ppid in transport metadata if they have changed

* Only update the process metadata if it already exists

* Fix os.getppid for windows python 2.7

* Add to changelog

* Rebuild metadata on pid change

Also made build_metadata a public method in the client

* Generate metadata only during start_thread

* Remove metadata kwarg from transport completely

* Remove metadata :param
romulorosa pushed a commit to romulorosa/apm-agent-python that referenced this pull request Oct 15, 2020
…ic#825)

* Update the pid/ppid in transport metadata if they have changed

* Only update the process metadata if it already exists

* Fix os.getppid for windows python 2.7

* Add to changelog

* Rebuild metadata on pid change

Also made build_metadata a public method in the client

* Generate metadata only during start_thread

* Remove metadata kwarg from transport completely

* Remove metadata :param
beniwohli pushed a commit to beniwohli/apm-agent-python that referenced this pull request Sep 14, 2021
…ic#825)

* Update the pid/ppid in transport metadata if they have changed

* Only update the process metadata if it already exists

* Fix os.getppid for windows python 2.7

* Add to changelog

* Rebuild metadata on pid change

Also made build_metadata a public method in the client

* Generate metadata only during start_thread

* Remove metadata kwarg from transport completely

* Remove metadata :param
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Process pid get master pid
3 participants