Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix micro package pull from PyPI #1848

Merged
merged 9 commits into from
Sep 30, 2022
Merged

Fix micro package pull from PyPI #1848

merged 9 commits into from
Sep 30, 2022

Conversation

FlorianGD
Copy link
Contributor

@FlorianGD FlorianGD commented Sep 12, 2022 •
edited
Loading

Description

This PR fixes a bug when installing micro packages from PyPI

Development notes

The way the package name was computed relied on the assumption that the package path could compute the name of the folder where the egg info file is. This is not true if the package is downloaded from PyPI

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes

Fix #1811

@FlorianGD
Fix micro package pull from PyPI
6573694 
Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
@FlorianGD
Add note to release.md
6973e28 
Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
@FlorianGD
fix tests for micropkg pull from PyPI
608f7d1 
Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
@FlorianGD FlorianGD marked this pull request as ready for review September 12, 2022 16:26
@FlorianGD
Copy link
Contributor Author

FlorianGD commented Sep 19, 2022 •
edited
Loading

I believe the failing tests came after the merge of main. They seem Windows related, which I cannot easily test locally. The weird thing is that it passed on windows for python 3.9.

I am not sure what I should do.

Edit: solved now

@FlorianGD
Merge branch 'main' into master
4daa6d0
@FlorianGD
Merge branch 'main' into master
d553010
@FlorianGD
update RELEASE.md
9b21026 
Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
@SajidAlamQB
Update RELEASE.md
8ffafc8 
Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
SajidAlamQB
SajidAlamQB reviewed Sep 26, 2022
View reviewed changes
Copy link
Contributor

@SajidAlamQB SajidAlamQB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up, your solution looks good! I am thinking, do we need to update some of the other PyPI tests as well?

@FlorianGD
Copy link
Contributor Author

I do not think other tests need to be updated. The tests regarding PyPI are:

  • test_invalid_pull_from_pypi
  • test_pull_from_pypi_more_than_one_sdist_file
  • test_pull_from_pypi

The first 2 should fail before reaching the changed part of the code (in _unpack_sdist). The last one is updated.

merelcht
merelcht approved these changes Sep 26, 2022
View reviewed changes
Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for investigating and fixing this @FlorianGD ! 👍

I wonder if we could have caught this earlier.. I guess our assumption that package name on PyPi has the version was wrong looking at the test change?

@FlorianGD
Copy link
Contributor Author

Yes, I think that this assumption was indeed incorrect. Maybe having an integration test on https://test.pypi.org would have caught it also?

@merelcht merelcht merged commit e2cf980 into kedro-org:main Sep 30, 2022
AhdraMeraliQB pushed a commit that referenced this pull request Oct 21, 2022
@FlorianGD
Fix micro package pull from PyPI (#1848)
57384cc 
Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
nickolasrm pushed a commit to ProjetaAi/kedro that referenced this pull request Oct 26, 2022
@FlorianGD @nickolasrm
Fix micro package pull from PyPI (kedro-org#1848)
9467300 
Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
Signed-off-by: nickolasrm <nickolasrochamachado@gmail.com>
AhdraMeraliQB added a commit that referenced this pull request Nov 9, 2022
@AhdraMeraliQB @noklam
@jstammers @merelcht @FlorianGD @ankatiyar @yetudada @jmholzer @yash6318 @deepyaman @carlaprv @dinotuku @SajidAlamQB @dependabot
Validate code snippets in datasets documentation - Part 1 (#1962)
f52249b 
* Release/0.18.3 (#1856)

* Update release version and release notes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update missing release notes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* update vresion

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* update release notes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove comment from code example

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove more comments

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add YAML formatting

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add missing import

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove even more comments

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove more even more comments

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add pickle requirement to extras_require

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Try fix YAML docs

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Try fix YAML docs pt 2

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Fix code snippets in docs (#1876)

* Fix code snippets

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Separate code blocks

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Lint

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Fix issue with specifying format for SparkHiveDataSet (#1857)

Signed-off-by: jstammers <jimmy.stammers@cgastrategy.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update RELEASE.md (#1883)

* Update RELEASE.md

* fix broken link

* Update RELEASE.md

Co-authored-by: Merel Theisen <49397448+MerelTheisenQB@users.noreply.github.com>

Co-authored-by: Merel Theisen <49397448+MerelTheisenQB@users.noreply.github.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Deprecate `kedro test` and `kedro lint` (#1873)

* Deprecating `kedro test` and `kedro lint`

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Deprecate commands

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Make kedro looks prettier

* Update Linting

Signed-off-by: Nok <nok_lam_chan@mckinsey.com>

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok_lam_chan@mckinsey.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Fix micro package pull from PyPI (#1848)

Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update Error message for `VersionNotFoundError` to handle Permission related issues better (#1881)

* Update message for VersionNotFoundError

Signed-off-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>

* Add test for VersionNotFoundError for cloud protocols

* Update test_data_catalog.py

Update NoVersionFoundError test

* minor linting update

* update docs link + styling changes

* Revert "update docs link + styling changes"

This reverts commit 6088e00.

* Update test with styling changes

* Update RELEASE.md

Signed-off-by: ankatiyar <ankitakatiyar2401@gmail.com>

Signed-off-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Signed-off-by: ankatiyar <ankitakatiyar2401@gmail.com>
Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update experiment tracking documentation with working examples (#1893)

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add NHS AI Lab and ReSpo.Vision to companies list (#1878)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Document how users can use pytest instead of kedro test (#1879)

* Add best_practices.md with introductory sections

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add pytest and pytest-cov sections

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add pytest-cov coverage report

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add sections on pytest-cov

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add automated_testing to index.rst

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Reformat third-party library names and clean grammar.

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add link to virtual environment docs

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add example of good test naming

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Improve link accessibility

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Improve pytest docs link accessibility

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add reminder link to virtual environment docs

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix formatting in link to coverage docs

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove reference to /src under 'Run your tests'

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Modify references to <project_name> to <package_name>

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix sentence structure

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix broken databricks doc link

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Capitalise Kedro-Viz in the "Visualize layers" section (#1899)

* Capitalised kedro-viz

Signed-off-by: yash6318 <yash.agrawal.cse21@iitbhu.ac.in>

* capitalised Kedro viz

Signed-off-by: yash6318 <yash.agrawal.cse21@iitbhu.ac.in>

* Updated set_up_experiment_tracking.md

Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: yash6318 <yash.agrawal.cse21@iitbhu.ac.in>

Signed-off-by: yash6318 <yash.agrawal.cse21@iitbhu.ac.in>
Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Fix linting on autmated test page (#1906)

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add _SINGLE_PROCESS property to CachedDataSet (#1905)

Signed-off-by: Carla Vieira <carlaprv@hotmail.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update the tutorial of "Visualise pipelines" (#1913)

* Change a file extention to match the previous article

Signed-off-by: dinotuku <kuan.tung@epfl.ch>

* Add a missing import

Signed-off-by: dinotuku <kuan.tung@epfl.ch>

* Change both preprocessed datasets to parquet files

Signed-off-by: dinotuku <kuan.tung@epfl.ch>

* Change data type to ParquetDataSet for parquet files

Signed-off-by: dinotuku <kuan.tung@epfl.ch>

* Add a note for installing seaborn if it is not installed

Signed-off-by: dinotuku <kuan.tung@epfl.ch>

Signed-off-by: dinotuku <kuan.tung@epfl.ch>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Document how users can use linting tools instead of `kedro lint` (#1904)

* Add documentation for linting tools

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Revert changes to commands_reference.md

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update linting docs with suggestions

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update linting doc

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Make core config accessible in dict get way  (#1870)

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Create dependabot.yml configuration file for version updates (#1862)

* Create dependabot.yml configuration file

* Update dependabot.yml

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add target-branch

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update dependabot.yml

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* limit dependabot to just dependency folder

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update test_requirements.txt

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update MANIFEST.in

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* fix e2e

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update continue_config.yml

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update requirements.txt

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update requirements.txt

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* fix link

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* revert

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Delete requirements.txt

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update dependabot config (#1928)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update robots.txt (#1929)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* fix broken link (#1950)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update dependabot.yml config  (#1938)

* Update dependabot.yml

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin jupyterlab_services to requirments

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* lint

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update setup.py Jinja2 dependencies (#1954)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update pip-tools requirement from ~=6.5 to ~=6.9 in /dependency (#1957)

Updates the requirements on [pip-tools](https://github.com/jazzband/pip-tools) to permit the latest version.
- [Release notes](https://github.com/jazzband/pip-tools/releases)
- [Changelog](https://github.com/jazzband/pip-tools/blob/master/CHANGELOG.md)
- [Commits](jazzband/pip-tools@6.5.0...6.9.0)

---
updated-dependencies:
- dependency-name: pip-tools
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update toposort requirement from ~=1.5 to ~=1.7 in /dependency (#1956)

Updates the requirements on [toposort]() to permit the latest version.

---
updated-dependencies:
- dependency-name: toposort
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sajid Alam <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add deprecation warning to package_name argument in session create() (#1953)

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove redundant `resolve_load_version` call (#1911)

* remove a redundant function call

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Remove redundant resolove_load_version & fix test

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix HoloviewWriter tests with more specific error message pattern & Lint

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Rename tests

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Make docstring in test starter match real starters (#1916)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Try to fix formatting error

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>

* Specify pickle import

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Signed-off-by: jstammers <jimmy.stammers@cgastrategy.com>
Signed-off-by: Nok <nok_lam_chan@mckinsey.com>
Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
Signed-off-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Signed-off-by: ankatiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Signed-off-by: yash6318 <yash.agrawal.cse21@iitbhu.ac.in>
Signed-off-by: Carla Vieira <carlaprv@hotmail.com>
Signed-off-by: dinotuku <kuan.tung@epfl.ch>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Nok <mediumnok@gmail.com>
Co-authored-by: Jimmy Stammers <jimmy.stammers@gmail.com>
Co-authored-by: Merel Theisen <49397448+MerelTheisenQB@users.noreply.github.com>
Co-authored-by: Florian Gaudin-Delrieu <9217921+FlorianGD@users.noreply.github.com>
Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>
Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Co-authored-by: Yash Agrawal <96697569+yash6318@users.noreply.github.com>
Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Co-authored-by: Carla Vieira <carlaprv@hotmail.com>
Co-authored-by: Kuan Tung <kuan.tung@epfl.ch>
Co-authored-by: Sajid Alam <90610031+SajidAlamQB@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Merel Theisen <merel.theisen@quantumblack.com>
@antonymilne antonymilne mentioned this pull request Feb 1, 2023
Closed
@astrojuanlu astrojuanlu mentioned this pull request May 8, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@merelcht merelcht merelcht approved these changes

@SajidAlamQB SajidAlamQB SajidAlamQB approved these changes

@idanov idanov Awaiting requested review from idanov

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

kedro micropkg pull from pypi is not working in 0.18.2
3 participants
@FlorianGD @merelcht @SajidAlamQB