Skip to content

Conversation

@pombredanne
Copy link
Member

There are a few issues with the Ubuntu Oval importer that was stuck.
This resolves some of these issues:

  • Do not fail when there is a version range that cannot be parsed as a RangeSpecifier().
  • Improve error handling
  • Add simple logging using print statements to help monitor import progress
  • Also apply minor code formatting.
  • I also added a code comment about the Etag handling which IMHO are more problematic than helpful at this stage, especially when there is a failure during import. See also Disable Etag #321

Signed-off-by: Philippe Ombredanne pombredanne@nexb.com

@pombredanne
Copy link
Member Author

Note that I will need to add tests before we can merge and also the Ubuntu importer still does NOT complete after running for a day.

@pombredanne pombredanne requested a review from sbs2001 January 29, 2021 13:36
@sbs2001
Copy link
Collaborator

sbs2001 commented Jan 29, 2021

@pombredanne Any luck with completing the whole import ?

@pombredanne
Copy link
Member Author

@pombredanne Any luck with completing the whole import ?

@sbs2001 I stopped it after several hours and about ~8000 completed (e.g. after the first archive)

@sbs2001
Copy link
Collaborator

sbs2001 commented Jan 29, 2021

@sbs2001
Copy link
Collaborator

sbs2001 commented Feb 12, 2021

@pombredanne I think we should ditch pycodestyle and use black for both checking + formatting like aboutcode-org/skeleton#13 (comment)

pombredanne and others added 14 commits March 4, 2021 17:12
Correct ABOUT file

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
We are not using etag correctly. Also the function name is misleading
as we are checking the Etag in this create_etag function

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This importer no longer worked with failures to process some version
ranges as RangeSpecifier(). These change imprivae error handling and
crude logging using print statements for now.
Also apply minor code formatting.

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This is an "external" library that contains non-passing doctest-like
docs.

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This makes it easy to reproduce errors by copy/pasting the faulty
command.

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This is triggered when using black-like style

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
…allback mapping

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
sbs2001 added 2 commits March 4, 2021 17:18
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
@sbs2001 sbs2001 merged commit e4246aa into main Mar 4, 2021
@sbs2001 sbs2001 deleted the improve-ubuntu branch March 12, 2021 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants