Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve download #43

Merged
merged 29 commits into from
Dec 1, 2023
Merged

Improve download #43

merged 29 commits into from
Dec 1, 2023

Conversation

agrouaze
Copy link
Member

@agrouaze agrouaze commented Nov 22, 2023 •
edited
Loading

  • test multithread 4 //
  • test multithread >4 //
  • make sure we also can use the sequential simple download
  • add a .tmp or .dl to the file with on-going download
  • try except for multi thread
  • [ ] escape password or login containing quotes (won't fix)
  • separate tokens and active session semaphore files
  • add blacklist feature (to avoid black-hole effect when an account is unusable)
  • add a pre-spool directory feature so that sorting chain won't touch to a file with on-going download
  • fix issue with multi-thread and result that seems to be always identical: it was due to usage of a token that dont belongs to the session account

@agrouaze agrouaze marked this pull request as draft November 22, 2023 14:58
@agrouaze agrouaze self-assigned this Nov 22, 2023
@agrouaze agrouaze added the enhancement New feature or request label Nov 22, 2023
@agrouaze agrouaze linked an issue Nov 22, 2023 that may be closed by this pull request
handle the different code status return by OData API #41
Closed
@agrouaze
Copy link
Member Author

agrouaze commented Nov 22, 2023 •
edited
Loading

close #41 with 69638fa

agrouaze and others added 21 commits November 22, 2023 16:52
take into account for status meaning for multi thread
d11c7b6
add the test for multi threads
7eed84d
@agrouaze
change to be able to handle multi account CDSE and multi threads
4b38e24
@agrouaze
tokens maangmeent
1441563
@agrouaze
test for logins
364dd48
@agrouaze
improve test simple
b433c77
@agrouaze
improve multi thread tests
bb9606e
@agrouaze
add example of listings
917c282
@agrouaze
add active_session_directory: in confi.yml
ef1103c
@agrouaze
fix issue with multi account multi tokens for download, clean osbolet…
a08b003 
… methods
@agrouaze
add securities for token read/write functions
758d1ca
@agrouaze
add session manager in the lib
6e4d145
@agrouaze
improve testing the different account
520f798
@agrouaze
new high level test script for IW
a95a52c
@agrouaze
make broader options for fetch test script
b510e9f
@agrouaze
fix to have multi thread and sequentials methods working
41004a7
@agrouaze
put listings in dataframes
20b5800
@agrouaze
docstring
b73ba56
@agrouaze
introduce capability to switch between groups of logins
bbefcf5
@agrouaze
capability to test the 2 sequentials methods
b8da8b1
@agrouaze
logins groups
7ea225d
This was linked to issues Dec 1, 2023
handle round-robin multiple CDSE account #42
Closed
test multithread downloads #40
Closed
@agrouaze agrouaze marked this pull request as ready for review December 1, 2023 14:16
@agrouaze agrouaze merged commit 252a762 into main Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@agrouaze agrouaze

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
1 participant
@agrouaze