Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration with database version #45

Merged
merged 14 commits into from
Aug 30, 2022
Merged

Integration with database version #45

merged 14 commits into from
Aug 30, 2022

Conversation

aramoto99
Copy link
Collaborator

  • Internal control by the file system has been transferred to the database.
  • All contents stored in the work directory are managed in the database.
    They are stored in work/storage/storage.db.
  • The database uses sqlite3.
  • Database codes are in aiaccel/storage.

Note: The changes in this update are vast.

aramoto99 and others added 14 commits July 4, 2022 16:17
@aramoto99
Data management function by database is added.
55e7856 
By adding the `--fs` option argument to the startup command, data management functions by the conventional file system can be used.
Extensive refactoring was performed around the addition of this functionality.
@aramoto99
Update setup.py
c018c1f
@aramoto99
Merge branch 'main' of https://github.com/aistairc/aiaccel into devel…
f762acb 
…op/database
@aramoto99
Merge branch 'main' of github:aistairc/aiaccel into develop/database
9cd4b86
@aramoto99
Merge branch 'main' of https://github.com/aistairc/aiaccel into devel…
c10b929 
…op/database
@aramoto99
Delete tests/test_data/work/state/ as it is not used in develop/datab…
d1f6a82 
…ase.
@KanaiYuma-aist
Develop/database fix test (#18)
3f18217 
* fix for tests

* delete space

* pInfoDict -> p_info
@aramoto99 @tkokada
@KanaiYuma-aist @yoshipon
Merge main into debelop/database (#31)
ebaeafb 
* Fix #19 (#20)

* Fix #21 (#22)

* fix README (#23)

* fix codestyle test (#26)

* fix for pycodestyle

* add flake8==4.0.1

* Feature/update readme (#25)

* update README

* add bash

Co-authored-by: Yoshiaki Bando <yoshipon@users.noreply.github.com>

* Fix resumption test for DB

Co-authored-by: Takashi OKADA <tk.okada.github@gmail.com>
Co-authored-by: KanaiYuma-aist <105629713+KanaiYuma-aist@users.noreply.github.com>
Co-authored-by: Yoshiaki Bando <yoshipon@users.noreply.github.com>
@aramoto99 @tkokada @KanaiYuma-aist
Feature/database (#36)
3511f87 
* Fix #19 (#20)

* Fix #21 (#22)

* fix README (#23)

* Fixed a problem in which Snapshot could not be saved during parallel execution.

* Add function of delete any data in DB/FS.

* Fix code style

Co-authored-by: Takashi OKADA <tk.okada.github@gmail.com>
Co-authored-by: KanaiYuma-aist <105629713+KanaiYuma-aist@users.noreply.github.com>
@aramoto99 @tkokada @KanaiYuma-aist
Request to merge cumulative changes . (#37)
c4e48fa 
* Fix #19 (#20)

* Fix #21 (#22)

* fix README (#23)

* The report generation function can now be called by command.

* Fix data inconsistency when excute resume. (#39)

* Fix resume

* Fixed problem with sirialize not working in fs mode

* Remove unused package imports

* Fixed a problem that the resume function did not work in fs mode.

* The name workspace/state is the name of the old specification, so it was changed to the appropriate name workspace/snapshot.

* Delete unnecessary directories in Workspace.

* Delete unnecessary directories in Workspace.

* Delete unnecessary directories in Workspace.

* Delete unnecessary directories in Workspace.

Co-authored-by: Takashi OKADA <tk.okada.github@gmail.com>
Co-authored-by: KanaiYuma-aist <105629713+KanaiYuma-aist@users.noreply.github.com>
@aramoto99 @tkokada
@KanaiYuma-aist @yoshipon
Add dynamic loading of optimizer, And fix resumtion test. (#42)
24a7401 
* Fix #19 (#20)

* Fix #21 (#22)

* fix README (#23)

* fix codestyle test (#26)

* fix for pycodestyle

* add flake8==4.0.1

* Feature/update readme (#25)

* update README

* add bash

Co-authored-by: Yoshiaki Bando <yoshipon@users.noreply.github.com>

* Fix TPE resumtion test (#29)

* Fix tpe resumption test

* fix tpe resumption test

* Fix tep resumption test

* add retry (#38)

* Add dynamic loading of optimizer (#34)

* (WIP) Add dynamic loading of optimizer by importlib.

* (WIP) Add dynamic loading of optimizer by importlib.

* Add dynamic loading of optimizer by importlib

* Add dynamic loading of optimizer by importlib

* Fix test code

* fix test

* Fix resumtioin test

* Add dynamic loading of optimizer

* Fix dynamic loading of optimizer

* Function names and comments were rolled back.

* Changed the standard optimizer specification method

* changed the specification method for importing the standard optimizer.

* change deserialize (#40)

Co-authored-by: aramoto99 <aramoto-masahiro@aist.go.jp>

* Merge branch 'main' of github:aistairc/aiaccel into feature/database

* Fix codestyle

* (WIP) Change TPE resume

* Fix resume test

* Feature/fix resume tpe (#41)

* fix resume tpe

* fix resume tpe

* add if len(running_trials) > 0

* fix about trial number

* Fix TPE resumption

Co-authored-by: Takashi OKADA <tk.okada.github@gmail.com>
Co-authored-by: KanaiYuma-aist <105629713+KanaiYuma-aist@users.noreply.github.com>
Co-authored-by: Yoshiaki Bando <yoshipon@users.noreply.github.com>
@aramoto99 @tkokada
@KanaiYuma-aist @yoshipon
Fix conflicts with main branch (#43)
c89ce05 
* Fix #19 (#20)

* Fix #21 (#22)

* fix README (#23)

* fix codestyle test (#26)

* fix for pycodestyle

* add flake8==4.0.1

* Feature/update readme (#25)

* update README

* add bash

Co-authored-by: Yoshiaki Bando <yoshipon@users.noreply.github.com>

* Fix TPE resumtion test (#29)

* Fix tpe resumption test

* fix tpe resumption test

* Fix tep resumption test

* add retry (#38)

* Add dynamic loading of optimizer (#34)

* (WIP) Add dynamic loading of optimizer by importlib.

* (WIP) Add dynamic loading of optimizer by importlib.

* Add dynamic loading of optimizer by importlib

* Add dynamic loading of optimizer by importlib

* Fix test code

* fix test

* Fix resumtioin test

* Add dynamic loading of optimizer

* Fix dynamic loading of optimizer

* Function names and comments were rolled back.

* Changed the standard optimizer specification method

* changed the specification method for importing the standard optimizer.

* change deserialize (#40)

Co-authored-by: aramoto99 <aramoto-masahiro@aist.go.jp>

* Merge branch 'main' of github:aistairc/aiaccel into feature/database

* Fix codestyle

* (WIP) Change TPE resume

* Fix resume test

* Feature/fix resume tpe (#41)

* fix resume tpe

* fix resume tpe

* add if len(running_trials) > 0

* fix about trial number

* Fix TPE resumption

Co-authored-by: Takashi OKADA <tk.okada.github@gmail.com>
Co-authored-by: KanaiYuma-aist <105629713+KanaiYuma-aist@users.noreply.github.com>
Co-authored-by: Yoshiaki Bando <yoshipon@users.noreply.github.com>
@aramoto99
Merge branch 'develop/database' of github:aistairc/aiaccel into featu…
5893f3b 
…re/patch_main
@aramoto99
Fix codestyle
168b407
@github-actions
Copy link

Unit Test Results

355 tests  +117   355 ✔️ +117   1m 9s ⏱️ +30s
    1 suites ±    0       0 💤 ±    0 
    1 files   ±    0       0 ❌ ±    0 

Results for commit 168b407. ± Comparison against base commit 8102997.

yoshipon
yoshipon approved these changes Aug 30, 2022
View reviewed changes
Copy link
Collaborator

@yoshipon yoshipon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Let's merge first :p

@yoshipon yoshipon merged commit 77519b2 into main Aug 30, 2022
@yoshipon yoshipon deleted the feature/patch_main branch August 30, 2022 01:35
@aramoto99 aramoto99 mentioned this pull request Aug 30, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yoshipon yoshipon yoshipon approved these changes

@tkokada tkokada Awaiting requested review from tkokada

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@aramoto99 @yoshipon @KanaiYuma-aist