Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented MysqlFileManager #3

Merged
merged 9 commits into from
Feb 26, 2019
Merged

Implemented MysqlFileManager #3

merged 9 commits into from
Feb 26, 2019

Conversation

lwj5
Copy link
Member

@lwj5 lwj5 commented Feb 14, 2019

Added an implementation of filemanager using MySQL for recording and indexing and file system to store files.

@lwj5 lwj5 added the enhancement New feature or request label Feb 14, 2019
@lwj5 lwj5 self-assigned this Feb 14, 2019
@lwj5 lwj5 requested a review from sitfoxfly February 14, 2019 06:09
@coveralls
Copy link

coveralls commented Feb 14, 2019

Pull Request Test Coverage Report for Build 113

  • 262 of 403 (65.01%) changed or added relevant lines in 16 files are covered.
  • 2 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-0.1%) to 74.749%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/main/java/ai/preferred/venom/fetcher/AsyncResponseConsumer.java 2 3 66.67%
src/main/java/ai/preferred/venom/job/PriorityQueueScheduler.java 1 2 50.0%
src/main/java/ai/preferred/venom/Crawler.java 2 4 50.0%
src/main/java/ai/preferred/venom/response/StorageResponse.java 9 11 81.82%
src/main/java/ai/preferred/venom/storage/FileManagerCallback.java 9 11 81.82%
src/main/java/ai/preferred/venom/request/StorageFetcherRequest.java 12 16 75.0%
src/main/java/ai/preferred/venom/fetcher/StorageFetcher.java 53 59 89.83%
src/main/java/ai/preferred/venom/storage/DummyFileManager.java 23 33 69.7%
src/main/java/ai/preferred/venom/storage/MysqlFileManager.java 87 200 43.5%
Files with Coverage Reduction New Missed Lines %
src/main/java/ai/preferred/venom/fetcher/AsyncResponseConsumer.java 1 83.33%
src/main/java/ai/preferred/venom/job/AbstractQueueScheduler.java 1 63.16%
Totals Coverage Status
Change from base Build 97: -0.1%
Covered Lines: 1042
Relevant Lines: 1394

💛 - Coveralls

@sitfoxfly
Copy link
Member

Only test phase requires mysql-connector-java dependency, right?

@lwj5
Copy link
Member Author

lwj5 commented Feb 22, 2019

No, it is needed to use MysqlFileManager, the user can override it if they require a different version.
For now we will consolidate and include the storage dependencies. If the project grows or if we require other implementations of FileManager, we will move to a multi-module maven project.

@lwj5 lwj5 merged commit 0ad77a5 into master Feb 26, 2019
@lwj5 lwj5 deleted the storage branch February 26, 2019 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants