Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refresh wiki input data periodically #54

Open
Akronix opened this issue Oct 15, 2018 · 0 comments
Open

Refresh wiki input data periodically #54

Akronix opened this issue Oct 15, 2018 · 0 comments

Comments

@Akronix
Copy link
Collaborator

Akronix commented Oct 15, 2018

For the very long term, it could be nice to have a way to refresh data dumps, ideally in a incremental fashion. Currently, dumps have to be downloaded, parsed and uploaded and that data get outdated with the past of the time.

The best approach will be to have a smart DB and some periodical scripts that add to that database the downloaded data. Something like InfluxDB can serve as an inspiration.

Another simpler approach is to just have some scripts that periodically download the data, processes it, upload to the DATA_DIR folder and restart WikiChron automatically. Note that it can be tricky to make them incremental and that downloading and processing large wikis can take a lot of time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant