Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various "minor" changes (4) #1435

Merged
merged 21 commits into from
Jun 20, 2023
Merged

Conversation

Hans5958
Copy link
Member

@Hans5958 Hans5958 commented Apr 25, 2023

Sequel to #1432. This is yet another set of changes. I suggest viewing the commits tab to see (and check) the changes made. Anyone is welcome to test it, as well as giving feedback.

Some highlights:

  • Add schema-based validation (7398fd3)
    This is only used on the validation workflow due to the relatively long time it take. This could be added on aformatter.py if wanted to.
  • Use tqdm for progress bar, and write directly (it made it way faster) (0a23227)
  • Use -1 as placeholder ID for new entries (689007a)
    0 could meant as the very first entry, index-wise, whereas -1 have a more obvious meaning of an invalid ID.
  • Per-entry patches (cdd21f9)
    See Various "minor" changes (4) #1435 (comment)

@Hans5958 Hans5958 requested a review from a team as a code owner April 25, 2023 06:24
@netlify
Copy link

netlify bot commented Apr 25, 2023

Deploy Preview for place-atlas ready!

Name Link
🔨 Latest commit 414ee14
🔍 Latest deploy log https://app.netlify.com/sites/place-atlas/deploys/6491d38f5f93a700084d668e
😎 Deploy Preview https://deploy-preview-1435--place-atlas.netlify.app/
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@Hans5958 Hans5958 force-pushed the live-love-life/4 branch 6 times, most recently from 019cb28 to 689007a Compare April 25, 2023 06:53
@Hans5958 Hans5958 force-pushed the live-love-life/4 branch from 689007a to daa5213 Compare May 12, 2023 06:13
@Codixer
Copy link
Member

Codixer commented May 16, 2023

Atlas.json conflict @Hans5958, please resolve.

Copy link
Member

@Codixer Codixer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meep

@Hans5958
Copy link
Member Author

Wilco. I am quite busy so expect in around 24h.

@Hans5958 Hans5958 force-pushed the live-love-life/4 branch from daa5213 to 0f5b442 Compare May 17, 2023 15:35
@Hans5958 Hans5958 requested a review from Codixer June 1, 2023 06:46
@Hans5958 Hans5958 force-pushed the live-love-life/4 branch from eeebc19 to 9d38f4c Compare June 1, 2023 06:46
@github-actions

This comment was marked as outdated.

@github-actions

This comment was marked as outdated.

@Hans5958
Copy link
Member Author

Hans5958 commented Jun 15, 2023

An experiemental of per-entry patches has been implemented to reduce dependency on Reddit, especially on creating new entries.

Contributors now can submit patches by creating separate .json files on data/patches for each submission. For redditcrawl.py, instead generating a temporary JSON file, it will create the relevant patches instead. These patches work independently (except the cases on conflicts to which fewer manual intervention have to be done), and merging would be easy enough by using merge_out.py.

Contributors can submit patches by creating a new .json patch on data/patches, or use the included tool, create_patch.py, that assists the script user on creating a patch.

This is inspired from how changeset works on merging the information.

This gives the following advantages:

  • Contributors can update the Atlas data through GitHub, and only one person have to merge it to the main Atlas data. All of this is done without dealing with the magnitude of potential conflicts that have to be solved if everyone has to change the singular, main atlas.json.
  • Numerical IDs can be used again. This is either a pro or a con. I saw this as a pro, but in any way this is a better way to support both sources, next to a generating meaningless, random string.
  • Reducing the dependency of Reddit. Even though we can somehow work the way through the Reddit API, some people didn't want to use Reddit as a boycott, or just for the fact that the subreddit is now closed for a while. With this, people can use multiple avenues, all without creating a overhead to the developers/staff members.

TODO

  • Add info to CONTRIBUTING.md (how to create patches, how to merge, etc)
  • Add info on "Create New Entry" modal
  • Use the position index instead of the id for sorting by Oldest and Newest

@github-actions

This comment was marked as outdated.

3 similar comments
@github-actions

This comment was marked as outdated.

@github-actions

This comment was marked as outdated.

@github-actions

This comment was marked as outdated.

@github-actions

This comment was marked as outdated.

2 similar comments
@github-actions

This comment was marked as outdated.

@github-actions

This comment was marked as outdated.

@Hans5958 Hans5958 requested review from a team and removed request for AnonymousRandomPerson June 16, 2023 11:36
@github-actions
Copy link

github-actions bot commented Jun 18, 2023

Qodana for JS

It seems all right 👌

No new problems were found according to the checks applied
💡 Qodana analysis was run in the pull request mode: only the changed files were checked

View the detailed Qodana report

To be able to view the detailed Qodana report, you can either:

  1. Register at Qodana Cloud and configure the action
  2. Use GitHub Code Scanning with Qodana
  3. Host Qodana report at GitHub Pages
  4. Inspect and use qodana.sarif.json (see the Qodana SARIF format for details)
Contact Qodana team

Contact us at qodana-support@jetbrains.com

Hans5958 added 21 commits June 20, 2023 23:24
Preventing a potential risk of having 0 as the first ever entry, while -1 have a meaning of none
For testing
Instead of one temporary JSON file, contributors now can submit patches in form of separate files per entry, that will be merged by `merge_out.py` without dealing with the potential conflicts to the main `atlas.json`.
Continue when patch errors instead of ending the whole process
Usually I would use Pushshift, but we don't have it anymore. This is done by seeking the subreddit's timeline and using the search feature. Hopefully this covers all of it.

153 (+ me) new contributors, thank you!
@Codixer Codixer merged commit 5992be6 into placeAtlas:master Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants