Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query url parameters #878

Merged
merged 3 commits into from
Oct 22, 2020
Merged

Conversation

WithoutPants
Copy link
Collaborator

I found a need to manipulate the values that are fed into queryURL when using sceneByFragment. This change introduces queryURLReplace section which contains a map of placeholder fields (filename, oshash, checksum, title) to regex replacement lists.

I'm using this to enable scraping from filename on the Voyeurweb scraper. It's files are named xxx-yyy.mp4 and only the xxx part is used in the query URL. See the following sceneByFragment section:

  queryURLReplace:
    filename:
      - regex: ^.*[/\\](\d*)-\d*\..*
        with: $1

@WithoutPants WithoutPants added the improvement Something needed tweaking. label Oct 21, 2020
@WithoutPants WithoutPants added this to the Version 0.4.0 milestone Oct 21, 2020
@bnkai
Copy link
Collaborator

bnkai commented Oct 21, 2020

This looks/works ok
It makes it possible to simplify the JavLibrary scraper by using

  queryURL: https://www.javlibrary.com/en/vl_searchbyid.php?keyword={filename}
  queryURLReplace:
    filename:
      - regex: (.*[^a-zA-Z0-9])*([a-zA-Z0-9]+-\d+).*
        with: $2

to search for movie ids.

@WithoutPants WithoutPants merged commit 109e55a into stashapp:develop Oct 22, 2020
Tweeticoats pushed a commit to Tweeticoats/stash that referenced this pull request Feb 1, 2021
@WithoutPants WithoutPants deleted the query-url-params branch February 4, 2021 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Something needed tweaking.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants