Not fetching new SHiFT codes

[flammable]

I think something is wrong with Orcicorn's website - Gearbox released a new box set of most of the Borderlands games, and has been posting a crazy amount of SHiFT codes, but none of them have appeared on the website.

It looks like it used the Twitter API, which is now basically dead. I'm not sure how to proceed here. I hope this project isn't dead as a result.

If so, this was really great while it lasted, and I really appreciate all of the work you've put into it!

[ugoogalizer]

Oricorn's SHIFTBot is officially dead: https://www.twitlonger.com/show/n_1ss9el4

For now I've forked Fabbi's autoshift and made a couple of changes so I can limp on:

• I'm manually/temporarily populating a json file here with shift codes. This isn't sustainable, but haven't addressed automating that yet. Step one would be finding a way to scrape X The site formally known as Twitter without costing thousands of dollars.
• I've repointed autoshift to the above json
• Fixed bug in autoshift that meant "universal" keys were only actually redeemed for the first platform in the list you pass at runtime
• Fixed bug in autoshift that meant some successful redemptions were marked as previously redeemed at command line (suspect this is due to a change to the https://shift.gearboxsoftware.com/rewards website)
• Added some extra debug output to the code to help me unpick the above two bugs

[alambers]

This is admittedly a crude hack but it works. I've added support for mentalmars.com which is still tracking keys. Apply this diff and you'll be getting keys again. Make adjustments to the platform and games queried as needed. It doesn't tie into the platform/game selections from the command line so a proper integration would be awesome, but it works for me as-is so I won't be doing that. Diff for query.py: 300a301,303

#for k in row.keys(): #print(k, row[k])

402a406,534

def parse_mental_bl2(): import json key_url = "https://mentalmars.com/game-news/borderlands-2-golden-keys/

"

resp = requests.get(key_url) if not resp: _L.error(f"Error querying for new keys: {resp.reason}") return None #data: dict = json.loads(resp.text)[0] pattern =

re.compile(r"((?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?))")

keys = list() for match in re.findall(pattern, resp.text): #print (match) key = Key() key.set(code = match) key.set(game = "bl2") key.set(platform = "psn") key.set(reward = "Golden Keys") key.set(type = "shift") key.set(redeemed = "0") keys.append(key) del key yield from keys def parse_mental_bl3(): import json key_url = "https://mentalmars.com/game-news/borderlands-3-golden-keys/

"

resp = requests.get(key_url) if not resp: _L.error(f"Error querying for new keys: {resp.reason}") return None #data: dict = json.loads(resp.text)[0] pattern =

re.compile(r"((?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?))")

keys = list() for match in re.findall(pattern, resp.text): #print (match) key = Key() key.set(code = match) key.set(game = "bl3") key.set(platform = "psn") key.set(reward = "Golden Keys") key.set(type = "shift") key.set(redeemed = "0") keys.append(key) del key yield from keys def parse_mental_ttw(): import json key_url = "

https://mentalmars.com/game-news/tiny-tinas-wonderlands-shift-codes/"

resp = requests.get(key_url) if not resp: _L.error(f"Error querying for new keys: {resp.reason}") return None #data: dict = json.loads(resp.text)[0] pattern =

re.compile(r"((?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?))")

keys = list() for match in re.findall(pattern, resp.text): #print (match) key = Key() key.set(code = match) key.set(game = "ttw") key.set(platform = "psn") key.set(reward = "Golden Keys") key.set(type = "shift") key.set(redeemed = "0") keys.append(key) del key yield from keys def parse_mental_blps(): import json key_url = "https://mentalmars.com/game-news/bltps-golden-keys/" resp = requests.get(key_url) if not resp: _L.error(f"Error querying for new keys: {resp.reason}") return None #data: dict = json.loads(resp.text)[0] pattern =

re.compile(r"((?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?)(?:-)(?:\w{5}?))")

keys = list() for match in re.findall(pattern, resp.text): #print (match) key = Key() key.set(code = match) key.set(game = "blps") key.set(platform = "psn") key.set(reward = "Golden Keys") key.set(type = "shift") key.set(redeemed = "0") keys.append(key) del key yield from keys

407a540

print("Grabbing keys from orcicorn")

408a542,550

…

print("Grabbing keys from mental bl2") keys.extend(list(parse_mental_bl2())) print("Grabbing keys from mental bl3") keys.extend(list(parse_mental_bl3())) print("Grabbing keys from mental ttw") keys.extend(list(parse_mental_ttw())) print("Grabbing keys from mental blps") keys.extend(list(parse_mental_blps())) On Fri, Sep 22, 2023 at 7:59 AM ugoogalizer ***@***.***> wrote: Oricorn's SHIFTBot is officially dead: https://www.twitlonger.com/show/n_1ss9el4 For now I've forked Fabbi's autoshift and made a couple of changes so I can limp on: - I'm manually/temporarily populating a json file here <https://raw.githubusercontent.com/ugoogalizer/autoshift/master/shiftcodes.json> with shift codes. This isn't sustainable, but haven't addressed automating that yet. Step one would be finding a way to scrape X The site formally known as Twitter without costing thousands of dollars. - I've repointed autoshift to the above json - Fixed bug in autoshift that meant "universal" keys were only actually redeemed for the first platform in the list you pass at runtime - Fixed bug in autoshift that meant some successful redemptions were marked as previously redeemed at command line (suspect this is due to a change to the https://shift.gearboxsoftware.com/rewards website) - Added some extra debug output to the code to help me unpick the above two bugs — Reply to this email directly, view it on GitHub <#53 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHMXQ5LKKGFM47CUQVAVELX3V4RLANCNFSM6AAAAAA47DKYSU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[flammable]

Thank you both. I've been manually redeeming via this Reddit post, but I miss having things happen automatically. 😄

Tagging @Fabbi in case he wants to merge either (or both?) sets of code, so maybe they can be rolled into new Docker images.

[Fabbi]

Well I should've seen that one coming..

We definitely have to find a new source for keys that is not Reddit or Twitter (I refuse to call it "X").
The website @alambers uses seems to be the best bet yet when combined with Orcicorn for all the still active keys from older games. Could you reformat your diff so that it is more readable here?
I'll take a look at the fork of @ugoogalizer and see what I can do when I have the time.

Thanks for your work everyone!

[alambers]

I've attached the diff via github this time. It's not pretty but the core of it is one webpage per game, a regex to gather the keys on the webpage, then appending the keys to the usual query process. Brute-force style.

mentalmars_query.diff.txt

[superxtc]

I've attached the diff via github this time. It's not pretty but the core of it is one webpage per game, a regex to gather the keys on the webpage, then appending the keys to the usual query process. Brute-force style.

mentalmars_query.diff.txt

Can you post the patched file please? It won't apply for me for some reason

[ugoogalizer]

Hey all, I've been writing a parser for mentalmars that would output into the same structure that @Fabbi's autoshift accepts. It's not quite finished yet as I still need to automate the publishing side. First cut of its output from mental mars is here. (I just realised I skipped TTW though).
I haven't figured out where the best place to publish is, so am thinking a dedicated GitHub repo at this stage, but am open to ideas.

Will keep this post updated as I finish.

[alambers]

OK, here's my query.py file as requested.

query.py.txt

[ugoogalizer]

I've built an automated scraper to populate from mentalmars and track required metadata. Currently checking every 2 hours that now automatically updates a json file. There's still a few things to finish, but it's funcitoning well for me should anyone wish to use it too.

I've repointed Fabbi's autoshift to this file and subbmitted PR #54, which also includes the bugfixes I mentioned above.

[ugoogalizer]

Until the PR is merged, updated container image found here: https://hub.docker.com/r/ugoogalizer/autoshift