Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: app_abfallplus_de no results #1989

Closed
6 of 7 tasks
ITTV-tools opened this issue Apr 18, 2024 · 75 comments
Closed
6 of 7 tasks

[Bug]: app_abfallplus_de no results #1989

ITTV-tools opened this issue Apr 18, 2024 · 75 comments

Comments

@ITTV-tools
Copy link

I Have A Problem With:

A specific source

What's Your Problem

I want to use de.k4systems.awbrastatt. I tried several citys. Muggensturm, Bühl, Bühlertal. But the log always says the region was not found.

I tried serveral config. With City and street and without street. But always the same error.

Source (if relevant)

No response

Logs

fetch failed for source Apps by Abfall+: Traceback (most recent call last): File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch entries = self._source.fetch() ^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/app_abfallplus_de.py", line 140, in fetch for d in self._app.generate_calendar(): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 786, in generate_calendar self.select_kommune() File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 502, in select_kommune raise Exception(f"Region {self._region_search} not found.") Exception: Region Muggensturm not found.

Relevant Configuration

waste_collection_schedule:
    sources:
    - name: app_abfallplus_de
      args:
        app_id: de.k4systems.awbrastatt
        city: Muggensturm

Checklist Source Error

  • Use the example parameters for your source (often available in the documentation) (don't forget to restart Home Assistant after changing the configuration)
  • Checked that the website of your service provider is still working
  • Tested my attributes on the service provider website (if possible)
  • I have tested with the latest version of the integration (master) (for HACS in the 3 dot menu of the integration click on "Redownload" and choose master as version)

Checklist Sensor Error

  • Checked in the Home Assistant Calendar tab if the event names match the types names (if types argument is used)

Required

  • I have searched past (closed AND opened) issues to see if this bug has already been reported, and it hasn't been.
  • I understand that people give their precious time for free, and thus I've done my very best to make this problem as easy as possible to investigate.
@ITTV-tools
Copy link
Author

I can find the citys in the app

@aklausing
Copy link

Hi. I have the same problem. It worked a week ago and suddenly all schedules are gone and the log states:

2024-04-18 22:19:46.575 ERROR (SyncWorker_3) [waste_collection_schedule.source_shell] fetch failed for source Apps by Abfall+:
Traceback (most recent call last):
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch
    entries = self._source.fetch()
              ^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/app_abfallplus_de.py", line 140, in fetch
    for d in self._app.generate_calendar():
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 760, in generate_calendar
    self.select_kommune()
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 500, in select_kommune
    raise Exception(f"Region {self._region_search} not found.")
Exception: Region Bassum not found.

Tried downgrading to 1.46.0 ... no luck.

@SuperMasterPhoenix
Copy link

SuperMasterPhoenix commented Apr 18, 2024

And also the same issue here for de.k4systems.abfallapploe:

2024-04-18 22:42:30.382 ERROR (SyncWorker_8) [waste_collection_schedule.source_shell] fetch failed for source Apps by Abfall+:
Traceback (most recent call last):
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch
    entries = self._source.fetch()
              ^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/app_abfallplus_de.py", line 140, in fetch
    for d in self._app.generate_calendar():
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 786, in generate_calendar
    self.select_kommune()
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 502, in select_kommune
    raise Exception(f"Region {self._region_search} not found.")
Exception: Region <XYZ> not found.

City, street and house number are defined exactly as in the app which worked until today morning.
HA version: 2024.4.3
Waste collection version: 1.47.0

@fatalyzm
Copy link

fatalyzm commented Apr 18, 2024

same here, HA version: 2024.4.3, Waste collection version: 1.47.0

2024-04-18 23:31:21.459 ERROR (SyncWorker_6) [waste_collection_schedule.source_shell] fetch failed for source Apps by Abfall+:
Traceback (most recent call last):
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch
    entries = self._source.fetch()
              ^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/app_abfallplus_de.py", line 140, in fetch
    for d in self._app.generate_calendar():
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 786, in generate_calendar
    self.select_kommune()
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 502, in select_kommune
    raise Exception(f"Region {self._region_search} not found.")
Exception: Region XXXXX not found.
- name: app_abfallplus_de
  args:
    app_id: de.k4systems.meinawblm

@Alek19901
Copy link

Alek19901 commented Apr 19, 2024

Same here, HA Version: 2024.4.3, Waste collection Version 1.47.0

fetch failed for source Apps by Abfall+: Traceback (most recent call last): File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch entries = self._source.fetch() ^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/app_abfallplus_de.py", line 140, in fetch for d in self._app.generate_calendar(): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 786, in generate_calendar self.select_kommune() File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 502, in select_kommune raise Exception(f"Region {self._region_search} not found.") Exception: Region Thierstein not found. 
  - name: app_abfallplus_de
    args:
      app_id: de.k4systems.kufiapp

@SvenJ76
Copy link

SvenJ76 commented Apr 19, 2024

Same here:
waste_collection_schedule:
sources:
- name: app_abfallplus_de
args:
app_id: de.k4systems.abfallappwug
city: Haundorf
bezirk: Obererlbach

I opened a separate Bugreport by mistake, I will close it right now.

@Anubis666FFW
Copy link

Same on app_id: de.abfallwecker

@crshvrstl
Copy link

Same problem
app_id: de.k4systems.leipziglk
city: Naunhof
bezirk: Naunhof

HA Version 2024.4.3 --> it seems to be a problem with the new HA version

@SuperMasterPhoenix
Copy link

HA Version 2024.4.3 --> it seems to be a problem with the new HA version

I don't think so. My update to 2024.4.3. was done in 12th of April and waste collection schedule was working fine with this version until yesterday. And I didn't do any update on that time.

@ITTV-tools
Copy link
Author

ITTV-tools commented Apr 19, 2024

I use the ICS file form this Website now. Looks like the are using abfall.io
https://www.awb-landkreis-rastatt.de/abfallkalender

waste_collection_schedule:
  sources:
    - name: ics
      args:
        url: https://api.abfall.io/?key=<key>&mode=export&idhousenumber=<number>&wastetypes=576,573,20,66,157,572&timeperiod={%Y}0101-{%Y}1231&showinactive=false&type=ics

@ReneNulschDE
Copy link
Contributor

I have created a PR to fix this. Looks like the servers are probing for the user-agent now. Tested with

    - name: app_abfallplus_de
      args:
        app_id: de.k4systems.awbrastatt
        city: Muggensturm
        strasse: Adlergasse

@SvenJ76
Copy link

SvenJ76 commented Apr 19, 2024

@ReneNulschDE Perfect! I tried your code and works for me too...

@5ila5
Copy link
Collaborator

5ila5 commented Apr 19, 2024

Fixed by #1993 (thanks @ReneNulschDE)

This will be part of release 1.48.0 or is available now as master (git) version (3-dot menu -> redownload -> change version to master -> fully restart HA (this will increase update cycle by a lot)

@daSchizo
Copy link

daSchizo commented Apr 19, 2024

Unfortunately it still doesn't work for me.

  • name: app_abfallplus_de
    args:
    app_id: de.idcontor.abfallwbd
    city: Duisburg

But the error hast changed:

Logger: waste_collection_schedule.source_shell
Quelle: custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py:136
Integration: waste_collection_schedule (Dokumentation)
Erstmals aufgetreten: 22:11:06 (1 Vorkommnisse)
Zuletzt protokolliert: 22:11:06

fetch failed for source Apps by Abfall+: Traceback (most recent call last): File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch entries = self._source.fetch() ^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/app_abfallplus_de.py", line 140, in fetch for d in self._app.generate_calendar(): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 795, in generate_calendar self.select_kommune() File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 493, in select_kommune regions = self.get_kommunen() ^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 471, in get_kommunen r.raise_for_status() File "/usr/local/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://app.abfallplus.de/assistent/kommune/

@ReneNulschDE
Copy link
Contributor

ReneNulschDE commented Apr 20, 2024

Correct, only solved for some of the apps. Needs more time to analyze the new flow.

Analysis so far:

  1. The apps have specific User-Agents.
    Tested with "LK Leipzig", when I change the user agent to "LK Leipzig/9.1.0.0 iOS/17.5 Device/iPhone Screen/1170x2532" the "Brandis" test case is working
  2. The user id has changed from guid to random_hex(48)

@promufa
Copy link

promufa commented Apr 20, 2024

Thanks @ReneNulschDE for your efforts. Much like the comment above from @daSchizo, this is still not working for me.
I had that as a Q&A thread (#1991) is there a way to merge that here?

@schmidt-mai-marcel
Copy link

Hello. I have the same problem. Service is unknown since April 17, 2024.
My values ​​for Waste Collection Schedule:

waste_collection_schedule:
sources:
- name: app_abfallplus_de
args:
app_id: de.k4systems.abfalllkbz
city: Hoyerswerda
bezirk: WK VIII

Thank you for your support..

@ReneNulschDE
Copy link
Contributor

ReneNulschDE commented Apr 20, 2024

Saturday evening update, all test cases except "Bonn orange" are working, I have added some testcases based on the input in this issue. I will prepare a new PR now. (Bonn orange needs to be checked, maybe a different bin type)

Testing source app_abfallplus_de ...
  found 36 entries for de.albagroup.app Braunschweig Hauptstraße 7A  
  de.k4systems.bonnorange Auf dem Hügel failed: Street 'Auf dem Hügel' not found., available: []
  found 192 entries for de.ucom.abfallavr Brühl Habichtstr. 4A
  found 66 entries for de.k4systems.abfallappwug Bergen hauptstr. 1
  found 109 entries for de.k4systems.abfallappcux Wurster Nordseeküste Aakweg Alle Hausnummern
  found 57 entries for de.abfallwecker Mutzschen, Am Lindigt 1
  found 66 entries for de.k4systems.leipziglk Brandis Brandis
  found 91 entries for de.k4systems.lkgoettingen, Abfallwirtschaft Altkreis Göttingen,  Adelebsen, Alle Straßen
  found 65 entries for de.k4systems.zakb Fürth Ahornweg 3
  found 289 entries for de.k4systems.avea Leverkusen Haberstr.
  found 52 entries for de.k4systems.abfallappog Bad Peterstal-Griesbach alle Straßen
  found 79 entries for de.k4systems.abfallappfuerth Großhabersdorf Am Dürren Grund 1 a
  found 87 entries for de.k4systems.awbgp Bad Boll Ahornstraße Alle Hausnummern
  found 157 entries for de.k4systems.abfalllkbz Hoyerswerda bezirk: WK VIII
  found 77 entries for de.idcontor.abfallwbd Duisburg, Rahm Am Junkersknappen 6
  found 141 entries for de.k4systems.awbrastatt Muggensturm Adlergasse

@promufa
Copy link

promufa commented Apr 20, 2024

Confirmed working again for de.k4systems.abfallappbh.
Thanks @ReneNulschDE

@ReneNulschDE
Copy link
Contributor

Additional note: Bonn Orange needs the parameter "city" with the first letter of the street as value.

Testing source app_abfallplus_de ...
  found 97 entries for de.k4systems.bonnorange Auf dem Hügel

@threadstone83
Copy link

Since yesterday I have the same problem for Nüdlingen. Here my configuration:

waste_collection_schedule:
    sources:
    - name: app_abfallplus_de
      args:
        app_id: de.k4systems.abfallscout
        city: Nüdlingen
        bezirk: Nüdlingen
        strasse: Alle Straßen

I already did a reload with changing to master, but this didn't help.

@ReneNulschDE
Copy link
Contributor

You have to wait until the PR #2001 got merged into the master branch and then check again. I tested your config in my env and it is working.

@threadstone83
Copy link

threadstone83 commented Apr 21, 2024

Sounds good. Thank you very much!
Any idea when it is merged / how long it will last?

@mabla7
Copy link

mabla7 commented Apr 22, 2024

Thank you @ReneNulschDE

Tried to use de.k4systems.abfallinfoapp with city Hellenthal, but still get
401 Client Error: Unauthorized for url: https://app.abfallplus.de/assistent/kommune/

@chma73
Copy link

chma73 commented Apr 23, 2024

@ReneNulschDE: tested your fixed code, also worked for me :-) thanks, hope you are now the cat ;-)

@5ila5
Copy link
Collaborator

5ila5 commented Apr 23, 2024

Just merged the fix. tested: as mentioned by @ReneNulschDE

waste_collection_schedule:
    sources:
    - name: app_abfallplus_de
      args:
        app_id: de.albagroup.app
        landkreis: Oberhavel # not Kreis Oberhavel
        bezirk: Marwitz
        city: Oberkrämer
        strasse: Dreihügelweg

found 64 entries

so this should work now after updating the master version

@threadstone83
Copy link

I can confirm - for me this fix works (at the moment).
Thank you very much!!

@mfraedrich
Copy link

@ReneNulschDE: tested your fixed code, also worked for me :-) thanks, hope you are now the cat ;-)

@ReneNulschDE : thanks for your fix, is working now !

@Alek19901
Copy link

Tested again - Today!

Now it works :-) Thanks for your work!

@BIGede
Copy link

BIGede commented Apr 26, 2024

I have tried it with Master and I use the following area

waste_collection_schedule:
sources:
- name: app_abfallplus_de
args:
app_id: de.k4systems.neustadtaisch
city: Scheinfeld
strasse: Schnodsenbach
hnr: Alle Hausnummern

Error message Region not found:

Logger: waste_collection_schedule.source_shell
Quelle: custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py:136
Integration: waste_collection_schedule (Dokumentation)
Erstmals aufgetreten: 13:34:36 (1 Vorkommnisse)
Zuletzt protokolliert: 13:34:36

fetch failed for source Apps by Abfall+: Traceback (most recent call last): File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch entries = self._source.fetch() ^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/app_abfallplus_de.py", line 140, in fetch for d in self._app.generate_calendar(): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 786, in generate_calendar self.select_kommune() File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 502, in select_kommune raise Exception(f"Region {self._region_search} not found.") Exception: Region Scheinfeld not found.

With 1.47.0 I had the error message: Street not found

@ReneNulschDE
Copy link
Contributor

@BIGede : Your config is not correct. I tried the following config and this works.

waste_collection_schedule:
  sources:
    - name: app_abfallplus_de
      args:
        app_id: de.k4systems.neustadtaisch
        city: Scheinfeld
        strasse: Schnodsenbach (Schnodsenbach)
        hnr: 1

@specularius
Copy link

No luck with Master

waste_collection_schedule:
sources:
- name: app_abfallplus_de
args:
app_id: de.k4systems.lkemmendingen
city: Biederbach
strasse: Alle Straßen

Also error message "Region not found"

Logger: waste_collection_schedule.source_shell
Quelle: custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py:136
Integration: waste_collection_schedule (Dokumentation)
Erstmals aufgetreten: 09:43:43 (1 Vorkommnisse)
Zuletzt protokolliert: 09:43:43

fetch failed for source Apps by Abfall+: Traceback (most recent call last): File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch entries = self._source.fetch() ^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/app_abfallplus_de.py", line 140, in fetch for d in self._app.generate_calendar(): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 786, in generate_calendar self.select_kommune() File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py", line 502, in select_kommune raise Exception(f"Region {self._region_search} not found.") Exception: Region Biederbach not found.

@chma73
Copy link

chma73 commented Apr 27, 2024

@specularius that it works, you have to follow this instruction (you can also check above):

ReneNulschDE commented 5 days ago
@schmidt-mai-marcel :

Danke für den Link. Wo muss man diesen Code einfügen?
Der Code aus dem link muß in der Datei HA-Config-Folder/custom_components/waste_collection_schedule/waste_collection_schedule/service/AppAbfallplusDe.py eingefügt werden. (alles ersetzen) - Wenn Du damit wenig Erfahrung hast, erstelle dir eine Sicherung...

@specularius
Copy link

@chma73 wrote that, but the link is not (or no longer) valid . Error: 404: Not Found

@chma73
Copy link

chma73 commented Apr 27, 2024

@specularius hm maybe you can check with @ReneNulschDE
or you need to wait till the new master version is available

@specularius
Copy link

@chma73 Thanks for the hint. I found it under "hacs_waste_collection_schedule/custom_components/waste_collection_schedule/waste_collection_schedule
/service/" and there is the new "AppAbfallplusDe.py". Copied the file to the local directory and bingo. It worked for me

@BIGede
Copy link

BIGede commented Apr 27, 2024

@ReneNulschDE
Hi,

I use exactly this code. Error message is still region not found.

@ReneNulschDE
Copy link
Contributor

Then you do not have the right version out of the master branch. HACS has sometimes problems pulling the master versions. You can try to switch to the actual version, install, restart, switch to master, reboot, install, reboot. Or copy the master branch version manually.

@BIGede
Copy link

BIGede commented Apr 27, 2024

Thank you, now it works again. Reinstalling the master was the solution.

@BIGede
Copy link

BIGede commented Apr 28, 2024

New small problem, there is no longer an entry for "Biomüll" in the calendar. Apparently this is not collected via the API.

@5ila5 5ila5 changed the title [Bug]: app_abfallplus_de de.k4systems.awbrastatt no results [Bug]: app_abfallplus_de no results Apr 30, 2024
@5ila5 5ila5 mentioned this issue Apr 30, 2024
7 tasks
@threadstone83
Copy link

Since today it stopped working again (de.k4systems.abfallscout) :-(

@Breakingx
Copy link

de.k4systems.zakb doesn't work with the master branch

@ReneNulschDE
Copy link
Contributor

@5ila5 : I'm traveling through southern Europe currently (and the next weeks) and have limited time to analyze it. Maybe you can take a look. I'll try it whenever possible.

@5ila5
Copy link
Collaborator

5ila5 commented May 4, 2024

@Breakingx zakb worked for me:

{
        "app_id": "de.k4systems.zakb",
        "strasse": "Ahornweg",
        "hnr": "3",
        "city": "Fürth",
    }

@threadstone83 abfallscout worked for me when providing strasse and hnr:

{
        "app_id": "de.k4systems.abfallscout",
        "city": "Hammelburg",
        "bezirk": "Morlesau",
        "strasse": "Alle Straßen",
        "hnr": "Alle Hausnummern"
 }

but I made some minor changes, making strasse and hnr optional if only one is possible anyway in #2051

Maybe HACS did some weird version change? Or they had an outage right when fetching?

I just released 1.48.0 so HACS should not make chaos anymore. I will close this for now as all the tested sources do return data now

@5ila5 5ila5 closed this as completed May 4, 2024
@threadstone83
Copy link

@5ila5 after I updated to 1.48.0 it workes again. Thank you very much!!

@Shooter182
Copy link

Hello Guys,

big thanks for the update!
now it works for:

  • name: app_abfallplus_de
    args:
    app_id: de.cmcitymedia.hokwaste

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests