Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tesla API: Retry in 60497 seconds #3957

Closed
1 task done
mrgaolei opened this issue Jun 14, 2024 · 43 comments
Closed
1 task done

Tesla API: Retry in 60497 seconds #3957

mrgaolei opened this issue Jun 14, 2024 · 43 comments
Labels
area:tesla api Related to the Tesla API undetermined Not sure if this is considered a real bug

Comments

@mrgaolei
Copy link
Contributor

mrgaolei commented Jun 14, 2024

Is there an existing issue for this?

  • I have searched the existing issues

What happened?

Can't fetch car data.
I found that Teslamate upgrade API endpoint from OwnerAPI to FleetAPI, but what ClientID is?
How can I set my ClientID?
I think there's no common ClientID

Sorry!, updated:

I was wrong, is OwnerAPI, not FleetAPI.
but why prompt me "Retry in 60466 seconds"?
I test another TeslaID on my same Server goes OK.
Is this TeslaID blocked?

Expected Behavior

No response

Steps To Reproduce

No response

Relevant log output

2024-06-14 07:12:14.086 [info] GET https://owner-api.vn.cloud.tesla.cn/api/1/vehicles/366692546757877/vehicle_data -> 429 (44.648 ms)
2024-06-14 07:12:14.086 [warning] TeslaApi.Error / "Retry in 60466 seconds\n"
2024-06-14 07:12:14.087 car_id=1 [error] Error / :unknown

Screenshots

No response

Additional data

No response

Type of installation

Docker

Version

v20.10.12

@yanxuanm
Copy link

yanxuanm commented Jun 14, 2024

I am encountering the same issue starting at June 14, 12AM EDT, might related to tesla server side problem.

I updated to the latest version of Teslamate, restarted the container, issue still persists.

2024-06-14 03:41:47.434 [info] Starting logger for '****'
2024-06-14 03:41:47.612 [info] GET /
2024-06-14 03:41:47.617 [info] Sent 200 in 5ms
2024-06-14 03:41:47.641 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/*******/vehicle_data -> 429 (88.300 ms)
2024-06-14 03:41:47.641 [warning] TeslaApi.Error / "Retry in 58693 seconds\n"
2024-06-14 03:41:47.641 car_id=1 [error] Error / :unknown
2024-06-14 03:41:50.552 [info] GET /
2024-06-14 03:41:50.556 [info] Sent 200 in 3ms

@4U6U57
Copy link

4U6U57 commented Jun 14, 2024

Don't have great logs since I rebuilt my container, thinking it was my fault. My dashboards stop showing data at around 20:30 Pacific (https://time.is/1718336220), although my retry time suggests 21:00 Pacific exactly (if cooldown time is 20 hours). Hope this isn't the end of the OwnerAPI.

2024-06-14T07:46:12.700194475Z	stdout	2024-06-14 07:46:12.698 car_id=1 [error] Error / :unknown
2024-06-14T07:46:12.699488915Z	stdout	2024-06-14 07:46:12.697 [warning] TeslaApi.Error / "Retry in 58428 seconds\n"
2024-06-14T07:46:12.698609125Z	stdout	2024-06-14 07:46:12.697 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/***/vehicle_data -> 429 (66.601 ms)

As an aside, I'm super conflicted since I just got 2024.14.9, should I wait to update until I can have my statistics reflected properly? 😶

@yanxuanm
Copy link

Don't have great logs since I rebuilt my container, thinking it was my fault. My dashboards stop showing data at around 20:30 Pacific (https://time.is/1718336220), although my retry time suggests 21:00 Pacific exactly (if cooldown time is 20 hours). Hope this isn't the end of the OwnerAPI.

2024-06-14T07:46:12.700194475Z	stdout	2024-06-14 07:46:12.698 car_id=1 [error] Error / :unknown
2024-06-14T07:46:12.699488915Z	stdout	2024-06-14 07:46:12.697 [warning] TeslaApi.Error / "Retry in 58428 seconds\n"
2024-06-14T07:46:12.698609125Z	stdout	2024-06-14 07:46:12.697 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/***/vehicle_data -> 429 (66.601 ms)

As an aside, I'm super conflicted since I just got 2024.14.9, should I wait to update until I can have my statistics reflected properly? 😶

Looks like not only on 2024.14.9,
my car is on [2024.20.1], 2023 Model Y LR

@jheredianet
Copy link
Contributor

Same constant warnings here: M3 2020

image

@TurboChum
Copy link

Same issues as @jheredianet

appears to have flat lined around 11:20pm Eastern on both my 2021 MY and 2022 M3. Hopefully there’s a way to fix this soon!
Tesla mate reports battery % to my home automation system which then controls charging based on what’s left in the house batteries. Only noticed this morning as the car is rather full and the house is rather empty.
Suppose I will need to add some watchdogs.

@prato1976
Copy link

prato1976 commented Jun 14, 2024

Same here stop working around 03:23 UTC
Tried also to new tokens and no luck.

Log GMT+1

2024-06-13 23:59:23.420 car_id=1 [info] Start / :online
2024-06-13 23:59:28.130 car_id=1 [info] Connecting ...
2024-06-14 00:00:19.553 car_id=1 [info] Charging detected: -3 kW
2024-06-14 00:00:19.986 [info] GET https://nominatim.openstreetmap.org/reverse -> 200 (253.411 ms)
2024-06-14 00:00:19.994 car_id=1 [info] Charging / SOC: 34% / Casa
2024-06-14 00:00:19.994 car_id=1 [info] Disconnecting ...
2024-06-14 04:23:08.246 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (62.917 ms)
2024-06-14 04:23:08.247 [warning] TeslaApi.Error / "Retry in 74212 seconds\n"
2024-06-14 04:23:08.247 car_id=1 [error] Error / :unknown
2024-06-14 04:23:23.312 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (64.860 ms)
2024-06-14 04:23:23.313 [warning] TeslaApi.Error / "Retry in 74197 seconds\n"
2024-06-14 04:23:23.313 car_id=1 [error] Error / :unknown
2024-06-14 04:23:38.381 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (67.520 ms)
2024-06-14 04:23:38.381 [warning] TeslaApi.Error / "Retry in 74182 seconds\n"
2024-06-14 04:23:38.381 car_id=1 [error] Error / :unknown
2024-06-14 04:23:53.446 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (63.722 ms)
2024-06-14 04:23:53.446 [warning] TeslaApi.Error / "Retry in 74167 seconds\n"
2024-06-14 04:23:53.446 car_id=1 [error] Error / :unknown

@mattenz
Copy link

mattenz commented Jun 14, 2024

I’m seeing the same issue, however the Tesla add-on in Home Assistant which I still have setup via the old owners API is able to poll and control the car.

@mrgaolei
Copy link
Contributor Author

via

I’m seeing the same issue, however the Tesla add-on in Home Assistant which I still have setup via the old owners API is able to poll and control the car.

FleetAPI is wave updata. All vehicle can command via Fleet and some old vehicle also command via OwnerAPI. But those will updata to FleetAPI only final.

@b89dan
Copy link

b89dan commented Jun 14, 2024

Same here, as of today 14 Jun:

2024-06-14 11:06:18.449 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/9290931261318626/vehicle_data -> 429 (69.356 ms)
2024-06-14 11:06:18.449 [warning] TeslaApi.Error / "Retry in 46422 seconds\n"
2024-06-14 11:06:18.449 car_id=1 [error] Error / :unknown

@arthvpatel
Copy link

arthvpatel commented Jun 14, 2024

Mine was down earlier but it seems to have reconnected now. Healthcheck failed but can still see the data updated

@mattenz
Copy link

mattenz commented Jun 14, 2024

Yep, mine seems to have recovered now too

@b89dan
Copy link

b89dan commented Jun 14, 2024

indeed, recovered now on my side too.

@prato1976
Copy link

back online

2024-06-14 12:24:27.721 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (58.695 ms)
2024-06-14 12:24:27.721 [warning] TeslaApi.Error / "Retry in 45333 seconds\n"
2024-06-14 12:24:27.722 car_id=1 [error] Error / :unknown
2024-06-14 12:24:57.866 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (62.658 ms)
2024-06-14 12:24:57.866 [warning] TeslaApi.Error / "Retry in 45303 seconds\n"
2024-06-14 12:24:57.866 car_id=1 [error] Error / :unknown
2024-06-14 12:25:28.246 car_id=1 [info] Start / :online
2024-06-14 12:25:28.256 car_id=1 [info] Connecting ...
2024-06-14 12:29:38.208 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9110.879 ms)
2024-06-14 12:29:38.208 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-14 12:29:38.209 car_id=1 [error] Error / :unknown
2024-06-14 12:29:38.505 [notice]     :alarm_handler: {:clear, TeslaMate.Vehicles.Vehicle_1_api_error}
2024-06-14 12:30:07.411 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9201.884 ms)
2024-06-14 12:30:07.411 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-14 12:30:07.412 car_id=1 [error] Error / :unknown

@jlestel
Copy link
Contributor

jlestel commented Jun 14, 2024

Tesla changed the 429 to display the instruction in the body for developers who didn't see the header.

@jheredianet @prato1976 408 response codes (Device Not Available) are not counted against the WAF quota.

Only calls returning a 200 are taken into account in the quota which then causes the 429 (too many requests).

@mrgaolei @prato1976 did it work (with response code 200 on the vehicle_data call) after the indicated time? or sooner ?

I see that @b89dan should have waited a long time according to the message received...

@florianfish
Copy link

Same problem here since this night.

@jensb
Copy link

jensb commented Jun 15, 2024

I am also seeing this since yesterday. There goes my solar overflow charging automation :-(
The ioBroker tesla-motors adapter also has the same issue.

Can we collaborate on a solution which still allows getting vehicle charge / battery charge often enough so that solar overflow charging still is feasible, i.e. once every minute at least? I'm willing to help & discuss.

@ykii
Copy link

ykii commented Jun 15, 2024

Same here, has anyone found a solution yet?

@ToniA
Copy link
Contributor

ToniA commented Jun 15, 2024

Same here, starting 2024-06-15 13:04 EET. Legacy Model S, '16 S75 with MCU1

@jheredianet
Copy link
Contributor

jheredianet commented Jun 15, 2024

@vixium
Copy link

vixium commented Jun 15, 2024

Same problem here...but Home Assistant complement still working with the same Owner API

@mortenmoulder
Copy link

Getting the same retrying error in Teslamate version 1.29.1. Home Assistant also does not work.

429 error (Too Many Requests) by the looks of it:

2024-06-15 14:52:01.715 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/xxxx/vehicle_data -> 429 (56.356 ms)
2024-06-15 14:52:01.715 [warning] TeslaApi.Error / "Retry in 32879 seconds\n"
2024-06-15 14:52:01.715 car_id=1 [error] Error / :unknown
2024-06-15 14:52:31.857 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/xxxx/vehicle_data -> 429 (58.280 ms)
2024-06-15 14:52:31.857 [warning] TeslaApi.Error / "Retry in 32849 seconds\n"
2024-06-15 14:52:31.857 car_id=1 [error] Error / :unknown
2024-06-15 14:53:02.002 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/xxxx/vehicle_data -> 429 (63.761 ms)
2024-06-15 14:53:02.002 [warning] TeslaApi.Error / "Retry in 32819 seconds\n"
2024-06-15 14:53:02.002 car_id=1 [error] Error / :unknown

@mandreasen
Copy link

mandreasen commented Jun 15, 2024

Same issue for the past ~5 hours.

2024-06-15 16:51:16.319 [info] Version: 1.29.1
2024-06-15 16:51:16.683 [info] POST https://auth.tesla.com/oauth2/v3/token -> 200 (304.887 ms)
2024-06-15 16:51:16.683 [info] Refreshed api tokens
2024-06-15 16:51:16.687 [info] Scheduling token refresh in 6 h
2024-06-15 16:51:16.691 [info] Running TeslaMateWeb.Endpoint with cowboy 2.10.0 at :::4000 (http)
2024-06-15 16:51:16.693 [info] Access TeslaMateWeb.Endpoint at http://localhost
2024-06-15 16:51:16.936 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/<hidden>/vehicle_data -> 429 (44.827 ms)
2024-06-15 16:51:16.937 [warning] TeslaApi.Error / %TeslaApi.Error{
  reason: :unknown,
  message: "\"Retry in 32924 seconds\\n\"",
  env: %Tesla.Env{
    method: :get,
    url: "https://owner-api.teslamotors.com/api/1/vehicles/<hidden>/vehicle_data",
    query: [
      endpoints: "charge_state;climate_state;closures_state;drive_state;gui_settings;location_data;vehicle_config;vehicle_state;vehicle_data_combo"
    ],
    headers: [
      {"content-type", "text/plain"},
      {"ratelimit-limit", "600"},
      {"ratelimit-remaining", "0"},
      {"ratelimit-reset", "32924"},
      {"retry-after", "32924"},
      {"x-request-id", "<hidden>"},
      {"vary", "Accept-Encoding"},
      {"x-envoy-upstream-service-time", "14"},
      {"x-frame-options", "DENY"},
      {"x-content-type-options", "nosniff"},
      {"strict-transport-security", "max-age=31536000; includeSubDomains"},
      {"cache-control", "no-cache, no-store, private, s-max-age=0"},
      {"date", "Sat, 15 Jun 2024 14:51:16 GMT"},
      {"server", "envoy"},
      {"transfer-encoding", "chunked"}
    ],
    body: "Retry in 32924 seconds\n",
    status: 429,
    opts: [
      access_token: "<hidden>"
    ],
    __module__: TeslaApi,
    __client__: %Tesla.Client{fun: nil, pre: [], post: [], adapter: nil}
  }
}
2024-06-15 16:51:16.937 [info] Starting logger for ''
2024-06-15 16:51:16.966 [info] MQTT connection has been established
2024-06-15 16:51:17.078 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/<hidden>/vehicle_data -> 429 (42.320 ms)
2024-06-15 16:51:17.078 [warning] TeslaApi.Error / "Retry in 32923 seconds\n"
2024-06-15 16:51:17.079 car_id=1 [error] Error / :unknown

Running TeslaMate with Docker on Linux with only Docker and TeslaMate running.

@donok1
Copy link

donok1 commented Jun 15, 2024

Same problem here.
It started for me at 01:13 UTC on June 14, in the middle of the night in Europe, while the car was parked in our underground parking without internet connection (just like every night).
Then 429 where coming and going every few hours.

Since 01:15 UTC on June 15, only 429 answer are logged without interruption. Although the time counter jumps quite quickly to 30s, and then decreases. Once reaching 1s it always jumps back to several hours.
Only the refresh token call is successful:

POST https://auth.tesla.com/oauth2/v3/token -> 200

I have no other services or apps using the owner API.
Model Y 2023 RWD, running 2024.20.1 since a few days.

@top-gun
Copy link
Contributor

top-gun commented Jun 15, 2024

I ran into the same issue, and found there seems to be a relation to me also running Home Assistant with Tesla integration 3.23.0. There seems to be an issue in 3.23.0 where the integration, despite chosing the option "use MQTT to retrieve data from Teslamate" periodically polls the car directly via the Tesla API. It's too early for conclusions, but it's possible that the HA integration triggered some rate limit on the Tesla server side. There is a new version 3.23.1 which I just installed. For Teslamate, I will have to wait until the rate limit is lifted in 30039 seconds.

@JakobLichterfeld JakobLichterfeld added undetermined Not sure if this is considered a real bug area:tesla api Related to the Tesla API labels Jun 15, 2024
@prato1976
Copy link

hi @jlestel,
not sure if if understand well since i'm not an expert :)

I was driving till 19:39

2024-06-13 19:39:20.722 car_id=1 [info] Driving / Ended / 15 km – 15 min
2024-06-13 19:39:20.722 car_id=1 [info] Start / :online
2024-06-13 19:44:48.791 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9108.288 ms)
2024-06-13 19:44:48.791 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-13 19:44:48.791 car_id=1 [error] Error / :unknown
2024-06-13 19:45:17.901 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9108.773 ms)
2024-06-13 19:45:17.902 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-13 19:45:17.902 car_id=1 [error] Error / :unknown
2024-06-13 19:45:47.031 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9127.993 ms)
2024-06-13 19:45:47.031 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-13 19:45:47.031 car_id=1 [error] Error / :unknown
2024-06-13 19:45:58.071 [notice]     :alarm_handler: {:set, {TeslaMate.Vehicles.Vehicle_1_api_error, :fuse_blown}}
2024-06-13 19:46:16.182 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9150.314 ms)
2024-06-13 19:46:16.182 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-13 19:46:16.182 car_id=1 [error] Error / :unknown
2024-06-13 19:46:45.806 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9622.001 ms)
2024-06-13 19:46:45.806 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-13 19:46:45.806 car_id=1 [error] Error / :unknown
2024-06-13 19:47:14.486 car_id=1 [info] Suspending logging
2024-06-13 19:52:58.078 [notice]     :alarm_handler: {:clear, TeslaMate.Vehicles.Vehicle_1_api_error}
2024-06-13 19:59:22.558 [info] Streaming API: Vehicle offline
2024-06-13 19:59:22.558 car_id=1 [warning] Stream reports vehicle as offline, fetching vehicle state ...
2024-06-13 19:59:22.773 car_id=1 [info] Start / :offline
2024-06-13 19:59:22.776 car_id=1 [info] Disconnecting ...

Refres token

2024-06-13 22:45:59.553 [info] Refreshing access token ...
2024-06-13 22:46:00.272 [info] POST https://auth.tesla.com/oauth2/v3/token -> 200 (718.294 ms)
2024-06-13 22:46:00.274 [info] Scheduling token refresh in 6 h

my car start charging midnight

2024-06-13 23:59:23.420 car_id=1 [info] Start / :online
2024-06-13 23:59:28.130 car_id=1 [info] Connecting ...
2024-06-14 00:00:19.553 car_id=1 [info] Charging detected: -3 kW
2024-06-14 00:00:19.986 [info] GET https://nominatim.openstreetmap.org/reverse -> 200 (253.411 ms)
2024-06-14 00:00:19.994 car_id=1 [info] Charging / SOC: 34% / Casa
2024-06-14 00:00:19.994 car_id=1 [info] Disconnecting ...

then at 04:23 no more info/data available on teslamate and starting to get the 429

2024-06-14 04:23:08.246 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (62.917 ms)
2024-06-14 04:23:08.247 [warning] TeslaApi.Error / "Retry in 74212 seconds\n"
2024-06-14 04:23:08.247 car_id=1 [error] Error / :unknown
2024-06-14 04:23:23.312 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (64.860 ms)
2024-06-14 04:23:23.313 [warning] TeslaApi.Error / "Retry in 74197 seconds\n"
2024-06-14 04:23:23.313 car_id=1 [error] Error / :unknown
2024-06-14 04:23:38.381 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (67.520 ms)
2024-06-14 04:23:38.381 [warning] TeslaApi.Error / "Retry in 74182 seconds\n"
2024-06-14 04:23:38.381 car_id=1 [error] Error / :unknown
2024-06-14 04:23:53.446 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (63.722 ms)
2024-06-14 04:23:53.446 [warning] TeslaApi.Error / "Retry in 74167 seconds\n"
2024-06-14 04:23:53.446 car_id=1 [error] Error / :unknown
2024-06-14 04:23:58.589 [notice]     :alarm_handler: {:set, {TeslaMate.Vehicles.Vehicle_1_api_error, :fuse_blown}}
2024-06-14 04:24:08.510 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (62.014 ms)
2024-06-14 04:24:08.510 [warning] TeslaApi.Error / "Retry in 74152 seconds\n"
2024-06-14 04:24:08.510 car_id=1 [error] Error / :unknown
2024-06-14 04:24:23.575 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (64.874 ms)
2024-06-14 04:24:23.576 [warning] TeslaApi.Error / "Retry in 74137 seconds\n"
2024-06-14 04:24:23.576 car_id=1 [error] Error / :unknown
2024-06-14 04:24:38.639 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (62.356 ms)
2024-06-14 04:24:38.639 [warning] TeslaApi.Error / "Retry in 74122 seconds\n"
2024-06-14 04:24:38.639 car_id=1 [error] Error / :unknown
2024-06-14 04:24:53.703 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (63.242 ms)
2024-06-14 04:24:53.703 [warning] TeslaApi.Error / "Retry in 74107 seconds\n"
2024-06-14 04:24:53.703 car_id=1 [error] Error / :unknown

and i continue getting 429 till 12:24:57 and starting having 408 at 12:25:28

2024-06-14 12:23:57.579 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (58.696 ms)
2024-06-14 12:23:57.579 [warning] TeslaApi.Error / "Retry in 45363 seconds\n"
2024-06-14 12:23:57.579 car_id=1 [error] Error / :unknown
2024-06-14 12:24:27.721 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (58.695 ms)
2024-06-14 12:24:27.721 [warning] TeslaApi.Error / "Retry in 45333 seconds\n"
2024-06-14 12:24:27.722 car_id=1 [error] Error / :unknown
2024-06-14 12:24:57.866 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 429 (62.658 ms)
2024-06-14 12:24:57.866 [warning] TeslaApi.Error / "Retry in 45303 seconds\n"
2024-06-14 12:24:57.866 car_id=1 [error] Error / :unknown
2024-06-14 12:25:28.246 car_id=1 [info] Start / :online
2024-06-14 12:25:28.256 car_id=1 [info] Connecting ...
2024-06-14 12:29:38.208 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9110.879 ms)
2024-06-14 12:29:38.208 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-14 12:29:38.209 car_id=1 [error] Error / :unknown
2024-06-14 12:29:38.505 [notice]     :alarm_handler: {:clear, TeslaMate.Vehicles.Vehicle_1_api_error}
2024-06-14 12:30:07.411 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9201.884 ms)
2024-06-14 12:30:07.411 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-14 12:30:07.412 car_id=1 [error] Error / :unknown
2024-06-14 12:45:43.516 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9111.254 ms)
2024-06-14 12:45:43.516 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-14 12:45:43.516 car_id=1 [error] Error / :unknown
2024-06-14 12:46:12.631 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9114.831 ms)
2024-06-14 12:46:12.631 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-14 12:46:12.631 car_id=1 [error] Error / :unknown
2024-06-14 12:53:46.766 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/1492930661955577/vehicle_data -> 408 (9276.637 ms)
2024-06-14 12:53:46.766 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-06-14 12:53:46.766 car_id=1 [error] Error / :unknown

since 2024-06-14 12:25:28 i don't see any 429 till now and working normal

regarding the 200 i got
2024-06-14 04:46:00.678 [info] POST https://auth.tesla.com/oauth2/v3/token -> 200 (403.638 ms)
2024-06-14 10:04:38.847 [info] POST https://auth.tesla.com/oauth2/v3/token -> 200 (371.817 ms) (i have restarted container)
2024-06-14 10:18:05.280 [info] POST https://auth.tesla.com/oauth2/v3/token -> 200 (318.079 ms) (I have forced the tokens)

@djpalmis
Copy link

djpalmis commented Jun 15, 2024

#3958 it happened to me, too

Just an idea: the real weakness is that teslamate doesn't give any feedback about those errors in the Web page.
In my case it works like everything is OK.. but it is not for sure!
I don't know: we need an "alive counter check" and/or an error feedback

@kimmilde
Copy link

Same error here. We need to limit the Api request or go to fleetapi asap

@hierro1981
Copy link

Reading from Tessie api status seems that Tesla have turned off the deprecated Owner API https://status.tessie.com/incident/384149

@realJustinLee
Copy link
Contributor

Same here

teslamate-1  | 2024-06-16 07:14:36.061 [info] GET https://owner-api.vn.cloud.tesla.cn/api/1/vehicles/XXXXXXXXXXXXX/vehicle_data -> 429 (263.169 ms)
teslamate-1  | 2024-06-16 07:14:36.061 [warning] TeslaApi.Error / :too_many_request 2725
teslamate-1  | 2024-06-16 07:14:36.062 car_id=2 [error] Too many request / Retry after 2725 seconds

@mortenmoulder
Copy link

This morning my Teslamate started working again randomly. I did not generate a new token.

@chiefjoejj
Copy link

I had the same issues starting yesterday. No connection to the two connected cars without showing any errors in the teslamte ui. just realized the missing connection because even after charging the cars for a few hours the soc got stuck at low percentages.

updated my teslamate-docker-stack-thingy yesterday -> no change

stopped all services today morning for around one hour and now teslamte gets updated from both cars again while charging. didnt change anything in the settings and never renewed any keys, tokens, whatsoever. lets see how long it is working again...

@mandreasen
Copy link

Mine works again. I shut it down for ~10 hours since the "Retry after X seconds" was around 9 hours. Now it's working once again, so either the API just works once again or it helpt to keep it down for the retry time.

@jensb
Copy link

jensb commented Jun 16, 2024

Here too. Today everything works again - so far.
But I disabled solar based charging which reduces the amount of requests significantly.

This incident might also have been a warning shot by Tesla - so we should take this seriously and upgrade the api.

@brianmay
Copy link
Collaborator

Several points:

  • Moving to the Fleet API will not help, I think that is also rate limited.

  • A change has been merged into master that respects the time in the 429 header. Respect Retry-After header on 429 response code #3943 - but while this means we want be loading the servers as much, it doesn't help the issue that the time is excessive.

  • Possibly we might need to increase the polling intervals. But doing so would reduce the responsiveness of Teslamate to non-streaming changes, and is yuck.

@4U6U57
Copy link

4U6U57 commented Jun 16, 2024

@brianmay one thing to note is that retry-after value can lie.

A couple of us who originally posted this issue were given retry times of 20h, but with the original constant polling behavior, I was able to get a successful response after less than 8h (as visible from my logs), and a couple others posted similar behavior.

2024-06-14 11:22:12.161 car_id=1 [info] Connecting ...
2024-06-14 11:22:02.044 car_id=1 [info] Start / :online
2024-06-14 11:21:31.235 car_id=1 [error] Error / :unknown
2024-06-14 11:21:31.235 [warning] TeslaApi.Error / "Retry in 45509 seconds\n"
2024-06-14 11:21:31.235 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/*****/vehicle_data -> 429 (63.234 ms)
2024-06-14 11:21:01.064 car_id=1 [error] Error / :unknown
2024-06-14 11:21:01.064 [warning] TeslaApi.Error / "Retry in 45539 seconds\n"
2024-06-14 11:21:01.064 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/*****/vehicle_data -> 429 (62.404 ms)
2024-06-14 11:20:30.914 car_id=1 [error] Error / :unknown
2024-06-14 11:20:30.914 [warning] TeslaApi.Error / "Retry in 45569 seconds\n"

@brianmay
Copy link
Collaborator

@brianmay one thing to note is that retry-after value can lie.

A couple of us who originally posted this issue were given retry times of 20h, but with the original constant polling behavior, I was able to get a successful response after less than 8h (as visible from my logs), and a couple others posted similar behavior.

Yuck. Does that mean we should be putting a cap on the upper retry time? Or is this just a transit issue while they implement the feature correctly?

@jlestel
Copy link
Contributor

jlestel commented Jun 16, 2024

@brianmay it's yuck but it's a lesser evil since the new streaming only sends events every minute

@4U6U57 i made the same observation. It would be interesting to know at what time you were able to access again. I have the impression that this is a daily limit. maybe the counters are finally reset daily at a certain time...

@ToniA
Copy link
Contributor

ToniA commented Jun 16, 2024

Mine started working at 03:00 AM EET, when the car started charging. I don't know if updating to the latest build helped, but I don't see any rate-limiting errors in the log any more. I haven't driven today, but at least the charge got recorded.

docker-compose.yml from

services:
  teslamate:
    image: teslamate/teslamate:latest

to

services:
  teslamate:
    image: teslamate/teslamate:edge

i.e. from latest release to the latest build.

@top-gun
Copy link
Contributor

top-gun commented Jun 16, 2024

Mine is on "Latest" and simply worked as soon as the ban from Tesla was expired.

@olexs
Copy link
Contributor

olexs commented Jun 18, 2024

Possibly we might need to increase the polling intervals. But doing so would reduce the responsiveness of Teslamate to non-streaming changes, and is yuck.

Would it be an option to make the polling interval(s) configurable? This way, users can decide between higher responsiveness with a higher risk of running into the API limits, or slower updates but lower risk.

@jlestel
Copy link
Contributor

jlestel commented Jun 18, 2024

Would it be an option to make the polling interval(s) configurable?

This is what I prepared in this PR.

@JakobLichterfeld JakobLichterfeld changed the title v 1.29.1 "Retry in 60497 seconds\n" "Retry in 60497 seconds" Jun 18, 2024
@JakobLichterfeld JakobLichterfeld changed the title "Retry in 60497 seconds" Tesla API: Retry in 60497 seconds Jun 18, 2024
@cwanja
Copy link
Collaborator

cwanja commented Jun 19, 2024

Mine is on "Latest" and simply worked as soon as the ban from Tesla was expired.

Between this quoted comment and release 1.29.2, is this resolved?

@top-gun
Copy link
Contributor

top-gun commented Jun 19, 2024

It did not happen again for me. In my case, it was enough to install a new version of the Tesla integration for Home Assistant, no updates to TM were necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:tesla api Related to the Tesla API undetermined Not sure if this is considered a real bug
Projects
None yet
Development

No branches or pull requests