Skip to content

Commit

Permalink
Update deltalake version
Browse files Browse the repository at this point in the history
Working with Mage AI in a local environment on M1 Mac I faced the following error when trying to write data to datalake.

```
Generic S3 error: Error after 10 retries in 2.612435942s, max_retries:10, retry_timeout:180s, source:error sending request for url (https://s3..../_delta_log/_last_checkpoint): error trying to connect: invalid peer certificate: BadSignature
```
Seems it's a [known issue](delta-io/delta-rs#2551). And can be solved by upgrading deltalake lib to 0.18.2 version.

This PR updates deltalake version from 0.17.4 to the most recent 0.20.0., however [`overwrite_schema` parameter has been dropped](https://github.com/delta-io/delta-rs/releases#:~:text=fix%3A%20remove%20deprecated%20overwrite_schema%20configuration%20which%20has%20incorrect%20behavior%20by) since 0.18.0 due to

> fix: remove deprecated overwrite_schema configuration which has incorrect behavior
  • Loading branch information
anton-at-cs authored Oct 16, 2024
1 parent 8aec8f9 commit 91c5636
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ attrs==17.4.0; python_version < '3.12'
attrs==24.2.0; python_version >= '3.12'
backoff
clickhouse_sqlalchemy
deltalake==0.17.4
deltalake==0.20.0
facebook_business==17.0.2
gnupg==2.3.1
google-analytics-data==0.14.2; python_version < '3.11'
Expand Down

0 comments on commit 91c5636

Please sign in to comment.