Only use fsspec obj to scan remote csv if polars version is less than 1.34

[charles-turner-1]

Change Summary

• Only use fsspec obj to scan remote csv if polars version is less than 1.34
• I added this change to fix Panic in Polars when loading ESM catalog from S3-hosted CSV (begin <= end (10 <= 9) slicing error) #744 - however the polars 1.34 release (https://github.com/pola-rs/polars/releases?q=%2324450&expanded=true) fixed it (PR fix: Fix panic scanning from AWS legacy global endpoint URL pola-rs/polars#24450)
• This reduces the memory footprint on esm_datastore instantiation down from ~70MB to ~8MB.,

I have tested it against the datastore linked in #744 and can confirm it works for polars 1.33,1.34 and 1.36 (latest).

Memory usage will onIy have reduced if polars >= 1.34.

We could also bump the minimum polars version & keep the code simpler...

Related issue number

#744
#753

Checklist

• Unit tests for the changes exist
• Tests pass on CI
• Documentation reflects the changes where applicable

[mgrover1]

@charles-turner-1 - which PR should I look at first? 😄

[charles-turner-1]

Sorry, it's been a busy morning! This one is pretty straightforward, 767 is quite a bit more complicated - so I'd start here.

Is there a way to disable/delay auto-pinging you with every PR I open, short of editing the code owners file? It's probably a bit irritating!

[mgrover1]

Sorry, it's been a busy morning! This one is pretty straightforward, 767 is quite a bit more complicated - so I'd start here.

Is there a way to disable/delay auto-pinging you with every PR I open, short of editing the code owners file? It's probably a bit irritating!

No worries at all :) I like getting a ping to see the updates! You can always tag as a draft to let me know when it is still not ready, but no pressure/rush!! Thanks for getting these PRs in

[charles-turner-1]

I think I did move the other back to draft actually - shame it doesn't cancel the review request with the change 😞

[mgrover1]

Looks good! Makes sense - thanks so much!!