`No such file or directory` warnings

[debajyoti-truefoundry]

I am seeing multiple No such file or directory warnings on version: v2.3.2.

2025-06-09T08:34:55.044209637Z  WARN         tokio-runtime-worker ThreadId(73) parseable::parseable::streams:512: File (/parseable/staging/loadgen6/39cda3db93356721.date=2025-06-09.hour=08.minute=25.parseable-54ffcd64dc-mw6cs.data.arrows) not found; Error = No such file or directory (os error 2)
2025-06-09T08:34:55.044248068Z  WARN         tokio-runtime-worker ThreadId(73) parseable::parseable::streams:512: File (/parseable/staging/loadgen6/96b23bab87956fd2.date=2025-06-09.hour=08.minute=25.parseable-54ffcd64dc-mw6cs.data.arrows) not found; Error = No such file or directory (os error 2)
2025-06-09T08:34:55.044257199Z  WARN         tokio-runtime-worker ThreadId(73) parseable::parseable::streams:512: File (/parseable/staging/loadgen6/ed3634890caf4839.date=2025-06-09.hour=08.minute=25.parseable-54ffcd64dc-mw6cs.data.arrows) not found; Error = No such file or directory (os error 2)
2025-06-09T08:34:55.044272079Z  WARN         tokio-runtime-worker ThreadId(73) parseable::parseable::staging::reader:89: Error when trying to read file: "/parseable/staging/loadgen6/96b23bab87956fd2.date=2025-06-09.hour=08.minute=18.parseable-644556c9b-dmsl7.data.arrows"
2025-06-09T08:34:55.044281189Z  WARN         tokio-runtime-worker ThreadId(73) parseable::parseable::staging::reader:89: Error when trying to read file: "/parseable/staging/loadgen6/39cda3db93356721.date=2025-06-09.hour=08.minute=18.parseable-644556c9b-dmsl7.data.arrows"
2025-06-09T08:34:55.044287309Z  WARN         tokio-runtime-worker ThreadId(73) parseable::parseable::staging::reader:89: Error when trying to read file: "/parseable/staging/loadgen6/ed3634890caf4839.date=2025-06-09.hour=08.minute=18.parseable-644556c9b-dmsl7.data.arrows"

Are these normal?

[linear]

DEV-803 `No such file or directory` warnings

[nitisht]

We'll get back on this one shortly @debajyoti-truefoundry

[debajyoti-truefoundry]

Noticing this again.

2025-06-09T08:59:26.137225221Z  WARN         tokio-runtime-worker ThreadId(41) parseable::parseable::streams:512: File (/parseable/staging/loadgen7/ed3634890caf4839.date=2025-06-09.hour=08.minute=50.parseable-5d857bf459-bs2jz.data.arrows) not found; Error = No such file or directory (os error 2)
2025-06-09T08:59:26.141225959Z  WARN         tokio-runtime-worker ThreadId(40) parseable::parseable::streams:512: File (/parseable/staging/loadgen7/39cda3db93356721.date=2025-06-09.hour=08.minute=50.parseable-5d857bf459-bs2jz.data.arrows) not found; Error = No such file or directory (os error 2)
2025-06-09T08:59:26.173893448Z  WARN         tokio-runtime-worker ThreadId(41) parseable::parseable::streams:512: File (/parseable/staging/loadgen7/96b23bab87956fd2.date=2025-06-09.hour=08.minute=50.parseable-5d857bf459-bs2jz.data.arrows) not found; Error = No such file or directory (os error 2)
20

I also do not see any data being synced to S3. Is this due to the error mentioned above?

[nikhilsinhaparseable]

@debajyoti-truefoundry I suspect the user running parseable does not have proper priviledges to the staging directory because of which the server is unable to create files.
I need some more information to understand and debug the issue, can you please join slack?

[nikhilsinhaparseable]

@debajyoti-truefoundry we have identified this as a resource management issue and are developing enhancements to improve server stability under resource constraints. We will provide an update here once the fix is tested and ready for release.