Running as file watcher in container with PyCharm truncates file

[daveisfera]

I've setup a file watcher that run ruff --fix and ruff format in a docker image. I've run just one of those commands to isolate the issue and have seen it happen when just ruff --fix or ruff format is running so it's happening with either command run separately

Here's an example of the error output and when I look at the file, several characters from the last few lines have been removed (i.e. the file was truncated):

/Users/dlj/projects/myproject/local/format_file.sh myfile.py
error: Failed to parse myfile.py:117:5: Unexpected token 'exc'
Found 1 error (1 fixed, 0 remaining).
error: Failed to format myfile.py: source contains syntax errors: ParseError { error: UnrecognizedToken(Name { name: "exc" }, None), offset: 3254, source_path: "<filename>" }

Process finished with exit code 2

Here's the scripts that I use as the watcher (which I have setup in the same manner that black recommends):

#!/usr/bin/env sh

cd "$(dirname "$0")/.." || exit

if ! local/_run_ruff.sh --fix "$@"; then
  exit 1
fi
local/_run_ruff.sh format "$@"
exit $?

#!/usr/bin/env sh

cd "$(dirname "$0")/.." || exit

docker run --rm \
  -v "${PWD}":/usr/src/app:z,cached \
  -w /usr/src/app \
  ruff:0.1.4 \
  /usr/local/bin/ruff "$@"
exit $?

On a related note, I know there's a PyCharm plugin but I like running it in a container, because then CI and every developer machine is automagically kept in sync

Here's my ruff.toml

line-length = 120

select = ["C", "I", "U", "W"]
ignore = ["E", "F", "C901", "W191"]

target-version = "py39"

[isort]
combine-as-imports = true

[format]
# Like Black, use double quotes for strings.
quote-style = "double"

# Like Black, indent with spaces, rather than tabs.
indent-style = "space"

# Like Black, automatically detect the appropriate line ending.
line-ending = "auto"

[daveisfera]

I thought that maybe this was an issue with the cache, but I tried setting RUFF_NO_CACHE=true from 0.1.5 and the problem still happens

[daveisfera]

I've tried playing around with different things when this happens to understand what's going on and the oddest thing I've noticed is that undoing the format and then editing the file usually doesn't change anything about the result. Here's an example:

This was the function at the end of a file I just edited:

@require_POST
  def video(request: HttpRequest, stream_name: str) -> HttpResponse:
      return _handle_data(request, stream_name, HlsType.video, UploadVideoForm, "data")

And it was truncated to this:

@require_POST
def video(request: HttpRequest, stream_name: str) -> Http

As mentioned, I undid the change from the format applied on save and then copied a second copy of the function just below it and saved again and it truncated to the same spot. I then did that 2 more times, so there were 4 copies of the function and it did the same thing. I added newlines above the function and after doing that a few times, it returned a result without the truncation and a warning about the multiple definitions of the function.

So it appears that the issue is happening on the reading of the file (or else the warning about the multiple definitions would have happened) and it appears that it's using the old length of the file when reading the data and that's what is causing the truncation

[charliermarsh]

@daveisfera -- Any idea if this is still an issue?

[daveisfera]

Yes, it is unfortunately still an issue. I'm guessing it's something to do with the way that PyCharm handles files and edits that isn't playing nice with how ruff inputs and/or outputs the file inside of a container on macOS

[MichaReiser]

Sounds mysterious. Does this only happen in a container or also when running it locally? Is my understanding correct that you created a pycharm watcher to run the script or is there another watch task?

[daveisfera]

Only when I run it in a container so I've switched to running ruff natively to workaround this problem (but then all the developers on our team have to synchronize the upgrade instead of it being controlled by a script)

And yes, it's a PyCharm watcher that's running the script, and I set it up the same way that I used to run black

[MichaReiser]

That's very strange. Could you try adding a random 10ms timeout before calling ruff to emulate Python's startup time 🤣

Have you tried https://plugins.jetbrains.com/plugin/20574-ruff ?

[MichaReiser]

Do you run into the same issue when using ruff as an external tool? https://docs.astral.sh/ruff/editors/setup/#via-external-tool

[daveisfera]

I prefer to run it in a container, because then the version being run is controlled by the code itself and changes automatically roll out to all developers and CI as the commit with the change is merged

From my testing so far, putting sleep 0.1 before running ruff has prevented the problem from happening, so I'll keep testing with that and see if the issue ever happens