Skip to content

fix: add explicit utf-8 encoding to file open calls#7480

Open
IgnazioDS wants to merge 1 commit intomicrosoft:mainfrom
IgnazioDS:fix/playwright-utf8-encoding
Open

fix: add explicit utf-8 encoding to file open calls#7480
IgnazioDS wants to merge 1 commit intomicrosoft:mainfrom
IgnazioDS:fix/playwright-utf8-encoding

Conversation

@IgnazioDS
Copy link
Copy Markdown

Summary

  • Add encoding='utf-8' to open() calls across the codebase (15 files)
  • The original fix for playwright_controller.py (line 67) was already merged in PR add utf encoding in websurfer read file #6094, but the issue remains open because similar missing encoding parameters exist throughout the codebase
  • Prevents UnicodeDecodeError on non-English Windows systems (e.g., CP950 for Traditional Chinese)

Problem

On Windows systems with non-UTF-8 default encoding, open() without explicit encoding crashes with UnicodeDecodeError: 'cp950' codec can't decode byte. See #5566.

Changes

magentic-one-cli:

  • _m1.py — config file reading

autogen-ext:

  • docker_jupyter/_docker_jupyter.py — HTML file writing

autogen-studio:

  • database/schema_manager.py — alembic config read/write (7 calls)
  • web/auth/manager.py — YAML config reading
  • lite/studio.py — env file writing
  • cli.py — env file writing
  • gallery/builder.py — JSON file writing

agbench:

  • run_cmd.py — env/config file reading (4 calls)
  • tabulate_cmd.py — console log reading (2 calls)
  • remove_missing_cmd.py — console log reading
  • linter/cli.py — log file reading
  • linter/coders/oai_coder.py — cache file read/write
  • benchmarks/process_logs.py — log/prompt file reading (8 calls)
  • benchmarks/GAIA/Scripts/custom_tabulate.py — answer/log file reading

Other:

  • fixup_generated_files.py — generated file read/write

Test plan

  • No regression on Linux/macOS where UTF-8 is typically the default
  • Files read/write correctly on Windows with non-UTF-8 locale
  • All existing tests pass (encoding='utf-8' is a safe default for all text files in this repo)

Fixes #5566

Add encoding='utf-8' to open() calls across the codebase to prevent
UnicodeDecodeError on non-English Windows systems where the default
encoding is not UTF-8 (e.g., CP950 for Traditional Chinese).

Files fixed:
- magentic-one-cli/_m1.py (config file reading)
- autogen-ext/code_executors/docker_jupyter/_docker_jupyter.py (HTML writing)
- autogen-studio/database/schema_manager.py (alembic config read/write)
- autogen-studio/web/auth/manager.py (YAML config reading)
- autogen-studio/lite/studio.py (env file writing)
- autogen-studio/cli.py (env file writing)
- autogen-studio/gallery/builder.py (JSON file writing)
- agbench run_cmd.py, tabulate_cmd.py, remove_missing_cmd.py (log/config reading)
- agbench linter cli.py and oai_coder.py (file read/write)
- agbench benchmarks process_logs.py, custom_tabulate.py (log reading)
- fixup_generated_files.py (generated file read/write)

Fixes microsoft#5566
@IgnazioDS
Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

open needs encoding='utf-8' for non-english environment, error in playwright_controller.py

1 participant