[python] Fix pyarrow module missing attribute

[tonymtu]

Purpose

Fix AttributeError: module 'pyarrow' has no attribute 'parquet' in format table read/write.

pyarrow.parquet is a submodule that needs explicit import. Previously this wasn't caught maybe because other modules imported pyarrow.parquet as a side effect, making it available globally. When those modules are absent or import order changes, the error surfaces.

Tests

API and Format

Documentation

Generative AI tooling

No

[XiaoHongbo-Hope]

Thanks! Can we use a small case to reproduce the issue, and add it in CI to avoid such issues in future?

[tonymtu]

Thanks! Can we use a small case to reproduce the issue, and add it in CI to avoid such issues in future?

Maybe it is hard to cover the issue with a reliable test. When running the full test suite, import in earlier tests can silently mask the issue.

A static analysis check (e.g. mypy) can be a proper way to catch this. However, introducing mypy into CI is a broader change that can surface many existing type issues across the codebase?

[XiaoHongbo-Hope]

Thanks! Can we use a small case to reproduce the issue, and add it in CI to avoid such issues in future?

Maybe it is hard to cover the issue with a reliable test. When running the full test suite, import in earlier tests can silently mask the issue.

A static analysis check (e.g. mypy) can be a proper way to catch this. However, introducing mypy into CI is a broader change that can surface many existing type issues across the codebase?

Got your point. Can we use a subprocess to test the issue, or we just introduce mypy into CI ? We should try our best to catch these issues in CI rather than on the user side. Can you provide us some suggestion on it? @JingsongLi

[tonymtu]

Additionally, AI helps to find other "optional modules" in pyarrow that need explicit import across the codebase.

# refer to pyarrow/__init.py__#show_info()
print("\nOptional modules:")
modules = ["csv", "cuda", "dataset", "feather", "flight", "fs", "gandiva", "json",
           "orc", "parquet"]

[JingsongLi]

Hi @tonymtu @XiaoHongbo-Hope , maybe

Thanks! Can we use a small case to reproduce the issue, and add it in CI to avoid such issues in future?

Maybe it is hard to cover the issue with a reliable test. When running the full test suite, import in earlier tests can silently mask the issue.
A static analysis check (e.g. mypy) can be a proper way to catch this. However, introducing mypy into CI is a broader change that can surface many existing type issues across the codebase?

Got your point. Can we use a subprocess to test the issue, or we just introduce mypy into CI ? We should try our best to catch these issues in CI rather than on the user side. Can you provide us some suggestion on it? @JingsongLi

I don't have any good suggestions. I'm not very familiar with Python engineering testing. Perhaps it's a problem with our CI testing?

[tonymtu]

Hi @JingsongLi @XiaoHongbo-Hope . I'd prefer to split this into two separate efforts:

• This PR: fix the immediate bug (explicit import pyarrow.xxx).
• A follow-up effort: enable mypy checking in CI to prevent this class of issues.

I noticed there's already a [mypy] section in dev/cfg.ini but it's not wired into lint-python.sh, presumably because enabling it across the full codebase would surface too many existing issues at once.

Suggestions on enabling [mypy] by Claude Opus:
flake8 (our current linter) cannot catch this — it's a style checker, not a type checker. Only mypy can detect unresolved submodule attribute access. A practical approach for the follow-up would be running mypy only on files changed in each PR (git diff --name-only), so we can incrementally enforce type checking without a massive one-time fix.

Is there any better ideas?

[JingsongLi]

Sounds good to me. I will merge this PR.