Description
In #47 we are addressing critical deficiencies in the check-c-globals.py
(and adding c-globals.py
). Here we are focusing on nice-to-haves for c-globals.py
(at the least identifying them). While they are not the focus of #47, some of them might be partially addressed there.
Here are some non-critical deficiencies:
- only identifies statics for a single set of defines (i.e. for applicable ifdefs)
- isn't great at auto-detecting the correct runtime level of statics (requiring extra info in
ignored-globals.txt
) ignored-globals.txt
isn't great- could provide more info to the script (e.g. filename, scope, reason)
- imperfect statics detection
- could identify
static const
, etc. - bpo-36876: Avoid static locals. python/cpython#13372
- bpo-36876: Use a consistent variable name for kwlist. python/cpython#13531
- could identify
- doesn't identify thread-safety of variables
- detection relies on having a built binary (hence post-preprocessor)
...
<TBD>
...
(Everything past here is just dumped WIP text...)
Directory structure:
Tools/c-analyzer/
c_statics/
README (including "Resolving Failures" section)
__main__.py
find.py
show.py
known.py
supported.py
ignored.tsv (temporary)
c_parser/
files.py
info.py
statics.py
declarations.py
(pycparser / ply) <--------------- new
c_symbols/ <--------------- new
symbols.py (wrapper around "nm" command, etc.)
c-statics.py
check-c-statics.py
Lib/test/
test_tools/test_tool_c_analyzer/
testdata/...
test_c_statics/...
test_c_parser/...
test_c_symbols/... <--------------- new
test_check_c_statics.py
TODO:
Formatting:
-
--formatter FORMATTER,...
CLI arg -
"single-line" formatter
-
"basic" formatter
-
"table" formatter (columns, sort, group-by)
-
"json" formatter
-
"tsv" formatter
-
show.py
: addshow_table(rows, columns)
-
__main__.py
: add "formatter" arg to main() and command funcs -
__main__.py
: add "--formatter" CLI arg to "check" & "show" command -
info.py
: addStaticVar.level
andStaticVar.reason
-
known.py
: add "resolveVar(StaticVar)" -
add more functionality (e.g. better info) to
c-globals.py
+ identify static vars, including their scope, type, and runtime level + identify macros that produce static vars + identify types, including their runtime level +
-
_cg/__main__
: show-known (table of statics, table of macros, table of types) -
_cg/info.py
: StaticVar, Macro, Struct -
_cg/__main__
: show-found (table of statics, table of macros, table of types) -
_cg/__main__
: structs (show table) - parsing: find statics (w/scope), macros, structs
- identify per-interpreter struct defs (e.g. PyObject), macros, and statics
-
Here are things we can do later:
- add
StaticVar.macro
_cg/info.py
: addMacro
,Struct
-
_cg/__main__
: show-known (table of statics, table of macros, table of types)-
_cg/__main__
: show-found (table of statics, table of macros, table of types) -
_cg/__main__
: structs (show table) -
parsing: find statics (w/scope), macros, structs
-
identify per-interpreter struct defs (e.g. PyObject), macros, and statics
-
-
CLI:
[-v|--verbose] [-q|--quiet]
-
CLI:
show [--all] [--filter FILTER] [--columns COLUMNS] [--sort COL[,COL,...]] [--group-by COL[,COL,...]]
-
track ifdefs
-
unfold where ifdefs split a block (e.g. func, struct)
-
stop using preprocessor (or compare with results with preprocessor)
-
option to skip self check?
bpo-36876: "Global C variables are a problem."
1. update check-c-globals.py to identify static locals (and thread-locals)
2. deal with any identified globals
* move them to _PyRuntimeState (or thread-locals to PyThreadState, etc.)
* ignore them by adding them to Tools/c-globals/ignored-globals.txt
3. add check-c-globals.py to "make check"
4. (if "make check" isn't already there), ensure check-c-globals.py is run at
some point in CI
...
Also, Tools/c-globals/ignored-globals.txt is a bit out of date (some vars have
been removed, renamed, or moved to another file). That should get cleaned
up. It might also make sense to update check-c-globals.py to verify that all
variables in ignored-globals.txt actually exist.