Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stubgen: import Union if generated stub code includes #12929

Closed
eggplants opened this issue Jun 2, 2022 · 5 comments · Fixed by #14601
Closed

stubgen: import Union if generated stub code includes #12929

eggplants opened this issue Jun 2, 2022 · 5 comments · Fixed by #14601

Comments

@eggplants
Copy link
Contributor

eggplants commented Jun 2, 2022

Feature

I would like to insert from typing import Union when generated stub code by stubgen includes Union.

Pitch

ref: #12920

Now:

# Python 3.10.4, mypy 0.960
$ cat test.py
import random


def a() -> int | str:
    if random.random() > 0.5:
        return 1
    else:
        return "hey!"

def b() -> str | None:
    if random.random() > 0.5:
        return "hey!"

$ stubgen test.py
Processed 1 modules
Generated out/test.pyi

$ cat out/test.pyi
def a() -> Union[int, str]: ...
def b() -> Union[str, None]: ...

Expect:

from typing import Union

def a() -> Union[int, str]: ...
def b() -> Union[str, None]: ...
@JelleZijlstra
Copy link
Member

Thanks! I expect this is an easy fix (just need to ad some function call to make stubgen emit the import), so if you're willing to dive into the code, feel free to make a PR.

@eggplants
Copy link
Contributor Author

OK, I'll try it...

@bo-dani
Copy link

bo-dani commented Jan 4, 2023

Is there any update on this fix?

potiuk added a commit to potiuk/airflow that referenced this issue Jan 21, 2023
There were couple of problems with static checks generating source
files including generated stubs in the common.sql package:

* black formatting was implemented in multiple separate scripts
  making it harded to fix problems in all of them
* generated stub files were not formatted with is_pyi=True and
  black had no way to figure it out because it was working on strings
* black formatting was not consistently applied in all places
* EOL at the end of generated stub file was missing, leading to EOL
  fixer adding them after generation leading to multiple pre-commit
  passes needed
* there was (already unused) deprecated dev dict generator that used
  its own black formatting.

There were also couple of problems with the files generated by
stubgen itself:

* Union was missing in the generated stubs (this is a known issue
  with stubgen: python/mypy#12929
* Intellij complained on Incomplete import from _typeshed missing

This PR fixes all the problems:

* black formatting is now consistenly extracted and applied everywhere
* when needed, is_pyi flag is passed to black so that it knows
  that .pyi file is being fomratted
* EOL is added at the end of file when the file is generated
* Union is added to the generated stub
* noqa is added to _typeshed import
* the dict generator is removed

As the end result, generated stub files are fully importable
(no errors reported by IntelliJ IDE) and consistently formatted
every time.
potiuk added a commit to apache/airflow that referenced this issue Jan 23, 2023
There were couple of problems with static checks generating source
files including generated stubs in the common.sql package:

* black formatting was implemented in multiple separate scripts
  making it harded to fix problems in all of them
* generated stub files were not formatted with is_pyi=True and
  black had no way to figure it out because it was working on strings
* black formatting was not consistently applied in all places
* EOL at the end of generated stub file was missing, leading to EOL
  fixer adding them after generation leading to multiple pre-commit
  passes needed
* there was (already unused) deprecated dev dict generator that used
  its own black formatting.

There were also couple of problems with the files generated by
stubgen itself:

* Union was missing in the generated stubs (this is a known issue
  with stubgen: python/mypy#12929
* Intellij complained on Incomplete import from _typeshed missing

This PR fixes all the problems:

* black formatting is now consistenly extracted and applied everywhere
* when needed, is_pyi flag is passed to black so that it knows
  that .pyi file is being fomratted
* EOL is added at the end of file when the file is generated
* Union is added to the generated stub
* noqa is added to _typeshed import
* the dict generator is removed

As the end result, generated stub files are fully importable
(no errors reported by IntelliJ IDE) and consistently formatted
every time.
hamdanal added a commit to hamdanal/mypy that referenced this issue Feb 3, 2023
When a PEP 604 Union exists in the runtime, stubgen was generating a
`Union[...]` syntax without importing `Union` from `typing`. With this
change, stubgen preserves the ` | `-unions in the output.

Fixes python#12929
Closes python#13428
Ref python#12920
@sguidotti
Copy link

If no one else is working on this I can pick it up

@eggplants
Copy link
Contributor Author

eggplants commented Feb 7, 2023

@sguidotti

@hamdanal has already created: #14601

hauntsaninja pushed a commit that referenced this issue Feb 7, 2023
When a PEP 604 Union exists in the runtime, stubgen was generating a
`Union[...]` syntax without importing `Union` from `typing`. With this
change, stubgen preserves the ` | `-unions in the output.

Fixes #12929
Closes #13428
Ref #12920
ephraimbuddy pushed a commit to apache/airflow that referenced this issue Mar 8, 2023
There were couple of problems with static checks generating source
files including generated stubs in the common.sql package:

* black formatting was implemented in multiple separate scripts
  making it harded to fix problems in all of them
* generated stub files were not formatted with is_pyi=True and
  black had no way to figure it out because it was working on strings
* black formatting was not consistently applied in all places
* EOL at the end of generated stub file was missing, leading to EOL
  fixer adding them after generation leading to multiple pre-commit
  passes needed
* there was (already unused) deprecated dev dict generator that used
  its own black formatting.

There were also couple of problems with the files generated by
stubgen itself:

* Union was missing in the generated stubs (this is a known issue
  with stubgen: python/mypy#12929
* Intellij complained on Incomplete import from _typeshed missing

This PR fixes all the problems:

* black formatting is now consistenly extracted and applied everywhere
* when needed, is_pyi flag is passed to black so that it knows
  that .pyi file is being fomratted
* EOL is added at the end of file when the file is generated
* Union is added to the generated stub
* noqa is added to _typeshed import
* the dict generator is removed

As the end result, generated stub files are fully importable
(no errors reported by IntelliJ IDE) and consistently formatted
every time.

(cherry picked from commit 129f082)
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Mar 31, 2023
There were couple of problems with static checks generating source
files including generated stubs in the common.sql package:

* black formatting was implemented in multiple separate scripts
  making it harded to fix problems in all of them
* generated stub files were not formatted with is_pyi=True and
  black had no way to figure it out because it was working on strings
* black formatting was not consistently applied in all places
* EOL at the end of generated stub file was missing, leading to EOL
  fixer adding them after generation leading to multiple pre-commit
  passes needed
* there was (already unused) deprecated dev dict generator that used
  its own black formatting.

There were also couple of problems with the files generated by
stubgen itself:

* Union was missing in the generated stubs (this is a known issue
  with stubgen: python/mypy#12929
* Intellij complained on Incomplete import from _typeshed missing

This PR fixes all the problems:

* black formatting is now consistenly extracted and applied everywhere
* when needed, is_pyi flag is passed to black so that it knows
  that .pyi file is being fomratted
* EOL is added at the end of file when the file is generated
* Union is added to the generated stub
* noqa is added to _typeshed import
* the dict generator is removed

As the end result, generated stub files are fully importable
(no errors reported by IntelliJ IDE) and consistently formatted
every time.

GitOrigin-RevId: 129f0820cd03c721ebebe3461489f255bb9e752c
ahidalgob pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Nov 7, 2023
There were couple of problems with static checks generating source
files including generated stubs in the common.sql package:

* black formatting was implemented in multiple separate scripts
  making it harded to fix problems in all of them
* generated stub files were not formatted with is_pyi=True and
  black had no way to figure it out because it was working on strings
* black formatting was not consistently applied in all places
* EOL at the end of generated stub file was missing, leading to EOL
  fixer adding them after generation leading to multiple pre-commit
  passes needed
* there was (already unused) deprecated dev dict generator that used
  its own black formatting.

There were also couple of problems with the files generated by
stubgen itself:

* Union was missing in the generated stubs (this is a known issue
  with stubgen: python/mypy#12929
* Intellij complained on Incomplete import from _typeshed missing

This PR fixes all the problems:

* black formatting is now consistenly extracted and applied everywhere
* when needed, is_pyi flag is passed to black so that it knows
  that .pyi file is being fomratted
* EOL is added at the end of file when the file is generated
* Union is added to the generated stub
* noqa is added to _typeshed import
* the dict generator is removed

As the end result, generated stub files are fully importable
(no errors reported by IntelliJ IDE) and consistently formatted
every time.

GitOrigin-RevId: 129f0820cd03c721ebebe3461489f255bb9e752c
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Sep 19, 2024
There were couple of problems with static checks generating source
files including generated stubs in the common.sql package:

* black formatting was implemented in multiple separate scripts
  making it harded to fix problems in all of them
* generated stub files were not formatted with is_pyi=True and
  black had no way to figure it out because it was working on strings
* black formatting was not consistently applied in all places
* EOL at the end of generated stub file was missing, leading to EOL
  fixer adding them after generation leading to multiple pre-commit
  passes needed
* there was (already unused) deprecated dev dict generator that used
  its own black formatting.

There were also couple of problems with the files generated by
stubgen itself:

* Union was missing in the generated stubs (this is a known issue
  with stubgen: python/mypy#12929
* Intellij complained on Incomplete import from _typeshed missing

This PR fixes all the problems:

* black formatting is now consistenly extracted and applied everywhere
* when needed, is_pyi flag is passed to black so that it knows
  that .pyi file is being fomratted
* EOL is added at the end of file when the file is generated
* Union is added to the generated stub
* noqa is added to _typeshed import
* the dict generator is removed

As the end result, generated stub files are fully importable
(no errors reported by IntelliJ IDE) and consistently formatted
every time.

GitOrigin-RevId: 129f0820cd03c721ebebe3461489f255bb9e752c
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Nov 8, 2024
There were couple of problems with static checks generating source
files including generated stubs in the common.sql package:

* black formatting was implemented in multiple separate scripts
  making it harded to fix problems in all of them
* generated stub files were not formatted with is_pyi=True and
  black had no way to figure it out because it was working on strings
* black formatting was not consistently applied in all places
* EOL at the end of generated stub file was missing, leading to EOL
  fixer adding them after generation leading to multiple pre-commit
  passes needed
* there was (already unused) deprecated dev dict generator that used
  its own black formatting.

There were also couple of problems with the files generated by
stubgen itself:

* Union was missing in the generated stubs (this is a known issue
  with stubgen: python/mypy#12929
* Intellij complained on Incomplete import from _typeshed missing

This PR fixes all the problems:

* black formatting is now consistenly extracted and applied everywhere
* when needed, is_pyi flag is passed to black so that it knows
  that .pyi file is being fomratted
* EOL is added at the end of file when the file is generated
* Union is added to the generated stub
* noqa is added to _typeshed import
* the dict generator is removed

As the end result, generated stub files are fully importable
(no errors reported by IntelliJ IDE) and consistently formatted
every time.

GitOrigin-RevId: 129f0820cd03c721ebebe3461489f255bb9e752c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants