pyproject.toml (PEP 518) support

[asottile]

In GitLab by @valignatev on Jun 11, 2018, 10:58

Hi! Is there a plan to support pyproject.toml from PEP 518?
Specifically, it would be nice if flake8 could understand [tool.flake8] section in the pyproject.toml file.

[asottile]

In GitLab by @jgirardet on Jul 28, 2018, 06:04

Hi,
I'm ok to do it.
I have a question for maintainers concerning the toml thing, which way would you like it to be implemented ?
some possibilities:

• add a toml libray to flake8 dependency, which one (toml, tomlkit, other)
• only use pyproject.toml if a toml libray is importable (which would let to the user to add a toml lib in his requirements. still the question, which toml libray would we support ?
• simply parse the pyproject.toml to extract the [tool.flake] section and deal with. not the best, probably not every usecase, but could be enough.

As simple implementation, all those possibility should return a dict to add to a configPArser.

Thank for your advice.
Jimmy

[asottile]

In GitLab by @valignatev on Jul 28, 2018, 07:22

I'm nobody, of course (in the terms of flake8 project), but I personally think that pyproject.toml should be the first-class citizen since it's in the PEP. So the first option.

[asottile]

In GitLab by @tmose1106 on Jul 29, 2018, 11:58

I am also not affiliate with flake8 in any way, but here is my two cents:

At the moment, is hard to tell which toml library to choose. As of today I have found the toml library to work just fine for parsing. It is also worth noting that if poetry picks up usage in the future, tomlkit could very well become the dominant toml library.

Specifically to @jgirardet's question, I feel that the second option would make the most sense at the moment. Simply make the toml library an optional dependency and if there is a pyproject.toml file present, use it. I feel that the first option can be best handled as a future feature request once the dust has settled some more on poetry adoption so that we can make an informed choice on a mandatory toml library dependency and phase out some of the other configuration methods.

[asottile]

In GitLab by @jgirardet on Jul 30, 2018, 03:47

mentioned in merge request !245

[asottile]

In GitLab by @d_ on Jan 30, 2019, 02:38

mentioned in commit d_/dlc-gui@248a6b88d5d620dac793b7856fc217b0e1210e5d

[asottile]

In GitLab by @d_ on Jan 30, 2019, 02:50

mentioned in commit d_/dlc-gui@ba4aadd9fccb0568c8712321c664fa83a7bca36a

[asottile]

In GitLab by @radek-sprta on Feb 28, 2019, 24:56

A temporary solution could be what tox did -- just inline the ini configuration under tool.flake8.legacy_flake8_ini key.

[asottile]

In GitLab by @asottile on Feb 28, 2019, 07:37

@radek-sprta iirc tox did that because they use a special ini parser with extra features (substitutions etc.). Doing that for flake8 wouldn't really help (you'd still need toml to parse that)

[asottile]

In GitLab by @rossmacarthur on Mar 27, 2019, 03:26

I would really like this to be done 🙏, I don't mind working on it.

@jgirardet @tmose1106 my opinion is to use the toml library it seems to be much more widely used than tomlkit (based on https://pypistats.org) and we don't need the style preserving that tomlkit offers.

I don't think pyproject.toml support should be a feature/optional dependency because it's in the PEP.

[asottile]

In GitLab by @asottile on Mar 27, 2019, 08:31

I don't think pyproject.toml support should be a feature/optional dependency because it's in the PEP.

Just want to point out that both of the "pyproject.toml PEPs" (517 and 518) are about defining a build system for pip and aren't speccing that projects should stuff configuration in there (PEP 518 briefly mentions the tool table but is referring to configuration of PEP517/PEP518 build tools). See also this thread from pypa/pip further about that topic.

[asottile]

In GitLab by @d_ on Mar 30, 2019, 13:13

mentioned in commit d_/dlc-gui@74b0d42bdfb742217720dc2d806d2fbbcda93341

[asottile]

In GitLab by @ethanhs on May 15, 2019, 15:30

FWIW, I read the PEP as allowing configuration of any tool, and asked on the packaging discourse. Based on https://discuss.python.org/t/pep-518-clarify-scope-of-tool-section/1682/2 I think it is safe to say that all tools are allowed in pyproject.toml, and it isn't meant just for build tools.

[asottile]

In GitLab by @sigmavirus24 on May 18, 2019, 11:11

@ethanhs As I understand Anthony's comment, he was responding to the assertion that Flake8 should grow this because the PEP allows for tools to use the file. The PEP doesn't require tools to use pyproject.toml and thus the assertion that "it's in the PEP" doesn't make sense.

If Flake8 were a tool like homebrew or pip that could reasonably report usage to us, we could say "Well we can deprecate X, Y, and Z because no one is using it to allow for the surface area to expand to supporting pyproject.toml." As it is, we have a poor understanding of what's used, what isn't, and what makes sense to adopt.

I don't think anyone is saying "pyproject.toml" is out of the question. It's just a matter of the fact that the existing configuration processing in Flake8 is ridiculously complex because of historical reasons and backwards compatibility. I would feel far more comfortable (if I were still the primary Flake8 maintainer) if I could deprecate a bunch of the config related things before adding support for another file.

Further, while there hasn't been an explicit "yes" or "no", I'd like to remind everything that "No is temporary, but yes is 'forever'" as far as any responsibly maintained project is concerned. In other words, given our existing attempts to provide backwards compatibility, it's entirely likely that we'd never be able to get rid of this once it was added. That's a long term maintenance nightmare, sadly.

[asottile]

In GitLab by @Stavros on May 27, 2019, 10:33

If Flake8 were a tool like homebrew or pip that could reasonably report usage to us

Why can't it? I'd opt in.

It's just a matter of the fact that the existing configuration processing in Flake8 is ridiculously complex because of historical reasons and backwards compatibility

Is there not, essentially, an order with which ini files are processed? If so, adding one more file to that list doesn't sound terrible, although I do see your argument about having to then maintain that for ever...

[asottile]

In GitLab by @jgirardet on May 27, 2019, 12:09

Initially I thought it was great to add it then I started !245.
Only one config file for everything : config, build system, packaging, third party tools : a dream ❤️

And I read the opinion of Guido in rejecting pyproject.toml support in mypy : python/mypy#5205

Maybe he is right. Maybe old tools should keep .ini support and don't bother pyproject. Maybe only new tools should try to support pyproject.toml and ignore .ini files (this is what black did).

So I will not go further for now in !245. I think we should close !245. But if maintainers really want to add it, I'll then finish the job.