Allow a dedicated dependency file for magic (not package.json or readme)

[jakenvac]

First off, I'd like to say that I love the concept of this project and I really like the idea of a readme outlining (and defining) a projects dependencies. but hear me out!

I have put together a couple of reasons why I think restricting the dependencies to existing files/formats hurts the usage of this project more than it aids it. Particularly when working in a team. (I understand that motivating developers to use the readme is likely the main reason this isn't already a feature)

• By only allowing a magic environment to be defined in a file that is likely to already exist in a project makes tea harder to sell to colleagues/peers/other devs. Different people have different opinions on what a readme should/shouldn't contain, likewise for a package.json. By having a dedicated config in source control, devs can learn the advantages of tea at their own pace. Once they see how great it is, convincing them that maybe it's time to move this to the readme becomes an easier sell.

• In a similar vein, not all teams are going to want to adopt this tool, if there were a dedicated config file one could add it to the .gitignore, use tea to their hearts desire without impacting the project or team directly and still reap most of the benefits of the tool. This currently doesn't seem possible in a nodejs project with an existing readme.

• While markdown is great, there are other great markup formats such as asciidoc that all have their pros and cons. Forcing a particular format goes against the beauty of having different tools to suit different needs. Especially if a project has an extensive readme already written up in a different format. Personally I love markdown, it's my preferred syntax but I think it's wrong to ignore where it falls short and why it may not always be the best fit for someone.

• But you can also use package.json! Sure but why? Package.json is a project specific file that exists to outline the metadata about a node.js project, why should this dictate the environmental config for what could be a project much larger than just the node component? Furthermore the mere existence of a package.json implies a lot about a project. It's out of place and potentially misleading to exist in a golang project for example. This leads on to my next point.

• By glancing at a directory we can typically make relatively safe assumptions. Eg. 'Ah so there's a package.json, a tsconfig.json and an .env file, it's very likely this is a node project, written in typescript with some environmental config'. This goes for all kinds of tooling and their respective configs. By having, lets say, a teaxyz.json, it's immediately obvious that there's some environmental config I can use to get up and running. This is almost as powerful as having the deps right there in the readme.

These criticisms are with the best of intentions, I truly believe this project could change the face of development. Apologies if they come across too harshly, I have a tendency to over explain things.

I'm definitely open to hearing why my outlined points are all wrong!

If this is something that the maintainers of the project would be willing to accept then I'll gladly do the work and submit a PR.

[jhheider]

I don't think your points are wrong, @jakeHL. I've long thought that allowing a new Regex("^\.?tea.(json|ya?ml|toml$") as a second source of truth would be advantageous to some. Are you interested in implementing that? It obviously makes configuration search more expensive (if we have to check 9 possibilities rather than 1, though that isn't necessarily a killer). One way to do it without incurring that cost universally might be to add a --config, -c option to read a specific config file (though it'll require a syntax and parser support at present if it's not Markdown).

We definitely want this tool to be useful to the way teams work, which is why we didn't make all these choices in final fashion up front. tea is for everyone, and we're willing to entertain all sorts of PRs for new features.

I will say that philosophically, we like tools that Just Work they way people expect them to (so, obeying the principal of least surprise, and preventing simple interfaces to complexity). One of the simplest way to get there is to have opinionated tools, because linear path from A to B is easy to reason about, test, and debug. But I do see this idea as a natural extension of modern tooling idioms.

[jakenvac]

Hello, I'd definitely be interested in doing the work.

I'm not familiar with the codebase yet so maybe I'm being naive but if by 'expensive' you mean in terms of performance I'd say it's not unreasonable to clearly define a hierarchy of priority.

For example if a tea.json existed, can we assume that there would be no need to check the readme? I can't think of a reason tea.json would exist if a developer still intended for the readme to be the source of truth.

After that I think it comes down to setting some ground rules, such as json takes precedence over yaml takes precedence over toml. It's arbitrary but I'd argue that's probably the order of popularity in terms of those formats so it'd solve the performance issue for the majority of users.

I guess it's important to ask does it even need to support different structured data files?
Why not just json(c)? This keeps Tea opinionated while still allowing the flexibility of a dedicated config. I somewhat arbitrarily suggested json, it could be any format, but seeing as the project is built on Deno, why not leverage the jsonc parser from the standard library.

I personally think by picking one format it enhances the speed of "visually grepping" a directory to infer things about a project.

You can easily counter that argument with the same argument I made about markdown vs asciidoc, however!

[jhheider]

If we (or you) only want to implement JSON(C), I don't see any issue with that as a first step. I'll note that we use YAML front matter in README.md, and the package definitions in the pantry are in YAML as well, so that might be a more natural "first" config format. But I certainly don't dictate terms to those excited to implement their favorite feature. ;)

[jakenvac]

I'm absolutely happy to go with yaml, I'd like to keep it consistent. You raise a good point regarding the front matter. It would make it easy to lift and shift from one place to another.

Thanks for the discussion! I'll get started on a PR.

[mxcl]

I feel package.json is generic. Node doesn’t own it. Right now it does mean node but it doesn’t have to—tools like ours choosing it can effect that. It is literally package DOT json, not node.json.

I don’t quite buy the arguments that teams won't adopt this because there isn’t a “proper file”. README is the right place for this information and markdown makes the data machine readable. Every project has a README. Teams don’t care about files they care about their toolsets meeting their needs. They try out new tools based on those tools meeting needs that are not met and the requirements to make use of those tools being low enough for experimentation.

Having said this I'm fine with 9 files. 18 files. 100 files. The performance is not to worry about for checking files. The UX is mostly fine (we will need a verbose output that says which files were discarded or used and that).

So, sure. Let's leave this open a couple weeks so others can chime in though. IME open source works because we garner lots of feedback and we may be wrong.

I propose: tea.yaml also allowing .github/tea.yaml (because some people really hate more files in their project root, and I get that).

[jakenvac]

Teams don’t care about files they care about their toolsets meeting their needs. They try out new tools based on those tools meeting needs that are not met and the requirements to make use of those tools being low enough for experimentation.

I completely agree that this is the way that it should be, but anecdotally I can confidently say that it's not always the case. I work as an independent software contractor so find myself working as part of all kinds of teams - there's always people with different views of what they think the purpose of a readme is. Especially when working with a monorepo.

On the flip side, however, I do think its perfectly okay to want to stay opinionated. It's okay to state "if you want to use tea, this is the way it is" after all it is ultimately your project, @mxcl. I just thought I'd weigh in with my views and experiences.

I think I feel so strongly about this as I can see tea really change of the face of development specifically in a team context. A solo dev can do whatever they want with their environment. Keeping it consistent for many developers is where its strengths lie, in my opinion.

I will say I completely disagree with your point re package.json I understand where you are coming from, sure it's not named after the framework it's used within, such as cargo.toml for example. However it's a very widely known and commonly used file for node.js. It'd be like adding tea dependencies to a python requirements.txt. Developers start to get accustomed to the files they see in projects they visit.

I started work on adding tea.yml support yesterday and I'll continue with it and submit when it's ready. There's no pressure from me to merge or not, I'd like to contribute to tea in one way or another and this has been a good introduction to the codebase.

Thanks for your hard work on tea so far and for your work on brew in the past. Both great tools!

[mxcl]

Part of the reason I picked README and package.json is in my experience the people who make choices about what tools to use are far more apposed to adding “yet another f**ing file to the project root*. So we certainly are coming at this with different anecdotes.

This is also part of the reason we try to infer the environment automatically with our “magic” (though this only works for a couple of situations currently).

I just don’t really feel that this issue makes much difference: devs pick tools based on needs, not file choices. I removed a pain point by choosing existing files (not another f***ing file) which is a plus. If they hate README they can use package.json (it's node’s file, but also it's not, we're not the first tool to add custom namespaces there. Additionally I don't really see how it matters if people see the package.json and think node, they can make all the assumptions they like about a project from its file structure, they will still need to read the README to know how to “get started” where they will learn they actually need whatever and they can just do tea start if they want. As team devs or solo devs how often do people actually come across new projects? Not that often. I personally rarely read the file listing as my first step, I scroll to the README, there's a reason GitHub emphasize it so).

My worry with adding tea.yml is we are then making things more complex rather than less. We already allow two files.

I said we can have 8 but really I was thinking about how our magic works.

Maybe it doesn't matter. I'd need to think about it more.

[mxcl]

I guess really it should be package.yml and use the same format as the pantry.

We have a messy story currently with all this stuff TBH.

[jakenvac]

I do understand where you are coming from entirely. It's completely fair to be frustrated at the state of 'every tool needs some config and a file in the root'. Personally I'd like to see a source control agnostic version of the .github folder to solve that issue, but that's a topic for another day lol. So for now I guess I'm used to the stick it all in the root approach. I think trying not to make the tool too complex is a valid concern, too. That's a challenge that will be ever present throughout the life of a tool such as this, so I can't fault you for wanting to get ahead of it.

I think having package.yaml is a good compromise. Especially if it's consistent with other schemas used in the project already.

I have a draft PR open for this work that has pretty much complete support for a tea.yml now. It'll not be any effort at all to switch it to package.yml if you are interested in this behavior. For now I will probably park the remaining work outlined in the PR until there is a more concrete decision. I'm also happy to continue it if you think having something to test might help make that decision, regardless of the outcome.

As I said previously, it has been a good intro to the codebase so I can look for other areas to contribute in hopefully more meaningful ways.

[jhheider]

Personally I'd like to see a source control agnostic version of the .github folder to solve that issue, but that's a topic for another day lol.

Possibly normalizing:

mkdir .gitignore.d
echo .gitignore.d >>.gitignore

or adding .gitignore.d support to git.

[jakenvac]

@mxcl I've just come across what I think could be another reason not to use package.json. I guess it doesn't necessarily argue for another file as the readme.md is still an option.

But just for visibility:

Many text editors (I tested with Kakoune, Neovim and Helix) that work with the Language Server Protocol will use project specific files to identify the root of your workspace. (Often paired with a .git directory). I'm currently working on a Deno project and added a package.json with tea deps and immediately my editor started thinking it was a node.js project and started giving me incorrect auto complete, editor hints, errors etc.

There are fixes that can be done, such as changing the config in your editor to look for a deno.jsonc before a package.json etc. but I feel like that goes against the ethos of keeping Tea as simple as possible.

Again, not necessarily an argument for/against package.yml but could be worth at least mentioning in the readme that package.json might have unintended side effects.

Sorry for typos/formatting, I wrote this on mobile.

[mxcl]

immediately my editor started thinking it was a node.js project

this is a pretty good reason to not standardize on it.

I don't wish to fight with existing tooling, that's a bad approach.

I think standardizing on package.yml is maybe a great idea. I'm still noodling it.

[jhheider]

Since (currently), tea is usable with any project, but not required, preferring a dotfile over not might be better. In my head, it's always been .tea.yml for this reason. Then third party tooling (like editor plugins) can reliably detect tea features and behave accordingly. I also don't like clutter in my project root, but dotfiles feel like what I expect, as in a traditional home directory.

[jakenvac]

I'll try and get some time this weekend to work a little more on the PR I have open then, see what you guys think. Might be useful to play with it a bit to get a feel for if it seems like the right solution.