Avoid arg limits for internal uses of Bazel

[ofek]

Bazel supports supplying arguments that may exceed length limits via files. This uses the --target_pattern_file option for commands that accept targets and the --query_file option for commands that accept a query.

An exception is made for users invoking Bazel through the wrapper since there is no need for checking what was already successful.

The command bzl now matches the name of our internal Bazel wrapper and satisfies a comment from the previous review.

[Ishirui]

The approach seems a bit unsafe to me.
Maybe it would be better to go through the command list and pull out only things that look like targets or queries, instead of relying on the order or the presence of a --. We can then write these targets or queries into a file and leave the rest of the command intact.

[Ishirui]

I don't think this is super safe - from my testing click does not include the -- in its command list.

If we disable the ignore_arg_limits context manager in src/dda/tools/bzl/__init__.py, and debug we get the following:

dda bzl --version -- build hello
...
print(command)
>>> ["--version", "build", "hello"]

Now I get that in most contexts commands passed directly from click will have __ignore_arg_limits == True, but it seems dangerous to me to rely on a part of the command that our CLI framework itself omits.

[ofek]

I totally forgot about that, great catch, thanks! I added a new utility that does the proper thing. In a separate PR I'll apply the utility to any other existing commands that exclusively forward arguments.

[Ishirui]

I also don't think this approach will work for every possible invocation of bazel.
For example:

bazel build -- --color no //target

Is a valid invocation, but we don't want the --color no part to be included in the file containing
We need to enforce a stricter structure to the command, where anything after -- is only arguments to the command (i.e. targets or queries), and disallow passing flags there.

This is going to be a bit tricky tho:

1. As mentioned above, click does not pass the -- marker, so it will be hard to rely on it
2. As seen in my example, some flags can themselves accept arguments, so we can't just rely on the presence of - at the beginning of the arg.

[ofek]

Maybe it would be better to go through the command list and pull out only things that look like targets or queries

I considered this but in order to do that you need knowledge of every supported option to understand whether it's a flag or if it accepts a value (and flags can also accept values apparently). Relevant documentation:

• https://bazel.build/reference/command-line-reference
• https://bazel.build/run/build#specifying-build-targets

As you can see in the latter link, targets may also begin with a hyphen to indicate negation.

I went through various approaches over the weekend and realized any parser would be fraught with issues so I chose to enforce the separator for internal uses.