Manual deployment hangs with a deployment folder that starts with a dot

[d4h0]

Describe the bug

Executing 'netlify deploy -d .build' does not find any files, and hangs during the upload.

Here is the output of a tried deployment:

Deploying to draft URL...
✔ Finished hashing 0 files
✔ CDN requesting 0 files
✔ Finished uploading 0 assets
⧇ Waiting for deploy to go live...

You see, no files were found (even so there are files).

To Reproduce

Steps to reproduce the behavior:

1. Setup a Netlify project
2. on Linux, create a hidden directory (a directory that starts with a dot, i.e. ".deploy")
3. Put some HTML files into the hidden directory
4. Start the deployment of the hidden folder

Configuration

• If possible, please copy/paste below your netlify.toml.

I'm using the default netlify.toml (I didn't change anything).

• Please enter the following command in a terminal and copy/paste its output:

  System:
    OS: Linux 5.8 Arch Linux
    CPU: (8) x64 Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz
    Memory: 1.96 GB / 15.52 GB
    Container: Yes
    Shell: 5.0.18 - /bin/bash
  Binaries:
    Node: 14.10.1 - /usr/bin/node
    npm: 6.14.8 - /usr/bin/npm
    Watchman: 4.9.0 - /usr/bin/watchman
  npmGlobalPackages:
    netlify-cli: 2.61.2

Expected behavior

The CLI finds files in the deployment folder, and the upload succeeds (at least, if there is a problem, there should be an error message)

CLI Output

See above

Additional context

After changing the name of the deployment folder from ".build" to "build", everything works as expected

[monsonjeremy]

@erezrokah Any advice on how this should be handled?
Here's the discovery I did:

It looks like this is the desired behavior, here the hashFiles function is called using the given dir
https://github.com/netlify/js-client/blob/4cb1f240bac527305ff98032a801accf22dea5bf/src/deploy/index.js#L49

The hashFiles function uses the defaultFilter which ignores hidden files/directories
https://github.com/netlify/js-client/blob/4cb1f240bac527305ff98032a801accf22dea5bf/src/deploy/hash-files.js#L20
https://github.com/netlify/js-client/blob/4cb1f240bac527305ff98032a801accf22dea5bf/src/deploy/util.js#L12

This seems like the intended behavior in order to prevent walking all hidden directories (.git and others). It seems to me that the CLI should throw an error if the -d flag is passed a hidden directory.

[d4h0]

Throwing an error would already be an improvement, but why should it be impossible to deploy from a hidden directory? A hidden directory seems like a reasonable choice for generated files that the user normally doesn't interact with.

[monsonjeremy]

Throwing an error would already be an improvement, but why should it be impossible to deploy from a hidden directory? A hidden directory seems like a reasonable choice for generated files that the user normally doesn't interact with.

I agree that it's a valid use case. I'm not very familiar with the code, but it seems it would require a bit more effort to build this feature.

I'd be happy to implement, just asking for some direction before taking this one.

[d4h0]

Okay, thanks. I guess it wouldn't be the end of the world if hidden directories are forbidden (just a little odd).

[monsonjeremy]

@d4h0 I've opened two PR's to address the issue. If that isn't the desired outcome then it will be pretty easy to implement code to throw an error.

[erezrokah]

Thanks @monsonjeremy, seems like this is also related to the second bullet here #1225 (comment)

[erezrokah]

I was able to reproduce the issue, but only if there is no netlify.toml file in the project, see https://github.com/erezrokah/netlify-build-reproductions/tree/cli/issue_1227.

git clone git@github.com:erezrokah/netlify-build-reproductions.git
git checkout cli/issue_1227
netlify deploy -d .build

@d4h0 can you confirm that running touch netlify.toml (creates an empty file) serves as a workaround?

[monsonjeremy]

@erezrokah I can confirm the same behavior as you have seen. Interesting. Do you think we should still handle the case where there is no netlify.toml? If not, maybe we can throw an error if the directory is a hidden one and the netlify.toml file does not exist?

I'm not sure that adding netlify.toml actual causes all the files to be found? From my local testing, it logs out the following

✔ Finished hashing 1 files
...

I believe that is only the netlify.toml file that is being hashed though.

I tested this on the branch with my changes, and it logs out:

✔ Finished hashing 2 files
...

To me this confirms that the issue still exists, even with an empty netlify.toml.

[erezrokah]

There are 2 separate issues here I think:

1. Not having a netlify.toml makes the deploy stuck (happens when dir starts with a . or trying to deploy an empty directory).
2. Netlify CLI ignore hidden files/directories while deploying.

[monsonjeremy]

@erezrokah I agree, however I think they are both related. Fixing the second bullet will also fix the first one since the hidden directory will be picked up, allowing the deploy to work because files are found.

[erezrokah]

Fixing the second bullet will also fix the first one since the hidden directory will be picked up, allowing the deploy to work because files are found.

Correct, but it will fix only one variation of the first bug. Trying to deploy an empty folder (without a netlify.toml file) will still get stuck).