Using terragrunt scaffold with GitLab URLs

[gmaghera]

Describe the bug
Scaffold does not seem to work with GitLab URLs

To Reproduce

remote: The project you were looking for could not be found or you don't have permission to view it.
fatal: repository 'https://gitlab.com/rivian/dc.git/' not found
WARN[0000] Failed to find last release tag for git::https://gitlab.com/rivian/dc.git
INFO[0000] Scaffolding a new Terragrunt module git::https://gitlab.com/rivian/dc.git//platform/terraform-modules/s3.git to /Users/gmaghera/Workspaces/dc-terraform/tools/us-east-2/scaffold-sandbox
ERRO[0002] error downloading 'https://gitlab.com/rivian/dc.git': /usr/local/bin/git exited with 128: Cloning into '/var/folders/1q/b7d7p2g91fddxq6jwt237cdm6xcxmy/T/getter2517239202/temp'...
remote: The project you were looking for could not be found or you don't have permission to view it.
fatal: repository 'https://gitlab.com/rivian/dc.git/' not found
ERRO[0002] Unable to determine underlying exit code, so Terragrunt will exit with error code 1

or

❯ terragrunt scaffold https://gitlab.com/rivian/dc/platform//terraform-modules//s3
remote: The project you were looking for could not be found or you don't have permission to view it.
fatal: repository 'https://gitlab.com/rivian/dc.git/' not found
WARN[0000] Failed to find last release tag for git::https://gitlab.com/rivian/dc.git
INFO[0000] Scaffolding a new Terragrunt module git::https://gitlab.com/rivian/dc.git//platform/terraform-modules/s3 to /Users/gmaghera/Workspaces/dc-terraform/tools/us-east-2/scaffold-sandbox
ERRO[0002] error downloading 'https://gitlab.com/rivian/dc.git': /usr/local/bin/git exited with 128: Cloning into '/var/folders/1q/b7d7p2g91fddxq6jwt237cdm6xcxmy/T/getter1330873649/temp'...
remote: The project you were looking for could not be found or you don't have permission to view it.
fatal: repository 'https://gitlab.com/rivian/dc.git/' not found
ERRO[0002] Unable to determine underlying exit code, so Terragrunt will exit with error code 1

not applicable

Expected behavior
Please provide an example for how to do this against a GitLab repository.

Nice to have

• Terminal output
• Screenshots

Versions

• Terragrunt version: 0.55.20
• Terraform version: 1.5.5
• Environment details (Ubuntu 20.04, Windows 10, etc.): Darwin 23.3.0, macOS 14.3.1
  Additional context
  Add any other context about the problem here.

[denis256]

Hi,
git ssh URLs are also not handled correctly?

[denis256]

I suspect wasn't used .git URL to clone repo
In my tests for https://gitlab.com/denis256/terragrunt-tests I used:

$ terragrunt scaffold https://gitlab.com/denis256/terragrunt-tests.git//scaffold/default-template
INFO[0000] Scaffolding a new Terragrunt module git::https://gitlab.com/denis256/terragrunt-tests.git//scaffold/default-template?ref=v10.0.0 to /tmp/test1 
INFO[0004] Running boilerplate generation to /tmp/test1 
[boilerplate] 2024/03/28 20:43:07 Loading boilerplate config from /tmp/boilerplate967725454/boilerplate.yml
[boilerplate] 2024/03/28 20:43:07 Loading boilerplate config from /tmp/boilerplate967725454/boilerplate.yml
INFO[0004] Scaffolding completed

$ terragrunt scaffold gitlab.com/denis256/terragrunt-tests.git//scaffold/default-template
INFO[0000] Scaffolding a new Terragrunt module git::https://gitlab.com/denis256/terragrunt-tests.git//scaffold/default-template?ref=v10.0.0 to /tmp/test1 
INFO[0004] Running boilerplate generation to /tmp/test1 
INFO[0004] /tmp/test1/terragrunt.hcl was updated        
INFO[0004] Scaffolding completed    

[lorengordon]

Using .git on the end of the clone url is something of a github thing. It's not strictly necessary, and for example, codecommit doesn't use/support it at all.

[gmaghera]

Hi @denis256, thanks for looking into this. If you look at my original quote, I did try without the .git piece terragrunt scaffold https://gitlab.com/rivian/dc/platform//terraform-modules//s3 failed.

The error suggests to me that there is an assumption about directory level. Your example apparently uses a personal GitLab project which is always two levels deep: gitlab.com/<username>/<projectname> -- and that works. Compare that to the error I am getting:

❯ terragrunt scaffold https://gitlab.com/rivian/dc/platform//terraform-modules//s3
remote: The project you were looking for could not be found or you don't have permission to view it.
fatal: repository 'https://gitlab.com/rivian/dc.git/' not found
WARN[0000] Failed to find last release tag for git::https://gitlab.com/rivian/dc.git
INFO[0000] Scaffolding a new Terragrunt module git::https://gitlab.com/rivian/dc.git//platform/terraform-modules/s3 to /Users/gmaghera/Workspaces/dc-terraform/tools/us-east-2/scaffold-sandbox
ERRO[0002] error downloading 'https://gitlab.com/rivian/dc.git': /usr/local/bin/git exited with 128: Cloning into '/var/folders/1q/b7d7p2g91fddxq6jwt237cdm6xcxmy/T/getter1330873649/temp'...
remote: The project you were looking for could not be found or you don't have permission to view it.
fatal: repository 'https://gitlab.com/rivian/dc.git/' not found
ERRO[0002] Unable to determine underlying exit code, so Terragrunt will exit with error code 1

The project path is https://gitlab.com/rivian/dc/platform//terraform-modules/s3,
but the error say this fatal: repository 'https://gitlab.com/rivian/dc.git/' not found.
"dc" is a group not a git project, and you gotta go two levels deeper before the s3 project is found.

(and I see that I used the // notation twice, by accident, but I get the same exact error with just one or none, too.)

[gmaghera]

@lorengordon I am not sure about GitHub, but with GitLab if you go into the web UI both SSH and HTTPS URLs end in .git. But they do work without it too.

[gmaghera]

Hi, git ssh URLs are also not handled correctly?

Apparently not. Or I'm not using the call signature correctly...

~/Workspaces/dc-terraform/tools/us-east-2/scaffold-sandbox experiment-with-scaffold* 1h 36s
(⎈|dev-us-east-1:kube-system)❯ terragrunt scaffold git@gitlab.com:rivian/dc/platform/terraform-modules/s3.git
INFO[0002] Scaffolding a new Terragrunt module git::ssh://git@gitlab.com/rivian/dc/platform/terraform-modules/s3.git?ref=4.2.0 to /Users/gmaghera/Workspaces/dc-terraform/tools/us-east-2/scaffold-sandbox
ERRO[0002] error downloading 'ssh://git@gitlab.com/rivian/dc/platform/terraform-modules/s3.git?ref=4.2.0': /usr/local/bin/git exited with 128: fatal: not a git repository (or any of the parent directories): .git
ERRO[0002] Unable to determine underlying exit code, so Terragrunt will exit with error code 1

~/Workspaces/dc-terraform/tools/us-east-2/scaffold-sandbox experiment-with-scaffold*
(⎈|dev-us-east-1:kube-system)❯ terragrunt scaffold git@gitlab.com:rivian/dc/platform/terraform-modules/s3
INFO[0001] Scaffolding a new Terragrunt module git::ssh://git@gitlab.com/rivian/dc/platform/terraform-modules/s3?ref=4.2.0 to /Users/gmaghera/Workspaces/dc-terraform/tools/us-east-2/scaffold-sandbox
ERRO[0001] error downloading 'ssh://git@gitlab.com/rivian/dc/platform/terraform-modules/s3?ref=4.2.0': /usr/local/bin/git exited with 128: fatal: not a git repository (or any of the parent directories): .git
ERRO[0001] Unable to determine underlying exit code, so Terragrunt will exit with error code 1

[denis256]

Hi,
yes, there is an issue... happens in my tests too, after some debugging looks like it is an issue in go-getter library which is used to pull content, something similar happens in go-getter CLI too

$ go-getter "github.com/denis256/s3.git//module2" /tmp/qwe
2024/04/01 18:01:56 success!

$ go-getter "gitlab.com/denis25-tests/dc/platform/terraform-modules/s3.git//module2" /tmp/qwe
2024/04/01 18:02:19 Error downloading: error downloading 'https://gitlab.com/denis25-tests/dc.git': /usr/bin/git exited with 128: Cloning into '/tmp/getter72444115/temp'...
remote: The project you were looking for could not be found or you don't have permission to view it.
fatal: repository 'https://gitlab.com/denis25-tests/dc.git/' not found

Will raise an issue in the library GH project

[denis256]

hashicorp/go-getter#479

[gmaghera]

Thanks for the update @denis256.

[gmaghera]

@denis256 could you remove the "awaiting response" label? Is there something else I can provide?