404 error when creating files using github_repository_file resource

[kunalnanda]

Terraform Version

= v1.0.5

Affected Resource(s)

github_repository_file

Terraform Configuration Files

resource "github_repository" "repos" {
  for_each = local.repos

  name         = each.key
  description  = join("", [each.value.description, " - Managed by Terraform"])
  visibility   = each.value.visibility
  homepage_url = each.value.homepage_url
}

resource "github_repository_file" "gitignore" {
  for_each = local.repos

  content             = file("templates/.gitignore")
  file                = ".gitignore"
  repository          = github_repository.repos[each.key].name
  branch              = "main"
  overwrite_on_create = true
  commit_message      = "Managed by Terraform"
  commit_author       = "user"
  commit_email        = "user@email.com"
}

Expected Behavior

The repositories should be created, and a default .gitignore file should be created with contents from the given template file.

Actual Behavior

The repositories are created. The .gitignore file fails with the following error:

Error: unexpected status code: 404 Not Found

with github_repository_file.gitignore["terraform"],
on repos.tf line 10, in resource "github_repository_file" "gitignore":
10: resource "github_repository_file" "gitignore" {

This error is repeated 27 times, one for each repository.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids

I am running this against my personal Github account.
My Github token has the following permissions:

• repo
  • repo:status
  • repo_deployment
  • public_repo
  • repo:invite
  • security_events

[serain]

I'm facing this but only when trying to create files in nested directories.

For example this works: ./circle/config.yaml
And this fails: ./github/workflows/build.yaml

I can only create 1 directory deep.

EDIT: Fixed my issue, in my case it's because to update ./github/worklows you need the workflows scope on your token. Had nothing to do with nested directory.

[jcudit]

Can we get more detailed debug logging posted to the issue? Adding TF_LOG=DEBUG will help us determine which request is causing the 404.

[kunalnanda]

@jcudit Here is the debug log:

2021-09-29T11:24:49.630+1000 [DEBUG] Adding temp file log sink: /var/folders/q_/bp_xl53j0ms3k5y0b7d_c8g40000gn/T/terraform-log994684574
2021-09-29T11:24:49.631+1000 [INFO]  Terraform version: 1.0.5
2021-09-29T11:24:49.631+1000 [INFO]  Go runtime version: go1.16.4
2021-09-29T11:24:49.631+1000 [INFO]  CLI args: []string{"/usr/local/Cellar/tfenv/2.2.2/versions/1.0.5/terraform", "apply", "--auto-approve"}
2021-09-29T11:24:49.631+1000 [DEBUG] Attempting to open CLI config file: /Users/blah/.terraformrc
2021-09-29T11:24:49.631+1000 [INFO]  Loading CLI configuration from /Users/blah/.terraformrc
2021-09-29T11:24:49.634+1000 [INFO]  Loading CLI configuration from /Users/blah/.terraform.d/credentials.tfrc.json
2021-09-29T11:24:49.634+1000 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-09-29T11:24:49.635+1000 [DEBUG] ignoring non-existing provider search directory /Users/blah/.terraform.d/plugins
2021-09-29T11:24:49.635+1000 [DEBUG] ignoring non-existing provider search directory /Users/blah/Library/Application Support/io.terraform/plugins
2021-09-29T11:24:49.635+1000 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2021-09-29T11:24:49.636+1000 [INFO]  CLI command args: []string{"apply", "--auto-approve"}
2021-09-29T11:24:49.652+1000 [DEBUG] Service discovery for app.terraform.io at https://app.terraform.io/.well-known/terraform.json
2021-09-29T11:24:50.943+1000 [DEBUG] Retrieve version constraints for service tfe.v2.1 and product terraform
2021-09-29T11:24:53.116+1000 [DEBUG] checking for provisioner in "."
2021-09-29T11:24:53.116+1000 [DEBUG] checking for provisioner in "/usr/local/Cellar/tfenv/2.2.2/versions/1.0.5"
2021-09-29T11:24:53.117+1000 [INFO]  Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2021-09-29T11:24:53.401+1000 [INFO]  backend/remote: starting Apply operation

[rtjfarrimond]

I was also facing this issue, adding auto_init = true to the github_repository fixed it for me. I did have to manually delete the repo and reapply the terraform config to get it to work.

The issue seems to have been that the repository was empty and had no branches. Adding auto_init = true created an initial commit with empty README (docs here), after which terraform apply was able to successfully create the files in the main branch. I didn't see it directly, but I would imagine that the underlying API call to create the files contains the branch name in the path, which would explain the 404 response received when the branch did not exist.

I also used the github_branch data source to access the main branch that was created by setting auto_init = true and replaced any literal "main" with data.github_branch.main.branch.

[FalconerTC]

Seeing the same thing with the following TF

resource "github_repository_file" "test" {
  repository = "RentTheRunway/bell_labs"
  branch     = "master"
  file       = ".testfile"
  content    = "**/*.tfstate"
}

Produces the following debug logs

2021/11/02 17:25:45 [DEBUG] EvalApply: ProviderMeta config value set
2021/11/02 17:25:45 [DEBUG] module.repositories.github_repository_file.test: applying the planned Create change
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: 2021/11/02 17:25:45 [DEBUG] Github API Request Details:
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: ---[ REQUEST ]---------------------------------------
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: GET /repos//RentTheRunway/bell_labs/branches/master HTTP/1.1
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Host: api.github.com
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: User-Agent: go-github
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Accept: application/vnd.github.v3+json
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Accept-Encoding: gzip
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0:
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0:
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: -----------------------------------------------------
2021-11-02T17:25:45.445-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: 2021/11/02 17:25:45 [TRACE] Acquiring lock for GitHub API request ("RentTheRunway/bell_labs:master")
2021-11-02T17:25:45.755-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: 2021/11/02 17:25:45 [TRACE] Releasing lock for GitHub API request ("RentTheRunway/bell_labs:master")
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: 2021/11/02 17:25:45 [DEBUG] Github API Response Details:
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: ---[ RESPONSE ]--------------------------------------
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: HTTP/2.0 404 Not Found
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Access-Control-Allow-Origin: *
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Content-Security-Policy: default-src 'none'
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Content-Type: application/json; charset=utf-8
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Date: Tue, 02 Nov 2021 21:25:45 GMT
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Server: GitHub.com
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: Vary: Accept-Encoding, Accept, X-Requested-With
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Content-Type-Options: nosniff
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Frame-Options: deny
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Github-Media-Type: github.v3; format=json
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Github-Request-Id: E962:0E29:24C546:471F11:6181ACD9
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Limit: 60
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Remaining: 49
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Reset: 1635890283
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Resource: core
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Ratelimit-Used: 11
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: X-Xss-Protection: 0
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0:
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: {
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0:  "message": "Not Found",
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0:  "documentation_url": "https://docs.github.com/rest"
UG] plugin.terraform-provider-github_v4.17.0: }
2021-11-02T17:25:45.756-0400 [DEBUG] plugin.terraform-provider-github_v4.17.0: -----------------------------------------------------
2021/11/02 17:25:45 [DEBUG] module.repositories.github_repository_file.test: apply errored, but we're indicating that via the Error pointer rather than returning it: unexpected status code: 404 Not Found
2021/11/02 17:25:45 [ERROR] eval: *terraform.EvalApplyPost, err: unexpected status code: 404 Not Found
2021/11/02 17:25:45 [ERROR] eval: *terraform.EvalSequence, err: unexpected status code: 404 Not Found

[lerljaku]

it has something to do with the set PAT.
If i dont set it, url is malformed like that GET /repos//RentTheRunway/bell_labs/branches/master HTTP/1.1

if i set it it fills the username /repos/lerljaku/RentTheRunway/bell_labs/branches/master HTTP/1.1
but it doesnt work if the repo owner is organization instead of user

EDIT:

when passing filling owner to github provider, it works then

provider "github" {
  token = var.gh_pat
  owner = var.gh_org
}

[lerrigatto]

I actually had this error because in my state I have the resource pointing to a branch that doesn't exist anymore, even if the new one I specify was there.
I solved deleting the resource from the state and then re-apply.

[jamesggraf-m1]

I ran into this error and discovered the cause in my case was that there are two Github providers: hashicorp/github and integrations/github. The integrations/github provider works fine, the hashicorp/github provider throws 404 errors. So if your module doesn't explicitly require integrations/github, it'll use the hashicorp provider and blow up.

[ahasna]

Lost 3 hours of my life to this and the solution was to provide owner either explicitly in the provider or through GITHUB_OWNER 😭

Documentation is misleading mentioning that owner is optional.

[rotilho]

This problem happens when there's a mismatch between both implementations (integrations and hashicorp).

How to reproduce:

• Set up a top level provider to integrations/terraform-provider-github with owner property different from the current user
• Create a module that don't define integrations/terraform-provider-github (so it will use the hashicorp/terraform-provider-github by defalt)
• Define a github_repository_file inside the module

Expected behaviour:

• File is created

Current behaviour:

• github_repository_file from a module don't see the integrations/terraform-provider-github owner and try to create the file in the current user repository

[rmn-lux]

I have the same problem. My repo was empty, so i recreate it with non empty readme file and the problem has gone.

[douglascayers]

In my case,

• I already was using both github owner/token variables as suggested previously
• I already was using the integrations/github terraform provider as suggested previously
• My repository already is in use so I didn't want to delete/recreate it

So what I ended up doing was log in to my postgres database and manually update the state JSON. Since I had successfully been able to use github_repository_file resource with other repositories, I copied a good value and tweaked it for the repository that wasn't working. Then, I deleted my local terraform state (rm -f .terraform/terraform.tfstate) and reinitialized from my backend (terraform init --backend-config=${CONN_STR}).

Here's the JSON snippet I added to my state:

{
  "version": 4,
  "terraform_version": "1.1.9",
  "resources": [
    {
      "mode": "managed",
      "type": "github_repository_file",
      "name": "readme",
      "provider": "provider[\"registry.terraform.io/integrations/github\"]",
      "instances": [
        {
          "index_key": "my-awesome-repo",
          "schema_version": 0,
          "attributes": {
            "branch": "main",
            "commit_author": "my-git-username",
            "commit_email": "me@example.com",
            "commit_message": "Created by Terraform",
            "commit_sha": "af1d88db0dc7a616eca89d88ef25235de7cfe41e",
            "content": "my-awesome-repo",
            "file": "README.md",
            "id": "my-awesome-repo/README.md",
            "overwrite_on_create": false,
            "repository": "my-awesome-repo",
            "sha": "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed"
          },
          "sensitive_attributes": [],
          "private": "cyQstM==",
          "dependencies": [
            "github_repository.my-awesome-repo"
          ]
        }
      ]
    }
  ]
}

Note, commit_sha should be the commit that represents the file content, usually the most recent commit to the file in GitHub.

Note, sha is the SHA1 hash of the file contents. To get the known value in GitHub, use the following curl command against the API:

GITHUB_TOKEN="ghp_your_token"
OWNER="your-repo-owner"
REPO="your-repo-name"
COMMIT="commit-hash-of-your-file"

curl -X GET -H "Authorization: token ${GITHUB_TOKEN}" "https://api.github.com/repos/${OWNER}/${REPO}/git/blobs/${COMMIT}"

The JSON response includes the sha value.

[shubham030]

I have the same issue, have checked all the above fixes/work arounds and nothing seems to work.
I am trying to create 20+ files and some of them get created but the remaining end up with the same error
unexpected status code: 404 Not Found

But if I apply the changes again then the remaining files get created.
I have also checked the API rate limit and it looks good.

I am using the following config
cdktf 0.13.1
terraform v1.3.2
github provider -> registry.terraform.io/integrations/github 4.31.0