Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using the emulator, upload a file then downloading it return 404 #209

Open
Raffaello opened this issue Oct 21, 2022 · 4 comments
Open

using the emulator, upload a file then downloading it return 404 #209

Raffaello opened this issue Oct 21, 2022 · 4 comments

Comments

@Raffaello
Copy link

Describe the bug
A clear and concise description of what the bug is.
uploading a file succesfully, then when try to downloading is returning 404.

[SERVER] All services started
"POST /upload/storage/v1/b/local-bucket/o?name=Example.xml&uploadType=resumable HTTP/1.1" 200 -
No file to remove '_resumable/6effa36d984da5caa33e4f5fb3a0f70e54ab18f801d03504f7cd2fdaa38e2cc5'
"PUT /upload/storage/v1/b/local-bucket/o?name=Example.xml&uploadType=resumable&upload_id=local-bucket%3AExample.xml%3A2022-10-21+13%3A31%3A18.531160 HTTP/1.1" 200 -
Resource not found:
**resource 'b' not found**
"GET /b/local-bucket/o/Example.xml?alt=media HTTP/1.1" 404 -

the url to get /b is the standard GCP Storage url bucket /o obect.
it isn't expected to fail on a GET /b url

To Reproduce
A minimal, complete, and reproducible example to reproduce the behavior.

  1. upload a file with NodeJS
  2. downaload a file with NodeJS
import { Storage } from "@google-cloud/storage"
import process from "process"

const HOST = "localhost"
const PORT = 9023
const BUCKET = "local-bucket"

process.env.STORAGE_EMULATOR_HOST = `http://${HOST}:${PORT}`

const storage = new Storage()

async function upload(filePath, destinationFilename) {
    const resp = await storage.bucket(BUCKET).upload(filePath,{
        destination: destinationFilename
    })
}
// USAGE needs filepath to upload and the name of the file at destination:
// e.g. local_storage_upload mypath/filename.txt filename.txt
upload(process.argv[2], process.argv[3]).catch(console.error)

// usage npx local-storage_upload.ts filetoUpload.txt filetoUpload.txt

download:

const [fileData] = await this.storage.bucket("local-bucket").file("fileToUpload.txt).download({ validation: "md5" });
// this will trigger 404 and throw exception as file not found

Expected behavior
it should download the file

System (please complete the following information):

  • gcp-storage-emulator version: latest, there is no way to retrieve from -h argument or a --version argument.

Additional context
Add any other context about the problem here.

Basically what is saying is "url not found" `b/" the b/ part of the path is the "bucket" in GCP storage. so it should just work without any further issue as it is the basic GCP storage url

@EmandM
Copy link

EmandM commented Feb 16, 2023

Bumping because I also have this issue, however mine is failing on creating a bucket and then trying to create a signedURL on that bucket.

Relevant logs:

storage_emulator_1    | "GET /storage/v1/b/test-bucket?alt=json&prettyPrint=false&projection=full HTTP/1.1" 404 -
storage_emulator_1    | [BUCKETS] Received request to create bucket with name test-bucket
storage_emulator_1    | "POST /storage/v1/b?alt=json&prettyPrint=false&project=test-project-id HTTP/1.1" 200 -
storage_emulator_1    | Resource not found:
storage_emulator_1    | resource 'storage' not found
storage_emulator_1    | "GET /storage/v1/b/test-bucket/iam?alt=json&optionsRequestedPolicyVersion=1&prettyPrint=false HTTP/1.1" 404 -

@Raffaello
Copy link
Author

@EmandM
if i recall it correctly i had to use an ENV parameter detection: when there is STORAGE_EMULATOR_URL or something i was changing slightly the URL i don't remember if it was for mounting the emulator or in the URL of the application . but something like that. anyway i can't help you more than this as i don't remember the workaround that i adopted completely.

@EmandM
Copy link

EmandM commented Feb 16, 2023

This doesn't solve the issue I am having, but is the solution to the original issue googleapis/google-cloud-go#4444 (comment)

The issue is if you use STORAGE_EMULATOR_HOST=http://localhost:9090 the go storage api will overwrite a whole bunch of things. You have to set it as in the issue above. If you use just STORAGE_EMULATOR_HOST=localhost:9090 (i.e. without the http:// then it works fine.

@i-am-ian
Copy link

i-am-ian commented Jan 5, 2024

I ran into this issue today. There's an open issue to address this in the nodejs-storage project.

In the meantime, I found a workaround - rather than setting the STORAGE_EMULATOR_HOST variable, set the apiEndpoint property in the options provided to the Storage constructor.

const storage = new Storage({ apiEndpoint: 'http://127.0.0.1:9023' });

That's working for both uploading and downloading a file for me. I haven't tried other operations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants