UFAL/Download instructions for command line don't work

[milanmajchrak]

Original issue: ufal/clarin-dspace#1130 (comment)

Why curl {} brace expansion cannot be used for download commands

Background

The original requirement was to use curl's URL globbing ({}) for compact download commands:

curl -o "#1" "https://server/api/.../handle/123/42{/file1.txt,/file2.txt}"

After thorough investigation, this approach is not viable. Here's why:

Problem 1: Raw (non-encoded) filenames in {} produce invalid HTTP requests

Filenames with spaces, diacritics, or special characters (Médiá (3).jfif) placed directly in {} result in curl sending raw bytes in the URL path. This violates HTTP spec (RFC 7230) — the server returns 400 Bad Request.
If you are trying to use this CURL on windows you must have the right encoding in you terminal.
You find it out using the command chcp

Problem 2: Percent-encoded filenames in {} produce wrong output filenames

curl -o "#1" "url/{M%C3%A9di%C3%A1.jfif,simple.txt}"

The #1 variable is substituted with the literal text from {} — including percent sequences. The file is saved as M%C3%A9di%C3%A1.jfif instead of Médiá.jfif. curl does not URL-decode #1. From the official documentation: "curl does not attempt to decode %-sequences (yet) in the provided file name". There is no --url-decode or similar option.

Problem 3: -J (Content-Disposition) fails on Windows

curl -OJ "url/{M%C3%A9di%C3%A1.jfif,simple.txt}"

The -J flag reads the filename from the server's Content-Disposition header. However, curl on Windows fails with "Invalid argument" when the header contains non-ASCII characters (diacritics, CJK, etc.).

Solution

Use separate -o "filename" "url" pairs for each file:

curl -o "Médiá (3).jfif" "https://server/.../M%C3%A9di%C3%A1%20%283%29.jfif" -o "data.csv" "https://server/.../data.csv"

• -o receives the real filename (with diacritics) directly from the shell — works on all OS
• The URL is properly percent-encoded — valid HTTP
• Works on Windows, Linux, and macOS for any filename

[Paurikova2]

@milanmajchrak
He checked: curl -O <a href="https://dspace-dev.ufal.mff.cuni.cz/repository/server/api/bitstream/11234/1-5814%7B/0/pdtvallex-4.5.xml,/1/vallex.changes%22 rel="noreferrer noopener" title="https://dspace-dev.ufal.mff.cuni.cz/repository/server/api/bitstream/11234/1-5814%7b/0/pdtvallex-4.5.xml,/1/vallex.changes%22 target="_blank">https://dspace-dev.ufal.mff.cuni.cz/repository/server/api/bitstream/11234/1-5814{/0/pdtvallex-4.5.x…

I found item: https://dspace-dev.ufal.mff.cuni.cz/repository/items/34ed276a-fcb5-4199-b160-a3d20c79f8f6
Copied curl and it works for me. curl -O https://dspace-dev.ufal.mff.cuni.cz/repository/server/api/bitstream/11234/1-5814{/0/pdtvallex-4.5.xml,/1/vallex.changes}

[Paurikova2]

@milanmajchrak

THIS IS IMPORTANT

There is unexising url.
Correct url> http://localhost:4000/bitstream/123456789/2-5912/0/test.jfif

And now there is problem:

The Core Problem

In DSpace 5, everything ran on one port (8080).
In DSpace 7+, the system is split:Port 4000 (Frontend): Just a JavaScript (Angular) app. It doesn't have the files.Port 8080 (Backend): The REST API where the files actually live.

Why Browser works vs. curl fails

Browser: Loads the Frontend $\rightarrow$ Runs JavaScript $\rightarrow$ Automatically fetches the file from the Backend.
Curl: Only downloads what is at the URL. Since the Frontend is just an "empty shell," you get 404 or HTML, because curl cannot run the JavaScript needed to find the actual file.

Solutions for the next developer1.

1. Call the Backend directly (Recommended)Identify the internal UUID of the bitstream and use the API endpoint:curl -L http://localhost:8080/server/api/core/bitstreams//content -o test.jfif
2. Find the "True" URLOpen Browser DevTools (F12) > Network tab.Refresh the page and find the request for the file.Use the Request URL found there (it will likely be port 8080).
3. Fix via Proxy (If port 4000 must work)If you need the frontend to serve these files, ensure the Express server is configured to proxy /bitstream requests to the backend:JavaScript// Example in Express
   app.use('/bitstream', createProxyMiddleware({ target: 'http://localhost:8080' }));

Maybe helpfull

dataquest-dev/DSpace#1048