-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stream large binary file over declarative HTTP client. #2906
Comments
If you must work with an input stream you can probably do something like: Flowable bodyReader = Flowable.create(emitter -> {
try (InputStream in = getInputStream()) {
byte[] buffer = new byte[1024];
int len;
while ((len = in.read(buffer)) != -1) {
if (buffer.length == len) {
emitter.onNext(buffer);
} else {
emitter.onNext(Arrays.copyOf(buffer, len));
}
}
emitter.onComplete();
} catch (Throwable e) {
emitter.onError(e);
}
}, BackpressureStrategy.BUFFER).subscribeOn(Schedulers.io()); Then declare:
And pass the above flowable. |
thanks for your quick feedback, after trying your suggestion we figure out that GitHub upload API does not accept streams it expects the asset data in its raw binary form as a body. any suggestions on how can we send a binary file as a body without load it all into memory using micronaut declarative client. thanks again |
Looks like there is a library from the rxjava folks to handle converting an inputstream to a byte array observable https://github.com/ReactiveX/RxJavaString. I don't see anywhere in that document that says the api doesn't accept streams. Even if you buffered the whole file in memory it will still be streamed because it cannot send the entire file in a single chunk. You likely need to choose a media type that matches the files contents. |
@soufianenassih my suggestion should work the same, just change the content type to |
I'm having a similar issue, but rather than streaming to Github I'm trying to stream from one micronaut service to another. I created a How is it supposed to be consumed on the receiving service? I have the matching controller implementation, and am subscribing to the flowable but receiving nothing. I verified the InputStream is reading data correctly. |
I am doing the same except i am using reactor library. The problem is I am trying to restream the data from a @controller endpoint to @client. It works as expected for small files but I am limited to 10mbs. |
provide examples of what doesn't work and maybe we can help |
Thanks for quick response :) io.micronaut.http.exceptions.ContentLengthExceededException: The content length [1073741824] exceeds the maximum allowed content length [10485760]
|
so for these kinds of proxying use cases involving the client today you have to write a filter https://docs.micronaut.io/latest/guide/#proxyClient Certainly it would be nice to be able to use a controller but due to the way buffering works in Netty it makes it hard to implement. |
Thank you very much for your help. Big father of Micronaut 😇 Will definitely rewrite it then. |
we need to upload large files "nightly GraalVm artifacts" by using GitHub release API and the declarative micronaut HTTP client we try
MediaType.MULTIPART_FORM_DATA
and the@body
as MultipartBody but without success because GitHub upload needs a stream of binary data.after that, we try
MediaType.APPLICATION_OCTET_STREAM
and it works as expected withbyte[]
the only limitation with this approach is that it needs an array of bytes, and for our situation loading a large file to memory is not a good idea.Question
Are they any examples to stream large files using declarative HTTP client or any intention to support
InputStream
for theMediaType.APPLICATION_OCTET_STREAM
Actuale usage
Maybe supporting this on the future
The text was updated successfully, but these errors were encountered: