Uploading a new version to a target box file using box source file not working after SDK upgrade

[hmousaiti]

• I have checked that the SDK documentation doesn't solve my issue.
• I have checked that the API documentation doesn't solve my issue.
• I have searched the Box Developer Forums and my issue isn't already reported (or if it has been reported, I have attached a link to it, for reference).
• I have searched Issues in this repo and my issue isn't already reported.

Description of the Issue

1. App user is an owner of a folder, and also is a collaborator with role “Viewer” in another user’s file.
2. App user is trying to download the file from the user’s box account and upload it as new version to the owned folder.
3. The download and upload process happen in parallel and in an asynchronous way, downloading the file from box and writing it to an output stream, meanwhile uploading the downloaded part using an input stream connected to the output stream.

The above behavior is being done using PipedOutputStream, and PipedInputStream. It is working fine using Box SDK 3.3.0. But, when we migrated to use SDK 4.0.1, we started to get and exception that:

java.util.concurrent.CompletionException: java.lang.RuntimeException: java.io.IOException: Pipe closed
at org.springframework.aop.interceptor.AsyncExecutionAspectSupport.lambda$doSubmit$3(AsyncExecutionAspectSupport.java:281) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.interceptor.AsyncExecutionAspectSupport$$Lambda$1300.000000001A83A8F0.get(Unknown Source) ~[na:na]

Steps to Reproduce

1. Spring boot application which is downloading a file from box and upload that downloaded file as a new version to another target file.
2. Given PipedOutputStream out, and PipedInputStream in = new PipedInputStream(8192).
3. in.connect(out);
4. Download the file to the "out"
5. Upload the new version from the "in"

Expected Behavior

Error Message, Including Stack Trace

java.util.concurrent.CompletionException: java.lang.RuntimeException: java.io.IOException: Pipe closed
at org.springframework.aop.interceptor.AsyncExecutionAspectSupport.lambda$doSubmit$3(AsyncExecutionAspectSupport.java:281) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.interceptor.AsyncExecutionAspectSupport$$Lambda$1300.000000001A83A8F0.get(Unknown Source) ~[na:na]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1601) ~[na:1.8.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) ~[na:1.8.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:1.8.0]

Screenshots

Versions Used

Java SDK: 4.0.1
Java: 1.8.0_291

[arjankowski]

hi @hmousaiti ,

Can you give us a complete code sample that works and generates the mentioned error, without using springframework?
This will help us a lot in further work on this bug.

Thanks!

[antusus]

Hello @hmousaiti,
you can even replicate that with a simple service using spring framework. It looks like somehow it interferes with SDK. Both Spring Framework and SDK are using OkHttp maybe updating spring framework can help?

[hmousaiti]

@antusus I am attaching a separate app without using spring framework where the issue can be replicated.
lh-box-integration-external-main.zip

[mhagmajer]

Thanks @hmousaiti for providing an example, we'll take a look

[antusus]

@hmousaiti
I was able to look at your code and you have a race issue in your code. You are running two threads. One (call it thread1 reads data into input stream. This input stream also feeds output stream. Now second thread reads from output stream and uploads new version of the file. Then you start both threads ad once and this is the problem - you loose control over which threads works when.
And it might happen that thread1 reads from box 319 bytes and then thread2 runs and wants to send 8192 bytes but there are not enough bytes to sent and all thing collapses.
You can observe that by running this simple example:

public class BoxIssue
{
    public static void main( String[] args ) throws IOException
    {
    	Thread t1=new Thread(){
    	public void run(){
			// loops make it wasier to spot
			for (int i = 0; i < 10; i++) {
				System.out.println("Thread 1");
			}
	    }};


    	Thread t2=new Thread(){
    	public void run(){
			for (int i = 0; i < 10; i++) {
				System.out.println("Thread 2");
			}
    	}};
		
    	t1.start();
    	t2.start();
    }
}

You might see Thread 1 and Thread 2 mixed.
In order for this to work you will need to wait for the whole file to be read into the input stream. Or create a synchronisation mechanism to split the writing in 8192 bytes batches. Last batch can be smaller than 8192. This number comes from here.

[hmousaiti]

@antusus

The same exact code works perfectly with box versions lower than 4 without facing any issues. The issue appeared only when we migrated to 4.0.1.

Also, the application I attached above is a working application and the instructions to run it is in the readme file. I appreciate if you can apply the solution you are suggesting there so that I can better understand it.