Server Response Errror (Bad Request) when overwriting large hyperfile since v0.26

[kykrueger]

Describe the bug
When updating from version 0.25 to 0.26, we found that one of our scripts which overwrites a datasource started failing.
We have narrowed down the source of the bug to the following change:
307d8a2#r130310838

This error occurs with our servers running version 2022.1.13,

Versions
Details of your environment, including:

• Tableau Server version: bug present when using 2022.1.13, but not with 2022.1.16
• Python version 3.9
• TSC library version 0.26+

To Reproduce
Upload a large hyperfile, ours causing the error is 150+ MB.

Results

    raise ServerResponseError.from_response(server_response.content, self.parent_srv.namespace, url)
tableauserverclient.server.endpoint.exceptions.ServerResponseError:

        400011: Bad Request
                There was a problem publishing the file

[kykrueger]

The problem with the threaded request seems to be a race condition. While debugging, I'd added a breakpoint here:

server-client-python/tableauserverclient/server/endpoint/endpoint.py

Lines 109 to 112 in 307d8a2

	sleep(1)
	if self.async_response != -1:
	# a quick return for any immediate responses
	return self.async_response

, and the upload ran fine. Increasing the sleep time also hid the problem.

[kykrueger]

I've created a PR to fix the issues we were having.
The use of a custom datetime module caught me off guard in the project.
I'd suggest using datetime and timedelta as the source for seconds and minutes instead of manually keeping track of what the seconds and minutes ought to be, but didn't want to make any more changes than necessary to fix the problem.

[kykrueger]

To push this along I've added a check to see if exceptions are returned from the blocking request. This was already returned before this PR, but the type hinting didn't reveal the possibility.

When making my changes, I fixed a pyright error where it had complained that Exceptions could be returned, but were not hinted. This broke the checks for mypy.

Now the returned exceptions will be raised. Before a typeerror would have probably occurred, but now the proper exception will be raised.

[robbertvs]

With tableauserverclient v0.28 and tableau server 2023.1.7 we encountered a similar problem, except TSC did not throw an error. The logs showed that the uploaded chunks were just about 100 bytes:

Publishing <filename>.hyper to server with chunking method (datasource over 64MB, chunk size 50MB)
Initiated file upload session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:30:39 Published 0.00010013580322265625MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:30:42 Published 0.0002002716064453125MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:30:44 Published 0.00030040740966796875MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
...
06:33:38 Published 0.0062999725341796875MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:33:40 Published 0.006400108337402344MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:33:42 Published 0.006428718566894531MB
File upload finished (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
Published <filename>.hyper (JOB_ID: 050ce884-adf5-4c69-b93d-18b1a3afc2c7)

We resolved this issue by downgrading to 0.25

[vizyourdata]

I am seeing the same error in tableauserverclient==0.28

Versions
Tableau Online
Python 3.9.13
tableauserverclient==0.28

My hyper file was very small. The file published successfully and then threw the same error
400011: Bad Request There was a problem publishing the file 'my.hyper'

I downgraded to tableauserverclient==0.25 as suggested and no error.

--Update--
I found this at the bottom of the hyperd log file from the version 0.28 run.
{"ts":"2023-11-02T09:45:37.803769","k":"connection-communication-failure","v":{"std-exception":{"type":"system_error","message":"No process is on the other end of the pipe.","error-code":233,"error-category":"system","error-message":"No process is on the other end of the pipe."},"source":"CallbackConnectionCheck"}}