Modify request body field

[matiasinsaurralde]

Solves #1724 however the Python bindings behave differently, if you had a middleware with the following code:

from tyk.decorators import *
from gateway import TykGateway as tyk

@Hook
def TestHook(request, session, metadata, spec):
  if "mystring" in request.object.body:
    print("found mystring")
  return request, session, metadata

After this change it would throw this error:

 Can't dispatch, error: <class 'TypeError'> a bytes-like object is required, not 'str'

It would need this small change:

from tyk.decorators import *
from gateway import TykGateway as tyk

@Hook
def TestHook(request, session, metadata, spec):
  if "mystring" in str(request.object.body):
    print("found mystring")
  return request, session, metadata

The request.object.body field is generated automatically by the PB tool so I'm not sure if we should modify it?
Bindings for other languages could be affected too.

[buger]

Since it is backward incompatible, we need to find another solution :(

[buger]

@matiasinsaurralde I wonder if we can inject some manual code to bindings we generate, so we add some byte->string conversion?

[matiasinsaurralde]

@buger we could but we would need to do it everytime the bindings are re-generated and for the different languages? Also some people might be generating their own bindings

[buger]

What about plan on adding the second field, for compatibility?

[matiasinsaurralde]

I've refactored this and included a test that sends multipart data and a normal JSON request, the approach is as follows:

• When building the object we check if the data is valid UTF8 (using the unicode package), if it's valid we store a stringified version in object.body. Previously this was the standard behavior for all scenarios and the reason of the panics we had.
• If the data isn't valid UTF8, we store it as bytes using object.raw_body (this is a new field so that we don't break existing code).

[buger]

Nice! I would also suggest that probably raw_body should be populated all the time. So if you are writing a plugin you do not have to think about this logic of switching different fields.

[matiasinsaurralde]

@buger Cool, will tweak it!

[matiasinsaurralde]

Updated, raw_body is always populated now

[matiasinsaurralde]

Added gRPC tests and implemented a helper for dispatcher errors

[buger]

Now it looks really nice 👌