[ Question ] How do you manipulate proxy response

[iroy2000]

Hi, I'm using Express 4,

In normal way, I can do the following

    app.get('/v1/users/:username', function(request, response, next) {
        var username = request.params.username;
        findUserByUsername(username, function(error, user) {
            if (error) return next(error);
            return response.render('user', user);
        });
    });

But how do I execute custom logic if I'm using proxy, let's say I want to manipulate some of the data before response to the user? Is there a good pattern to do that with this middleware ?

app.use('/api', proxy({target: 'http://www.example.org', changeOrigin: true}));

[chimurai]

I don't have the answer unfortunately.

You are not the first one the ask this question. (#91 (comment))

Happy to add this one to the http-proxy-middleware recipes.

Maybe this example might help you:
https://github.com/nodejitsu/node-http-proxy/blob/master/examples/middleware/modifyResponse-middleware.js

Can you post this question on StackOverflow (with backlink)? Hopefully your question will get an answer there.

Thanks!

---

Update 2021-04-18:

responseInterceptor in now available in http-proxy-middleware@1.2.0

• README: https://github.com/chimurai/http-proxy-middleware#intercept-and-manipulate-responses
• recipes: https://github.com/chimurai/http-proxy-middleware/blob/master/recipes/response-interceptor.md#readme
• code example: https://github.com/chimurai/http-proxy-middleware/blob/master/examples/response-interceptor/index.js

[iroy2000]

I tried to use https://github.com/langjt/node-http-proxy-json

And it seems to solve my problem. Note that the library is still fairly new, but it's definitely take out some pain points from many people that has the same issues.

Let me just quote my code here for people who is interested, Hope it helps :)

@chimurai You can even add that to your recipe if you think it is good enough.

import express from 'express';
import proxy from 'http-proxy-middleware';
import config from 'config';
import modifyResponse from 'node-http-proxy-json';

const router = new express.Router();

const myProxy = proxy(config.get('some.json.api.Url'), {
  changeOrigin: true,
  logLevel: 'debug',
  pathRewrite: (path, req) => path.replace('/api', '/_ajax'),
  onProxyRes(proxyRes, req, res) {

    delete proxyRes.headers['content-length'];

    modifyResponse(res, proxyRes.headers['content-encoding'], function (body) {
      if (body) {
        // modify some information
        body.version = 2;
        body.props = {
          "nestedProps": true
        };
      }
      return body;
    });
  },
});


router.use('/api/example', myProxy);

export default router;

[silverprize]

@iroy2000 Thanks for your approach. But I'd like to add custom header from body's property.
I tried and faced "Can't set headers after they are sent.".
I suggest to use multi proxy module.
express-http-proxy provide async proxy response.
express-http-proxy is inconvenient than http-proxy-middleware thus keep http-proxy-middleware as main proxy.
Few apis that need to modify header or body are called via express-http-proxy.

[cokeknight]

here is my answer,

onProxyRes :function(proxyRes, req, res){
      var _write = res.write;
      var output;
      var body = "";
      proxyRes.on('data', function(data) {
          data = data.toString('utf-8');
          body += data;
      });
      res.write = function (data) {
        try{
          eval("output="+body)
          output = mock.mock(output)
          _write.call(res,JSON.stringify(output));
        } catch (err) {}
      }
    }

add onProxyRes option on the http-proxy-middleware
use the data event on the proxyRes to get the output
then modify the output in res.write

[enl31]

@cokeknight thank you, can you help me more? How do you change header in this approach?
I want to have custom header containgn length of (json) body in response like this:

  res.write = data => {
    data = data.toString('utf-8');
    body += data;
    try{
      body = JSON.parse( body);
       res.set({
        'Content-Type': 'Application/json',
        'count': body.meta.count
      });
      _write.call(res,JSON.stringify(body.data));
    } catch (err) {}
  }

[zaaack]

Not working for async case, I need return some async data, but it just return nothing.

[benjamin658]

@enl31 You can set content-length by this way:

proxyRes.headers['content-length'] = data.length;

[westmark]

This was my final solution (inspired by @kokarn 's solution over at http-party/node-http-proxy#796

I'm using this in a webpack dev server setup by create-react-app, hence the comment further down.

onProxyRes: ( proxyRes, req, res ) => {
    const _writeHead = res.writeHead;
    let _writeHeadArgs;
    const _end = res.end;
    let body = '';

    proxyRes.on( 'data', ( data ) => {
      data = data.toString( 'utf-8' );
      body += data;
    } );

    // Defer writeHead
    res.writeHead = ( ...writeHeadArgs ) => {
      _writeHeadArgs = writeHeadArgs;
    };

    // Defer all writes
    res.write = () => {};

    res.end = ( ...endArgs ) => {
      // Do everything in the end. Means we wont be streaming the response but who cares in a dev env..
      const output = body
        .replace( /stufftoreplace/g, '/leweb' );

      if ( proxyRes.headers && proxyRes.headers[ 'content-length' ] ) {
        res.setHeader( 'content-length', output.length );
      }

      // This disables chunked encoding
      res.setHeader( 'transfer-encoding', '' );

      // Disable cache for all http as well
      res.setHeader( 'cache-control', 'no-cache' );

      _writeHead.apply( res, _writeHeadArgs );

      if ( body.length ) {
         // Write everything via end()
        _end.apply( res, [ output ] );
      } else {
        _end.apply( res, endArgs );
      }
    };
  }

[youngzhao-xyz]

If proxyRes is gzipped, I will need to unzip it before pass it to res. Is there a better way to work around this?

Here is my code

onProxyRes: (proxyRes, req, res) => {
    const end = res.end;
    const writeHead = res.writeHead;
    let writeHeadArgs;
    let body; 
    let buffer = new Buffer('');

    // Concat and unzip proxy response
    proxyRes
      .on('data', (chunk) => {
        buffer = Buffer.concat([buffer, chunk]);
      })
      .on('end', () => {
        body = zlib.gunzipSync(buffer).toString('utf8');
      });

    // Defer write and writeHead
    res.write = () => {};
    res.writeHead = (...args) => { writeHeadArgs = args; };

    // Update user response at the end
    res.end = () => {
      const output = manipulateBody(body); // some function to manipulate body
      res.setHeader('content-length', output.length);
      res.setHeader('content-encoding', '');
      writeHead.apply(res, writeHeadArgs);

      end.apply(res, [output]);
    };
  }

[nickgrealy]

I'm currently using @cokeknight 's solution (#97 (comment)), and I'm finding my JSON responses are being truncated after 8192 characters.

Is anyone else finding this?

N.B. It could be (probably is) because I'm not using the following code snippet... can someone explain what the intention is, and why you wouldn't just do _write.call(res, body);?

eval("output="+body)
output = mock.mock(output)
_write.call(res, JSON.stringify(output));

(e.g. assign body reference to variable output (why use eval?), then wrap it in a mock?)

[ortonomy]

Allright, I also used : https://github.com/langjt/node-http-proxy-json

and it solved my problem. I recommend it over the manual re-writing of res.write methods - it the same thing as the solutions above, but it's captured in a library. 😅