Add support for MySQL compressed protocol

[vlasky]

Can we please add support for the MySQL compressed protocol.

Perhaps the compressed protocol code from node-mysql2 can be used to help implement this support in mysqljs:mysql.

Support for the compressed protocol greatly reduces bandwidth usage when connecting over a network to a remote MySQL server.

I have a need for this because my webapp relies on the zongji binlog listener package which is tightly integrated with mysqljs and would require a massive rewrite to use node-mysql2.

[dougwilson]

Hi, this is a great idea. I believe this is the first request for this feature in all the time that this module existed (at least I could not find any previous issue requesting it). You're always welcome to submit a pull request with an implementation of the feature 🎉

If you're not sure how to get started, there is a contributing section https://github.com/mysqljs/mysql/blob/master/Readme.md#contributing with an overview.

[sidorares]

happy to assist with implementation @vlasky ( also happy to try to add missing binlog features so that you can use mysql2 without zongji )

[dougwilson]

That'd be sweet. I was also looking over the protocol docs, as I figured it was going to be similar to SSL. I noticed in mysql2 that it just tries to compress everything outbound, is that right? I couldn't tell if it tried to coalesce multiple packets together and/or implemented a min size threshold before trying to deflate.

[sidorares]

It does not tries to batch packets together, but if compressed packet is bigger than uncompressed then compression for that packet is not used. I think in real life scenarios for outgoing packets ( typically command packets / short sql queries ) it does not make sense to complicate code with delaying / batching for very little extra compression efficiency

https://github.com/sidorares/node-mysql2/blob/72bba61802cb30b62ea5a93b3699bfcfd51b0699/lib/compressed_protocol.js#L84

[dougwilson]

Thanks @sidorares . I got a basic implementation working. Gotta get all the tests added and some things ironed out.

[dougwilson]

I have an early WIP on the compression branch. You can enable compression, but it doesn't currently compress the data from client to server, only opting into the compression protocol and decompressing the packets sent from the server to the client.

You can install using npm install mysqljs/mysql#compression and to enable compression add compress: true to the connection options object.

The implementation is trying to fit in the compression layer without major changes in the parser/protocol/packetwriter classes, as there are some modules out there currently reaching in and using these internal classes. This will allow landing this as a minor version change, and cleaning up the interface can be deferred to work towards the next major, when that's necessary.

[Bnaya]

@dougwilson your branch (after rebase) works great for my use case.
What are the needed steps to make it happen? i will happily help with that.

[vlasky]

Hi @dougwilson, what hurdles are preventing the code from the compression branch from being merged into the master branch?