Skip to content

packets: implemented compression protocol #649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 37 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
e6c682c
packets: implemented compression protocol
Jul 31, 2017
77f6792
packets: implemented compression protocol CR changes
Aug 16, 2017
a0cf94b
packets: implemented compression protocol: remove bytes.Reset for bac…
Aug 16, 2017
4cdff28
Merge branch 'master' of https://github.com/go-sql-driver/mysql
Oct 8, 2017
d0ea1a4
reading working
Aug 18, 2017
477c9f8
writerly changes
Aug 18, 2017
996ed2d
PR 649: adding compression (second code review)
Oct 8, 2017
f74faed
do not query max_allowed_packet by default (#680)
julienschmidt Oct 12, 2017
b3a093e
packets: do not call function on nulled value (#678)
julienschmidt Oct 16, 2017
5eaa5ff
ColumnType interfaces (#667)
julienschmidt Oct 17, 2017
ee46028
Add Aurora errno to rejectReadOnly check (#634)
jeffcharles Oct 17, 2017
93aed73
allow successful TravisCI runs in forks (#639)
jmhodges Oct 17, 2017
4f10ee5
Drop support for Go 1.6 and lower (#696)
julienschmidt Nov 12, 2017
59b0f90
Make gofmt happy (#704)
julienschmidt Nov 14, 2017
3fbf53a
Added support for custom string types in ConvertValue. (#623)
dsmontoya Nov 15, 2017
f9c6a2c
Implement NamedValueChecker for mysqlConn (#690)
pushrax Nov 16, 2017
6046bf0
Fix Valuers by returning driver.ErrSkip if couldn't convert type inte…
randomjunk Nov 16, 2017
385673a
statement: Fix conversion of Valuer (#710)
linxGnu Nov 17, 2017
9031984
Fixed imports for appengine/cloudsql (#700)
rrbrussell Nov 17, 2017
6992fad
Fix tls=true didn't work with host without port (#718)
methane Dec 4, 2017
386f84b
Differentiate between BINARY and CHAR (#724)
kwoodhouse93 Jan 10, 2018
f853432
Test with latest Go patch versions (#693)
AlekSi Jan 10, 2018
d1a8b86
Fix prepared statement (#734)
methane Jan 13, 2018
3167920
driver.ErrBadConn when init packet read fails (#736)
Jan 25, 2018
fb33a2c
packets: implemented compression protocol
Jul 31, 2017
f174605
packets: implemented compression protocol CR changes
Aug 16, 2017
dbd1e2b
third code review changes
Mar 23, 2018
3e12e32
PR 649: minor cleanup
Mar 23, 2018
17a06f1
Merge branch 'master' into master
methane Mar 26, 2018
60bdaec
Sort AUTHORS
methane Mar 26, 2018
422ab6f
Update dsn.go
methane Mar 26, 2018
26ea544
cr4 changes
Jul 23, 2018
f339392
Merge branch 'master' into cr4
Jul 23, 2018
3e559a8
saving work with SimpleReader present
Sep 12, 2018
6ceaef6
removed buf from mysqlConn
Sep 26, 2018
97afd8d
Merge pull request #1 from bLamarche413/cr4
Oct 8, 2018
f617170
removed comment
Oct 8, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix prepared statement (#734)
* Fix prepared statement

When there are many args and maxAllowedPacket is not enough,
writeExecutePacket() attempted to use STMT_LONG_DATA even for
0byte string.
But writeCommandLongData() doesn't support 0byte data. So it
caused to send malfold packet.

This commit loosen threshold for using STMT_LONG_DATA.

* Change minimum size of LONG_DATA to 64byte

* Add test which reproduce issue 730

* TestPreparedManyCols test only numParams = 65535 case

* s/as possible//
  • Loading branch information
methane authored and Brigitte Lamarche committed Mar 23, 2018
commit d1a8b86f7fef0f773e55ca15defb2347be22a106
17 changes: 14 additions & 3 deletions driver_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1669,24 +1669,35 @@ func TestStmtMultiRows(t *testing.T) {
// Regression test for
// * more than 32 NULL parameters (issue 209)
// * more parameters than fit into the buffer (issue 201)
// * parameters * 64 > max_allowed_packet (issue 734)
func TestPreparedManyCols(t *testing.T) {
const numParams = defaultBufSize
numParams := 65535
runTests(t, dsn, func(dbt *DBTest) {
query := "SELECT ?" + strings.Repeat(",?", numParams-1)
stmt, err := dbt.db.Prepare(query)
if err != nil {
dbt.Fatal(err)
}
defer stmt.Close()

// create more parameters than fit into the buffer
// which will take nil-values
params := make([]interface{}, numParams)
rows, err := stmt.Query(params...)
if err != nil {
stmt.Close()
dbt.Fatal(err)
}
defer rows.Close()
rows.Close()

// Create 0byte string which we can't send via STMT_LONG_DATA.
for i := 0; i < numParams; i++ {
params[i] = ""
}
rows, err = stmt.Query(params...)
if err != nil {
dbt.Fatal(err)
}
rows.Close()
})
}

Expand Down
10 changes: 8 additions & 2 deletions packets.go
Original file line number Diff line number Diff line change
Expand Up @@ -927,6 +927,12 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error {
const minPktLen = 4 + 1 + 4 + 1 + 4
mc := stmt.mc

// Determine threshould dynamically to avoid packet size shortage.
longDataSize := mc.maxAllowedPacket / (stmt.paramCount + 1)
if longDataSize < 64 {
longDataSize = 64
}

// Reset packet-sequence
mc.sequence = 0
mc.compressionSequence = 0
Expand Down Expand Up @@ -1055,7 +1061,7 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error {
paramTypes[i+i] = byte(fieldTypeString)
paramTypes[i+i+1] = 0x00

if len(v) < mc.maxAllowedPacket-pos-len(paramValues)-(len(args)-(i+1))*64 {
if len(v) < longDataSize {
paramValues = appendLengthEncodedInteger(paramValues,
uint64(len(v)),
)
Expand All @@ -1077,7 +1083,7 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error {
paramTypes[i+i] = byte(fieldTypeString)
paramTypes[i+i+1] = 0x00

if len(v) < mc.maxAllowedPacket-pos-len(paramValues)-(len(args)-(i+1))*64 {
if len(v) < longDataSize {
paramValues = appendLengthEncodedInteger(paramValues,
uint64(len(v)),
)
Expand Down