Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: services/httpd: parse correctly Accept header with extra test cases #14485

Merged
merged 1 commit into from
Mar 12, 2020
Merged

Conversation

jsign
Copy link

@jsign jsign commented Jul 26, 2019

Closes #14296

Until the standard lib decides to include a standard function to correctly parse Accept headers, we can rely on goautoneg library which is a Github port used by Prometheus.

I also added sub-tests as Table Tests for multiples scenarios around this problem.

Adding a new dependency for this amount of code may hurt a bit, so maybe we could add the code directly too.

  • CHANGELOG.md updated with a link to the PR (not the Issue)
  • Rebased/mergeable
  • Tests pass
  • Signed CLA (if not already signed)

@jsign
Copy link
Author

jsign commented Jul 26, 2019

The failed test case seems to be a reported flaky test

@jsign jsign changed the title services/httpd: parse correctly Accept headers & add test cases fix: services/httpd: parse correctly Accept headers & add test cases Jul 26, 2019
@jsign jsign changed the title fix: services/httpd: parse correctly Accept headers & add test cases fix: services/httpd: parse correctly Accept header with extra test cases Jul 26, 2019
@jsign
Copy link
Author

jsign commented Aug 31, 2019

@russorat , I rebased it to 1.8 to keep it clean.
Is 1.8 OK for merging or do you prefer on master?

@jsternberg
Copy link
Contributor

Can you copy the code from the dependency into the code as an unexported function? Make a reference that the code is based on that person's original library and a link to the license as ours would become a derivative work so we would need to give credit as part of the license.

There's things I want to change in both your code and the dependency code to ensure this is done correctly and won't require fixes in the future and it is easier to do that if we're not pulling in a dependency we don't control for something so small.

@jsign
Copy link
Author

jsign commented Nov 10, 2019

@jsternberg, pushed changes in the direction you mentioned.

services/httpd/response_writer.go Outdated Show resolved Hide resolved
services/httpd/accept.go Outdated Show resolved Hide resolved
services/httpd/response_writer.go Outdated Show resolved Hide resolved
services/httpd/response_writer_test.go Show resolved Hide resolved
@jsign
Copy link
Author

jsign commented Nov 16, 2019

@jsternberg, I applied your suggestions, and also switched to performance mode since the code had many allocations that could be avoided. And considering this is a hot-path, deserved some extra attention.

Some points about the changes (apart from your comments):

  • make csvFormatter stateless
  • all known content types and formatters are defined as vars, so to a ton of allocations. This is the reason why I made csvFormatter stateless (since it wasn't).
  • avoided string concats to save extra allocations (that's why full exist apart from splited parts)
  • added * test case
  • in case of */* the switch isn't biased to a definite option. This is strictly correct (since any option complies to request), but maybe another default is wanted.
  • as can be seen, adding future content types is much less verbose
  • rebased again to 1.8

@jsternberg
Copy link
Contributor

Sorry I haven't had time to review this again. I will review by end of day tomorrow.

@dgnorton
Copy link
Contributor

dgnorton commented Mar 6, 2020

@jsternberg can you review this for the 1.8 release coming up?

@jsternberg
Copy link
Contributor

I reverted the changes to the formatters and modified the creation of those formatters to a factory method. I think that the state is actually used by the code so a global instance doesn't work, but I also don't remember well enough to know for certain so I don't want those to change.

I am going to wait for the tests and run some myself and then merge this.

@jsternberg jsternberg merged commit 409de34 into influxdata:1.8 Mar 12, 2020
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request May 12, 2021
v1.8.4 [2021-01-27]
-------------------

### Bugfixes

-	[#19696](influxdata/influxdb#19697): fix(flux): add durations to Flux logging

v1.8.3 [2020-09-30]
-------------------

### Features

-	[#19187](influxdata/influxdb#19187): feat: Collect values written stats.
-	[#19611](influxdata/influxdb#19611): feat: Add -lponly flag to export sub-command.

### Bugfixes

-	[#19409](influxdata/influxdb#19409): chore: update uuid library from satori to gofrs.
-	[#19439](influxdata/influxdb#19439): fix(storage): ArrayFilterCursor truncation for multi-block data.
-	[#19460](influxdata/influxdb#19460): chore: Use latest version of influxql package.
-	[#19512](influxdata/influxdb#19512): chore: Quiet static analysis tools.
-	[#19592](influxdata/influxdb#19592): fix(services/storage): multi measurement queries return all applicable series.
-	[#19612](influxdata/influxdb#19612): fix: lock map before writes.

v1.8.2 [2020-08-13]
-------------------

### Bugfixes

-	[#19253](influxdata/influxdb#19253): fix(tsdb): Revert disable series id set cache size by default.

v1.8.1 [2020-07-08]
-------------------

### Features

-	[#18457](influxdata/influxdb#18457): feat(query): Parallelize field iterator planning.
-	[#18886](influxdata/influxdb#18886): feat(http): Allow user supplied HTTP headers.

### Bugfixes

-	[#17319](influxdata/influxdb#17319): fix(flux): buckets call no longer panics.
-	[#18212](influxdata/influxdb#18212): fix(tsdb): Defer closing of underlying SeriesIDSetIterators.
-	[#18286](influxdata/influxdb#18286): fix(tsdb): Disable series id set cache size by default.
-	[#18299](influxdata/influxdb#18299): refactor(http): Simplify Authorizer.
-	[#18694](influxdata/influxdb#18694): fix(tsi1): wait deleting epoch before dropping shard.
-	[#18687](influxdata/influxdb#18687): perf(tsi1): batch write tombstone entries when dropping/deleting.
-	[#18826](influxdata/influxdb#18826): fix: gracefully handle errors when creating snapshots.
-	[#18849](influxdata/influxdb#18849): chore(build): remove all of the go1.12 references from build.

v1.8.0 [2020-04-11]
-------------------

### Features

-	[#15952](influxdata/influxdb#15952): Add influx_inspect verify-tombstone tool.
-	[#16542](influxdata/influxdb#16542): Add offline series compaction to influx_inspect buildtsi.
-	[#16599](influxdata/influxdb#16599): Make influx CLI support custom HTTP endpoint.
-	[#16908](influxdata/influxdb#16908): Add support for InfluxDB 2.0 write API.
-	[#17621](influxdata/influxdb#17621): Update Flux to v0.65.0.
-	[#17188](influxdata/influxdb#17188): Enhance support for bound parameters.

### Bugfixes

-	[#10503](influxdata/influxdb#10503): Delete rebuilds series index when series to be deleted are only found in cache.
-	[#10504](https://github.com/influxdata/influxdb/issue/10504): Delete rebuilds series index when series to be deleted are outside timerange.
-	[#14485](influxdata/influxdb#14485): Parse Accept header correctly.
-	[#16524](influxdata/influxdb#16524): Upgrade compaction error log from `Info` to `Warn`.
-	[#16525](influxdata/influxdb#16525): Remove double increment of meta index.
-	[#16595](influxdata/influxdb#16595): Improve series cardinality limit for inmem index.
-	[#16606](influxdata/influxdb#16606): Ensure all block data returned.
-	[#16627](influxdata/influxdb#16627): Skip WriteSnapshot during backup if snapshotter is busy.
-	[#16709](influxdata/influxdb#16709): Reduce influxd and influx startup time if Flux isn't used.
-	[#16762](influxdata/influxdb#16762): Fix bugs in -compact-series-file.
-	[#16944](influxdata/influxdb#16944): Update to Go 1.13.8 and Go modules.
-	[#17032](influxdata/influxdb#17032): Fix a SIGSEGV when accessing tsi active log.
-	[#17656](influxdata/influxdb#17656): Verify precision in write requests.
-	[#17698](influxdata/influxdb#17698): Enable configuration of TLS 1.3.
jperkin added a commit to TritonDataCenter/pkgsrc that referenced this pull request May 12, 2021
v1.8.4 [2021-01-27]
-------------------

### Bugfixes

-	[#19696](influxdata/influxdb#19697): fix(flux): add durations to Flux logging

v1.8.3 [2020-09-30]
-------------------

### Features

-	[#19187](influxdata/influxdb#19187): feat: Collect values written stats.
-	[#19611](influxdata/influxdb#19611): feat: Add -lponly flag to export sub-command.

### Bugfixes

-	[#19409](influxdata/influxdb#19409): chore: update uuid library from satori to gofrs.
-	[#19439](influxdata/influxdb#19439): fix(storage): ArrayFilterCursor truncation for multi-block data.
-	[#19460](influxdata/influxdb#19460): chore: Use latest version of influxql package.
-	[#19512](influxdata/influxdb#19512): chore: Quiet static analysis tools.
-	[#19592](influxdata/influxdb#19592): fix(services/storage): multi measurement queries return all applicable series.
-	[#19612](influxdata/influxdb#19612): fix: lock map before writes.

v1.8.2 [2020-08-13]
-------------------

### Bugfixes

-	[#19253](influxdata/influxdb#19253): fix(tsdb): Revert disable series id set cache size by default.

v1.8.1 [2020-07-08]
-------------------

### Features

-	[#18457](influxdata/influxdb#18457): feat(query): Parallelize field iterator planning.
-	[#18886](influxdata/influxdb#18886): feat(http): Allow user supplied HTTP headers.

### Bugfixes

-	[#17319](influxdata/influxdb#17319): fix(flux): buckets call no longer panics.
-	[#18212](influxdata/influxdb#18212): fix(tsdb): Defer closing of underlying SeriesIDSetIterators.
-	[#18286](influxdata/influxdb#18286): fix(tsdb): Disable series id set cache size by default.
-	[#18299](influxdata/influxdb#18299): refactor(http): Simplify Authorizer.
-	[#18694](influxdata/influxdb#18694): fix(tsi1): wait deleting epoch before dropping shard.
-	[#18687](influxdata/influxdb#18687): perf(tsi1): batch write tombstone entries when dropping/deleting.
-	[#18826](influxdata/influxdb#18826): fix: gracefully handle errors when creating snapshots.
-	[#18849](influxdata/influxdb#18849): chore(build): remove all of the go1.12 references from build.

v1.8.0 [2020-04-11]
-------------------

### Features

-	[#15952](influxdata/influxdb#15952): Add influx_inspect verify-tombstone tool.
-	[#16542](influxdata/influxdb#16542): Add offline series compaction to influx_inspect buildtsi.
-	[#16599](influxdata/influxdb#16599): Make influx CLI support custom HTTP endpoint.
-	[#16908](influxdata/influxdb#16908): Add support for InfluxDB 2.0 write API.
-	[#17621](influxdata/influxdb#17621): Update Flux to v0.65.0.
-	[#17188](influxdata/influxdb#17188): Enhance support for bound parameters.

### Bugfixes

-	[#10503](influxdata/influxdb#10503): Delete rebuilds series index when series to be deleted are only found in cache.
-	[#10504](https://github.com/influxdata/influxdb/issue/10504): Delete rebuilds series index when series to be deleted are outside timerange.
-	[#14485](influxdata/influxdb#14485): Parse Accept header correctly.
-	[#16524](influxdata/influxdb#16524): Upgrade compaction error log from `Info` to `Warn`.
-	[#16525](influxdata/influxdb#16525): Remove double increment of meta index.
-	[#16595](influxdata/influxdb#16595): Improve series cardinality limit for inmem index.
-	[#16606](influxdata/influxdb#16606): Ensure all block data returned.
-	[#16627](influxdata/influxdb#16627): Skip WriteSnapshot during backup if snapshotter is busy.
-	[#16709](influxdata/influxdb#16709): Reduce influxd and influx startup time if Flux isn't used.
-	[#16762](influxdata/influxdb#16762): Fix bugs in -compact-series-file.
-	[#16944](influxdata/influxdb#16944): Update to Go 1.13.8 and Go modules.
-	[#17032](influxdata/influxdb#17032): Fix a SIGSEGV when accessing tsi active log.
-	[#17656](influxdata/influxdb#17656): Verify precision in write requests.
-	[#17698](influxdata/influxdb#17698): Enable configuration of TLS 1.3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants