Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(flux): Reduce influxd and influx startup time if Flux unused #16709

Merged
merged 2 commits into from
Mar 2, 2020
Merged

fix(flux): Reduce influxd and influx startup time if Flux unused #16709

merged 2 commits into from
Mar 2, 2020

Conversation

stuartcarnie
Copy link
Contributor

@stuartcarnie stuartcarnie commented Feb 3, 2020 •
edited
Loading

This PR lazily initializes Flux built-in functions when Flux is used.

It significantly reduces the startup time of the influxd and influx binaries when Flux is disabled / unused.

Before (4.66s):

↳ time bin/18/influx
bin/18/influx  4.66s user 0.19s system 198% cpu 2.441 total

After (10ms):

↳ time bin/18/influx
bin/18/influx  0.01s user 0.01s system 88% cpu 0.021 total

Closes #

Describe your proposed changes here.

  • CHANGELOG.md updated with a link to the PR (not the Issue)

@stuartcarnie
fix(flux): Flux startup costs should only be paid if Flux is used
5b58a81 
This PR lazily initializes Flux built-in functions when Flux is used.

It significantly reduces the startup time of the `influxd` and `influx`
binaries.

Before (4.66s):

```
↳ time bin/18/influx
bin/18/influx  4.66s user 0.19s system 198% cpu 2.441 total
```

After (10ms):

```
↳ time bin/18/influx
bin/18/influx  0.01s user 0.01s system 88% cpu 0.021 total
```
@stuartcarnie stuartcarnie self-assigned this Feb 3, 2020
@stuartcarnie stuartcarnie changed the title fix(flux): Flux startup costs should only be paid if Flux is used fix(flux): Reduce influxd and influx startup time if Flux unused Feb 3, 2020
@stuartcarnie stuartcarnie requested review from ayang64 and removed request for dgnorton February 3, 2020 19:50
@russorat russorat added the 1.x label Feb 27, 2020
ayang64
ayang64 approved these changes Mar 2, 2020
View reviewed changes
flux/control/controller.go
@@ -17,6 +17,8 @@ type MetaClient = coordinator.MetaClient
type Authorizer = influxdb.Authorizer

func NewController(mc MetaClient, reader influxdb.Reader, auth Authorizer, authEnabled bool, logger *zap.Logger) *Controller {
builtin.Initialize()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is awesome!

@stuartcarnie stuartcarnie merged commit f71caf5 into 1.8 Mar 2, 2020
@stuartcarnie stuartcarnie deleted the sgc/18/flux-cost branch March 2, 2020 22:16
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request May 12, 2021
@jperkin
influxdb: Update to v1.8.4.
5b86395 
v1.8.4 [2021-01-27]
-------------------

### Bugfixes

-	[#19696](influxdata/influxdb#19697): fix(flux): add durations to Flux logging

v1.8.3 [2020-09-30]
-------------------

### Features

-	[#19187](influxdata/influxdb#19187): feat: Collect values written stats.
-	[#19611](influxdata/influxdb#19611): feat: Add -lponly flag to export sub-command.

### Bugfixes

-	[#19409](influxdata/influxdb#19409): chore: update uuid library from satori to gofrs.
-	[#19439](influxdata/influxdb#19439): fix(storage): ArrayFilterCursor truncation for multi-block data.
-	[#19460](influxdata/influxdb#19460): chore: Use latest version of influxql package.
-	[#19512](influxdata/influxdb#19512): chore: Quiet static analysis tools.
-	[#19592](influxdata/influxdb#19592): fix(services/storage): multi measurement queries return all applicable series.
-	[#19612](influxdata/influxdb#19612): fix: lock map before writes.

v1.8.2 [2020-08-13]
-------------------

### Bugfixes

-	[#19253](influxdata/influxdb#19253): fix(tsdb): Revert disable series id set cache size by default.

v1.8.1 [2020-07-08]
-------------------

### Features

-	[#18457](influxdata/influxdb#18457): feat(query): Parallelize field iterator planning.
-	[#18886](influxdata/influxdb#18886): feat(http): Allow user supplied HTTP headers.

### Bugfixes

-	[#17319](influxdata/influxdb#17319): fix(flux): buckets call no longer panics.
-	[#18212](influxdata/influxdb#18212): fix(tsdb): Defer closing of underlying SeriesIDSetIterators.
-	[#18286](influxdata/influxdb#18286): fix(tsdb): Disable series id set cache size by default.
-	[#18299](influxdata/influxdb#18299): refactor(http): Simplify Authorizer.
-	[#18694](influxdata/influxdb#18694): fix(tsi1): wait deleting epoch before dropping shard.
-	[#18687](influxdata/influxdb#18687): perf(tsi1): batch write tombstone entries when dropping/deleting.
-	[#18826](influxdata/influxdb#18826): fix: gracefully handle errors when creating snapshots.
-	[#18849](influxdata/influxdb#18849): chore(build): remove all of the go1.12 references from build.

v1.8.0 [2020-04-11]
-------------------

### Features

-	[#15952](influxdata/influxdb#15952): Add influx_inspect verify-tombstone tool.
-	[#16542](influxdata/influxdb#16542): Add offline series compaction to influx_inspect buildtsi.
-	[#16599](influxdata/influxdb#16599): Make influx CLI support custom HTTP endpoint.
-	[#16908](influxdata/influxdb#16908): Add support for InfluxDB 2.0 write API.
-	[#17621](influxdata/influxdb#17621): Update Flux to v0.65.0.
-	[#17188](influxdata/influxdb#17188): Enhance support for bound parameters.

### Bugfixes

-	[#10503](influxdata/influxdb#10503): Delete rebuilds series index when series to be deleted are only found in cache.
-	[#10504](https://github.com/influxdata/influxdb/issue/10504): Delete rebuilds series index when series to be deleted are outside timerange.
-	[#14485](influxdata/influxdb#14485): Parse Accept header correctly.
-	[#16524](influxdata/influxdb#16524): Upgrade compaction error log from `Info` to `Warn`.
-	[#16525](influxdata/influxdb#16525): Remove double increment of meta index.
-	[#16595](influxdata/influxdb#16595): Improve series cardinality limit for inmem index.
-	[#16606](influxdata/influxdb#16606): Ensure all block data returned.
-	[#16627](influxdata/influxdb#16627): Skip WriteSnapshot during backup if snapshotter is busy.
-	[#16709](influxdata/influxdb#16709): Reduce influxd and influx startup time if Flux isn't used.
-	[#16762](influxdata/influxdb#16762): Fix bugs in -compact-series-file.
-	[#16944](influxdata/influxdb#16944): Update to Go 1.13.8 and Go modules.
-	[#17032](influxdata/influxdb#17032): Fix a SIGSEGV when accessing tsi active log.
-	[#17656](influxdata/influxdb#17656): Verify precision in write requests.
-	[#17698](influxdata/influxdb#17698): Enable configuration of TLS 1.3.
jperkin added a commit to TritonDataCenter/pkgsrc that referenced this pull request May 12, 2021
@jperkin
influxdb: Update to v1.8.4.
4f5507f 
v1.8.4 [2021-01-27]
-------------------

### Bugfixes

-	[#19696](influxdata/influxdb#19697): fix(flux): add durations to Flux logging

v1.8.3 [2020-09-30]
-------------------

### Features

-	[#19187](influxdata/influxdb#19187): feat: Collect values written stats.
-	[#19611](influxdata/influxdb#19611): feat: Add -lponly flag to export sub-command.

### Bugfixes

-	[#19409](influxdata/influxdb#19409): chore: update uuid library from satori to gofrs.
-	[#19439](influxdata/influxdb#19439): fix(storage): ArrayFilterCursor truncation for multi-block data.
-	[#19460](influxdata/influxdb#19460): chore: Use latest version of influxql package.
-	[#19512](influxdata/influxdb#19512): chore: Quiet static analysis tools.
-	[#19592](influxdata/influxdb#19592): fix(services/storage): multi measurement queries return all applicable series.
-	[#19612](influxdata/influxdb#19612): fix: lock map before writes.

v1.8.2 [2020-08-13]
-------------------

### Bugfixes

-	[#19253](influxdata/influxdb#19253): fix(tsdb): Revert disable series id set cache size by default.

v1.8.1 [2020-07-08]
-------------------

### Features

-	[#18457](influxdata/influxdb#18457): feat(query): Parallelize field iterator planning.
-	[#18886](influxdata/influxdb#18886): feat(http): Allow user supplied HTTP headers.

### Bugfixes

-	[#17319](influxdata/influxdb#17319): fix(flux): buckets call no longer panics.
-	[#18212](influxdata/influxdb#18212): fix(tsdb): Defer closing of underlying SeriesIDSetIterators.
-	[#18286](influxdata/influxdb#18286): fix(tsdb): Disable series id set cache size by default.
-	[#18299](influxdata/influxdb#18299): refactor(http): Simplify Authorizer.
-	[#18694](influxdata/influxdb#18694): fix(tsi1): wait deleting epoch before dropping shard.
-	[#18687](influxdata/influxdb#18687): perf(tsi1): batch write tombstone entries when dropping/deleting.
-	[#18826](influxdata/influxdb#18826): fix: gracefully handle errors when creating snapshots.
-	[#18849](influxdata/influxdb#18849): chore(build): remove all of the go1.12 references from build.

v1.8.0 [2020-04-11]
-------------------

### Features

-	[#15952](influxdata/influxdb#15952): Add influx_inspect verify-tombstone tool.
-	[#16542](influxdata/influxdb#16542): Add offline series compaction to influx_inspect buildtsi.
-	[#16599](influxdata/influxdb#16599): Make influx CLI support custom HTTP endpoint.
-	[#16908](influxdata/influxdb#16908): Add support for InfluxDB 2.0 write API.
-	[#17621](influxdata/influxdb#17621): Update Flux to v0.65.0.
-	[#17188](influxdata/influxdb#17188): Enhance support for bound parameters.

### Bugfixes

-	[#10503](influxdata/influxdb#10503): Delete rebuilds series index when series to be deleted are only found in cache.
-	[#10504](https://github.com/influxdata/influxdb/issue/10504): Delete rebuilds series index when series to be deleted are outside timerange.
-	[#14485](influxdata/influxdb#14485): Parse Accept header correctly.
-	[#16524](influxdata/influxdb#16524): Upgrade compaction error log from `Info` to `Warn`.
-	[#16525](influxdata/influxdb#16525): Remove double increment of meta index.
-	[#16595](influxdata/influxdb#16595): Improve series cardinality limit for inmem index.
-	[#16606](influxdata/influxdb#16606): Ensure all block data returned.
-	[#16627](influxdata/influxdb#16627): Skip WriteSnapshot during backup if snapshotter is busy.
-	[#16709](influxdata/influxdb#16709): Reduce influxd and influx startup time if Flux isn't used.
-	[#16762](influxdata/influxdb#16762): Fix bugs in -compact-series-file.
-	[#16944](influxdata/influxdb#16944): Update to Go 1.13.8 and Go modules.
-	[#17032](influxdata/influxdb#17032): Fix a SIGSEGV when accessing tsi active log.
-	[#17656](influxdata/influxdb#17656): Verify precision in write requests.
-	[#17698](influxdata/influxdb#17698): Enable configuration of TLS 1.3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ayang64 ayang64 ayang64 approved these changes

@bthesorceror bthesorceror bthesorceror approved these changes

Assignees

@stuartcarnie stuartcarnie

Labels
1.x
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@stuartcarnie @bthesorceror @ayang64 @russorat