Skip to content

feat: bulk streamed processing#22

Draft
stevenh wants to merge 10 commits intomasterfrom
feat/bulk-stream-processing
Draft

feat: bulk streamed processing#22
stevenh wants to merge 10 commits intomasterfrom
feat/bulk-stream-processing

Conversation

@stevenh
Copy link
Collaborator

@stevenh stevenh commented Jan 22, 2022

Add streamed processing of bulk, eliminating the limits needed to prevent memory requirements as well as recreation of svrquery.Client for each processed element by implementing a reusable BulkClient.

This changes the processing to best effort and can result in entries without an address being output.

Also:

  • Output to a provided stream.
  • Eliminate the use of log.Fatal in the library.
  • Include element details when parsing fails.

Stephen Birch and others added 10 commits November 29, 2021 16:53
…dd-encryption-to-tf2e

� Conflicts:
�	cmd/cli/main.go
�	go.sum
Add streamed processing of bulk, eliminating the limits needed to
prevent memory requirements as well as recreation of svrquery.Client for
each processed element by implementing a reusable BulkClient.

This changes the processing to best effort and can result in entries
without an address being output.

Also:
* Output to a provided stream.
* Eliminate the use of log.Fatal in the library.
* Include element details when parsing fails.
@stevenh stevenh force-pushed the feat/bulk-stream-processing branch from de8d660 to 33ed3b7 Compare January 22, 2022 00:59
Base automatically changed from feat/add-encryption-to-tf2e to master January 24, 2022 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants