Tags: gauravgcet/DataflowJavaSDK
Tags
Fixes a bug in custom unbounded readers Custom unbounded readers are read in bundles of at most 10k elements or 10 seconds. A recent change accidentally removed the 10k element limit. This change reintroduces it and adds a test. The previous test also was passing vacuously because the iteration limit was incorrect (it would always have only one iteration). ----Release Notes---- [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=112723469
Occasionally compact combined state on write. This is needed to avoid arbitrarily large blowup for many-element windows. Now, with a fixed probability (currently 0.2%) when writing the state, instead of doing a blind write we read in all accumulated values, combine, and replace everything with the final combined value. ----Release Notes---- [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=109239045
Version management Declare version 1.2.0. Prepare for version 1.2.1. ----Release Notes---- [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=104700491
Handle exceptions from within the SDK while processing work The DataflowWorkerHarness when processing work wasn't handling general exceptions from being thrown causing the processing threads to all die leaving a zombie worker. ----Release Notes---- [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=104664789
Updates dependency google-api-services-datastore-protobuf. ----Release Notes---- Updating dependency google-api-services-datastore-protobuf from v1beta2-rev1-2.1.2 to v1beta2-rev1-3.0.2. [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=103114238
Dataflow GA: update GitHub's README.md to remove Beta language ----Release Notes---- [] ------------- Created by MOE: http://code.google.com/p/moe-java MOE_MIGRATED_REVID=100331508
Move GroupByKey validation into validate method ----Release Notes---- Re-enabled verification of GroupByKey usage. Specififically, the key must have a deterministic coder and using GroupByKey with an Unbounded PCollection requires windowing or triggers. [] ------------- Created by MOE: http://code.google.com/p/moe-java MOE_MIGRATED_REVID=99194010
Change ExpectedLogs to include better stack-traces Also add support for verifying the absence of a message. ----Release Notes---- [] ------------- Created by MOE: http://code.google.com/p/moe-java MOE_MIGRATED_REVID=94922273
PreviousNext