-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Closed
Description
What would you like to happen?
Summary
Add a filter() method to the PCollection class in the TypeScript SDK, consistent with the existing map() and flatMap() methods.
Motivation
The TypeScript SDK currently provides map() and flatMap() methods on PCollection, but lacks a filter() method. This is explicitly mentioned as a TODO in the README-dev.md:
"Also add a [do]Filter, and possibly a [do]Reduce?"
The Python SDK already provides beam.Filter() for this purpose. Adding filter() to the TypeScript SDK would:
- Improve developer experience - JavaScript developers expect
filter()alongsidemap() - Improve code readability -
pcoll.filter(x => x > 0)is cleaner thanpcoll.flatMap(x => x > 0 ? [x] : []) - Maintain consistency with the Python SDK which has
beam.Filter()
Proposed API
// Filter even numbers
const evens = pcoll.filter(x => x % 2 === 0);
// Filter with context (for side inputs, etc.)
const filtered = pcoll.filter((x, ctx) => x > ctx.threshold, { threshold: sideInput });
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components
- Component: Python SDK
- Component: Java SDK
- Component: Go SDK
- Component: Typescript SDK
- Component: IO connector
- Component: Beam YAML
- Component: Beam examples
- Component: Beam playground
- Component: Beam katas
- Component: Website
- Component: Infrastructure
- Component: Spark Runner
- Component: Flink Runner
- Component: Samza Runner
- Component: Twister2 Runner
- Component: Hazelcast Jet Runner
- Component: Google Cloud Dataflow Runner