Local Test before push

Hi, i'm new working with Dataflow, i was trying to parse a nginx log file with gcstobigquery, writing the parser on javascript UDF, i tought i'll be simple, but no, i got errors everywhere, and when i solved them, new errors come.

I just want to now if i could do a local test before the push, compile the TextIOToBigQuery locally changing the output, instead BigQuery just a local output of the json.

Thanks!

[sabhyankar]

Hi @rafalitox2 - Yes in fact the recommended workflow would be to develop and execute the pipeline locally (using the DirectRunner) after you validate the individual components (custom DoFns, PTransforms etc) via unit tests.

See Testing Beam pipelines