Skip to content

Implement a common testing framework for Transport UDFs #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 14, 2018

Conversation

shardulm94
Copy link
Contributor

  • Add transportable-udfs-test-spi containg the SPI to be implemented by the testers. Also contains an extended SPI for SQL-specific testers
  • Add transportable-udfs-test-api which contains the user facing API for the test framework
  • Add Hive, Presto, Spark and generic implementations for the transportable-udfs-test-spi. For every platform
    • Add a TestStdUDFWrapper which takes in classes enclosed by the wrapper as constructor arguments
    • Add an implementation of StdTester for that platform
    • For SQL-based testers add a SqlFunctionCallGenerator to generate a SQL string from user input arguments
    • For SQL-based testers add a ToPlatformTestOutputConverter to convert user-provided expected output data to output received from the platform's SQL query execution
  • For the generic tester platform, implement StdData, StdFactory and all classed from transportable-udfs-typesystem
  • Add support for unknown data type in Hive, Presto and Spark

@wmoustafa
Copy link
Contributor

Thanks @shardulm94 for this feature!

This patch has been reviewed internally at LinkedIn since the feature started before open sourcing the Transport framework. I will go ahead and merge.

@wmoustafa wmoustafa merged commit 6514827 into linkedin:master Dec 14, 2018
@shardulm94 shardulm94 deleted the testframework branch March 23, 2019 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants