UDF Examples for Spark SQL
This is a hive udf.
Hive UDF's can be used in Spark SQL, without modification.
It is important to note that for performance reasons you may choose to re-implement your hive udf's as spark udf's when migrating from hadoop to spark sql. This example is meant to provide a demonstration that the functionality is supported, not as a recommendation.
Not sure if all hive UDF's would work, but certainly this simple one does.
This takes a string and converts to lower case.
See the README.md for build information.
See the instructions in each sub-project
This is a hive udf.
Hive UDF's can be used in Spark SQL, without modification.
It is important to note that for performance reasons you may choose to re-implement your hive udf's as spark udf's when migrating from hadoop to spark sql. This example is meant to provide a demonstration that the functionality is supported, not as a recommendation.
Not sure if all hive UDF's would work, but certainly this simple one does.
This takes a string and reverses it.
See the README.md for build information.
See the instructions in each sub-project
These examples are used in our hadoop migration training course
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.