Performance Comparison Tests #91
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: rscales <robscales@icloud.com>
Add initial framework for odbc dll
- Add ARROW_FLIGHT_SQL_ODBC option. If we set `ARROW_FLIGHT_SQL_ODBC=ON`, the flightsql odbc folder will be built
- Add odbc api layer for entry_point.cc
- builds odbc dll file, with ODBC APIs exported in odbc.def
Address James' comments
Fix `odbcabstraction` build errors and partially fix `flightsql_odbc` errors
Fix boost-variant not found error
- Adding dependencies from odbc/vcpkg.json to cpp/vcpkg.json
- Fix whereami.cc and .h dependency; ported lates code
Update whereami.cc
- use `long` instead of `int64`. Fixed namespace issues.
- PR CI fix: Add `parquet-testing` back
Partial build fix for `flight_sql` folder
- Replaced `namespace arrow` and `namespace odbcabstraction` with `using namespace ...`
- fix flight_sql_connection.cc
Fix `util::nullopt` to use `std::nullopt`
- fix std::optional
- fix BufferReader
- Fix GetSchema
- fix json_converter.cc
- partial fix configuration.h
- partial fix get_info_cache.cc
- Fix winsock build error
- Comment out `flight_sql` files that cannot build
- Comment out configuration and unit tests
- Comment out get info cache and system trust store
Create initial odbc tests folder
Implement SQLAllocEnv
Fix cmake build
Implement SQLFreeEnv
Fix rest of build errors from `flightsql-odbc`
- Fix get info errors
- Fix for configuration window
- added odbcinst library
- Fix system trust store
- unit test fixes
- Add dependency of ARROW_COMPUTE. `arrow/compute/api.h` is used in
`flight_sql`. Adding `ARROW_COMPUTE=ON` during build fixed run time unit
tests failures.
Implement SQLAllocConnect and SQLFreeConnect
Fix build issue from static flight sql driver
Lint and code style fixes
Re-add deleted submodule parquet-testing
clang-format lint fix
cpplint lint fix
Exclude whereami in rat exclude list
C++/CLI lint fix
Update parquet-testing to match commit from `main`
Address Kou's comments
ODBC directory lint fixes
Catching the lint fixes outside of `flightsql-odbc` code
Fix build warnings that get treated as error
Implement SQLSetEnvAttr and SQLGetEnvAttr
Implement use of ExecuteWithDiagnostics
Doxygen Error Fixes and Address comments from Kou and James
Address comments from Kou
- Updates License.txt
- Update cmake toolchain
- Move whereami to `vendored`
- Use string_view instead of NOLINT std::string
Remove `whereami.cc` from arrow util build
We are building whereami.cc as part of odbc
Fix include headers to replace <> with ""
Address comments from James
Implement SQLGetDiagField
Implement stubs for SQLGetInfo, SQLGetDiagField and SQLGetDiagRec Separate RegisterDsn and UnregisterDsn from windows build Update code to save driver value from connection string Add ReadMes for ODBC and tests Fix test issues with string_view Address code reviews Update entry_points.cc to fix build issue Remove Dremio references Use emplace properly Address comment from Rob and add SQLDisconnect test case
Accidentally committed the change during git rebase
Tests are skipped for now.
Add todo to update logging system later Add logs
* Add todo for noauth validation * mock server with token auth Add tests * run same test with both modes * Enable ODBC tests in workflow * Switch current test cases to use FlightSQLODBCTestBase So the tests can be skipped when `TEST_CONNECT_STR` is not set. * Change tests to run on both mock and remote modes Wrap usage of TEST_CONNECT_STR where possible * Rename test fixtures and make connection string functions virtual * Fix lint issue * Attempt to enable ODBC build on Windows platforms * Attempt to fix clang64 and MinGW errors * Attempt to register ODBC * Address James' comments Use constant string for token * use ServerMiddleware to validate token
PopulateCallOptions before making a connection Fix dsn window bug with advance properties Fix seg fault issue from empty string
* Let compiler append `W` to ODBC APIs where applicable.
As per changes from apache#46261, we need to initialize Kernel library explicitly to get the functions registered
* Move `connection_test.cc` to last in `CMakeLists.txt`, this resolves the seg fault. The seg fault was not reproducible on my local environment, even when I use the msys2 shell. * Rename test executable to flight_sql_odbc_test
* SQLGetStmtAttr stub implementation * Fix missing break statement in SQLGetDiagRec * Run ShutdownProtobufLibrary as part of test environment * Add comment for apacheGH-46889 * Pass call options to executed prepared statement `call_options_` contains the authentication token which is needed
SQLGetStmtAttr stub implementation Stub call for SQLGetStmtAttr Enable statement handle allocation Implement SQLExecDirect - test hang issue Update odbc_api.cc Run `ShutdownProtobufLibrary` after all ODBC tests Resolves test hanging problem Fix missing break statement in SQLGetDiagRec Add tests Lint fixes Run ShutdownProtobufLibrary as part of test environment Add debug messages Draft code Remove drafts Add comment Add comment for apacheGH-46889 Update statement test headers Pass call options to executed prepared statement `call_options_` contains the authentication token which is needed SQLFetch & SQLGetData initial impl EVERYTHING before this commit is for SQLExecDirect SQLFetch and SQLGetData TestSQLExecDirectSimpleQuery test Make GetData() return SQLRETURN `TestSQLExecDirectDataQuery` for remote and mock servers Remove unneeded test case Add more data types * Add SQL_GUID in getCTypeForSQLType * Add data types in query * Add checks for columns 1 to 25 * Update statement_test.cc Add varbinary test * Implement SQLNumResultCols Will leave tests for later * Implement SQLRowCount * Implement SQLMoreResults * Work on date retrieval fix The `TestSQLExecDirectDataQuery` checks for date is not working. * Fix date retrieval error by importing fix from Paul * imported fix from dremio/flightsql-odbc@d44d862, an Apache-Licensed repository * Add more checks for get data * Add more tests * Continue work on float truncation test * Disable float truncation test * Address comments from James - do not return errors for invalid rowCountPtr and columnCountPtr - add static cast for columnCount - Add tests with SQL_C_DEFAULT as target type - Add test checks for indicator - Fix SQL_NUMERIC, SQL_BIT, SQL_FLOAT target type * Fix build issues in CI * More fix for CI * Add test for invalid buffer length
* Add tests * Address comments from Rob
* Move ifdefs outside of test cases * use `stmt` * Move logs to first line * fix test for `SQL_DRIVER_HSTMT` * Use camel case for tests
* SQL_UNBIND implementation + tests * Add tests for `SQL_ATTR_ROW_ARRAY_SIZE` * Add check for `SQL_ATTR_ROWS_FETCHED_PTR`
* Descriptor allocation initial impl * Add descriptor handle tests * Add diagnostic error test for descriptor handle * the error is from driver manager as our driver doesn't have descriptor-specific APIs implemented yet * Add SQLGetDescField test from driver manager * Add doxygen doc comment
* revert back to passing empty string in handshake
* Bug: system trust store has default value of "true" regardless of "encryption" value. * But if encryption is set to false, system trust store cannot be used, so system trust store should be saved as false if user does not enable encryption. * It may confuse the user if they see encryption is set to false but system trust store is set to true. In this case, the driver will not do system trust store verification. The DSN window should reflect this accurately.
Add ARROW_FLIGHT_SQL_ODBC_INSTALLER environment variable to enable creation of Windows installer. * Use CPack + WiX to create a `msi` installer. * run command `cpack` under the build directory to generate the installer.
* In `wxs`, cannot use `package`, need to use `fragment` instead * Use component as feature will automatically be generated * Set Patch version + add installer instructions
* replaces default banner with Arrow logo banner
* use versioninfo.rc.in template * use `1 VERSIONINFO` for it to work properly * Set version variables * Add `.rc` to gitignore * Add `@` variables to rc template
Since the connection string says Driver= and not DSN=, it makes sense to change the command line to driver to indicate that we are passing the driver name value, not the DSN value
rscales
force-pushed
the
performance-baseline-test
branch
from
0e286b0 to
7dceaa8
Compare
alinaliBQ
force-pushed
the
apache-odbc
branch
from
380e405 to
8041b2b
Compare
alinaliBQ
force-pushed
the
apache-odbc
branch
6 times, most recently
from
c383805 to
bdf2fdb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Initial version of table read test
See following for usage: