Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jj #3

Merged
merged 519 commits into from
May 23, 2020
Merged

jj #3

Changes from 1 commit
Commits
Show all changes
519 commits
Select commit Hold shift + click to select a range
e094af1
Move non-contextual state from Context to field
findepi Apr 27, 2020
2b02894
Fix incorrect visitor result type
findepi Apr 27, 2020
1ce21bc
Enforce DefaultTraversalVisitor Void result
findepi Apr 27, 2020
aa89cb6
Use Zulu JDK 11 to run product tests
electrum Apr 22, 2020
ea8f7bd
Remove fallback for Java 8
electrum Feb 11, 2020
a4ed120
Properly initialize HDFS configuration in RubixInitializer
losipiuk Apr 27, 2020
573dc38
Rename variable
losipiuk Apr 27, 2020
7d211ef
Expose JdbcClient invocation statistics in JMX
Praveen2112 Apr 19, 2020
dc78d33
Remove "Java 8" phrase from "stream API" comments
electrum Apr 28, 2020
44be687
Update server RPM for Java 11
electrum Apr 28, 2020
9432607
Remove documentation references to Java 8
electrum Apr 28, 2020
5bee687
Add random suffix to all tables in TestDistributedQueries
alexjo2144 Apr 27, 2020
69f8c05
Clean up planning of unnest
martint Apr 20, 2020
75eabcd
Port join operator to WorkProcessor internally
sopel39 Feb 7, 2020
1ab3968
Fix javadoc
findepi Apr 28, 2020
d5bcdf9
Remove unsound check
findepi Apr 28, 2020
a246feb
Add Like predicate to SHOW COLUMNS
Praveen2112 Mar 6, 2020
bc35fdb
Make Hive caching and S3 security mapping mutually exclusive
losipiuk Apr 28, 2020
6912112
Make Hive caching and GCS access token mutually exclusive
losipiuk Apr 28, 2020
5972bd3
Verify support for correlation during analysis
martint Apr 27, 2020
c14d78f
Check Limit with ties rewritten
kasiafi Apr 29, 2020
052214e
Support order sensitivity in RowNumberNode
kasiafi Apr 25, 2020
ed1d66c
Allow reading ORC files which do not have row-group information
Apr 15, 2020
eafbdcd
Do not used Rubix parallel warmup by default
losipiuk Apr 30, 2020
c970ae0
Replace hive.cache.parallel-warmup-enabled with hive.cache.read-mode
losipiuk Apr 29, 2020
cfd3350
Fix wrong return value in geospatial document
Apr 30, 2020
a380c5d
Add data mapping tests with padded chars to TestDistributedQueries
alexjo2144 Apr 29, 2020
780e272
Verify number of deserialized pages
findepi Apr 15, 2020
4d80836
Verify data integrity in exchanges
findepi Apr 15, 2020
b1a2969
Skip whitespace padded char data mapping test for Kudu
alexjo2144 May 1, 2020
19abcb7
Quote special row field names containing special characters
Praveen2112 Apr 8, 2020
4da1814
Add link to book and paper pages
mosabua Apr 15, 2020
2169c41
Remove duplicated CacheStatsMBean
alexjo2144 Apr 10, 2020
1694c17
Remove unused method
findepi Apr 27, 2020
6843bf9
Simplify Field#toString
findepi Apr 27, 2020
ead1407
Correct constant to represent TIME
findepi Apr 27, 2020
ad48ae0
Simplify type conversion code
findepi Apr 27, 2020
3eef328
Remove redundant test case
findepi May 3, 2020
a51e625
Move some tests to appropriate place
findepi May 3, 2020
13a7cd4
Add some missing types to TestingTypeManager
MiguelWeezardo May 3, 2020
1df1c54
Preserve first close exception
findepi May 4, 2020
0024db7
Rename variable name and extract id column name in Cassandra
ebyhr Apr 22, 2020
8c9b405
Allow INSERT statement for Cassandra table having hidden id column
ebyhr Apr 21, 2020
be31183
Remove usage of deprecated immedateCheckedFuture
electrum Apr 29, 2020
1bf8167
Fix error message for aggregation annotation parser
electrum May 3, 2020
1166b5a
Update to Airbase 100
electrum Apr 28, 2020
c72a820
Add catalog session property details
mosabua Mar 31, 2020
41046f7
Reduce RPM size using symlinks
electrum Apr 30, 2020
ae220a9
Simplify user mapping rule matching
electrum May 4, 2020
a6c7da4
Validate that user mapping rules list is not empty
electrum May 4, 2020
db7fcc9
Update to Airlift 0.196
electrum May 5, 2020
d937ceb
Add documentation for SHOW CREATE SCHEMA
electrum May 4, 2020
eee6045
Document LIKE clause for SHOW COLUMNS
electrum May 4, 2020
b09a619
Remove UseGCOverheadLimit which has no effect for G1
electrum May 4, 2020
3c2ac9f
Remove legacy GC debug flags
electrum May 4, 2020
f33ce93
Fix release note section naming and location
electrum May 5, 2020
45bfd76
Add 333 release notes
electrum May 4, 2020
f754f56
[maven-release-plugin] prepare release 333
electrum May 5, 2020
40ce327
[maven-release-plugin] prepare for next development iteration
electrum May 5, 2020
e76f642
Remove redundant declaration
findepi Apr 30, 2020
cd7c3da
Make table wider in MongoDB document
ebyhr May 2, 2020
43b55cb
Support case insensitive identifiers in MongoDB
ebyhr May 2, 2020
c9731f9
Enable tests related to DROP TABLE in MongoDB
ebyhr May 5, 2020
2ca9b5b
Rename id column to key and shorten name in testColumnName
ebyhr May 5, 2020
fdbeea3
Use Metadata.quote in CassandraCqlUtils
ebyhr May 5, 2020
652673e
Skip valueIsNull to ShortArrayBlock if possible
kokosing May 4, 2020
1b457a1
Change Hive $file_modified_time column to timestamp with timezone type
alexjo2144 May 4, 2020
2aa3de4
Add basic Domain, ValueSet, Range and Marker #toString
findepi Dec 13, 2018
c1bdd86
Inline trivial method
findepi May 4, 2020
6efc6c5
Use better predicate in test
findepi May 4, 2020
75fe578
Use assertThat for better messages on failure
findepi May 4, 2020
1027a89
Extract counting mock connector
findepi May 4, 2020
058556e
Reduce number of objects in test
findepi May 4, 2020
6620434
Add new catalog in TestPrestoDatabaseMetaData
findepi May 4, 2020
550b8a6
Add metadata operation counting JDBC test
findepi May 4, 2020
56dc6bb
Fix failure when unnest inputs are repeated
martint Apr 20, 2020
445aa5d
Remove unused RowNumberNode with non empty partitionBy
kasiafi May 5, 2020
e333672
Use test queries that select some data
findepi May 7, 2020
41642c9
Remove redundant condition
findepi May 7, 2020
a299e32
Use Optional instead of @Nullable parameter
findepi May 6, 2020
8748ffe
Operate on Slice directly
findepi May 6, 2020
75db23a
Add patternConstantPrefixBytes
findepi May 6, 2020
059061d
Use JDK Predicate interface
findepi May 6, 2020
f9dd802
Fix indentation
findepi May 6, 2020
6aea881
Translate LIKE predicate to Domain
findepi May 6, 2020
e776d98
MoveWebUiModule to UI package
dain Apr 14, 2020
98173c2
Add utility for servlet security handling
dain Apr 14, 2020
6224441
Support forwarded https in web ui authentication
dain Apr 14, 2020
3c7a30d
Add extensible web ui authentication bindings
dain Apr 14, 2020
489effa
Add fixed user web ui authenticator
dain Apr 14, 2020
ae6efb8
Add HTTP protocol based authenticators to web ui
dain Apr 14, 2020
7c09049
Move delete analysis to the analyzer
martint May 6, 2020
971b32c
Constraint cleanup
findepi May 7, 2020
77a5913
Rename method and make more flexible
findepi May 6, 2020
a3763b8
Replace FilterUtil.filter method
findepi May 6, 2020
ce9abcd
Simplify control flow
findepi May 6, 2020
1d7cba2
Move empty table name check to MetadataManager
findepi May 7, 2020
fe8fcf0
Optimize system.jdbc.columns metadata calls
findepi May 7, 2020
febbff6
Extract statistics conversion method
findepi Apr 29, 2020
bcf4e6f
Reject unsupported type
findepi Apr 29, 2020
a71fa9b
Add isStatisticsOverflow test
findepi Apr 29, 2020
7d2d9db
Use Parquet statistics for decimal with 0 scale
findepi Apr 29, 2020
82fb9a5
Minor cleanup of BigQuery connector
ebyhr May 6, 2020
c7d969c
Add random functions with range
victormazevedo Oct 24, 2019
c569203
Embed version in commit stacktrace
findepi May 8, 2020
b33c4ff
Update jQuery to 3.5.1 and Bootstrap to 3.4.1
ShashwatArghode May 6, 2020
7d54901
Fix formatting
kasiafi May 6, 2020
1ff463a
Add static import
kasiafi May 6, 2020
7a02a48
Add project-off rule for TopNRowNumberNode
kasiafi May 7, 2020
d7f53eb
Add project-off rule for SortNode
kasiafi May 8, 2020
1d595b6
Prune all unreferenced symbols in PruneUnreferencedOutputs.visitWindow()
kasiafi May 7, 2020
2ebbc64
Verify event-listener.name validity
findepi May 8, 2020
ca0851f
Remove usage of Unsafe from UnscaledDecimal128Arithmetic
electrum May 6, 2020
8973a79
Use signal support from jline
electrum May 6, 2020
9377f58
Extract error handling processing to a separate method
martint Apr 29, 2020
176d229
Optimize Elasticsearch queries that fetch no columns
martint Apr 29, 2020
6a90baa
Remove extra empty line
sopel39 Mar 14, 2020
b5b9c87
Fix handling of EmptySplit
sopel39 Mar 18, 2020
8f91739
Ensure current group is materialized before fetching next page
sopel39 Mar 22, 2020
3afbdda
Remove dynamic filters from spatial join
sopel39 May 11, 2020
bbc538c
Static import PlanBuilder#expression
sopel39 May 11, 2020
984fa4c
Fix formatting
sopel39 May 11, 2020
b0fce29
Avoid adding local exchanges around trivial projection
kasiafi May 2, 2020
58c58bd
Add overwrite insert for Hive connector
mosabua Apr 7, 2020
eac91c0
Document how JMX is enabled
mosabua May 8, 2020
a1b3b42
Check filter dependencies in constructor for join involving unnest
kasiafi May 8, 2020
87adeb0
Refactor and fix UnnestNode symbol pruning
kasiafi May 8, 2020
2af61a7
Add column-pruning rule for UnnestNode
kasiafi May 8, 2020
a6a4d0a
Add project-off rule for UnnestNode
kasiafi May 8, 2020
42a1b77
Improve Domain#overlaps
raunaqmorarka May 5, 2020
2558c28
Fix typo on drop-schema docs
chhsiao90 May 13, 2020
8935300
Correct uses of TupleDomain#isAll
findepi Apr 24, 2020
23eb6c6
Use TupleDomain#transform
findepi Apr 24, 2020
cc97b7b
Add TupleDomain transformations
findepi Apr 24, 2020
ef3a5bc
Prefix dynamic filter IDs
findepi May 13, 2020
f4d7de3
Remove configuration for Presto version
electrum May 11, 2020
f3cfb01
Fix formatting
sopel39 May 13, 2020
150b9c7
Reverse if
sopel39 May 14, 2020
e77109d
Add basic Rubix caching unit test
sopel39 May 13, 2020
315f567
Refactor RubixConfigurationInitializer into dynamic provider
sopel39 May 14, 2020
8d4baa3
Make sure Rubix cache is not used with dynamic config providers
sopel39 May 14, 2020
c8e4f39
Add rubix module test
sopel39 May 14, 2020
b526481
Update TestJmxHistoricalData for newer Java versions
electrum May 14, 2020
6bc60f3
Build server for Java 11
electrum May 9, 2020
685ae60
Run PrestoServer main on Java 8 for version check
electrum May 12, 2020
e7eeeed
Replace JavaVersion with JDK API
electrum May 12, 2020
8ca42c0
Switch back to platform class loader for plugins
electrum May 14, 2020
cd269e9
Verify arguments to RowNumberNode constructor
kasiafi May 3, 2020
4bafa84
Improve Filter predicate push into RowNumber
kasiafi May 3, 2020
71dad31
Add Remote exchange forcefully if aggregation has empty grouping set
Praveen2112 May 9, 2020
2d01e65
Fix Result type precision in decimal operator document
tom-kita May 3, 2020
cad2f86
Add memsql plugin to server
findepi May 15, 2020
d1c7fd9
Use expression lambdas
sopel39 May 15, 2020
eb220d4
Optimizer for merging set operations
Lewuathe Jan 21, 2020
d5982fa
Remove fake properties from Elastic Search column metadata
losipiuk May 15, 2020
69804f4
Enable testShowCreateTable for Elastic Search
losipiuk May 15, 2020
2b39b6f
Replace x-forwarded support with Airlift version
dain May 13, 2020
066648d
Minor cleanup of EventListenerManager
electrum May 14, 2020
e92ed27
Set thread context class loader for EventListenerFactory
electrum May 14, 2020
b757e52
Use JDK unmodifiable collections in SPI
electrum May 15, 2020
d130780
Add pinot connector
elonazoulay Nov 15, 2019
e92ff99
Fix typo in list of functions and operators document
ebyhr May 15, 2020
5a0320c
Add milliseconds to final time format in case of short queries
Apr 2, 2020
af4bca3
Compare datetime types with equals()
martint May 15, 2020
50bb37d
Extract dataMapping tableName creation
eskabetxe Apr 20, 2020
f75484f
Add Oracle connector
eskabetxe May 15, 2020
6809887
Fix hash for -0
findepi May 15, 2020
1cdea0f
Rename S3 host to bucket
electrum Dec 1, 2019
0b18706
Show distributed plan by default in EXPLAIN
electrum May 13, 2020
dbf6aab
Use global AWS SDK version for Kinesis
electrum May 5, 2020
93aedae
Move special modules to the end
electrum Apr 15, 2020
5a20c2a
Fix visibility of methods in Server
electrum May 15, 2020
bcb15be
Remove unused method arguments
electrum May 15, 2020
23fff72
Improve documentation for Hive directory listing caching
electrum Apr 9, 2020
6b846be
Cleanup documentation for binary functions
electrum Apr 20, 2020
74eb484
Reorganize math function documentation
electrum Apr 20, 2020
ac6ac5a
Use same font for function parentheses
electrum Apr 20, 2020
dbd3cdf
Prevent OutputBufferMemoryManager from becoming negative
electrum May 4, 2020
7ff3ace
Use Instant to track session start time
martint May 17, 2020
6d9d47e
Use Optional.isEmpty() method
electrum May 15, 2020
107716a
Use Optional.stream() method
electrum May 15, 2020
37c74ad
Simplify Optional usage
electrum May 15, 2020
ffc4524
Fix InMemoryNodeManager
sopel39 May 15, 2020
98d0383
Add large file test for Rubix
sopel39 May 15, 2020
6fbf2d9
Improve testCacheRead test
sopel39 May 15, 2020
9943fb5
Simplify RubixInitializer#initializeRubix method
sopel39 May 18, 2020
d50efd0
Reduce methods visibility
sopel39 May 18, 2020
be328f5
Mark fields as @Nullable
sopel39 May 18, 2020
f782f4b
Add missing null checks
sopel39 May 18, 2020
5af5d74
Fix formatting
sopel39 May 18, 2020
8787f6c
Use method reference instead of lambda
sopel39 May 18, 2020
14afa2d
Use RetryDriver to wait for coordinator node
sopel39 May 18, 2020
faadbba
Extract Rubix startup into separate method
sopel39 May 18, 2020
5128d1f
Initialize more fields in TestRubixCaching
sopel39 May 18, 2020
81c14d6
Wait for Rubix to start during Hive connector initialization
sopel39 May 18, 2020
789cb9a
Use @PostConstruct to initialize Rubix instead of explicit method call
sopel39 May 18, 2020
5be57b6
Add missing binding
findepi May 14, 2020
c2d3421
Add cache write test
sopel39 May 18, 2020
1d256a2
Update Rubix version to 0.3.9
sopel39 May 18, 2020
28b6425
Allow Rubix caching for all Azure filesystems
losipiuk May 5, 2020
f7e9c44
Allow Rubix caching for HDFS filesystem
losipiuk May 5, 2020
9afc43f
Remove unused method
sopel39 May 18, 2020
1d66614
Use warning for principal rules deprecation
electrum May 19, 2020
f449611
Disable testSelectInformationSchemaColumns in Oracle connector
ebyhr May 19, 2020
47f1019
Stop Rubix in tests and connector shutdown
sopel39 May 18, 2020
5118e55
Pass FileSystems directly to Closer
sopel39 May 19, 2020
f54c1c6
Use Docker Hub for testcontainers' Ryuk
findepi May 19, 2020
118b06b
Remove unused local variable in ExpressionAnalysis
Lewuathe May 19, 2020
42669fe
Add presto-ml doc
Lewuathe May 9, 2020
204783d
Add AVRO product test group
sopel39 May 20, 2020
43f2be5
Add Rubix HDFS product test
sopel39 May 19, 2020
81ca81f
Rename tempto config file
findepi May 20, 2020
d33500e
Implement toString() for JDBC driver Array result
electrum May 20, 2020
24fcd00
Fix matching Hive bucket filenames without leading zeros
ArvinZheng May 11, 2020
ca82775
Add pluggable CertificateAuthenticator
electrum May 11, 2020
d312077
Improved concurrency for Elasticsearch calls
aalbu May 4, 2020
66c3609
Fix support for creating Hive external tables using CTAS
alexjo2144 May 15, 2020
e1a0c28
Remove deprecated missingAttributeException usage
Lewuathe May 20, 2020
eb3dce7
Remove obsolete comment
sopel39 May 21, 2020
d877593
Mark test class as single-threaded
findepi May 21, 2020
2754c55
Remove redundant synchronization
sopel39 May 21, 2020
360ed2b
Make it explicit that none of the futures should fail
sopel39 May 21, 2020
07eefdf
Static import MoreFutures#getFutureValue
sopel39 May 21, 2020
ed96422
Use dedicated cache key for internal Rubix configuration
sopel39 May 21, 2020
0680bef
Test Hive caching with two tables created in sequence
sopel39 May 21, 2020
7be38e6
Remove unused method
phd3 May 4, 2020
7537ef0
Fix error message in WindowNode.Function constructor
phd3 May 5, 2020
f00fae5
Support multiple unnest outputs in unnest matcher
phd3 May 19, 2020
69ef682
Pushdown dereference expressions in the query plan
phd3 Aug 30, 2019
5dedde4
Plan assertions for end to end dereference pushdown in hive
phd3 Apr 15, 2020
2fe05db
Add support for passthrough Elasticsearch queries
martint May 14, 2020
e471fed
Eliminate deprecated stream method usage
Lewuathe May 22, 2020
75a539d
Fix Launcher local startup failure
findepi May 22, 2020
f6c3773
Don't start Rubix server by default on coordinator
sopel39 May 22, 2020
532bdba
Use MBean to obtain caching stats in tests
sopel39 May 21, 2020
dc2c82e
Use assertEventually in Rubix tests
sopel39 May 21, 2020
73b93c2
Add Rubix read mode tests
sopel39 May 20, 2020
f921ef0
Use generated table data in Rubix product test
sopel39 May 21, 2020
9c8c815
Use Rubix async read mode by default
sopel39 May 20, 2020
fa92f95
Add Java requirement to JDBC driver documentation
mosabua May 15, 2020
097bffb
Use varchar for Elasticsearch passthrough query result
martint May 22, 2020
1d9c37f
Add ISO date and time examples
mosabua May 12, 2020
b4bd966
Fix documentation beginQuery and cleanupQuery
electrum May 22, 2020
c8f7a93
show join distribution type in webui live plan
charlesjmorgan May 18, 2020
eae0a0a
Make TestHiveCaching#testReadFromTable more resilient
sopel39 May 22, 2020
a2b0697
Add requirements, pspg and authentication for CLI
mosabua May 19, 2020
5969330
Add software requirements
mosabua May 12, 2020
d91c816
Detail Hive connector requirements
mosabua May 19, 2020
3e0574c
Increase process numbers to 1000 from 500 in Oracle test
ebyhr May 20, 2020
a9f2b2c
Remove unused code in TypeSignatureParser
VicoWu May 22, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use generated table data in Rubix product test
  • Loading branch information
sopel39 committed May 22, 2020
commit f921ef077471c0d38b8bc1c7b9be67b4f14d694f
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
import io.prestosql.tempto.query.QueryResult;
import org.testng.annotations.Test;

import java.util.Random;

import static com.google.common.collect.Iterables.getOnlyElement;
import static io.airlift.testing.Assertions.assertGreaterThan;
import static io.prestosql.tempto.assertions.QueryAssert.Row.row;
Expand All @@ -44,15 +46,14 @@ private void testReadFromTable(String tableNameSuffix)
String cachedTableName = "hive.default.test_cache_read" + tableNameSuffix;
String nonCachedTableName = "hivenoncached.default.test_cache_read" + tableNameSuffix;

query("DROP TABLE IF EXISTS " + nonCachedTableName);
query("CREATE TABLE " + nonCachedTableName + " WITH (format='ORC') AS SELECT 'Hello world' as col");
String tableData = createTestTable(nonCachedTableName);

QueryResult beforeCacheStats = getCacheStats();
long beforeRemoteReads = getRemoteReads(beforeCacheStats);
long beforeCachedReads = getCachedReads(beforeCacheStats);

assertThat(query("SELECT * FROM " + cachedTableName))
.containsExactly(row("Hello world"));
.containsExactly(row(tableData));

assertEventually(
new Duration(10, SECONDS),
Expand All @@ -68,7 +69,7 @@ private void testReadFromTable(String tableNameSuffix)
long firstCachedReads = getCachedReads(firstCacheStats);

assertThat(query("SELECT * FROM " + cachedTableName))
.containsExactly(row("Hello world"));
.containsExactly(row(tableData));

assertEventually(
new Duration(10, SECONDS),
Expand All @@ -82,6 +83,25 @@ private void testReadFromTable(String tableNameSuffix)
query("DROP TABLE " + nonCachedTableName);
}

/**
* Creates table with text files that are larger than 1MB
*/
private String createTestTable(String tableName)
{
StringBuilder randomDataBuilder = new StringBuilder();
Random random = new Random();
for (int i = 0; i < 500_000; ++i) {
randomDataBuilder.append(random.nextInt(10));
}
String randomData = randomDataBuilder.toString();

query("DROP TABLE IF EXISTS " + tableName);
// use `format` to overcome SQL query length limit
query("CREATE TABLE " + tableName + " WITH (format='TEXTFILE') AS SELECT format('%1$s%1$s%1$s%1$s%1$s', '" + randomData + "') as col");

return randomData.repeat(5);
}

private QueryResult getCacheStats()
{
return query("SELECT cachedreads, remotereads FROM " +
Expand Down