Skip to content

Commit

Permalink
Updated dependencies to address major CVEs
Browse files Browse the repository at this point in the history
- updated LICENSE and NOTICE files accordingly
- updated sub-module links
- fixed formatting in release notes
  • Loading branch information
sumwale committed Jun 12, 2022
1 parent be8d170 commit c7b84fa
Show file tree
Hide file tree
Showing 10 changed files with 210 additions and 765 deletions.
657 changes: 65 additions & 592 deletions LICENSE

Large diffs are not rendered by default.

169 changes: 65 additions & 104 deletions NOTICE

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion aqp
17 changes: 9 additions & 8 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,14 @@ buildscript {
classpath 'de.undercouch:gradle-download-task:3.4.3'
classpath 'net.rdrei.android.buildtimetracker:gradle-plugin:0.11.+'
classpath 'com.netflix.nebula:gradle-ospackage-plugin:5.2.+'
// classpath 'org.owasp:dependency-check-gradle:6.5.3'
classpath 'org.owasp:dependency-check-gradle:7.1.0.1'
}
}

apply plugin: 'wrapper'
apply plugin: 'distribution'
apply plugin: 'nebula.ospackage-base'
apply plugin: "nebula.ospackage"
// apply plugin: 'org.owasp.dependencycheck'

// def isEnterpriseProduct = rootProject.hasProperty('snappydata.enterprise')

Expand Down Expand Up @@ -70,6 +69,7 @@ allprojects {
apply plugin: 'com.github.johnrengelman.shadow'
apply plugin: 'idea'
apply plugin: "build-time-tracker"
apply plugin: 'org.owasp.dependencycheck'

group = 'io.snappydata'
version = '1.3.1'
Expand Down Expand Up @@ -112,7 +112,7 @@ allprojects {
snappySparkMetricsLibVersion = '2.0.0.1'
log4j2Version = '2.17.2'
slf4jVersion = '1.7.32'
junitVersion = '4.12'
junitVersion = '4.13.2'
mockitoVersion = '1.10.19'
hadoopVersion = '3.2.0'
awsSdkVersion = '1.11.375'
Expand All @@ -126,8 +126,8 @@ allprojects {
fastutilVersion = '8.5.6'
kryoVersion = '4.0.1'
thriftVersion = '0.9.3'
jacksonVersion = '2.13.1'
jacksonDatabindVersion = '2.13.1'
jacksonVersion = '2.13.3'
jacksonDatabindVersion = '2.13.3'
hiveVersion = '1.21.2.7.0.3.2-3'
metricsVersion = '4.0.3'
metrics2Version = '2.2.0'
Expand All @@ -140,6 +140,7 @@ allprojects {
objenesisVersion = '3.0.1'
rabbitMqVersion = '4.9.1'
akkaVersion = '2.3.16'
nettyAkkaVersion = '3.10.6.Final'
sprayVersion = '1.3.4'
sprayJsonVersion = '1.3.5'
sprayShapelessVersion = '1.3.3'
Expand All @@ -148,7 +149,7 @@ allprojects {
jodaTimeVersion = '2.10.1'
slickVersion = '2.1.0'
h2Version = '1.3.176'
commonsIoVersion = '2.6'
commonsIoVersion = '2.11.0'
commonsPoolVersion = '1.6'
dbcpVersion = '1.4'
shiroVersion = '1.2.6'
Expand Down Expand Up @@ -990,8 +991,8 @@ gradle.taskGraph.whenReady { graph ->
Set<String> skipPublishFor = [
'snappydata_' + scalaBinaryVersion, 'snappy-spark', 'snappy-store',
'snappy-dtests_' + scalaBinaryVersion, 'snappy-compatibility-tests_' + scalaBinaryVersion,
'snappydata-native', 'snappydata-store-prebuild',
'snappy-spark-assembly_' + scalaBinaryVersion
'gemfire-junit', 'gemfire-tests', 'snappydata-store-tests', 'snappydata-native',
'snappydata-store-prebuild', 'snappy-spark-assembly_' + scalaBinaryVersion
]

task publishLocal() {
Expand Down
6 changes: 5 additions & 1 deletion cluster/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,11 @@ dependencies {
//exclude(group: 'org.scala-lang', module: 'scala-library')
//exclude(group: 'org.apache.avro', module: 'avro')
}
compile(group:'com.google.cloud.bigdataoss', name:'gcs-connector', version: gcsHadoop3ConnectorVersion, classifier: 'shaded')
compile(group:'com.google.cloud.bigdataoss', name:'gcs-connector', version: gcsHadoop3ConnectorVersion, classifier: 'shaded') {
// shaded jar includes all dependencies but the pom still lists all dependencies as per
// the non-shaded jar, so skip all dependencies explicitly
transitive = false
}

testCompile project(':dunit')
testCompile "org.eclipse.collections:eclipse-collections:${eclipseCollectionsVersion}"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ trait SnappyJobTestSupport extends Logging {
* exceeds specified time, the method will throw exception.
*/
def submitAndWaitForCompletion(classFullName: String, jobCmdAffix: String = "",
waitTimeMillis: Int = 60000): Unit = {
waitTimeMillis: Int = 120000): Unit = {
val consoleLog: String = submitJob(classFullName, jobCmdAffix)
logInfo("Job submission response:" + consoleLog)
val jobId = getJobId(consoleLog)
Expand Down
116 changes: 61 additions & 55 deletions docs/release_notes/release_notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ SnappyData 1.3.1 release includes the following new features over the previous 1
to the latest Log4J 2.x (2.17.2) from the previous Log4J 1.x. Patches were ported for the Spark components
(where support for Log4J 2.x will land only with the 3.3.0 release), while other components were updated
to use Log4J/SLF4J. The Spark connector component supports both Log4J 2.x and Log4J 1.x to allow compatibility
with upstream Spark releases which the SnappyData's Spark distribution only uses Log4J 2.x.
with upstream Spark releases while the SnappyData's Spark distribution only uses Log4J 2.x.


## Stability and Security Improvements
Expand All @@ -88,68 +88,74 @@ SnappyData 1.3.1 includes the following changes to improve stability and securit
* Fixed a race condition in old entries cleaner thread deleting in-use snapshot entries.

* Allow retry in startup failure even for data nodes. In some rare cases region initialization may fail
due to colocated region still being initialized, so retry region initialization.
due to colocated region still being initialized, so retry region initialization for such cases.

* Fixed UDF names lookups to do exact regex match in the CSV list in the meta-data region.
* Fixed UDF name lookups to do exact regex match in the CSV list in the meta-data region.

* Apart from Log4J, following dependencies were updated to address known security issues:
- Jetty upgraded to 9.4.44.v20210927
- jackson-mapper-asl and jackson-core-asl upgraded to 1.9.14-atlassian-6
- jackson and jackson-databind upgraded to 2.13.1
- Kafka upgraded to 2.2.2
- [SPARK-34110](https://issues.apache.org/jira/browse/SPARK-34110): Upgrade Zookeeper to 3.6.2
- [SPARK-37901](https://issues.apache.org/jira/browse/SPARK-37901): Upgrade Netty to 4.1.73
- gcs-hadoop-connector upgraded to hadoop3-2.1.2
- Jetty upgraded to 9.4.44.v20210927
- jackson-mapper-asl and jackson-core-asl upgraded to 1.9.14-atlassian-6
- jackson and jackson-databind upgraded to 2.13.1
- Kafka upgraded to 2.2.2
- [SPARK-34110](https://issues.apache.org/jira/browse/SPARK-34110): Upgrade Zookeeper to 3.6.2
- [SPARK-37901](https://issues.apache.org/jira/browse/SPARK-37901): Upgrade Netty to 4.1.73
- gcs-hadoop-connector upgraded to hadoop3-2.1.2

* Ported patches for the following issues from Apache Geode:
- [GEODE-1252](https://issues.apache.org/jira/browse/GEODE-1252): Modify bits field atomically
- [GEODE-2802](https://issues.apache.org/jira/browse/GEODE-2802): Tombstone version vector to contain only
- [GEODE-1252](https://issues.apache.org/jira/browse/GEODE-1252): Modify bits field atomically
- [GEODE-2802](https://issues.apache.org/jira/browse/GEODE-2802): Tombstone version vector to contain only
the members that generate the tombstone
- [GEODE-5278](https://issues.apache.org/jira/browse/GEODE-5278): Unexpected CommitConflictException caused by
- [GEODE-5278](https://issues.apache.org/jira/browse/GEODE-5278): Unexpected CommitConflictException caused by
faulty region synchronization
- [GEODE-4083](https://issues.apache.org/jira/browse/GEODE-4083): Fix infinite loop caused by thread race
changing version
- [GEODE-3796](https://issues.apache.org/jira/browse/GEODE-3796): Changes are made to validate region version
after the region is initialized
- [GEODE-6058](https://issues.apache.org/jira/browse/GEODE-6058): recordVersion should allow update higher local
version if for non-persistent region
- [GEODE-6013](https://issues.apache.org/jira/browse/GEODE-6013): Use expected initial image requester's
rvv information
- [GEODE-2159](https://issues.apache.org/jira/browse/GEODE-2159): Add serialVersionUIDs to exception classes
not having them
- [GEODE-5559](https://issues.apache.org/jira/browse/GEODE-5559): Improve runtime of
RegionVersionHolder.canonicalExceptions
- [GEODE-5612](https://issues.apache.org/jira/browse/GEODE-5612): Fix RVVExceptionB.writeReceived()
- [GEODE-7085](https://issues.apache.org/jira/browse/GEODE-7085): Ensure that the bitset stays within
BIT_SET_WIDTH and is flushed in all code paths
- GFE-50415: Wait for membership change in persistence advisor can hang if the event was missed
- [GEODE-5111](https://issues.apache.org/jira/browse/GEODE-5111): Set offline members to null only when done
waiting for them
- [GEODE-4083](https://issues.apache.org/jira/browse/GEODE-4083): Fix infinite loop caused
by thread race changing version
- [GEODE-3796](https://issues.apache.org/jira/browse/GEODE-3796): Changes are made to
validate region version after the region is initialized
- [GEODE-6058](https://issues.apache.org/jira/browse/GEODE-6058): recordVersion should
allow update higher local version if for non-persistent region
- [GEODE-6013](https://issues.apache.org/jira/browse/GEODE-6013): Use expected initial
image requester's rvv information
- [GEODE-2159](https://issues.apache.org/jira/browse/GEODE-2159): Add serialVersionUIDs to
exception classes not having them
- [GEODE-5559](https://issues.apache.org/jira/browse/GEODE-5559): Improve runtime of
RegionVersionHolder.canonicalExceptions
- [GEODE-5612](https://issues.apache.org/jira/browse/GEODE-5612):
Fix RVVExceptionB.writeReceived()
- [GEODE-7085](https://issues.apache.org/jira/browse/GEODE-7085): Ensure that the bitset
stays within BIT_SET_WIDTH and is flushed in all code paths
- GFE-50415: Wait for membership change in persistence advisor can hang if the member
join event was missed
- [GEODE-5111](https://issues.apache.org/jira/browse/GEODE-5111): Set offline members to
null only when done waiting for them

* Merged patches for the following Spark issues:
- [SPARK-6305](https://issues.apache.org/jira/browse/SPARK-6305): Migrate from log4j1 to log4j2
- Followups SPARK-37684, SPARK-37774 to upgrade log4j to 2.17.x
- [SPARK-37791](https://issues.apache.org/jira/browse/SPARK-37791): Use log4j2 in examples
- [SPARK-37794](https://issues.apache.org/jira/browse/SPARK-37794): Remove internal log4j bridge api usage
- [SPARK-37746](https://issues.apache.org/jira/browse/SPARK-37746): log4j2-defaults.properties is not working
since log4j 2 is always initialized by default
- [SPARK-37792](https://issues.apache.org/jira/browse/SPARK-37792): Fix the check of custom configuration in
SparkShellLoggingFilter
- [SPARK-37795](https://issues.apache.org/jira/browse/SPARK-37795): Add a scalastyle rule to ban `org.apache.log4j`
imports
- [SPARK-37805](https://issues.apache.org/jira/browse/SPARK-37805): Refactor `TestUtils#configTestLog4j` method
to use log4j2 api
- [SPARK-37889](https://issues.apache.org/jira/browse/SPARK-37889): Replace Log4j2 MarkerFilter with RegexFilter
- [SPARK-26267](https://issues.apache.org/jira/browse/SPARK-26267): Retry when detecting incorrect offsets
from Kafka
- [SPARK-37729](https://issues.apache.org/jira/browse/SPARK-37729): Fix SparkSession.setLogLevel that is not
working in Spark Shell
- [SPARK-37887](https://issues.apache.org/jira/browse/SPARK-37887): Fix the check of repl log level
- [SPARK-37790](https://issues.apache.org/jira/browse/SPARK-37790): Upgrade SLF4J to 1.7.32
- [SPARK-22324](https://issues.apache.org/jira/browse/SPARK-22324): Upgrade Arrow to 0.8.0
- [SPARK-25598](https://issues.apache.org/jira/browse/SPARK-25598): Remove flume connector in Spark
- [SPARK-37693](https://issues.apache.org/jira/browse/SPARK-37693): Fix ChildProcAppHandleSuite failed in Jenkins
maven test
- [SPARK-6305](https://issues.apache.org/jira/browse/SPARK-6305): Migrate from log4j1 to log4j2
- Followups SPARK-37684, SPARK-37774 to upgrade log4j to 2.17.x
- [SPARK-37791](https://issues.apache.org/jira/browse/SPARK-37791): Use log4j2 in examples
- [SPARK-37794](https://issues.apache.org/jira/browse/SPARK-37794): Remove internal log4j
bridge api usage
- [SPARK-37746](https://issues.apache.org/jira/browse/SPARK-37746):
log4j2-defaults.properties is not working since log4j 2 is always initialized by default
- [SPARK-37792](https://issues.apache.org/jira/browse/SPARK-37792): Fix the check of
custom configuration in SparkShellLoggingFilter
- [SPARK-37795](https://issues.apache.org/jira/browse/SPARK-37795): Add a scalastyle rule
to ban `org.apache.log4j` imports
- [SPARK-37805](https://issues.apache.org/jira/browse/SPARK-37805):
Refactor `TestUtils#configTestLog4j` method to use log4j2 api
- [SPARK-37889](https://issues.apache.org/jira/browse/SPARK-37889): Replace Log4j2
MarkerFilter with RegexFilter
- [SPARK-26267](https://issues.apache.org/jira/browse/SPARK-26267): Retry when detecting
incorrect offsets from Kafka
- [SPARK-37729](https://issues.apache.org/jira/browse/SPARK-37729):
Fix SparkSession.setLogLevel that is not working in Spark Shell
- [SPARK-37887](https://issues.apache.org/jira/browse/SPARK-37887): Fix the check of REPL
log level
- [SPARK-37790](https://issues.apache.org/jira/browse/SPARK-37790): Upgrade SLF4J to 1.7.32
- [SPARK-22324](https://issues.apache.org/jira/browse/SPARK-22324): Upgrade Arrow to 0.8.0
- [SPARK-25598](https://issues.apache.org/jira/browse/SPARK-25598): Remove flume connector
in Spark
- [SPARK-37693](https://issues.apache.org/jira/browse/SPARK-37693):
Fix ChildProcAppHandleSuite failed in Jenkins maven test


## Resolved Issues
Expand All @@ -162,7 +168,7 @@ SnappyData 1.3.1 resolves the following major issues apart from the patches note
## Known Issues

The known issues noted in [1.3.0 release notes](https://tibcosoftware.github.io/snappydata/1.3.0/release_notes/release_notes/#known-issues)
still apply in 1.3.1 release.
still apply in 1.3.1 release. These have been reproduced below for reference:

| Key | Item | Description | Workaround |
| --- | ---- | ----------- | ---------- |
Expand Down
2 changes: 1 addition & 1 deletion spark
Submodule spark updated from cba053 to 0169c3
2 changes: 1 addition & 1 deletion spark-jobserver
Submodule spark-jobserver updated 1 files
+1 −1 build.gradle

0 comments on commit c7b84fa

Please sign in to comment.