Spring Cloud Data Flow 2.11.5

Dependent Projects and Compatibility

Component	Version
Spring Cloud Deployer	2.9.5
Spring Cloud Skipper	2.11.5
Spring Cloud Dataflow UI	3.4.6
Spring Cloud Dataflow	2.11.5

Issues

• spring-cloud/spring-cloud-deployer#465 Allow setting up more than one initContainer per platform
• spring-cloud/spring-cloud-deployer#462 Node Selector doesn't get applied on K8s platforms
• #5909 User should be able to set app version when creating schedule.
• #5854 Task table prefix is not resolved correctly in CTR

🔨 Dependency Upgrades

• Spring Framework to 5.3.39
• Reactor to 2020.0.47
• Netty to 4.1.113.Final
• Tomcat to 9.0.93
• Snappy Java 1.1.10.6

What's Changed in Spring Cloud Dataflow and Skipper from v2.11.4 to v2.11.5

#5931 Update various dependencies patch versions @onobc
#5905 [CI] Remove actions that don't add value. @corneil
#5904 Update to 2.11.5-SNAPSHOT @corneil
#5887 Handle case where docker-compose executable is absent. @corneil
#5866 Update documentation for initContainers addition in spring-cloud-deployer @corneil
#5857 CTR needs to support boot property styles for the tablePrefix property @cppwfs
#5852 Port download-apps.sh to macOs @cachescrubber
#5817 Added verification of tasks/thinexecutions in DataflowOAuthIT. @corneil

Full Changelog: v2.11.4...v2.11.5

Spring Cloud Data Flow 2.11.4

Dependent Projects and Compatibility

Component	Version
Spring Cloud Dataflow	2.11.4
Spring Cloud Skipper	2.11.4
Spring Cloud Deployer	2.9.4
Spring Cloud Dataflow UI	3.4.5

Issues

• #5848 BadSqlGrammarException for Spring Boot 3 Task Applications
• #5825 Security issues in transitive dependency of xmlunit-core-2.9.1.jar
• #5705 Task schedule is not using the version defined in the properties file

🔨 Dependency Upgrades

• Spring Framework to 5.3.37
• Jackson to 2.17.2
• Reactor to 2020.0.45
• Netty to 4.1.111.Final
• Tomcat to 9.0.90
• nimbus-jose-jwt to 9.37.3
• testcontainers to 1.19.8

What's Changed in Spring Cloud Dataflow and Skipper from v2.11.3 to v2.11.4

• Update scope of spring-boot-starter-test by @corneil in #5826
• Add spring-cloud-dataflow-server to version-info. by @corneil in #5822
• All src/deploy scripts must be updated to the 2.11.4 release. by @cppwfs in #5827
• [CI] Remove unused labeler workflow by @onobc in #5840
• last task execution performance by @corneil in #5843
• Remove redundant dep. mgmt for Bouncycastle by @onobc in #5855
• [CI] Create workflow for uaa-test container. by @corneil in #5861
• Update Spring Framework to 5.3.37 by @corneil in #5863
• Fix build-image name for containers. by @corneil in #5864
• Update Tomcat for 9.0.90 by @corneil in #5865
• Update various lib deps patch versions by @onobc in #5868
• Remove Dependency Mgmt for H2 and rely on Boot Bom by @cppwfs in #5870
• Ensure PackageMetadata is created with SafeConstructor by @cppwfs in #5871

Full Changelog: v2.11.3...v2.11.4

What's Changed in Spring Cloud Deployer from v2.9.3 to v2.9.4

• Update Spring Framework to 5.3.34 by @corneil in spring-cloud/spring-cloud-deployer#449
• Add dep. mgmt. for commons-io and commons-compress by @onobc in spring-cloud/spring-cloud-deployer#452
• [CI] Remove unused labeler workflow file by @onobc in spring-cloud/spring-cloud-deployer#456
• Update Jackson to 2.17.1 by @onobc in spring-cloud/spring-cloud-deployer#457
• Update hashids to 1.0.3 by @corneil in spring-cloud/spring-cloud-deployer#458
• Fix cfenv detection by @jvalkeal in spring-cloud/spring-cloud-deployer#461
• Put dependency versions in properties by @onobc in spring-cloud/spring-cloud-deployer#467

Full Changelog: spring-cloud/spring-cloud-deployer@v2.9.3...v2.9.4

Spring Cloud Data Flow 2.11.3

Dependent Projects and Compatibility

Component	Version
Spring Cloud Deployer	2.9.3
Spring Cloud Skipper	2.11.3
Spring Cloud Dataflow UI	3.4.3
Spring Cloud Dataflow	2.11.3

Issues

• spring-cloud/spring-cloud-deployer#444 Update Kubernetes Support Matrix for 2.9.x
• spring-cloud/spring-cloud-dataflow-ui#1996 Update Task Execution list to use tasks/thinexecutions
• #5794 Fix code scanning alert - Spring-Kafka has Java Deserialization vulnerability When Improperly Configured
• #5783 --propertiesFile option on scdf shell command "stream update" not actually available.
• #5782 Fix the thin task execution page query for CTR status
• #5780 Vulnerabilities affecting Spring Cloud Dataflow dependencies
• #5778 Publish Skipper 2.11.x reference docs
• #5718 Unable to get list of task executions with a large amount of task execution rows
• #5695 Further performance improvements for job executions (for oracle, other dbs might vary)
• #5643 Address json-path CVE-2023-51074
• #5640 Re-enable SimpleJobServicePostgresTests

Spring Cloud Data Flow 2.11.2

Dependent Projects and Compatibility

Component	Version
Spring Cloud Dataflow Build	2.11.2
Spring Cloud Dataflow Common	2.11.2
Spring Cloud Deployer	2.9.2
Spring Cloud Deployer Local	2.9.2
Spring Cloud Deployer CF	2.9.2
Spring Cloud Deployer K8S	2.9.2
Spring Cloud Common Security Config	2.11.2
Spring Cloud Skipper	2.11.2
Spring Cloud Dataflow UI	3.4.2
Spring Cloud Dataflow	2.11.2

Issues

#5593 Update logback to handle CVE-2023-6378
spring-cloud/spring-cloud-deployer#407 CF Deployer needs to use LogCacheClient to retrieve logs
spring-cloud/spring-cloud-dataflow-ui#1974 Creating Schedule of Task prevents version property
spring-cloud/spring-cloud-dataflow-ui#1944 Task Execution Log doesn't scroll horizontally
#5615 Unable to sort tasks by Schema Target
#5611 Update to Spring Cloud 2021.0.9
#5609 Remove of JOB_CONFIGURATION_LOCATION broke some queries
#5591 The BOOT3 batch schema has old columns in table
#5576 Mariadb Schema needs to change BOOT3 Sequence tabels to sequences.
#5568 Schedule Task with Arguments and Properties does not work
#5564 Check dependencies for Spring Boot 2.7.18
#5561 Update Spring Boot 2.7.18
#5522 Update to Spring Boot 2.7.17
#5557 Step Context section is not populated on the UI
#5555 BOOT3_BATCH_JOB_EXECUTION should not contain JOB_CONFIGURATION_LOCATION
#5551 Batch Remote Parititioning does not pass Boot3 info to partitions
#5549 getJobExecutionsWithStepCountFilteredByTaskExecutionId in JdbcSearchableJobExecutionDao does not support task prefixes
#5548 Unable to configure Harbor as container registry
#5539 As a developer I need to run Oracle locally on my Mac ARM64
#5534 Task re-launch fails when using BOOT3 and Oracle
#5533 Add Boot3 Support for SimpleJobService
#5531 Unable to navigate through the job execution pages
#5524 Controller /jobs/thinexecutions is very slow with a large amount of job executions and no navigation possible oracle db
#5520 Stopping a batch with Spring Boot 3/Spring Batch 5 doesn't work properly.
#5518 Relaunch of a Spring Boot 3/Spring Batch 5 batch doesn't work properly.
#5507 Add commit info to the about controller
#5496 spring.cloud.dataflow.task.platform.kubernetes.accounts.[platformid].initContainer no longer injects the initContainer
#5484 Inconsistent data when consulting jobs/instances
#5464 Add documentation for launching multiple versions of a task.
#5408 Error during "Clean up all task/job executions"

Update

Kubernetes Deployment scripts updated.

Spring Cloud Data Flow 2.11.1

Notable changes in 2.11.1

• Updated versions and mitigations for CVEs.
  • json-smart: 2.4.11
  • nimbus-jose-jwt: 9.31
  • snappy-java: 1.1.10.4
  • commons-compress: 1.24.0
• Ensure TaskOperations launch API is backwards compatible.
• Fix issues in dependency managements of modules after creating mono-repo

Dependent Projects and Compatibility

Component	Version
Spring Cloud Dataflow Build	2.11.1
Spring Cloud Dataflow Common	2.11.1
Spring Cloud Deployer	2.9.1
Spring Cloud Deployer Local	2.9.1
Spring Cloud Deployer CF	2.9.1
Spring Cloud Deployer K8S	2.9.1
Spring Cloud Common Security Config	2.11.1
Spring Cloud Skipper	2.11.1
Spring Cloud Dataflow UI	3.4.1
Spring Cloud Dataflow	2.11.1

Issues

#5492 TaskOperations launch API is not backwards compatible
#5488 Update dependencies for latest CVEs
#5485 Add common-security-config modules to dependency management

Installer

The scripts for installing the Carvel package on Kubernetes and local development is attached below

Spring Cloud Data Flow 2.11.0

Noteworthy

Spring Boot 3 Applications

• Adds support for Spring Boot 3.x based stream applications.
• Adds support for Spring Cloud Task 3.x based task applications.
• Adds support for Spring Batch 5.x based batch applications.

More info can be found in the Boot 3 Appendix.

Library Updates

• Spring Boot 2.7.15
• Spring Cloud 2021.0.8

Kubernetes Update

• The long-awaited update of Kubernetes cronjobs from batch/v1beta -> batch/v1 which allows users to run Kubernetes >= 1.25.0.

Dependent Projects and Compatibility

Component	Version
Spring Cloud Dataflow Build	2.11.0
Spring Cloud Dataflow Common	2.11.0
Spring Cloud Deployer	2.9.0
Spring Cloud Deployer Local	2.9.0
Spring Cloud Deployer CF	2.9.0
Spring Cloud Deployer K8S	2.9.0
Spring Cloud Common Security Config	2.11.0
Spring Cloud Skipper	2.11.0
Spring Cloud Dataflow UI	3.4.0
Spring Cloud Dataflow	2.11.0

Issues

spring-cloud/spring-cloud-deployer#399 Duplicate Kubernetes pod ids when tasks are launched at the same time
spring-cloud/spring-cloud-dataflow-ui#1947 App registration details panel not updating when switch between boot versions
spring-cloud/spring-cloud-dataflow-ui#1946 Spring Boot 3 / Batch 5: Job execution is empty although shown in DB
#5467 Default version is always used to pick the table prefixes
#5459 The task configured as boot2 application cannot be executed in SCDF 2.11.0 RC1.
#5456 Error Value too long for column "AUDIT_DATA CHARACTER VARYING(4000)
#5452 Sensitive data still displayed in audit dashboard
#5450 As a user I can easily tell that this repo is no longer active and now lives in the mono repo.
#5446 Add tests to the spring-cloud-dataflow-aggregate-task module
#5443 [Docs] Missing task 3.x coordinates in ref guide
#5442 Add num days option to task cleanup
#5441 SCDF 2.11.0-RC1 composed task runner documentation
#5439 LOB-Elements broken after migration with PostgreSQLTextToOID

Spring Cloud Data Flow 2.10.3

Notable changes

Library updates

• Spring Boot 2.7.11

Dependent Projects and Compatibility

Component	Version
Spring Cloud Dataflow Build	2.10.3
Spring Cloud Dataflow Common	2.10.3
Spring Cloud Deployer	2.8.3
Spring Cloud Deployer Local	2.8.3
Spring Cloud Deployer CF	2.8.3
Spring Cloud Deployer K8S	2.8.3
Spring Cloud Common Security Config	1.8.3
Spring Cloud Skipper	2.9.3
Spring Cloud Dataflow UI	3.3.3
Spring Cloud Dataflow	2.10.3

CVE Fixes

#5329 Fix code scanning alert - Constructor Deserialization Remote Code Execution
#5303 Security issues in transitive dependency of spring-security-oauth2-client-5.4.2
#5302 Security issues in transitive dependency of spring-expression-5.2.11
#5301 Security issues in transitive dependency of spring-webmvc-5.3.25
#5300 Security issues in transitive dependency of json-smart-2.3
#5299 Security issues in transitive dependency of jettison-1.51
#5248 Misalignment of spring-security-oauth2-client versions in spring-cloud-dataflow-server module

Issues

#5325 Switch maven repos off of libs-xyz variants
#5321 default spring repo at local scdf is not available
#5317 Unregistering multiple applications doesn't work while unregistering single application works.
#5297 [2.10.x] Documentation cleanup
#5252 Simplify Carvel packages (Part 1)
#4791 Hibernate statistics are not exposed even if corresponding configuration are applied
spring-cloud/spring-cloud-deployer#381 (Backport) Respect initContainer from global properties
spring-cloud/spring-cloud-deployer#386 (Backport) Add pod-level 'terminationGracePeriodSeconds'

Spring Cloud Data Flow 2.10.2

Notable changes

Library updates

• Spring Boot 2.7.9
• Spring Cloud 2021.0.6

Dependent Projects and Compatibility

Component	Version
Spring Cloud Dataflow Build	2.10.2
Spring Cloud Dataflow Common	2.10.2
Spring Cloud Deployer	2.8.2
Spring Cloud Deployer Local	2.8.2
Spring Cloud Deployer CF	2.8.2
Spring Cloud Deployer K8S	2.8.2
Spring Cloud Common Security Config	1.8.2
Spring Cloud Skipper	2.9.2
Spring Cloud Dataflow UI	3.3.2
Spring Cloud Dataflow	2.10.2

Issues

#5232 Changed default repo.spring.io/snapshot to repo.spring.io/libs-snapshot
#5207 Removed invalid deprecation on StepExecutionHistory.getReadCount
#5246 Fixed by #5232

Spring Cloud Data Flow 2.10.1

Notable changes

Library updates

• Spring Boot 2.7.8
• Spring Framework 5.3.25
• Spring Shell 2.1.5

Kubernetes

• Added ability to fully configure pod and container security contexts.
• Container security context is propagated to init container and additional containers.

Dependent Projects and Compatibility

Component	Version
Spring Cloud Dataflow Build	2.10.1
Spring Cloud Dataflow Common	2.10.1
Spring Cloud Deployer	2.8.1
Spring Cloud Deployer Local	2.8.1
Spring Cloud Deployer CF	2.8.1
Spring Cloud Deployer K8S	2.8.1
Spring Cloud Common Security Config	1.8.1
Spring Cloud Skipper	2.9.1
Spring Cloud Dataflow UI	3.3.1
Spring Cloud Dataflow	2.10.1

Issues

spring-cloud/spring-cloud-dataflow-build#95 Update spring-shell 2.1.5
spring-cloud/spring-cloud-deployer-local#219 maxConcurrentExecutionsReached() has a race-condition where multiple instances can start before it returns a result
spring-cloud/spring-cloud-deployer-kubernetes#512 Add support to configure SecurityContext/allowPrivilegeEscalation for the deployed containers
spring-cloud/spring-cloud-dataflow-ui#1904 Task executions status label
#5200 Clarify docs on extending classpath
#5194 Remove workaround for missing pivotal-cloudfoundry-client-reactor
#5193 Remove all uses of repo.spring.io/release (main)
#5191 Update to Spring Boot 2.7.8
#5188 Flyway error migrating from SCDF 2.9.5 to 2.10.0
#5187 AppRegistryCommands missing ApplicationType completion
#5184 As user I want to set runAsGroup property in podSecurityContext
#5183 Task execustion view report wrong status
#5174 dataflow.spring.io Batch related documentation issues
#5172 spring-cloud-dataflow-shell fails to parse kebab-case arguments
#5171 Misalignment of spring-security-oauth2-client versions in spring-cloud-dataflow-server module
#5170 Spring Batch run in K8S pod in SCDF fails to load MariaDB driver class
#5164 MySQL 5.7 not working w/ SCDF 2.10 when using config server
#5163 Documentation for 2.10.0 local SCDF still points to 2.10.0-SNAPSHOT for manual downloads
#4971 Deprecate Count attribute for Step History

Spring Cloud Data Flow 2.10.0

Notable changes

Spring updates

Updates to the following versions:

• Spring Boot 2.7.6
• Spring Framework 5.3.24
• Spring Cloud 2021.0.5

Spring Shell

Spring Shell has been updated to 2.1.4.

Documentation Updates

Container images for ARM.

Spring Cloud Data Flow and Skipper, and Stream Application projects contain scripts to use in creating containers when running on an ARM platform. The Reference Guide has a section describing the use of the scripts.

Scripts for local Kuberenetes Development

We have added a collection of scripts to the Spring Cloud Data Flow that makes it easy to launch a local Kuberenetes cluster and install SCDF with MariaDB and RabbitMQ or Kafka. See the Reference Guide for more info.

Dependent Projects and Compatibility

Component	Version
Spring Cloud Dataflow Build	2.10.0
Spring Cloud Dataflow Common	2.10.0
Spring Cloud Deployer	2.8.0
Spring Cloud Deployer Local	2.8.0
Spring Cloud Deployer CF	2.8.0
Spring Cloud Deployer K8S	2.8.0
Spring Cloud Common Security Config	1.8.0
Spring Cloud Skipper	2.9.0
Spring Cloud Dataflow UI	3.3.0
Spring Cloud Dataflow	2.10.0

Issues

spring-cloud/spring-cloud-dataflow-build#93 Update spring-shell 2.1.4
spring-cloud/spring-cloud-deployer-kubernetes#503 Improve documentation on kuberenetes deployment properites
spring-cloud/spring-cloud-deployer-kubernetes#494 Support for concurrency policy of a Job Schedule
spring-cloud/spring-cloud-skipper#1068 Ensure reference doc is up-to-date and valid
#5161 Shell fails with stream deploy using properties
#5151 Review outstanding issues for dataflow.spring.io
#5113 Obtain OSM license for SCDF PRO
#5108 Support for TAS 3.0
#5083 Communicate best practice for extending the classpath of Dataflow/Skipper
#4946 Revisit shell changes
#4944 Arm64 docker images for Apple M1 usage
#4830 spring cloud dataflow launch task on other the kubernetes cluster
#4807 Ensure reference doc is up-to-date and valid
#4792 Update K8s compatibility in test-plan and docs
#4780 Add to docker-compose install docs info on env-vars and ssl.
#4779 Issue with MariaDB when using skipper-deployment.yaml