Skip to content

Pull Job scheduler from maven instead of jenkins #663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ps48 merged 3 commits into from
Mar 7, 2023
Merged

Pull Job scheduler from maven instead of jenkins #663

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1d23ee7
use maven
derek-ho Feb 28, 2023
d33fa4e
bwc version to 2.7.0 bump
derek-ho Feb 28, 2023
c59a017
hard code zips
derek-ho Mar 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 7 additions & 23 deletions build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ configurations {
ktlint
testCompile
testRuntime
zipArchive
}

detekt {
Expand Down Expand Up @@ -167,6 +168,7 @@ repositories {
}

dependencies {
zipArchive group: 'org.opensearch.plugin', name:'opensearch-job-scheduler', version: "${opensearch_build}"
implementation "org.opensearch:opensearch:${opensearch_version}"
implementation "org.jetbrains.kotlin:kotlin-stdlib:${kotlin_version}"
implementation "org.jetbrains.kotlin:kotlin-stdlib-common:${kotlin_version}"
Expand Down Expand Up @@ -292,8 +294,6 @@ Zip bundle = (Zip) project.getTasks().getByName("bundlePlugin");
integTest.dependsOn(bundle)
integTest.getClusters().forEach{c -> c.plugin(project.getObjects().fileProperty().value(bundle.getArchiveFile()))}

String jobSchedulerURL = "https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/" + opensearch_version.replace("-SNAPSHOT", "") + "/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-job-scheduler-" + opensearch_build.replace("-SNAPSHOT", "") + ".zip"

testClusters.integTest {
testDistribution = "INTEG_TEST"
// need to install job-scheduler first, need to assemble job-scheduler first
Expand All @@ -303,15 +303,7 @@ testClusters.integTest {
return new RegularFile() {
@Override
File getAsFile() {
File dir = new File("src/test/resources/job-scheduler")
if (!dir.exists()) {
dir.mkdirs()
}
File file = new File(dir, "opensearch-job-scheduler-" + opensearch_build + ".zip")
if (!file.exists()) {
new URL(jobSchedulerURL).withInputStream{ ins -> file.withOutputStream{ it << ins }}
}
return fileTree("src/test/resources/job-scheduler").getSingleFile()
return configurations.zipArchive.asFileTree.getSingleFile()
}
}
}
Expand All @@ -333,12 +325,12 @@ testClusters.integTest {
}

// For job-scheduler and reports-scheduler, the latest opensearch releases appear to be 1.1.0.0.
String baseVersion = "2.6.0"
String baseVersion = "2.7.0"
String bwcVersion = baseVersion + ".0"
String baseName = "reportsSchedulerBwcCluster"
String bwcFilePath = "src/test/resources/bwc"
String bwcJobSchedulerURL = "https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/${baseVersion}/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-job-scheduler-${bwcVersion}.zip"
String bwcReportsSchedulerURL = "https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/${baseVersion}/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-reports-scheduler-${bwcVersion}.zip"
String bwcJobSchedulerURL = "https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0-SNAPSHOT/opensearch-job-scheduler-2.7.0.0-20230305.020625-11.zip"
String bwcReportsSchedulerURL = "https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/opensearch-reports-scheduler/2.7.0.0-SNAPSHOT/opensearch-reports-scheduler-2.7.0.0-20230306.195720-1.zip"
Comment on lines +332 to +333
Copy link
Collaborator Author

@derek-ho derek-ho Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#671

Copy link
Member

@joshuali925 joshuali925 Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a known solution to fix 671?

Copy link
Collaborator Author

@derek-ho derek-ho Mar 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably. We just need to differentiate/store it in different path and then retrieve it appropriately


2.times {i ->
testClusters {
Expand Down Expand Up @@ -403,15 +395,7 @@ task prepareBwcTests {
return new RegularFile() {
@Override
File getAsFile() {
File dir = new File(bwcFilePath + "/job-scheduler/" + project.version)
if (!dir.exists()) {
dir.mkdirs()
}
File file = new File(dir, "opensearch-reports-scheduler-" + project.version + ".zip")
if (!file.exists()) {
new URL(jobSchedulerURL).withInputStream{ ins -> file.withOutputStream{ it << ins }}
}
return fileTree(bwcFilePath + "/job-scheduler/" + project.version).getSingleFile()
return configurations.zipArchive.asFileTree.getSingleFile()
}
}
}
Expand Down