Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task/jenkins 42120 git creation credential changes #895

Merged
merged 40 commits into from
Mar 16, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
1c2e937
JENKINS-41397# validate git credential
Jan 25, 2017
0b207f7
Field specific error message
Jan 27, 2017
f029779
Merge remote-tracking branch 'origin/master' into task/JENKINS-41397
Jan 31, 2017
b0fb5dc
bug fixes
Jan 31, 2017
8dc82e1
credential validation is now 400 errors.
Jan 31, 2017
19d8621
Merge remote-tracking branch 'origin/master' into task/JENKINS-41397
Feb 14, 2017
718b731
Merge remote-tracking branch 'origin/master' into task/JENKINS-41397
Feb 14, 2017
6c11a94
[JENKINS-42120] fix a bug where some content from the FlowSteps (inac…
Mar 2, 2017
82821ab
[JENKINS-42120] rework Git creation flow to use modal for creating ne…
Mar 2, 2017
23d5978
Merge branch 'master' into task/JENKINS-42120-git-creation-credential…
Mar 6, 2017
954a68e
[JENKINS-42120] remove "System SSH" as an option
Mar 6, 2017
236b875
[JENKINS-42120] width of username / password inputs
Mar 6, 2017
8e9304b
[JENKINS-42120] remove obsolete code for credential type in "Connect"…
Mar 6, 2017
485d986
Merge branch 'task/JENKINS-41397' into task/JENKINS-42120-git-creatio…
Mar 6, 2017
830a0fb
[JENKINS-42120] add a step to Git creation flow that can display unha…
Mar 6, 2017
867f127
[JENKINS-42120] reworking Git creation to support different validatio…
Mar 6, 2017
9e9f778
[JENKINS-42120] fix bug where credentialId was not sent correctly; ad…
Mar 6, 2017
e852705
Merge remote-tracking branch 'origin/master' into task/JENKINS-41397
Mar 7, 2017
43bbde0
Use StandardCredentials in case of git.
Mar 7, 2017
699e1e8
List all credentials that user has access to.
Mar 7, 2017
eac6262
Merge branch 'task/JENKINS-41397' into task/JENKINS-42120-git-creatio…
Mar 8, 2017
3b49071
[JENKINS-42120] fix a ridiculous bug where credentials were being cre…
Mar 8, 2017
ada0ffd
[JENKINS-42120] add an API to remove all steps after the specified step
Mar 9, 2017
e0aad62
[JENKINS-42120] transform Git TransportException "authentication is r…
Mar 9, 2017
dae30b2
[JENKINS-42120] improve the error handling for Git creation, respondi…
Mar 9, 2017
f608ec8
[JENKINS-42120] delint
Mar 9, 2017
594b8a2
[JENKINS-42120] fix broken test
Mar 9, 2017
a8ceb12
Frogot to remove StandardCredentials.
vivek Mar 10, 2017
c8a986c
Merge branch 'task/JENKINS-41397' into task/JENKINS-42120-git-creatio…
Mar 10, 2017
393a791
[JENKINS-42120] use beta JDL with dropdown fixes and enhancements; re…
Mar 13, 2017
8614e64
[JENKINS-42120] fix a bug where selecting a credential would not allo…
Mar 13, 2017
9b95dde
[JENKINS-42120] adjust backend to transform the SSH "connection is no…
Mar 13, 2017
c7250b2
[JENKINS-42120] italicize the special "none" credential
Mar 14, 2017
3da646d
[JENKINS-42120] change Git creation UI to expose "System Default" opt…
Mar 15, 2017
fe568b1
[JENKINS-42120] use JDL that includes useful Dropdown fixes but inclu…
Mar 15, 2017
dfdb8fc
Merge branch 'master' into task/JENKINS-42120-git-creation-credential…
Mar 16, 2017
66ff520
release core-js 0.0.89
Mar 16, 2017
e606c7f
bump core-js version to 0.0.90-SNAPSHOT
Mar 16, 2017
b59388c
[JENKINS-42120] use latest JDL (and core-js, since we have to)
Mar 16, 2017
988d4d0
Merge branch 'master' into task/JENKINS-42120-git-creation-credential…
Mar 16, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
bug fixes
- Don't create org folder if there was error
- collect various 400 errors and return in same response
  • Loading branch information
Vivek Pandey authored and Vivek Pandey committed Jan 31, 2017
commit b0fb5dc58661358dc081dfffa2c94fe3daa4aa4f
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import javax.annotation.Nonnull;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;

/**
Expand Down Expand Up @@ -42,6 +43,12 @@ public ErrorMessage add(Error error){
return this;
}

@JsonIgnore
public ErrorMessage addAll(Collection<Error> errors){
this.errors.addAll(errors);
return this;
}

@JsonProperty("errors")
public List<Error> getErrors(){
return errors;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
package io.jenkins.blueocean.blueocean_git_pipeline;

import com.cloudbees.plugins.credentials.common.StandardUsernameCredentials;
import hudson.model.Cause;
import hudson.model.Failure;
import hudson.model.TopLevelItem;
import hudson.plugins.git.GitException;
import io.jenkins.blueocean.commons.ErrorMessage;
import io.jenkins.blueocean.commons.ServiceException;
import io.jenkins.blueocean.rest.Reachable;
Expand All @@ -15,26 +18,27 @@
import jenkins.plugins.git.GitSCMSource;
import org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject;
import org.kohsuke.stapler.DataBoundConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

/**
* @author Vivek Pandey
*/
public class GitPipelineCreateRequest extends AbstractPipelineCreateRequestImpl {

private static final String MODE = "org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject";

private static final Logger logger = LoggerFactory.getLogger(GitPipelineCreateRequest.class);

private BlueScmConfig scmConfig;

@DataBoundConstructor
public GitPipelineCreateRequest(String name, BlueScmConfig scmConfig) {
validate(name, scmConfig);
setName(name);
if(scmConfig == null){
throw new ServiceException.BadRequestExpception(new ErrorMessage(400, "Failed to create Git pipeline:"+name)
.add(new ErrorMessage.Error("scmConfig", ErrorMessage.Error.ErrorCodes.MISSING.toString(), "scmConfig is required")));
}
this.scmConfig = scmConfig;
}

Expand All @@ -43,16 +47,10 @@ public BluePipeline create(Reachable parent) throws IOException {

String sourceUri = scmConfig.getUri();

if (sourceUri == null) {
throw new ServiceException.BadRequestExpception(new ErrorMessage(400, "Failed to create Git pipeline:"+getName())
.add(new ErrorMessage.Error("scmConfig.uri", ErrorMessage.Error.ErrorCodes.MISSING.toString(), "uri is required")));
}

TopLevelItem item = create(Jenkins.getInstance(), getName(), MODE, MultiBranchProjectDescriptor.class);

if (item instanceof WorkflowMultiBranchProject) {
WorkflowMultiBranchProject project = (WorkflowMultiBranchProject) item;
GitUtils.validateCredentials(project, sourceUri, scmConfig.getCredentialId());

//XXX: set credentialId to empty string if null or we get NPE later on
String credentialId = scmConfig.getCredentialId() == null ? "" : scmConfig.getCredentialId();
Expand All @@ -70,4 +68,51 @@ public BluePipeline create(Reachable parent) throws IOException {
return null;
}

private void validate(String name, BlueScmConfig scmConfig){
if(scmConfig == null){
throw new ServiceException.BadRequestExpception(new ErrorMessage(400, "Failed to create Git pipeline")
.add(new ErrorMessage.Error("scmConfig", ErrorMessage.Error.ErrorCodes.MISSING.toString(), "scmConfig is required")));
}

List<ErrorMessage.Error> errors = new ArrayList<>();

String sourceUri = scmConfig.getUri();

if (sourceUri == null) {
errors.add(new ErrorMessage.Error("scmConfig.uri", ErrorMessage.Error.ErrorCodes.MISSING.toString(), "uri is required"));
}else {
try {
StandardUsernameCredentials credentials = null;
if(scmConfig.getCredentialId() != null){
credentials = GitUtils.getCredentials(Jenkins.getInstance(), sourceUri, scmConfig.getCredentialId());
if (credentials == null) {
errors.add(new ErrorMessage.Error("scmConfig.credentialId",
ErrorMessage.Error.ErrorCodes.NOT_FOUND.toString(),
String.format("credentialId: %s not found", scmConfig.getCredentialId())));
}
}
GitUtils.validateCredentials(sourceUri, credentials);
} catch (GitException e) {
logger.error("Error validating credential: " + e.getMessage(), e);
errors.add(new ErrorMessage.Error("scmConfig.uri", ErrorMessage.Error.ErrorCodes.INVALID.toString(),
e.getMessage()));
}
}

try {
Jenkins.getInstance().getProjectNamingStrategy().checkName(getName());
}catch (Failure f){
errors.add(new ErrorMessage.Error("scmConfig.name", ErrorMessage.Error.ErrorCodes.INVALID.toString(), getName() + "in not a valid name"));
}

if(Jenkins.getInstance().getItem(name)!=null) {
errors.add(new ErrorMessage.Error("name",
ErrorMessage.Error.ErrorCodes.ALREADY_EXISTS.toString(), name + " already exists"));
}

if(!errors.isEmpty()){
throw new ServiceException.BadRequestExpception(new ErrorMessage(400, "Failed to create Git pipeline:"+getName()).addAll(errors));
}
}

}
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
package io.jenkins.blueocean.blueocean_git_pipeline;

import com.cloudbees.plugins.credentials.common.StandardUsernameCredentials;
import hudson.model.Cause;
import hudson.model.CauseAction;
import hudson.model.Item;
import hudson.plugins.git.GitException;
import hudson.security.ACL;
import hudson.util.PersistedList;
import io.jenkins.blueocean.commons.ErrorMessage;
import io.jenkins.blueocean.commons.ServiceException;
import io.jenkins.blueocean.rest.model.BluePipeline;
import io.jenkins.blueocean.rest.model.BluePipelineUpdateRequest;
Expand All @@ -15,15 +18,21 @@
import jenkins.plugins.git.GitSCMSource;
import org.acegisecurity.Authentication;
import org.kohsuke.stapler.DataBoundConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.annotation.CheckForNull;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;

/**
* @author Vivek Pandey
*/
public class GitPipelineUpdateRequest extends BluePipelineUpdateRequest {
private static final Logger logger = LoggerFactory.getLogger(GitPipelineUpdateRequest.class);

private final BlueScmConfig scmConfig;

@DataBoundConstructor
Expand Down Expand Up @@ -64,10 +73,34 @@ private BranchSource getGitScmSource(MultiBranchProject mbp){

if(scmConfig != null){
sourceUri = scmConfig.getUri();
List<ErrorMessage.Error> errors = new ArrayList<>();

StandardUsernameCredentials credentials = null;
if(scmConfig.getCredentialId() != null){
credentials = GitUtils.getCredentials(Jenkins.getInstance(), sourceUri, scmConfig.getCredentialId());
if (credentials == null) {
errors.add(new ErrorMessage.Error("scmConfig.credentialId",
ErrorMessage.Error.ErrorCodes.NOT_FOUND.toString(),
String.format("credentialId: %s not found", scmConfig.getCredentialId())));
}
}

if(sourceUri != null) {
GitUtils.validateCredentials(mbp, sourceUri, scmConfig.getCredentialId());
try {
GitUtils.validateCredentials(sourceUri, credentials);
}catch (GitException e){
logger.error("Error validating git request: "+e.getMessage(), e);
errors.add(new ErrorMessage.Error("scmConfig.uri", ErrorMessage.Error.ErrorCodes.INVALID.toString(),
e.getMessage()));
}
}
credentialId = scmConfig.getCredentialId() == null ? "" : scmConfig.getCredentialId();

if (!errors.isEmpty()){
throw new ServiceException.BadRequestExpception(new ErrorMessage(400, "Failed to create Git pipeline")
.addAll(errors));

}
}

PersistedList<BranchSource> sources = mbp.getSourcesList();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,16 @@
import com.cloudbees.plugins.credentials.common.StandardUsernameCredentials;
import com.cloudbees.plugins.credentials.domains.URIRequirementBuilder;
import hudson.EnvVars;
import hudson.model.ItemGroup;
import hudson.model.TaskListener;
import hudson.plugins.git.GitException;
import hudson.security.ACL;
import io.jenkins.blueocean.commons.ErrorMessage;
import io.jenkins.blueocean.commons.ServiceException;
import jenkins.scm.api.SCMSourceOwner;
import org.jenkinsci.plugins.gitclient.Git;
import org.jenkinsci.plugins.gitclient.GitClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.annotation.Nonnull;
import javax.annotation.Nullable;
Expand All @@ -22,24 +24,15 @@
* @author Vivek Pandey
*/
class GitUtils {
private static final Logger logger = LoggerFactory.getLogger(GitUtils.class);

/**
* Calls 'git ls-remote -h uri' to check if git uri or supplied credentials are valid
*
* @param owner SCM owner, such as MultiBranchProject
* @param uri git repo uri
* @param credentialId credential id to use when accessing git
* @param credentials credential to use when accessing git
*/
static void validateCredentials(@Nonnull SCMSourceOwner owner, @Nonnull String uri, @Nullable String credentialId){
StandardUsernameCredentials credentials = null;
if(credentialId != null) {
credentials = getCredentials(owner, uri, credentialId);
if (credentials == null) {
throw new ServiceException.BadRequestExpception(new ErrorMessage(400, "Failed to create Git pipeline")
.add(new ErrorMessage.Error("scmConfig.credentialId",
ErrorMessage.Error.ErrorCodes.NOT_FOUND.toString(),
String.format("credentialId: %s not found", credentialId))));
}
}
static void validateCredentials(@Nonnull String uri, @Nullable StandardUsernameCredentials credentials) throws GitException{
Git git = new Git(TaskListener.NULL, new EnvVars());
try {
GitClient gitClient = git.getClient();
Expand All @@ -48,21 +41,30 @@ static void validateCredentials(@Nonnull SCMSourceOwner owner, @Nonnull String u
}
gitClient.getRemoteReferences(uri,null, true,false);
} catch (IOException | InterruptedException e) {
logger.error("Error running git remote-ls: " + e.getMessage(), e);
throw new ServiceException.UnexpectedErrorException("Failed to create pipeline due to unexpected error: "+e.getMessage(), e);
} catch (GitException e){
throw new ServiceException.BadRequestExpception(new ErrorMessage(400, "Failed to create Git pipeline")
.add(new ErrorMessage.Error("scmConfig.uri",
ErrorMessage.Error.ErrorCodes.INVALID.toString(),
"Invalid uri: " + uri)), e);
} catch (IllegalStateException e){
throw new ServiceException.ForbiddenException(new ErrorMessage(403, "Failed to create Git pipeline")
.add(new ErrorMessage.Error("scmConfig.credentialId",
ErrorMessage.Error.ErrorCodes.INVALID.toString(),
"Invalid credentialId: " + credentialId)), e);
} catch (IllegalStateException | GitException e){
logger.error("Error running git remote-ls: " + e.getMessage(), e);
if(credentials != null) {
// XXX: check for 'not authorized' is hack. Git plugin API (org.eclipse.jgit.transport.TransportHttp.connect())does not send
// back any error code so that we can distinguish between unauthorized vs bad url or some other type of errors.
// Where org.eclipse.jgit.transport.SshTransport.connect() throws IllegalStateException in case of unauthorized,
// org.eclipse.jgit.transport.HttpTransport.connect() throws TransportException with error code 'not authorized'
// appended to the message.
if(e instanceof IllegalStateException || e.getMessage().endsWith("not authorized")){
throw new ServiceException.ForbiddenException(new ErrorMessage(403, "Failed to create Git pipeline")
.add(new ErrorMessage.Error("scmConfig.credentialId",
ErrorMessage.Error.ErrorCodes.INVALID.toString(),
"Invalid credentialId: " + credentials.getId())), e);
}
throw e; // throw GitException so that later on it can be reported as 400 error
} else{
throw new GitException(e);
}
}
}

private static StandardUsernameCredentials getCredentials(SCMSourceOwner owner, String uri, String credentialId){
static StandardUsernameCredentials getCredentials(ItemGroup owner, String uri, String credentialId){
return CredentialsMatchers
.firstOrNull(
CredentialsProvider.lookupCredentials(StandardUsernameCredentials.class, owner,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,7 @@
import java.util.List;
import java.util.Map;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.*;

/**
* @author Vivek Pandey
Expand Down Expand Up @@ -240,6 +238,7 @@ public void shouldFailOnValidation2(){

assertEquals(errors.get(0).get("field"), "scmConfig");
assertEquals(errors.get(0).get("code"), "MISSING");
assertNull(Jenkins.getInstance().getItem("demo"));
}

@Test
Expand All @@ -255,6 +254,8 @@ public void shouldFailOnValidation3(){

assertEquals(errors.get(0).get("field"), "scmConfig.uri");
assertEquals(errors.get(0).get("code"), "MISSING");
assertNull(Jenkins.getInstance().getItem("demo"));

}


Expand All @@ -273,13 +274,38 @@ public void shouldFailOnValidation4(){
resp = post("/organizations/jenkins/pipelines/",
ImmutableMap.of("name", "demo",
"$class", "io.jenkins.blueocean.blueocean_git_pipeline.GitPipelineCreateRequest",
"scmConfig", ImmutableMap.of("uri", sampleRepo.fileUrl())
"scmConfig", ImmutableMap.of("uri", sampleRepo.fileUrl(), "credentialId", "sdsdsd")
), 400);
List<Map> errors = (List<Map>) resp.get("errors");
List<Map<String,String>> errors = (List<Map<String,String>>) resp.get("errors");

boolean nameFound = false;
boolean credentialIdFound = false;
for(Map<String,String> error:errors){
if(error.get("field").equals("name")){
nameFound = true;
assertEquals(error.get("code"), "ALREADY_EXISTS");
}else if(error.get("field").equals("scmConfig.credentialId")){
credentialIdFound = true;
assertEquals(error.get("code"), "NOT_FOUND");
}
}
assertTrue(nameFound);
assertTrue(credentialIdFound);
}

assertEquals(errors.get(0).get("field"), "name");
assertEquals(errors.get(0).get("code"), "ALREADY_EXISTS");
@Test
public void shouldFailOnValidation5(){

Map<String,Object> resp = post("/organizations/jenkins/pipelines/",
ImmutableMap.of("name", "demo",
"$class", "io.jenkins.blueocean.blueocean_git_pipeline.GitPipelineCreateRequest",
"scmConfig", ImmutableMap.of("uri", sampleRepo.fileUrl(), "credentialId", "sdsdsd")
), 400);
List<Map<String,String>> errors = (List<Map<String,String>>) resp.get("errors");

assertEquals("scmConfig.credentialId", errors.get(0).get("field"));
assertEquals("NOT_FOUND", errors.get(0).get("code"));
assertNull(Jenkins.getInstance().getItem("demo"));
}

private String createMbp(){
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,10 @@ static void validateCredentialId(String credentialId, OrganizationFolder item, G
try {
item.delete();
} catch (InterruptedException e) {
throw new ServiceException.UnexpectedErrorException("Invalid credentialId: " + credentialId + ". Failure during cleaing up folder: " + item.getName() + ". Error: " + e.getMessage(), e);
throw new ServiceException.UnexpectedErrorException("No Credentials instance found for credentialId: " + credentialId + ". Failure during cleaing up folder: " + item.getName() + ". Error: " + e.getMessage(), e);
}
throw new ServiceException.BadRequestExpception(new ErrorMessage(400, "Failed to create Git pipeline")
.add(new ErrorMessage.Error("credentialId", ErrorMessage.Error.ErrorCodes.INVALID.toString(), "Invalid credentialId")));
.add(new ErrorMessage.Error("credentialId", ErrorMessage.Error.ErrorCodes.NOT_FOUND.toString(), "No Credentials instance found for credentialId: "+credentialId)));

}
}
Expand Down
Loading