1517 discovery of template sets (#1585)

* First implementation of settings.xml mapping

* Completed mapping of the needed components, refactoring, added JavaDocs, added log output

* Added test class, removed unnecessary code

* #1530 added maven settings password decryption
added org.codehaus.plexus sec-dispatcher dependency
added org.codehaus.plexus cipher dependency
added MavenSettingsUtil
added MavenSettingsUtilTest and resources

* #1530 adjusted test method name

* Added tests, did refactoring

* Added missing JavaDocs, tests are now using asserThat(), more improvements addressing change requests on the pull requests

* #1530 added basic authentication type
replaced authToken with password
added basic username and password authentication

* #1530 renamed REST search API constants

* #1530 handle template-set.xml
made sure that only template-set.xml files get returned as download links
added new template-set.xml to ConfigurationConstants
added a valid template-set.xml example to test resource files
removed extra outputs (non template-set.xml) from test assertions
added classifier to Nexus2Response
renamed retrieveDownloadUrls to retrieveTemplateSetXmlDownloadURLs

* #1530 added some fixes
fixed credentials not being used
fixed too many httpclient calls being used
added new test for a failed basic authentication
added new test for a failed token authentication
enabled bad url test

* #1530 cleaned up exceptions
removed JsonMappingException because it is already being handled by JsonProcessingException

* Switched from javax to jakarta, resolved comments from the pull request

* Tidied up code

* Changed path generation

* Added unmarshalling for mirrors, added loading of active profiles, added preperation of settings.xml

* added function, which returns repositories of all active profiles

* #1530 improved log messages
adjusted log messages (converted to debug, made factory process messages more transparent)
converted RestSearchResponseException status code from string to integer

* Added utils to operate with mirrors specified in maven's settings.xml

* Optimized MavenSettingsUtilTest, added first code of MavenMirrorUtilTest

* Fixed bugs, added tests for matchPattern

* Completed MavenMirrorUtil class, completed tests

* determined active profiles, added method to determine repositories of active profiles and inject mirrors in repository urls

* Minor optimisations

* Added collection of servers private key and passphrase

* Refactoring of classes in a new maven module

* #1530 added exception messages to constants

* #1530 adjusted exception handling
replaced exception strings with constants
added retrieveRestSearchApiTargetLink method
adjusted SearchResponseException with dynamic message
adjusted tests to check for exception types and message content

* Fixed merge bugs

* #1517 Refactoring
Refactored changes from #1530 into #1517
Moved #1530 files into MavenSearch package

* #1517 Added proxy to http client

* #1517 added eclipse project files

* Removed MavenSettingsUtil

* #1517 changed the process of getting active profiles
added artifact retriever
removed unsecessary code

* #1517 added multiple tests

* #1517 improved WireMock log messages
moved logback-test.xml from core-api to core-artifact-retriever

* #1517 introduced server credentials
moved all server credentials into new object
added new MavenSearchArtifactRetriever class
added new ServerCredentials class
adjusted tests to utilize server credentials
deleted MavenArtifactsUtil

* #1517 Added proxy util and tests

* #1517 removed exceptions
replaced exceptions with debug log messages
adjusted tests

* #1517 Reworked MavenProxyUtil, updated tests

* #1517 replaced more exceptions with log messages
removed obsolete retrieveMavenArtifactsByGroupId method

* #1517 implemented requested changes
converted null returns to empty lists
added new debug message to indicate that the search is proceeding
converted MavenSearchArtifactRetriever to static class
added more tests
adjusted test descriptions

* #1517 Combined ArtifactRetriever with MavenSearch
added artifactRetriever logic
prepared artifactRetriever tests

* #1517 ArtifactRetriever test preperation

* #1517 added first ArtifactRetriever tests
added example template-set.xml
added example json response for WireMock
added WireMock tests for proxy and non proxy connections as well as basic authentication
added more log messages for proxy usage and basic authentication usage
fixed NPE in MavenSettingsUtil and ArtifactRetriever

* #1517 added proxy server authentication
added basic authentication to proxy connection
added tests for proxy authentication
added proxyUsername and proxyPassword to ServerCredentials

* #1517 split test classes
split test classes of maven search

* #1517 re-organized test resources
simplified json test resource files for MavenSearchResponse tests

* #1517 fixed ArtifactRetriever proxy test
added 2nd WireMock server
adjusted settings.xml to use proxy properly

* #1517 Added ArtivactRetrieverReader

Co-authored-by: Leonie leonie-heidi.schuessler@capgemini.com

* Removed unused test

* #1517 removed util package from mavensearch

* #1517 optimizations
fixed typo
moved file reader from test to generateMavenTemplateSetConfiguration method
adjusted javadocs
added more debug information

* #1517 renamed reader models

* #1517 removed util package from mavensearch

* #1517 added template set version and tests
added retrieveTemplateSetData method (converts a list of template set files to TemplateSetArtifactReader)
added more tests and test resources
converted TemplateSetArtifactReader to stateful class
added simple template set version retrieval
adjusted tests
removed tag names and increment descriptions from TemplateSetArtifactReader
removed setupClass method from TemplateSetArtifactReaderTest (moved initialization to each test)

* #1517 renamed MavenReaderTest resource folder
renamed MavenReaderTest resource folder to TemplateSetArtifactReaderTest

* #1517 set java version to 11

* #1517 Removed java 11 dependency

* #1517 removed java 11 dependencies

* #1517 replaced List.of with simple List

* #1517 implemented requested changes
renamed get... methods
adjusted/added javadocs

* #1517 implemented requested changes
removed .classpath and .project files
moved timeout values to constants
converted SEARCH_RESPONSES from Object to AbstractSearchResponse
moved regex version number detection to constant

* #1517 implemented requested changes
saved the return of Systemutils.getUserHome() and determineMvnPath in a static variable to avoid multiple calls of the method

* #1517 implemented requested changes
added javadoc descriptions

* #1517 adjusted javadocs and class name
fixed type of Nexus2SearchResponseArtifactLinks

* #1517 Removed exceptions from tests, added javaDocs

* #1517 implemented requested changes
removed unnecessary returns

* #1517 adjusted artifact reader
adjusted artifact reader to latest TemplateSetConfiguration
added ContextConfiguration and TemplatesConfiguration entities
adjusted tests

* #1517 introduced new TemplateSet entity
replaced TemplateSetArtifactReader constructor with new retrieve method
adjusted tests

* #1517 implemented requested changes
removed unnecessary caching of user home and maven paths

* #1517 implemented requested changes
added zeroturnaround zt-exec 1.12 to the root pom as a managed dependency

* #1517 implemented requested changes
removed unnecessary build of test-jar

* #1517 #1517 implemented requested changes
adjusted debug log message
made sure that missing files won't break the retrieval process
added new testRetrieveTemplateSetArtifactWithMissingFile test

* #1517 implemented requested changes
changed wiremock dependency to wiremock-jre8 2.35.0

* #1517 implemented requested changes
changed java version to 11

Co-authored-by: jan-vcapgemini <jan-vincent.hoelzle@capgemini.com>

cobigen-templates/pom.xml

@@ -1,5 +1,6 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
+xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.devonfw.cobigen.templates</groupId>
   <artifactId>templates-parent</artifactId>

cobigen/cobigen-core-api/pom.xml

@@ -1,6 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
-  xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <modelVersion>4.0.0</modelVersion>
   <artifactId>core-api</artifactId>
   <name>CobiGen Core - API</name>
@@ -32,7 +31,6 @@
     <dependency>
       <groupId>org.zeroturnaround</groupId>
       <artifactId>zt-exec</artifactId>
-      <version>1.12</version>
     </dependency>
 
     <dependency>
@@ -66,6 +64,19 @@
       <version>2.27.2</version>
       <scope>test</scope>
     </dependency>
+
+    <!-- Needed for settings.xml decryption -->
+    <dependency>
+      <groupId>org.codehaus.plexus</groupId>
+      <artifactId>plexus-sec-dispatcher</artifactId>
+      <version>2.0</version>
+    </dependency>
+
+    <dependency>
+      <groupId>org.codehaus.plexus</groupId>
+      <artifactId>plexus-cipher</artifactId>
+      <version>2.0</version>
+    </dependency>
   </dependencies>
 
   <build>
@@ -83,4 +94,4 @@
       </plugin>
     </plugins>
   </build>
-</project>
+</project>

cobigen/cobigen-core-api/src/main/java/com/devonfw/cobigen/api/constants/ConfigurationConstants.java

@@ -20,6 +20,9 @@ public class ConfigurationConstants {
   /** Templates configuration file name */
   public static final String TEMPLATES_CONFIG_FILENAME = "templates.xml";
 
+  /** Template Set configuration file name */
+  public static final String TEMPLATE_SET_CONFIG_FILENAME = "template-set.xml";
+
   /** Filename of the {@link Properties} used to customize cobigen properties and template relocation. */
   public static final String COBIGEN_PROPERTIES = "cobigen.properties";
 

cobigen/cobigen-core-api/src/main/java/com/devonfw/cobigen/api/util/MavenUtil.java

@@ -27,9 +27,6 @@
 
 import com.devonfw.cobigen.api.constants.MavenConstants;
 import com.devonfw.cobigen.api.exception.CobiGenRuntimeException;
-import com.devonfw.cobigen.api.exception.RestSearchResponseException;
-import com.devonfw.cobigen.api.util.to.SearchResponseFactory;
-import com.fasterxml.jackson.core.JsonProcessingException;
 import com.google.common.collect.Lists;
 import com.google.common.hash.Hashing;
 import com.google.common.io.ByteSource;
@@ -198,6 +195,19 @@ public static Path determineMavenRepositoryPath() {
     return Paths.get(m2Repo);
   }
 
+  /**
+   * Returns maven's settings.xml as a string by using maven evaluate
+   *
+   * @return the maven's settings.xml as string
+   */
+  public static String determineMavenSettings() {
+
+    LOG.info("Determine content of maven's settings.xml");
+    String mavenSettings = runCommand(SystemUtils.getUserHome().toPath(), Lists.newArrayList(
+        SystemUtil.determineMvnPath().toString(), "help:evaluate", "-Dexpression=settings", "-DforceStdout"));
+    return mavenSettings;
+  }
+
   /**
    * Execute any command in the given execution directory with the given arguments
    *
@@ -343,27 +353,4 @@ public static Path getProjectRoot(Path inputFile, boolean topLevel) {
     LOG.debug("Project root could not be found.");
     return null;
   }
-
-  /**
-   * Retrieves a list of download URLs by groupId from the specified repository search REST API using authentication
-   * with bearer token
-   *
-   * @param baseUrl String of the repository server URL
-   * @param groupId the groupId to search for
-   * @param authToken bearer token to use for authentication
-   * @return List of artifact download URLS or null if an error occurred
-   */
-  public static List<URL> retrieveMavenArtifactsByGroupId(String baseUrl, String groupId, String authToken) {
-
-    try {
-
-      return SearchResponseFactory.searchArtifactDownloadLinks(baseUrl, groupId, authToken);
-    } catch (RestSearchResponseException | JsonProcessingException | MalformedURLException e) {
-      LOG.error("Unable to get artifacts from {} by groupId {}", baseUrl, groupId, e);
-      // TODO: Handle Eclipse, CLI and MavenPlugin here (f.e. with a new Exception)
-      return null;
-    }
-
-  }
-
 }