Skip to content

Commit

Permalink
Added JSCover to compute coverage of Javascript files
Browse files Browse the repository at this point in the history
  • Loading branch information
jerome-netguardians committed Dec 16, 2019
1 parent 0695a8d commit eb96894
Show file tree
Hide file tree
Showing 17 changed files with 319 additions and 29 deletions.
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ addons:

script:
# the following command line builds the project, runs the tests with coverage and then execute the SonarCloud analysis
- mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install sonar:sonar -Dsonar.projectKey=eskimo-sh_eskimo
- mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install sonar:sonar -PjsCoverage -Dsonar.projectKey=eskimo-sh_eskimo
8 changes: 4 additions & 4 deletions doc/guides/eskimo-guide/eskimo-guide.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -900,8 +900,8 @@ available from within Eskimo.

They are disposed in the menu under "_Eskimo Services_".

The pre-packaged web consoles with Eskimo are Zeppelin, Gdash, Kibana, Grafana, Cerebro, Spark History Server, Kafka
Manager and Mesos Console.
The pre-packaged web consoles with Eskimo are Zeppelin, Gdash, Kibana, Grafana, Cerebro, Spark History Server,
Flink App Manager, Kafka Manager and Mesos Console.


== Eskimo pre-Packaged services
Expand Down Expand Up @@ -934,7 +934,7 @@ Since every restart of a service creates actually a new docker container, contai
freshly restarted every time. +
This is why the persistent data is stored under sub-folders if `/var/lib` which is mounted to the docker container.

==== Commands wrappers for kafka, logstash and spark
==== Commands wrappers for kafka, logstash, spark and flink

Commands such as kafka `create-producer.sh` or spark's `spark-submit` work only from within the respective kafka or spark
executor docker containers.
Expand Down Expand Up @@ -1138,7 +1138,7 @@ enabling fault-tolerant and elastic distributed systems to easily be built and r

Mesos is a distributed system kernel. Mesos is built using the same principles as the Linux kernel, only at a
different level of abstraction. +
The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elasticsearch) with
The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Flink) with
API’s for resource management and scheduling across entire datacenter and cloud environments.

http://mesos.apache.org/
Expand Down
14 changes: 14 additions & 0 deletions eskimo-ce.iml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,20 @@
<orderEntry type="library" scope="TEST" name="Maven: org.eclipse.jetty:jetty-io:9.4.15.v20190215" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: org.eclipse.jetty.websocket:websocket-common:9.4.15.v20190215" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: org.eclipse.jetty.websocket:websocket-api:9.4.15.v20190215" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.github.tntim96:JSCover:2.0.8" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.javascript:closure-compiler:v20190618" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.javascript:closure-compiler-externs:v20190618" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: args4j:args4j:2.0.26" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.errorprone:error_prone_annotations:2.3.1" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.guava:guava:25.1-jre" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: org.checkerframework:checker-qual:2.0.0" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.j2objc:j2objc-annotations:1.1" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: org.codehaus.mojo:animal-sniffer-annotations:1.14" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.protobuf:protobuf-java:3.0.2" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.code.gson:gson:2.8.5" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.code.findbugs:jsr305:3.0.1" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: com.google.jsinterop:jsinterop-annotations:1.0.0" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: org.mozilla:rhino:1.7.11" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: org.eclipse.jetty:jetty-util:9.4.16.v20190411" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: org.apache.sshd:sshd-core:2.2.0" level="project" />
<orderEntry type="library" scope="TEST" name="Maven: org.apache.sshd:sshd-common:2.2.0" level="project" />
Expand Down
46 changes: 46 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,12 @@ Software.
<version>2.35.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.github.tntim96</groupId>
<artifactId>JSCover</artifactId>
<version>2.0.8</version>
<scope>test</scope>
</dependency>

<!-- for HTML Unit -->
<dependency>
Expand Down Expand Up @@ -273,6 +279,46 @@ Software.
</build>

<profiles>
<profile>
<id>jsCoverage</id>
<build>
<plugins>
<!-- Set the JS coverage run flag -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<id>setFlag</id>
<phase>compile</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>/bin/touch</executable>
<arguments>
<argument>target/jsCoverageFlag</argument>
</arguments>
</configuration>
</execution>
<execution>
<id>computeLcov</id>
<phase>verify</phase>
<goals>
<goal>java</goal>
</goals>
<configuration>
<includeProjectDependencies>true</includeProjectDependencies>
<mainClass>ch.niceideas.eskimo.utils.GenerateLCOV</mainClass>
<classpathScope>test</classpathScope>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>pdfDoc</id>
<activation>
Expand Down
134 changes: 134 additions & 0 deletions src/test/java/ch/niceideas/eskimo/utils/GenerateLCOV.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
package ch.niceideas.eskimo.utils;

import ch.niceideas.common.utils.FileUtils;
import jscover.report.BranchData;
import jscover.report.FileData;
import jscover.report.JSONDataMerger;
import jscover.report.lcov.LCovGenerator;
import org.apache.log4j.Logger;

import java.io.File;
import java.util.List;
import java.util.SortedMap;
import java.util.TreeMap;

import static java.lang.String.format;

public class GenerateLCOV {

public static final String SOURCE_DIRECTORY = "src/main/webapp";
private static Logger logger = Logger.getLogger(GenerateLCOV.class);

private static String jsCoverReportDir = "target/jscov-report";

private static String mergedJsCoverReportDir = "target/jscov-report-merged";

private static String lcovReportDir = "target/jscov-lcov";

private static JSONDataMerger jsonDataMerger = new JSONDataMerger();

private static LCovGenerator lCovGenerator = new LCovGenerator();

public static void main (String[] args) {

try {

File reportDir = new File (jsCoverReportDir);
if (!reportDir.exists()) {
logger.warn(jsCoverReportDir + " doesn't exist");
System.exit (-1);
}

SortedMap<String, FileData> total = new TreeMap<>();
for (File reportSubdir : reportDir.listFiles()) {
File reportFile = new File (reportSubdir, "jscoverage.json");
if (!reportFile.exists()) {
logger.warn(reportFile + " doesn't exist");
} else {

logger.info("Merging : " + reportFile);
String reportContent = FileUtils.readFile(reportFile);
total = jsonDataMerger.mergeJSONCoverageMaps(total, jsonDataMerger.jsonToMap(reportContent));
}
}

String merged = GenerateLCOV.toJSON(total);
File targetFile = new File (mergedJsCoverReportDir, "jscoverage.json");
targetFile.getParentFile().mkdirs();
FileUtils.writeFile(targetFile, merged);


File lcovFile = new File(lcovReportDir, "jscover.lcov");
lCovGenerator.saveData(jsonDataMerger.jsonToMap(merged).values(), SOURCE_DIRECTORY, lcovFile);

} catch (Exception e) {
logger.error(e, e);
}

}

public static String toJSON(SortedMap<String, FileData> map) {
StringBuilder json = new StringBuilder("{");
int scriptCount = 0;
for (String scriptURI : map.keySet()) {
StringBuilder coverage = new StringBuilder();
StringBuilder branchData = new StringBuilder();
FileData coverageData = map.get(scriptURI);
for (int i = 0; i < coverageData.getLines().size(); i++) {
if (i > 0)
coverage.append(",");
coverage.append(coverageData.getLines().get(i));
}

// Function Coverage (HA-CA)
StringBuilder functions = new StringBuilder();
for (int i = 0; i < coverageData.getFunctions().size(); i++) {
if (i > 0)
functions.append(",");
functions.append(coverageData.getFunctions().get(i));
}

addBranchData(branchData, coverageData);
if (scriptCount++ > 0) {
json.append(",");
}

StringBuilder scriptJSON = new StringBuilder(format("\"%s\":{", scriptURI));
scriptJSON.append(format("\"lineData\":[%s]", coverage));
if (functions.length() > 0)
scriptJSON.append(format(",\"functionData\":[%s]", functions));
if (branchData.length() > 0)
scriptJSON.append(format(",\"branchData\":{%s}", branchData));
scriptJSON.append("}");
json.append(scriptJSON);
}
json.append("}");
return json.toString();
}

static void addBranchData(StringBuilder branchData, FileData coverageData) {
int count = 0;
for (Integer i: coverageData.getBranchData().keySet()) {
List<BranchData> conditions = coverageData.getBranchData().get(i);
if (count++ > 0)
branchData.append(",");
branchData.append(format("\"%s\":[",i));
addBranchConditions(branchData, conditions);
branchData.append("]");
}
}

static void addBranchConditions(StringBuilder branchData, List<BranchData> conditions) {
for (int j = 0; j < conditions.size(); j++) {
if (j > 0)
branchData.append(",");
BranchData branchObj = conditions.get(j);
if (branchObj == null) {
branchData.append("null");
} else {
String branchJSON = "{\"position\":%d,\"nodeLength\":%d,\"evalFalse\":%d,\"evalTrue\":%d}";
branchData.append(format(branchJSON, branchObj.getPosition(), branchObj.getNodeLength(), branchObj.getEvalFalse(), branchObj.getEvalTrue()));
}
}
}
}
109 changes: 103 additions & 6 deletions src/test/javawebtests/ch/niceideas/eskimo/html/AbstractWebTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -34,26 +34,123 @@

package ch.niceideas.eskimo.html;

import ch.niceideas.common.utils.FileUtils;
import ch.niceideas.common.utils.ResourceUtils;
import ch.niceideas.eskimo.utils.GenerateLCOV;
import com.gargoylesoftware.htmlunit.AjaxController;
import com.gargoylesoftware.htmlunit.AlertHandler;
import com.gargoylesoftware.htmlunit.Page;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import jscover.Main;
import jscover.report.FileData;
import jscover.report.JSONDataMerger;
import org.apache.log4j.Logger;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.*;
import org.junit.rules.TestName;

import java.io.File;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
import java.util.SortedMap;
import java.util.TreeMap;

public abstract class AbstractWebTest {

private static final Logger logger = Logger.getLogger(AbstractWebTest.class);

private static Thread server;
private static Main main = null;

private static File jsCoverageFlagFile = new File("target/jsCoverageFlag");

private static String jsCoverReportDir = "target/jscov-report";
private static String[] jsCoverArgs = new String[]{
"-ws",
"--document-root=src/main/webapp",
"--port=9001",
//"--no-branch",
//"--no-function",
//"--no-instrument=example/lib",
"--log=INFO",
"--report-dir=" + jsCoverReportDir
};

private static String className = null;
private static List<String> coverages = new ArrayList<>();

private static JSONDataMerger jsonDataMerger = new JSONDataMerger();
@Rule
public TestName name = new TestName();

protected WebClient webClient;
protected HtmlPage page;

@BeforeClass
public static void setUpOnce() {
if (isCoverageRun()) {
main = new Main();
server = new Thread(() -> main.runMain(jsCoverArgs));
server.start();
}
}

@AfterClass
public static void tearDownOnce() throws Exception {
if (isCoverageRun()) {
main.stop();

File targetFile = new File(jsCoverReportDir + "/" + className, "jscoverage.json");
targetFile.getParentFile().mkdirs();
FileUtils.writeFile(targetFile, mergeJSON());
}
}

@Before
public void setUpClassName() throws Exception {
Class clazz = this.getClass(); //if you want to get Class object
className = clazz.getCanonicalName(); //you want to get only class name
}

@After
public void tearDown() throws Exception {
if (isCoverageRun()) {
page.executeJavaScript("window.jscoverFinished = false;");
page.executeJavaScript("jscoverage_report('', function(){window.jscoverFinished=true;});");

int attempt = 0;
while ((!((Boolean) (page.executeJavaScript("window.jscoverFinished").getJavaScriptResult())).booleanValue()) && attempt < 10) {
logger.debug("Waiting for coverage report to be written ...");
Thread.sleep(500);
attempt++;
}

String json = (String) (page.executeJavaScript("jscoverage_serializeCoverageToJSON();")).getJavaScriptResult();
coverages.add(json);
}
}

private static String mergeJSON() {
SortedMap<String, FileData> total = new TreeMap<>();
for (int i = 0; i < coverages.size(); i++) {
String json = coverages.get(i);
total = jsonDataMerger.mergeJSONCoverageMaps(total, jsonDataMerger.jsonToMap(json));
}
return GenerateLCOV.toJSON(total);
}

protected static final boolean isCoverageRun() {
//return true;
return jsCoverageFlagFile.exists();
}

protected final void loadScript (HtmlPage page, String script) {
if (isCoverageRun()) {
page.executeJavaScript("loadScript('http://localhost:9001/scripts/"+script+"')");
} else {
page.executeJavaScript("loadScript('../../src/main/webapp/scripts/"+script+"')");
}
}

@Before
public void init() throws Exception {
webClient = new WebClient();
Expand Down Expand Up @@ -112,7 +209,7 @@ public void init() throws Exception {
page.executeJavaScript("eskimoMain.setAvailableNodes = function () {};");
page.executeJavaScript("eskimoMain.menuResize = function () {};");

page.executeJavaScript("loadScript('../../src/main/webapp/scripts/jquery-3.3.1.js')");
loadScript (page, "jquery-3.3.1.js");

// override jquery load
page.executeJavaScript("$.fn._internalLoad = $.fn.load;");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ public class EskimoConsolesTest extends AbstractWebTest {
@Before
public void setUp() throws Exception {

page.executeJavaScript("loadScript('../../src/main/webapp/scripts/eskimoConsoles.js')");
loadScript (page, "eskimoConsoles.js");

// mock ajax term
page.executeJavaScript("ajaxterm = {};");
Expand Down
Loading

0 comments on commit eb96894

Please sign in to comment.