Skip to content

Merge redesign into develop #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Nov 9, 2021
Merged

Merge redesign into develop #50

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
df479f6
Upgrade to OPAL 4.0
johannesduesing Jun 17, 2021
d6f09a0
Adopt tests: Change http to https, adapt to OPAL upgrade
johannesduesing Jun 17, 2021
3717948
Merge pull request #1 from johannesduesing/feature/upgrade-opal
johannesduesing Jun 18, 2021
73664f6
Upgrade to latest version of Elastic4s. Upgrade to latest AKKA versio…
johannesduesing Oct 6, 2021
b9de0e9
Upgrade to scala 2.12.15, move to SBT 1.5.5, cleanup and upgrade depe…
johannesduesing Oct 6, 2021
4453790
Upgrade to OPAL 4.0.0, fixed test errors, removed unused code
johannesduesing Oct 18, 2021
da60e4e
Fixed various deprecation issues
johannesduesing Oct 18, 2021
3412b30
Crawler now mines POM file information only. Functionality for calcul…
johannesduesing Oct 19, 2021
f5e55e5
Add publish date to index
johannesduesing Oct 19, 2021
0fe70c0
Switched to an external library for resolving effective POM models. T…
johannesduesing Oct 20, 2021
3ec7241
Reintroduced Hermes Analyzer into pipeline, fixed bug related to OPAL…
johannesduesing Oct 20, 2021
63a8469
Add error handling for different processing phases
johannesduesing Oct 20, 2021
0171ae5
Removed unused code
johannesduesing Oct 20, 2021
42b8e3b
Add Tests for PomFileReadActor. Removed unused GitIdentifier. Fixed b…
johannesduesing Oct 21, 2021
7fb1f48
Removed unused library
johannesduesing Oct 21, 2021
1d168c2
Merge remote-tracking branch 'origin/develop' into feature/redesign
johannesduesing Oct 21, 2021
422e828
Fixed some unecessary merge artefacts
johannesduesing Oct 21, 2021
8afd7cb
Upgrade to latest version of JWT core to mitigate vulnerabilties dete…
johannesduesing Oct 21, 2021
d5b1a20
Pinned secure version of guava
johannesduesing Oct 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Reintroduced Hermes Analyzer into pipeline, fixed bug related to OPAL… 
… upgrade
  • Loading branch information
@johannesduesing
johannesduesing committed Oct 20, 2021
commit 3ec72417acfa6cbeaf6bac32815ebf2449d5fc91
2 changes: 1 addition & 1 deletion src/main/scala/de/upb/cs/swt/delphi/crawler/Crawler.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ object Crawler extends App with AppLogging {
private val configuration = new Configuration()
implicit val system: ActorSystem = ActorSystem("delphi-crawler")

OPALLogAdapter.setOpalLoggingEnabled(true)
OPALLogAdapter.setOpalLoggingEnabled(false)
HermesAnalyzer.setConfig()

sys.addShutdownHook({
Expand Down
6 changes: 5 additions & 1 deletion src/main/scala/de/upb/cs/swt/delphi/crawler/discovery/maven/MavenDiscoveryProcess.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,11 @@ class MavenDiscoveryProcess(configuration: Configuration, elasticPool: ActorRef)

isValid
}
.runWith(createSinkFromActorRef[MavenArtifact](elasticPool))
.alsoTo(createSinkFromActorRef[MavenArtifact](elasticPool))
.mapAsync(8)(artifact => (hermesPool ? artifact).mapTo[Try[HermesResults]])
.filter(_.isSuccess)
.map(_.get)
.runWith(createSinkFromActorRef[HermesResults](elasticPool))

Success(0L)
}
Expand Down
7 changes: 6 additions & 1 deletion src/main/scala/de/upb/cs/swt/delphi/crawler/processing/HermesActor.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import akka.actor.{Actor, ActorLogging, ActorRef, Props}
import de.upb.cs.swt.delphi.core.model.MavenIdentifier
import de.upb.cs.swt.delphi.crawler.model.MavenArtifact

import scala.util.Try
import scala.util.{Failure, Success, Try}

class HermesActor() extends Actor with ActorLogging with OPALFunctionality with HermesFunctionality {

Expand All @@ -32,6 +32,11 @@ class HermesActor() extends Actor with ActorLogging with OPALFunctionality with
computeHermesResult(m, reifyProject(m))
}

hermesResult match {
case Success(r) => log.info(s"Hermes run successful for ${m.identifier.toUniqueString}")
case Failure(ex) => log.error(ex, s"Hermes run failed for ${m.identifier}")
}

sender() ! hermesResult
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/main/scala/de/upb/cs/swt/delphi/crawler/processing/OPALFunctionality.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import scala.util.Try
trait OPALFunctionality {

def reifyProject(m: MavenArtifact): Project[URL] = {
val project = ClassStreamReader.createProject(m.identifier.toJarLocation.toURL,true)
val project = ClassStreamReader.createProject(m.identifier.toJarLocation.toURL, m.jarFile.get.is, true)
Try(m.jarFile.get.is.close())
project
}
Expand Down
6 changes: 6 additions & 0 deletions src/main/scala/de/upb/cs/swt/delphi/crawler/tools/ClassStreamReader.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,12 @@ object ClassStreamReader {
createProject(source, jarInputStream, projectIsLibrary)
}

def createProject(source: URL, is: InputStream, projectIsLibrary: Boolean): Project[URL] = {
val jis = new JarInputStream(is)

createProject(source, jis, projectIsLibrary)
}

def createProject(source: URL,
jarInputStream: JarInputStream,
projectIsLibrary: Boolean): Project[URL] = {
Expand Down