Help running scan with FossID

[LuisMartinPoloESA]

Hi everyone,

I'm trying to run the scan with our FOSSID instance and I was able to configure the scan in the config-yml file, it looks like the connection works but I get the following error message. Can somebody help me to understand what's going on?

Many thanks!

12:59:09.301 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - waitForResult parameter is set to 'true'
12:59:09.309 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is .
12:59:09.310 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is .
12:59:09.330 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdUrlProvider - Mapping https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}.
12:59:09.418 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server URL is http://localhost.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by retrofit2.Platform (file:/home/luismartin/Software/ort/cli/build/install/ort/lib/retrofit-2.9.0.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
12:59:09.703 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server is running version 2023.1.0.
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - waitForResult parameter is set to 'true'
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is .
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is .
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdUrlProvider - Mapping https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}.
12:59:09.704 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server URL is http://localhost.
Scanning projects with:
        FossId (version 2023.1.0)
Scanning packages with:
        FossId (version 2023.1.0)
Exception in thread "main" com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `org.ossreviewtoolkit.clients.fossid.model.Project` from Array value (token `JsonToken.START_ARRAY`)
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 61] (through reference chain: org.ossreviewtoolkit.clients.fossid.EntityResponseBody["data"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1767)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1541)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1488)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeFromArray(BeanDeserializer.java:654)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:211)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:440)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1493)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2125)
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1533)
	at retrofit2.converter.jackson.JacksonResponseBodyConverter.convert(JacksonResponseBodyConverter.java:33)
	at retrofit2.converter.jackson.JacksonResponseBodyConverter.convert(JacksonResponseBodyConverter.java:23)
	at retrofit2.OkHttpCall.parseResponse(OkHttpCall.java:243)
	at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:153)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

[sschuberth]

This looks like a case of #8462. The work-around (or actually the fix) is to resolve the underlying access problem.

[nnobelis]

@sschuberth I am not so sure as the error happens now in getProject....

I am a bit concerned that there are no output Parameterizing the name with pattern 'XXX'.

Could you please post your config-yml, without the sensitive data ?

[nnobelis]

12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is .
12:59:09.704 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is .

I think the problem is that you have no naming pattern in the configuration.

Try to add the following options to your config.yml:

ort:
  scanner:
    config:
      FossId:
        options:
          namingProjectPattern: "$FossidProjectName"
          namingScanPattern: "scan_#projectName_#branch_#currentTimestamp_#deltaTag"
          namingVariableFossidProjectName: '<put your project name here>'

[LuisMartinPoloESA]

Hi @nnobelis ,

many thanks for the quick answer. This is an extract of my config now, after the changes:

  FossId:
    options:
      serverUrl: 'http://localhost'

      namingProjectPattern: '$FossidProjectName'
      namingScanPattern: 'scan_#projectName_#branch_#currentTimestamp_#deltaTag'
      namingVariableFossidProjectName: 'BH34ORT'

      waitForResult: true

      keepFailedScans: false
      deltaScans: true
      deltaScanLimit: 10

      detectLicenseDeclarations: true
      detectCopyrightStatements: true

      timeout: 60

      urlMappingExample: "https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}"

    secrets:
      user: xxx
      apiKey: xxx

And the error:

13:38:29.814 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - waitForResult parameter is set to 'true'
13:38:29.823 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is $FossidProjectName.
13:38:29.824 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is scan_#projectName_#branch_#currentTimestamp_#deltaTag.
13:38:29.844 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdUrlProvider - Mapping https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}.
13:38:29.927 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server URL is http://localhost.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by retrofit2.Platform (file:/home/luismartin/Software/ort/cli/build/install/ort/lib/retrofit-2.9.0.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
13:38:30.286 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server is running version 2023.1.0.
13:38:30.287 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - waitForResult parameter is set to 'true'
13:38:30.287 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for projects is $FossidProjectName.
13:38:30.287 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdConfig - Naming pattern for scans is scan_#projectName_#branch_#currentTimestamp_#deltaTag.
13:38:30.288 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossIdUrlProvider - Mapping https://my-repo.example.org(?<repoPath>.*) -> ssh://my-mapped-repo.example.org${repoPath}.
13:38:30.288 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server URL is http://localhost.
13:38:30.296 [main] INFO  org.ossreviewtoolkit.clients.fossid.FossIdRestService - The FossID server is running version 2023.1.0.
Scanning projects with:
        FossId (version 2023.1.0)
Scanning packages with:
        FossId (version 2023.1.0)
13:38:31.582 [main] INFO  org.ossreviewtoolkit.plugins.scanners.fossid.FossId - Found project name 'gaia-bhthree' in URL https://github.com/esa/gaia-bhthree.git.
Exception in thread "main" com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `org.ossreviewtoolkit.clients.fossid.model.Project` from Array value (token `JsonToken.START_ARRAY`)
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 61] (through reference chain: org.ossreviewtoolkit.clients.fossid.EntityResponseBody["data"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1767)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1541)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1488)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeFromArray(BeanDeserializer.java:654)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:211)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:440)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1493)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2125)
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1533)
	at retrofit2.converter.jackson.JacksonResponseBodyConverter.convert(JacksonResponseBodyConverter.java:33)
	at retrofit2.converter.jackson.JacksonResponseBodyConverter.convert(JacksonResponseBodyConverter.java:23)
	at retrofit2.OkHttpCall.parseResponse(OkHttpCall.java:243)
	at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:153)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Maybe there is some info missing in the analyser result?

[nnobelis]

No I think it's an error in the communication format with FossID.

Could you please parametrize the following curl request, run it and post us the response ?

curl --request POST \
  --url <url of your fossid instance>/api.php \
  --header 'Content-Type: application/json' \
  --data '{
	"action" : "get_information",
	"group" : "projects",
	
	"data" : {
		"username":"<username>",
		"key":"<api key>",
		"project_code": "gaia-bhthree"
	}
} 

[LuisMartinPoloESA]

This is all I can get:
{"operation":"projects_get_information","status":"0","data":null,"error":"An error occurred. Please check WebApp logs.","message":"An error occurred. Please check WebApp logs.","message_parameters":[]}

I was trying with the web user/password but was always getting an authentication error. When I changed to the fossid database user/password, then it returns this error. When I check the nginx logs, nothing really useful appears there.

FYI, I am using php 8.2 with fossid, not sure of this might be causing the problem.

[nnobelis]

The apikey is not the Web user's password.
You need to generate one in the UI:

[LuisMartinPoloESA]

Indeed, that was the issue. Everything works now, thank you for you help!

[nnobelis]

Glad to be of help.
I created #8605 to improve the error reporting.