Scanner fails with "multiple scan results for the same provenance" error

[daniel-kr]

Describe the bug

When scanning projects containing AngularJS dependencies (angular, angular-sanitize, and angular-animate in version 1.8.3), ORT version 70.1.0 fails with an IllegalArgumentException stating "Found multiple scan results for the same provenance and scanner". This issue prevents successful completion of the scanning process. ORT version 61.0.0 does not exhibit this problem.

The root cause seems to be that npm info returns incorrect git revision information from bower repositories for these three AngularJS packages instead of the correct angular.js repository. This leads to duplicate scan results that fail validation.

To Reproduce

Steps to reproduce the behavior:

1. Create a project with the following dependencies in package.json:

   {
     "dependencies": {
       "angular": "1.8.3",
       "angular-sanitize": "1.8.3",
       "angular-animate": "1.8.3"
     }
   }

   A minimal reproduction repository is available at: https://github.com/daniel-kr/minimal-frontend-repo/tree/tmp/angularjs-duplicate-prevenances

2. Run ORT analyzer on the project

3. Run ORT scanner on the analyzer result

Expected behavior

ORT should handle the incorrect npm metadata gracefully and complete the scan. The merge logic should account for revision corrections that occur during the scan phase.

Console / log output

IllegalArgumentException: Found multiple scan results for the same provenance and scanner.
Scanner:
---
name: "ScanCode"
version: "32.4.1"
configuration: "--copyright --license --info --strip-root --timeout 300 --json"

Provenance:
---
vcs_info:
  type: "Git"
  url: "https://github.com/angular/angular.js.git"
  revision: "cf16b241e1c61c22a820ed8211bc2332ede88e62"
  path: ""
resolved_revision: "cf16b241e1c61c22a820ed8211bc2332ede88e62"

No scanner run was created.

Environment

Stock ORT docker version 70.1.0

config.yml:

ort:
  forceOverwrite: true
  analyzer:
    skipExcluded: true
    allow_dynamic_versions: true
    enabled_package_managers: [ NPM ]
  downloader:
    sourceCodeOrigins: [VCS, ARTIFACT]
    skipExcluded: true
  scanner:
    skipExcluded: true
    storages:
      local:
        backend:
          localFileStorage:
            directory: ${user.home}/.ort/scanner/results
            compression: false
    storageReaders: [local]
    storageWriters: [local]

Additional context

Root Cause Analysis

The issue seems to occur due to the following sequence of events:

1. NPM Analyzer Phase: npm info returns incorrect git revision information for the three AngularJS packages:

   • Returns revisions from separate bower repositories instead of the main angular.js repository
   • Each package gets a different incorrect revision
   • All three packages actually share the same repository (angular/angular.js)
   • merge logic does not merge them because their revisions differ.

2. Scan Phase: ORT attempts to check out the incorrect revisions and fails (they don't exist in the angular.js repo)

   • ORT falls back to checking out the tag v1.8.3
   • All three packages now have the same correct revision
   • With the same repository and revision, they become duplicates.

3. Validation Phase: Three scan results with identical revisions fail the duplicate check

   • It appears that the merge logic that should merge duplicates might run BEFORE the revision correction (this is an assumption based on the observed behavior)
   • After correction, the three results become duplicates but are no longer deduplicated

Related Changes

This issue appears to be introduced by changes in ORT 65.0.0:

• PR feat(model): Guard duplicate scan results / file lists per provenance #10584
• PR feat(scanner): Merge duplicate scan results that share a provenance #10502

Workaround

A package curation can be applied to correct the revision information in this particular case:

- id: "NPM::angular:1.8.3"
  curations:
    comment: "Correct VCS information to point to the specific release tag. For some reason, npm info shows the revision of the bowser repositories which is wrong."
    vcs:
      type: "Git"
      url: "https://github.com/angular/angular.js.git"
      revision: "v1.8.3"
- id: "NPM::angular-animate:1.8.3"
  curations:
    comment: "Correct VCS information to point to the specific release tag. For some reason, npm info shows the revision of the bowser repositories which is wrong."
    vcs:
      type: "Git"
      url: "https://github.com/angular/angular.js.git"
      revision: "v1.8.3"
- id: "NPM::angular-sanitize:1.8.3"
  curations:
    comment: "Correct VCS information to point to the specific release tag. For some reason, npm info shows the revision of the bowser repositories which is wrong."
    vcs:
      type: "Git"
      url: "https://github.com/angular/angular.js.git"
      revision: "v1.8.3"