Support for verifying authenticity of artifacts w.r.t. update sources (MD5 or signatures)

[gberche-orange]

Thanks again for sharing the java-buildpack-dependency-builder, and its replicate script to be able to replicate artifacts within an offline private paas instance.

Following @nebhale 's suggestion to have users request most needed features in the java buildpack, it would be great if the java-buildpack-dependency-builder provided a way to trace back to the update sources and verify the integrity/authenticity of the artifact binaries.

Most artifacts repos make available checksums or digital signatures that enables their users to validate that binaries in downstream distributions properly match original distribution. This allows users to get protected against ~"man-in-the-middle" attacks that would inject malicious code into the replicated artifacts run by the java buildpack.

E.g. bintray provides a way to digitally sign artifacts using PGP keys, see also related binary manual.
As an example, the upload source for groovy is bintray, onto which the PGP signature is available as HTTPS download urls. Groovy ships and digital signature associated to each release (e.g. groovy-binary-2.2.2.zip.asc )

So the need is for java-buildpack users or paas operators using the replicate script to be able to verify that their locally replicated artifact match the checksum/hash or digital signatures of the corresponding update locations. Ideally this verification would be done automatically.

As an example of the value associated to this feature, some company policies require that binaries run in production can be tracked back to their origin, being either commercial vendors, distributions or opensource communities. This could prevent CloudFoundry java-buildpack adoption in those.

Some possible ways this could be provided:

• for users to semi-manually verify origin of replicated artifacts, provide a machine readable description of each artifacts with the following (could that fit into the current index.yml file) ?
  • the local file path (e.g. groovy/groovy-2.2.1.zip) of the artifact
  • the original update location (e.g. http://dl.bintray.com/groovy/maven/groovy-binary-2.4.0-beta-2.zip )
  • the traceability/authenticity verification method (eg 'pgp' or 'md5') and associated params (e.g. http://dl.bintray.com/groovy/maven/#groovy-binary-2.2.2.zip.asc and https://bintray.com/user/downloadSubjectPublicKey?username=groovy)
• users trusting the buildand replicate scripts to verify authenticity of the fetched/ replicated artifacts. The script would then automatically verify the authenticity of fetched/replicated artifacts against update locations, and fail on differences. Paas users or Paas operator would merely have to rate their confidence on the update source to be confident about code they use in their app.

[nebhale]

Wow. Interesting idea. Big change, but with the recent kerfluffel over Maven Central, perfectly justifiable. I think the biggest barrier is that most of the artifacts aren't verifiable from their source. I think it's reasonable that we put the infrastructure in place to verify replicates, and then explore extending that chain of trust to the original sources as a second step.

I suspect this won't happen in the near term, but let @rmorgan, @cgfrost, and me kick around some ideas.

[gberche-orange]

great, thanks Ben!

[nebhale]

@gberche-orange Finally dug through the open tabs on my desktop. Issue for this is here: https://www.pivotaltracker.com/story/show/76436618

[gberche-orange]

Thanks @nebhale

It would be interesting to see if any convergence/synergies could be made with the buildpacks dependencies epics such as including the signature + checksums/hash into the json-formatted dependencies description

[nebhale]

Yup, we're working with that team as we speak.