Keep track of checksum per gem in the lockfile

[coilysiren]

From @Bertg on September 30, 2013 8:49

More and more people are getting worried about the safety of gems and the ecosystem around it. There are some nice project on their way (signed gems etc...) but it might still be some time before these are main stream. And that might still not fix issues with git based gems etc.

What I'm suggesting here is not a 100% fail safe system. What I'd like to do here is add to Bundler's "The gems you run locally will be the gems on your server" guarantee.

I'm suggesting to add a checksum to the Gemfile.lock file per gem. When creating the lockfile we create a checksum per gem and store it in the file.
On install, specifically when retrieving a gem from a foreign source, we again make the checksum and compare.
This should add a significant amount of confidence that the source of the gem isn't compromised and that I'm running the exact same version on all machines.

What do you think?

Copied from original issue: rubygems/bundler-features#27

[coilysiren]

From @xaviershay on September 30, 2013 14:51

cc @tarcieri thoughts? Is this worth doing independently of other rubygems security features?

[coilysiren]

From @tarcieri on September 30, 2013 16:24

@xaviershay seems useful, even for non-security purposes

[coilysiren]

From @indirect on September 30, 2013 17:45

This is actually already on our list of things we'd like to do. If it ships with Bundler < 2.0, the checksums will have to be in a different file. If it ships with Bundler 2.0+, we should store the gem checksums in the lock file.

Either way, this gives Bundler users assurance that they are installing the same .gem files on their servers as they installed on their development machines, which is most of the benefit provided by Rubygems' current signing mechanism without any of the complex and hard-to-use bits.

On Sep 30, 2013, at 9:24 AM, Tony Arcieri notifications@github.com wrote:

@xaviershay seems useful, even for non-security purposes

—
Reply to this email directly or view it on GitHub.

[coilysiren]

From @bbhoss on October 1, 2013 20:5

I'm going to be taking a look at this soon, I posted basically this exact question last night to the list.

[coilysiren]

From @tarcieri on October 2, 2013 3:3

So here's a thought... if Bundler could:

• Ensure that any gems that have an associated hash are cryptographically checksummed
• Download a "revocation list" of any gem checksums known to be malicious (which is signed by the RubyGems master key)
• Provide a way to digitally sign a Gemfile

...what you wind up with is effectively a Merkle tree of a known good state of the system, along with a way for RubyGems to centrally revoke known malicious gems.

I think this covers most of the bases of the threat model we'd like to protect against with a cryptographic signature scheme for RubyGems.

[coilysiren]

From @bbhoss on October 2, 2013 3:18

The last two sound cool, but I think the first point you have there is what we're proposing. The rest seem like things we could add on later. As far as the first thing though, where would we store them in the lockfile? I propose a section named CHECKSUMS which would basically be a list of each of the gems in the specs list under the GEM section. I figure we can use the Digest::SHA2 since we have that all the way back to 1.8.7 (does bundler still support 1.8?) As far as git refs though, I think we should probably just ignore those since the hash itself is already a semi-secure reference point.

[coilysiren]

From @tarcieri on October 2, 2013 3:20

Yeah, everything I'm describing could be added later. Making sure you implement that first bullet point correctly would be a good start ;)

More specifically, it'd be good if Bundler could record the checksums of every gem in the lockfile.

And perhaps it might be worth using the term "DIGESTS" to better indicate you're using a cryptographically secure hash function for the checksumming.

[coilysiren]

From @bbhoss on October 2, 2013 3:21

Cool. Any thoughts on my lockfile proposal? I'm not super familiar with the internals of bundler yet, and I know we want to maintain compatibility with old versions.

[coilysiren]

From @tarcieri on October 2, 2013 3:23

Compatibility seems easy: you use the digests if they're available, and ignore them if they're not there, unless a flag is passed that mandates that the Gemfile.lock contains digests of the gems

[coilysiren]

From @Bertg on October 2, 2013 11:21

@bbhoss & @tarcieri What I was originally proposing is that the client that builds the lockfile makes a checksum for each gem independent of gem itself.

Neither info in the gem or info based on it's delivery mechanism should be trusted; as these could be compromised.
We should assume that the git-SHA could be faked, certificates could be compromised.

We can only trust the code itself, if we make a checksum the entire gem's code. When the code code gets compromised, we will know.

[coilysiren]

From @indirect on October 2, 2013 20:7

I agree that we should make a checksum for each gem (specifically, the .gem file that the gem is installed from). If we're talking to a compromised gem source, it would obviously self-report that the checksums are correct.

Git shas aren't supplied by the server, they are calculated by the git client. As a result, we already do this for git gems, because we store the target sha in the Gemfile.lock.

All that to say, the scope of this feature would be adding checksums to non-git, non-path gems, and I think it would be very helpful.

[coilysiren]

From @bbhoss on October 2, 2013 20:8

I spent a few hours trying to grok the bundler source, but couldn't find a good place to add this. Could someone with more knowledge give me a high level overview of how they think it should be added?

[coilysiren]

From @indirect on October 2, 2013 20:40

I'm not sure where the best place would be, but probably something that calculates the hashes of the gem about to be installed in the installer code, and then checks the checksum against the checksum provided for that gem, if there is one.

As for writing the checksums, I think it probably makes sense to make that part of the code that writes each gem to the DEPENDENCIES section of the lockfile, and puts the checksum after the gem name.

[coilysiren]

From @indirect on October 2, 2013 20:40

(If you'd like more help, feel free to ask here, join #bundler on freenode, or email me directly.)

[coilysiren]

From @TimMoore on October 2, 2013 20:44

Won't this break platform-specific gems?