please do not depend on python-pip3 - security risk - use dependencies available from distribution package repositories #2326
Comments
|
As far I know bitpay/copay like issue could also happen on any Linux repository: at that case someone took over the maintenance of a no longer maintained package, then commited a problematic code. After a quick look the only packages that are missing from Debian repo: If someone creates package for these, then it should work |
|
I'm against using distribution package repositories. Packages in them are often outdated, take a lot of time to update, and many of them simply aren't available. They also aren't portable. How would distribution package repositories prevent their developers from submitting malicious version? The same happened with |
🤦♂️ |
|
Filip Š:
I'm against using distribution package repositories. Packages in them are often outdated, take a lot of time to update, and many of them simply aren't available.
It works for thousands of applications available from
packages.debian.org. Among them, popular applcations such as firefox,
gimp, libreoffice, and whatnot.
How would distribution package repositories prevent their developers from submitting malicious version?
Debian developers don't allow malware into Debian. That fortunately
hasn't happened to Debian yet.
|
But that applications are not written in Python and are not using Python dependencies.
NPM and PyPI also don't allow malware. But malicious maintainer of the package can still push it. |
Actually Firefox includes the dependencies inside the package: Just pdf.js (the part they use to display pdf files) has over 200 dependencies that are not installed using apt repository: https://github.com/mozilla/pdf.js |
|
Filip Š:
> It works for thousands of applications available from
But that applications are not written in Python and are not using Python dependencies.
Many python based applications in Debian are using python dependencies
also variable from Debian.
Here is a list of popular python based applications:
https://en.wikipedia.org/wiki/List_of_Python_software#Applications
Among them deludge (bittorrent client) and pip just to pick two examples.
Some of them are available from Debian package sources.
> Debian developers don't allow malware into Debian.
NPM and PyPI also don't allow malware. But malicious maintainer of the package can still push it.
This has never happened in Debian yet which has thousands of packages.
|
|
Linux ≠ Debian. While popular, Debian-based Linux distributions occupy only a small fraction of the Linux space. Devolving ZeroNet from its current distro-agnostic installation instructions to distro-specific installation instructions makes little to no sense. Mostly, no sense. The current distro-agnostic installation instructions are fine. What's not fine is the sluggish pace of package publication by Debian developers – but that's entirely on them. ZeroNet isn't to blame here. When even Gentoo and Arch Linux provide up-to-date packages for ZeroNet, you know that something's wrong with your rolling release model... or lack thereof.
Relative perspective: it matters.
No, because distro-specific package repositories are (wait for it)... distro-specific. ZeroNet would only be installable under Debian-based Linux distributions – a significant regression from ZeroNet's current applicability to all Linux distributions. Does that actually sound like a good idea to anyone in attendance who is not @adrelanos? If we really want to walk down this road in a sane way, we might consider publishing a distro-agnostic AppImage for ZeroNet. Unlike distro-specific packages, AppImages portably run on all Linux distributions. They also bundle all requisite dependencies in a safe, secure, robust, reproducible manner isolated from any package manager – If portability, security, and ease of installation are your concerns, AppImage is your answer. Debian is a non-answer to a question that no one asked.
No. The blockers here are Debian developers – not ZeroNet. Again, Gentoo and Arch Linux both publish ZeroNet packages. If the little guys with effectively no funding can do it, the big guys with absurd piles of multinational corporate venture capital absolutely can, too. This is entirely on Debian. Debian's well-known internal torpor is entirely a Debian issue. tl;drThis issue is a non-issue and should probably be closed. |
|
pip still has no idea about code signing in this day and age. It was conisdered a big leap forward when they implemented fetching over HTTPS and reject plaintext connections which is really sad considering the poor security assumptions that even this would have: Here are two different real world cases of malware snuck into PyPi being executed becuase of pip's non existent security: The problem is ultimately the result of developers and administrators who fail to inspect packages thoroughly. Adding to the insecurity, the widely used pip package management system (pictured above), which most Python developers rely on, doesn't require cryptographic signature before executing code when a package is installed. https://www.zdnet.com/article/twelve-malicious-python-libraries-found-and-removed-from-pypi/ Please consider an alternative packaging/installation model that doesn't rely on pip. Either using stuff in Debian backports or just making an AppImage (which includes code that is audited by you beforehand) - Electrum does the latter since their release cycle is far too rapid for it to sync with Debian's. We really love Zeronet and want to include it in our privacy distro by default, but unless we can have a secure installation mechanism, we just can't do it. PS. I am only interested in Zeronet dev replies. If you are an enthusiast please express yourself in another thread. |
|
APT by default uses HTTP and there were already vulnerabilities to allow man-in-the-middle attacks (here and here. Cryptographically signing packages does not prevent their authors to publish malicious packages, which happened in the case of npm and pip. The only way to prevent this would me to manually review and check every single release of every single package but I don't think this happens on APT either. Also, distribution package managers are often outdated and take a lot of time to update. Many APT packages do not have their latest version published, even if it exists on other package repositories. This means that we would need to wait a long time to get new features. And sometimes, if you use a bit old distribution, they are just not updated. And using distribution package managers would require to set things for each of package managers. There are many Linux distributions so this would require a lot of work. Pip, on the other hand, is platform-independent and you don't need to configure packages for each platform separately. And you will still need to use yet another thing for Windows and macOS. So, distro-specific managers are not a solution. The proper solution is to use distro-agnostic managers (AppImage, Flatpack, Snap) where they are available and continue using Pip where they are not. |
Right. What we want to do is ensure no one else, but the package author has modified its code. The transport layer security is a nice plus in case the package manger itself has a RCE bug, but this is besides the point for code integrity. I guess we are on the same page about AppImage which is a good sign. We know how to deal with those and they suit us. |
|
Malicious code has been found on pypi https://www.zdnet.com/article/twelve-malicious-python-libraries-found-and-removed-from-pypi/ but Debian and Ubuntu have been running insecure apt for years https://www.guardicore.com/labs/a-vulnerability-in-debians-apt-allows-for-easy-lateral-movement-in-data-centers-2/ The only solution if you want a secure system is to run Gentoo which has a package manager that supports signing the original source with the originator's key and builds from source. I can't believe in the day of state MITM that ZeroNet ships unsigned releases or binaries - they all should be signed with an originator's key. All the ZeroNet forks @TwinLizzie @canewsin @geekless and releases @HelloZeroNet should be signed and the should make the effort to be supported on Gentoo. leycec/raiagent#101 |
|
The bug in CVE-2019-3462 was introduced in 2009, so taking 10 years to fix it is "promptly fixed" on Debian terms? Last time I looked at apt in Debian wasn't even using https:// for reasons they made famous. And I'm certain that Debian and Devuan do not use secure curl with protection against SSL protocol downgrades. or redirects, to download debs, even though the susceptibility of SSL to these attacks has been well know for a decade or more. Debian is famous in professional security circles for having refused to address these weaknesses, and it is assumed at this point to be intentional. |
|
Please use words and sentences, as just posting links that are 5+ years out of date with regard to the point being made, is not helpful. To recall, the discussion was about a package manager that supports signing the original source with the originator's key. |
It doesn't matter when the issue was introduced. You haven't shown evidence that it was introduced 10 years ago either. If a vulnerability stays undetected for years, then no bad intention can be deducted. Once CVE-2019-3462 was reported, CVE-2019-3462 was promptly fixed.
Long time since you didn't look then.
Link to bug report(s) required.
Citation required.
By how that bug reports went, I don't expect anything has changed in 5+ years. An interested reader that cares about the gentoo package manager security is free to check if it's resolved meanwhile and/or to rehash the discussion.
Since I opened the ticket...
Since Whonix is based on Debian... Then went off-topic by making it about Debian vs Gentoo package manager security. So for this issue to be resolved... Well, see original post. |
|
So if you'r requesting packaging for Gentoo, upload to Gentoo... I suggest opening your own ticket. Though, not sure how much sense that makes due to #2326. But any of the forks my have a higher chance to implement this. |
|
The decision on a Gentoo overlay was to drop ZeroNet entirely leycec/raiagent@e3cbb3b leycec/raiagent#101 My suggestion for you is to change the title to "please do not depend on python-pip3 or apt - security risk - use dependencies available from originator signed distributions" in case you wanted meaningful security for ZeroNet. @filips123 what Gentoo is doing is integrating the originator's keys into its package management, and then distributing the orginator's signed source plus a Gentoo signed recipe to build from the source. Or for WhoNixxed distributions that are maintained using unsigned apt like KickSecure; it always struck me as curious that anything aiming to be "watertight" would use unsigned apt. There was a port of most of KickSecure plus to Gentoo, but it was abandoned we they realized there is no way of doing updates securely. ( The links to the "required" information you overlooked are in the links @filips123 posted.) |
|
A distribution with originator signed source code sounds nice. It's certainly a nice item to have for a (security) features checklist for a Linux distribution comparison. There's a lot other checklist items. Added here:
APT metadata isn't unsigned. It's signed. Can be easily seen here for example:
It even has a valid-until field which prevents indefinite freeze attacks and rollback attacks. https://wiki.debian.org/DebianRepository/Format Check out The Update Framework (TUF) threat model: It's really excellent. Years ago the security researchers from TUF found many security issues with package managers in many distributions (Debian among others) which have been fixed. Among the improvements was the valid-until field. https://www2.cs.arizona.edu/stork/packagemanagersecurity/attacks-on-package-managers.html As for Gentoo vs TUF, see above links in my previous posts. On top of the signed APT metadata, Debian offers downloads TLS and even onion. Onion provides an end-to-end (Tor to Tor) transport encryption alternative to TLS which is independent from any TLS certificate authorities. I guess your critical point here is basically that Debian isn't a source-based distribution. The question and answer here seems very much on point: Source packages for Debian that are signed by Debian Maintainers and uploaded to Debian are verified by buildd (build server) which then builds binaries which are then distributed with signed APT metadata. There's indeed no direct chain from the Debian Maintainer digital signature to the APT running on the user's computer verifying the package was signed by the maintainer's key. That's I guess what you mean by originator signed. Debian indeed isn't using the source-based distribution model. This affects all binary based distributions. The solution for that reproducible builds and it is being worked on. https://reproducible-builds.org/ This feature request to avoid python-pip3 is useful for both, Debian and Gentoo users.
Where?
We? They? Who?
All of Kicksecure source code is signed for years. All git commits are signed as well as all git tags are also signed. If there's something missing, please report a bug since I'd be interested to know and most likely would accommodate the request. APT isn't needed at all to acquire signed Kicksecure source code. |
Your links in the previous posts are years out of date, as is the trivial discussion of Gentoo in https://www.whonix.org/wiki/Dev/Operating_System. In the last 2 years, Gentoo has introduced originator signed source (and in some cases binary) packaging which is what I was referring to. It's a huge undertaking to have the package manager manage the originator keys. I think you can get part of the way there in Debian with signed-by= in sources.list.d, and it surprised me that KickSecure chose not to do this: https://forums.whonix.org/t/apt-seccomp-bpf-sandboxing/7702 |
|
emdee-net:
> That's I guess what you mean by originator signed.
No I don't mean maintainer signed; I mean by the originator of the package.
Any link for that?
I am reading
https://wiki.gentoo.org/wiki/Project:Portage/Repository_verification#How_does_it_work.3F
where it's written "Portage verifies the latest commit, which will be
signed automatically (not by a developer) ..."
> As for Gentoo vs TUF, see above links in my previous posts.
Your links in the previous posts are years out of date,
These things don't magically change. I've shown evidence that Gentoo
does not pass TUF. Kinda proving a negative.
Now you're claiming Gentoo has fixed it. Therefore the burden of proof
is on you.
But let's check again. The issue described in
Whonix/Gentoo-Port#19 /
https://bugs.gentoo.org/show_bug.cgi?id=539954 is still valid.
I went to the stage 3 folder.
https://gentoo.osuosl.org/releases/amd64/autobuilds/current-stage3-amd64-systemd/
There are signatures. But no valid-until field.
Since https://bugs.gentoo.org/show_bug.cgi?id=539954 is 7 years old, if
you like to see the security of Gentoo improved, you could ask if the
ticket could be re-opened and re-considered or if it's already resolved
meanwhile.
as is the trivial discussion of Gentoo in https://www.whonix.org/wiki/Dev/Operating_System. In the last 2 years, Gentoo has introduced originator signed source (and in some cases binary) packaging which is what I was referring to.
It's a huge undertaking to have the package manager manage the originator keys.
That's for sure. However, that's different from valid-until field / TUF.
I think you can get part of the way there in Debian with signed-by= in sources.list.d,
signed-by is different. signed-by can pin a deb repository line to a
specific signing key.
https://forums.whonix.org/t/apt-repository-signing-keys-per-apt-sources-list-signed-by/12302
and it surprised me that KickSecure chose not to do this: https://forums.whonix.org/t/apt-seccomp-bpf-sandboxing/7702
Kicksecure is doing that as per reply in that forum thread. Here's the
related git commit:
Kicksecure/security-misc@3558a99
|
|
I was surprised to see no signed-by: in /etc/apt/sources.list.d/whonix.list of KickSecure. You're reading Portage/Repository_verification which is assumed, and not what I'm talking about: I'm talking about originator signing.
Look in /usr/portage/sec-keys/ of any current gentoo. Or run "eix sec-keys/*" on any Gentoo. Maybe you can add a sentence about originator signed source (and in some cases binary) packaging in the Gentoo section of https://www.whonix.org/wiki/Dev/Operating_System . That would double you coverage of Gentoo there. (You also might make the point that you can also can avoid systemd too which is a real bonus, or for some, a requirement.) But in the case of Zeronet you might as well close this issue because there is no Linux distribution on the planet that contains the obsolete/abandoned/vulnerable prerequisite requirements of Zeronet (e.g. merkletools). Look at what a Gentoo overlay had to do to build a Zeronet leycec/raiagent@e3cbb3b#diff-86f2ac264ef17e6d26b9986736b59a61f52bf22d7cae7b73923e81f3c54cf045 Cecil eventually decided to drop ZeroNet completely. A more helpful issue is to request that abandoned or obsolete requirements in ZeroNet be replaced or vendored. See Cecil's suggestions for doing that in leycec/raiagent#101 But in the case of merkletools you would have to replace it as it is known to be vulnerable to attacking-merkle-trees-with-a-second-preimage-attack - see Tierion/pymerkletools#13 (comment) They never did update the README to disclose this. |
|
emdee-net:
I was surprised to see no signed-by: in /etc/apt/sources.list.d/whonix.list of KickSecure.
Kicksecure uses signed-by.
Kicksecure doesn't use whonix.list. /etc/apt/sources.list.d/whonix.list
is deprecated. Migrated. No longer in use.
In Kicksecure sources list that is file
/etc/apt/sources.list.d/derivative.list signed-by is being used:
To check:
cat /etc/apt/sources.list.d/derivative.list
The deb line will include:
deb [signed-by=/usr/share/keyrings/derivative.asc] ...
Link to source code where this is implemented:
https://github.com/Kicksecure/repository-dist/blob/master/usr/bin/repository-dist#L361
|
|
There's no /etc/apt/sources.list.d/derivative.list in my KickSecure and no signed-by in any of the sources.list.d files. Not a problem - there is no distribution I know of that ships merkletools (and coincurve and pyelliptic and others?) so to support for it with dependencies available from distribution package repositories is not known to be possible. |
|
emdee-net:
There's no /etc/apt/sources.list.d/derivative.list in my KickSecure and no signed-by in any of the sources.list.d files.
Probably an outdated version.
If already updated, any other files related to Whonix / Kicksecure in
/etc/apt/sources.list.d/ folder are safe to delete.
Then just run repository-dist to have
/etc/apt/sources.list.d/derivative.list created.
https://www.kicksecure.com/wiki/APT_Repository
If it created the old version, then it wasn't updated yet.
New downloadable releases use /etc/apt/sources.list.d/derivative.list
and signed-by by default.
|
Instructions on Linux currently require to run
sudo python3 -m pip install -r requirements.txt. Please consider to drop this if possible.python pip is a security risk since it does not verify software signatures on packages it downloads. It is also a third party package manager.
Previously a bitcoin wallet was hacked that depended on a third party package manager that didn't verify software signatures.
https://bitpay.com/blog/npm-package-vulnerability-copay/
Could you please use dependencies available from distribution package repositories such as packages.debian.org?
This is also a blocker for ever getting ZeroNet accepted into official packages.debian.org.
This is also a blocker for pre-installation of ZeroNet by default in Whonix.
Related packaging issues:
#382
#241
#1786
The text was updated successfully, but these errors were encountered: