Add cargo-deny check to CI #2713

emilk · 2023-03-02T13:52:11Z

cargo-deny is an amazing tool that protects from:

duplicated crates (code bloat)
copy-left licenses in the dependency tree
RUSTSEC advisories

Try it:

cargo install cargo-deny
cargo update
cargo deny --all-features --log-level error check

(EDIT: removed wrong stuff about rustsec and duplicated deps due to me using an outdated Cargo.lock)

Tested on all platforms changed
Compilation warnings were addressed
cargo fmt has been run on this branch
cargo doc builds successfully
Added an entry to CHANGELOG.md if knowledge of this change could be valuable to users
Updated documentation to reflect any user-facing changes, including notes of platform-specific behavior
Created or updated an example program if it would help users understand this functionality
Updated feature matrix, if new features were added or implemented

kchibisov · 2023-03-02T13:57:21Z

wayland-cursor v0.29.1

I think it tested minimal versions? I'm not sure how we can get wayland-cursor v0.29.1 if we have

winit/Cargo.toml

Line 112 in 79ac236

    
           wayland-client = { version = "0.29.5", default_features = false,  features = ["use_system_lib"], optional = true }

. In my lockfile it's 0.29.5 for example, it's a family of wayland crates, they have the same version.

copy-left licenses in the dependency tree

That's a good idea to ensure, but new deps are always rechecked manually (at least I try to) and ensure we don't add some crap.

wayland-cursor v0.29.1

Duplicate on nix is related to the way you test, though, there should be duplicate on nix, because calloop uses nix 0.25, but wayland-client is at 0.24

emilk · 2023-03-02T14:11:07Z

I'm not sure how we can get wayland-cursor v0.29.1

Like so:

   = nix v0.22.0
     └── wayland-cursor v0.29.1
         └── smithay-client-toolkit v0.16.0
             ├── sctk-adwaita v0.5.2
             │   └── winit v0.28.1
             └── winit v0.28.1 (*)

So we need a new smithay-client-toolkit to fix the duplication of nix.

kchibisov · 2023-03-02T14:15:26Z

So we need a new smithay-client-toolkit to fix the duplication of nix.

No, if you run cargo update -p wayland-cursor you'll get wayland-cursor 0.29.5.

That's for example my output of cargo tree from winit:

kchibisov@Zodiac:winit-workspace/fork [ sync-0282] ┃ cargo tree
winit v0.28.2 (/home/kchibisov/src/rust/winit-workspace/fork)
├── bitflags v1.3.2
├── instant v0.1.12
│   └── cfg-if v1.0.0
├── libc v0.2.139
├── log v0.4.17
│   └── cfg-if v1.0.0
├── mio v0.8.5
│   ├── libc v0.2.139
│   └── log v0.4.17 (*)
├── once_cell v1.17.0
├── percent-encoding v2.2.0
├── raw-window-handle v0.5.0
│   └── cty v0.2.2
├── sctk-adwaita v0.5.3
│   ├── ab_glyph v0.2.20
│   │   ├── ab_glyph_rasterizer v0.1.8
│   │   └── owned_ttf_parser v0.18.1
│   │       └── ttf-parser v0.18.1
│   ├── log v0.4.17 (*)
│   ├── memmap2 v0.5.8
│   │   └── libc v0.2.139
│   ├── smithay-client-toolkit v0.16.0
│   │   ├── bitflags v1.3.2
│   │   ├── calloop v0.10.5
│   │   │   ├── log v0.4.17 (*)
│   │   │   ├── nix v0.25.1
│   │   │   │   ├── bitflags v1.3.2
│   │   │   │   ├── cfg-if v1.0.0
│   │   │   │   ├── libc v0.2.139
│   │   │   │   └── memoffset v0.6.5
│   │   │   │       [build-dependencies]
│   │   │   │       └── autocfg v1.1.0
│   │   │   │   [build-dependencies]
│   │   │   │   └── autocfg v1.1.0
│   │   │   ├── slotmap v1.0.6
│   │   │   │   [build-dependencies]
│   │   │   │   └── version_check v0.9.4
│   │   │   ├── thiserror v1.0.38
│   │   │   │   └── thiserror-impl v1.0.38 (proc-macro)
│   │   │   │       ├── proc-macro2 v1.0.49
│   │   │   │       │   └── unicode-ident v1.0.6
│   │   │   │       ├── quote v1.0.23
│   │   │   │       │   └── proc-macro2 v1.0.49 (*)
│   │   │   │       └── syn v1.0.107
│   │   │   │           ├── proc-macro2 v1.0.49 (*)
│   │   │   │           ├── quote v1.0.23 (*)
│   │   │   │           └── unicode-ident v1.0.6
│   │   │   └── vec_map v0.8.2
│   │   ├── dlib v0.5.0
│   │   │   └── libloading v0.7.4
│   │   │       └── cfg-if v1.0.0
│   │   ├── lazy_static v1.4.0
│   │   ├── log v0.4.17 (*)
│   │   ├── memmap2 v0.5.8 (*)
│   │   ├── nix v0.24.3
│   │   │   ├── bitflags v1.3.2
│   │   │   ├── cfg-if v1.0.0
│   │   │   ├── libc v0.2.139
│   │   │   └── memoffset v0.6.5 (*)
│   │   ├── wayland-client v0.29.5
│   │   │   ├── bitflags v1.3.2
│   │   │   ├── downcast-rs v1.2.0
│   │   │   ├── libc v0.2.139
│   │   │   ├── nix v0.24.3 (*)
│   │   │   ├── scoped-tls v1.0.1
│   │   │   ├── wayland-commons v0.29.5
│   │   │   │   ├── nix v0.24.3 (*)
│   │   │   │   ├── once_cell v1.17.0
│   │   │   │   ├── smallvec v1.10.0
│   │   │   │   └── wayland-sys v0.29.5
│   │   │   │       ├── dlib v0.5.0 (*)
│   │   │   │       └── lazy_static v1.4.0
│   │   │   │       [build-dependencies]
│   │   │   │       └── pkg-config v0.3.26
│   │   │   └── wayland-sys v0.29.5 (*)
│   │   │   [build-dependencies]
│   │   │   └── wayland-scanner v0.29.5
│   │   │       ├── proc-macro2 v1.0.49 (*)
│   │   │       ├── quote v1.0.23 (*)
│   │   │       └── xml-rs v0.8.4
│   │   ├── wayland-cursor v0.29.5
│   │   │   ├── nix v0.24.3 (*)
│   │   │   ├── wayland-client v0.29.5 (*)
│   │   │   └── xcursor v0.3.4
│   │   │       └── nom v7.1.2
│   │   │           ├── memchr v2.5.0
│   │   │           └── minimal-lexical v0.2.1
│   │   └── wayland-protocols v0.29.5
│   │       ├── bitflags v1.3.2
│   │       ├── wayland-client v0.29.5 (*)
│   │       └── wayland-commons v0.29.5 (*)
│   │       [build-dependencies]
│   │       └── wayland-scanner v0.29.5 (*)
│   │   [build-dependencies]
│   │   └── pkg-config v0.3.26
│   └── tiny-skia v0.8.3
│       ├── arrayref v0.3.6
│       ├── arrayvec v0.7.2
│       ├── bytemuck v1.12.3
│       ├── cfg-if v1.0.0
│       ├── png v0.17.7
│       │   ├── bitflags v1.3.2
│       │   ├── crc32fast v1.3.2
│       │   │   └── cfg-if v1.0.0
│       │   ├── flate2 v1.0.25
│       │   │   ├── crc32fast v1.3.2 (*)
│       │   │   └── miniz_oxide v0.6.2
│       │   │       └── adler v1.0.2
│       │   └── miniz_oxide v0.6.2 (*)
│       └── tiny-skia-path v0.8.3
│           ├── arrayref v0.3.6
│           ├── bytemuck v1.12.3
│           └── strict-num v0.1.0
├── smithay-client-toolkit v0.16.0 (*)
├── wayland-client v0.29.5 (*)
├── wayland-commons v0.29.5 (*)
├── wayland-protocols v0.29.5 (*)
└── x11-dl v2.20.1
    ├── lazy_static v1.4.0
    └── libc v0.2.139
    [build-dependencies]
    └── pkg-config v0.3.26
[build-dependencies]
├── cfg_aliases v0.1.1
└── wayland-scanner v0.29.5 (*)
[dev-dependencies]
├── image v0.24.5
│   ├── bytemuck v1.12.3
│   ├── byteorder v1.4.3
│   ├── color_quant v1.1.0
│   ├── num-rational v0.4.1
│   │   ├── num-integer v0.1.45
│   │   │   └── num-traits v0.2.15
│   │   │       [build-dependencies]
│   │   │       └── autocfg v1.1.0
│   │   │   [build-dependencies]
│   │   │   └── autocfg v1.1.0
│   │   └── num-traits v0.2.15 (*)
│   │   [build-dependencies]
│   │   └── autocfg v1.1.0
│   ├── num-traits v0.2.15 (*)
│   └── png v0.17.7 (*)
└── simple_logger v2.3.0
    └── log v0.4.17 (*)

And that's the duplicates

nix v0.24.3
├── smithay-client-toolkit v0.16.0
│   ├── sctk-adwaita v0.5.3
│   │   └── winit v0.28.2 (/home/kchibisov/src/rust/winit-workspace/fork)
│   └── winit v0.28.2 (/home/kchibisov/src/rust/winit-workspace/fork)
├── wayland-client v0.29.5
│   ├── smithay-client-toolkit v0.16.0 (*)
│   ├── wayland-cursor v0.29.5
│   │   └── smithay-client-toolkit v0.16.0 (*)
│   ├── wayland-protocols v0.29.5
│   │   ├── smithay-client-toolkit v0.16.0 (*)
│   │   └── winit v0.28.2 (/home/kchibisov/src/rust/winit-workspace/fork)
│   └── winit v0.28.2 (/home/kchibisov/src/rust/winit-workspace/fork)
├── wayland-commons v0.29.5
│   ├── wayland-client v0.29.5 (*)
│   ├── wayland-protocols v0.29.5 (*)
│   └── winit v0.28.2 (/home/kchibisov/src/rust/winit-workspace/fork)
└── wayland-cursor v0.29.5 (*)

nix v0.25.1
└── calloop v0.10.5
    └── smithay-client-toolkit v0.16.0 (*)

emilk · 2023-03-02T14:16:33Z

Oh I see - I have an old Cargo.lock file 🤦‍♂️

emilk · 2023-03-02T14:24:33Z

So one can fix the nix duplication with:

> cargo update
> cargo update -p calloop --precise 0.10.2                                            
    Updating crates.io index
    Updating calloop v0.10.3 -> v0.10.2
    Removing nix v0.25.1

but since winit doesn't have a Cargo.lock checked in (nor should it), it's very hard to get cargo-deny to understand this.

kchibisov · 2023-03-02T14:29:04Z

Yes, but calloop fixes some critical bugs, so we can't downgrade.

emilk · 2023-03-02T14:32:15Z

No - so I've told deny.toml to ignore the nix duplication. Let's see if this gets a green light from CI…

emilk · 2023-03-02T14:57:02Z

So - I find cargo-deny very useful, and use it in all my big projects. It can be configured in deny.toml to ignore problems (e.g. the duplicated nix crate), so it should never become a hard blocker for anything.

But - YMMV :)

madsmtm · 2023-03-02T15:44:51Z

It would actually be beneficial for our CI if Cargo.lock was checked in, since that would allow us to cache cargo stuff

kchibisov

In general that looks good to me.

The nix issue will be resolved once #2676 is merged (will take a while though).

.github/workflows/ci.yml

deny.toml

emilk · 2023-03-11T14:44:17Z

By adding more targets, we find more duplicate crates:

❯ cargo deny --all-features --log-level error check
error[duplicate]: found 2 duplicate entries for crate 'foreign-types'
   ┌─ /Users/emilk/code/forks/winit/Cargo.lock:37:1
   │  
37 │ ╭ foreign-types 0.3.2 registry+https://github.com/rust-lang/crates.io-index
38 │ │ foreign-types 0.5.0 registry+https://github.com/rust-lang/crates.io-index
   │ ╰─────────────────────────────────────────────────────────────────────────^ lock entries
   │  
   = foreign-types v0.3.2
     ├── cocoa v0.24.1
     │   └── crossfont v0.5.1
     │       └── sctk-adwaita v0.5.3
     │           └── winit v0.28.1
     ├── cocoa-foundation v0.1.0
     │   └── cocoa v0.24.1 (*)
     ├── core-graphics v0.22.3
     │   ├── cocoa v0.24.1 (*)
     │   ├── core-text v19.2.0
     │   │   └── crossfont v0.5.1 (*)
     │   ├── crossfont v0.5.1 (*)
     │   └── winit v0.28.1 (*)
     ├── core-graphics-types v0.1.1
     │   ├── cocoa-foundation v0.1.0 (*)
     │   └── core-graphics v0.22.3 (*)
     └── core-text v19.2.0 (*)
   = foreign-types v0.5.0
     └── crossfont v0.5.1
         └── sctk-adwaita v0.5.3
             └── winit v0.28.1

error[duplicate]: found 2 duplicate entries for crate 'foreign-types-shared'
   ┌─ /Users/emilk/code/forks/winit/Cargo.lock:40:1
   │  
40 │ ╭ foreign-types-shared 0.1.1 registry+https://github.com/rust-lang/crates.io-index
41 │ │ foreign-types-shared 0.3.1 registry+https://github.com/rust-lang/crates.io-index
   │ ╰────────────────────────────────────────────────────────────────────────────────^ lock entries
   │  
   = foreign-types-shared v0.1.1
     └── foreign-types v0.3.2
         ├── cocoa v0.24.1
         │   └── crossfont v0.5.1
         │       └── sctk-adwaita v0.5.3
         │           └── winit v0.28.1
         ├── cocoa-foundation v0.1.0
         │   └── cocoa v0.24.1 (*)
         ├── core-graphics v0.22.3
         │   ├── cocoa v0.24.1 (*)
         │   ├── core-text v19.2.0
         │   │   └── crossfont v0.5.1 (*)
         │   ├── crossfont v0.5.1 (*)
         │   └── winit v0.28.1 (*)
         ├── core-graphics-types v0.1.1
         │   ├── cocoa-foundation v0.1.0 (*)
         │   └── core-graphics v0.22.3 (*)
         └── core-text v19.2.0 (*)
   = foreign-types-shared v0.3.1
     └── foreign-types v0.5.0
         └── crossfont v0.5.1
             └── sctk-adwaita v0.5.3
                 └── winit v0.28.1

error[duplicate]: found 2 duplicate entries for crate 'redox_syscall'
   ┌─ /Users/emilk/code/forks/winit/Cargo.lock:88:1
   │  
88 │ ╭ redox_syscall 0.2.16 registry+https://github.com/rust-lang/crates.io-index
89 │ │ redox_syscall 0.3.4 registry+https://github.com/rust-lang/crates.io-index
   │ ╰─────────────────────────────────────────────────────────────────────────^ lock entries
   │  
   = redox_syscall v0.2.16
     └── orbclient v0.3.43
         └── winit v0.28.1
   = redox_syscall v0.3.4
     └── winit v0.28.1

emilk · 2023-03-11T14:46:31Z

Interestingly, foreign-types 0.5 was release three years ago, so one would think cocoa would have had time to update

kchibisov · 2023-03-11T14:49:27Z

@emilk but that looks simply wrong. Because sctk-adwaita is a wayland only dependency, yet you're showing me that it's in the graph for macOS. (it can't even build on macOS)

kchibisov · 2023-03-11T14:50:23Z

cc @jackpot51 wrt redox duplicates.

emilk · 2023-03-11T15:46:37Z

Ah, I think I'm maybe finally understanding how cargo-deny acts wrt target. If you specify a --target (e.g. in a matrix-config on the CI), it will check only that target. If you do not specify a --target it will check all targets at once, i.e. as if you were compiling for all targets at the same time, which of course is never going to happen. So I think we need to switch back to several calls to cargo check with different --target for it to work as expected.

Ah yes, here is the related issue: EmbarkStudios/cargo-deny#324

emilk · 2023-03-11T16:01:19Z

So this remains:

 error[duplicate]: found 2 duplicate entries for crate 'redox_syscall'
   ┌─ /github/workspace/Cargo.lock:25:1
   │  
25 │ ╭ redox_syscall 0.2.16 registry+https://github.com/rust-lang/crates.io-index
26 │ │ redox_syscall 0.3.4 registry+https://github.com/rust-lang/crates.io-index
   │ ╰─────────────────────────────────────────────────────────────────────────^ lock entries
   │  
   = redox_syscall v0.2.16
     └── orbclient v0.3.43
         └── winit v0.28.2
   = redox_syscall v0.3.4
     └── winit v0.28.2

Because of https://gitlab.redox-os.org/redox-os/orbclient/-/issues/46

I'll ignore it for now

kchibisov

Could you rebase, I hope that should fix the CI.

emilk · 2023-03-11T18:31:26Z

Seems like CI is running latest Rust, which means 1.68.0, which means new clippy errors.

emilk · 2023-03-11T18:37:47Z

Clippy fix: #2729

kchibisov · 2023-03-16T20:05:49Z

Thanks!

emilk mentioned this pull request Mar 2, 2023

Rerun PRs rerun-io/opensource#2

Open

emilk marked this pull request as ready for review March 2, 2023 14:57

kchibisov requested changes Mar 11, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

.github/workflows/ci.yml Outdated Show resolved Hide resolved

deny.toml Outdated Show resolved Hide resolved

kchibisov approved these changes Mar 11, 2023

View reviewed changes

emilk force-pushed the cargo-deny branch from c1b0f46 to 804198c Compare March 11, 2023 18:19

emilk added 8 commits March 12, 2023 19:38

Add cargo-deny check to CI

b9e2f5e

fix ci

28f43c4

Try again to get cargo-deny running on CI

813f99d

allow duplicates of nix crate

896d0b9

Check for copy-left licenses

6917f88

encourage running cargo update before cargo deny check

9d12d75

Properly use the target fields

bb0b903

Go back to a matrix on CI

f1c7d88

skip redox_syscall

23da80b

emilk force-pushed the cargo-deny branch from c160683 to 23da80b Compare March 12, 2023 18:38

kchibisov merged commit fbea75d into rust-windowing:master Mar 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add cargo-deny check to CI #2713

Add cargo-deny check to CI #2713

emilk commented Mar 2, 2023 •

edited

Loading

kchibisov commented Mar 2, 2023 •

edited

Loading

emilk commented Mar 2, 2023 •

edited

Loading

kchibisov commented Mar 2, 2023

emilk commented Mar 2, 2023

emilk commented Mar 2, 2023

kchibisov commented Mar 2, 2023

emilk commented Mar 2, 2023

emilk commented Mar 2, 2023

madsmtm commented Mar 2, 2023

kchibisov left a comment

emilk commented Mar 11, 2023

emilk commented Mar 11, 2023

kchibisov commented Mar 11, 2023 •

edited

Loading

kchibisov commented Mar 11, 2023

emilk commented Mar 11, 2023

emilk commented Mar 11, 2023 •

edited

Loading

kchibisov left a comment

emilk commented Mar 11, 2023

emilk commented Mar 11, 2023

kchibisov commented Mar 16, 2023

Add cargo-deny check to CI #2713

Add cargo-deny check to CI #2713

Conversation

emilk commented Mar 2, 2023 • edited Loading

kchibisov commented Mar 2, 2023 • edited Loading

emilk commented Mar 2, 2023 • edited Loading

kchibisov commented Mar 2, 2023

emilk commented Mar 2, 2023

emilk commented Mar 2, 2023

kchibisov commented Mar 2, 2023

emilk commented Mar 2, 2023

emilk commented Mar 2, 2023

madsmtm commented Mar 2, 2023

kchibisov left a comment

Choose a reason for hiding this comment

emilk commented Mar 11, 2023

emilk commented Mar 11, 2023

kchibisov commented Mar 11, 2023 • edited Loading

kchibisov commented Mar 11, 2023

emilk commented Mar 11, 2023

emilk commented Mar 11, 2023 • edited Loading

kchibisov left a comment

Choose a reason for hiding this comment

emilk commented Mar 11, 2023

emilk commented Mar 11, 2023

kchibisov commented Mar 16, 2023

emilk commented Mar 2, 2023 •

edited

Loading

kchibisov commented Mar 2, 2023 •

edited

Loading

emilk commented Mar 2, 2023 •

edited

Loading

kchibisov commented Mar 11, 2023 •

edited

Loading

emilk commented Mar 11, 2023 •

edited

Loading