Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Switch tarpaulin to llvm engine #2432

Closed
wants to merge 1 commit into from

Conversation

RageKnify
Copy link
Contributor

Trying this to see if it makes coverage numbers more accurate

@RageKnify RageKnify added the Internal Category for changelog label Nov 13, 2022
Trying this to see if it makes coverage numbers more accurate
@github-actions
Copy link

Test262 conformance changes

Test result main count PR count difference
Total 93,815 93,815 0
Passed 69,562 69,562 0
Ignored 18,422 18,422 0
Failed 5,831 5,831 0
Panics 0 0 0
Conformance 74.15% 74.15% 0.00%

@codecov
Copy link

codecov bot commented Nov 13, 2022

Codecov Report

Merging #2432 (1a6bb60) into main (fdac8ec) will increase coverage by 13.17%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main    #2432       +/-   ##
===========================================
+ Coverage   38.86%   52.03%   +13.17%     
===========================================
  Files         316      320        +4     
  Lines       24123    34262    +10139     
===========================================
+ Hits         9375    17829     +8454     
- Misses      14748    16433     +1685     
Impacted Files Coverage Δ
boa_engine/src/vm/opcode/value/mod.rs 50.00% <0.00%> (-50.00%) ⬇️
boa_engine/src/context/icu.rs 54.54% <0.00%> (-25.46%) ⬇️
boa_engine/src/vm/opcode/push/mod.rs 75.00% <0.00%> (-25.00%) ⬇️
boa_engine/src/vm/opcode/push/numbers.rs 80.00% <0.00%> (-20.00%) ⬇️
boa_engine/src/vm/opcode/return_stm/mod.rs 7.40% <0.00%> (-17.60%) ⬇️
...a_engine/src/vm/opcode/binary_ops/macro_defined.rs 85.71% <0.00%> (-14.29%) ⬇️
boa_engine/src/vm/opcode/new/mod.rs 75.67% <0.00%> (-13.22%) ⬇️
boa_engine/src/vm/opcode/try_catch/mod.rs 82.05% <0.00%> (-10.81%) ⬇️
boa_engine/src/builtins/map/mod.rs 76.21% <0.00%> (-10.31%) ⬇️
boa_engine/src/value/integer.rs 58.97% <0.00%> (-10.26%) ⬇️
... and 302 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@RageKnify
Copy link
Contributor Author

Both engines seem to mostly suffer from false negatives, this engine seems to considers 10139 more lines. Looking at the codecov.io report the differences are a little weird so I'm not sure how to feel.
Just because of this one reporting on more lines I tend to preffer it, but I want to hear others' opinion on this.
CC: @Razican @raskad @jedel1043 @HalidOdat @jasonwilliams

@raskad
Copy link
Member

raskad commented Nov 14, 2022

After reading the relevant issue in tarpaulin, my understanding is that the long term plan is to use llvm instead of ptrace. From that point of view it makes sense to me to switch over. I looked trough the report diff a bit and to me it seems like the llvm engine is more accurate in the lines it reports a covered.

I am in favor of switching.

@jedel1043
Copy link
Member

jedel1043 commented Nov 14, 2022

I agree with @raskad. The LLVM engine also has the advantage of having more support for other architectures.

Copy link
Member

@HalidOdat HalidOdat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, switching seems like the right choice :)

Copy link
Member

@Razican Razican left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the change :) tarpaulin is moving in the good direction here. If we find false positives/negatives, we should report them.

@Razican
Copy link
Member

Razican commented Nov 14, 2022

Bors r+

bors bot pushed a commit that referenced this pull request Nov 14, 2022
Trying this to see if it makes coverage numbers more accurate
@bors
Copy link

bors bot commented Nov 14, 2022

Pull request successfully merged into main.

Build succeeded:

@bors bors bot changed the title Switch tarpaulin to llvm engine [Merged by Bors] - Switch tarpaulin to llvm engine Nov 14, 2022
@bors bors bot closed this Nov 14, 2022
@bors bors bot deleted the tarpaulin-llvm branch November 14, 2022 06:48
@RageKnify RageKnify added this to the v0.17.0 milestone Nov 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Internal Category for changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants