Coverage misses lines

[iamed2]

For this file: https://github.com/invenia/Memento.jl/blob/master/src/stdlib.jl

The coverage report is:

        - if VERSION > v"0.7.0-DEV.2980"
        -     import Base.CoreLogging:
        -         AbstractLogger,
        -         handle_message,
        -         min_enabled_level,
        -         shouldlog,
        -         global_logger,
        -         Debug
        - 
        -     const LEVEL_MAP = Dict{AbstractString}
        -     struct CoreLogger <: AbstractLogger
        -     end
        - 
        -     min_enabled_level(logger::CoreLogger) = Debug
        4     shouldlog(logger::CoreLogger, arags...) = true
        - 
        -     function handle_message(::CoreLogger, cl_level, msg, mod, group, id, filepath, line; kwargs...)
        2         logger = getlogger(mod)
        -         level = lowercase(string(cl_level))
        2         log(logger, logger.record(logger.name, level, getlevels(logger)[level], msg))
        -     end
        - 
        -     function substitute!()
        1         global_logger(CoreLogger())
        -         notice(getlogger(@__MODULE__), "Substituting global logging with Memento")
        -     end
        - else
        -     function substitute!()
        -         warn(
        -             getlogger(@__MODULE__),
        -             "Global logging substitution is not support for julia $VERSION"
        -         )
        -     end
        - end
        - 

Notice that level = lowercase(string(cl_level)) is between two covered lines but somehow isn't reported as covered.

[vtjnash]

Now is perhaps a good time to update LineInfoNode to track line-number ranges in the parser (but might be hard still for the optimizer to handle updating this information). And see also https://github.com/JuliaLang/julia/pull/11802/files

Perhaps we should inject this information before optimization (like we are doing at the llvm level), as this'll problem will only get worse as our optimizer gets better. We could more-easily do it then at the basic-block level. This would be more likely to require recompiling the system image to switch the flag though.

[maxbennedich]

We are seeing the same thing starting in Julia 0.7. Every Julia 0.7 and 1.0 project with test coverage I've seen suffer from the same thing. I've tried with --inline=no but it's still incorrect for us.

[navidcy]

Any updates on this issue?
I haven't been able to use Coveralls or Codecov.io since v0.7. This was actually very useful in developing tests for my packages. Without that I have no way to find which lines of codes are actually covered. Is there a way to do that and I'm missing it out?

[sbromberger]

I don't know whether this is related, but LightGraphs just dropped to from 99.8% to 66% codecov (and the reported coverage gaps don't make sense).

[matteoacrossi]

I have the same problem in one of my packages. It simply misses lines, as in the OP.
Any news on this?

[fingolfin]

To clarify, this issue is certainly amplified by recent changes to https://github.com/JuliaCI/Coverage.jl These are my "fault", although I "only" enabled code that was used on Julia 0.6 to also work correctly on Julia 0.7 and 1.0, which made the issue much more visible. Some discussions on mitigating the problem:

• Coverage information does not seem to be right JuliaCI/Coverage.jl#187
• Let amend_coverage_from_src! skip partially covered code (WIP) JuliaCI/Coverage.jl#188
• Turn of inlining when computing coverage (WIP) Pkg.jl#866
• Base.julia_cmd() should propagate --inline=no #29855