Inconsistency between single-arg MvNormal and Normal

[sethaxen]

On Distributions v0.24.1, the mean-only constructor for Normal defaults to a variance of 1:

julia> using Distributions

julia> Normal(0)
Normal{Float64}(μ=0.0, σ=1.0)

while the mean-only constructor for MvNormal defaults to a covariance of 0:

julia> MvNormal(zeros(2))
ZeroMeanDiagNormal(
dim: 2
μ: 2-element Zeros{Float64}
Σ: [0.0 0.0; 0.0 0.0]
)

It seems to me that by providing the mean only, the user would usually expect an identity covariance matrix, analogous to Normal.

[johnczito]

Apparently it's not a mean-only constructor. It's a "covariance"-only constructor:

Distributions.jl/src/multivariate/mvnormal.jl

Line 223 in 6a22a99

MvNormal(Σ::AbstractVecOrMat{<:Real}) = MvNormal(Zeros{eltype(Σ)}(size(Σ, 1)), Σ)

Given a vector it always sets the mean to zero and fills in the diagonal of the covariance, which is why you got a zero matrix in your case. Alternatively:

julia> MvNormal([1., 2.])
ZeroMeanDiagNormal(
dim: 2
μ: 2-element Zeros{Float64}
Σ: [1.0 0.0; 0.0 4.0]
)

I prefer MvNormal(x::AbstractVector) = MvNormal(x, I) to MvNormal(x::AbstractVector) = MvNormal(0, diagm(x.^2)), and that would be consistent with the univariate behavior, but I guess the current version isn't "wrong" if people like it.

[sethaxen]

Wow, that's even more surprising. I can see how that could make sense, but the inconsistency with Normal seems like it could introduce bugs in users' code.

e.g. in Turing when you have n variables iid drawn from the same Normal(0, 1) distribution, you get an efficiency boost by using MvNormal(zeros(n), 1). I encountered this issue by converting Normal(0) to MvNormal(zeros(n)) and watching my model go insane.

[devmotion]

e.g. in Turing when you have n variables iid drawn from the same Normal(0, 1) distribution, you get an efficiency boost by using MvNormal(zeros(n), 1).

Actually, if possible you should just specify MvNormal(n, 1) since internally Zeros (from FillArrays) is used in such cases.

BTW there is some discussion about the inconsistencies between Normal and MvNormal also in #584.

[mschauer]

A vector is a bad pun for a diagonal matrix, we should fade out that method.

[sethaxen]

A vector is a bad pun for a diagonal matrix, we should fade out that method.

In this case though the vector is not the diagonal of a diagonal covariance matrix. It's the corresponding standard deviations. If we drop the vector constructor, I'd argue we should also drop the other standard deviation constructor (MvNormal(μ, σ::Real)), in favor of the user explicitly using MvNormal(μ, σ^2*I).

[andreasnoack]

Let's continue this conversation in #826