How to remove noise from an observed sequence

[fenaux]

I try to remove noise from an observed sequence

The observed points are the blue points.
They correspond to team_id[in_track]

The desired behaviour corresponds to the orange curve with only one change around index 600
The I use
a = mc.HiddenMarkovModel().from_seq(team_id[in_track], team_id[in_track])
and get
a.tp

array([[ 0.94824, 0.018484, 0.033272],
[ 0.018692, 0.9486, 0.03271],
[ 0.12931, 0.2069, 0.66379]])

As I know
transition form state 0 to 1 or 1 to 0 are rare divide by 10 correspondings terms
transitions form state 0 or 1 to 2 should not occur corresponding terms are changed to 1e-9

Leading to
atp_hat

array([[ 0.99815, 0.0018484, 1e-09],
[ 0.0018692, 0.99813, 1e-09],
[ 0.12931, 0.2069, 0.66379]])

I then apply
vs, vsi = a.viterbi(obs_seq=team_id[in_track], tp=atp_hat)
but both vs and vsi are exactly indentical to blue points.

Is there a way to get what I expect ?

[maximtrp]

Looking at your case, the issue is that you're using the same sequence for both observations
and states in from_seq(), which creates an identity mapping. Here's how to achieve the noise
removal you want:

When you use from_seq(team_id[in_track], team_id[in_track]), the emission matrix becomes an
identity matrix where each state only emits itself. Even with modified transition
probabilities, Viterbi will still prefer the "correct" emissions.

Solution 1: Modify the Emission Matrix

import mchmm as mc
import numpy as np

# Train the model
a = mc.HiddenMarkovModel().from_seq(team_id[in_track], team_id[in_track])

# Your modified transition matrix
atp_hat = np.array([
    [0.99815, 0.0018484, 1e-09],
    [0.0018692, 0.99813, 1e-09],
    [0.12931, 0.2069, 0.66379]
])

# Create a "noisy" emission matrix that allows states to emit other observations
# This gives each state some probability of emitting other values
ep_smooth = np.array([
    [0.85, 0.10, 0.05],  # State 0 mostly emits 0, but can emit 1,2
    [0.10, 0.85, 0.05],  # State 1 mostly emits 1, but can emit 0,2
    [0.15, 0.15, 0.70]   # State 2 mostly emits 2
])

# Run Viterbi with both modified matrices
vs, vsi = a.viterbi(obs_seq=team_id[in_track], tp=atp_hat, ep=ep_smooth)

Solution 2: Use Baum-Welch with Constraints

# Start with your constrained transition matrix
hmm = mc.HiddenMarkovModel().from_baum_welch(
    team_id[in_track],
    states=[0, 1, 2],
    tp=atp_hat  # Use your modified transitions as initialization
)

# The algorithm will learn emission probabilities that account for noise
vs, vsi = hmm.viterbi(team_id[in_track])

Solution 3: Post-processing with Median Filter

from scipy.signal import medfilt

# Apply median filter to smooth the sequence
smoothed = medfilt(team_id[in_track], kernel_size=5)

Try solution 1 first—adjusting the emission probabilities to allow "emission noise" should
give you the smoothing behavior you're looking for.