Add option to set default value for habits on unknown days

[KristianTashkov]

Implements: #125

This change propagates to a lot of places so list of things:
Implemented

1. History card can have a default square depending on the default value
2. Scores can now start from values above 0.
3. Main screen can start with completed habits, but they are not filtered unless you specifically enter the value.

Things to do left

• Happy to get guidance of the order of numerical EditHabit dialog and the titles for the new value to make it prettier
• Score is mostly consistent, but score chart starts from the first repetition. Do we want to change that?
• Streaks are not calculated for default value YES. Are we okay with them being calculated from the first "actual" occurence?
• Frequency/History/Total in the habit page is inconsistent with these default values too
• I expect some tests to crash, but I prefer some guidance on the above before I continue
• We probably need to add a lot of new tests to cover this functionality as well

@iSoron Waiting on some guidance for all of this before I continue cause it's a lot and don't want to spend more effort if you don't like where this is going.

[hiqua]

Streaks are not calculated for default value YES. Are we okay with them being calculated from the first "actual" occurrence?

This seems to indicate that with the current implementation, if I create a new habit today with a default value, this default value will be applied to all the previous days (yesterday, etc.). I don't think that should be the case, IMHO the default value should apply to every day starting from the creation date, if possible.

Other things:

• the UI could make it easier to select YES / NO, it should be some kind of toggle button, instead of two options to choose from.
  I would rename the box to "Check (or tick) by default", with gray background if not, and highlighted if yes, or something like this.
• we need to check how this interacts with the question mark option

[KristianTashkov]

the UI could make it easier to select YES / NO, it should be some kind of toggle button, instead of two options to choose from.

I would rename the box to "Check (or tick) by default", with gray background if not, and highlighted if yes, or something like this.
there is a third option if you have SKIP enabled which is why I made it a toggle button. It's not shown in the screenshot above since it's disabled by default.

we need to check how this interacts with the question mark option

In what sense? I've left the question mark working, but it does have color if by default you complete the habit.

[KristianTashkov]

@iSoron now that AT_MOST are merged I need to start updating this PR to reflect it as well. Can you go over the questions I have so I have all the information what you want implemented out of this feature for this PR?

[iSoron]

Hi @KristianTashkov, could you please clarify what feature request is this PR trying to address? It's still unclear to me why is this feature needed. Is this #125? If so, note that is only applies to YES/NO habits, instead of numerical ones.

[KristianTashkov]

As long as you enable a default value for YES/NO, I see no reason to not have the same functionality for numerical habits as well. If you are wondering about use cases:

1. The current AT_MOST that starts at 100% isn't intuitive IMO, if I set myself a new habit the prior is that i'm failing at it, not that I'm already perfect. I would want to set the default at some failing value so I can start from 0% as usual.
2. You can create habits where by default on unknown days you get a value different than 0, (similarly to wanting an automatic YES by default for YES/NO) and to decrease them based on behavior during the day.
3. This doesn't work as well with the current implementation, but the number picker starting from 0 every time is also unwanted in some cases, you can set it to something closer to the real values you submit every day and make input easier

[rootmeanclaire]

As long as you enable a default value for YES/NO, I see no reason to not have the same functionality for numerical habits as well.

It seems like this PR was originally designed to address #125, so maybe additional functionality, even if it's related, should be in a separate pull request.

[hiqua]

@iSoron what do you think of @KristianTashkov's points? FWIW I could use the default numeric values myself, and I think the AT_MOST example is a good one (thinking of cigarettes and you want to reduce gradually). I could also imagine people wanting to track their sleep, being fairly stable and wanting to record deviations from their average more easily.

[iSoron]

Hi @hiqua, I have not taken any action on this feature because there are still many open questions to me. At this point, I'm still not sure if the added complexity (both in the code and in the UI) justifies adding this feature.

1. Let's assume that the user can provide a default value when creating a habit. How do we use this value? Do we assume that all UNKNOWN entries have the default value? Or do we actually create an entry at the start of the day with the default value? There are problems with both alternatives.
2. If we don't create any actual entries, but we simply assume that UNKNOWN entries have this default value:
   1. How do we distinguish YES_MANUAL and UNKNOWN in the user interface, when "show question marks" is enabled or disabled? On one hand, this distinction is important to let the user know that some data is missing. On the other hand, making UNKNOWN different from YES_MANUAL could make the UI less satisfying (many users like to see the main screen filled with colorful checkmarks; if the default checkmarks are different, this effect is lost).
   2. How do we display UNKNOWN in the calendar chart? We already have too many shades/patterns. Do we add yet another one? Now the chart is becoming really hard to interpret.
   3. Changing the default value would cause all scores and streaks to change. Sure, this already happens when we change the habit frequency, but that's also undesirable, and now we are making the problem worse.
3. If the app actually creates new entries:
   1. When exactly do we create new entries? When the user launches the main activity? What if they interact with the app mainly through widgets? It seems like now we would need a background task to add entries daily, which adds complexity and points of failure, since background tasks are notoriously unreliable on Android.
   2. When this event triggers, what entries exactly do we add? One for each UNKNOWN in the entire habit history? Then how can the user override it? Just the UNKNOWNs added since the event was last triggered? Then now we need to keep track of that, which adds complexity.

Answering @KristianTashkov questions:

The current AT_MOST that starts at 100% isn't intuitive IMO, if I set myself a new habit the prior is that i'm failing at it, not that I'm already perfect. I would want to set the default at some failing value so I can start from 0% as usual.

Currently, AT_MOST starts at 100% score because: (i) the default value is zero; and (ii) we assume that all missing entries (including the ones before the habit was created) has the default value. If this feature is implemented and the user sets a default value to a large number (beyond the habit target), then the score should start at 0%. If the suggestion, however, is to allow both a low default value, and low starting score, I don't think we can do it consistently.

You can create habits where by default on unknown days you get a value different than 0, (similarly to wanting an automatic YES by default for YES/NO) and to decrease them based on behavior during the day.

OK, our understanding is the same.

This doesn't work as well with the current implementation, but the number picker starting from 0 every time is also unwanted in some cases, you can set it to something closer to the real values you submit every day and make input easier

This could also be implemented at the UI layer, by having the picker default to the last entered value. No changes to the database, or to the data classes, are required. This solution doesn't have any of the issues above, and we could easily merge it (in a separate PR).

[hiqua]

1. we create a new entry at the start of the day / once the app is opened
2. i. At a high-level, I would have a task running whenever the app is open, which would keep track of some timestamp t. Set all unknown entries between the previously recorded timestamp and the new one to the default value.

In pseudo-code, more or less:

last_t = null
while true {
  if(last_t!=null) {
    for (day : daysBetween(last_t, current_t) {
      check(day, default)
    }
  }
  last_t = current_t
}

[iSoron]

At a high-level, I would have a task running whenever the app is open, which would keep track of some timestamp t. Set all unknown entries between the previously recorded timestamp and the new one to the default value.

A few questions/comments about this solution:

1. We also need to run this at midnight, otherwise the widgets would not update correctly.
2. What happens when the user adds a default value to a habit that previously didn't have any? Should we retroactively replace all unknowns? I'm concerned that, if the user accidentally enters an incorrect value, that mistake cannot be easily reversed; the app would quickly "corrupt" the entire habit history with automatically generated entries. This is particularly troublesome for boolean habits, since I assume that most users only have YES_MANUAL and UNKNOWN entries (not many NO entries). Perhaps a better idea is to also update last_t for all habits that don't have a default value?
3. For boolean habits, are we allowing three default values (YES, NO, UNKNOWN) or just two (YES, NO)? If it's the first option, how do we handle users that have "show question marks" disabled? If the second, do we actually add NO entries, or just leave them as unknown?

[hiqua]

Should we retroactively replace all unknowns?

No

For boolean habits, are we allowing three default values (YES, NO, UNKNOWN) or just two (YES, NO)?

In the UI, the default value can only be set to YES or NO. Removing the default value in the UI does what's done currently, meaning that the default value is actually UNKNOWN.

Perhaps a better idea is to also update last_t for all habits that don't have a default value?

It can even be a global task iterating over all habits with just one last_t common to all habits.

To summarize, I suggest that the default value implementation is just like the user setting the default value as early as possible, but automatically. So no change in the underlying data model.

[iSoron]

It can even be a global task iterating over all habits with just one last_t common to all habits.
To summarize, I suggest that the default value implementation is just like the user setting the default value as early as possible, but automatically. So no change in the underlying data model.

I think that could work. A PR implementing it would be very welcome.

[hiqua]

@KristianTashkov do you have all your answers / are you fine with the suggestions above?

[KristianTashkov]

Hey @hiqua . At this point this PR is very stale and I would probably have to redo it anyway so I'm closing this.
For the last discussion you had, I'm not sure I'm on board with this design of default value so if this is the decision the app is taking I won't be the one implementing it. Sorry!
Also it seems that this new design doesn't cover the main use case I cared about, having the ability to have AT_MOST numerical habit to start at 0% (By specifying that by default it's a failure).

[hiqua]

having the ability to have AT_MOST numerical habit to start at 0%

I don't know how it is currently, but I think we should consider all unknown entries as failing for numerical habits, which would solve this problem.

[iSoron]

I don't know how it is currently, but I think we should consider all unknown entries as failing for numerical habits, which would solve this problem.

Currently, all UNKNOWN entries are treated (and displayed in the UI) as zero. For AT_MOST habits, that's a success, so AT_MOST habits currently start at 100% score, which is what @KristianTashkov dislikes. I'm also not sure that the current behavior is the best.

[hiqua]

I'm also not sure that the current behavior is the best.

Exactly, I'm advocating for a failing value which depends on the habit goal.