v0.7.0

See the majority of changes in the RC1 release

New Since RC1

• Updated flatpak appstream metadeta to conform with new guidelines
• Fixed flatpak build manifest to work with new ESM module configuration

v0.7.0-RC1

This is a pre-release that makes some major changes to dependency versions and internal(ish) tooling but no changes to configuration or primary functionality.

What's New?

Improved Logging

Replaced logging solution, which was based on an outdated (100's of commits behind) fork of winston, with @foxxmd/logging:

• Built on top of Pino with modern code and fewer dependencies
• Adds coloring to all parts of the logs! Coloring is supported by console, docker logs, and UI
  • UI log coloring is rendered using existing log coloring (ansi)
  • Labels and error stacktraces are dimmer to make main messages stand out more
• (BREAKING) Log timestamp format has been changed to be more human-friendly IE 2024-03-21 16:18:35.520 -0400
• (BREAKING) The filenames for log files has been changed from scrobble-2024-03-22.log.X to scrobble-2024-03-22.X.log

Bumped Minimum Node Version

To support simpler testing with Mocha the minimum node version required for development and testing has been bumped from v18.16.0 to v18.19.1 in order to enable --import usage.

Bug Fixes and Improvements

Several bug fixes and general improvements are also included in this release. Specifically, improvements for subsonic sources and duplicate scrobble matching. See the full changelog below for all changes.

Full Changelog

Bug Fixes

• (flatpak) Add missing developer name
• (flatpak) Reduce summary length
• (webscrobbler) Fix missing wildcard to match slug route #137
• (No Category) Handle when no existing scrobble match with score greater than 0 #130
• (No Category) Fix ts schema generation import issue
• (No Category) Remove alt single quote when normalizing strings
• (No Category) Refactor static arrow function class fields to normal functions #143
• (No Category) Add comment for future typed linting

Documentation

• (No Category) Fix which host debian image is available for

Features

• (ui) Use ansi-to-react for log coloring
• (No Category) Add legacy fallback options for subsonic communication #136
• (No Category) Log more URL when route is unknown
• (No Category) Add initial linting config
• (No Category) Enable colorizing docker log output

Miscellaneous Tasks

• (!) Upgrade Node to LTS for --import support
• (docker) Remove unused dependencies
• (No Category) Remove unused es6-error package
• (No Category) Upgrade string-sameness
• (No Category) Bump mocha version

Refactor

• (logging) (!) Replace winston with @foxxmd/logging
• (No Category) Replace dbus-next with dbus-ts #142

Testing

• (No Category) Add test that handles no existing scrobble match with score greater than 0 #130
• (No Category) Use tsx through mocha config

Ci

• (No Category) Implement test checks for PRs and image publishing

0.6.5

What's New?

Upstream Recent

Prior to 0.5.0 MS let you see "recently played" directly returned from a Source's upstream API (for those that have it IE spotify, lastfm, listenbrainz...) This was useful for determining if the Source's API was "behind" what had actually been played -- for instance sometimes Spotify's "recently played" can be delayed by minutes/hours which could result in backlogging not returning what you "know" was played more recently.

This is entirely a convenience for the user, rather than new functionality for MS, but had been missing as a feature. This release returns this feature to the frontend.

UI Improvements for Source-of-Truth

For some sources (Lastfm, Listenbrainz) MS always uses the recently played data as the source-of-truth for scrobbling to your clients, rather than MS's "player". This is due to information returned for "now playing" for these sources being less accurate than recently played data. This has functionally not changed but this discrepancy is now more clearly presented to the user in the UI via a ? tooltip icon.

Google Cast Stability Improvements

I have rewritten parts of the third-party library providing google cast (chromecast) functionality to be more stable and less prone to crashing the entire MS application. Reconnecting events should now be more reliable as well.

Last.fm Timeout Fixes

An issue causing requests to Last.fm to hang forever has been fixed and will now cause the request to timeout in a reasonable amount of time instead of freeze up MS #134

Existing Scrobble checks for non-english characters

A bug causing scrobbles with non-english character-symbols to potentially not match existing scrobbles from client has been fixed. #121

Docker Image Alternatives

An alternative debian-based docker image is now provided for x86/x64 host architectures through a *-debian flavored tag. This image may be used as a workaround if you experience sporadic networking issues with the regular docker images. #126

Full Changelog

Features

• (chrome) Used improved library
• (lastfm) Implement nowPlaying lfm api call
• (source) Implement recent from API
• (sources) Improve Source of Truth usage and presentation to user #134
• (No Category) Add debug logging for user api requests
• (No Category) Improved logging for last.fm api network errors #115
• (No Category) Implement debian-based image #126

Bug Fixes

• (frontend) Fix scss division deprecation warnings
• (lastfm) Patch lfm library to actually implement a request timeout #134
• (scrobbler) Fix erasing non-english characters #121
• (No Category) Add patch-package as postinstall script so packages actually get patched

Documentation

• (No Category) Add workarounds for rollup issues and flatpak disclaimer
• (No Category) Add debian docker image network workaround to docs #126 #134

Miscellaneous Tasks

• (No Category) Remove some unused packages
• (No Category) Ignore apple cert extensions
• (No Category) Make container image build tags agnostic to default branch name
• (No Category) Add matrix for building debian flavor tag

v0.6.4

What's New?

Google Cast Source

Support for Google Cast devices as Sources. This is particularly exciting because music platforms that do not have an API for exposing user Now Playing info (Pandora, Tidal, MixCloud, Soundcloud) can now be monitored if you play them through a Google Cast device. Additionally, since the Cast platform exposes generic Now Playing information virtually any app that plays music through Casting can now be monitored by MS.

Application tooling restructure and frontend platform migration

This is no functional change for end-users. This is an internal/development refactor but brings with it many benefits:

• Refactored project from cjs to esm (new module standard)
• Moved away from dead frontend platform (CRA) to Vite
  • Removed dependency on webpack
  • Removed api proxy required by CRA
  • Simplified app entry -- Vite is served through express middleware instead of having nodemon spawn separate processes for front/backend
• Refactored project to use tsx runtime wrapper -- no longer need to compile to js to run dev/prod
  • Side-effect of this is stack traces reported in prod now point to actual source files with correct lines numbers

The only visible changes are:

• a slight increase in docker image size due to bundling typescript/source files
• developers do not need to use API_PORT anymore -- use PORT for both dev/prod now.

Source init improvements

All sources have been refactored to have more granular init stages:

• build data -- any data that needs to be parsed (URLs for mopidy), built, or verified (credentials) happens here
• test connection -- if the source can be checked for reach-ability it is done here
• test auth -- if the source requires auth it is tested here

If any stage fails more descriptive errors are logged for the failure and the stage it failed at is also described, giving users more insight into why a source failed to start as well as improving debugging.

App Crash Logging

App-crashing errors should now be logged before the app crashes. The easiest way to capture these logs is to set file logging to warn log level so they are stored to file. In your file-based config.json`:

{
  "logging": {
    "level": "debug",
    "file": "warn"
  }
  // ...
}

Deprecations and Future Changes

Maloja

• Support for Maloja versions below 3.0.0 is now deprecated
  • Likely no one is using these versions as they are almost 2 years old but MS still supports them with additional logic that could (should) be simplified
• A warning is emitting if Maloja version is below 3.2.0 due to lack of support for scrobbling album information

Spotify Credentials

A future release will include a migration from the current spotify api library to the official spotify web sdk. The information stored for Spotify credentials in releases prior to 0.6.4 is insufficient and will require reauthentication by the user if, for example, the user upgrades directly from 0.6.2 -> 0.6.5+.

0.6.4 will store all required credential information needed for future releases. Simply upgrade to this version and wait a few hours for Spotify credentials to be refreshed.

Full Changelog

Features

• (chrome) Improve mdns discovery for docker
• (chrome) Implement user filtering media by type
• (chrome) Allow manual device connections
• (chrome) Refactor mdns discovery to run on source heartbeat as well as startup
• (chrome) Catch buffering media
• (deezer) Fix credentials check during build data stage
• (maloja) Improve error handling
• (spotify) Prepare for future spotify lib migration by writing completing credential data
• (subsonic) Improve error/response parsing and report errors correctly on startup #129
• (No Category) Improve discovery for network exceptions in error causes
• (No Category) Improve data init stage log wording
• (No Category) Log when heartbeat tasks start

Bug Fixes

• (No Category) Actually throw auth failure so initialization fails correctly
• (No Category) Fix isReady logic
• (No Category) Fix status indicator color based on status text
• (No Category) Fix status text based on initialization OK values
• (No Category) Fix return values for source init startup
• (No Category) Fix when to check connection
• (No Category) Fix where retries are checked
• (No Category) Better handling of uncaught exceptions and rejections

Documentation

• (No Category) Chromecast docs and schema
• (No Category) Add local source reference to flatpak manifest

Miscellaneous Tasks

• (chrome) Switch to scoped chromecast-client to fix flatpak source generation limitation
• (maloja) Deprecate support for versions below 3.0.0

v0.6.3

Changelog

Features

• (player) Handle state transfer for single-platform sources #124
• (player) Improved player deletion behavior
• (scrobbler) Change fuzzy diff threshold to be inclusive
• (scrobbler) Use play data to extract non-joined track name when comparing for duplicate check
• (scrobbler) Improve log readability for duplicate checking
• (scrobblers) Implement user-initiated (re)start mechanism #114
• (source) Improve (re)start behavior and ui
• (webscrobbler) Log the best-guess URL for requests using configured slug
• (No Category) Implement play completion context #121
• (No Category) Improve track credits parsing to recognize post-feat suffixes #121

Bug Fixes

• (jellyfin) Split musicbrainz mbid artist identifier semicolon #114
• (lastfm) Fix handling of non-string mbid string values #114
• (listenbrainz) Treat 400 responses as non-showstopping #114
• (mopidy) Fix album artist parsing #110
• (player) Fix listen range default value
• (spotify) Undefined player state data when in private session - thanks @brunohpaiva
• (webscrobbler) Correct ENV names to match those documented #111

Refactor

• (scrobblers) Consolidate scrobbler client startup surface area
• (source) Start consolidating isReady behavior
• (No Category) Improve string comparison robustness
• (No Category) Move Play time comparison code into own file
• (No Category) Temporal comparison result data improvements #121

Documentation

• (webscrobbler) Add where to find connector list #112
• (webscrobbler) Move slug into data #111
• (No Category) Update schema for logging enums
• (No Category) Add contributing guidelines and PR template
• (No Category) Correct json-schema.app links - thanks @Bujiraso

Miscellaneous Tasks

• (tests) Fix TS compile warning on test for bad data

v0.6.2

What's New?

Album Artists

MS now tracks Album Artists and correctly scrobbles this info if the client supports it.

Targeted (Smarter) Polling for Play Durations #108

For polling sources that report track duration (Spotify), MS will now decrease polling interval near the beginning and end of the track in order to get more accurate "listened duration" and track start timestamp. NOTE: Spotify's Automix feature may interfere with this.

Changelog

Features

• (player) Pad scrobble listen duration when confidence is high #108
• (scrobbler) Use listenedFor when checking for existing scrobbles
• (source) Smart interval calculation based on track position and end time #108
• (No Category) Allow disabling console and file logging through ENV
• (No Category) Refactor auth testing to differentiate between network issues and actual auth errors #102
• (No Category) Implement album artist and scrobble client payload function
• (No Category) Implement album artist for all sources #104
• (No Category) Add app version output

Refactor

• (No Category) Simplify network error detection during auth test
• (No Category) Don't stringify error when using upstream options

Bug Fixes

• (lastfm) Fix api error when listening history is private by sending session key #105
• (maloja) Fix missing return value for auth test
• (player) Correctly output value of play timestamps for player based on undefined or not #107
• (plex) Use album artist if present and remove "Various" artist #104
• (source) Fix polling interval drift
• (spotify) Improve api error handling #103
• (No Category) Test auth correctly on auth flow completion
• (No Category) Catch errors during source backlog processing #103

Documentation

• (spotify) Add note about duration accuracy when using Spotify's Automix feature

Miscellaneous Tasks

• (flatpak) Update metainfo and sources
• (No Category) Update string-sameness version

v0.6.1

What's New?

Webscrobbler Source

The Webscrobbler browser extension has been added as a source using its native webhooks behavior.

Display and Controls for Queued/Dead Letter Scrobbles

Quietly implemented in 0.6.0, MS has two new control structures for making scrobbling more resilient to offline clients and network issues:

• Queued Scrobbles -- When a Source discovers a new Play it is added to a queue for each Client that should scrobble it. If the Client fails to scrobble due to networking or "showstopping" errors from the downstream API then it stopped processing the queue until the next heartbeat. This ensures all scrobbles are preserved until the downstream service is ready to accept scrobbles again.
• Dead Letter Scrobbles -- If the downstream service fails to scrobble for reasons that are expected by MS (EX Last.fm's "ignored" response) then the Scrobble is added to the Dead Letter Queue, a queue of failed scrobbles.
  • The queue is retried a couple times on heartbeat to account for ephemeral issues downstream.
  • If scrobbles still fail the MS user can at least see what failed, when, and why. The queue display in the dashboard also allows users to retry scrobbling manually.

Changelog

Features

• (No Category) Implement WebScrobbler source
• (No Category) Implement dead letter endpoints
• (No Category) Add dead letter error to data
• (No Category) Log storage improvements and limit controls
• (No Category) Implement retry/remove all dead letter scrobbles
• (No Category) Implement live updates for dead letter and scrobble counts
• (No Category) Implement live client/source status updates
• (No Category) Implement queued scrobbles display

Bug Fixes

• (No Category) Deadletter endpoint responses

Documentation

• (webscrobbler) Add disclaimer about hostname for firefox
• (No Category) Add queued scrobble feature highlight

Refactor

• (No Category) Reduce noisy logging at INFO level

v0.6.0

What's New?

UI Improvements and Now Playing

When a Source player state is detected a "Now Playing" player will render in the Source card showing how MS tracks plays for scrobbling.

The UI has been reworked to reduce whitespace, compact the layout, use headers more effectively, and improve wording/visualization.

Recently played tracks are now linked to their source (Spotify, Maloja) if MS can parse them.

Backend Tests and Scrobbling Fixes

A suite of tests have been added to test core backend functionality. These tests helped surface multiple bugs that should, now fixed, reduce duplicate scrobbles and improve consistency for all sources/clients.

Other Notable Improvements and Fixes

• Backlog on startup is configurable #100
  • In a source config options property use "scrobbleBacklog": false to disable backlogging on startup
• Enable/disable configs via property
  • In a source/client config at the top level use "enable": false" to disable using that config
• Scrobbling deferred until track finishes playing or is changed
  • This ensures an accurate "listened to" time is recorded for the scrobble, rather than scrobbling as soon as "listened to" time is valid
  • "listened to" is now sent in Maloja scrobble payload
• Jellyfin endpoint now accepts more header content-type values #101
• Updated youtube source library to work better with non-chromium browsers #98

Breaking Changes

Docker Image Base Updated (Linuxserver.io)

• Updates alpine 3.17 -> 3.18
    * Main reason is to potentially fix DNS issues related to musl #88
    * Bumps npm 9.1.2 -> 9.6.6

If you are using Portainer to manage containers you should remove all non-MS related ENVs from the container template before pulling the latest image.

Increased Scrobble Threshold Defaults #73

Previously, the default settings for scrobbling a track specified to scrobble if it was played for more than 30 seconds. This has been increased to:

• Played more than 240 seconds (4 minutes) OR
• More than 50% of the track played (for sources that support reporting track duration)

If you have scrobbleThresholds set in config.json or in individual source configs this does not affect you.

Increased Default Polling Interval

Due to the above change the default polling interval for sources have been adjusted like so:

• Default Interval: 30s -> 10s
• Max Interval: 60s -> 30s

This provides more accurate "listened to" recording and, generally, better source tracking resolution. This should not adversely affect your usage of each provider other than increasing bandwidth a little. These default settings can be overridden using the interval and maxInterval properties in individual source configs.

Changelog

Bug Fixes

• (backend) Fix how port is determined and used for localUrl
• (deezer) Fix deezer passport generation
• (docker) Normalize ownership of node_modules files
• (flatpak) Add missing release node
• (jellyfin) Enable parsing requests with more content-types and add logging on invalid types
• (player) Fix some bugs and add param for easier testing
• (player state) Fix stale/dead logic checks
• (scrobble) Fix config destructing default value
• (scrobble) Fix wrong artists used for comparison
• (scrobble) Fix not using reference duration for fuzzy comparison
• (scrobblers) Test auth when client completes auth callback
• (ui) Use normal match parameters
• (ui) Log line parsing missing line terminator capture
• (ui) Fix wrong route for client auth #97
• (ui) Remove duration percent display when duration is zero
• (ui) For non-polling sources use last activity date for status
• (youtube) Use updated yt music library #98
• (No Category) Load variables from .env for backend
• (No Category) Pass localUrl to scrobble clients for use by lastfm
• (No Category) Fix missing destructure default value

Documentation

• (jellyfin) Add request header workaround for jellyfin #101
• (No Category) Update dashboard screenshot
• (No Category) Clarify subsonic api compatibility

Features

• (config) Implement property to control whether a source/client is used
• (jellyfin) Initial testing suite
• (lastfm) Add player support
• (listenbrainz) Add player support #74
• (maloja) Generate web links for scrobbled tracks
• (maloja) Add listened duration to scrobble payload
• (player) Add sourceOfTruth variable to control logging and future use
• (player) Defer play discovery so listenedFor is accurate
• (player) Log when play fails discovery on player change
• (player) Implement seek and repeat detection
• (player state) Preserve position when player is not in stopped status
• (player state) Implement player status and listened duration in UI
• (scrobble) Use new string approach for track
• (scrobble) Use new string approach for artists
• (scrobble) Implement artist dup bonus scoring heuristic
• (scrobble) Initial scrobble queue implementation
• (scrobble) Implement dead letter queue processing
• (scrobble) Implement heartbeat task for scrobblers
• (scrobble) Improve existing scrobble comparison logic
• (source) Control backlogging via config #100
• (ui) Remove player when dead
• (ui) Compact ui
• (ui) Make player dates human friendly
• (ui) Remove unnecessary wording for track stats
• (ui) Expand displayed plays
• (ui) Implement status indicator
• (No Category) Rework source api to include player states
• (No Category) BREAKING Set default scrobble thresholds to standard values #73
• (No Category) BREAKING Decrease default polling interval
• (No Category) Implement Base URL
• (No Category) Implement new string comparison logic

Miscellaneous Tasks

• (docker) BREAKING Bump lsio base image version to 3.18
• (No Category) Use constant for default retry multiplier
• (No Category) Remove unused dashboard backend endpoint
• (No Category) Ensure production mode for webpack without relying on env

Refactor

• (player) Move listen progress/range functionality into own class (SRP)
• (player state) Refactor handling of non-updated players
• (player state) Remove old play logic
• (scrobble) Print score breakdown in different statement
• (scrobble) Adjust dup score weights
• (scrobble) Refactor scrobble delay and scrobble processing sleep behavior
• (ui) Prevent nodemon from restarting on tailwind config change

Testing

• (player) Add player tests
• (scrobble) Implement basic scrobble tests
• (scrobble) Add more tests for similar-but-unique scrobble detection
• (scrobble) Add test for duplicate when exact title/time
• (scrobble) Improve test durability by using mixed duration sources
• (scrobble) Add tests for scrobble processing queue and existing comparisons

v0.5.2

What's New?

MS now has a scheduled "heartbeat" that runs every 20 minutes. It's purpose, for now, is to is start (or restart) sources that have stopped due to upstream issues. The use-case for a Source is:

• request/polling retry attempts are low
• source was initially OK (auth OK, initial communication OK)
• and there is a non-auth related upstream communication issue

The heartbeat tasks will automatically try to restart polling for this Source so MS can recover from an upstream API communication error without user intervention.

Changelog

Features

• (No Category) Schedule heartbeat task to restart sources that should be polling #6 #65 #93

Bug Fixes

• (listenbrainz) Artist parsing joiner phrases must begin with non-word character #94

v0.5.1

Changelog

Bug Fixes

• (tautulli) fix redirect path
• (plex/jellyfin) fix redirect allowing relative paths that breaks url when source endpoint has a trailing slash #92