3.0.0 - Continuous Recordings

This release is a big one. It mainly focuses on continuous recordings, but also includes a lot of other features and fixes.

I have been working on this release for a long time and I am very happy with the result. I hope you will enjoy it as much as I do.
The release cadence has been very slow lately and for that I am sorry. I will try to make smaller releases more often in the future.

Breaking changes

• New docker volumes are required. Please see the updated docker documentation
• Old recordings will not be compatible with the new version and wont be viewable in the UI
• Files will no longer be stored in the /recordings folder, and should be removed from your docker volumes
• Recordings are now in HLS format. If you need to have the recordings saved as full MP4 files you can use the new config option create_event_clip, or use the new Export feature in the UI.
• mp4 file extension is now forced and extension is removed
• segments_folder is removed in favor of tier paths from the new storage component
• Thumbnails are now stored in /thumbnails folder (depending on the tier config)
• filename_pattern in thumbnail is removed, stored as ID of the recording
• audio_codec behavior changed. It is now only used as stream information. To transcode Audio, set audio_codec under recorder
• trigger_recorder is removed in favor of trigger_event_recording
• The FFmpeg command has been changed, so if you use raw_command you need to update it

New features

24/7 recordings

The main feature included in this release is the option of having 24/7 recordings.
With this comes a new component, storage which will allow you to retain data based on time (like it is today) and based one consumed space.
This allows you to use your storage to its maximum.

Documentation for how the recordings work can be found here

Updated screenshots for the frontend can be found here

Storage Tiers

Another feature of the storage component are tiers.
They allow you to store recordings on multiple media.
Say you have one 256GB SSD and a 1TB HDD, you can keep the most recent recordings on the SSD for fast access while offloading the older recordings to the HDD for archiving.

Timeline

To easily view 24/7 recordings, a new Timeline view is available.
It currently shows continuous recordings and different events such as motion, detected objects, face recognition etc.

Events

The Events view is a more concise view than the Timeline.
Here you can scroll through and view all the Events.

Other new features

• A new safe mode has been added. If Viseron cant startup due to config errors, it will restore the last known good config and start up with a minimum amount of components. This will allow you to fix the config in the UI and restart Viseron without having to manually restore the config.
• Allow empty password for cameras
• A logfile is now created in the /config folder. This will allow you to see the logs without having to enter the container.
• New telegram component provided by @smelis, which allows sending of clips and thumbnails using Telegram.
• New ptz component provided by @smelis, which can be used by the Telegram component to control a camera.
  The PTZ functionality will be ported to the web UI as well at some point.
• max_recording_time has been added to the recorder component. This will allow you to set a maximum recording time for each recording. This is useful if you have a lot of motion events and want to keep the recordings short.
• record_only has been added to the recorder component. This will allow you to record without running any detectors.
• still_image_width and still_image_height has been added to the camera component. This will allow you to set the resolution of the still images.

Changes

• A PostgreSQL database is now running inside the container. It is not possible to use an external database at the moment.
• Upgraded to Ubuntu 22.04 which gives updated drivers to support newer hardware
• Updated to React 18
• Updated to Python 3.10
• Updated to OpenCV 4.10.0
• Updated to FFmpeg 5.1.2
• Updated CUDA
• Updated DLIB
• Updated OpenCL
• Updated VA-API
• Updated all Python dependencies
• The documentation has been updated with a lot of new chapters

Fixes

• Camera passwords are now escaped, meaning that passwords with special characters are now supported.
  This has been a long standing issue which caused FFProbe to fail when trying to connect to the camera.
• Properly redact sensitive information in logs
• Reduce the occurence of KeyError: UUID in the logs
• Fix a long standing bug where frames with drawn items (like bounding boxes, masks etc) would bleed through to other frames.

Docker images will be on Docker Hub shortly

v3.0.0b13 - Final beta

This beta will be the last one before the release of 3.0.0, unless some big error is found.
The release focuses on stability improvements and polishing of existing features.
Some new features are also included.

New documentation chapters have also been added which aims to explain how recordings and Viseron in general is working.
New sections will be added in the future to highlight common usecases.
https://dev--viseron.netlify.app/docs/documentation/configuration

Breaking changes

• The /recordings folder is no longer used, and should be replaced with the /event_clips folder.
  This namechange was made in order to clear up some confusion on what this folder is storing in v3.
  Docker instructions have been updated: https://dev--viseron.netlify.app/docs/documentation/installation#running-viseron
• trigger_recorder has been deprecated in favor of trigger_event_recording

New features

• Faces to recognize can now be pulled from Compreface directly
• A logfile is now created in the config directory. Docs have been updated accordingly: https://dev--viseron.netlify.app/components-explorer/components/logger#view-logs
• You can now download events and snapshots from the Events page.
  You can also specify a timespan for 24/7 recordings which will be downloaded as an MP4 file.
  Documentation can be found here: https://dev--viseron.netlify.app/docs/documentation/configuration/recordings#downloading-recordings
• New config option record_only which makes it possible to create continuous recordings without running any type of detectors.
• New config option width and height for still_image which allows setting the resolution of the still image
• You can now click individual events in the popup windows on the Events/timeline view to seek to the event
• New telegram component provided by @smelis, which allows sending of clips and thumbnails using Telegram.
• New ptz component provided by @smelis, which can be used by the Telegram component to control a camera.
  The PTZ functionality will be ported to the web UI as well at some point.

What's Changed

• add empty config folder to allow devcontainer to start by @roflcoopter in #843
• remove devcontainer config mount by @roflcoopter in #846
• pin types-requests to 2.31.0.6 by @roflcoopter in #847
• added new functionality to use compreface subjects for facial recognition by @tantonj in #845
• Documentation updates by @roflcoopter in #850
• add logging to file by @roflcoopter in #852
• wait for move_on_shutdown to complete by @roflcoopter in #853
• allow dates with only recorded video to be selected by @roflcoopter in #854
• use yield_per to avoid cursor closing in _check_tier by @roflcoopter in #856
• dont materialize when using yield_per by @roflcoopter in #857
• fetch all rows to avoid cursor closing by @roflcoopter in #858
• Avoid importing dlib when it is not used by @roflcoopter in #865
• Allow download of events and snapshots by @roflcoopter in #869
• check for nan value when clicking the LIVE button by @roflcoopter in #870
• reload hls every 5 seconds on fatal errors by @roflcoopter in #871
• reload page when getting a vite preloadError by @roflcoopter in #872
• use getCameraNameFromQueryCache in Breadcrumbs by @roflcoopter in #873
• apply object detection mask the same way as motion detection by @roflcoopter in #874
• properly cleanup after test_setup_components by @roflcoopter in #875
• add category and subcategory to OrphanedRecordingsCleanup query by @roflcoopter in #876
• allow pressing enter to login by @roflcoopter in #877
• remove future=True from sessionmaker by @roflcoopter in #878
• add screenshots and videos to event download instructions by @roflcoopter in #879
• change bash to shell in docs codeblocks by @roflcoopter in #880
• Continuous recording config parameter by @roflcoopter in #881
• Async Websocket API by @roflcoopter in #883
• improve shutdown sequence and make sure all threads are stopped by @roflcoopter in #884
• Doc typos and FFmpeg timestamp example by @roflcoopter in #885
• Update docker.mdx - --env-file vs --env by @nneul in #887
• fix ffmpeg log pipe closing by @roflcoopter in #888
• remove files_meta table to avoid costly joins by @roflcoopter in #890
• add gitsponsors to README by @roflcoopter in #891
• record_only config option by @roflcoopter in #892
• fix mjpeg stream exception by @roflcoopter in #893
• further asyncio event loop improvements by @roflcoopter in #894
• upgrade mypy to 1.14.1 by @roflcoopter in #895
• Mount config folder in repo to /config in devcontainer by @roflcoopter in #896
• fix so that opening drawer doesnt rerender page by @roflcoopter in #897
• add config option to set still image resolution by @roflcoopter in #898
• remove gitsponsors by @roflcoopter in #899
• add error boundaries to frontend by @roflcoopter in #900
• Events and timeline navigation improvements by @roflcoopter in #904
• deprecate trigger_recorder in favor of trigger_event_recording by @roflcoopter in #905
• Component explorer link fix by @roflcoopter in #911
• Rename /recordings to /event_clips by @roflcoopter in #912
• add direct navigation to timeline by @roflcoopter in #913
• allow overrides for each snapshot domain by @roflcoopter in #916
• PTZ and Telegram Components by @smelis in #788
• Fixes to the new PTZ and Telegram components by @roflcoopter in #918
• update images and descriptions for ptz and telegram by @roflcoopter in #919

New Contributors

• @tantonj made their first contribution in #845
• @smelis made their first contribution in #788

Full Changelog: v3.0.0b12...v3.0.0b13

v3.0.0b12 - Bugfixes and cleanup jobs

This release includes a multitude of fixes and changes to improve the stability of Viseron.
It also adds multiple clean up jobs to keep the database and filesystem in sync in case of any crashes.

If you have tried the v3 beta before and experienced issues, i highly suggest trying this release out since it contains loads of fixes.

Fixes

• Remove volumes from the Dockerfile to avoid issues with bind mounts
• Use the process watchdog for Darknet so that it is restarted in case of crashes
• Add extra logging to shutdown procedure to pinpoint threads and processes that is slowing it down
• Allow empty passwords for cameras
• Run every thread using RestartableThread so that it gets awaited during shutdown
• Fixes the move_on_shutdown: true config option. Previously Viseron did not wait for the callback to finish before exiting.
• A condition was missing when searchng for files for move_on_shutdown causing files to be moved multiple times
• Delete from FilesMeta table manually instead of using a trigger
• Properly throttle the function that checks if any files should be moved. This cause issues with attempting to move files multiple times
• Event clips are now generated in the servers timezone instead of UTC

Cleanup jobs

Multiple cleanup jobs now run regularly to make sure to keep the database and filesystem clutter free.
For instance if you delete files manually while Viseron is not started, the cleanup jobs will make sure that they are removed from the database.

They are intentionally throttled to run quite slow to not degrade performance of your host.

• OrphanedFilesCleanup (Mondays, 3 AM)

  • Removes files that are no longer associated with any recordings or events

• OrphanedDatabaseFilesCleanup (Wednesdays, 3 AM)

  • Cleans up database entries that point to non-existent files

• OrphanedFilesMetaCleanup (Fridays, 3 AM)

  • Removes metadata for files that no longer exist

• EmptyFoldersCleanup (Daily at midnight)

  • Removes empty folders to keep storage structure tidy

• OrphanedThumbnailsCleanup (Daily at midnight)

  • Removes thumbnail images that are no longer linked to any recordings

• OrphanedClipsCleanup (Daily at midnight)

  • Removes video clips that are no longer associated with any events

• OrphanedRecordingsCleanup (Daily at midnight)

  • Removes recordings that are no longer linked to any events

• OrphanedPostProcessorResultsCleanup (Daily at midnight)

  • Removes post-processing results that are no longer needed

• OrphanedObjectsCleanup (Daily at midnight)

  • Removes detected object data that is no longer associated with any events

• OrphanedMotionCleanup (Daily at midnight)

  • Removes motion detection data that is no longer linked to any events

• OldEventsCleanup (Daily at midnight)

  • Removes expired system events based on retention settings

For discussions, please use this discussion: #721

Full Changelog: v3.0.0b11...v3.0.0b12

v3.0.0b11 - Timeline player syncing

This release mainly contains new functionality to keep all the players on the Event/Timeline views in sync.
It can currently be a bit unclear if a stream is paused or not when it gets out of sync due to missing segments, that will be improved in the next beta.

Many performance improvements have also been made:

• Fetching available timespans is now done over websocket instead of through the API which reduces the amount of requests made significantly.
• If many events came close together it could cause stalls since each event would trigger a refetch from the API. These refetches are now debounced to avoid this.
• The entries in the event table is now virtualized (and the thumbnails only load when you stop scrolling).
  This significantly reduces lag and sluggishness of the table when having lots of events displayed.

And lastly, React has been upgrade to version 18, along with MUI v6 and React Query v5.
Togheter these upgrades should also equate to some performance improvements.

Details of all changes

• Sync all active players on the Events/Timeline page
• Upgrade to React 18
• Upgrade to MUI v6
• Upgrade to react-query v5
• Update timespans over using websockets
• Use system event listeners to refetch event queries
• Custom video controls
• Pick correct stream format config for substreams
• Massive performance increase for the Event table when having many events

For discussions, please use this discussion: #721

Full Changelog: v3.0.0b10...v3.0.0b11

v3.0.0b10 - Streaming improvements

This release is focused mainly on a new layout for the Events page.
You can now select multiple cameras at once and they will be shown in a grid. Camera streams will automatically resize to give as much coverage as possible.
Selecting cameras is done using the floating action button in the bottom right corner.
The streams are not yet properly synchronized. I will work on that for a coming beta.
Here is a short demo of the grid:

There is also a new filter button located next to the Events/Timeline tabs that allows you to filter out only the relevant event types.

Details of all changes

• Adds filters to the Events and Timeline tabs so you can filter out only the events you want (Motion, object, ALPR etc)
• Upgrades HLS.js
• Fix calling loadSource twice when viewing 24/7 recordings, resulting in a browser console error
• Generate HLS segments directly from ffmpeg instead of writing mp4 files and manually fragmenting.
  This significantly improves the stream quality and removes some choppyness.
• Make Events page more compact by removing the camera selection grid. Selecting cameras is done using the floating action button in the bottom right corner
• Never delete a frame from memory if it is still the current frame.
  Should reduce some intermittent errors in the logs
• Use the ffmpeg hls_flags program_date_time option to get the exact creation time for segments. Previously milliseconds were discarded causing some timing issues
• Use settings under recorder for transcoding the generated segments
• Events page now allows selection of multiple cameras and shows them in a grid with a dynamic layout based on the screen size
• FFmpeg now deletes segments on its own in case of a Viseron crash
• RegEx performance improved to avoid hanging on large m3u8 playlists
• Improve speed of SQL queries
• Camera cards now has status text and indicators showing various states
• Frontend optimizations
• Fix missing sahred frames of motion and object detecors
• Enable safe mode if config fails to load (if a secret is missing for instance)
• Other minor bug fixes

For discussions, please use this discussion: #721

What's Changed

• Fix small doc typo by @nneul in #782
• Update NVIDIA link on installation doc by @afaulconbridge in #786
• rm sponsor by @gitsponsored in #784
• Events page and streaming improvements by @roflcoopter in #793
• Properly delete segments by @roflcoopter in #796
• Events page multiple camera layout and stability improvements by @roflcoopter in #802

New Contributors

• @nneul made their first contribution in #782
• @afaulconbridge made their first contribution in #786

Full Changelog: v3.0.0b9...v3.0.0b10

v3.0.0b9 - EdgeTPU fix

• Fix for edgetpu crashing (#721 (comment))

For discussions, please use this discussion: #721

Full Changelog: v3.0.0b8...v3.0.0b9

v3.0.0b8 - License plate recognition events

• OpenCV 4.10.0
• Annotate images with supervision for nicer looking text and bounding boxes
• Fix HLS playback bug when not using auth
• Darknet on CPU now runs in a separate python process
• Store License plate recognition snapshots
• Show License plate recognition in the timeline/events tabs
• Store Motion detection snapshots
• Show Motion detection in the events tab
• Correctly calculate event amounts in the date picker
• Allow running object and motion detectors with trigger_recorder: true at the same time without using scan_on_motion_only: true or recorder_keepalive

For discussions, please use this discussion: #721

Full Changelog: v3.0.0b7...v3.0.0b8

v3.0.0b7 - Face recognition events

Changes

• Show Face Recognition events on the Events and Timeline tabs
• Events on the Events tab is now grouped to avoid clutter
• Fix auth check always being enabled when viewing the timeline
• Escape camera passwords
• Add better logic for redacting sensitive information from the logs
• New config option check_interval for each storage tier to control the minimum time between moving/deleting files
• Allow running without any detectors to only store continuous recordings
• Upgrade all python dependencies

For discussions, please use this discussion: #721

Full Changelog: v3.0.0b6...v3.0.0b7

v3.0.0b6 - GStreamer fix and VAAPI support

Fixes

• Small fix to GStreamer. A bug was introduced when upgrading to Ubuntu 22.04 which caused pipelines to hang
• Support for VAAPI decoding of streams in GStreamer

Full Changelog: v3.0.0b5...v3.0.0b6

v3.0.0b5 - Delete unwanted files instead of moving

Fixes

• Make sure to instantly delete files instead of moving them to succeeding tiers that wont store them
• Crossbuild gpac so it works on aarch64, jetson-nano etc
• Make sure jetson-nano finds the correct postgresql version

Full Changelog: v3.0.0b4...v3.0.0b5