Cadmus

Added

• SSO support with Google SSO PoC (frontend)
• accessibility (frontend)
• firebase notifications (for mobile applications)
• every API call now has a rate limit. The limit is per API endpoint, so it is not a global counter. The default value is 20 requests per 15 minutes. The user has to define 2 extra environmental variables:
  • LIMIT_WINDOW specifies the time window in ms.
  • LIMIT_REQUESTS specifies the number of allowed requests in that secific time window.
• GRIP API integration. When there is 1 or more GRIP events related to the ARTEMIS event, a dropdown menu with the event ids is added to the hijack page. When clicked, a new tab to grip event page is opened.
• signup and Login forms now include a captcha field for bot protection. The user has to additionally define the following .env variables:
  • CAPTCHA_SECRET. This would preferably be a long random hash value.
  • CAPTCHA_WINDOW specifies the time where the false login attempt count is valid.
  • CAPTCHA_TRIES is the number of unsuccessful login attempts that are needed to trigger the CAPTCHA generation.
• new env var ARTEMIS_WEB_BASE_DIR specifies the ARTEMIS web base directory (default: empty).
• condition to limit GRIP events association to 1 hour time window w.r.t. ARTEMIS alerts.
• a web command line search experience based on super-cmd. To trigger the search modal:
  • Windows + Linux OS: ctrl + Windows key (super) + k
  • MacOS : cmd + k
• logout alerts (frontend)
• live button in dashboard page (frontend)
• 401 notification for unauthorized users (frontend)
• visual testing on cypress (frontend)
• install_pybgpstreamv2_for_ubuntu.sh utility for pybgpstrem installation

Changed

• upgraded NGINX to 1.23
• upgraded RabbitMQ to 3.9.20
• (breaking) upgraded timescale DB to 2.8.1-pg14
• upgraded postgrest to v10
• upgraded hasura GraphQL engine v2.10.0
• changed bgpstream_retrieve_prefix_records.py to support BGPStream v2 (new version)
• enforce ssl_protocols TLSv1.2 TLSv1.3 in default nginx.conf
• bumped caida/bgpstream to 2.2.0 and monitor containers to related 1.0.3
• migrated our web application to latest Nextjs, React and Webpack
• updated artemis-utils to 1.0.14 (service name fix for docker-compose)
• password policy change
• increased size limit for POST /api/config to host larger configuration files
• removed process parallelism for internal reconfiguration tasks for thread-safety
• default frontend password for admin

Fixed

• Split prefixtree recalculation indicator and fixed prefix trie bug with v4/v6
• LDAP behavior and correct vars (frontend)
• Admin login bug (frontend)
• Bug with logout warning popup
• GQL ports in frontend
• login and JWT auth calls and documentation
• LDAP button is hidden when no LDAP service is being used (frontend)
• tooltips bug (frontend)
• timezone and timestamps bug (frontend)
• pending users treatment
• hijack time started timestamp fix (frontend)
• broken hijack URL in logs

Backwards Incompatible Changes

🚨 Since we migrated to a new version of the backend database, you will need to do the following process to upgrade ARTEMIS:

• Deactivate current running instance: docker-compose -f ... down
• Stash any local changes that should not conflict with upstream: git stash
• Checkout the master branch git checkout master
• Pull most recent code (including .env, versions, etc.) git pull origin master
• (IMPORTANT) Delete old DB data sudo rm -r postgres-data-current && sudo rm -r postgres-data-backup
• Pop local changes and resolve conflicts by preferring upstream: git stash pop
• Pull latest containers docker-compose -f ... pull
• Start ARTEMIS! docker-compose -f ... up -d

Bellerophon

Added

• new frontend (React) in new repo and container
• modular component-oriented UI
• CSRF protection
• improved LDAP support
• responsive layout and window scaling fix
• user feedback when tables are not properly rendered
• allow creation of new user from admin user management page
• admins can now set user's password
• correct auto-logout
• configurable session and inactivity timeout in .env
• "json" encoding accepted for messages coming from frontend (ignore/resolve/seen/delete/(un-)mitigate)
• frontend REST API (OpenAPI-swagger)

Changed

• changes in "dataplane_msms" table and "view_dataplane_msms" view, in order to support the new design of the "dataplane_view" module.
• improved server-side auth
• upgraded artemis-utils to 1.0.10 to include the slacker-log-handler==1.7.1 dep
• migrating from travis to GH actions
• downgraded to six==1.11.0 to achieve compatibility
• changed tap invalid format logging level to debug (instead of warning)
• updated artemis-utils to 1.0.11
• updated k8s documentation
• updated artemis-chart
• replaced bcrypt with argon2

Fixed

• autoconfiguration subprefix bug in prefixtree plus new autoconf tests
• vagrant docker-compose.yaml file fix (sync with master)
• session timeout behavior

Deprecated

• old frontend (Flask, custom JS)

Security

• updated PyYAML to 5.4 (fixing sec vulnerabilitiy)

Backwards Incompatible Changes

🚨 You can follow the classic update process for the transition to the latest version, however note that the new frontend requires the following changes that are not backwards compatible:

Deprecation of initial frontend user DB

We have migrated to mongoDB from sqlite. Due to security implications
for migrating salted hashed passwords between the two databases based on the algorithms we used initially and the ones we use currently, we opted for simply deprecating the sqlite DB. Locally managed users will
have to be recreated per instance. However, to ease this process, the new frontend now provides admins with the capability to create new users and set their password accordingly.

New and deleted env variables

• CONFIG_HOST=configuration [NEW, NO ACTION REQUIRED]
• CONFIG_PORT=3000 [NEW, NO ACTION REQUIRED]
• DATABASE_HOST=database [NEW, NO ACTION REQUIRED]
• BIND_IP=0.0.0.0 [DELETED]
• WEBAPP_PORT=4200 [CORRECT VALUE CHANGED FROM 8000 to 4200]
• ADMIN_USER=admin [DELETED, NOW USING ONLY EMAIL]
• JS_VERSION=1.0.4 [DELETED]
• GUNICORN_WORKERS=4 [DELETED]
• SESSION_TIMEOUT=1800 [NEW]
• INACTIVITY_TIMEOUT=900 [NEW]
• MONGODB_USER=admin [NEW]
• MONGODB_PASS=pass [NEW, SECRET]
• MONGODB_HOST=mongodb [NEW]
• MONGODB_PORT=27017 [NEW]
• MONGODB_NAME=artemis-web [NEW]
• LDAP_ENABLED=true [NEW]
• LDAP_HOST=ldap [NEW]
• LDAP_PORT=10389 [NEW]
• LDAP_PROTOCOL=ldap [NEW]
• LDAP_BIND_DN="cn=admin,dc=planetexpress,dc=com" [NEW]
• LDAP_BIND_SECRET="GoodNewsEveryone" [NEW, SECRET]
• LDAP_SEARCH_BASE="ou=people,dc=planetexpress,dc=com" [NEW]
• LDAP_SEARCH_FILTER="(mail={{username}})" [NEW]
• LDAP_SEARCH_ATTRIBUTES="mail, uid" [NEW]
• LDAP_EMAIL_FIELDNAME=mail [NEW]
• LDAP_ADMIN_GROUP=admin_staff [NEW]
• LDAP_USER_GROUP= [NEW]
• FLASK_SECRET_KEY [DELETED]
• SECURITY_PASSWORD_SALT [DELETED]
• CSRF_SECRET [NEW, SECRET]
• API_KEY [NEW, SECRET]

Deprecated/removed files

• Entire frontend code section. The current code is in the mono-repo https://github.com/FORTH-ICS-INSPIRE/artemis-web .
• local_configs/frontend/webapp.cfg
• local_configs/frontend/config.py
• local_configs/frontend/logging.yaml
• local_configs/frontend/__init__.py
• frontend/db

docker-compose.yaml

• Heavy revisions in frontend section to sync with new env variables. Please make sure you properly resolve conflicts with your on-premise ARTEMIS instance.
• Addition of mongodb section, user data is now kept at the mapped ./mongo_data folder.

Boreas

To upgrade, please follow the guide here and pay attention to the important 2.0.0 upgrade instructions.

Added

• Action and necessary logic to un-mitigate a hijack
• Ability to reload targeted services based on what conf section changed
• GET config endpoints to backend and monitor services
• Makefile for local commands (dev/ci)
• Added healthchecks to docker-compose (docker-compose version bumped to 3.4)
• Service reconfiguring flag and status
• Redis no-data-received key expiry logic (default: 60 seconds)
• Hijack detail JSON export

Changed

• Decoupled microservice architecture for backend and frontend
• Possible states constraint on DB hijacks table to allow orthogonal mitigation tagging
• Orthogonal mitigation/unmitigation tagging
• Heavily refactored docs
• Replaced bulk timer thread with separate process
• Replaced autoignore checker thread with separate process
• Replaced exabgp autoconf thread with separate process
• Display info on new microservices, grouping backend/monitor
• Upgraded requests to 2.25.1
• Upgraded to new bgpstream containers (using also ris and routeviews live sources)
• Full refactoring and upgrade of artemis-utils, refactored all imports (1.0.7)
• Upgraded Flask-JWT-Extended to 3.25.0 to address JWT issue with new browsers
• Upgraded artemis-utils to 1.0.8

Fixed

• Discard messages with timestamps in future datetime (updated to artemis-utils==1.0.5)
• Check for very old timestamps in DB update handling
• Replaced sets that are not supported by shared manager with lists
• Removed unneeded read locks
• Non-gracefull SIGKILL for taps in case graceful stop delays a lot
• Redis configuration file input

Deprecated

• 'caida' project in bgpstreamlive
• supervisor support and configuration

Notes:

1. if you encounter issues with detection essentially getting "stuck" (see #564) please comment under this issue and give us a heads-up in Slack. This was a bug we encountered only in a certain local deployment and we would like to double-check with the users whether it is an actual problem. If it is, please consult the issue workaround comment and we will patch on demand.
2. If you see any redis-related errors probably you have not set the configuration file of redis (new 2.0.0 addition correctly). Just run:

cp backend-services/configs/redis.conf local_configs/backend/redis.conf

and restart artemis.

Achilles

To upgrade, follow the guide here.

Added

• Redirection to login page when session is expired (#404)
• Support RADIUS integration for user authentication (#409)
• Support for GRAPHQL_URI to be set via environment variables, or to take a lazy approach and set HASURA_HOST and HASURA_PORT env variables
• Flag to override BGPStream timestamp (#415)
• Hijack alert auto-ignore mechanism for alerts of limited impact/visibility (#373)
• Env variable to invoke intended process states recovery mechanism
• Reinstated RPKI tests for PRs
• Implicit withdrawals are now generated also on correct (direct) subprefix routing
• Support for fork PRs in travis CI/CD builds and tests
• New dataplane-related table and view (pings and traceroutes)
• PoC mitigation mechanism and setup (deaggregation)
• Coverage for unit tests
• AS Path pre-pending sequence matching support in configuration
• Docs pages

Changed

• Refactor: Changed exchange / queue names
• Heavily refactored and improved exaBGP-based auto-configuration for speed/scale

Fixed

• Deleting user also deletes the role mapping (#412)
• Backend Makefile to use currently installed python version
• Frontend Makefile to use specific closure compiler version
• Minor opt fix in Redis RTT calls in DB
• Observer fix (RMQ connection establishment) to deal with threading issues

Removed

• Autoconf support in historical BGPStream (due to deprecation)

Deprecated

• Wiki pages

Aeolus

Added

• Additional ports (8080, 8443) to nginx configuration for rootless docker support
• Application names for all postgres connections
• Database migration check between latest release and current version
• Mechanism to show which modules are (re)loading upon configuration change
• GUNICORN_WORKERS and RABBITMQ_IO_THREAD_POOL_SIZE variables smaller deployments
• ZAP Active and Passive scanning
• Select Page button on hijacks page to select all hijacks on current page

Changed

• Upgraded nginx to latest release (v1.17)
• Upgrade rabbitmq to latest release (v3.8.3)
• Upgrade postgrest to latest release version (v7.0.0)
• Upgraded hasura-engine to v1.1.0
• Updated version of BGPStream on custom image (mavromat/bgpstream-redis:v1.3-bgpstream)

Fixed

• LIMIT query planner bug (http://datamangling.com/2014/01/17/limit-1-and-performance-in-a-postgres-query/)
• Added stderr output of supervisor processes on docker-compose logs (Please update your custom supervisor confs if using local_configs)
• Issue with multiple rules for same prefix(es) in detection
• Update RIPE Stat API call for ASN to country mapping
• Minor issue with deprecated version of routinator (upgraded to 0.6.4)
• Observer ujson encoding bug

Security

• Secured "proxy_api" endpoint from unauthorized access. Thanks to @marcgweg for finding this issue

Ajax

This release includes the following notable changes:

• Important monitoring + detection performance boost
• Grafana chart support
• Passive autoconfiguration via exaBGP or bgpstreamhist, via "autoconf" flag
• ENV variable "WITHDRAWN_HIJACK_THRESHOLD" to consider hijacks withdrawn based on a withdrawn monitor percentage threshold
• TravisCI and CodeCov support (replaced SemaphoreCI and coveralls)
• Database fixes and optimizations
• Replaced betabmp client with "caida" project within bgpstreamlive
• Added bgpstreamkafka tap (can be used for private BMP feeds)
• RPKI status in hijack alerts; can be enabled via ENV variables
• Optional routinator RPKI validator microservice
• Automatic YAML config file formatting upon config modification
• ENV vars "DB_AUTOCLEAN" and "DB_HIJACK_DORMANT" set to 1 day (24h) by default
• ARTEMIS recovery upon server rebooting
• Fixed important RabbitMQ ack issue with messages

Ares

Note for this version: please take extra care since we have introduced the following backwards-incompatible changes (logging and Kubernetes):

• Changed frontend logger from webapp_logger to artemis_logger in logging.yaml and config.py under frontend/webapp/configs. For old versions you need to replace the name from webapp_logger to artemis_logger in these two files under your local_configs/frontend folder.
• Changed helm chart labels to support multiple releases per namespace and this breaks helm upgrades. To migrate to the new version you need to deploy a new installation and migrate the Persistent Volumes Claims (PVC) to the new release by attaching them on the current Persistent Volumes.

This release includes the following changes:

Added

• IPv6 tests (backend testing)
• PR labeler (GitHub actions)
• Env variable and formatter for selecting hijack log fields
• Env variable for ARTEMIS web host for backend logging
• Logging also withdrawn and outdated hijacks in mail and hijack loggers
• Extra end_tag, outdated_parent and hijack_url in hijacks for logging
• Reinstating intended modules on ARTEMIS startup
• Support of multiple artemis instances on a single namespace (k8s)
• Added benchmarker and PR commenter

Changed

• py-radix, substituted with pytricia tree
• dep-licenses updated (with currently used modules)
• Fixed bug with outdated hijacks that remain hijacks
• Fixed bug with BGP updates related to more than one hijacks
• Added outdated_parent logging field to inform logging systems

Fixed

• Support for millions of prefixes in configuration file
• Ensure ARTEMIS config file is not clobbered on pod restart

Removed

• ipaddress requirement from frontend (not needed)

Arktos

This release includes the following changes:

Added

• Slack logging package and example
• ARTEMIS logo
• Monitor peers count in stats table (overview)
• Implicit withdrawals generated for benign BGP updates correcting hijacked peer-prefix combinations
• Support for configured ASN ranges
• Community-based annotation support for annotating hijacks
• Local pre-commit hook to check versioning
• Bgpstream historical BGP record collector (per prefix)
• Support for RIPE whois service-resolved AS-SETs in configuration
• Ingress resource for helm-chart (by default supports NGINX Ingress Controller)
• Redis ping/pong check on connection initialization
• In hijack page count the number of peers seen announcement and withdrawal

Changed

• Refactoring frontend (views, templates and static files are organized inside the folder render)
• Update hasura (1.0.0alpha42 -> 1.0.0alpha45)
• Replaced $.ajax with fetch
• Hijack logger output is now a JSON string
• Decoupled monitor module to another container
• Optional deployment of NGINX container or NGINX Ingress on Kubernetes

Fixed

• Bug with rule learning (hijack to rule dict) when empty neighbor
• Minor fix in custom mitigation trigger
• Updated configurations for k8s deployment
• Fixed flake8 warnings
• Wrong data.sql for testing
• Bug with single-quoted communities being interpreted as sexagesimal upon automated conf reload
• Monitoring of monitors and restarting them if not seen any BGP updates within user-defined interval
• Avoid detectors being stuck on blpop if token erased
• Fix for historical replay of withdrawals plus bgpstreamhist rate-limit
• Minor: creating implicit withdrawals 1 second after the related benign announcement
• Minor: in handling BGP withdrawals, consider announcements that are strictly newer than the withdrawals
• Various helm (k8s) chart fixes and improvements (see #221)

Security

• Bumped SQLAlchemy from 1.2.16 to 1.3.3 in /frontend

You can view a detailed list of changes in the CHANGELOG (https://github.com/FORTH-ICS-INSPIRE/artemis/blob/master/CHANGELOG.md). You can follow the wiki guide here (https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#upgrading-artemis-to-a-new-version) to upgrade to the new version.

Note for this version: please take extra care following the wiki upgrade guide in order to properly build/alter the local_configs to include the new monitor container configuration. Make sure you check docker-compose.yaml and make sure that the local configs are properly mapped.

Athena

This release includes the following changes:

Added

• Support for dormant flags in hijacks
• Storing hijack update (origin, neighbor) combinations in redis
• Learn rule action for ongoing hijacks in frontend after ignore action
• Misc UI additions
• Initial kubernetes/helm (helm-charts) support
• Reject old updates from taps and have a "HISTORIC" variable to enable/disable
• Initial support for LDAP authentication
• Hijack deletion functionality
• Abuse contact details for each ASN (Extracted from RIPEStat)
• Functionality to copy ASN details on clickboard
• Support to filter BGP Updates based on their AS Path
• Display distinct values of BGP Updates for the following fields: "Origin AS", "Peer AS" and "service" in hijack view

Changed

• Using prefix lists in json file format as monitoring taps input to avoid problematic ultra long arguments
• Refactored environment variables
• Use of RIPE RIS firehose stream instead of the websocket clientui8
• Misc UI improvements
• Update hasura (1.0.0alpha31 -> 1.0.0alpha42)
• Hijack view now uses hasura graphql to fetch BGP Updates

Fixed

• Correct RFC2622 translation when needed in frontend and backend
• When learning ignore rule, escape special character ":" (IPv6)
• Problematic start of RIS and exaBGP monitors, even if not configured
• BGP update redis bootstraping from DB
• UI support for multiple instances of a module in overview and system page
• Misc UI bugs

Removed

• Configured prefix graph visualization (needs redesign)
• Config data field from configs DB table

Security

• Using yaml dump and safe_load instead of pickling/unpickling

You can view a more detailed list of changes in the CHANGELOG (https://github.com/FORTH-ICS-INSPIRE/artemis/blob/master/CHANGELOG.md). You can follow the wiki guide here (https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#upgrading-artemis-to-a-new-version) to upgrade to the new version. Note: this release requires a DB migration, which is automatically handled by the system.

Atlas

Added

• Tooltip support for peers seen BGP Announcement/Withdrawal on hijack view.
• Support for rfc2622 ^+, ^-, ^n and ^n-m prefix operators in configuration
• Coverage tests for frontend actions
• Pre-commit hooks
• Policy dimension on hijacks (route leak detection based on no-export)
• Support for auto-cleaning unneeded BGP updates
• Automated DB backups
• View hijack by key
• Enable sorting for columns: # Peers Seen/# ASes Infected
• Added DB version on overview page

Changed

• Testing refactoring
• RIPE RIS live python websocket client

Fixed

• Updated/optimized db query for removing withdrawn peers (newer announcement)
• Support for different user/pass on rabbitmq
• Solved bug with randomized config hashing
• Fixed expected behavior when trying to run old containers on new DBs

You can view a more detailed list of changes in the CHANGELOG (https://github.com/FORTH-ICS-INSPIRE/artemis/blob/master/CHANGELOG.md). You can follow the wiki guide here (https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#upgrading-artemis-to-a-new-version) to upgrade to the new version. Note: this release requires a DB migration, which is automatically handled by the system.