Make add handle duplicate articles #576

🚀 Feature

Currently, the bbs_database add is going to error out when an article is already in the database. When trying to add all articles in a folder It would be more convenient to compute new_articles = set(parsed_uids) - set(existing_uids) and only add the new articles.

Motivation

In the overall pipeline we would want to bulk add all articles from a given source and a given month to the database. However, one duplicate article would lead to errors.

Alternatives

• Compute the above set difference outside of add and create a new folder of symlinks only holding new articles
• Add articles to the database one by one and just ignore failures