Skip to content

SQLite DB with SQLAlchemy #190

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,12 +182,11 @@ _Skip these steps if launching from the .sh script on Linux/macOS._
- Windows w/Command Prompt: `.venv\Scripts\activate.bat`
- Linux/macOS: `source .venv/bin/activate`

3. Install the required packages:
3. Install the required packages:

- required to run the app: `pip install -r requirements.txt`
- required to develop: `pip install -r requirements-dev.txt`


To run all the tests use `python -m pytest tests/` from the `tagstudio` folder.

_Learn more about setting up a virtual environment [here](https://docs.python.org/3/tutorial/venv.html)._
Expand All @@ -213,10 +212,13 @@ _Learn more about setting up a virtual environment [here](https://docs.python.or

### What State Is the Project Currently In?

As of writing (Alpha v9.2.0) the project is in a useable state, however it lacks proper testing and quality of life features.
As of writing (Alpha v9.2.1) the project is in a useable state, however it lacks proper testing and quality of life features.

### What Features Are You Planning on Adding?

> [!NOTE]
> **_See [Planned Features](/doc/planned_features.md) documentation._**

Of the several features I have planned for the project, these are broken up into “priority” features and “future” features. Priority features were originally intended for the first public release, however are currently absent from the Alpha v9.x.x builds.

#### Priority Features
Expand Down
119 changes: 119 additions & 0 deletions database_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
from datetime import datetime
from pathlib import Path

from sqlalchemy import select
from sqlalchemy.orm import Session

from tagstudio.src.database.manage import drop_tables, make_engine, make_tables
from tagstudio.src.database.table_declarations.entry import Entry
from tagstudio.src.database.table_declarations.field import (
DatetimeField,
FloatField,
IntegerField,
TagBoxField,
TextField,
)
from tagstudio.src.database.table_declarations.tag import Tag, TagCategory

CONNECTION_STRING = "sqlite:///./example_database.sqlite"

ENGINE = make_engine(connection_string=CONNECTION_STRING)

# If prior schema defs
drop_tables(engine=ENGINE)
make_tables(engine=ENGINE)

# Populate with mock data
with Session(bind=ENGINE) as session, session.begin():
mock_entry = Entry(path=Path("/mock_path/to/item.jpg"))

archived_tag = Tag(name="Archived", category=TagCategory.meta_tag)
favorited_tag = Tag(name="Favorited", category=TagCategory.meta_tag)

user_tag = Tag(name="User Made Tag", category=TagCategory.user_tag)
user_subtag = Tag(
name="Subtag", category=TagCategory.user_tag, parent_tags=set([user_tag])
)

note_field = TextField(
value="Example text value",
name="Example Note",
)
integer_field = IntegerField(
value=100,
name="Example Integer",
)
float_field = FloatField(
value=100.10,
name="Example Float",
)
datetime_field = DatetimeField(
value=datetime.now(),
name="Example Datetime",
)
meta_tag_box_field = TagBoxField(
value=TagCategory.meta_tag,
name="Meta Tag Box",
)
user_tag_box_field = TagBoxField(
value=TagCategory.user_tag,
name="User Tag Box",
)

mock_entry.tags.update(
[
archived_tag,
favorited_tag,
user_tag,
]
)

mock_entry.fields.extend(
[
note_field,
integer_field,
float_field,
datetime_field,
meta_tag_box_field,
user_tag_box_field,
]
)

session.add(mock_entry)


with Session(bind=ENGINE) as session, session.begin():
entry = session.scalars(select(Entry).where(Entry.id == 1)).one()

print("Entry information:")
print(
f"\tMeta tags: {[tag.name for tag in entry.category_tags(category=TagCategory.meta_tag)]}"
)
print(
f"\tUser tags: {[tag.name for tag in entry.category_tags(category=TagCategory.user_tag)]}"
)
print(
f"\t\tUser Tags' Subtags: {[(tag.name, [tag.name for tag in tag.subtags]) for tag in entry.category_tags(category=TagCategory.user_tag)]}"
)
print(f"\tIs archived: {entry.archived}")
print(f"\tIs favorited: {entry.archived}")
print(
f"\tOrdered Fields: \n\t\t{"\n\t\t".join([f"{(i, field.name, field.value)}" for i, field in enumerate(entry.fields)])}" # type: ignore
)

print("\nMoving first field to end, reordering, and committing to DB...\n")

# Move field entry
first_field = entry.fields.pop(0)
entry.fields.append(first_field)
# Reorder entries
entry.fields.reorder()
# Commit on context close

with Session(bind=ENGINE) as session, session.begin():
entry = session.scalars(select(Entry).where(Entry.id == 1)).one()

print("Entry information:")
print(
f"\tOrdered Fields: \n\t\t{"\n\t\t".join([f"{(i, field.name, field.value)}" for i, field in enumerate(entry.fields)])}" # type: ignore
)
23 changes: 0 additions & 23 deletions doc/Field.md

This file was deleted.

19 changes: 0 additions & 19 deletions doc/Home.md

This file was deleted.

10 changes: 0 additions & 10 deletions doc/Library.md

This file was deleted.

Binary file added doc/assets/db_schema.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/assets/tag_override_ex-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/assets/tag_override_ex-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
24 changes: 24 additions & 0 deletions doc/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Welcome to the TagStudio Documentation!

> [!WARNING]
> This documentation is still a work in progress, and is intended to aide with deconstructing and understanding of the core mechanics of TagStudio and how it operates.

<div align="center">
<img src="../github_header.png" alt="TagStudio Alpha" height="100">
<img src="https://i0.wp.com/www.bapl.org/wp-content/uploads/2019/02/old-under-construction-gif.gif" alt="Under Construction" height="100">
</div>

## Table of Contents

- [Library](/doc/library/library.md)
- [Entries](/doc/library/entry.md)
- [Fields](/doc/library/field.md)
- [Tags](/doc/library/tag.md)
- [Tools & Macros](/doc/utilities/macro.md)
- [Planned Features](/doc/updates/planned_features.md)

---

### [Database Migration](/doc/updates/db_migration.md)

The "Database Migration", "DB Migration", or "SQLite Migration" is an upcoming update to TagStudio which will replace the current JSON [library](/doc/library/library.md) with a SQL-based one, and will additionally include some fundamental changes to how some features such as [tags](/doc/library/tag.md) will work.
8 changes: 4 additions & 4 deletions doc/Entry.md → doc/library/entry.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Entry

Entries are the units that fill a [library](/doc/Library.md). Each one corresponds to a file, holding a reference to it along with the metadata associated with it.
Entries are the units that fill a [library](/doc/library/library.md). Each one corresponds to a file, holding a reference to it along with the metadata associated with it.

### Entry Object Structure

Expand All @@ -14,12 +14,12 @@ Entries are the units that fill a [library](/doc/Library.md). Each one correspon
3. `path`:
- String, **Required**, OS Agnostic
- The folder path in which the media file is located in.
4. [`fields`](/doc/Field.md):
4. [`fields`](/doc/library/field.md):
- List of dicts, Optional
- A list of Field ID/Value dicts.

NOTE: _Entries currently have several unused optional fields intended for later features._

## Retrieving Entries based on [Tag](/doc/Tag.md) Cluster
## Retrieving Entries based on [Tag](/doc/library/tag.md) Cluster

By default when querying Entries, each Entry's `tags` list (stored in the form of Tag `id`s) is compared against the Tag `id`s in a given Tag cluster (list of Tag `id`s) or appended clusters in the case of multi-term queries. The type of comparison depends on the type of query and whether or not it is an inclusive or exclusive query, or a combination of both. This default searching behavior is done in _O(n)_ time, but can be sped up in the future by building indexes on certain search terms. These indexes can be stored on disk and loaded back into memory in future sessions. These indexes will also need to be updated as new Tags and Entries are added or edited.
By default when querying Entries, each Entry's `tags` list (stored in the form of Tag `id`s) is compared against the Tag `id`s in a given Tag cluster (list of Tag `id`s) or appended clusters in the case of multi-term queries. The type of comparison depends on the type of query and whether or not it is an inclusive or exclusive query, or a combination of both. This default searching behavior is done in _O(n)_ time, but can be sped up in the future by building indexes on certain search terms. These indexes can be stored on disk and loaded back into memory in future sessions. These indexes will also need to be updated as new Tags and Entries are added or edited.
3 changes: 3 additions & 0 deletions doc/library/entry_groups.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Entry Groups (Upcoming Feature)

Entries can be grouped via tags marked as “groups” which when applied to different entries will signal TagStudio to treat those entries as a single group inside of searches and browsing.
34 changes: 34 additions & 0 deletions doc/library/field.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Field

Fields are the building blocks of metadata stored in [entries](/doc/library/entry.md). Fields have several base types for representing different kinds of information, including:

#### `text_line`

- A string of text, displayed as a single line.
- e.g: Title, Author, Artist, URL, etc.

#### `text_box`

- A long string of text displayed as a box of text.
- e.g: Description, Notes, etc.

#### `tag_box`

- A box of [tags](/doc/library/tag.md) defined and added by the user.
- Multiple tag boxes can be used to separate classifications of tags.
- e.g: Content Tags, Meta Tags, etc.

#### `datetime` [WIP]

- A date and time value.
- e.g: Date Created, Date Modified, Date Taken, etc.

#### `checkbox` [WIP]

- A simple two-state checkbox.
- Can be associated with a tag for quick organization.
- e.g: Archive, Favorite, etc.

#### `collation` [obsolete]

- Previously used for associating files to be used in a [collation](/doc/utilities/macro.md#create-collage), will be removed in favor of a more flexible feature in future updates.
11 changes: 11 additions & 0 deletions doc/library/library.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Library

The library is how TagStudio represents your chosen directory, with every file inside of it being displayed as an [entry](/doc/library/entry.md). You can have as many or few libraries as you wish, since each libraries' data is stored within a "`.TagStudio`" folder at its root.
Note that this means [tags](/doc/library/tag.md) you create only exist _per-library_.

### Library Contents

- [Entries](/doc/library/entry.md)
- [Fields](/doc/library/field.md)
- [Tags](/doc/library/tag.md)
- [Macros](/doc/utilities/macro.md)
31 changes: 24 additions & 7 deletions doc/Tag.md → doc/library/tag.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,54 @@
# Tag

Tags are user-defined attributes made up of one or more keywords, aliases, and relationships to other tags. A person, place, thing, concept, you name it! Tags allow for a more sophisticated way to organize and search [entries](/doc/Entry.md) thanks to their aliases, parent tags, and more.
Tags are user-defined attributes made up of one or more keywords, aliases, and relationships to other tags. A person, place, thing, concept, you name it! Tags allow for a more sophisticated way to organize and search [entries](/doc/library/entry.md) thanks to their aliases, parent tags, and more.
Tags can be as simple or complex as wanted, so that any user can tune TagStudio to fit their needs.

Among the things that make tags so useful, aliases give the ability to contain alternate names and spellings, making searches intuitive and expansive. Furthermore, parent-tags/subtags offer relational organization capabilities for the structuring and connection of the [library's](/doc/Library.md) contents.
Among the things that make tags so useful, aliases give the ability to contain alternate names and spellings, making searches intuitive and expansive. Furthermore, parent-tags/subtags offer relational organization capabilities for the structuring and connection of the [library's](/doc/library/library.md) contents.

## Tag Object Structure

#### `id`

ID for the tag.

- Int, Unique, Required
- Used for internal processing

#### `name`

The normal name of the tag, with no shortening or specification.

- String, Required
- Doesn't have to be unique
- Used for display, searching, and storing

#### `shorthand`

The shorthand name for the tag. Works like an alias but is used for specific display purposes.

- String, Optional
- Doesn't have to be unique
- Used for display and searching

#### `aliases`

Alternate names for the tag.

- List of Strings, Optional
- Recommended to be unique to this tag
- Used for searching

#### `subtags`

Other Tags that make up properties of this tag. Also called "parent tags".

- List of Strings, Optional
- Used for display (first parent tag only) and searching.

#### `color`

A color name string for customizing the tag's display color

- String, Optional
- Used for display

Expand All @@ -41,11 +58,11 @@ Using for example, a library of files including some tagged with the following t

| Tag | `name` | `shorthand` | `aliases` | `parent tags` |
| ------------------- | ------------------- | ----------- | ---------------------- | -------------------------------------------- |
| *League of Legends* | "League of Legends" | "LoL" | ["League"] | ["Game", "Fantasy"] |
| *Arcane* | "Arcane" | "" | [] | ["League of Legends", "Cartoon"] |
| *Jinx (LoL)* | "Jinx Piltover" | "Jinx" | ["Jinxy", "Jinxy Poo"] | ["League of Legends", "Arcane", "Character"] |
| *Zander (Arcane)* | "Zander Zanderson" | "Zander" | [] | ["Arcane", "Character"] |
| *Mr. Legend (LoL)* | "Mr. Legend" | "" | [] | ["League of Legends", "Character"] |
| _League of Legends_ | "League of Legends" | "LoL" | ["League"] | ["Game", "Fantasy"] |
| _Arcane_ | "Arcane" | "" | [] | ["League of Legends", "Cartoon"] |
| _Jinx (LoL)_ | "Jinx Piltover" | "Jinx" | ["Jinxy", "Jinxy Poo"] | ["League of Legends", "Arcane", "Character"] |
| _Zander (Arcane)_ | "Zander Zanderson" | "Zander" | [] | ["Arcane", "Character"] |
| _Mr. Legend (LoL)_ | "Mr. Legend" | "" | [] | ["League of Legends", "Character"] |

**The query "Arcane" will display results tagged with:**

Expand Down
3 changes: 3 additions & 0 deletions doc/library/tag_categories.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Tag Categories (Upcoming Feature)

Replaces [Tag Fields](/doc/library/field.md#tag_box). Tags are able to be marked as a “category” which then displays as tag fields currently do, with any tags inheriting from that category being displayed underneath.
16 changes: 16 additions & 0 deletions doc/library/tag_overrides.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Tag Overrides (Upcoming Feature)

Tag overrides are the ability to add or remove [parent tags](/doc/library/tag.md#subtags) from a [tag](/doc/library/tag.md) on a per- [entry](/doc/library/entry.md) basis. Relies on the [Database Migration](/doc/updates/db_migration.md) update being complete.

## Examples

<figure>
<img src="../assets/tag_override_ex-1.png" alt="Example 1" height="300">
<figcaption>Ex. 1 - Comparing standard tag composition vs additive and subtractive inheritance overrides.</figcaption>
</figure>

<figure>
<img src="../assets/tag_override_ex-2.png" alt="Example 2" height="300">

<figcaption>Ex. 2 - Parent tag swap using tag overrides.</figcaption>
</figure>
Loading
Loading