vtf txb _buffercomponents swap

↑ vtf-txb-_buffercomponents

swap

Objects

class swap._Swap ¶

Optionally Buffer Component for swapping data chunks into an SQL database.

The chunks are cut out by _Trimmer when reaching a defined number of _Row's in the TextBuffer, depending on the current cursor position, and passed to Swap as DumpData. DumpData contains the cut rows, the corresponding data points and the parameter for the chunk position addressing. For runtime reduction, only the content of the rows is written to the database, the metadata and position addressing remain in memory.

Swap stores the data chunks in the database under unique slot numbers that never repeat for the existence of a database. Position addressing is done using an associative dictionary __slot_index__ and the list current_chunk_ids that indicates the above (index 0) and below (index 1) adjacent chunks to those loaded in the TextBuffer. A 0 on a position indicates that there are no chunks on that side. If chunks are "shifted to the side", the position number is extended by 1, starting from -1 for chunks above and 1 for chunks below the currently loaded buffer, the data chunks directly adjacent to the TextBuffer thus always have the highest absolute position number, the uppermost always -1 and the lowest always 1.

The storage, processing and adjustment of metadata (data points) is done under the slot numbers with ChunkMetaItem's in _MetaIndex (modified dictionary).

The handling of the swap is performed in the main methods of TextBuffer and via _Trimmer. The item ChunkLoad is created by TextBuffer and gives information about the actions done in the swap during an edit of the current buffer.

Information about parameterization and interaction for a smooth program flow:

• Even if in _Trimmer the size of the chunks is specified, it can NOT be guaranteed, because in runtime it is possible to store all rows of the current buffer as one chunk and chunks can be edited inplace (See ChunkIter, ChunkBuffer).
• Note that cursor movement via the cursor_move method in the TextBuffer is limited by the loaded rows in the TextBuffer.
• To ensure a display that is always filled, the number of displayable rows should be considered when parameterizing rows_maximal in _Swap and _Trimmer, and load_distance in _Swap.
• In addition, an appropriate average value should be found when selecting the chunk_size; larger to reduce accesses to the database and cache memory, smaller for faster processing of the chunks.

WARNING: the connection of the SQL Database is NOT threadsave, but the cursor (_sql.SQLTSCursor).

_¶

__meta_index__: _MetaIndex

__slot_index__: dict[int, int]

current_chunk_ids: list[int, int]

db_attached: bool

db_in_mem: bool

db_path: str | Literal[":memory:", ":history:", "file:...<uri>"]

sql_connection: SQLConnection

sql_cursor: _sql.SQLTSCursor

__call__() -> tuple[list[list[_Row]] | None, list[list[_Row]] | None, int | None, int | None] ¶

Automation. Load the next chunk when the cursor is in the load distance.

[+] __trimmer__.trim

Returns: (

• <_Rowchunk's cut from top> | None,
• <_Rowchunk's cut from bottom> | None,
• <n loaded chunks from top> | None,
• <n loaded chunks from bottom> | None

)

__init__(**kwargs) ¶

Create a new Swap object.

Keyword parameter:

• __buffer__

  The TextBuffer object for which Swap is created.

• db_path

  The location of the database can be specified as a filepath using an ordinal or "Uniform Resource Identifier" (URI) string; to create the database temporarily in RAM, the expression ":memory:" can be used; another special expression is ":history:" to create the database in the same location of the database in _LocalHistory.

• from_db

  To build the database and the object from an existing database, the origin can be passed as an ordinal path, a URI, or an existing SQL connection. The connection will be closed automatically afterwards, unless an SQL connection was passed.

• unlink_atexit

  Registers the deletion of the database when exiting the Python interpreter.

  The process is performed depending on the location of the database:

  • If the database was created as simple files, the connection is closed and the files are deleted from disk (including journals);
  • If an existing connection was passed (internal usage) or the database was created in RAM, the connection will be closed;
  • If the expression ":history:" was used during creation, all Swap entries in the database are deleted (unless the database was closed before).

• rows_maximal

  Used as the limit for fillings.

• keep_top_row_size

  After loading chunks, adjusts the top row in the buffer to the allocated size of the "top row".

• load_distance

  Distance between the cursor and the edge of the currently loaded chunks in the buffer at which loading is triggered.

raises:	ConfigurationError: if ":history:" is used and there is no connection to a database in __local_history__. DatabaseFilesError: db_path already exists. DatabaseTableError: if the database tables already exist in the destination.

__new_db__(**kwargs) -> _Swap ¶

Create a new _Swap with the same parameters, except from_db is set to None by default. Parameters can be overwritten via kwargs.

raises:	ConfigurationError: if ":history:" is used and there is no connection to a database in __local_history__. DatabaseFilesError: db_path already exists. DatabaseTableError: if the database tables already exist in the destination.

auto_fill() -> tuple[list[list[_Row]] | None, list[list[_Row]] | None, int | None, int | None] ¶

Fill the buffer with chunks up to the upper limit. Then remove the overhang.

[+] __trimmer__.trim

Returns: (

• <_Rowchunk's cut from top> | None,
• <_Rowchunk's cut from bottom> | None,
• <n loaded chunks from top> | None,
• <n loaded chunks from bottom> | None

)

backup(dst) -> None ¶

Clones all chunks in the swap, the metadata of the _Swap, the _MetaIndex and the currently loaded buffer in dst. Close the connection to the backup-db if dst is defined as path or URI.

raises:	ConfigurationError: if ":history:" or ":memory:" is used as dst. DatabaseFilesError: dst already exists. DatabaseTableError: if the database tables already exist in the destination.

chunk_buffer(position_id, *, sandbox=False, delete_empty=False) -> ChunkBuffer ¶

Fabricate an ChunkBuffer. Use the sandbox mode and do not overwrite the actual data if sandbox is True. Delete the entry of the chunk in the swap if the chunk became empty due to an edit of the ChunkBuffer and delete_empty is set to True (is ignored in sandbox mode).

clone(to_db, with_current_buffer=False, unlink_origin=False, **kwargs) -> _Swap ¶

Clone the database and metadata into a new _Swap object, and include the currently loaded chunks in TextBuffer if with_current_buffer is set to True. The standard parameterization is the same as the original and can be overridden via keyword arguments. If unlink_origin is set to True, the existing database is deleted depending on its location:

db origin	unlink
Filepath	- > Connection.close ; os.unlink
:memory:	- > Connection.close
:history:	- > dropall
SQL-Connection	- > Connection.close

raises:	ConfigurationError: if ":history:" is used and there is no connection to a database in __local_history__. DatabaseFilesError: db_path already exists. DatabaseTableError: if the database tables already exist in the destination.

dropall() -> None ¶

Delete all entries in the tables of the database and discard the indexes.

dump_metas(index_too=True) -> None ¶

Dump the metadata of the _Swap into the database and do commit. Include the _MetaIndex if index_too is True.

get_by_slot(slot) -> ChunkData ¶

Return the chunk of slot from the swap as ChunkData.

get_chunk(position_id) -> ChunkData ¶

Read and return the chunk of the position_id from the swap as ChunkData.

raises:	KeyError: if the position is not present.

load(side, goto) -> bool ¶

Load the next chunk of side side to the TextBuffer. Then set the cursor to the data point goto. Return False if no chunk of the side is present, otherwise True.

@property

positions_bottom_ids() -> tuple[int, ...] | tuple ¶

All position numbers of the chunks below the current buffer ordered from top to bottom.

Scheme: 7, 6, 5, 4, 3, 2, 1

@property

positions_ids() -> tuple[int, ...] | tuple ¶

All position numbers of the chunks ordered from top to bottom.

Scheme: -1, -2, -3, -4, 2, 1

@property

positions_top_ids() -> tuple[int, ...] | tuple ¶

All position numbers of the chunks above the current buffer ordered from top to bottom.

Scheme: -1, -2, -3

remove_chunk_positions(*position_ids) -> None ¶

Irretrievably remove chunks from swap based on position numbers and adjust positioning addressing.

WARNING: May disturb the stability of _Swap and TextBuffer if non-empty chunks are removed and the metadata index is not recalculated or adjusted.

slot(position_id) -> int ¶

Return the slot number for position_id.

raises:	KeyError: if the position is not present.

suit(mode="commit", leave_active=False) -> _Suit[_Swap] | _Suit[_Trimmer] ¶

Return a _Suit depending on mode.

When entering __swap__ is returned:

>>> with __swap__.suit(<mode>)[ as swap]:
>>>     ...

Modes:

• "commit" :

  Commit when exiting the suit instead of after each dump.

• "fill" :

  Disable automatic filling with chunks over __call__ and auto_fill within the suit.

• "_metas" :

  Bypasses the adjustment of metadata for actions on the swap and writes the differences separately. The final adjustment of the metadata is then done only after leaving the suit. SPECIAL Usage

If a suit is active and leave_active is True, __exit__ of the active suit is executed and a new one is created, otherwise a dummy is returned.

unlink() -> None ¶

Execute the unlink function, depending on where the database is located, and remove the execution entry when exiting the Python interpreter. It can be assumed that the database will no longer exist and will be deleted (even if the database is on the disk as files).

db origin	unlink
Filepath	- > Connection.close ; os.unlink
:memory:	- > Connection.close
:history:	- > dropall
SQL-Connection	- > Connection.close

Components

class swap._MetaIndex(dict[int, ChunkMetaItem], ContextManager) ¶

An index for storing, processing, and adjusting metadata associated with chunks. The metadata is stored as ChunkMetaItem at the slot number.

Handling is ensured by the methods in the TextBuffer and its components when swap is active.

The shadow-mode:

For a SPECIAL usage, the object can be used as a contextmanager/suit to register inplace edits of chunks separately without direct adjustment of the index. This allows a sequential processing of several chunks without changing the original data points (see also ChunkIter), only when leaving the differences resulting from the editing are processed and the index is adjusted.

_¶

__shadow_diffs__: dict[int | None, tuple[int, int, int, int]]

__shadow_ln_count__: dict[int, tuple[int, int]]

__enter__() -> None ¶

Start the shadow adjustment of the MetaIndex.

The following adjustments to the index are saved separately and are only finally performed when shadow_commit is executed.

Overwrites the core methods for the adjustments, the remake method and the execution of pop_*.

__exit__(exc_type, exc_val, exc_tb) -> None ¶

Finish the shadow adjustment of the MetaIndex and perform the adjustments.

Resets the core methods for the adjustment, the remake method and the execution of pop_* to the original functions.

adjust_bottom_auto() -> tuple[int, int, int, int] | None ¶

Automatically adjust the items below the TextBuffer.

adjust_by_adjacent(adjacent_pos_ids, next_abs_dat, next_abs_cnt, next_row_num, next_lin_num) -> tuple[int, int, int, int] ¶

Calculate the differences with the data start points of the meta item for the first entry in adjacent_pos_ids and the data end points from the chunk before it.

Then call the interface to adjust the sequence of ChunkMetaItem's by the differences.

Return the differences: ( <dif_dat>, <dif_cnt>, <dif_row>, <dif_lin> )

adjust_by_position(origin_position, next_abs_dat, next_abs_cnt, next_row_num, next_lin_num) -> tuple[int, int, int, int] ¶

Calculate the differences with the data start points of item adjacent to origin_position and the data end points from the origin_position.

Then call the interface to adjust the origin_position following ChunkMetaItem's [ and TextBuffer ] by the differences.

Return the differences: ( <dif_dat>, <dif_cnt>, <dif_row>, <dif_lin> )

copy() -> _MetaIndex ¶

Create a deepcopy and return a new _MetaIndex

diff_by_adjacent(adjacent_pos_ids, next_abs_dat, next_abs_cnt, next_row_num, next_lin_num) -> tuple[int, int, int, int] ¶

Calculate the differences with the data start points of the meta item for the first entry in adjacent_pos_ids and the data end points from the chunk before it.

Return the differences: ( <dif_dat>, <dif_cnt>, <dif_row>, <dif_lin> )

diff_by_position(origin_position, next_abs_dat, next_abs_cnt, next_row_num, next_lin_num) -> tuple[int, int, int, int] ¶

Calculate the differences with the data start points of item adjacent to origin_position and the data end points from the origin_position.

Return the differences: ( <dif_dat>, <dif_cnt>, <dif_row>, <dif_lin> )

get_by_slot(slot) -> ChunkMetaItem ¶

Return the ChunkMetaItem on slot

get_meta_indices() -> tuple[tuple[int | None], dict[int | None, tuple[int, int, int, int, int, int]]] ¶

Create an index of the meta start points for each chunk.

Return a top-down sorted sequence of position numbers at index 0:

(-1, -2, -3, -4, None, 2, 1)

and an unordered dictionary at index 1:

{ -3: ( <data start>, <content start>, <row start>, <line start>, <n rows>, <n newlines> ), ... }

The currently loaded chunks in the buffer are indicated by key None.

get_metaitem(position_id) -> ChunkMetaItem ¶

Return the ChunkMetaItem for position_id

pop(slot) -> ChunkMetaItem ¶

The original pop method.

Remove ChunkMetaItem on slot from the index and return it.

Overwritten when shadow mode is active.

pop_by_slot(slot) -> ChunkMetaItem ¶

The interface method to pop ChunkMetaItem's by the slot-number.

pop_metaitem(position_id) -> ChunkMetaItem ¶

The interface method to pop ChunkMetaItem's by the position_id.

remake(*, bottom_too=True, bottom_only=False) -> None ¶

Recalculate the metaindex by reading all chunks.

shadow_commit() -> None ¶

Finish the shadow adjustment of the MetaIndex and perform the adjustments.

Resets the core methods for the adjustment, the remake method and the execution of pop_* to the original functions.

shadow_start() -> None ¶

Start the shadow adjustment of the MetaIndex.

The following adjustments to the index are saved separately and are only finally performed when shadow_commit is executed.

Overwrites the core methods for the adjustments, the remake method and the execution of pop_*.

---

Date:	20 Dec 2022
Version:	0.1
Author:	Adrian Hoefflin [srccircumflex]
Doc-Generator:	"pyiStructure-RSTGenerator" <prototype>