Skip to content

Commit

Permalink
Update type hints in helpers.write_timdex_records_to_json (#96)
Browse files Browse the repository at this point in the history
* Update type hints in helpers.write_timdex_records_to_json

Why these changes are being introduced:
* After some refactoring in 07/2022 that added an iterator pattern to Transformer, it looks as though downstream type hints were not updated.
* While this did not affect runtime, it made reading the code somewhat confusing to someone new to the codebase.

How this addresses that need:
* Update the `records` input argument of `helpers.write_timdex_records_to_json()` from `Iterator[TimdexRecord]` to `Transformer`
* rename `records` to `transformer_instance`, matching variable names from calling `cli.py`
* Add explicit `TimdexRecord` type hint for result of `next(transformer_instance)`

Side effects of this change:
* None

Relevant ticket(s):
* None

Additional maintenance:
* Updated syntax for isort command in Makefile
  * Based on new results from isort, update import order in helpers.py
* update dependencies and revert click workaround
  * In PR #91, while updating dependencies, `click` and `mypy` had a conflict.
  * A temporary workaround was put into place that reordered the `click` decorators, but this deviated from other projects.
  * Now that `click` has updated again, that workaround is no longer needed.
  * Resolves #92
  • Loading branch information
ghukill authored Jul 18, 2023
1 parent e9c5212 commit 2f7fdaa
Show file tree
Hide file tree
Showing 4 changed files with 69 additions and 65 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ flake8:
pipenv run flake8 .

isort:
pipenv run isort . --diff
pipenv run isort . --check --diff

mypy:
pipenv run mypy transmogrifier
Expand Down
118 changes: 59 additions & 59 deletions Pipfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion transmogrifier/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
logger = logging.getLogger(__name__)


@click.command()
@click.option(
"-i",
"--input-file",
Expand All @@ -41,7 +42,6 @@
@click.option(
"-v", "--verbose", is_flag=True, help="Pass to log at debug level instead of info"
)
@click.command()
def main(source, input_file, output_file, verbose):
START_TIME = perf_counter()
root_logger = logging.getLogger()
Expand Down
12 changes: 8 additions & 4 deletions transmogrifier/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import logging
import os
from datetime import datetime
from typing import Iterator, Optional
from typing import TYPE_CHECKING, Iterator, Optional

from attrs import asdict
from bs4 import BeautifulSoup, Tag
Expand All @@ -16,6 +16,10 @@
from transmogrifier.config import DATE_FORMATS
from transmogrifier.models import TimdexRecord

# import Transformer only when type checking to avoid circular dependency
if TYPE_CHECKING: # pragma: no cover
from transmogrifier.sources.transformer import Transformer

logger = logging.getLogger(__name__)


Expand Down Expand Up @@ -173,11 +177,11 @@ def write_deleted_records_to_file(deleted_records: list[str], output_file_path:


def write_timdex_records_to_json(
records: Iterator[TimdexRecord], output_file_path: str
transformer_instance: "Transformer", output_file_path: str
) -> int:
count = 0
try:
record = next(records)
record: TimdexRecord = next(transformer_instance)
except StopIteration:
return count
with open(output_file_path, "w") as file:
Expand All @@ -195,7 +199,7 @@ def write_timdex_records_to_json(
"Status update: %s records written to output file so far!", count
)
try:
record = next(records)
record: TimdexRecord = next(transformer_instance) # type: ignore[no-redef] # noqa: E501
except StopIteration:
break
file.write(",\n")
Expand Down

0 comments on commit 2f7fdaa

Please sign in to comment.