-
-
Notifications
You must be signed in to change notification settings - Fork 151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/performance test #749
base: main
Are you sure you want to change the base?
Feature/performance test #749
Conversation
…hanges in config.py).
…xture for test_calculator recording.
…scriptions for testing purposes. Changed get_all_recordings() in crud.py to get_recordings(), added a parameter max_rows to get a specified number of rows.
…RL to build off DB_FILE_PATH.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the great start @seanmcguire12 👍
Please let me know if you have any questions 🙏
|
||
return data | ||
|
||
def format_sql_insert(table_name, rows): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add types.
Also, what do you think about something like this:
from sqlalchemy import insert, Table, MetaData
from sqlalchemy.engine import Engine # Assuming you have an engine instance
def generate_sql_insert(engine: Engine, table_name: str, rows: list) -> str:
"""Generates a SQL INSERT statement using SQLAlchemy for given rows and table.
Args:
engine (Engine): SQLAlchemy Engine connected to the database.
table_name (str): Name of the table to insert data into.
rows (list): List of dictionaries representing the rows to insert.
Returns:
str: A string representation of the SQL INSERT statement suitable for fixtures.sql.
"""
metadata = MetaData(bind=engine)
table = Table(table_name, metadata, autoload_with=engine)
stmt = insert(table).values(rows)
compiled = stmt.compile(engine, compile_kwargs={"literal_binds": True})
return str(compiled) + ";"
Please fill out the type for rows
, e.g. list[dict]
, list[BaseModelType]
, list[db.Base]
, or similar.
|
||
if data["memory_stats"]: | ||
file.write("-- Insert sample memory_stats\n") | ||
file.write(format_sql_insert("memory_stat", data["memory_stats"])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about something more DRY, e.g.:
rows_by_table_name = fetch_data(session)
for table_name, rows in rows_by_table_name.items():
if not rows:
logger.warning(f"no rows for table_name=}")
continue
with open(file_path, "a") as file:
logger.info(f"writing {len(rows)=} to {file_path=} for {table_name=}")
file.write(f"-- Insert sample rows for {table_name}\n")
file.write(format_sql_insert(table_name, rows))
Related: #707 |
…ctions for interacting with the macos accessibility tree.
…st is as we expect. Still to do: add conditional statements to ensure that the tests run correctly based on the users OS.
…r is not closed during the recording (so that we can run tests on the final state).
|
||
|
||
logger.info(f"Value match: '{element_value}' == '{expected_value}'") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think about:
assert actual_value == expected_value, f"{actual_value=} {expected_value=}"
Looks great so far, thank you @seanmcguire12 ! 🙏 |
This PR creates performance tests for evaluating the various replay strategies against a constant (recording(s) in the database). Work in progress.
Summary
This PR addresses #704
Checklist
How can your code be run and tested?
This code can be run with
pytest -s /OpenAdapt/tests/openadapt/test_performance.py