Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add memory & conn collector, collector manager, tests #460

Merged
merged 199 commits into from
Mar 28, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
199 commits
Select commit Hold shift + click to select a range
fac6a87
add memory & conn collector, collector manager, tests
ankona Jan 19, 2024
8a81a59
mypy
ankona Jan 19, 2024
cd12d30
add sinks, timestamps, execute collect from telmon event loop
ankona Jan 19, 2024
d8aa972
fix refactor mistakes
ankona Jan 22, 2024
655b0c2
working collectors, no threading, shutdown bug
ankona Jan 22, 2024
640d8b5
tweak sink/collector api, log addresses in errors, add MockSink to tests
ankona Jan 22, 2024
4477401
revert autoshutdown change
ankona Jan 22, 2024
de64065
use proper log level
ankona Jan 22, 2024
57f9192
get rid of sink cast
ankona Jan 22, 2024
70c9042
add mocked collector exec test
ankona Jan 22, 2024
d9992ef
fix async testing bug for autoshutdown
ankona Jan 22, 2024
8b9a039
add telemetry collection timeout, make Sink.save async
ankona Jan 23, 2024
0ea28a1
fix failure to await async method
ankona Jan 24, 2024
3356dbe
remove unused import
ankona Jan 24, 2024
935237b
isort
ankona Jan 24, 2024
0c2c228
isort
ankona Jan 24, 2024
5b6fedd
mypy
ankona Jan 24, 2024
cc1d0b0
validate filesink init, separate telemetry tests
ankona Jan 24, 2024
d9b88c9
move more fixtures to conftest
ankona Jan 24, 2024
3d9bd7a
Enable always-on manifest generation
ankona Jan 24, 2024
0d9b35d
update tests for always-on manifest write
ankona Jan 24, 2024
ea4b847
add collector shutdown, lint, mypy
ankona Jan 24, 2024
3564690
rename entity config field
ankona Jan 24, 2024
50074ad
add telmon min/max collection frequency
ankona Jan 24, 2024
2e4779d
increase max telemetry freq allowed
ankona Jan 24, 2024
294b278
checkpoint. file headers, file names, err handling, tracing
ankona Jan 25, 2024
53b8fc7
fix formatting lint err
ankona Jan 26, 2024
54574c3
apply isort
ankona Jan 26, 2024
32c65e7
header write bugd
ankona Jan 26, 2024
dbd6144
whitespace
ankona Jan 26, 2024
e5b1f72
black
ankona Jan 26, 2024
3b023f6
force black
ankona Jan 26, 2024
79e7cc9
reduce header boilerplate/duplicated code
ankona Jan 26, 2024
6ddf9a1
fix typo
ankona Jan 26, 2024
1ef9533
remove commented code blocks
ankona Jan 26, 2024
40f90be
fix test bugs introduced in header refactor
ankona Jan 26, 2024
210a7d1
fix test bugs
ankona Jan 26, 2024
b8ff4ff
disable some autostop/telemetry config tests temporarily
ankona Jan 26, 2024
d2b94a9
fix broken boolean evaluation for enabling db collectors
ankona Jan 26, 2024
773a32a
shift collector output config for integration w/dashboard
ankona Jan 26, 2024
79b4371
py38 black
ankona Jan 26, 2024
7647f15
Add client_id to output from client list collector
ankona Jan 29, 2024
68ebf3c
refactor telemetry producer to avoid inheritance
ankona Jan 29, 2024
20016e7
apply isort
ankona Jan 29, 2024
7b8f251
ensure default telemetry enable behaviors, add some missing typehints…
ankona Jan 29, 2024
e8c9059
modify default behavior to write exp logs w/no telmon config dependency
ankona Jan 29, 2024
45361be
handle missing paths in manifest
ankona Jan 29, 2024
2133628
separate instance config from collector output config
ankona Jan 30, 2024
dd69204
fix copypasta in test description
ankona Jan 30, 2024
d0b13e9
black
ankona Jan 30, 2024
a2a1fb4
blackify
ankona Jan 30, 2024
f3db48e
move telmon logs to subdirectory
ankona Jan 30, 2024
ec1c91c
remove extraneous call to super init
ankona Jan 30, 2024
1f8ac89
Implement collector callback to disable collection after unmanaged en…
ankona Jan 30, 2024
e484adc
share collector mocks, add hints
ankona Jan 31, 2024
4e71660
fix ms accuracy bug
ankona Jan 31, 2024
1572d11
solve early shutdown bug
ankona Jan 31, 2024
acd8d95
whitespace
ankona Jan 31, 2024
045e2e3
formatting
ankona Jan 31, 2024
6947e01
remove extraneous optional params
ankona Jan 31, 2024
1cfccb2
fmt
ankona Jan 31, 2024
62ff1e7
fix `can_shutdown` test
ankona Jan 31, 2024
df9eb7e
fmt
ankona Jan 31, 2024
c9a460c
fix bad fstring in test data
ankona Feb 2, 2024
a78821d
fix missing ping mock method on mock_redis
ankona Feb 2, 2024
6df2708
default telemetry to off for entity fixture
ankona Feb 2, 2024
97c9410
add tests for collector manager.find_collectors
ankona Feb 2, 2024
0686af3
add timeout tests back in
ankona Feb 2, 2024
626174a
avoid multiline reformats
ankona Feb 2, 2024
0d24a05
fix tests
ankona Feb 2, 2024
c30c230
fmt
ankona Feb 2, 2024
1d6dfa6
use self in place of class on static
ankona Feb 2, 2024
342976c
remove extraneous list creation
ankona Feb 2, 2024
27781c3
fix observer->directory chicken/egg problem
ankona Feb 2, 2024
4d28929
do not use classname in static call
ankona Feb 2, 2024
a662a49
do not use classname in static call
ankona Feb 2, 2024
b3b83b3
unnecessary dataclass inheritance
ankona Feb 2, 2024
1c8755b
review mods
ankona Feb 3, 2024
69d0e71
rename collectors w/correct naming standard
ankona Feb 3, 2024
9f36992
remove extranous value properties from collectors
ankona Feb 3, 2024
e8aa6f2
import bug
ankona Feb 3, 2024
2b5f1c9
remove unused collector _value attribute
ankona Feb 3, 2024
4f5e597
remove unnecessary lambda
ankona Feb 3, 2024
8949d36
simplify dupe check
ankona Feb 3, 2024
8cf7c31
make removal of collectors async
ankona Feb 3, 2024
816ea89
fix iterable vs *args bug
ankona Feb 3, 2024
33d6aaf
asyncify prepare
ankona Feb 3, 2024
fe2a896
fix hardcoded err messages, formatting
ankona Feb 3, 2024
6c0bb08
avoid duplication of timestamp code
ankona Feb 3, 2024
b46805c
improve chaining, fix broken test mock
ankona Feb 3, 2024
bfae744
hint fix
ankona Feb 3, 2024
262ea31
fmt
ankona Feb 3, 2024
d2a9922
mypy
ankona Feb 3, 2024
bcdea65
fmt
ankona Feb 3, 2024
13e5528
remove unnecessary exception handler
ankona Feb 5, 2024
7de8acf
fix incorrect comments
ankona Feb 6, 2024
d1e9e3c
move local import to global import
ankona Feb 6, 2024
e1ff30e
fix misleading test description
ankona Feb 6, 2024
3f50b5d
fix tests writing outputs to test root directory
ankona Feb 6, 2024
c0fbf59
reduce assert to necessary clause
ankona Feb 6, 2024
b47893b
remove unused mock parameter
ankona Feb 6, 2024
80c61e8
fix to meet naming standard
ankona Feb 6, 2024
7becfde
remove unnecessary assert clause
ankona Feb 6, 2024
fc249ae
remove extraneous param
ankona Feb 6, 2024
a1f2664
improve test assertion
ankona Feb 6, 2024
d5a87c2
remove duplicate test
ankona Feb 6, 2024
ad07036
use {} instead of set()
ankona Feb 6, 2024
510f1a9
use equality operator & mock timestamp
ankona Feb 6, 2024
c8de925
restructure ugly whiles to for loops
ankona Feb 6, 2024
b6eebb3
fix incorrect parametrized value
ankona Feb 6, 2024
4113098
remove extraneous assert
ankona Feb 6, 2024
664e3a6
fix collector task timeout ignoring pending tasks
ankona Feb 6, 2024
60a36d8
rename methods that should be private
ankona Feb 6, 2024
81c5e74
remove unnecessary method
ankona Feb 6, 2024
db487d1
replace kwargs with args in Sink API
ankona Feb 6, 2024
e4fc192
tweak sink save calls
ankona Feb 6, 2024
7964047
remove unnecessary result object comparison
ankona Feb 6, 2024
573ff6e
remove incorrect docstring
ankona Feb 6, 2024
0c98b83
Separate task completed logic from base TaskStatusHandler
ankona Feb 6, 2024
34f71af
tweak file handler overrides
ankona Feb 6, 2024
f8b3dfb
fmt
ankona Feb 6, 2024
b489f5b
use presence of collector config to indicate enabled/disabled
ankona Feb 7, 2024
081071c
remove unused type:ignore
ankona Feb 7, 2024
ee42761
limit mock client count to real-world max
ankona Feb 7, 2024
fc57472
remove anyio dep
ankona Feb 9, 2024
527e531
revert removal of watchdog type:ignore
ankona Feb 9, 2024
c72698b
fix sec/ms comparison; early telmon shutdown bug
ankona Feb 22, 2024
caa5e75
fix bug when rerunning experiment without cleaning exp_dir
ankona Feb 22, 2024
31e93b1
fmt
ankona Feb 23, 2024
b29c7cc
update tests for exp_id
ankona Feb 23, 2024
667791d
lint
ankona Feb 23, 2024
0d88154
black
ankona Feb 23, 2024
44f3d1f
import bug
ankona Feb 23, 2024
d18f4cb
fix for telemetry monitor hijacking experiment output
ankona Feb 29, 2024
70601e9
handle unexpected process failure
ankona Mar 1, 2024
954c15c
fix lint error (unused ex)
ankona Mar 1, 2024
68b4425
fix unformatted telemetry monitor logs
ankona Mar 1, 2024
4dd88d5
avoid possibility of NotImplementedError
ankona Mar 1, 2024
5d20d05
avoid unpacking unused result
ankona Mar 1, 2024
52eb4ae
remove unnecessary branch
ankona Mar 1, 2024
7105944
readability
ankona Mar 1, 2024
1fd528c
mypy
ankona Mar 1, 2024
ad999b8
make collector / sink interactions more consistent
ankona Mar 1, 2024
a987879
fix seconds/milliseconds mismatch
ankona Mar 1, 2024
734ed6d
fix ms/sec mismatches in tests, reduce snooze times for output file w…
ankona Mar 1, 2024
e4433fb
reduce nesting by reversing implementation location
ankona Mar 1, 2024
24656c0
fix invalid name assignment
ankona Mar 1, 2024
b1926ea
fix key typo
ankona Mar 1, 2024
da39ce0
replace iterable typehint w/sequence
ankona Mar 4, 2024
73e3e12
use monkeypatch.setenv instead of manual env patching
ankona Mar 4, 2024
e060988
cancel tasks after asyncio.wait timeout
ankona Mar 4, 2024
5c4fab1
Merge branch 'develop' into 612
ankona Mar 4, 2024
d78815e
fix fake pid that can be found on a runner by accident
ankona Mar 4, 2024
e9d82ed
test fix
ankona Mar 4, 2024
5423995
split telmon into subpackage and entrypoint
ankona Mar 4, 2024
3e3f4c8
fix filling telmon w/unmanaged tasks
ankona Mar 4, 2024
02f92fc
allow manifest writing to complete prior to processing
ankona Mar 4, 2024
d63cbdb
avoid file system
ankona Mar 6, 2024
be88062
avoid looking for physical stop events in telmon test
ankona Mar 6, 2024
826d515
fix failure to count
ankona Mar 6, 2024
0a38e66
ensure databases are tracked
ankona Mar 7, 2024
57ede39
separate status checks for unmanaged entities
ankona Mar 7, 2024
f0ec254
lint
ankona Mar 7, 2024
0933107
merge develop into feature
ankona Mar 13, 2024
f1e81b7
Reorganize telemetry monitor code into subpackage (#12)
ankona Mar 13, 2024
4811284
revert unintended removal
ankona Mar 13, 2024
8d35c87
make telmon docstring more consistent w/originals
ankona Mar 13, 2024
359dc5e
docstring improvements
ankona Mar 13, 2024
2130f42
docstrings
ankona Mar 13, 2024
fb278fb
cleanup
ankona Mar 13, 2024
ddafa04
revert tiny change
ankona Mar 13, 2024
63a4920
complete docstring
ankona Mar 13, 2024
70a25d9
missed collector docstrings
ankona Mar 13, 2024
7df028a
Add missing collectormanager class docstring
ankona Mar 13, 2024
339675d
moar
ankona Mar 13, 2024
11986f6
Tweak class name descriptiveness, add docstrings
ankona Mar 13, 2024
2d20687
review tweaks
ankona Mar 13, 2024
10afab8
don't keep tasks list around on class
ankona Mar 13, 2024
e1826e3
fix completion status bug
ankona Mar 14, 2024
bcb18a5
remove commented code
ankona Mar 14, 2024
c0aeaa0
remove commented code
ankona Mar 14, 2024
06ecc69
remove "multi-exp driver" support (exp_id)
ankona Mar 14, 2024
5be84ad
formatting
ankona Mar 14, 2024
1534bea
test hostname filter to satisfy message format
ankona Mar 14, 2024
ce029c4
mani test
ankona Mar 15, 2024
4e2c24a
black fmt fail
ankona Mar 15, 2024
edf990b
tweak & add missing docstrings
ankona Mar 15, 2024
9750528
reduce retry delay on manifest reading
ankona Mar 15, 2024
6276e08
add test confirming non-overwriing of write_event
ankona Mar 15, 2024
776f654
update changelog
ankona Mar 15, 2024
df03761
merge develop
ankona Mar 15, 2024
6262486
add comment for clarity
ankona Mar 15, 2024
eaadfc5
Merge branch 'develop' into 612
ankona Mar 16, 2024
cbe0170
docstring fixes
ankona Mar 19, 2024
6252e57
Merge branch 'develop' into 612
ankona Mar 19, 2024
ffa9d58
fixes per code review
ankona Mar 20, 2024
21dbe24
fix failure to return early bug
ankona Mar 20, 2024
5da35ef
Merge branch 'develop' into 612
ankona Mar 20, 2024
104d189
Merge branch 'develop' into 612
ankona Mar 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
black
  • Loading branch information
ankona committed Feb 22, 2024
commit e5b1f728fead50e456947ba15c263435879f4363
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,6 @@ smartsim/_core/lib
**/*.err
**/*.out
**/.smartsim/*

# optional dev tools
.pre-commit-config.yaml
2 changes: 1 addition & 1 deletion smartsim/_core/utils/serialize.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ def _dictify_db(
"step_id": step_id,
"task_id": task_id,
"managed": managed,
"collectors": db.is_telemetry_on,
"collectors": db.is_telemetry_on,
},
}
for dbnode, (
Expand Down
10 changes: 5 additions & 5 deletions tests/test_collector_sink.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,11 @@ async def test_sink_null_entity() -> None:


@pytest.mark.parametrize(
"name,type",
[
pytest.param("e1", "", id="No entity type"),
pytest.param("e2", "ensemble", id="Has entity type"),
]
"name,type",
[
pytest.param("e1", "", id="No entity type"),
pytest.param("e2", "ensemble", id="Has entity type"),
],
)
@pytest.mark.asyncio
async def test_sink_null_filename(mock_entity, name, type) -> None:
Expand Down
4 changes: 3 additions & 1 deletion tests/test_experiment.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,9 @@ def test_enable_disable_telemetry(monkeypatch, test_dir, config):
exp.start()
import pathlib

mani_path = pathlib.Path(test_dir) / config.telemetry_subdir / serialize.MANIFEST_FILENAME
mani_path = (
pathlib.Path(test_dir) / config.telemetry_subdir / serialize.MANIFEST_FILENAME
)
assert mani_path.exists()


Expand Down