-
Notifications
You must be signed in to change notification settings - Fork 630
Refactor PerfZero for simplicity and ease-of-use #292
Refactor PerfZero for simplicity and ease-of-use #292
Conversation
9ecb008 to
183975b
Compare
675bb9f to
561211d
Compare
3e464b8 to
d798e18
Compare
Add leading_indicators_test.py to benchmarks/scripts and run it in PerfZero Read benchmark results from local protobuf file written by tf.test.Benchmark.report_benchmark() Simplify Perfzero report logic implementation Rename environment variables from ROGUE_* to PERFZERO_* Replace print(..) with logging.info(..) and logging.debug(..) Print benchmark summary in a human readable format Use datetime string as execution id and include it in the output path name Specify full list of environment variables with documentation in README.md Configure bigquery table name via environment variable Upload gpu driver version to bigquery Print messages to both stdout and log file
d798e18 to
d4785fc
Compare
| conn.close() | ||
|
|
||
|
|
||
| #def upload_with_stream_mode(client, dataset, table, row): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Stream should be the default and needs to exist. I think you are swapping back in the current reporting. I think it makes sense to not change the reporting structure until we have a strong need. Changing it does not get us a lot of value right now and adds a lot of work. I realize the current structure is a bit messy right now.
|
|
||
| ml_framework_info = {} | ||
| ml_framework_info['framework'] = 'tensorflow' | ||
| ml_framework_info['version'] = tf.__version__ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left this off as I was testing pytorch and mxnet with the same setup and wanted to share reporting. I could be talked into keeping this here but I kind of prefer we not require tensorflow to do reporting.
|
|
||
| def build_execution_summary(execution_id, project_name, platform_name, | ||
| output_url, benchmark_result): | ||
| import tensorflow as tf |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you do keep this you need a pylint disable I think.
This patch makes the following improvement. It depends on cl/230280956 which adds leading_indicators_test.py to OSS tensorflow/benchmarks.
The information above is translated to old format and uploaded to bigquery.
The PerfZero configuration is now documented in README.md