Skip to content

Merge #11517: Tests: Improve benchmark precision #3620

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 20, 2020
Merged

Merge #11517: Tests: Improve benchmark precision #3620

merged 2 commits into from
Jul 20, 2020

Conversation

@PastaPastaPasta
Copy link
Member

@PastaPastaPasta PastaPastaPasta commented Jul 19, 2020

Decided to extract this out of #3618

There are a number of benchmarks we need to fix before this branch will compile. But I don't really understand where their numbers came from.

@laanwj @PastaPastaPasta
Merge bitcoin#11517: Tests: Improve benchmark precision
7ccd49a 
760af84 Removed CCheckQueueSpeed benchmark (Martin Ankerl)
00721e6 Improved microbenchmarking with multiple features. (Martin Ankerl)

Pull request description:

  The benchmark's KeepRunning() used to make a function call for each call, inflating measurement times for short running code. This change inlines the critical code that is executed each run and moves the slow timer updates into a new function.

  This change increases the average runtime for Trig from 0.000000082339208 sec to 0.000000080948591.

Tree-SHA512: 36b3bc55fc9b1d4cbf526b7103af6af18e9783e6b8f3ad3adbd09fac0bf9401cfefad58fd1e6fa2615d3c4e677998f912f3323d61d7b00b1c660d581c257d577
Signed-off-by: pasta <pasta@dashboost.org>

# Conflicts:
#	src/bench/bench.cpp
#	src/bench/bench_dash.cpp
#	src/bench/crypto_hash.cpp
#	src/bench/prevector_destructor.cpp
#	src/bench/verify_script.cpp
@UdjinM6 UdjinM6 added this to the 17 milestone Jul 19, 2020
UdjinM6
UdjinM6 requested changes Jul 19, 2020
View reviewed changes
Copy link

@UdjinM6 UdjinM6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pls see a22b937

re numbers: the idea is to target each benchmark to run ~1 second (for whatever reason) i.e. total should be ~5 seconds because default number of measurement evaluations is 5

UdjinM6
UdjinM6 approved these changes Jul 19, 2020
View reviewed changes
Copy link

@UdjinM6 UdjinM6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@UdjinM6 UdjinM6 merged commit 83726d2 into dashpay:develop Jul 20, 2020
gades pushed a commit to cosanta/cosanta-core that referenced this pull request Jun 30, 2021
@PastaPastaPasta @laanwj
@UdjinM6 @gades
Merge bitcoin#11517: Tests: Improve benchmark precision (dashpay#3620)
cf5d8c4 
* Merge bitcoin#11517: Tests: Improve benchmark precision

760af84 Removed CCheckQueueSpeed benchmark (Martin Ankerl)
00721e6 Improved microbenchmarking with multiple features. (Martin Ankerl)

Pull request description:

  The benchmark's KeepRunning() used to make a function call for each call, inflating measurement times for short running code. This change inlines the critical code that is executed each run and moves the slow timer updates into a new function.

  This change increases the average runtime for Trig from 0.000000082339208 sec to 0.000000080948591.

Tree-SHA512: 36b3bc55fc9b1d4cbf526b7103af6af18e9783e6b8f3ad3adbd09fac0bf9401cfefad58fd1e6fa2615d3c4e677998f912f3323d61d7b00b1c660d581c257d577
Signed-off-by: pasta <pasta@dashboost.org>

* More of 11517

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
@PastaPastaPasta PastaPastaPasta deleted the backport-11517 branch September 11, 2021 20:09
gades pushed a commit to cosanta/cosanta-core that referenced this pull request Mar 11, 2022
@PastaPastaPasta @laanwj
@UdjinM6 @gades
Merge bitcoin#11517: Tests: Improve benchmark precision (dashpay#3620)
7a12478 
* Merge bitcoin#11517: Tests: Improve benchmark precision

760af84 Removed CCheckQueueSpeed benchmark (Martin Ankerl)
00721e6 Improved microbenchmarking with multiple features. (Martin Ankerl)

Pull request description:

  The benchmark's KeepRunning() used to make a function call for each call, inflating measurement times for short running code. This change inlines the critical code that is executed each run and moves the slow timer updates into a new function.

  This change increases the average runtime for Trig from 0.000000082339208 sec to 0.000000080948591.

Tree-SHA512: 36b3bc55fc9b1d4cbf526b7103af6af18e9783e6b8f3ad3adbd09fac0bf9401cfefad58fd1e6fa2615d3c4e677998f912f3323d61d7b00b1c660d581c257d577
Signed-off-by: pasta <pasta@dashboost.org>

* More of 11517

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
dekm pushed a commit to unigrid-project/daemon that referenced this pull request Oct 27, 2022
@fanquake
Merge bitcoin#21655: build, qt: No longer need to set QT_RCC_TEST=1 f…
fae9a2c 
…or determinism

c799a19 build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)

Pull request description:

  The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
  > This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.

  Since dashpay#3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.

  Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
  See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](qt/qtbase@5283a6c).

ACKs for top commit:
  fanquake:
    ACK c799a19

Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
knst pushed a commit to knst/dash that referenced this pull request Feb 23, 2023
@fanquake @knst
Merge bitcoin#21655: build, qt: No longer need to set QT_RCC_TEST=1 f…
85ad344 
…or determinism

c799a19 build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)

Pull request description:

  The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
  > This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.

  Since dashpay#3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.

  Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
  See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](qt/qtbase@5283a6c).

ACKs for top commit:
  fanquake:
    ACK c799a19

Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
knst pushed a commit to knst/dash that referenced this pull request Mar 6, 2023
@fanquake @knst
Merge bitcoin#21655: build, qt: No longer need to set QT_RCC_TEST=1 f…
03a8f01 
…or determinism

c799a19 build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)

Pull request description:

  The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
  > This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.

  Since dashpay#3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.

  Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
  See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](qt/qtbase@5283a6c).

ACKs for top commit:
  fanquake:
    ACK c799a19

Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
knst pushed a commit to knst/dash that referenced this pull request Mar 13, 2023
@fanquake @knst
Merge bitcoin#21655: build, qt: No longer need to set QT_RCC_TEST=1 f…
88c99f7 
…or determinism

c799a19 build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)

Pull request description:

  The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
  > This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.

  Since dashpay#3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.

  Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
  See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](qt/qtbase@5283a6c).

ACKs for top commit:
  fanquake:
    ACK c799a19

Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
PastaPastaPasta pushed a commit to knst/dash that referenced this pull request Mar 26, 2023
@fanquake @PastaPastaPasta
Merge bitcoin#21655: build, qt: No longer need to set QT_RCC_TEST=1 f…
c62a180 
…or determinism

c799a19 build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)

Pull request description:

  The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
  > This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.

  Since dashpay#3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.

  Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
  See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](qt/qtbase@5283a6c).

ACKs for top commit:
  fanquake:
    ACK c799a19

Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@UdjinM6 UdjinM6 UdjinM6 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

17

Development

Successfully merging this pull request may close these issues.

3 participants

@PastaPastaPasta @UdjinM6 @laanwj