Skip to content

Conversation

@Kevin-Dekker
Copy link
Collaborator

  • Use dict(zip(keys, values)) instead of a for loop

  • Use itertuples to make each row until last column of the dataframe into a tuple

Efficiency Statistics (1000 iterations)

image

Larger dataframe:
image

Comparison in markdown tables:

Current (before commit/merge)

length: 1000 width: 3 length: 1000 width: 13 length: 5000 width: 3 length: 5000 width: 13 length: 15000 width: 3 length: 15000 width: 13
mean 0.0049 0.0179 0.0477 0.1101 0.1464 0.3459
std 0.0059 0.0079 0.0166 0.026 0.0308 0.0602
max 0.0472 0.0642 0.1056 0.2145 0.2607 0.6604

Suggested (after commit/merge)

length: 1000 width: 3 length: 1000 width: 13 length: 5000 width: 3 length: 5000 width: 13 length: 15000 width: 3 length: 15000 width: 13
mean 0.0016 0.0069 0.0157 0.0425 0.0473 0.1294
std 0.0036 0.0046 0.0053 0.0121 0.0123 0.0269
max 0.0167 0.0282 0.038 0.0934 0.094 0.2652

- Use dict(zip(keys, values)) instead of a for loop

- Use itertuples to make each row until last column of the dataframe into a tuple

Efficiency Statistics (1000 iterations)

Comparison in markdown tables:

Current (before commit/merge)
|      |   length: 1000 width: 3 |   length: 1000 width: 13 |   length: 5000 width: 3 |   length: 5000 width: 13 |   length: 15000 width: 3 |   length: 15000 width: 13 |
|:-----|------------------------:|-------------------------:|------------------------:|-------------------------:|-------------------------:|--------------------------:|
| mean |                  0.0049 |                   0.0179 |                  0.0477 |                   0.1101 |                   0.1464 |                    0.3459 |
| std  |                  0.0059 |                   0.0079 |                  0.0166 |                   0.026  |                   0.0308 |                    0.0602 |
| max  |                  0.0472 |                   0.0642 |                  0.1056 |                   0.2145 |                   0.2607 |                    0.6604 |

Suggested (after commit/merge)
|      |   length: 1000 width: 3 |   length: 1000 width: 13 |   length: 5000 width: 3 |   length: 5000 width: 13 |   length: 15000 width: 3 |   length: 15000 width: 13 |
|:-----|------------------------:|-------------------------:|------------------------:|-------------------------:|-------------------------:|--------------------------:|
| mean |                  0.0016 |                   0.0069 |                  0.0157 |                   0.0425 |                   0.0473 |                    0.1294 |
| std  |                  0.0036 |                   0.0046 |                  0.0053 |                   0.0121 |                   0.0123 |                    0.0269 |
| max  |                  0.0167 |                   0.0282 |                  0.038  |                   0.0934 |                   0.094  |                    0.2652 |
@MariusWirtz
Copy link
Collaborator

MariusWirtz commented Mar 16, 2022

Nice work @Kevin-Dekker. Especially the larger TM1py implementations will benefit from this.

I think we may even want to create a minor release on PyPI after the merge.

@MariusWirtz MariusWirtz merged commit 5fb7758 into cubewise-code:master Mar 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants