1.11

[MariusWirtz]

Highlights

1. Improved performance #882, #885

With this release TM1py makes better use of existing capabilities in the TM1 server to handle large read and write operations.

The write function

Pass use_blob=True to the write or write_dataframe function to use the new optimized write mode.

with TM1Service(**params) as tm1:
    cells = {
        ('Actual', 'Germany', 'T Series 4.0 L Sedan', 'Units', 'Jan'): 1500,
        ('Actual', 'Germany', 'T Series 4.0 L Sedan', 'Units', 'Feb'): 2100,
        ('Actual', 'Germany', 'T Series 4.0 L Sedan', 'Units', 'Mar'): 1100
    }
    tm1.cells.write("SalesCube", cells, use_blob=True)

On a sample of 1 million cell updates, this performs on par with Turbo Integrator and up to 6x faster than the previous write function. #882 (comment)

---

The write_async function

The write_async function now makes use of use_blob by default. The number of parallel worker threads in TM1 can be controlled through the max_workers argument. A reasonable value for slice_size needs to be provided.

E.g. if you are expecting 1 million cell updates, a reasonable choice would be 10 for max_workers and 100000 for slice_size.

with TM1Service(**params) as tm1:
    tm1.cells.write_async(
        cube_name="SalesCube",
        cells=cells,
        slice_size=100_000,
        max_workers=10)

On a sample of 1 million cell updates, this performs ~5 times better than plain Turbo Integrator and ~5 times better than the previous write_async function. #882 (comment)

---

The execute_mdx_dataframe, execute_view_dataframe functions

Pass use_blob=True to the execute_mdx_dataframe, and execute_view_dataframe functions to use the new optimized read mode.

with TM1Service(**sdata_params) as tm1:
    df = tm1.cells.execute_view_dataframe(cube_name="Sales Cube", view_name="Default", private=False, use_blob=True)

On large data sets this performs 20% to 40% better. It also reduces the memory footprint by ~70%. #885 (comment)

2. New shaped argument in the execute_view_dataframe function #893

Pass shaped=True to the execute_mdx_dataframe, and execute_view_dataframe functions to retrieve the data frame in the original shape of the cube view.

with TM1Service(**sdata_params) as tm1:
    df = tm1.cells.execute_view_dataframe(
        cube_name="Sales Cube",
        view_name="Default",
        shaped=True,
        use_blob=True)

	account1	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec
0	Gross Margin%	57.5155	56.4074	59.1279	59.5385	57.7353	57.1263	58.4302	59.7887	56.9305	56.9046	59.8059	56.2537
1	Price	19360	19532	19218.5	19631	19941	19870	19365	19216	19241	19099.5	19883.5	19432.5
2	Units	1	2	2	1	1	1	1	1	1	2	1	1
3	Sales	19.36	39.064	38.437	19.631	19.941	19.87	19.365	19.216	19.241	38.199	19.8835	19.4325
4	Variable Costs	8.225	17.029	15.71	7.943	8.428	8.519	8.05	7.727	8.287	16.462	7.992	8.501

Please note the months on the columns in the table above

3. Re-authenticate after session timeout #856

Now TM1py will re-authenticate and retry the operation once if the operation failed due to a session timeout.

This is helpful in situations where a TM1py script has been idle for a while and once it attempts to restart interaction with TM1 it fails due to a session timeout.

New Features, Improvements, and Bugfixes

• Fully support TM1 sandbox functionality by @adscheevel in Sandbox updates #844
• Support use of alternate hierarchies in get_elements_dataframe function by @MariusWirtz in Support hierarchies in get_elements_dataframe #876
• Allow dtype to be inferred by pandas with argument infer_dtype in execute_mdx_dataframe_shaped by @Kevin-Dekker in Allow dtype to be inferred by pandas with argument infer_dtype in execute_mdx_dataframe_shaped #879

Acknowledgments

Big thanks to @cwffonseca, @Kevin-Dekker, and @adscheevel for contributing code to this release, and many others for reporting bugs and requesting new features.

How to upgrade TM1py

pip install tm1py --upgrade

Full Changelog: 1.10.2...1.11

---

This discussion was created from the release 1.11.