Performance Profiling

[Flamenate]

I not an expert in profiling code and I've only used cProfile once on another project.
I was wondering how I could profile my Bot's memory usage & time per function call using the same tool or other tools if possible.

[Theelx]

For CPU profiling, line-by-line, I'd use line_profiler. The official library, which can be installed off PyPi with pip install line_profiler, doesn't have support for generating profiler results every x seconds, it only generates profiler results when the bot stops. I use kernprof -lvz (file name here, like run.py). That's why I made a fork that has a command line flag to output to a file every x seconds. With this, you can do something like kernprof -lvzi 15 (file name here), and it'll output results to a file every 15 seconds. You can view profiling results with python3 -m line_profiler ./run.py.lprof (replace run.py with your file name, ending with a .lprof suffix).

To profile a specific function, you can decorate the function with @profile, like

@profile
async def test():
    number = 100
    number /= 10
    print(number)
    return number

The best part is that kernprof injects @profile into the namespace at run time, so if you run using kernprof, you don't need to define the profile decorator yourself. However, to allow for running without kernprof also, you can apply the decorator later, like this:

async def test():
    number = 100
    number /= 10
    print(number)
    return number
    
try:
    test = profile(test)
except NameError:
    pass

.

Recap:
1a. You can profile with the pip installed version, which supports everything except outputting profile results without stopping the bot
1b. You can profile with my fork, which can be installed on Linux as so:

pip uninstall line_profiler
pip install git+https://github.com/Theelgirl/line_profiler.git

2. Decorate the functions you want to profile with @profile.
3. Use kernprof -lvz (file name here) or kernprof -lvzi 15 (file name here) depending on which version you install, to run the profiler
4. Use python3 -m line_profiler ./(file name here).lprof to view profile results

[Flamenate]

This tool is a lifesaver, thank you very much!

Although, I should add for anyone seeing this in the future. Make sure to have these installed before installing the tool:
-gcc
-cMake
-scikit-build (python package)
-python3-dev (if you're on linux)