Skip to content

Conversation

@trunglebka
Copy link
Contributor

I've done some profiling and found that bytes(data_ptr[: int(data_len)]) is an expensive operation and can be done by ctypes.string_at instead.

The benchmark result:

Correctness: Pass
Method 1 (slicing) avg time: 0.016509 seconds
Method 2 (string_at) avg time: 0.000130 seconds

The benchmarking code:

import ctypes
import random
import time


def create_cdata():
    test_data = bytes(
        random.getrandbits(8) for _ in range(10**6)
    )  # 1 MB of random data
    data_len = len(test_data)

    # Create a ctypes pointer to the test data
    c_array = (ctypes.c_uint8 * data_len)(*test_data)
    data_ptr = ctypes.cast(c_array, ctypes.POINTER(ctypes.c_uint8))
    return data_ptr, data_len


def benchmark_and_validate(iterations=100):
    # Generate random test data
    # Variables to accumulate time
    total_time1 = 0.0
    total_time2 = 0.0
    is_correct = True

    for _ in range(iterations):
        data_ptr, data_len=create_cdata()
        # Method 2: Using ctypes.string_at
        start_time = time.perf_counter()
        event_data2 = ctypes.string_at(data_ptr, data_len)
        total_time2 += time.perf_counter() - start_time

        # Method 1: Using slicing
        start_time = time.perf_counter()
        event_data1 = bytes(data_ptr[:data_len])
        total_time1 += time.perf_counter() - start_time

        # Check correctness
        if event_data1 != event_data2:
            is_correct = False
            break

    # Average times
    avg_time1 = total_time1 / iterations
    avg_time2 = total_time2 / iterations

    # Print results
    print(f"Correctness: {'Pass' if is_correct else 'Fail'}")
    print(f"Method 1 (slicing) avg time: {avg_time1:.6f} seconds")
    print(f"Method 2 (string_at) avg time: {avg_time2:.6f} seconds")

    return avg_time1, avg_time2, is_correct


if __name__ == "__main__":
    benchmark_and_validate(iterations=1000)


@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Charlie seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@davidzhao davidzhao requested a review from theomonnom January 7, 2025 07:06
@theomonnom
Copy link
Member

Hey, this lg! Can you sign the CLA?

@trunglebka
Copy link
Contributor Author

Hi @theomonnom,
I signed the CLA, but it seems like it got messed up with my profile, and there's another location to modify. I will create another PR

@trunglebka trunglebka closed this Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants