Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perftest: replace rand() with getrandom() during MR buffer initialization #177

Merged
merged 1 commit into from
Nov 13, 2022

Conversation

HelloTcc
Copy link

@HelloTcc HelloTcc commented Nov 8, 2022

rand() has very poor performance in some OS.

ib_send_bw will spend a lot of time during MR initialization when testing large packects in above scenario.

A simple test has been done:

#define HUGE_MR_SIZE 2147483647
int main(int argc, char *argv[])
{
        char *a = malloc(HUGE_MR_SIZE * sizeof(char));
        unsigned int i;
        char *tmp = a;
        int ret;

        srand(time(NULL));
        if (a == NULL)
                exit(1);

        if (argc <= 1)
                goto fall_back;

        for (i = HUGE_MR_SIZE; i > 0;) {
                ret = getrandom(tmp, i, 0);
                if (ret < 0)
                        goto fall_back;
                tmp += ret;
                i -= ret;
        }
        goto out;

fall_back:
        for(i = 0; i < HUGE_MR_SIZE; i++)
                a[i] = (char)rand();
out:
        free(a);
        return 0;
}
time ./a.out
real    5m35.033s
user    5m33.546s
sys     0m0.918s

time ./a.out 1

real    0m6.454s
user    0m0.000s
sys     0m6.449s

As shown in the test above, getrandom() has a much better performance, so replace rand() with it.
The test in perftest has also been done.

This patch fixes #89

Signed-off-by: Chengchang Tang tangchengchang@huawei.com

Copy link
Contributor

@HassanKhadour HassanKhadour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!

configure.ac Outdated
@@ -60,6 +60,7 @@ AC_PROG_LIBTOOL
AC_PROG_RANLIB
AC_HEADER_STDC
AC_CHECK_HEADERS([infiniband/verbs.h],,[AC_MSG_ERROR([ibverbs header files not found])])
AC_CHECK_HEADERS([sys/random.h],,[AC_MSG_ERROR([getrandom available])])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"getrandom available" is uncorrect message, should be not available/found instead

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks,my mistake.

configure.ac Outdated
@@ -60,6 +60,7 @@ AC_PROG_LIBTOOL
AC_PROG_RANLIB
AC_HEADER_STDC
AC_CHECK_HEADERS([infiniband/verbs.h],,[AC_MSG_ERROR([ibverbs header files not found])])
AC_CHECK_HEADERS([sys/random.h],,[AC_MSG_ERROR([getrandom not found])])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for this, I just checked witha setup that doesn't have sys/random.h and it failed to compile,I think it should be checked with ac_try_link because now if we don't have sys/random.h the configuration will fail and the HAVE_SYS_RANDOM_H will not help us in this situation

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it could be fixed by removing the AC_MSG_ERROR. I test it by removing my sys/random.h, and it works.

It seems that the log sys/random.h... no is enough

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great, it solved the issue.
Thanks for your contribution.

…tion

rand() has very poor performance in some OS.

ib_send_bw will spend a lot of time during MR initialization when
testing large packects in above scenario.

test has been done:
"""
\#define HUGE_MR_SIZE 2147483647
int main(int argc, char *argv[])
{
        char *a = malloc(HUGE_MR_SIZE * sizeof(char));
        unsigned int i;
        char *tmp = a;
        int ret;

        srand(time(NULL));
        if (a == NULL)
                exit(1);

        if (argc <= 1)
                goto fall_back;

        for (i = HUGE_MR_SIZE; i > 0;) {
                ret = getrandom(tmp, i, 0);
                if (ret < 0)
                        goto fall_back;
                tmp += ret;
                i -= ret;
        }
        goto out;

fall_back:
        for(i = 0; i < HUGE_MR_SIZE; i++)
                a[i] = (char)rand();
out:
        free(a);
        return 0;
}

time ./a.out
real    5m35.033s
user    5m33.546s
sys     0m0.918s

time ./a.out 1

real    0m6.454s
user    0m0.000s
sys     0m6.449s
"""

As shown in the test above, getrandom() has a much better performance,
so replace rand() with it.

Signed-off-by: Chengchang Tang <tangchengchang@huawei.com>
@HassanKhadour HassanKhadour merged commit 2f11cc8 into linux-rdma:master Nov 13, 2022
@HelloTcc HelloTcc deleted the rand branch November 22, 2022 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Initializing huge buffers with rand() takes too much time causing iWARP MPA timeout.
2 participants