-
-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bfs/exec.c: fix failure with large 64k ppc64le page size #34
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your pull request! A few comments:
test_exec_plus test fails because bfs_exec_arg_max() call to sysconf(_SC_ARG_MAX) is not large enough to contain two 64K pages (ppc64le default page size)
Any idea why it's so low? Note that the build seems to be fine on Debian ppc64, for example, so there might be some weird configuration in Alpine Linux. I'd be curious to see the output of
bfs -D exec -exec echo {} + -quit
and maybe
xargs --show-limits
(with GNU xargs) on an Alpine ppc64 system.
Since maximum argument size allowed by BFS is only 16K anyway (BFS_EXEC_ARG_MAX)
BFS_EXEC_ARG_MAX
is 16 MiB (16*1024*1024
).
then update arg size check to make sure it is at least 32k. Note that this is occurring on Alpine Linux edge build.
I'm a bit worried this is just papering over some other issue. I'd like to see the debugging output requested above before deciding the best way to fix this.
@@ -95,6 +95,10 @@ static size_t bfs_exec_arg_max(const struct bfs_exec *execbuf) { | |||
long page_size = sysconf(_SC_PAGESIZE); | |||
if (page_size < 4096) { | |||
page_size = 4096; | |||
} else { | |||
if (page_size > (BFS_EXEC_ARG_MAX/1024)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style: this should just be
} else if (page_size > BFS_EXEC_ARG_MAX/1024) {
(but see the other comments)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
~/bfs $ /home/mksully/aports/testing/bfs/src/bfs-1.2.2/bfs -D exec -exec echo {} + -quit
-exec: ARG_MAX: 131072 according to sysconf()
-exec: ARG_MAX: 130772 remaining after environment variables
-exec: ARG_MAX: 130751 remaining after fixed arguments
-exec: ARG_MAX: 97983 remaining after headroom
-exec: ARG_MAX: 97983 final value
-exec: Finishing execution, executing buffered command
-exec: Executing 'echo' ... [1 arguments] (size 10)
On Alpine xargs doesn't support the --show-limits options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah okay, the limit starts at only 128K, so obviously subtracting two 64K pages from that will leave no space. (That suggests that Alpine ppc64le has only a 1MiB limit by default which seems rather low, but whatever.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
*a 1MiB stack limit
@@ -95,6 +95,10 @@ static size_t bfs_exec_arg_max(const struct bfs_exec *execbuf) { | |||
long page_size = sysconf(_SC_PAGESIZE); | |||
if (page_size < 4096) { | |||
page_size = 4096; | |||
} else { | |||
if (page_size > (BFS_EXEC_ARG_MAX/1024)) { | |||
page_size = BFS_EXEC_ARG_MAX/1024; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a big fan of variable names that lie. It's not super obvious what the point of messing with page_size
in this way is -- I guess the idea is to avoid subtracting an excessive amount of headroom? Instead, maybe it's better to raise the lower limit at the end, like this:
if (arg_max < page_size) {
arg_max = page_size;
} else if (arg_max > BFS_EXEC_ARG_MAX) {
arg_max = BFS_EXEC_ARG_MAX;
}
That's not as dangerous as it would have been in the past, since bfs
now detects and recovers from E2BIG
if the limit really was super low.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, that is a much better and more straightforward solution. I tested it and it works fine on Alpine ppc64le. Do you want me to modify the PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thanks for testing! No need for you to update this PR, I ended up taking a slightly different approach in 30b4874. If that doesn't work let me know!
The issue should be fixed by 30b4874 |
test_exec_plus test fails because bfs_exec_arg_max() call to sysconf(_SC_ARG_MAX) is not large enough to contain two 64K pages (ppc64le default page size). Since maximum argument size allowed by BFS is only 16K anyway (BFS_EXEC_ARG_MAX) then update arg size check to make sure it is at least 32k. Note that this is occurring on Alpine Linux edge build.