Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bfs/exec.c: fix failure with large 64k ppc64le page size #34

Closed
wants to merge 1 commit into from

Conversation

mksully22
Copy link
Contributor

@mksully22 mksully22 commented Jul 6, 2018

test_exec_plus test fails because bfs_exec_arg_max() call to sysconf(_SC_ARG_MAX) is not large enough to contain two 64K pages (ppc64le default page size). Since maximum argument size allowed by BFS is only 16K anyway (BFS_EXEC_ARG_MAX) then update arg size check to make sure it is at least 32k. Note that this is occurring on Alpine Linux edge build.

Copy link
Owner

@tavianator tavianator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your pull request! A few comments:

test_exec_plus test fails because bfs_exec_arg_max() call to sysconf(_SC_ARG_MAX) is not large enough to contain two 64K pages (ppc64le default page size)

Any idea why it's so low? Note that the build seems to be fine on Debian ppc64, for example, so there might be some weird configuration in Alpine Linux. I'd be curious to see the output of

bfs -D exec -exec echo {} + -quit

and maybe

xargs --show-limits

(with GNU xargs) on an Alpine ppc64 system.

Since maximum argument size allowed by BFS is only 16K anyway (BFS_EXEC_ARG_MAX)

BFS_EXEC_ARG_MAX is 16 MiB (16*1024*1024).

then update arg size check to make sure it is at least 32k. Note that this is occurring on Alpine Linux edge build.

I'm a bit worried this is just papering over some other issue. I'd like to see the debugging output requested above before deciding the best way to fix this.

@@ -95,6 +95,10 @@ static size_t bfs_exec_arg_max(const struct bfs_exec *execbuf) {
long page_size = sysconf(_SC_PAGESIZE);
if (page_size < 4096) {
page_size = 4096;
} else {
if (page_size > (BFS_EXEC_ARG_MAX/1024)) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style: this should just be

} else if (page_size > BFS_EXEC_ARG_MAX/1024) {

(but see the other comments)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

~/bfs $ /home/mksully/aports/testing/bfs/src/bfs-1.2.2/bfs -D exec -exec echo {} + -quit
-exec: ARG_MAX: 131072 according to sysconf()
-exec: ARG_MAX: 130772 remaining after environment variables
-exec: ARG_MAX: 130751 remaining after fixed arguments
-exec: ARG_MAX: 97983 remaining after headroom
-exec: ARG_MAX: 97983 final value
-exec: Finishing execution, executing buffered command
-exec: Executing 'echo' ... [1 arguments] (size 10)

On Alpine xargs doesn't support the --show-limits options.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, the limit starts at only 128K, so obviously subtracting two 64K pages from that will leave no space. (That suggests that Alpine ppc64le has only a 1MiB limit by default which seems rather low, but whatever.)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*a 1MiB stack limit

@@ -95,6 +95,10 @@ static size_t bfs_exec_arg_max(const struct bfs_exec *execbuf) {
long page_size = sysconf(_SC_PAGESIZE);
if (page_size < 4096) {
page_size = 4096;
} else {
if (page_size > (BFS_EXEC_ARG_MAX/1024)) {
page_size = BFS_EXEC_ARG_MAX/1024;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big fan of variable names that lie. It's not super obvious what the point of messing with page_size in this way is -- I guess the idea is to avoid subtracting an excessive amount of headroom? Instead, maybe it's better to raise the lower limit at the end, like this:

	if (arg_max < page_size) {
		arg_max = page_size;
	} else if (arg_max > BFS_EXEC_ARG_MAX) {
		arg_max = BFS_EXEC_ARG_MAX;
	}

That's not as dangerous as it would have been in the past, since bfs now detects and recovers from E2BIG if the limit really was super low.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that is a much better and more straightforward solution. I tested it and it works fine on Alpine ppc64le. Do you want me to modify the PR?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for testing! No need for you to update this PR, I ended up taking a slightly different approach in 30b4874. If that doesn't work let me know!

@tavianator
Copy link
Owner

The issue should be fixed by 30b4874

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants