Skip to content

Fiber: add shadow stack support #9283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Conversation

chen-hu-97
Copy link
Contributor

Shadow stack is part of Intel's Control-Flow Enforcement Technology (CET).

Whenever a function is called, the return address is pushed onto both
the regular stack and the shadow stack. When that function returns, the
return addresses are popped off both stacks and compared; if they fail
to match, #CP raised.

With this commit, we create shadow stack for each fiber context and
switch the shadow stack accordingly during fcontext switch.

Signed-off-by: Chen, Hu hu1.chen@intel.com

@chen-hu-97
Copy link
Contributor Author

With this, all fiber test passed on CET machine. Otherwise, we see below failures.
$ make test TESTS="Zend/tests/fibers"
=====================================================================
Number of tests : 60 60
Tests skipped : 0 ( 0.0%) --------
Tests warned : 0 ( 0.0%) ( 0.0%)
Tests failed : 46 ( 76.7%) ( 76.7%)
Tests passed : 14 ( 23.3%) ( 23.3%)

This doesn't incur new test failure on my local test. Wait for CI results.

@devnexen devnexen requested a review from trowski August 9, 2022 11:54
@cmb69
Copy link
Member

cmb69 commented Aug 9, 2022

Thank you for the PR!

I'm uncomfortable to patch the bundled asm files from boost. Do you see a way to avoid that?

@trowski
Copy link
Member

trowski commented Aug 10, 2022

I would prefer to see the assembly files changed upstream in https://github.com/boostorg/context before modifying the assembly files here if possible.

Can we put the shadow stack into the same allocation as the normal stack? The normal stack pointer can be shifted lower in the make_context assembly with the shadow stack starting at the stack pointer given to make_context. This would avoid another allocation, special handling in zend_fiber_stack_allocate/free, and writing the pointer onto the stack memory.

The ucontext related functions have been updated to support shadow stacks, perhaps some inspiration can be taken from there? https://code.woboq.org/userspace/glibc/sysdeps/unix/sysv/linux/x86_64/

@chen-hu-97
Copy link
Contributor Author

Hi @trowski,
I will try upstreaming boost at first and then php. At the same time, let's polish the patch in the php test environment :)

Can we put the shadow stack into the same allocation as the normal stack?

Do you mean that we allcoate normal stack adjacent to shadow stack (SS)? For example, 0x1000 ~ 0x6000 for normal stack, 0x6000 ~ 0x7000 for SS and pass 0x7000 to make_fcontext?
If this understanding is correct, maybe it's not feasible now:
Say we allocate SS via syscall map_shadow_stack and get address 0x6000~0x7000. Now we allocate normal stack with desired region: mmap(MAP_FIXED, 0x1000~0x6000). The 2nd request is not always met.
Maybe I can talk to the author of map_shadow_stack to allocate adjacent normal/shadow stack at same time.

Current patch has much interaction between .C and .asm. One alternative is to handle all SS stuff in .asm, that is, we allocate SS in make_fcontext and record the base/size/pointer in fcontext. But I don't find a good place to munmap SS....

The ucontext related functions have been updated to support shadow stacks, perhaps some inspiration can be taken from there? https://code.woboq.org/userspace/glibc/sysdeps/unix/sysv/linux/x86_64/

The author give me several advises weeks ago. ucontext record SS info in C struct.

@trowski
Copy link
Member

trowski commented Aug 11, 2022

I was hoping to avoid two allocations for each fiber – but certainly is not a must, just would be nice. I'm looking forward to feedback from the author of map_shadow_stack.

If the shadow stack must be allocated with a separate syscall then I wouldn't attempt to request a specific range from mmap, that sounds too fragile. I don't have a better suggestion at the moment for passing the shadow stack pointer to make_fcontext.

@chen-hu-97
Copy link
Contributor Author

I was hoping to avoid two allocations for each fiber – but certainly is not a must, just would be nice. I'm looking forward to feedback from the author of map_shadow_stack.

If the shadow stack must be allocated with a separate syscall then I wouldn't attempt to request a specific range from mmap, that sounds too fragile. I don't have a better suggestion at the moment for passing the shadow stack pointer to make_fcontext.

Let me upstream this to boost at first. I will propose add one more arg to make_fcontext for passing back the shadow stack pointer. Current passing method in this PR doesn't look good. Once it's done, I will update this PR.

@chen-hu-97
Copy link
Contributor Author

chen-hu-97 commented Aug 12, 2022

Thank you for the PR!

I'm uncomfortable to patch the bundled asm files from boost. Do you see a way to avoid that?

:)
Actually I am somewhat afraid/frustrated of asm programming. But sometimes I can't avoid it .... :(

PeterYang12 added a commit to PeterYang12/context that referenced this pull request Sep 30, 2022
    Shadow stack is part of Intel's Control-Flow Enforcement Technology.

    Whenever a function is called, the return address is pushed onto both
    the regular stack and the shadow stack. When that function returns, the
    return addresses are popped off both stacks and compared; if they fail
    to match, #CP raised.

    Backport this commit from php/php-src#9283
    With this commit, we create shadow stack with syscall map_shadow_stack
    (no.451) for each fiber context and switch the shadow stack accordingly
    during fcontext switch.

Signed-off-by: PeterYang12 <yuhan.yang@intel.com>
Signed-off-by: chen-hu-97 <hu1.chen@intel.com>
@chen-hu-97
Copy link
Contributor Author

I would prefer to see the assembly files changed upstream in https://github.com/boostorg/context before modifying the assembly files here if possible.
...

The boost counterpart has been merged in boostorg/context#207.

@cmb69
Copy link
Member

cmb69 commented Oct 11, 2022

Thank you! I wonder whether we should rename ZEND_FIBER_SHADOW_STACK to BOOST_CONTEXT_SHADOW_STACK to avoid conflicts whenever we update the assembly files in the future. @trowski, thoughts?

@chen-hu-97
Copy link
Contributor Author

Thank you! I wonder whether we should rename ZEND_FIBER_SHADOW_STACK to BOOST_CONTEXT_SHADOW_STACK to avoid conflicts whenever we update the assembly files in the future. @trowski, thoughts?

Will rename ZEND_FIBER_SHADOW_STACK to BOOST_CONTEXT_SHADOW_STACK

@chen-hu-97 chen-hu-97 force-pushed the jit branch 2 times, most recently from ac964fc to f152c18 Compare November 2, 2022 07:00
Shadow stack is part of Intel's Control-Flow Enforcement Technology (CET).

Whenever a function is called, the return address is pushed onto both
the regular stack and the shadow stack. When that function returns, the
return addresses are popped off both stacks and compared; if they fail
to match, #CP raised.

With this commit, we create shadow stack for each fiber context and
switch the shadow stack accordingly during fcontext switch.

Signed-off-by: Chen, Hu <hu1.chen@intel.com>
@chen-hu-97
Copy link
Contributor Author

Update this PR to rename ZEND_FIBER_SHADOW_STACK to BOOST_CONTEXT_SHADOW_STACK

Thanks for @cmb69's comments. Sorry for the latency.

@cmb69
Copy link
Member

cmb69 commented Nov 2, 2022

Thank you! From a cursory glance, this looks good to me, but maybe @trowski can have a closer look.

@trowski
Copy link
Member

trowski commented Nov 4, 2022

This LGTM now.

@ramsey Can this be merged into the 8.2 branch?

@ramsey
Copy link
Member

ramsey commented Nov 4, 2022

8.2 is at the fifth RC right now. Can you help me understand the reason this should be merged to 8.2 and any risks associated with it?

@trowski
Copy link
Member

trowski commented Nov 4, 2022

My understanding is that a machine with CET enabled will not be able to use fibers without this change. As these changes only affect machines with shadow stack support, there's not much risk to these changes.

I have no idea how common CET is at this time though, so waiting until 8.3 may not have a large impact.

@cmb69
Copy link
Member

cmb69 commented Nov 4, 2022

Well, we treated #8339 as bug fix (went into PHP 8.1.8RC1), so I think it should be okay to target PHP-8.2 here.

@chen-hu-97, what do you think about the target branch?

@chen-hu-97
Copy link
Contributor Author

Hi @cmb69,
I prefer 8.3. I think this is not a bug fix, but add shadow stack support for fiber.

#8339 is bug fix. Gcc already support CET for some time and php binary has corresponding GNU properties. For some reason, it loses such properites since some commit and #8339 fix this.

@ramsey
Copy link
Member

ramsey commented Nov 7, 2022

I'm convinced; this should target 8.3.

@trowski
Copy link
Member

trowski commented Nov 7, 2022

So shadow stacks are an enhancement to CET support, not a requirement. Great, then targeting 8.3 makes sense to me too.

@cmb69 cmb69 closed this in 37b84b7 Nov 7, 2022
@cmb69
Copy link
Member

cmb69 commented Nov 7, 2022

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants