Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add attach_xdp to reference_guide.md #3450

Merged
merged 3 commits into from
Jun 6, 2021

Conversation

masibw
Copy link
Contributor

@masibw masibw commented May 24, 2021

Hi all, this PR will resolve #3414.
I wrote the document about attach_xdp, but I am a beginner at eBPF and XDP.

I am assuming that multiple calls to attach_xdp will overwrite the previous one and not run, is this correct? Is this correct and should I write this?

Can you give me the advice to improve the quality of documents?

Best Regards.

@yonghong-song
Copy link
Collaborator

[buildbot, test this please]

@yonghong-song
Copy link
Collaborator

I am assuming that multiple calls to attach_xdp will overwrite the previous one and not run, is this correct? Is this correct and should I write this?

Based on the kernel code, I think this is not true. But if you have a XDP_FLAGS_REPLACE, you will be able to do that.

Can you give me the advice to improve the quality of documents?

xdp itself is a pretty big topic. We have https://github.com/xdp-project and https://docs.cilium.io/en/stable/bpf/ has much more information about xdp and some of them uses iproute2 though. For bcc's perspecitve, I think you current version is a good first step. If you can dig into a little bit more and explain default and available flags, that will be great. You can refer to the flag definitions in src/python/bcc/init.py.

    # from xdp_flags uapi/linux/if_link.h
    XDP_FLAGS_UPDATE_IF_NOEXIST = (1 << 0)
    XDP_FLAGS_SKB_MODE = (1 << 1)
    XDP_FLAGS_DRV_MODE = (1 << 2)
    XDP_FLAGS_HW_MODE = (1 << 3)
    XDP_FLAGS_REPLACE = (1 << 4)

@masibw
Copy link
Contributor Author

masibw commented May 28, 2021

Hi, @yonghong-song Thanks for replying.

But if you have a XDP_FLAGS_REPLACE, you will be able to do that.

I see. Thank you very much. But I can't find those flags definition in src/python/bcc/init.py.
And is there any description of those flags? Should I read kernel source code?

@chenyuezhou
Copy link
Contributor

Hi, @yonghong-song Thanks for replying.

But if you have a XDP_FLAGS_REPLACE, you will be able to do that.

I see. Thank you very much. But I can't find those flags definition in src/python/bcc/init.py.
And is there any description of those flags? Should I read kernel source code?

Hi @masibw This feature is already supported in #3447.

This is my personal understanding and I don't know if it helps you.

1. XDP_FLAGS_UPDATE_IF_NOEXIST

If an XDP program is already attached to the specified driver, attaching the XDP program again will fail.

2. XDP_FLAGS_SKB_MODE

Driver doesn’t have support for XDP, but the kernel fakes it. XDP program works, but there’s no real performance benefit because packets are handed to kernel stack anyways which then emulates XDP – this is usually supported with generic network drivers used in home computers, laptops, and virtualized HW.

3. XDP_FLAGS_DRV_MODE

Driver has XDP support and can hand then to XDP without kernel stack interaction – Few drivers can support it and those are usually for enterprise HW.

4. XDP_FLAGS_HW_MODE

XDP can be loaded and executed directly on the NIC – just a handful of NICs can do that.

5. XDP_FLAGS_REPLACE

Provides a mechanism to safely replace one particular XDP program with another.

@masibw
Copy link
Contributor Author

masibw commented May 28, 2021

Hi @chenyuezhou, Thanks for the comments.
That's an amazing description of xdp flags.

Would you mind if I use your description in this PR?
and could you tell me how you get to know about that?

@chenyuezhou
Copy link
Contributor

chenyuezhou commented May 28, 2021

Hi @chenyuezhou, Thanks for the comments.
That's an amazing description of xdp flags.

Would you mind if I use your description in this PR?
and could you tell me how you get to know about that?

Never mind. It's okay.

BTW. Maybe you can come up with a better description.

https://pantheon.tech/what-is-af_xdp/ This article summarizes three modes of XDP. (XDP_FLAGS_SKB_MODE, XDP_FLAGS_DRV_MODE, XDP_FLAGS_HW_MODE)

you can find the describe of XDP_FLAGS_REPLACE. here https://lwn.net/Articles/816067/

Hope this helps you.

@masibw
Copy link
Contributor Author

masibw commented May 30, 2021

Thank you @chenyuezhou I read both posts.
I think your descriptions are pretty good:heart:

Then I have a question about the bcc's behaviour.
I think the default flag value is 0.

def attach_xdp(dev, fn, flags=0):

But flags started from 1. What is the default behaviour?

Maybe this is the function that determines behaviour, but I can't understand...
https://elixir.bootlin.com/linux/latest/C/ident/__bpf_set_link_xdp_fd_replace

@chenyuezhou
Copy link
Contributor

chenyuezhou commented May 31, 2021

Hi @masibw

bcc is using https://github.com/libbpf/libbpf. This is a mirror of bpf-next Linux source tree's tools/lib/bpf. This may be a bit different from your current kernel provides, here suggest that take a look at this.

As you said, bcc does call __bpf_set_link_xdp_fd_replace. For the kernel, do_setlink is eventually called. You can look at this function. (You can also use the tools/trace.py provided by bcc to dynamically observe the function in more detail, which may help you understand the function better.)

In my current kernel version (5.10.35), the default value of flags is 0 if flags is not set in attach_xdp.

net/core/rtnetlink.c@do_setlink+2886

        struct nlattr *xdp[IFLA_XDP_MAX + 1];
        u32 xdp_flags = 0;
        ...
        if (xdp[IFLA_XDP_FLAGS]) {
            xdp_flags = nla_get_u32(xdp[IFLA_XDP_FLAGS]);
            ...
        }
        ...

There may be different behavior in different kernel versions.

@yonghong-song
Copy link
Collaborator

yonghong-song commented Jun 1, 2021

@masibw Previously I said:

Based on the kernel code, I think this is not true. But if you have a XDP_FLAGS_REPLACE, you will be able to do that.

This is not true actually. In early days, kernel internal xdp is not bpf_link based, so you actually can replace a bpf program without flags = 0. But this changed when xdp utilized bpf_link (https://lore.kernel.org/bpf/20200722064603.3350758-5-andriin@fb.com/, bpf command BPF_LINK_CREATE) in which case, you can only replace a bpf program with XDP_FLAGS_REPLACE. Currently, bcc does not use BPF_LINK_CREATE, instead it still uses old netlink based approach, so XDP_FLAGS_REPLACE will not apply.

BTW, the current patch has a merge conflict with master branch. Please rebase when you submit your new pull request.

@yonghong-song
Copy link
Collaborator

I think the default flag value is 0.
def attach_xdp(dev, fn, flags=0):
But flags started from 1. What is the default behaviour?

The default is to replace the old program if there is an old program.

@masibw
Copy link
Contributor Author

masibw commented Jun 2, 2021

Thanks, @yonghong-song, @chenyuezhou. I now understand the default behavior and the history of xdp.

The default is to replace the old program if there is an old program.

So..bcc is using an old netlink based approach, Is it able to use flags? I mean if there is no use writing XDP_FLAGS_REPLACE, Writing to document is not needed?

BTW, the current patch has a merge conflict with master branch. Please rebase when you submit your new pull request.

Ok, I'll do rebase.

@masibw masibw force-pushed the docs_attach_xdp branch from 91b9012 to 272ac7b Compare June 2, 2021 02:46
@yonghong-song
Copy link
Collaborator

So..bcc is using an old netlink based approach, Is it able to use flags? I mean if there is no use writing XDP_FLAGS_REPLACE, Writing to document is not needed?

The documentation can skip XDP_FLAGS_REPLACE for now. Thanks!

@masibw
Copy link
Contributor Author

masibw commented Jun 5, 2021

Sorry for my late response. I have described the flag, what do you think?

Copy link
Collaborator

@yonghong-song yonghong-song left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Overall looks good. Added a few comments.

@@ -1721,6 +1722,55 @@ BPF.attach_raw_socket(bpf_func, ifname)

Examples in situ:
[search /examples](https://github.com/iovisor/bcc/search?q=attach_raw_socket+path%3Aexamples+language%3Apython&type=Code)
### 9. attach_xdp()
Syntax: ```BPF.attach_xdp(dev="device", fn=b.load_func("fn_name",BPF_XDP))```
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flags are also supported. BPF.attach_xdp(dev="device", fn=b.load_func("fn_name",BPF_XDP), flags)

### 9. attach_xdp()
Syntax: ```BPF.attach_xdp(dev="device", fn=b.load_func("fn_name",BPF_XDP))```

Instruments the network driver described by ```dev``` , and then receives the packet, run the BPF function ```fn_name()```.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run the BPF function fn_name() with flags.


You can use flags like this ```BPF.attach_xdp(dev="device", fn=b.load_func("fn_name",BPF_XDP), flags=BPF.XDP_FLAGS_UPDATE_IF_NOEXIST)```

For historical reasons, XDP_FLAGS_REPLACE does not make sense in BCC. The default value of 0 has the same meaning.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flag 0 means if there is no xdp program with device, the fn will run with that device. If there is an xdp program running with device, the old program will be replaced with new fn program.

Currently, bcc does not support XDP_FLAGS_REPLACE flag. The following are the descriptions of other flags.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, then how about this?

Suggested change
For historical reasons, XDP_FLAGS_REPLACE does not make sense in BCC. The default value of 0 has the same meaning.
The default value of flgas is 0. This means if there is no xdp program with `device`, the fn will run with that device. If there is an xdp program running with device, the old program will be replaced with new fn program.
Currently, bcc does not support XDP_FLAGS_REPLACE flag. The following are the descriptions of other flags.

#### 4. XDP_FLAGS_HW_MODE
XDP can be loaded and executed directly on the NIC – just a handful of NICs can do that.

#### 5. XDP_FLAGS_REPLACE
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this. It can only cause confusion. We can add it later if xdp load part is enhanced with BPF_LINK_CREATE syscall command.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All right, thank you.

@yonghong-song
Copy link
Collaborator

LGTM. Thanks!

@yonghong-song
Copy link
Collaborator

[buildbot, test this please]

@yonghong-song yonghong-song merged commit 5b9a5c6 into iovisor:master Jun 6, 2021
CrackerCat pushed a commit to CrackerCat/bcc that referenced this pull request Jul 31, 2024
- Add attach_xdp to reference_guide.md
- Add description about flags
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Description of attach_xdp
3 participants