-
Notifications
You must be signed in to change notification settings - Fork 110
OCPBUGS-18662:rps: trigger udev even per queue #816
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
SUBSYSTEM=="net", ACTION=="add", ENV{DEVPATH}!="/devices/virtual/net/*", TAG+="systemd", ENV{SYSTEMD_WANTS}="update-rps@%k.service" | ||
SUBSYSTEM=="queues", ACTION=="add", ENV{DEVPATH}=="/devices/pci*/queues/rx*", TAG+="systemd", ENV{SYSTEMD_WANTS}="update-rps@%p.service" | ||
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,38 +1,16 @@ | ||
#!/usr/bin/env bash | ||
|
||
dev=$1 | ||
[ -n "${dev}" ] || { echo "The device argument is missing" >&2 ; exit 1; } | ||
path=${1} | ||
[ -n "${path}" ] || { echo "The device path argument is missing" >&2 ; exit 1; } | ||
|
||
mask=$2 | ||
mask=${2} | ||
[ -n "${mask}" ] || { echo "The mask argument is missing" >&2 ; exit 1; } | ||
|
||
dev_dir="/sys/class/net/${dev}" | ||
queue_path=${path%rx*} | ||
|
||
function find_dev_dir { | ||
systemd_devs=$(systemctl list-units -t device | grep sys-subsystem-net-devices | cut -d' ' -f1) | ||
queue_num=${path#*queues/} | ||
# replace '/' with '-' | ||
queue_num="${queue_num/\//-}" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. minor nit: is this substitution required? If I read correctly $path is not escaped anymore and this is basically de-escaping the queue nr part Another minor note: you could possibly mention in the commit message that this change additionally avoid looping on all the rx queue, just for better future memory ;) both nits are not intended to block the MR, just to be considered if a new version should be needed for any reason There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
when the |
||
|
||
for systemd_dev in ${systemd_devs}; do | ||
dev_sysfs=$(systemctl show "${systemd_dev}" -p SysFSPath --value) | ||
|
||
dev_orig_name="${dev_sysfs##*/}" | ||
if [ "${dev_orig_name}" = "${dev}" ]; then | ||
dev_name="${systemd_dev##*-}" | ||
dev_name="${dev_name%%.device}" | ||
if [ "${dev_name}" = "${dev}" ]; then # disregard the original device unit | ||
continue | ||
fi | ||
|
||
echo "${dev} device was renamed to $dev_name" | ||
dev_dir="/sys/class/net/${dev_name}" | ||
break | ||
fi | ||
done | ||
} | ||
|
||
[ -d "${dev_dir}" ] || find_dev_dir # the net device was renamed, find the new name | ||
[ -d "${dev_dir}" ] || { sleep 5; find_dev_dir; } # search failed, wait a little and try again | ||
[ -d "${dev_dir}" ] || { echo "${dev_dir}" directory not found >&2 ; exit 0; } # the interface disappeared, not an error | ||
|
||
for i in "${dev_dir}"/queues/rx-*/rps_cpus; do | ||
echo "${mask}" > "${i}" | ||
done | ||
# set rps affinity for the queue | ||
echo "${mask}" > "/sys${queue_path}${queue_num}/rps_cpus" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
apiVersion: machineconfiguration.openshift.io/v1 | ||
kind: KubeletConfig | ||
metadata: | ||
creationTimestamp: null | ||
name: performance-manual | ||
ownerReferences: | ||
- apiVersion: performance.openshift.io/v2 | ||
kind: PerformanceProfile | ||
name: manual | ||
uid: "" | ||
spec: | ||
kubeletConfig: | ||
apiVersion: kubelet.config.k8s.io/v1beta1 | ||
authentication: | ||
anonymous: {} | ||
webhook: | ||
cacheTTL: 0s | ||
x509: {} | ||
authorization: | ||
webhook: | ||
cacheAuthorizedTTL: 0s | ||
cacheUnauthorizedTTL: 0s | ||
containerRuntimeEndpoint: "" | ||
cpuManagerPolicy: static | ||
cpuManagerPolicyOptions: | ||
full-pcpus-only: "true" | ||
cpuManagerReconcilePeriod: 5s | ||
evictionHard: | ||
imagefs.available: 15% | ||
memory.available: 100Mi | ||
nodefs.available: 10% | ||
nodefs.inodesFree: 5% | ||
evictionPressureTransitionPeriod: 0s | ||
fileCheckFrequency: 0s | ||
httpCheckFrequency: 0s | ||
imageMinimumGCAge: 0s | ||
kind: KubeletConfiguration | ||
kubeReserved: | ||
memory: 500Mi | ||
logging: | ||
flushFrequency: 0 | ||
options: | ||
json: | ||
infoBufferSize: "0" | ||
verbosity: 0 | ||
memoryManagerPolicy: Static | ||
memorySwap: {} | ||
nodeStatusReportFrequency: 0s | ||
nodeStatusUpdateFrequency: 0s | ||
reservedMemory: | ||
- limits: | ||
memory: 1100Mi | ||
numaNode: 0 | ||
reservedSystemCPUs: "0" | ||
runtimeRequestTimeout: 0s | ||
shutdownGracePeriod: 0s | ||
shutdownGracePeriodCriticalPods: 0s | ||
streamingConnectionIdleTimeout: 0s | ||
syncFrequency: 0s | ||
systemReserved: | ||
memory: 500Mi | ||
topologyManagerPolicy: single-numa-node | ||
volumeStatsAggPeriod: 0s | ||
machineConfigPoolSelector: | ||
matchLabels: | ||
machineconfiguration.openshift.io/role: worker-cnf | ||
status: | ||
conditions: null |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here the devpath for patching events is almost full sysfs queue directory - just missing the '/sys' prefix...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AFAIK the path should start from
/devices
and not from/sys
this is how it worked so far.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Tal-or: I was not clear in the sentence above, I'm sorry. What I mean is that since the devpath for patching events is almost full sysfs queue directory, we can exploit such fact to simplify the action, as hopefully better described in my comment on set-rps-mask.sh.
The above sentence and the comment there are supposed to be read one after another as part of a the same message