Skip to content

Linkstate pass-through (PIF -> VIF) #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
6 commits merged into from
Feb 8, 2011
Merged

Linkstate pass-through (PIF -> VIF) #10

6 commits merged into from
Feb 8, 2011

Conversation

@djs55
Copy link
Contributor

@djs55 djs55 commented Feb 4, 2011

No description provided.

@robhoes
Copy link
Member

robhoes commented Feb 7, 2011

Other than my comment on the GRE tunnel ports, I think the patches look good.

David Scott added 6 commits February 8, 2011 12:05
CP-2259: Add xenops function "set_carrier" to set the VIF frontend di…
bf78c94 
…sconnect flag.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Reviewed-by: Matthias Goergens <matthias.goergens@gmail.com>
Reviewed-by: Rob Hoes <rob.hoes@citrix.com>
CP-2259: set the initial value of "carrier" when the device is create…
ef830d3 
…d, in addition to changing it dynamically.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Reviewed-by: Matthias Goergens <matthias.goergens@gmail.com>
Reviewed-by: Rob Hoes <rob.hoes@citrix.com>
CP-2260: Whenever a physical device carrier status changes, find all …
88caaa4 
…the VIFs on the same bridge and any "tunnel" bridges and signal the change to them.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Reviewed-by: Matthias Goergens <matthias.goergens@gmail.com>
Reviewed-by: Rob Hoes <rob.hoes@citrix.com>
CA-50488: When adding a VIF to a bridge, if there are any non-virtual…
85b5a29 
… interfaces reflect the carrier. If there are only virtual interfaces (a virtual network) then we always consider a carrier is present.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Reviewed-by: Matthias Goergens <matthias.goergens@gmail.com>
Reviewed-by: Rob Hoes <rob.hoes@citrix.com>
CP-2261: Only pass through the PIF carrier to VIFs if Pool.other_conf…
484e630 
…ig:pass_through_pif_carrier=true for now.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Reviewed-by: Matthias Goergens <matthias.goergens@gmail.com>
Reviewed-by: Rob Hoes <rob.hoes@citrix.com>
CA-50488: When a VIF is first created, pass through the PIF carrier i…
8d5a0f5 
…f Pool.other_config:pass_through_pif_carrier=true.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Reviewed-by: Matthias Goergens <matthias.goergens@gmail.com>
Reviewed-by: Rob Hoes <rob.hoes@citrix.com>
mcclurmc pushed a commit to mcclurmc/xen-api that referenced this pull request Dec 12, 2011
@jonludlam
Merge pull request xapi-project#10 from mcclurmc/debian-fix-lintian-a…
9e7013f 
…nd-xe-package

Fix lintian warnings and add xcp-xe package
sharady pushed a commit to sharady/xen-api that referenced this pull request Mar 18, 2015
@robhoes
Merge pull request xapi-project#10 from johnelse/snapshot-revert-more…
4f38b76 
…-fixes-creedence-bugfix

More snapshot-revert fixes
sharady pushed a commit to sharady/xen-api that referenced this pull request May 11, 2015
Merge pull request xapi-project#10 from jonludlam/update-storage-inte…
7cddbe7 
…rface

Pull changes to storage layer from xen-api.git
sharady pushed a commit to sharady/xen-api that referenced this pull request May 11, 2015
@djs55
Merge pull request xapi-project#10 from djs55/domain-zero-policy
13a7d94 
memory: allow domain zero memory policy to be queried
koushikcgit pushed a commit to koushikcgit/xen-api that referenced this pull request Aug 7, 2015
@djs55
Merge pull request xapi-project#10 from robhoes/fix-vcpus
b75eb5d 
Set avail_vcpus in domain build info
lippirk pushed a commit to lippirk/xen-api that referenced this pull request Jul 20, 2020
@lindig
Merge pull request xapi-project#10 from kc284/master
2138d38 
Moved from jbuilder to dune and deprecated xcp in favour of xapi-idl.
kc284 pushed a commit to kc284/xen-api that referenced this pull request Mar 11, 2021
Merge pull request xapi-project#10 from xueyuan/master
4319e15 
Client side connection timeout and a check for if an object is null.
robhoes pushed a commit to robhoes/xen-api that referenced this pull request Jul 6, 2021
@djs55
Merge pull request xapi-project#10 from djs55/expose-open-connection
5aba4f1 
Expose the open_connection: Uri.t -> channels function
robhoes pushed a commit that referenced this pull request Sep 14, 2021
@johnelse
Merge pull request #10 from johnelse/remove-obuild-file
f584cce 
Remove obuild file
robhoes pushed a commit that referenced this pull request Sep 14, 2021
@johnelse
Merge pull request #10 from johnelse/release-0.6.1
8d7fa4d 
Release 0.6.1
psafont pushed a commit to psafont/xen-api that referenced this pull request Sep 20, 2021
Merge pull request xapi-project#10 from djs55/fix-arm
fc2c63b 
arm: CPUID is x86-specific
robhoes pushed a commit to robhoes/xen-api that referenced this pull request Sep 21, 2021
@djs55
Merge pull request xapi-project#10 from djs55/openchain
5e8cf34 
Adapt to new ocaml-vhd API
robhoes pushed a commit to robhoes/xen-api that referenced this pull request Sep 21, 2021
Merge pull request xapi-project#10 from jonludlam/great_stringext_rename
c8d31e8 
Great stringext rename
robhoes pushed a commit to robhoes/xen-api that referenced this pull request Sep 21, 2021
@mseri
Merge pull request xapi-project#10 from gaborigloi/sync_opam
a164d04 
Sync opam file with xs-opam
psafont pushed a commit to psafont/xen-api that referenced this pull request Oct 19, 2021
@robhoes
Merge pull request xapi-project#10 from robhoes/master
135f4a1 
Batch IP monitor changes for 1 second
robhoes pushed a commit to robhoes/xen-api that referenced this pull request Jan 10, 2022
@mseri
Merge pull request xapi-project#10 from mseri/master
eb7e6cb 
CP-24262: wsproxy cli code refactoring
robhoes pushed a commit to robhoes/xen-api that referenced this pull request Jan 10, 2022
Merge pull request xapi-project#10 from jonludlam/xcp-update-fix
aa1a3f1 
Update to newest storage interface
edwintorok pushed a commit to edwintorok/xen-api that referenced this pull request Jan 10, 2022
@lindig
Merge pull request xapi-project#10 from xapi-project/private/edvint/ci
0510b41 
Use github actions based on template
robhoes pushed a commit to robhoes/xen-api that referenced this pull request Apr 7, 2022
@johnelse
Merge pull request xapi-project#10 from johnelse/new-rrd-transport-in…
35acabb 
…terface

Update to new rrd-transport interface
psafont pushed a commit to psafont/xen-api that referenced this pull request Mar 17, 2023
@lindig
Merge pull request xapi-project#10 from lindig/master
a9adf69 
Update opam file
edwintorok added a commit to edwintorok/xen-api that referenced this pull request Oct 10, 2023
@edwintorok
Workaround: avoid sleep(1) call by crypt_r
c916e7d 
Using an off-cpu flamegraph I identified that concurrent PAM calls are slow due to a call to `sleep(1)`.
`pam_authenticate` calls `crypt_r` which calls `NSSLOW_Init` which on first use will try to initialize the just `dlopen`-ed library.
If it encounters a race condition it does a `sleep(1)`. This race condition can be quite reliably reproduced when performing a lot of PAM authentications from multiple threads in parallel.

GDB can also be used to confirm this by putting a breakpoint on `sleep`:
```
 #0  __sleep (seconds=seconds@entry=1) at ../sysdeps/unix/sysv/linux/sleep.c:42
 #1  0x00007ffff1548e22 in freebl_RunLoaderOnce () at lowhash_vector.c:122
 #2  0x00007ffff1548f31 in freebl_InitVector () at lowhash_vector.c:131
 #3  NSSLOW_Init () at lowhash_vector.c:148
 #4  0x00007ffff1b8f09a in __sha512_crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=0x7ffff31e17b8 "dIJbsXKc0",
 xapi-project#5  0x00007ffff1b8d070 in __crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=<optimized out>,
 xapi-project#6  0x00007ffff1dc9abc in verify_pwd_hash (p=p@entry=0x7fffd8005a60 "pamtest-edvint", hash=<optimized out>, nullok=nullok@entry=0) at passverify.c:111
 xapi-project#7  0x00007ffff1dc9139 in _unix_verify_password (pamh=pamh@entry=0x7fffd8002910, name=0x7fffd8002ab0 "pamtest-edvint", p=0x7fffd8005a60 "pamtest-edvint", ctrl=ctrl@entry=8389156) at support.c:777
 xapi-project#8  0x00007ffff1dc6556 in pam_sm_authenticate (pamh=0x7fffd8002910, flags=<optimized out>, argc=<optimized out>, argv=<optimized out>) at pam_unix_auth.c:178
 xapi-project#9  0x00007ffff7bcef1a in _pam_dispatch_aux (use_cached_chain=<optimized out>, resumed=<optimized out>, h=<optimized out>, flags=1, pamh=0x7fffd8002910) at pam_dispatch.c:110
 xapi-project#10 _pam_dispatch (pamh=pamh@entry=0x7fffd8002910, flags=1, choice=choice@entry=1) at pam_dispatch.c:426
 xapi-project#11 0x00007ffff7bce7e0 in pam_authenticate (pamh=0x7fffd8002910, flags=flags@entry=1) at pam_auth.c:34
 xapi-project#12 0x00000000005ae567 in XA_mh_authorize (username=username@entry=0x7fffd80028d0 "pamtest-edvint", password=password@entry=0x7fffd80028f0 "pamtest-edvint", error=error@entry=0x7ffff31e1be8) at xa_auth.c:83
 xapi-project#13 0x00000000005adf20 in stub_XA_mh_authorize (username=<optimized out>, password=<optimized out>) at xa_auth_stubs.c:42
 xapi-project#14 0x00000000004a0a6a in camlDune__exe__Bench_pam__pam_authenticate$27_320 () at ocaml/tests/bench/pam/bench_pam.ml:63
 xapi-project#15 0x00000000004a1113 in camlEzbechamel_concurrent__worker_loop_359 () at ocaml/tests/bench/lib/concurrent/ezbechamel_concurrent.ml:36
 xapi-project#16 0x00000000005935b9 in camlStdlib__Fun__protect_317 ()
 xapi-project#17 0x00000000004a1955 in camlThread__fun_850 ()
 xapi-project#18 0x00000000005d6401 in caml_start_program ()
 xapi-project#19 0x00000000005cd0fd in caml_callback_exn ()
 xapi-project#20 0x00000000005af810 in caml_thread_start ()
 xapi-project#21 0x00007ffff79b7e25 in start_thread (arg=0x7ffff31e2700) at pthread_create.c:308
 xapi-project#22 0x00007ffff71dbbad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
```

`pam_start` and `pam_end` doesn't help here, because on `pam_end` the library is `dlclose`-ed, so on next `pam_authenticate` it will have to go through the initialization code again.
(This initialization code would've belonged into `pam_start`, not `pam_authenticate`, but there are several layers here including a call to `crypt_r`).

To avoid this link with `libcrypt` and call `crypt_r` once ourselves (and ensure it loads `libfreeblpriv3` by using the sha512 prefix).
That way the library will stay loaded (we'll hold a reference count on it), and the `dlclose` done by PAM won't unload it.

Confirmed that there are no `sleep` calls now, and the results are also visible when running the benchmark targeted to the with and without fix code:
```
╭─────────────────────────────────────────────────┬───────────────────────────┬───────────────────────────┬───────────────────────────╮
│name                                             │  major-allocated          │  minor-allocated          │  monotonic-clock          │
├─────────────────────────────────────────────────┼───────────────────────────┼───────────────────────────┼───────────────────────────┤
│  concurrent authenticate (sleep fix, actual):8  │             0.0000 mjw/run│            50.0000 mnw/run│       27043467.0000 ns/run│
╰─────────────────────────────────────────────────┴───────────────────────────┴───────────────────────────┴───────────────────────────╯

╭────────────────────────────────────────┬───────────────────────────┬───────────────────────────┬───────────────────────────╮
│name                                    │  major-allocated          │  minor-allocated          │  monotonic-clock          │
├────────────────────────────────────────┼───────────────────────────┼───────────────────────────┼───────────────────────────┤
│  concurrent authenticate (no reuse):8  │             0.0000 mjw/run│            50.0000 mnw/run│     1029831372.0000 ns/run│
╰────────────────────────────────────────┴───────────────────────────┴───────────────────────────┴───────────────────────────╯
```

Without this fix using 2 threads to perform PAM authentication would result in a 38x slowdown compared to using no threads at all (which is what XAPI currently does).

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
edwintorok added a commit to edwintorok/xen-api that referenced this pull request Oct 11, 2023
@edwintorok
Pam: avoid sleep(1) call when multithreaded
d0d690c 
This enables PAM to be used in multithreaded mode (currently XAPI has a global lock around auth).

Using an off-cpu flamegraph I identified that concurrent PAM calls are slow due to a call to `sleep(1)`.
`pam_authenticate` calls `crypt_r` which calls `NSSLOW_Init` which on first use will try to initialize the just `dlopen`-ed library.
If it encounters a race condition it does a `sleep(1)`. This race condition can be quite reliably reproduced when performing a lot of PAM authentications from multiple threads in parallel.

GDB can also be used to confirm this by putting a breakpoint on `sleep`:
```
  #0  __sleep (seconds=seconds@entry=1) at ../sysdeps/unix/sysv/linux/sleep.c:42
  #1  0x00007ffff1548e22 in freebl_RunLoaderOnce () at lowhash_vector.c:122
  #2  0x00007ffff1548f31 in freebl_InitVector () at lowhash_vector.c:131
  #3  NSSLOW_Init () at lowhash_vector.c:148
  #4  0x00007ffff1b8f09a in __sha512_crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=0x7ffff31e17b8 "dIJbsXKc0",
  xapi-project#5  0x00007ffff1b8d070 in __crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=<optimized out>,
  xapi-project#6  0x00007ffff1dc9abc in verify_pwd_hash (p=p@entry=0x7fffd8005a60 "pamtest-edvint", hash=<optimized out>, nullok=nullok@entry=0) at passverify.c:111
  xapi-project#7  0x00007ffff1dc9139 in _unix_verify_password (pamh=pamh@entry=0x7fffd8002910, name=0x7fffd8002ab0 "pamtest-edvint", p=0x7fffd8005a60 "pamtest-edvint", ctrl=ctrl@entry=8389156) at support.c:777
  xapi-project#8  0x00007ffff1dc6556 in pam_sm_authenticate (pamh=0x7fffd8002910, flags=<optimized out>, argc=<optimized out>, argv=<optimized out>) at pam_unix_auth.c:178
  xapi-project#9  0x00007ffff7bcef1a in _pam_dispatch_aux (use_cached_chain=<optimized out>, resumed=<optimized out>, h=<optimized out>, flags=1, pamh=0x7fffd8002910) at pam_dispatch.c:110
  xapi-project#10 _pam_dispatch (pamh=pamh@entry=0x7fffd8002910, flags=1, choice=choice@entry=1) at pam_dispatch.c:426
  xapi-project#11 0x00007ffff7bce7e0 in pam_authenticate (pamh=0x7fffd8002910, flags=flags@entry=1) at pam_auth.c:34
  xapi-project#12 0x00000000005ae567 in XA_mh_authorize (username=username@entry=0x7fffd80028d0 "pamtest-edvint", password=password@entry=0x7fffd80028f0 "pamtest-edvint", error=error@entry=0x7ffff31e1be8) at xa_auth.c:83
  xapi-project#13 0x00000000005adf20 in stub_XA_mh_authorize (username=<optimized out>, password=<optimized out>) at xa_auth_stubs.c:42
```

`pam_start` and `pam_end` doesn't help here, because on `pam_end` the library is `dlclose`-ed, so on next `pam_authenticate` it will have to go through the initialization code again.
(This initialization code would've belonged into `pam_start`, not `pam_authenticate`, but there are several layers here including a call to `crypt_r`).
Upstream has fixed this problem >5 years ago by switching to libxcrypt instead.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
edwintorok added a commit to edwintorok/xen-api that referenced this pull request Oct 11, 2023
@edwintorok
CP-43755: Pam: avoid sleep(1) call when multithreaded
ba49ad0 
This enables PAM to be used in multithreaded mode (currently XAPI has a global lock around auth).

Using an off-cpu flamegraph I identified that concurrent PAM calls are slow due to a call to `sleep(1)`.
`pam_authenticate` calls `crypt_r` which calls `NSSLOW_Init` which on first use will try to initialize the just `dlopen`-ed library.
If it encounters a race condition it does a `sleep(1)`. This race condition can be quite reliably reproduced when performing a lot of PAM authentications from multiple threads in parallel.

GDB can also be used to confirm this by putting a breakpoint on `sleep`:
```
  #0  __sleep (seconds=seconds@entry=1) at ../sysdeps/unix/sysv/linux/sleep.c:42
  #1  0x00007ffff1548e22 in freebl_RunLoaderOnce () at lowhash_vector.c:122
  #2  0x00007ffff1548f31 in freebl_InitVector () at lowhash_vector.c:131
  #3  NSSLOW_Init () at lowhash_vector.c:148
  #4  0x00007ffff1b8f09a in __sha512_crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=0x7ffff31e17b8 "dIJbsXKc0",
  xapi-project#5  0x00007ffff1b8d070 in __crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=<optimized out>,
  xapi-project#6  0x00007ffff1dc9abc in verify_pwd_hash (p=p@entry=0x7fffd8005a60 "pamtest-edvint", hash=<optimized out>, nullok=nullok@entry=0) at passverify.c:111
  xapi-project#7  0x00007ffff1dc9139 in _unix_verify_password (pamh=pamh@entry=0x7fffd8002910, name=0x7fffd8002ab0 "pamtest-edvint", p=0x7fffd8005a60 "pamtest-edvint", ctrl=ctrl@entry=8389156) at support.c:777
  xapi-project#8  0x00007ffff1dc6556 in pam_sm_authenticate (pamh=0x7fffd8002910, flags=<optimized out>, argc=<optimized out>, argv=<optimized out>) at pam_unix_auth.c:178
  xapi-project#9  0x00007ffff7bcef1a in _pam_dispatch_aux (use_cached_chain=<optimized out>, resumed=<optimized out>, h=<optimized out>, flags=1, pamh=0x7fffd8002910) at pam_dispatch.c:110
  xapi-project#10 _pam_dispatch (pamh=pamh@entry=0x7fffd8002910, flags=1, choice=choice@entry=1) at pam_dispatch.c:426
  xapi-project#11 0x00007ffff7bce7e0 in pam_authenticate (pamh=0x7fffd8002910, flags=flags@entry=1) at pam_auth.c:34
  xapi-project#12 0x00000000005ae567 in XA_mh_authorize (username=username@entry=0x7fffd80028d0 "pamtest-edvint", password=password@entry=0x7fffd80028f0 "pamtest-edvint", error=error@entry=0x7ffff31e1be8) at xa_auth.c:83
  xapi-project#13 0x00000000005adf20 in stub_XA_mh_authorize (username=<optimized out>, password=<optimized out>) at xa_auth_stubs.c:42
```

`pam_start` and `pam_end` doesn't help here, because on `pam_end` the library is `dlclose`-ed, so on next `pam_authenticate` it will have to go through the initialization code again.
(This initialization code would've belonged into `pam_start`, not `pam_authenticate`, but there are several layers here including a call to `crypt_r`).
Upstream has fixed this problem >5 years ago by switching to libxcrypt instead.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
edwintorok added a commit to edwintorok/xen-api that referenced this pull request Oct 12, 2023
@edwintorok
CP-43755: Pam: avoid sleep(1) call when multithreaded
3b52b72 
This enables PAM to be used in multithreaded mode (currently XAPI has a global lock around auth).

Using an off-cpu flamegraph I identified that concurrent PAM calls are slow due to a call to `sleep(1)`.
`pam_authenticate` calls `crypt_r` which calls `NSSLOW_Init` which on first use will try to initialize the just `dlopen`-ed library.
If it encounters a race condition it does a `sleep(1)`. This race condition can be quite reliably reproduced when performing a lot of PAM authentications from multiple threads in parallel.

GDB can also be used to confirm this by putting a breakpoint on `sleep`:
```
  #0  __sleep (seconds=seconds@entry=1) at ../sysdeps/unix/sysv/linux/sleep.c:42
  #1  0x00007ffff1548e22 in freebl_RunLoaderOnce () at lowhash_vector.c:122
  #2  0x00007ffff1548f31 in freebl_InitVector () at lowhash_vector.c:131
  #3  NSSLOW_Init () at lowhash_vector.c:148
  #4  0x00007ffff1b8f09a in __sha512_crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=0x7ffff31e17b8 "dIJbsXKc0",
  xapi-project#5  0x00007ffff1b8d070 in __crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=<optimized out>,
  xapi-project#6  0x00007ffff1dc9abc in verify_pwd_hash (p=p@entry=0x7fffd8005a60 "pamtest-edvint", hash=<optimized out>, nullok=nullok@entry=0) at passverify.c:111
  xapi-project#7  0x00007ffff1dc9139 in _unix_verify_password (pamh=pamh@entry=0x7fffd8002910, name=0x7fffd8002ab0 "pamtest-edvint", p=0x7fffd8005a60 "pamtest-edvint", ctrl=ctrl@entry=8389156) at support.c:777
  xapi-project#8  0x00007ffff1dc6556 in pam_sm_authenticate (pamh=0x7fffd8002910, flags=<optimized out>, argc=<optimized out>, argv=<optimized out>) at pam_unix_auth.c:178
  xapi-project#9  0x00007ffff7bcef1a in _pam_dispatch_aux (use_cached_chain=<optimized out>, resumed=<optimized out>, h=<optimized out>, flags=1, pamh=0x7fffd8002910) at pam_dispatch.c:110
  xapi-project#10 _pam_dispatch (pamh=pamh@entry=0x7fffd8002910, flags=1, choice=choice@entry=1) at pam_dispatch.c:426
  xapi-project#11 0x00007ffff7bce7e0 in pam_authenticate (pamh=0x7fffd8002910, flags=flags@entry=1) at pam_auth.c:34
  xapi-project#12 0x00000000005ae567 in XA_mh_authorize (username=username@entry=0x7fffd80028d0 "pamtest-edvint", password=password@entry=0x7fffd80028f0 "pamtest-edvint", error=error@entry=0x7ffff31e1be8) at xa_auth.c:83
  xapi-project#13 0x00000000005adf20 in stub_XA_mh_authorize (username=<optimized out>, password=<optimized out>) at xa_auth_stubs.c:42
```

`pam_start` and `pam_end` doesn't help here, because on `pam_end` the library is `dlclose`-ed, so on next `pam_authenticate` it will have to go through the initialization code again.
(This initialization code would've belonged into `pam_start`, not `pam_authenticate`, but there are several layers here including a call to `crypt_r`).
Upstream has fixed this problem >5 years ago by switching to libxcrypt instead.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
edwintorok pushed a commit to edwintorok/xen-api that referenced this pull request Jan 22, 2024
@djs55
Merge pull request xapi-project#10 from djs55/release.0.13.0
d3ffd51 
Release.0.13.0
edwintorok pushed a commit to edwintorok/xen-api that referenced this pull request Jan 22, 2024
@djs55
Merge pull request xapi-project#10 from djs55/improve-api
ecbe67e 
Make the API easier to use
edwintorok pushed a commit to edwintorok/xen-api that referenced this pull request Jan 22, 2024
@mseri
Merge pull request xapi-project#10 from lindig/master
aa6f8dc 
Bump version to 1.0.2
lindig pushed a commit to lindig/xen-api that referenced this pull request Feb 28, 2024
@edwintorok
CP-43755: Pam: avoid sleep(1) call when multithreaded
920f9fe 
Backport of 3b52b72

This enables PAM to be used in multithreaded mode (currently XAPI has a global lock around auth).

Using an off-cpu flamegraph I identified that concurrent PAM calls are slow due to a call to `sleep(1)`.
`pam_authenticate` calls `crypt_r` which calls `NSSLOW_Init` which on first use will try to initialize the just `dlopen`-ed library.
If it encounters a race condition it does a `sleep(1)`. This race condition can be quite reliably reproduced when performing a lot of PAM authentications from multiple threads in parallel.

GDB can also be used to confirm this by putting a breakpoint on `sleep`:
```
  #0  __sleep (seconds=seconds@entry=1) at ../sysdeps/unix/sysv/linux/sleep.c:42
  xapi-project#1  0x00007ffff1548e22 in freebl_RunLoaderOnce () at lowhash_vector.c:122
  xapi-project#2  0x00007ffff1548f31 in freebl_InitVector () at lowhash_vector.c:131
  xapi-project#3  NSSLOW_Init () at lowhash_vector.c:148
  xapi-project#4  0x00007ffff1b8f09a in __sha512_crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=0x7ffff31e17b8 "dIJbsXKc0",
  xapi-project#5  0x00007ffff1b8d070 in __crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=<optimized out>,
  xapi-project#6  0x00007ffff1dc9abc in verify_pwd_hash (p=p@entry=0x7fffd8005a60 "pamtest-edvint", hash=<optimized out>, nullok=nullok@entry=0) at passverify.c:111
  xapi-project#7  0x00007ffff1dc9139 in _unix_verify_password (pamh=pamh@entry=0x7fffd8002910, name=0x7fffd8002ab0 "pamtest-edvint", p=0x7fffd8005a60 "pamtest-edvint", ctrl=ctrl@entry=8389156) at support.c:777
  xapi-project#8  0x00007ffff1dc6556 in pam_sm_authenticate (pamh=0x7fffd8002910, flags=<optimized out>, argc=<optimized out>, argv=<optimized out>) at pam_unix_auth.c:178
  xapi-project#9  0x00007ffff7bcef1a in _pam_dispatch_aux (use_cached_chain=<optimized out>, resumed=<optimized out>, h=<optimized out>, flags=1, pamh=0x7fffd8002910) at pam_dispatch.c:110
  xapi-project#10 _pam_dispatch (pamh=pamh@entry=0x7fffd8002910, flags=1, choice=choice@entry=1) at pam_dispatch.c:426
  xapi-project#11 0x00007ffff7bce7e0 in pam_authenticate (pamh=0x7fffd8002910, flags=flags@entry=1) at pam_auth.c:34
  xapi-project#12 0x00000000005ae567 in XA_mh_authorize (username=username@entry=0x7fffd80028d0 "pamtest-edvint", password=password@entry=0x7fffd80028f0 "pamtest-edvint", error=error@entry=0x7ffff31e1be8) at xa_auth.c:83
  xapi-project#13 0x00000000005adf20 in stub_XA_mh_authorize (username=<optimized out>, password=<optimized out>) at xa_auth_stubs.c:42
```

`pam_start` and `pam_end` doesn't help here, because on `pam_end` the library is `dlclose`-ed, so on next `pam_authenticate` it will have to go through the initialization code again.
(This initialization code would've belonged into `pam_start`, not `pam_authenticate`, but there are several layers here including a call to `crypt_r`).
Upstream has fixed this problem >5 years ago by switching to libxcrypt instead.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
lindig pushed a commit that referenced this pull request Feb 29, 2024
@edwintorok @lindig
CP-43755: Pam: avoid sleep(1) call when multithreaded
8f355ea 
Backport of 3b52b72

This enables PAM to be used in multithreaded mode (currently XAPI has a global lock around auth).

Using an off-cpu flamegraph I identified that concurrent PAM calls are slow due to a call to `sleep(1)`.
`pam_authenticate` calls `crypt_r` which calls `NSSLOW_Init` which on first use will try to initialize the just `dlopen`-ed library.
If it encounters a race condition it does a `sleep(1)`. This race condition can be quite reliably reproduced when performing a lot of PAM authentications from multiple threads in parallel.

GDB can also be used to confirm this by putting a breakpoint on `sleep`:
```
  #0  __sleep (seconds=seconds@entry=1) at ../sysdeps/unix/sysv/linux/sleep.c:42
  #1  0x00007ffff1548e22 in freebl_RunLoaderOnce () at lowhash_vector.c:122
  #2  0x00007ffff1548f31 in freebl_InitVector () at lowhash_vector.c:131
  #3  NSSLOW_Init () at lowhash_vector.c:148
  #4  0x00007ffff1b8f09a in __sha512_crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=0x7ffff31e17b8 "dIJbsXKc0",
  #5  0x00007ffff1b8d070 in __crypt_r (key=key@entry=0x7fffd8005a60 "pamtest-edvint", salt=<optimized out>,
  #6  0x00007ffff1dc9abc in verify_pwd_hash (p=p@entry=0x7fffd8005a60 "pamtest-edvint", hash=<optimized out>, nullok=nullok@entry=0) at passverify.c:111
  #7  0x00007ffff1dc9139 in _unix_verify_password (pamh=pamh@entry=0x7fffd8002910, name=0x7fffd8002ab0 "pamtest-edvint", p=0x7fffd8005a60 "pamtest-edvint", ctrl=ctrl@entry=8389156) at support.c:777
  #8  0x00007ffff1dc6556 in pam_sm_authenticate (pamh=0x7fffd8002910, flags=<optimized out>, argc=<optimized out>, argv=<optimized out>) at pam_unix_auth.c:178
  #9  0x00007ffff7bcef1a in _pam_dispatch_aux (use_cached_chain=<optimized out>, resumed=<optimized out>, h=<optimized out>, flags=1, pamh=0x7fffd8002910) at pam_dispatch.c:110
  #10 _pam_dispatch (pamh=pamh@entry=0x7fffd8002910, flags=1, choice=choice@entry=1) at pam_dispatch.c:426
  #11 0x00007ffff7bce7e0 in pam_authenticate (pamh=0x7fffd8002910, flags=flags@entry=1) at pam_auth.c:34
  #12 0x00000000005ae567 in XA_mh_authorize (username=username@entry=0x7fffd80028d0 "pamtest-edvint", password=password@entry=0x7fffd80028f0 "pamtest-edvint", error=error@entry=0x7ffff31e1be8) at xa_auth.c:83
  #13 0x00000000005adf20 in stub_XA_mh_authorize (username=<optimized out>, password=<optimized out>) at xa_auth_stubs.c:42
```

`pam_start` and `pam_end` doesn't help here, because on `pam_end` the library is `dlclose`-ed, so on next `pam_authenticate` it will have to go through the initialization code again.
(This initialization code would've belonged into `pam_start`, not `pam_authenticate`, but there are several layers here including a call to `crypt_r`).
Upstream has fixed this problem >5 years ago by switching to libxcrypt instead.

Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Christian Lindig <christian.lindig@cloud.com>
edwintorok pushed a commit to edwintorok/xen-api that referenced this pull request Sep 24, 2024
@mseri
Merge pull request xapi-project#10 from gaborigloi/sync_opam
4ee3ad6 
Sync opam file with xs-opam
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@djs55 @robhoes