Create NEST Extension Interface for NEST compatible with new registration scheme and unloading #3103

heplesser · 2024-02-12T17:15:00Z

The title says it all. This is still in draft stage as I need to iron out some problems, especially segfaults on lt_dlclose() of the module. ResetKernel() works and one can install the module again afterwards, but I have not yet checked what happens if the module changes between reloads. At present, I need to provide the (DY)LD_LIBRARY_PATH to the install location of the module. The old DynamicLoader is replaced by ModuleManager and NESTExtensionInterface is the base class for extension modules.

In nest/nest-extension-module#26 you will find a version of mymodule (in the modulemgr branch) that works with this branch of NEST.

This fixes #3064.

…still some TODOs

heplesser · 2024-02-13T08:04:52Z

Everything works now, including loading a modified module after ResetKernel(). One still needs to make sure that the path to the extension module is explicitly set in (DY)LD_LIBRARY_PATH, even if it is the same as the installation location for the NEST libs. Got to see if one can improve on this, would need some search-path expansion in the module manager. But otherwise, this is ready for review.

…ction

pnbabu

Thank you for these changes! I tested this with modules generated from NESTML and the install works fine. I have a few comments. Please find them below.

nestkernel/module_manager.cpp

nestkernel/nest_extension_interface.h

CMakeLists.txt

Co-authored-by: Pooja Babu <75320801+pnbabu@users.noreply.github.com>

…ix-3064

heplesser · 2024-02-20T21:13:57Z

@pnbabu Thank you for your review! I have fixed most items and commented on the two remaining.

heplesser · 2024-02-29T08:02:58Z

@clinssen This PR should definitely go into 3.7, but after discussion with @terhorstd I remove the milestone again for technical reasons.

terhorstd

Some minor points and some questions still open.

nestkernel/nestmodule.cpp

CMakeLists.txt

cmake/ProcessOptions.cmake

nestkernel/kernel_manager.h

nestkernel/module_manager.h

pnbabu

Looks good to me! Just some minor suggestions.

nestkernel/module_manager.h

Co-authored-by: Dennis Terhorst <terhorstd@users.noreply.github.com>

Co-authored-by: Pooja Babu <75320801+pnbabu@users.noreply.github.com>

…ix-3064

heplesser · 2024-03-12T11:27:22Z

@terhorstd Thanks for your comments! Could you take a look at my replies?

nestkernel/random_manager.cpp

.#foo

terhorstd

Looks good so far. Thanks for the complex work!

heplesser added 4 commits February 12, 2024 00:58

Replace dynamicloader with ModuleManager

fc81bd3

Make ModuleManager safe in case of missing ltdl

fc26af6

Add Install function to SLI interpreter

ee0f46b

Further significant revision of extension interface, now with class; …

1464290

…still some TODOs

heplesser added T: Enhancement New functionality, model or documentation S: High Should be handled next I: External API Developers of extensions or other language bindings may need to adapt their code labels Feb 12, 2024

heplesser requested review from clinssen, pnbabu and terhorstd February 12, 2024 17:15

heplesser added 3 commits February 12, 2024 18:35

Add missing include

bda895b

Paths to libraries can now be set correctly using -Dwith-...=... flags

1c2a0b0

First success with module closing (no python)

62ae400

heplesser marked this pull request as ready for review February 13, 2024 08:01

heplesser added 2 commits February 13, 2024 11:25

Fix Cmake _ROOT variables for compatibility with pre-3.27

6190fe0

Add NEST lib install dir to LTDL search path on ModuleManager constru…

ea33338

…ction

pnbabu suggested changes Feb 20, 2024

View reviewed changes

heplesser and others added 4 commits February 20, 2024 22:00

Update nestkernel/module_manager.cpp

0af2567

Co-authored-by: Pooja Babu <75320801+pnbabu@users.noreply.github.com>

Remove externa-modules from cmake options doc

45c167e

Fix comment

f404915

Merge branch 'fix-3064' of github.com:heplesser/nest-simulator into f…

68bfd1d

…ix-3064

clinssen added this to the NEST 3.7 milestone Feb 21, 2024

heplesser added 3 commits February 28, 2024 13:19

Prohibit module Install after nodes have been created

551543d

Merge branch 'master' of github.com:nest/nest-simulator into fix-3064

8a4d999

Add missing semicolon

31aee16

heplesser removed this from the NEST 3.7 milestone Feb 29, 2024

Merge branch 'master' of github.com:nest/nest-simulator into fix-3064

61be56b

Provide reinitialize function also if no ltdl

b1e3165

terhorstd requested changes Mar 12, 2024

View reviewed changes

pnbabu approved these changes Mar 12, 2024

View reviewed changes

nestkernel/module_manager.h Outdated Show resolved Hide resolved

nestkernel/module_manager.h Outdated Show resolved Hide resolved

nestkernel/module_manager.h Outdated Show resolved Hide resolved

terhorstd mentioned this pull request Mar 12, 2024

Revised module suitable for new NESTExtensionModule interface nest/nest-extension-module#26

Merged

heplesser and others added 6 commits March 12, 2024 12:10

Update nestkernel/module_manager.h

14dc62c

Co-authored-by: Dennis Terhorst <terhorstd@users.noreply.github.com>

Update nestkernel/module_manager.h

aacd3b1

Co-authored-by: Pooja Babu <75320801+pnbabu@users.noreply.github.com>

Update nestkernel/module_manager.h

6804084

Co-authored-by: Pooja Babu <75320801+pnbabu@users.noreply.github.com>

Merge branch 'master' of github.com:nest/nest-simulator into fix-3064

bef163e

Remove version number from nestmodule::name()

9f5a175

Merge branch 'fix-3064' of github.com:heplesser/nest-simulator into f…

e73b52b

…ix-3064

heplesser requested review from clinssen and terhorstd March 12, 2024 11:27

Fix formatting

0b4f406

heplesser mentioned this pull request Mar 12, 2024

Move registration of ConnBuilders, Backends, growth curves from nestmodule to respective managers, allowing unloading #3149

Merged

clinssen reviewed Mar 13, 2024

View reviewed changes

nestkernel/random_manager.cpp Outdated Show resolved Hide resolved

clinssen reviewed Mar 13, 2024

View reviewed changes

.#foo Outdated Show resolved Hide resolved

heplesser added 2 commits March 13, 2024 11:20

Remove unintentionally commited file.

4b380aa

Consistently name adjust_number_of..._only parameter

e17951e

heplesser requested a review from clinssen March 13, 2024 10:22

heplesser mentioned this pull request Mar 14, 2024

Add link to 3.7 whats new page in docs #3153

Merged

clinssen approved these changes Mar 18, 2024

View reviewed changes

terhorstd approved these changes Mar 19, 2024

View reviewed changes

abigailm merged commit 3ee7f98 into nest:master Mar 19, 2024

pnbabu mentioned this pull request Apr 4, 2024

Change model registration for NEST target nest/nestml#1002

Merged

BrewTestBot mentioned this pull request Apr 12, 2024

nest 3.7 Homebrew/homebrew-core#168752

Merged

heplesser deleted the fix-3064 branch April 24, 2024 14:09

clinssen mentioned this pull request May 17, 2024

nest.ResetKernel() resets installed modules nest/nestml#1054

Closed

heplesser mentioned this pull request Dec 1, 2025

Simplify test implementation and remove complex reusable code for better readability (replacement for #3510) #3661

Merged

heplesser added a commit to heplesser/nest-simulator that referenced this pull request Dec 1, 2025

Update comment to reflect behavior after merge of nest#3103.

a65c724

Create NEST Extension Interface for NEST compatible with new registration scheme and unloading #3103

Create NEST Extension Interface for NEST compatible with new registration scheme and unloading #3103

Uh oh!

Conversation

heplesser commented Feb 12, 2024

Uh oh!

heplesser commented Feb 13, 2024

Uh oh!

pnbabu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

heplesser commented Feb 20, 2024

Uh oh!

heplesser commented Feb 29, 2024

Uh oh!

terhorstd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pnbabu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

heplesser commented Mar 12, 2024

Uh oh!

Uh oh!

Uh oh!

terhorstd left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants