[Bug] OSGi shiro unable to restore rememberme session

[steinarb]

Search before asking

• I had searched in the issues and found no similar issues.

Environment

Apache karaf 4.4.7, Java 21.0.4+8-LTS-274, debian 12.10 "bookworm", amd64

Shiro version

Shiro 2.0.2

What was the actual outcome?

When restoring rememberme sessions I get the following stack traces in the log:

2025-04-12T21:00:38,643 | WARN  | qtp1776555921-646 | DefaultSecurityManager           | 199 - org.apache.shiro.core - 2.0.2 | Delegate RememberMeManager instance of type [org.apache.shiro.web.mgt.CookieRememberMeManager] threw an exception during getRememberedPrincipals().
org.apache.shiro.lang.io.SerializationException: Unable to deserialize argument byte array.
	at org.apache.shiro.lang.io.DefaultSerializer.deserialize(DefaultSerializer.java:90) ~[!/:2.0.2]
	at org.apache.shiro.mgt.AbstractRememberMeManager.deserialize(AbstractRememberMeManager.java:523) ~[!/:2.0.2]
	at org.apache.shiro.mgt.AbstractRememberMeManager.convertBytesToPrincipals(AbstractRememberMeManager.java:436) ~[!/:2.0.2]
	at org.apache.shiro.mgt.AbstractRememberMeManager.getRememberedPrincipals(AbstractRememberMeManager.java:399) ~[!/:2.0.2]
	at org.apache.shiro.mgt.DefaultSecurityManager.getRememberedIdentity(DefaultSecurityManager.java:618) ~[!/:2.0.2]
	at org.apache.shiro.mgt.DefaultSecurityManager.resolvePrincipals(DefaultSecurityManager.java:506) ~[!/:2.0.2]
	at org.apache.shiro.mgt.DefaultSecurityManager.createSubject(DefaultSecurityManager.java:350) ~[!/:2.0.2]
	at org.apache.shiro.subject.Subject$Builder.buildSubject(Subject.java:844) ~[!/:2.0.2]
	at org.apache.shiro.web.subject.WebSubject$Builder.buildWebSubject(WebSubject.java:148) ~[!/:2.0.2]
	at org.apache.shiro.web.servlet.AbstractShiroFilter.createSubject(AbstractShiroFilter.java:306) ~[!/:2.0.2]
	at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:374) ~[!/:2.0.2]
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:156) ~[!/:2.0.2]
	at org.ops4j.pax.web.service.spi.servlet.OsgiInitializedFilter.doFilter(OsgiInitializedFilter.java:176) ~[!/:?]
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) ~[!/:9.4.57.v20241219]
	at org.ops4j.pax.web.service.jetty.internal.PaxWebFilterHolder.doFilter(PaxWebFilterHolder.java:208) ~[!/:?]
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[!/:9.4.57.v20241219]
	at org.ops4j.pax.web.service.spi.servlet.OsgiFilterChain.doFilter(OsgiFilterChain.java:113) ~[!/:?]
	at org.ops4j.pax.web.service.jetty.internal.PaxWebServletHandler.doHandle(PaxWebServletHandler.java:334) ~[!/:?]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234) ~[!/:9.4.57.v20241219]
	at org.ops4j.pax.web.service.jetty.internal.PrioritizedHandlerCollection.handle(PrioritizedHandlerCollection.java:96) ~[!/:?]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) ~[!/:9.4.57.v20241219]
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) ~[!/:9.4.57.v20241219]
	at java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.lang.ClassNotFoundException: Unable to load ObjectStreamClass [org.apache.shiro.subject.SimplePrincipalCollection: static final long serialVersionUID = -6305224034025797558L;]: 
	at org.apache.shiro.lang.io.ClassResolvingObjectInputStream.resolveClass(ClassResolvingObjectInputStream.java:55) ~[!/:2.0.2]
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2061) ~[?:?]
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1927) ~[?:?]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2252) ~[?:?]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1762) ~[?:?]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:540) ~[?:?]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:498) ~[?:?]
	at org.apache.shiro.lang.io.DefaultSerializer.deserialize(DefaultSerializer.java:85) ~[!/:2.0.2]
	... 44 more
Caused by: org.apache.shiro.lang.util.UnknownClassException: Unable to load class named [org.apache.shiro.subject.SimplePrincipalCollection] from the thread context, current, or system/application ClassLoaders.  All heuristics have been exhausted.  Class could not be found.
	at org.apache.shiro.lang.util.ClassUtils.forName(ClassUtils.java:179) ~[!/:2.0.2]
	at org.apache.shiro.lang.io.ClassResolvingObjectInputStream.resolveClass(ClassResolvingObjectInputStream.java:53) ~[!/:2.0.2]
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2061) ~[?:?]
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1927) ~[?:?]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2252) ~[?:?]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1762) ~[?:?]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:540) ~[?:?]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:498) ~[?:?]
	at org.apache.shiro.lang.io.DefaultSerializer.deserialize(DefaultSerializer.java:85) ~[!/:2.0.2]
	... 44 more

What was the expected outcome?

No errors on rememberme restore.

How to reproduce

Run an application that uses shiro rememberme shiro on an OSGi platform

Debug logs

No response

[steinarb]

The core of the problem is this:

 Caused by: org.apache.shiro.lang.util.UnknownClassException: Unable to load class named [org.apache.shiro.subject.SimplePrincipalCollection] from the thread context, current, or system/application ClassLoaders.  All heuristics have been exhausted.  Class could not be found.
	at org.apache.shiro.lang.io.ClassResolvingObjectInputStream.resolveClass(ClassResolvingObjectInputStream.java:55) ~[!/:2.0.2]

The class ClassResolvingObjectInputStream is in the shiro-lang OSGi bundle (i.e. jar file with import-package and export-package declarations in the MANIFEST.MF).

For a class to be loaded and, found by ClassResolvingObjectInputStream, the package the class resides (i.e. org.apache.shiro.subject for SimplePrincipalCollection) in needs to be imported into shiro-lang.

And the package org.apache.shiro.subject (found in shiro-core) is not listed in the Import-Package declaration of the shiro-lang MANIFEST.MF (formatted for readability):

Import-Package:
 javax.servlet.jsp;resolution:=optional,
 java.beans,
 java.io,
 java.lang,
 java.lang.annotation,
 java.lang.invoke,
 java.lang.ref,
 java.lang.reflect,
 java.net,
 java.text,
 java.util,
 java.util.concurrent,
 java.util.concurrent.locks,
 org.apache.shiro.lang;version="[2.0,3)",
 org.apache.shiro.lang.codec;version="[2.0,3)",
 org.apache.shiro.lang.util;version="[2.0,3)",
 org.slf4j;version="[2.0,3)"

Below is the export-package of the MANIFEST.MF of shiro-core, also reformatted for readability, and I see one obvious problem: packages in shiro-core require packages from shiro-lang (which means shiro-lang can't import packages from shiro-core, because then you would have a loop).

So this requires changes to shiro to fix.

There are two ways of breaking the loop:

1. Put everything in one huge jar (the quick and dirty and wrong way)
2. Split out the stuff included both places into a separate bundle and
   import its packages in the bundles that need it

Export-package of the MANIFEST.MF of shiro-core, reformatted for readability:

Export-Package:
 org.apache.shiro;version="2.0.2";uses:="org.apache.shiro.lang,org.apache.shiro.mgt,org.apache.shiro.subject",
 org.apache.shiro.aop;version="2.0.2";uses:="org.apache.shiro.subject",org.apache.shiro.authc;version="2.0.2";uses:="org.apache.shiro.authz,org.apache.shiro.lang,org.apache.shiro.lang.util,org.apache.shiro.subject",
 org.apache.shiro.authc.credential;version="2.0.2";uses:="org.apache.shiro.authc,org.apache.shiro.crypto.hash,org.apache.shiro.crypto.hash.format,org.apache.shiro.lang.codec,org.apache.shiro.lang.util",
 org.apache.shiro.authc.pam;version="2.0.2";uses:="org.apache.shiro.authc,
 org.apache.shiro.realm,org.apache.shiro.subject",
 org.apache.shiro.authz;version="2.0.2";uses:="org.apache.shiro.authz.permission,org.apache.shiro.lang,org.apache.shiro.realm,org.apache.shiro.subject",
 org.apache.shiro.authz.annotation;version="2.0.2",
 org.apache.shiro.authz.aop;version="2.0.2";uses:="org.apache.shiro.aop,org.apache.shiro.authz",
 org.apache.shiro.authz.permission;version="2.0.2";uses:="org.apache.shiro.authz,org.apache.shiro.lang",
 org.apache.shiro.concurrent;version="2.0.2";uses:="org.apache.shiro.subject",
 org.apache.shiro.dao;version="2.0.2";uses:="org.apache.shiro.lang",org.apache.shiro.env;version="2.0.2";uses:="org.apache.shiro.config,org.apache.shiro.lang,org.apache.shiro.lang.util,org.apache.shiro.mgt",
 org.apache.shiro.ini;version="2.0.2";uses:="org.apache.shiro.config,org.apache.shiro.config.ogdl,org.apache.shiro.mgt,org.apache.shiro.realm,org.apache.shiro.util",
 org.apache.shiro.jndi;version="2.0.2";uses:="javax.naming,org.apache.shiro.lang.util",
 org.apache.shiro.ldap;version="2.0.2";uses:="org.apache.shiro.dao",
 org.apache.shiro.mgt;version="2.0.2";uses:="org.apache.shiro.authc,org.apache.shiro.authz,org.apache.shiro.cache,org.apache.shiro.crypto.cipher,org.apache.shiro.event,org.apache.shiro.lang.io,org.apache.shiro.lang.util,org.apache.shiro.realm,org.apache.shiro.session,org.apache.shiro.session.mgt,org.apache.shiro.subject",
 org.apache.shiro.realm;version="2.0.2";uses:="org.apache.shiro.authc,org.apache.shiro.authc.credential,org.apache.shiro.authz,org.apache.shiro.authz.permission,org.apache.shiro.cache,org.apache.shiro.lang.util,org.apache.shiro.subject",
 org.apache.shiro.realm.activedirectory;version="2.0.2";uses:="javax.naming,javax.naming.ldap,org.apache.shiro.authc,org.apache.shiro.authz,org.apache.shiro.realm.ldap,org.apache.shiro.subject",
 org.apache.shiro.realm.jdbc;version="2.0.2";uses:="javax.sql,org.apache.shiro.authc,org.apache.shiro.authz,org.apache.shiro.realm, org.apache.shiro.subject",
 org.apache.shiro.realm.jndi;version="2.0.2";uses:="org.apache.shiro.jndi,org.apache.shiro.realm",
 org.apache.shiro.realm.ldap;version="2.0.2";uses:="javax.naming,javax.naming.directory,javax.naming.ldap,org.apache.shiro.authc,org.apache.shiro.authz,org.apache.shiro.realm,org.apache.shiro.subject",
 org.apache.shiro.realm.text;version="2.0.2";uses:="org.apache.shiro.config,org.apache.shiro.lang.util,org.apache.shiro.realm",
 org.apache.shiro.session;version="2.0.2";uses:="org.apache.shiro.lang",
 org.apache.shiro.session.mgt;version="2.0.2";uses:="org.apache.shiro.authz,org.apache.shiro.cache,org.apache.shiro.event,org.apache.shiro.lang.util,org.apache.shiro.session,org.apache.shiro.session.mgt.eis,org.apache.shiro.util",
 org.apache.shiro.session.mgt.eis;version="2.0.2";uses:="org.apache.shiro.cache,org.apache.shiro.session",
 org.apache.shiro.subject;version="2.0.2";uses:="org.apache.shiro.authc,org.apache.shiro.authz,org.apache.shiro.lang,org.apache.shiro.mgt,org.apache.shiro.session",
 org.apache.shiro.subject.support;version="2.0.2";uses:="org.apache.shiro.authc,org.apache.shiro.authz,org.apache.shiro.mgt,org.apache.shiro.session,org.apache.shiro.session.mgt,org.apache.shiro.subject,org.apache.shiro.util",
 org.apache.shiro.util;version="2.0.2";uses:="org.apache.shiro.authz,org.apache.shiro.authz.permission,org.apache.shiro.lang.util,org.apache.shiro.mgt,org.apache.shiro.subject!"

[lprimak]

Is this not a duplicate of #1449? @steinarb ? I think it is...

[steinarb]

Nope. It is an OSGi import problem, which #1449 also is, but very different in nature (different packages and different bundles involved).

This issue is a class in shiro-lang not able to create an instance of a class in shiro-core (because shiro-lang doesn't import the package).

And shiro-lang can't import the package from shiro-core because shiro-core imports a package from shiro-lang and introducing a cross dependency would make the bundles refuse to load.

I tried splitting subject out into a separate bundle, but the classes in subject require a lot of packages in shiro-core (and two packages from shiro-lang):

org.apache.shiro.authc
org.apache.shiro.authc.pam
org.apache.shiro.authz
org.apache.shiro.lang
org.apache.shiro.lang.util
org.apache.shiro.mgt
org.apache.shiro.session
org.apache.shiro.util

Most troublesome are org.apache.shiro.lang because that means that the classes here and the classes in lang that wants to use them have a cross dependency (even if I were to split out all of the reset of the packages which are all in shiro-core).

Short story: I don't know any way of fixing this other than moving shiro-lang back into shiro-core (which IMO would be the wrong thing to do).

[steinarb]

Hm... maybe setting thread context class loader to the thread of the class calling ClassResolvingObjectInputStream (like in #1500 (comment)) would be a workaround?

But I don't know where the calling code is? Maybe in shiro-core somewhere?

Actually, the more I think about it, this issue is more similar to #1500 than #1449 because this is a case of reflection not being able to load a class by name.

And as I recall, there was code in Shiro trying several classloaders, and the thread context class loader was one of them? (my suggestion was to send a classloader in as a parameter, but setting the classloader as suggested by @lprimak worked).

#1449 is different because that means the OSGi bundle doesn't start up because all of its manifest imports can't be resolved (not missing classes during reflection, but problems with OSGi). I have no idea why that hasn't bitten me as well? Ah: I'm not using annotations from either javax.annotation.security or jakarta.annotation.security. I define access to paths in shiro.ini.

[lprimak]

@fpapon Possibly another OSGi / Karaf issue

[steinarb]

I have made a one-liner fix: #2084

I have tested it by rebuilding all apps in myapps-test with shiro 2.0.0-SNAPSHOT with my fix and the stack trace on restoring serialized remember me went away.

The fix is the same as I used as a workaround here: #1500 (comment)
(setting the classloader from the caller as thread context classloader so that reflection can find the classes it needs to restore)

[fpapon]

A good practice when switching the TCCL, is to:

1. store the current context in a local var
2. setting the new context
3. doing stuff
4. setting back the original context

[lprimak]

That's exactly what I did in #2085

[lprimak]

Also, thank you for your continued involvement Steinar, this is very helpful!

[steinarb]

A good practice when switching the TCCL, is to:

1. store the current context in a local var
2. setting the new context
3. doing stuff
4. setting back the original context

Check! Not much experience with TCCL #1500 (comment) was the first time I encountered the concept.