OustandingToken only created when refresh token is used to get a new key pair and not at creation ? Bug ?

[StitiFatah]

from the doc :

If the blacklist app is detected in INSTALLED_APPS, Simple JWT will add any generated refresh or sliding tokens to a list of outstanding tokens. It will also check that any refresh or sliding token does not appear in a blacklist of tokens before it considers it as valid.

I'm working with 'ROTATE_REFRESH_TOKENS': True and 'BLACKLIST_AFTER_ROTATION': True

When asking for a token pair through TokenObtainPairView it indeed directly create an OutstandingToken referencing the given refresh_token but when asking for a new pair via TokenRefreshView no OutstandingToken referencing the new given refresh token is created but instead it creates an OutstandingToken for the refresh_token which was used to ask a new pair and thus a BlacklistedToken.
Don't know if it's clear enough. In summary it's :

1-Login through /api/token --> 1 pair given ( let's call the refresh one refresh_token_0 ) --> outstanding_token_0 created from refresh_token_0
2-Claiming a new pair through /api/token/refresh ---> 1 new pair given, outstanding_token_0 is blacklisted but no outstanding_token_1 created
3-Claiming a new pair through /api/token/refresh with token_refresh_1, 1 new pair given and then oustanding_token_1 created + blacklisted

Expected behavior :

1-Login through /api/token --> 1 pair given ( let's call the refresh one refresh_token_0 ) --> outstanding_token_0 created from refresh_token_0
2-Claiming a new pair through /api/token/refresh ---> 1 new pair given, outstanding_token_0 is blacklisted and outstanding_token_1 created
3-Claiming a new pair through /api/token/refresh with token_refresh_1, new pair given, outsatnding_token_1 blacklisted then oustanding_token_2 created

So apart from the first claim when the user ask for credentials himself the new generated refresh tokens aren't creating outstanding token (when generated from /api/token/refresh), It waits for a new /api/token/refresh claim with this token to create outsanding token referecing it + blacklist in the same time.

Is this the expected behavior and I am missing something ? What's the point of Outstanding Token if it's directly blacklisted ( apart from the first time at user's login ) and not created one cycle before ? Is it possible to create them when generated ?

Thanks in advance

[Andrew-Chen-Wang]

Really late sorry. Will come back to this on the weekend. Sit tight.

[write2anitab]

I am also observing similar issue with the api/login/refresh not inserting the new generated refresh token to the outstanding list

The api/login correctly adds the generated token to the outstanding list

 select count(*) from token_blacklist_outstandingtoken; 
 count = 1

 select count(*) from token_blacklist_blacklistedtoken; 
  count = 0

The api/login/refresh correctly blacklists the refresh token , but does not add the generated refresh token to the outstanding list

 select count(*) from token_blacklist_outstandingtoken; (**expecting count 2 here **)
  count = 1 
  
 select count(*) from token_blacklist_blacklistedtoken;
  count = 1

django-rest-framework-simplejwt/rest_framework_simplejwt/serializers.py

For the api/login the TokenObtainPairSerializer implements the TokenObtainSerializer get_token method which adds to the outstanding list

class TokenObtainPairSerializer(TokenObtainSerializer):
    @classmethod
    def get_token(cls, user):
        return RefreshToken.for_user(user)
    ..........

However for the api/login/refresh the TokenRefreshSerializer is not implementing the TokenObtainSerializer . It blacklists the old access token, but does not add the new generated token to the outstanding list

class TokenRefreshSerializer(serializers.Serializer):
    refresh = serializers.CharField()
 ............
 if api_settings.ROTATE_REFRESH_TOKENS:
       if api_settings.BLACKLIST_AFTER_ROTATION:
            try:
                  # Attempt to blacklist the given refresh token
                  refresh.blacklist()
              except AttributeError:
                  # If blacklist app not installed, `blacklist` method will
                   # not be present
                   pass

[adeng322]

Im having the same issue as well

[Andrew-Chen-Wang]

Sorry, not sure why I was having a hard time parsing through this. @write2anitab the code doing refresh.blacklist() creates an outstanding token:

djangorestframework-simplejwt/rest_framework_simplejwt/tokens.py

Lines 196 to 213 in 1c7f005

	def blacklist(self):
	"""
	Ensures this token is included in the outstanding token list and
	adds it to the blacklist.
	"""
	jti = self.payload[api_settings.JTI_CLAIM]
	exp = self.payload['exp']
	# Ensure outstanding token exists with given jti
	token, _ = OutstandingToken.objects.get_or_create(
	jti=jti,
	defaults={
	'token': str(self),
	'expires_at': datetime_from_epoch(exp),
	},
	)
	return BlacklistedToken.objects.get_or_create(token=token)

We also have tests to verify this:

djangorestframework-simplejwt/tests/test_serializers.py

Lines 285 to 322 in 1c7f005

	def test_it_should_blacklist_refresh_token_if_tokens_should_be_rotated_and_blacklisted(self):
	self.assertEqual(OutstandingToken.objects.count(), 0)
	self.assertEqual(BlacklistedToken.objects.count(), 0)
	refresh = RefreshToken()
	refresh['test_claim'] = 'arst'
	old_jti = refresh['jti']
	old_exp = refresh['exp']
	# Serializer validates
	ser = TokenRefreshSerializer(data={'refresh': str(refresh)})
	now = aware_utcnow() - api_settings.ACCESS_TOKEN_LIFETIME / 2
	with override_api_settings(ROTATE_REFRESH_TOKENS=True, BLACKLIST_AFTER_ROTATION=True):
	with patch('rest_framework_simplejwt.tokens.aware_utcnow') as fake_aware_utcnow:
	fake_aware_utcnow.return_value = now
	self.assertTrue(ser.is_valid())
	access = AccessToken(ser.validated_data['access'])
	new_refresh = RefreshToken(ser.validated_data['refresh'])
	self.assertEqual(refresh['test_claim'], access['test_claim'])
	self.assertEqual(refresh['test_claim'], new_refresh['test_claim'])
	self.assertNotEqual(old_jti, new_refresh['jti'])
	self.assertNotEqual(old_exp, new_refresh['exp'])
	self.assertEqual(access['exp'], datetime_to_epoch(now + api_settings.ACCESS_TOKEN_LIFETIME))
	self.assertEqual(new_refresh['exp'], datetime_to_epoch(now + api_settings.REFRESH_TOKEN_LIFETIME))
	self.assertEqual(OutstandingToken.objects.count(), 1)
	self.assertEqual(BlacklistedToken.objects.count(), 1)
	# Assert old refresh token is blacklisted
	self.assertEqual(BlacklistedToken.objects.first().token.jti, old_jti)

Will need more debugging information and a reproducible git repository if possible.

[adeng322]

Sorry, not sure why you having a hard time parsing through this. The issue is "with the api/login/refresh not inserting the newly generated refresh token to the outstanding list". I believe the code snippet you provided has nothing to do with this issue but just to show how a refresh token is blacklisted. Correct me if I'm wrong, please. @StitiFatah @write2anitab provided such a detailed explanation. 3 people reflected the same issue. how can this just being simply labeled as invalid?

[Andrew-Chen-Wang]

@adeng322 They're great logs and fantastic explanations, just really long run-on sentences that personally made it hard for me to decipher...

For reproducibility, the settings ROTATE_REFRESH_TOKENS and BLACKLIST_AFTER_ROTATION are True. You need to send username/pw credentials to get your initial tokens. Then you refresh your access token and expect a new refresh and access token.

The "bug" is that the OutstandingToken is not created. The first permalink is where we generate the OutstandingToken and a BlacklistedToken. If that OutstandingToken is not created, then deduce: why is the BlacklistToken still created? It's because the get_or_create QuerySet method got an existing OutstandingToken that matched the token value.

The second snippet is our test code that I added for y'all to verify if we implemented our test case correctly. Please note the lines:

    now = aware_utcnow() - api_settings.ACCESS_TOKEN_LIFETIME / 2 
  
     with override_api_settings(ROTATE_REFRESH_TOKENS=True, BLACKLIST_AFTER_ROTATION=True): 
         with patch('rest_framework_simplejwt.tokens.aware_utcnow') as fake_aware_utcnow: 
             fake_aware_utcnow.return_value = now 
             self.assertTrue(ser.is_valid()) 

In other words, give it some time. Hope that explains it @adeng322

[j-zaballa]

I believe this is the same as #25. And I also believe this should not be the expected behavior.

Example: we want to give the user the option to logout on all devices. The function blacklists every OutstandingToken for the user. But, if token rotation is on, the rotation refresh tokens do not have a corresponding OutstandingToken record. Therefore, we can not blacklist those tokens and the user always gets a new access token with the refresh token.

[j-zaballa]

IMHO this is the relevant code for the question at hand. This is where, if ROTATE_REFRESH_TOKENS is enabled, we get a new refresh token. But this new refresh token is never stored into the database.

djangorestframework-simplejwt/rest_framework_simplejwt/serializers.py

Lines 104 to 125 in e5baf6a

	def validate(self, attrs):
	refresh = RefreshToken(attrs['refresh'])
	data = {'access': str(refresh.access_token)}
	if api_settings.ROTATE_REFRESH_TOKENS:
	if api_settings.BLACKLIST_AFTER_ROTATION:
	try:
	# Attempt to blacklist the given refresh token
	refresh.blacklist()
	except AttributeError:
	# If blacklist app not installed, `blacklist` method will
	# not be present
	pass
	refresh.set_jti()
	refresh.set_exp()
	refresh.set_iat()
	data['refresh'] = str(refresh)
	return data

I would be willing to contribute with a PR if this is indeed the way to go. Of course I might be missing something here. Thank you!

[Andrew-Chen-Wang]

@j-zaballa please provide a PR as it may be helpful for others if they disagree with me. I'm sure that, if this is included in the main package, this may require a major version bump again. Not that a PR will not get merged, I'm just really busy with school atm.

[j-zaballa]

@Andrew-Chen-Wang no worries :). I will submit a PR asap so we can discuss this topic there. Best regards!

[SumitJainUTD]

This is not fixed, I can still reproduce it

[ratataque]

This is not fixed, I can still reproduce it

This Pull Request fixes it

[Aniket-Singla]

I see 2 open PRs for this issue. Waiting for one to get reviewed and merged.