Skip to content

fix: OnNetworkDespawn not called during shutdown #1390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Nov 16, 2021

Conversation

NoelStephensUnity
Copy link
Collaborator

@NoelStephensUnity NoelStephensUnity commented Nov 3, 2021

When shutting down directly, NetworkObjects were not running through the OnNetworkDespawn process prior to being destroyed. This fixes the issue by assuring NetworkObjects are despawned prior to being destroyed (if applicable).

MTT-1542

Also Fixes:
MTT-1254

This was discovered by the samples group and relates to:
MTT-1472

PR Checklist

  • Have you added a backport label (if needed)? For example, the type:backport-release-* label. After you backport the PR, the label changes to stat:backported-release-*.
  • Have you updated the changelog? Each package has a CHANGELOG.md file.

Changelog

com.unity.netcode.gameobjects

  • Fixed NetworkObjects not being despawned before they are destroyed during shutdown for client, host, and server instances.

Testing and Documentation

  • Includes integration test that confirms this fix applies to client, host, and server instances.

Renaming and refactoring DestroyNonSceneObjects to be DespawnNetworkObjects as this is really what it is doing.  The destroy portion is either up to the handler, if an in-scene NetworkObject it lets the unloading of the scene destroy it, and the rest are always destroyed.
updates to the changes in NetworkSpawnManager
Adding a "ShutdownInProgress" flag to be able to easily check if we are shutting down.

Adding a check for the NetworkManager.LocalClient since we are despawning in the shutdown method.

Ignoring the de-parenting of NetworkObjects when shutting down (this will have to be discussed a bit).
Test fix:  I think this was just an unexposed bug in the NetworkRigidBody2D test, but will verify this before making this final.

Fixed some additional minor issues like checking for IsSpawned as opposed to NetworkManager.IsListening and IsSpawned as opposed to NetworkObject.IsSpawned.
Avoiding spam when despawning and shutting down.
Renaming DespawnNetworkObjects to DespawnAndDestroyNetworkObjects.
If a NetworkObject is not spawned it gets destroyed.
Fixing issue with editor tests and the NetworkManager.Singleton instance.
Adjusting the logic a bit on getting the NetworkObject associated with a NetworkBehaviour.
{
NetworkLog.LogWarning($"Could not get {nameof(NetworkObject)} for the {nameof(NetworkBehaviour)}. Are you missing a {nameof(NetworkObject)} component?");
if (NetworkLog.CurrentLogLevel < LogLevel.Normal)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this area has a high probability to generate a lot of spam, I reduced it to require the NetworkLog level be Developer as there can be several frames (during Update and FixedUpdate) where NetworkBehaviours will make checks that require a reference to the NetworkObject but it doesn't exist.

The question here is do we want to generate a warning here since NetworkObject can be null and should be valid to return a null value for m_NetworkObject.

After discussing this with Luke, it turned out there was a bug in the NetworkRigidBody2D where it was resetting the isKinematic to true.  This reverts the changes made to the NetworkRigidbody2DDynamicTest and fixes the bug with NetworkRigidBody2D.
@@ -668,17 +676,21 @@ internal void OnDespawnObject(NetworkObject networkObject, bool destroyGameObjec
return;
}

// Move child NetworkObjects to the root when parent NetworkObject is destroyed
foreach (var spawnedNetObj in SpawnedObjectsList)
// If we are shutting down the NetworkManager, then ignore resetting the parent
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MFatihMAR
This was my solution to handle the issue where changing the parent would cause NetworkObject.OnTransformParentChanged to log an exception because NetworkManager is null during shutdown when the NetworkObjects are being despawned. Adding an additional series of checks in NetworkObject.OnTransformParentChanged seemed less-than-optimal since this is specific to only despawning.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (NetworkManager == null || !NetworkManager.IsListening)
{
    transform.parent = m_CachedParent;
    Debug.LogException(new NotListeningException($"{nameof(NetworkManager)} is not listening, start a server or host before reparenting"));
    return;
}

is this the check you're referring to?

if so, we could split that logic up into 2 pieces: NetworkManager == null and NetworkManager.IsListening.
we'd return early if NetworkManager == null
we'd log exception if !NetworkManager.IsListening

also, I think we should not introduce the NetworkManager.ShutdownInProgress (see my other comment above) and also not inject this extra check here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is just below:

if (!NetworkManager.ShutdownInProgress)
{

Basically, if we a shutting down then ignore the parenting code...

Tests that validate OnNetworkDespawn is invoked during shutdown.
This tests server, host, and client relative instances.
update the changelog with the fixed information
@@ -333,6 +333,9 @@ public IReadOnlyList<ulong> ConnectedClientsIds
/// </summary>
public bool IsConnectedClient { get; internal set; }


public bool ShutdownInProgress { get; internal set; }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really, we should have a few states for the NetworkManager to provide the NetworkManager's current state.
This is a minimal fix in order to determine if we are in the middle of a shutdown.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of introducing this semi-fix/band-aid, what if we were to make NetworkManager.Singleton = null after we call NetworkManager.Shutdown (therefore destroy NOs first then set it to null)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would work when there is a single instance running, but our MultiInstsance tests would fail.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huh? why?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the NetworkManager.Singleton is one NetworkManager instance of (n) NetworkManager instances.
If you have 1 host and 4 clients running in a MultiInstance integration test and this code in NetworkManager is invoked during the shutdown of one of the clients:

        private void OnDestroy()
        {
            Shutdown();

            UnityEngine.SceneManagement.SceneManager.sceneUnloaded -= OnSceneUnloaded;

            if (Singleton == this)
            {
                Singleton = null;
            }
        }

But the NetworkManager of the client shutting down is not the same as the one assigned to the NetworkManager.Singleton, then it will not be set to null. Forcing it to null in the integration/unit test could lead to other issues where code may depend upon the Singleton. If only 1 client shuts down but the rest and the host stay active but we set the singleton to null... it would be not only introducing another semi-fix/band-aid...but the semi-fix/band-aid would be using a property for a purpose it was not intended and very likely would require us to fix other issues that might crop up by setting it to null while other NetworkManagers were still connected and active.

I didn't like adding yet another Boolean value to describe a "state of the NetworkManager". and is why I was thinking we might want to discuss coming up with some states for NetworkManager that let us know (some we already know but are various Boolean properties as opposed to just a single enum value):

  • The NetworkManager is being started
  • The NetworkManager is being initialized
  • The NetworkManager is connected at the transport
  • The NetworkManager is connected to network session
  • The NetworkManager is disconnecting the transport
  • The NetworkManager is shutting down
  • The NetworkManager is shut down.
    Perhaps not all of the above, but in that general direction.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not suggesting to force NetworkManager.Singleton = null, I'm saying if we call network destroy after something sets NetworkManager.Singleton = null then let's call network destroy on objects first and later let that other code do NetworkManager.Singleton = null

Copy link
Collaborator Author

@NoelStephensUnity NoelStephensUnity Nov 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, either case it appears we have another bool that signals the shutdown process that popped up recently.
For now, I will just make it this:
public bool ShutdownInProgress { get { return m_ShuttingDown; } }

I could make it internal but then users wouldn't be able to check this value for any edge case scenarios, which I could just use the m_ShuttingDown property but make it internal which would require me to change it to just ShuttingDown.

@NoelStephensUnity NoelStephensUnity marked this pull request as ready for review November 8, 2021 22:56
@unity-cla-assistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Comment on lines +285 to +286
if (m_NetworkObject == null || NetworkManager.Singleton == null ||
(NetworkManager.Singleton != null && !NetworkManager.Singleton.ShutdownInProgress))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NetworkManager.Singleton check should be another if-check with potentially another log message IMO.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will revisit this.

@@ -333,6 +333,9 @@ public IReadOnlyList<ulong> ConnectedClientsIds
/// </summary>
public bool IsConnectedClient { get; internal set; }


public bool ShutdownInProgress { get; internal set; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of introducing this semi-fix/band-aid, what if we were to make NetworkManager.Singleton = null after we call NetworkManager.Shutdown (therefore destroy NOs first then set it to null)?

@@ -668,17 +676,21 @@ internal void OnDespawnObject(NetworkObject networkObject, bool destroyGameObjec
return;
}

// Move child NetworkObjects to the root when parent NetworkObject is destroyed
foreach (var spawnedNetObj in SpawnedObjectsList)
// If we are shutting down the NetworkManager, then ignore resetting the parent
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (NetworkManager == null || !NetworkManager.IsListening)
{
    transform.parent = m_CachedParent;
    Debug.LogException(new NotListeningException($"{nameof(NetworkManager)} is not listening, start a server or host before reparenting"));
    return;
}

is this the check you're referring to?

if so, we could split that logic up into 2 pieces: NetworkManager == null and NetworkManager.IsListening.
we'd return early if NetworkManager == null
we'd log exception if !NetworkManager.IsListening

also, I think we should not introduce the NetworkManager.ShutdownInProgress (see my other comment above) and also not inject this extra check here.

NoelStephensUnity and others added 7 commits November 10, 2021 15:22
fixing issue with test and recent changes for when the internal shutdown method is invoked.
ShutdownInProgress now returns m_ShuttingDown.

Setting m_ShuttingDown to false at the end of ShutdownInternal in case the NetworkManager is going to be re-used.
line endings
This reverts commit e2ca11a.
fixing timing related issue for NetworkObjectOnNetworkDespawnTests.
@andrews-unity andrews-unity merged commit 1509c63 into develop Nov 16, 2021
@andrews-unity andrews-unity deleted the fix/OnNetworkDespawn-Not-Called-During-Shutdown branch November 16, 2021 00:34
andrews-unity pushed a commit that referenced this pull request Nov 24, 2021
* fix

Renaming and refactoring DestroyNonSceneObjects to be DespawnNetworkObjects as this is really what it is doing.  The destroy portion is either up to the handler, if an in-scene NetworkObject it lets the unloading of the scene destroy it, and the rest are always destroyed.

* fix

updates to the changes in NetworkSpawnManager

* fix

Adding a "ShutdownInProgress" flag to be able to easily check if we are shutting down.

Adding a check for the NetworkManager.LocalClient since we are despawning in the shutdown method.

Ignoring the de-parenting of NetworkObjects when shutting down (this will have to be discussed a bit).

* fix

Test fix:  I think this was just an unexposed bug in the NetworkRigidBody2D test, but will verify this before making this final.

Fixed some additional minor issues like checking for IsSpawned as opposed to NetworkManager.IsListening and IsSpawned as opposed to NetworkObject.IsSpawned.

* fix

Avoiding spam when despawning and shutting down.
Renaming DespawnNetworkObjects to DespawnAndDestroyNetworkObjects.
If a NetworkObject is not spawned it gets destroyed.

* test fix

Fixing issue with editor tests and the NetworkManager.Singleton instance.

* fix

Adjusting the logic a bit on getting the NetworkObject associated with a NetworkBehaviour.

* revert and fix

After discussing this with Luke, it turned out there was a bug in the NetworkRigidBody2D where it was resetting the isKinematic to true.  This reverts the changes made to the NetworkRigidbody2DDynamicTest and fixes the bug with NetworkRigidBody2D.

* test

Tests that validate OnNetworkDespawn is invoked during shutdown.
This tests server, host, and client relative instances.

* update changelog

update the changelog with the fixed information

* fix

fixing issue with test and recent changes for when the internal shutdown method is invoked.

* refactor

ShutdownInProgress now returns m_ShuttingDown.

Setting m_ShuttingDown to false at the end of ShutdownInternal in case the NetworkManager is going to be re-used.

* style

line endings

* Revert "style"

This reverts commit e2ca11a.

* fix

fixing timing related issue for NetworkObjectOnNetworkDespawnTests.
andrews-unity added a commit that referenced this pull request Nov 25, 2021
* fix

Renaming and refactoring DestroyNonSceneObjects to be DespawnNetworkObjects as this is really what it is doing.  The destroy portion is either up to the handler, if an in-scene NetworkObject it lets the unloading of the scene destroy it, and the rest are always destroyed.

* fix

updates to the changes in NetworkSpawnManager

* fix

Adding a "ShutdownInProgress" flag to be able to easily check if we are shutting down.

Adding a check for the NetworkManager.LocalClient since we are despawning in the shutdown method.

Ignoring the de-parenting of NetworkObjects when shutting down (this will have to be discussed a bit).

* fix

Test fix:  I think this was just an unexposed bug in the NetworkRigidBody2D test, but will verify this before making this final.

Fixed some additional minor issues like checking for IsSpawned as opposed to NetworkManager.IsListening and IsSpawned as opposed to NetworkObject.IsSpawned.

* fix

Avoiding spam when despawning and shutting down.
Renaming DespawnNetworkObjects to DespawnAndDestroyNetworkObjects.
If a NetworkObject is not spawned it gets destroyed.

* test fix

Fixing issue with editor tests and the NetworkManager.Singleton instance.

* fix

Adjusting the logic a bit on getting the NetworkObject associated with a NetworkBehaviour.

* revert and fix

After discussing this with Luke, it turned out there was a bug in the NetworkRigidBody2D where it was resetting the isKinematic to true.  This reverts the changes made to the NetworkRigidbody2DDynamicTest and fixes the bug with NetworkRigidBody2D.

* test

Tests that validate OnNetworkDespawn is invoked during shutdown.
This tests server, host, and client relative instances.

* update changelog

update the changelog with the fixed information

* fix

fixing issue with test and recent changes for when the internal shutdown method is invoked.

* refactor

ShutdownInProgress now returns m_ShuttingDown.

Setting m_ShuttingDown to false at the end of ShutdownInternal in case the NetworkManager is going to be re-used.

* style

line endings

* Revert "style"

This reverts commit e2ca11a.

* fix

fixing timing related issue for NetworkObjectOnNetworkDespawnTests.

Co-authored-by: Noel Stephens <noel.stephens@unity3d.com>
Co-authored-by: Unity Netcode CI <74025435+netcode-ci-service@users.noreply.github.com>
mollstam pushed a commit to Keepsake-Games/com.unity.netcode.gameobjects that referenced this pull request Feb 13, 2023
…1390)

* fix

Renaming and refactoring DestroyNonSceneObjects to be DespawnNetworkObjects as this is really what it is doing.  The destroy portion is either up to the handler, if an in-scene NetworkObject it lets the unloading of the scene destroy it, and the rest are always destroyed.

* fix

updates to the changes in NetworkSpawnManager

* fix

Adding a "ShutdownInProgress" flag to be able to easily check if we are shutting down.

Adding a check for the NetworkManager.LocalClient since we are despawning in the shutdown method.

Ignoring the de-parenting of NetworkObjects when shutting down (this will have to be discussed a bit).

* fix

Test fix:  I think this was just an unexposed bug in the NetworkRigidBody2D test, but will verify this before making this final.

Fixed some additional minor issues like checking for IsSpawned as opposed to NetworkManager.IsListening and IsSpawned as opposed to NetworkObject.IsSpawned.

* fix

Avoiding spam when despawning and shutting down.
Renaming DespawnNetworkObjects to DespawnAndDestroyNetworkObjects.
If a NetworkObject is not spawned it gets destroyed.

* test fix

Fixing issue with editor tests and the NetworkManager.Singleton instance.

* fix

Adjusting the logic a bit on getting the NetworkObject associated with a NetworkBehaviour.

* revert and fix

After discussing this with Luke, it turned out there was a bug in the NetworkRigidBody2D where it was resetting the isKinematic to true.  This reverts the changes made to the NetworkRigidbody2DDynamicTest and fixes the bug with NetworkRigidBody2D.

* test

Tests that validate OnNetworkDespawn is invoked during shutdown.
This tests server, host, and client relative instances.

* update changelog

update the changelog with the fixed information

* fix

fixing issue with test and recent changes for when the internal shutdown method is invoked.

* refactor

ShutdownInProgress now returns m_ShuttingDown.

Setting m_ShuttingDown to false at the end of ShutdownInternal in case the NetworkManager is going to be re-used.

* style

line endings

* Revert "style"

This reverts commit e2ca11a.

* fix

fixing timing related issue for NetworkObjectOnNetworkDespawnTests.
mollstam pushed a commit to Keepsake-Games/com.unity.netcode.gameobjects that referenced this pull request Feb 13, 2023
…1390) (Unity-Technologies#1460)

* fix

Renaming and refactoring DestroyNonSceneObjects to be DespawnNetworkObjects as this is really what it is doing.  The destroy portion is either up to the handler, if an in-scene NetworkObject it lets the unloading of the scene destroy it, and the rest are always destroyed.

* fix

updates to the changes in NetworkSpawnManager

* fix

Adding a "ShutdownInProgress" flag to be able to easily check if we are shutting down.

Adding a check for the NetworkManager.LocalClient since we are despawning in the shutdown method.

Ignoring the de-parenting of NetworkObjects when shutting down (this will have to be discussed a bit).

* fix

Test fix:  I think this was just an unexposed bug in the NetworkRigidBody2D test, but will verify this before making this final.

Fixed some additional minor issues like checking for IsSpawned as opposed to NetworkManager.IsListening and IsSpawned as opposed to NetworkObject.IsSpawned.

* fix

Avoiding spam when despawning and shutting down.
Renaming DespawnNetworkObjects to DespawnAndDestroyNetworkObjects.
If a NetworkObject is not spawned it gets destroyed.

* test fix

Fixing issue with editor tests and the NetworkManager.Singleton instance.

* fix

Adjusting the logic a bit on getting the NetworkObject associated with a NetworkBehaviour.

* revert and fix

After discussing this with Luke, it turned out there was a bug in the NetworkRigidBody2D where it was resetting the isKinematic to true.  This reverts the changes made to the NetworkRigidbody2DDynamicTest and fixes the bug with NetworkRigidBody2D.

* test

Tests that validate OnNetworkDespawn is invoked during shutdown.
This tests server, host, and client relative instances.

* update changelog

update the changelog with the fixed information

* fix

fixing issue with test and recent changes for when the internal shutdown method is invoked.

* refactor

ShutdownInProgress now returns m_ShuttingDown.

Setting m_ShuttingDown to false at the end of ShutdownInternal in case the NetworkManager is going to be re-used.

* style

line endings

* Revert "style"

This reverts commit e2ca11a.

* fix

fixing timing related issue for NetworkObjectOnNetworkDespawnTests.

Co-authored-by: Noel Stephens <noel.stephens@unity3d.com>
Co-authored-by: Unity Netcode CI <74025435+netcode-ci-service@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants