Merge branch 'dev'

Author: Arkatufus

RELEASE_NOTES.md

@@ -1,3 +1,27 @@
+#### 1.4.24 August 17 2021 ####
+**Maintenance Release for Akka.NET 1.4**
+
+**Bug Fixes and Improvements**
+
+* [Akka: Make `Router` open to extensions](https://github.com/akkadotnet/akka.net/pull/5201)
+* [Akka: Allow null response to `Ask<T>`](https://github.com/akkadotnet/akka.net/pull/5205)
+* [Akka.Cluster: Fix cluster startup race condition](https://github.com/akkadotnet/akka.net/pull/5185)
+* [Akka.Streams: Fix RestartFlow bug](https://github.com/akkadotnet/akka.net/pull/5181)
+* [Akka.Persistence.Sql: Implement TimestampProvider in BatchingSqlJournal](https://github.com/akkadotnet/akka.net/pull/5192)
+* [Akka.Serialization.Hyperion: Bump Hyperion version from 0.11.0 to 0.11.1](https://github.com/akkadotnet/akka.net/pull/5206)
+* [Akka.Serialization.Hyperion: Add Hyperion unsafe type filtering security feature](https://github.com/akkadotnet/akka.net/pull/5208)
+
+You can [see the full set of changes introduced in Akka.NET v1.4.24 here](https://github.com/akkadotnet/akka.net/milestone/55?closed=1)
+
+| COMMITS | LOC+ | LOC- | AUTHOR |
+| --- | --- | --- | --- |
+| 5 | 360 | 200 | Aaron Stannard |
+| 3 | 4 | 4 | dependabot[bot] |
+| 1 | 548 | 333 | Arjen Smits |
+| 1 | 42 | 19 | Martijn Schoemaker |
+| 1 | 26 | 27 | Andreas Dirnberger |
+| 1 | 171 | 27 | Gregorius Soedharmo |
+
 #### 1.4.23 August 09 2021 ####
 **Maintenance Release for Akka.NET 1.4**
 

docs/articles/networking/serialization.md

@@ -304,6 +304,57 @@ akka {
 }
 ```
 
+## Danger of polymorphic serializer
+One of the danger of polymorphic serializers is the danger of unsafe object type injection into 
+the serialization-deserialization chain. This issue applies to any type of polymorphic serializer,
+including JSON, BinaryFormatter, etc. In Akka, this issue primarily affects developers who allow third parties to pass messages directly 
+to unsecured Akka.Remote endpoints, a [practice that we do not encourage](https://getakka.net/articles/remoting/security.html#akkaremote-with-virtual-private-networks).
+
+Generally, there are two approaches you can take to alleviate this problem:
+1. Implement a schema-based serialization that are contract bound, which is more expensive to setup at first but fundamentally faster and more secure.
+2. Implement a filtering or blacklist to block dangerous types.
+
+An example of using a schema-based serialization in Akka can be read under the title "Using Google 
+Protocol Buffers to Version State and Messages" in [this documentation](https://petabridge.com/cluster/lesson3)
+
+Hyperion chose to implement the second approach by blacklisting a set of potentially dangerous types 
+from being deserialized:
+
+- System.Security.Claims.ClaimsIdentity
+- System.Windows.Forms.AxHost.State
+- System.Windows.Data.ObjectDataProvider
+- System.Management.Automation.PSObject
+- System.Web.Security.RolePrincipal
+- System.IdentityModel.Tokens.SessionSecurityToken
+- SessionViewStateHistoryItem
+- TextFormattingRunProperties
+- ToolboxItemContainer
+- System.Security.Principal.WindowsClaimsIdentity
+- System.Security.Principal.WindowsIdentity
+- System.Security.Principal.WindowsPrincipal
+- System.CodeDom.Compiler.TempFileCollection
+- System.IO.FileSystemInfo
+- System.Activities.Presentation.WorkflowDesigner
+- System.Windows.ResourceDictionary
+- System.Windows.Forms.BindingSource
+- Microsoft.Exchange.Management.SystemManager.WinForms.ExchangeSettingsProvider
+- System.Diagnostics.Process
+- System.Management.IWbemClassObjectFreeThreaded
+
+Be warned that these class can be used as a man in the middle attack vector, but if you need 
+to serialize one of these class, you can turn off this feature using this inside your HOCON settings:
+```
+akka.actor.serialization-settings.hyperion.disallow-unsafe-type = false
+```
+
+> [!IMPORTANT]
+> This feature is turned on as default since Akka.NET v1.4.24
+
+> [!WARNING]
+> Hyperion is __NOT__ designed as a safe serializer to be used in an open network as a client-server 
+> communication protocol, instead it is designed to be used as a server-server communication protocol, 
+> preferably inside a closed network system.
+
 ## Cross platform serialization compatibility in Hyperion
 There are problems that can arise when migrating from old .NET Framework to the new .NET Core standard, mainly because of breaking namespace and assembly name changes between these platforms.
 Hyperion implements a generic way of addressing this issue by transforming the names of these incompatible names during deserialization.

src/benchmark/Akka.Benchmarks/Akka.Benchmarks.csproj

@@ -7,7 +7,7 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="BenchmarkDotNet" Version="0.13.0" />
+    <PackageReference Include="BenchmarkDotNet" Version="0.13.1" />
     <PackageReference Include="Newtonsoft.Json" Version="$(NewtonsoftJsonVersion)" />
     <PackageReference Include="System.Collections.Immutable" Version="5.0.0" />
     <!-- FluentAssertions is used in some benchmarks to validate internal behaviors -->

src/benchmark/SerializationBenchmarks/SerializationBenchmarks.csproj

@@ -6,7 +6,7 @@
     </PropertyGroup>
 
     <ItemGroup>
-      <PackageReference Include="BenchmarkDotNet" Version="0.13.0" />
+      <PackageReference Include="BenchmarkDotNet" Version="0.13.1" />
     </ItemGroup>
 
     <ItemGroup>

src/common.props

@@ -11,7 +11,7 @@
   <PropertyGroup>
     <XunitVersion>2.4.1</XunitVersion>
     <TestSdkVersion>16.10.0</TestSdkVersion>
-    <HyperionVersion>0.11.0</HyperionVersion>
+    <HyperionVersion>0.11.1</HyperionVersion>
     <NewtonsoftJsonVersion>[12.0.3,)</NewtonsoftJsonVersion>
     <NBenchVersion>2.0.1</NBenchVersion>
     <ProtobufVersion>3.17.3</ProtobufVersion>

src/contrib/cluster/Akka.Cluster.Tools/Singleton/ClusterSingletonManager.cs

@@ -907,7 +907,7 @@ private void InitializeFSM()
                 if (e.FsmEvent is OldestChangedBuffer.OldestChanged oldestChanged && e.StateData is YoungerData youngerData)
                 {
                     _oldestChangedReceived = true;
-                    if (oldestChanged.Oldest.Equals(_selfUniqueAddress))
+                    if (oldestChanged.Oldest != null && oldestChanged.Oldest.Equals(_selfUniqueAddress))
                     {
                         Log.Info("Younger observed OldestChanged: [{0} -> myself]", youngerData.Oldest.Head()?.Address);
 

src/contrib/persistence/Akka.Persistence.Sql.Common/Journal/BatchingSqlJournal.cs

@@ -22,6 +22,7 @@
 using Akka.Event;
 using Akka.Pattern;
 using Akka.Persistence.Journal;
+using Akka.Persistence.Sql.Common.Journal;
 using Akka.Serialization;
 using Akka.Util;
 using Akka.Util.Internal;
@@ -233,7 +234,12 @@ public abstract class BatchingSqlJournalSetup
         /// <summary>
         /// The default serializer used when not type override matching is found
         /// </summary>
-        public string DefaultSerializer { get; } 
+        public string DefaultSerializer { get; }
+
+        /// <summary>
+        /// The fully qualified name of the type that should be used as timestamp provider.
+        /// </summary>
+        public string TimestampProviderTypeName { get; }
 
         /// <summary>
         /// Initializes a new instance of the <see cref="BatchingSqlJournalSetup" /> class.
@@ -293,6 +299,7 @@ protected BatchingSqlJournalSetup(Config config, QueryConfiguration namingConven
             ReplayFilterSettings = new ReplayFilterSettings(config.GetConfig("replay-filter"));
             NamingConventions = namingConventions;
             DefaultSerializer = config.GetString("serializer", null);
+            TimestampProviderTypeName = config.GetString("timestamp-provider", null);
         }
 
         /// <summary>
@@ -528,6 +535,11 @@ public RequestChunk(int chunkId, IJournalRequest[] requests)
         /// </summary>
         protected readonly bool CanPublish;
 
+        /// <summary>
+        /// The timestamp provider that will be used for the timestamp column when writing messages to the database.
+        /// </summary>
+        protected ITimestampProvider TimestampProvider { get; }
+
         /// <summary>
         /// Logging adapter for current journal actor .
         /// </summary>
@@ -562,7 +574,8 @@ public RequestChunk(int chunkId, IJournalRequest[] requests)
         protected BatchingSqlJournal(BatchingSqlJournalSetup setup)
         {
             Setup = setup;
-            CanPublish = Persistence.Instance.Apply(Context.System).Settings.Internal.PublishPluginCommands;
+            CanPublish = Persistence.Instance.Apply(Context.System).Settings.Internal.PublishPluginCommands;      
+            TimestampProvider = TimestampProviderProvider.GetTimestampProvider(setup.TimestampProviderTypeName, Context);      
 
             _persistenceIdSubscribers = new Dictionary<string, HashSet<IActorRef>>();
             _tagSubscribers = new Dictionary<string, HashSet<IActorRef>>();
@@ -1286,7 +1299,7 @@ private async Task<WriteMessagesResult> HandleWriteMessages(WriteMessages req, T
                         persistent = persistent.WithPayload(tagged.Payload);
                     }
 
-                    WriteEvent(command, persistent.WithTimestamp(DateTime.UtcNow.Ticks), tagBuilder.ToString());
+                    WriteEvent(command, persistent.WithTimestamp(TimestampProvider.GenerateTimestamp(persistent)), tagBuilder.ToString());
 
                     await command.ExecuteNonQueryAsync();
 

src/contrib/persistence/Akka.Persistence.Sql.Common/Journal/ITimestampProvider.cs

@@ -6,6 +6,7 @@
 //-----------------------------------------------------------------------
 
 using System;
+using Akka.Actor;
 
 namespace Akka.Persistence.Sql.Common.Journal
 {
@@ -34,4 +35,21 @@ public sealed class DefaultTimestampProvider : ITimestampProvider
         /// <returns>TBD</returns>
         public long GenerateTimestamp(IPersistentRepresentation message) => DateTime.UtcNow.Ticks;
     }
+
+    public static class TimestampProviderProvider 
+    {
+        public static ITimestampProvider GetTimestampProvider(string typeName, IActorContext context)
+        {
+            if (string.IsNullOrEmpty(typeName))
+            {
+                return new DefaultTimestampProvider();
+            }
+
+            var type = Type.GetType(typeName, true);
+            var withSystem = type.GetConstructor(new[] { context.System.GetType() }) != null;
+            return withSystem ?
+                (ITimestampProvider)Activator.CreateInstance(type, context.System) :
+                (ITimestampProvider)Activator.CreateInstance(type);
+        }
+    }
 }

src/contrib/persistence/Akka.Persistence.Sql.Common/Journal/SqlJournal.cs

@@ -514,21 +514,7 @@ protected virtual string GetConnectionString()
             return connectionString;
         }
 
-        #region obsoleted
-
-        /// <summary>
-        /// TBD
-        /// </summary>
-        /// <param name="typeName">TBD</param>
-        /// <returns>TBD</returns>
-        protected ITimestampProvider GetTimestampProvider(string typeName)
-        {
-            var type = Type.GetType(typeName, true);
-            var withSystem = type.GetConstructor(new[] { Context.System.GetType() }) != null;
-            return withSystem ?
-                (ITimestampProvider)Activator.CreateInstance(type, Context.System) :
-                (ITimestampProvider)Activator.CreateInstance(type);
-        }
-        #endregion
+        protected ITimestampProvider GetTimestampProvider(string typeName) =>
+            TimestampProviderProvider.GetTimestampProvider(typeName, Context);
     }
 }

src/contrib/persistence/Akka.Persistence.Sqlite/Akka.Persistence.Sqlite.csproj

@@ -16,7 +16,7 @@
   </ItemGroup>
 
   <ItemGroup>
-    <PackageReference Include="Microsoft.Data.SQLite" Version="5.0.8" />
+    <PackageReference Include="Microsoft.Data.SQLite" Version="5.0.9" />
   </ItemGroup>
 
   <PropertyGroup Condition=" '$(Configuration)' == 'Release' ">

src/contrib/serializers/Akka.Serialization.Hyperion.Tests/HyperionConfigTests.cs

@@ -35,6 +35,7 @@ public void Hyperion_serializer_should_have_correct_defaults()
                 Assert.True(serializer.Settings.VersionTolerance);
                 Assert.True(serializer.Settings.PreserveObjectReferences);
                 Assert.Equal("NoKnownTypes", serializer.Settings.KnownTypesProvider.Name);
+                Assert.True(serializer.Settings.DisallowUnsafeType);
             }
         }
 
@@ -50,6 +51,7 @@ public void Hyperion_serializer_should_allow_to_setup_custom_flags()
                     serialization-settings.hyperion {
                         preserve-object-references = false
                         version-tolerance = false
+                        disallow-unsafe-type = false
                     }
                 }
             ");
@@ -59,6 +61,7 @@ public void Hyperion_serializer_should_allow_to_setup_custom_flags()
                 Assert.False(serializer.Settings.VersionTolerance);
                 Assert.False(serializer.Settings.PreserveObjectReferences);
                 Assert.Equal("NoKnownTypes", serializer.Settings.KnownTypesProvider.Name);
+                Assert.False(serializer.Settings.DisallowUnsafeType);
             }
         }
 
@@ -82,6 +85,7 @@ public void Hyperion_serializer_should_allow_to_setup_custom_types_provider_with
                 Assert.True(serializer.Settings.VersionTolerance);
                 Assert.True(serializer.Settings.PreserveObjectReferences);
                 Assert.Equal(typeof(DummyTypesProviderWithDefaultCtor), serializer.Settings.KnownTypesProvider);
+                Assert.True(serializer.Settings.DisallowUnsafeType);
             }
         }
 
@@ -105,6 +109,7 @@ public void Hyperion_serializer_should_allow_to_setup_custom_types_provider_with
                 Assert.True(serializer.Settings.VersionTolerance);
                 Assert.True(serializer.Settings.PreserveObjectReferences);
                 Assert.Equal(typeof(DummyTypesProvider), serializer.Settings.KnownTypesProvider);
+                Assert.True(serializer.Settings.DisallowUnsafeType);
             }
         }
 

src/contrib/serializers/Akka.Serialization.Hyperion.Tests/HyperionSerializerSetupSpec.cs

@@ -1,6 +1,12 @@
 using System;
 using System.Collections.Generic;
+using System.Diagnostics;
+using System.IO;
 using System.Linq;
+using System.Runtime.InteropServices;
+using System.Runtime.Serialization;
+using System.Security.Claims;
+using System.Security.Principal;
 using System.Text;
 using System.Threading.Tasks;
 using Akka.Actor;
@@ -36,14 +42,16 @@ public void Setup_should_be_converted_to_settings_correctly()
         {
             var setup = HyperionSerializerSetup.Empty
                 .WithPreserveObjectReference(true)
-                .WithKnownTypeProvider<NoKnownTypes>();
+                .WithKnownTypeProvider<NoKnownTypes>()
+                .WithDisallowUnsafeType(false);
             var settings =
                 new HyperionSerializerSettings(
                     false, 
                     false, 
                     typeof(DummyTypesProvider), 
                     new Func<string, string>[] { s => $"{s}.." },
-                    new Surrogate[0]);
+                    new Surrogate[0],
+                    true);
             var appliedSettings = setup.ApplySettings(settings);
 
             appliedSettings.PreserveObjectReferences.Should().BeTrue(); // overriden
@@ -52,6 +60,7 @@ public void Setup_should_be_converted_to_settings_correctly()
             appliedSettings.PackageNameOverrides.Count().Should().Be(1); // from settings
             appliedSettings.PackageNameOverrides.First()("a").Should().Be("a..");
             appliedSettings.Surrogates.ToList().Count.Should().Be(0); // from settings
+            appliedSettings.DisallowUnsafeType.ShouldBe(false); // overriden
         }
 
         [Fact]
@@ -115,5 +124,31 @@ public void Setup_surrogate_should_work()
             surrogated.Count.Should().Be(1);
             surrogated[0].Should().BeEquivalentTo(expected);
         }
+
+        [Theory]
+        [MemberData(nameof(DangerousObjectFactory))]
+        public void Setup_disallow_unsafe_type_should_work(object dangerousObject, Type type)
+        {
+            var serializer = new HyperionSerializer((ExtendedActorSystem)Sys, HyperionSerializerSettings.Default);
+            var serialized = serializer.ToBinary(dangerousObject);
+            serializer.Invoking(s => s.FromBinary(serialized, type)).Should().Throw<SerializationException>();
+        }
+
+        public static IEnumerable<object[]> DangerousObjectFactory()
+        {
+            var isWindow = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
+            
+            yield return new object[]{ new FileInfo("C:\\Windows\\System32"), typeof(FileInfo) };
+            yield return new object[]{ new ClaimsIdentity(), typeof(ClaimsIdentity)};
+            if (isWindow)
+            {
+                yield return new object[]{ WindowsIdentity.GetAnonymous(), typeof(WindowsIdentity) };
+                yield return new object[]{ new WindowsPrincipal(WindowsIdentity.GetAnonymous()), typeof(WindowsPrincipal)};
+            }
+#if NET471
+            yield return new object[]{ new Process(), typeof(Process)};
+#endif
+            yield return new object[]{ new ClaimsIdentity(), typeof(ClaimsIdentity)};
+        }
     }
 }