Merge pull request #5294 from akkadotnet/dev

Akka.NET v1.4.26 Release

Author: Aaronontheweb

RELEASE_NOTES.md

@@ -1,3 +1,19 @@
+#### 1.4.26 September 28 2021 ####
+**Maintenance Release for Akka.NET 1.4**
+Akka.NET v1.4.26 is a very small release that addresses one wire format regression introduced in Akka.NET v1.4.20.
+
+**Bug Fixes and Improvements**
+* [Akka.Remote / Akka.Persistence: PrimitiveSerializers manifest backwards compatibility problem](https://github.com/akkadotnet/akka.net/issues/5279) - this could cause regressions when upgrading to Akka.NET v1.4.20 and later. We have resolved this issue in Akka.NET v1.4.26. [Please see our Akka.NET v1.4.26 upgrade advisory for details](https://getakka.net/community/whats-new/akkadotnet-v1.4-upgrade-advisories.html#upgrading-to-akkanet-v1426-from-older-versions).
+* [Akka.DistributedData.LightningDb: Revert #5180, switching back to original LightningDB packages](https://github.com/akkadotnet/akka.net/pull/5286)
+
+You can [see the full set of changes introduced in Akka.NET v1.4.26 here](https://github.com/akkadotnet/akka.net/milestone/57?closed=1)
+
+| COMMITS | LOC+ | LOC- | AUTHOR |
+| --- | --- | --- | --- |
+| 4 | 99 | 96 | Gregorius Soedharmo |
+| 3 | 79 | 5 | Aaron Stannard |
+| 1 | 1 | 1 | dependabot[bot] |
+
 #### 1.4.25 September 08 2021 ####
 **Maintenance Release for Akka.NET 1.4**
 Akka.NET v1.4.25 includes some _significant_ performance improvements for Akka.Remote and a number of important bug fixes and improvements.

docs/community/whats-new/akkadotnet-v1.4-upgrade-advisories.md

@@ -0,0 +1,67 @@
+---
+uid: akkadotnet-v14-upgrade-advisories
+title: Akka.NET v1.4 Upgrade Advisories
+---
+
+# Akka.NET v1.4 Upgrade Advisories
+This document contains specific upgrade suggestions, warnings, and notices that you will want to pay attention to when upgrading between versions within the Akka.NET v1.4 roadmap.
+
+## Upgrading to Akka.NET v1.4.20 from Older Versions
+
+> [!NOTE]
+> This is an edge-case issue that only affects users who are sending primitive data types (i.e. `int`, `long`, `string`) directly over Akka.Remote or Akka.Persistence.
+
+Between Akka.NET v1.4.19 and [v1.4.20](https://github.com/akkadotnet/akka.net/releases/tag/1.4.20) we introduced a regression in the wire format of the "primitive" serializer in Akka.NET v1.4.20 - the serializer responsible for transmitting `int`, `long`, and `string` as stand-alone messages in Akka.Remote and Akka.Persistence.
+
+The error message would look like this in clusters that are running a combination of v1.4.20 and any previous versions of Akka.NET:
+
+> `Serializer not defined for message with serializer id [6] and manifest []. Transient association error (association remains live). Cannot find manifest class [S] for serializer with id [17].`
+
+You can see the PR that introduced this regression here: https://github.com/akkadotnet/akka.net/issues/4986
+
+This change was originally introduced to assist with cross-platform wire compatibility between .NET Framework and .NET Core, because Microsoft changed the names of all primitive types between the two runtimes when .NET Core was originally introduced.
+
+**Workarounds**:
+
+To work around this issue, if you're affected by it (most users are not:)
+
+* Upgrade all of the nodes to v1.4.20 or later at once;
+* Or upgrade directly to Akka.NET v1.4.26 or later, which resolves this issue and prevents the regression from occurring.
+
+## Upgrading to Akka.NET v1.4.26 from Older Versions
+
+> [!NOTE]
+> This is an edge-case issue that only affects users who are sending primitive data types (i.e. `int`, `long`, `string`) directly over Akka.Remote or Akka.Persistence.
+
+In Akka.NET v1.4.26 we have introduced a new setting:
+
+```
+akka.actor.serialization-settings.primitive.use-legacy-behavior = on
+```
+
+This setting is set of `on` by default and it resolves the backwards compatibility issue introduced in the "primitives" serializer described in our [v1.4.20 upgrade advisory](#upgrading-to-akkanet-v1420-from-older-versions).
+
+> [!IMPORTANT]
+> If you have: 
+> * Previously upgraded to Akka.NET v1.4.20+ and you have not run into any issues;
+> * You have not yet upgraded to Akka.NET v1.4.20+; and
+> * You _do not_ plan on running both .NET Framework and .NET Core in the same cluster
+> Then you can safely upgrade to v1.4.26 using your normal deployment process. 
+
+If you are running a mixed .NET Core and .NET Framework cluster, see the process below.
+
+### Deploying v1.4.26 into Mixed .NET Core and .NET Framework Environments
+*However*, if you are attempting to run a mixed-mode cluster - i.e. some services running on .NET Framework and some running on .NET Core, you will eventually want to turn this setting to `off` in order to faciliate smooth operation between both platforms.
+
+#### Already Deployed v1.4.20 or Later
+If you've already deployed v1.4.20 and you have not had any issues with the primitives serializer, do the following:
+
+1. Before you upgrade to v1.4.26 or later set `akka.actor.serialization-settings.primitive.use-legacy-behavior = off` - so any future serialization of primitives will be handled correctly in a cross-platform way;
+2. Run your normal deployment process.
+
+#### Have Not Deployed v1.4.20 or Later
+If you have not previously deployed to v1.4.20 or later, then do the following:
+
+1. Deploy once with `akka.actor.serialization-settings.primitive.use-legacy-behavior = on` (the default);
+2. Once all nodes are completely upgraded to v1.4.26 and later, prepare a second deployment with `akka.actor.serialization-settings.primitive.use-legacy-behavior = off`;
+3. Complete the second deployment - you will be fine with all rolling upgrades moving forward.

docs/community/whats-new/toc.yml

@@ -1,2 +1,4 @@
 - name: New in Akka.NET v1.4
-  href: akkadotnet-v1.4.md
+  href: akkadotnet-v1.4.md
+- name: Upgrade Advisories (v1.4)
+  href: akkadotnet-v1.4-upgrade-advisories.md

src/contrib/cluster/Akka.Cluster.Sharding.Tests.MultiNode/Akka.Cluster.Sharding.Tests.MultiNode.csproj

@@ -1,9 +1,6 @@
 <Project Sdk="Microsoft.NET.Sdk">
     <Import Project="..\..\..\common.props" />
 
-    <!-- Needed to resolve https://github.com/akkadotnet/akka.net/pull/5180 and https://github.com/akkadotnet/akka.net/issues/5174 -->
-    <Import Project="../lib-hack/LightningDB.targets" />
-
   <PropertyGroup>
     <AssemblyTitle>Akka.Cluster.Sharding.Tests.MultiNode</AssemblyTitle>
     <TargetFrameworks>$(NetFrameworkTestVersion);$(NetTestVersion);$(NetCoreTestVersion)</TargetFrameworks>

src/contrib/cluster/Akka.DistributedData.LightningDB/Akka.DistributedData.LightningDB.csproj

@@ -4,7 +4,7 @@
   <PropertyGroup>
     <AssemblyTitle>Akka.DistributedData.LightningDB</AssemblyTitle>
     <Description>Replicated data using CRDT structures</Description>
-    <TargetFrameworks>$(NetStandardLibVersion)</TargetFrameworks>
+    <TargetFramework>$(NetStandardLibVersion)</TargetFramework>
     <PackageTags>$(AkkaPackageTags);network;cluster;crdt;replication;lightningdb;lmdb</PackageTags>
     <GenerateDocumentationFile>true</GenerateDocumentationFile>
   </PropertyGroup>
@@ -15,7 +15,7 @@
   </ItemGroup>
 
   <ItemGroup>
-    <PackageReference Include="LightningDB.Vendored.Akka" Version="0.14.0" />
+    <PackageReference Include="LightningDB" Version="0.14.0" />
   </ItemGroup>
 
   <PropertyGroup Condition=" '$(Configuration)' == 'Release' ">

src/contrib/cluster/Akka.DistributedData.Tests.MultiNode/Akka.DistributedData.Tests.MultiNode.csproj

@@ -1,9 +1,6 @@
 <Project Sdk="Microsoft.NET.Sdk">
     <Import Project="..\..\..\common.props" />
 
-    <!-- Needed to resolve https://github.com/akkadotnet/akka.net/pull/5180 and https://github.com/akkadotnet/akka.net/issues/5174 -->
-    <Import Project="../lib-hack/LightningDB.targets" />
-  
   <PropertyGroup>
     <AssemblyTitle>Akka.DistributedData.Tests.MultiNode</AssemblyTitle>
     <TargetFrameworks>$(NetFrameworkTestVersion);$(NetTestVersion);$(NetCoreTestVersion)</TargetFrameworks>

src/contrib/cluster/lib-hack/LightningDB.targets

@@ -16,7 +16,7 @@
   </ItemGroup>
 
   <ItemGroup>
-    <PackageReference Include="Microsoft.Data.SQLite" Version="5.0.9" />
+    <PackageReference Include="Microsoft.Data.SQLite" Version="5.0.10" />
   </ItemGroup>
 
   <PropertyGroup Condition=" '$(Configuration)' == 'Release' ">

src/contrib/cluster/lib-hack/runtimes/linux-arm/native/liblmdb.so

@@ -440,7 +440,7 @@ namespace Akka.Remote.Serialization
     }
     public sealed class PrimitiveSerializers : Akka.Serialization.SerializerWithStringManifest
     {
-        public PrimitiveSerializers(Akka.Actor.ExtendedActorSystem system) { }
+        public PrimitiveSerializers(Akka.Actor.ExtendedActorSystem system, Akka.Configuration.Config config) { }
         public override object FromBinary(byte[] bytes, string manifest) { }
         public override string Manifest(object obj) { }
         public override byte[] ToBinary(object obj) { }

src/contrib/cluster/lib-hack/runtimes/linux-arm64/native/liblmdb.so

@@ -9,6 +9,7 @@
 using System.Collections.Generic;
 using System.Linq;
 using System.Reflection;
+using Akka.Remote.TestKit;
 using Akka.TestKit;
 using Xunit;
 using FluentAssertions;
@@ -62,6 +63,7 @@ public void Only_MultiNodeConfig_role_count_used()
 
         private static Dictionary<string, List<NodeTest>> DiscoverSpecs()
         {
+            Environment.SetEnvironmentVariable(MultiNodeFactAttribute.MultiNodeTestEnvironmentName, "1");
             using (var controller = new XunitFrontController(AppDomainSupport.IfAvailable, new System.Uri(typeof(DiscoveryCases).GetTypeInfo().Assembly.CodeBase).LocalPath))	
             {
                 using (var discovery = new Discovery())

src/contrib/cluster/lib-hack/runtimes/linux-x64/native/liblmdb.so

@@ -135,8 +135,8 @@ class Program
         /// </summary>
         static void Main(string[] args)
         {
-            // Force load the args
-            CommandLine.GetPropertyOrDefault("force load", null);
+            CommandLine.Initialize(args);
+            
             if (CommandLine.ShowHelp)
             {
                 PrintHelp();
@@ -242,6 +242,7 @@ static void Main(string[] args)
             }
 #endif
 
+            Environment.SetEnvironmentVariable(MultiNodeFactAttribute.MultiNodeTestEnvironmentName, "1");
             using (var controller = new XunitFrontController(AppDomainSupport.IfAvailable, assemblyPath))
             {
                 using (var discovery = new Discovery())

src/contrib/cluster/lib-hack/runtimes/osx/native/lmdb.dylib

@@ -36,6 +36,8 @@ class Program
 
         static int Main(string[] args)
         {
+            CommandLine.Initialize(args);
+            
             var nodeIndex = CommandLine.GetInt32("multinode.index");
             var nodeRole = CommandLine.GetProperty("multinode.role");
             var assemblyFileName = CommandLine.GetProperty("multinode.test-assembly");
@@ -64,6 +66,7 @@ static int Main(string[] args)
 #endif
 
             Thread.Sleep(TimeSpan.FromSeconds(10));
+            Environment.SetEnvironmentVariable(MultiNodeFactAttribute.MultiNodeTestEnvironmentName, "1");
             using (var controller = new XunitFrontController(AppDomainSupport.IfAvailable, assemblyFileName))
             {
                 /* need to pass in just the assembly name to Discovery, not the full path

src/contrib/cluster/lib-hack/runtimes/win-x64/native/lmdb.dll

@@ -26,17 +26,19 @@ namespace Akka.Remote.TestKit
     ///     var testName = CommandLine.GetProperty("multinode.test-method");
     /// </code>
     /// </summary>
-    public class CommandLine
+    public static class CommandLine
     {
-        private static readonly StringDictionary Values;
+        private static readonly StringDictionary Values = new StringDictionary();
 
         static CommandLine()
         {
-            Values = new StringDictionary();
+            Initialize(Environment.GetCommandLineArgs());
+        }
 
+        public static void Initialize(string[] args)
+        {
             // Detect and fix PowerShell command line input.
             // PowerShell splits command line arguments on '.'
-            var args = Environment.GetCommandLineArgs();
             var fixedArgs = new List<string>();
             for (var i = 1; i < args.Length - 1; ++i)
             {
@@ -71,7 +73,7 @@ static CommandLine()
 
                 if (tokens.Length == 2)
                 {
-                    Values.Add(tokens[0], tokens[1]);
+                    Values[tokens[0]] = tokens[1].Trim().Trim('"');
                 }
                 else
                 {

src/contrib/cluster/lib-hack/runtimes/win-x64/native/lmdbautoresize.dll

@@ -16,23 +16,17 @@ public class MultiNodeFactAttribute : FactAttribute
         /// Set by MultiNodeTestRunner when running multi-node tests
         /// </summary>
         public const string MultiNodeTestEnvironmentName = "__AKKA_MULTI_NODE_ENVIRONMENT";
-        
-        public static Lazy<bool> ExecutedByMultiNodeRunner =
-            new Lazy<bool>(() =>
-            {
-                var args = Environment.GetCommandLineArgs();
-                if (args.Length == 0) return false;
-                var firstArg = args[0];
-                return firstArg.Contains("Akka.MultiNodeTestRunner") 
-                    || firstArg.Contains("Akka.NodeTestRunner")
-                    || Environment.GetEnvironmentVariable(MultiNodeTestEnvironmentName) != null;
-            });
+
+        private bool? _executedByMultiNodeRunner;
 
         public override string Skip
         {
             get
             {
-                return ExecutedByMultiNodeRunner.Value
+                if (_executedByMultiNodeRunner == null)
+                    _executedByMultiNodeRunner =
+                        Environment.GetEnvironmentVariable(MultiNodeTestEnvironmentName) != null;
+                return _executedByMultiNodeRunner != null && _executedByMultiNodeRunner.Value
                     ? base.Skip
                     : "Must be executed by multi-node test runner";
             }

src/contrib/cluster/lib-hack/runtimes/win-x86/native/lmdb.dll

@@ -176,6 +176,14 @@ public void Remoting_should_contain_correct_BatchWriter_settings_in_ReferenceCon
             s.BatchWriterSettings.EnableBatching.Should().BeTrue();
             s.BatchWriterSettings.MaxExplicitFlushes.Should().Be(BatchWriterSettings.DefaultMaxPendingWrites);
         }
+        
+        [Fact]
+        public void Remoting_should_contain_correct_PrimitiveSerializer_settings_in_ReferenceConf()
+        {
+            var c = RARP.For(Sys).Provider.RemoteSettings.Config.GetConfig("akka.actor.serialization-settings.primitive");
+            c.Should().NotBeNull();
+            c.GetBoolean("use-legacy-behavior").Should().BeTrue();
+        }
    }
 }
 

src/contrib/cluster/lib-hack/runtimes/win-x86/native/lmdbautoresize.dll

@@ -0,0 +1,43 @@
+// //-----------------------------------------------------------------------
+// // <copyright file="BugFix5279Spec.cs" company="Akka.NET Project">
+// //     Copyright (C) 2009-2021 Lightbend Inc. <http://www.lightbend.com>
+// //     Copyright (C) 2013-2021 .NET Foundation <https://github.com/akkadotnet/akka.net>
+// // </copyright>
+// //-----------------------------------------------------------------------
+
+using Akka.Actor;
+using Akka.Configuration;
+using Akka.Remote.Serialization;
+using Akka.TestKit;
+using Akka.Util;
+using Xunit;
+using FluentAssertions;
+
+namespace Akka.Remote.Tests.Serialization
+{
+    public class BugFix5279Spec: AkkaSpec
+    {
+        [Theory]
+        [InlineData(1, "I")]
+        [InlineData(1L, "L")]
+        [InlineData("1", "S")]
+        public void PrimitiveSerializer_without_useNeutralPrimitives_should_return_custom_manifest(object data, string manifest)
+        {
+            var config = ConfigurationFactory.ParseString("use-legacy-behavior = off");
+            var serializer = new PrimitiveSerializers((ExtendedActorSystem)Sys, config);
+            serializer.Manifest(data).Should().Be(manifest);
+        }
+        
+        [Theory]
+        [InlineData(1)]
+        [InlineData(1L)]
+        [InlineData("1")]
+        public void PrimitiveSerializer_without_useNeutralPrimitives_should_return_type_manifest(object data)
+        {
+            var config = ConfigurationFactory.ParseString("use-legacy-behavior = on");
+            var serializer = new PrimitiveSerializers((ExtendedActorSystem)Sys, config);
+            serializer.Manifest(data).Should().Be(data.GetType().TypeQualifiedName());
+        }
+        
+    }
+}