Consider a Streamlined/Slimmed-down Version 2.0

[csturm83]

Type of issue

[ ] Bug
[ ] Question (e.g. about handling/usage)
[X] Request for new feature/improvement

Expected Behavior

• Modern/lean/minimal .NET QRCoder v2.0 library
• Zero dependencies
• Easy to integrate cross-plat/Docker

Current Behavior

• Dependency on System.Drawing.Common causes cross-plat/Docker issues (libgdiplus)
• System.Drawing not really ideal for server based scenarios
• Locked into legacy design/public API for back-compat reasons
• Some degree of code bloat as code has aged

Possible Solution = QRCoder v2.0

• Cut some legacy cruft (support netstandard2.0+ only)
• Cleanup public API surface (flexibility to consider new designs)
• Remove System.Drawing.Common dependency completely
• Separate Payloads from QRCode generation (separate package maybe?)

Rationale

• Fresh start to target modern .NET platforms/scenarios (netstandard2.0+)
• Core logic/class/project structure can largely be reused
• Different/new designs/optimizations can be explored.
• Several existing renderers already have no dependency on System.Drawing
• Other renderers could potentially be refactored to remove System.Drawing dependency.
  • Does Base64QrCode really need to use Bitmap class? Can it instead work with raw bytes from PngByteQrCode? (not sure, I'm actually asking..)
  • Do simple System.Drawing structs (Size Color) really need to be used?
    • Maybe remove from public signatures (string instead of Color) or maybe roll similar custom structs as an implementation detail?
• Not all consumers need Payloads - maybe separate into separate package (at least more than one file ;) ).

In my mind, it wouldn't be a huge divergence from the current codebase - just mostly stripping out System.Drawing. Which would likely impact the embedded icons feature. Maybe layer a separate package (that depends on System.Drawing) on top of the "Core" package - so consumers can 'light up' features they need?

[PhonicUK]

I actually just ran into some headaches running this on Linux for some of the above reasons. The dependencies on PresentationFramework and PresentationCore prevented me from using it properly on Linux as things would fail trying to resolve those dependencies. I've forked the library ( https://github.com/CubeCoders/QRCoder ) to remove XAML/WPF support which gets it to play nice on Linux. libgdiplus isn't a huge deal, it's in the repos for most modern distributions. I may yet work on producing such a 'pure' build of this without any dependencies.

[PhonicUK]

@csturm83 turns out this was less work than I thought. My fork above has removed all references to System.Drawing and System.Drawing.Common, removes a handful of the renders (Base64QrCode is no-more, but as you noted you can just use the output from one of the other renderers to do that) and I've removed the cruft that allowed it to support .Net 3.5 so it's now the same codebase for .Net 4.5.2/Core (could be easily changed to target Standard instead) - I've not removed all the various random Payloads just yet but they didn't stop me from stripping the library down.

[csturm83]

@PhonicUK I'm curious why you would have issues related to XAML/WPF on Linux using the current version. Were you trying to use a .NET Framework binary/dll directly, or using a PackageReference?

An application running on Linux should be targeting a .NET Standard version of the library, which doesn't include the XAML/WPF code and references. If you're targeting a .NET Standard version of the library and still have an error, that is likely a bug.

Do you have any additional error information for the issues you were experiencing on Linux?

[PhonicUK]

My application targets .Net Framework rather than .Net Core, so Nuget didn't give me the option to use the .Net Standard version which resulted in it trying to resolve assemblies at runtime on Linux that were never going to exist. I was also running into issues because I'm using this library indirectly via an upstream dependency. I also needed to remove all of the System.Drawing references anyway to not rely on libgdiplus on Linux.

[CJPNActual]

FWIW I have a heavily-optimised version of QRCoder which generates raw QR data (which is really all I'm interested in, as drawing images is a separate problem) between 10x (1 data input and low-ECC) and 25x (max-input and high-ECC) faster than baseline QRCoder.

Considering a fork or some such in case anybody needs to generate a huge number of QR codes in a hurry.

[csturm83]

@CJPNActual I'd be curious if your speedup compared to QRCoder is strictly in comparison to its code that does the equivalent (raw data), or if you are including QRCoder rendering in your comparison. Also curious if your fork is netstandard2.0, or if it uses newer runtime features to achieve optimization. Also curious how far have you deviated from the current design (if at all). Could some of the optimizations be easily incorporated into the current code base?

I could see a potential use case where someone might want to throw simple QRCode generation behind a Web API. Maybe returning Base64 encoded strings?

[CJPNActual]

@csturm83

@CJPNActual I'd be curious if your speedup compared to QRCoder is strictly in comparison to its code that does the equivalent (raw data), or if you are including QRCoder rendering in your comparison.

This effort has focused entirely on the performance of QRCodeGenerator.CreateQrCode(string, ECCLevel) as in our usage scenario the generation of a QR image from binary is performed dynamically at vastly different points in time and space (servers in antipodean relation to each other). That said, the generation of an image from a raw QR is relatively trivial. The last time, long ago, I profiled .NETs image/graphics infrastructure it performed rather terribly, yet I understand it to have improved significantly in more recent times. Or, you know, the end user might be crazy and throw large numbers of raw QRs at a GPU and perform the final rendering in that manner. IIRC our production code utilises a custom method for image rendering, so beyond the scope of this optimisation work.

Also curious if your fork is netstandard2.0, or if it uses newer runtime features to achieve optimization. Also curious how far have you deviated from the current design (if at all). Could some of the optimizations be easily incorporated into the current code base?

Observed improvements in performance are achieved only by eliminating vastly-redundant CPU-thrashing along with improvements in algorithmic efficiency and memory use. No reliance on differences in optimisations of the underlying flavour of .NET, or any other such "black box" optimisations in the JITer or whatnot. Indeed I observe only slight improvements in performance between the various sub-flavours of reference-QRCoder, so one wonders whether optimisations implemented in .NET Standard/Core are being retroactively applied to .NET Framework 4.7.2 (really not my field of expertise).

You would instantly recognise the optimised version owing to the fact that the high-level structure is essentially identical to existing QRCoder. As such it is essentially a plug-and-play affair to incorporate in to current code base.

I could see a potential use case where someone might want to throw simple QRCode generation behind a Web API. Maybe returning Base64 encoded strings?

[csturm83]

Considering a fork or some such in case anybody needs to generate a huge number of QR codes in a hurry.

You would instantly recognise the optimised version owing to the fact that the high-level structure is essentially identical to existing QRCoder. As such it is essentially a plug-and-play affair to incorporate in to current code base.

@CJPNActual would you in fact consider sharing your fork (or submitting a PR)? I'm sure the community would be most appreciative 😄 .

[CJPNActual]

@csturm83 By all means, although I'm not yet finished with the project. As of today, alphanumeric V40(H) codes are issuing forth 36x faster.

[csturm83]

As of today, alphanumeric V40(H) codes are issuing forth 36x faster.

@CJPNActual That's awesome!

I'm curious if you had experimented at all with flattening QRCodeGenerator.ModuleMatrix into a single BitArray? Seems like a slight increase in lookup overhead (calculating a row offset) might be worth the decreased allocations and better data locality. I had thought about experimenting with it myself, but just haven't taken the time.

cc @jnyrup who has previously shown interest in performance within this repo.

[CJPNActual]

@csturm83

I'm curious if you had experimented at all with flattening [QRCodeGenerator.ModuleMatrix]

Indeed that's one of the very first strategies adopted, in fact going as far as abandoning BitArrray altogether for intensive operations. Instead I've (re)implemented BitArray's core functionality as follows:

public byte[] compactModuleMatrix;
public QRCodeData(int version)
{
    this.Version = version;
    Size = ModulesPerSideFromVersion(version);
    int compactSize = (int)Math.Floor((Size * Size) / 8f) + ((Size * Size) % 8);
    compactModuleMatrix = new byte[compactSize];
}

Then

public bool this[int Y, int X]
            {
            get {
                if (X >= Size || Y >= Size)
                    throw new Exception("Attempt to access QR module beyond bounds of array.");

                int baseAddress = ((Y * Size) + X) / 8;
                int interByteOffset = ((Y * Size) + X) % 8;
                byte offsetMask = (byte)(128 >> interByteOffset);
                int thisBit = compactModuleMatrix[baseAddress] & offsetMask;
                return thisBit == offsetMask;
            }
            set {
                if (X >= Size || Y >= Size)
                    throw new Exception("Attempt to access QR module beyond bounds of array.");

                int baseAddress = ((Y * Size) + X) / 8;
                int interByteOffset = ((Y * Size) + X) % 8;

                if (value) // true
                    compactModuleMatrix[baseAddress] |= (byte)(128 >> interByteOffset);
                else
                    compactModuleMatrix[baseAddress] &= (byte)~(128 >> interByteOffset);
            }
        }

Which allows:

var ourQRData = new QRCodeData(versionWhatever);
bool ourModuleValue = ourQRData[12, 34];
ourQRData[56, 78] = true;

There are some very good reasons for manually managing the QR's core memory structure further down the line.

P.S. Just cracked 40x faster for V40(H).

[CJPNActual]

Today's update:

68x faster/6.7ms per code for V40(H) and 16x faster/120μs per code for V1(L) on a single thread on an AMD 3900X.

62x faster/7.0ms per code for V40(H) and 15x faster/123μs per code for V1(L) on a single thread on an Intel Xeon E3-1270 v6.

Significant improvements still likely at the V1(L) end, although close to entering the validation phase as it's unlikely there will be further gains without resorting to code-gymnastics with CPU-intrinsics, SIMD or a fundamental reconsideration of the entire structure of QRCoder.

[CJPNActual]

@csturm83

Another update (AMD 3900X):

V1(L): 25x faster @ 76μs/code

V40(H): 77x faster @ 5.85ms/code