Python and Skia don't mix

[peardox]

I'm not positive about the exact criteria yet

Basically imports with Skia fully enabled causes Skia to hit problem with floating point

Take anything with an import and add this to your dproj (after installing Skia of course) and everything will go badly wrong.

Masking the FPU exception doesn't seem to help

As I don't know which package is causing the issue this is posted here

uses
  Skia, Skia.FMX, , ....

begin
  GlobalUseSkia := True;
  GlobalUseSkiaRasterWhenAvailable := False;
.
.

[viniciusfbb]

Hello,
The way Python4delphi is changing exception masks is not secure. Care is needed mainly to remove exceptions from the mask.
It's something that affects the entire application and other libraries like Skia, OpenGL, ActiveX, etc...

The problem is specifically in the MaskFPUExceptions function in PythonEngine.pas, which is used in the initialization of P4D-Data- Sciences

There are a few ways to resolve this. One of them would be to create a class to store the mask's state, change it, so that after a try finally you can restore the previous mask.

[peardox]

@viniciusfbb Thanks for the rapid reply and explanation. I sorta thought this was most likely MaskFPUExceptions (hence the issue being tentatively in this repo)

I wrote a replacement for MaskFPUExceptions that stores the current GetExceptionMask when called with True in a global then replaces it when called with false. This worked a treat and Skia now works perfectly again (Skia was really acting up without this fix...).

@lmbelo While my hack works I'm not submitting a PR for this one as this is a complicated issue which is situational - my 'fix' works for my app but other packages may well mess with the Exception Mask so a more robust solution is really required.

For reference here's what I did - you can enable/disable it via the define (good for testing)...

{$DEFINE USESAFEMASK}

var
  FPUMASK: TArithmeticExceptionMask;

procedure SafeMaskFPUExceptions(ExceptionsMasked : boolean;
  MatchPythonPrecision : Boolean = True);

implementation

uses
  Maths;

{$R *.fmx}

procedure SafeMaskFPUExceptions(ExceptionsMasked : boolean;
  MatchPythonPrecision : Boolean);
begin
  {$IFDEF USESAFEMASK}
  {$IF Defined(CPUX86) or Defined(CPUX64)}
  if ExceptionsMasked then
    begin
    FPUMASK := GetExceptionMask;
    SetExceptionMask([exInvalidOp, exDenormalized, exZeroDivide,
      exOverflow, exUnderflow, exPrecision]);
    end
  else
    SetExceptionMask(FPUMASK);
  {$WARN SYMBOL_PLATFORM OFF}
  {$IF Defined(FPC) or Defined(MSWINDOWS)}
  if MatchPythonPrecision then
      SetPrecisionMode(pmDouble)
    else
      SetPrecisionMode(pmExtended);
  {$WARN SYMBOL_PLATFORM ON}
  {$IFEND}
  {$IFEND}
  {$ELSE}
    MaskFPUExceptions(ExceptionsMasked, MatchPythonPrecision);
  {$IFEND}
end;

[lmbelo]

The exception mask is undone after the importation of any of the Python packages. If you have a look at the exception mask routine you'll see that the exception mask is being undone (for CPUX86 and CPUX64) to Delphi's default.
If we're changing exception mask in somewhere out of modules importation, we should be restoring to the defaults when we're done.
Can you show specifically where it is breaking down? Which platform are you targeting to?
Hope I can be of any help.

[peardox]

Ahh, that explains why using afterimport to reset the mask didn't work...

I tried two methods...

import(packagea)
import(packageb)
import(packagec)

with each of those using shared beforeinstall, afterinstall, beforeimport, afterimport which calls my SafeMaskFPUExceptions() function - this didn't work :(

So, I've not got

SafeMaskFPUExceptions(True)
import(packagea)
SafeMaskFPUExceptions(False)

See https://github.com/peardox/Lartis/blob/main/PythonSystem.pas#L501

[lmbelo]

You should decorate it by a try finally block.

MaskFPUExceptions(true);
try
  //do whatever you want to do
finally
  MaskFPUExceptions(false);
end;

[peardox]

I'm going to - this was a recent test, still unfinished - no proper error handlers yet

I just discovered that I need to handle errors properly anyway (Linux screws up horribly ATM - Issue coming soon...)

[lmbelo]

You need to get extra attention when targeting installed Python versions on Linux. You might hit across known issues reported before here. Have a look in all closed issues when you're going afters Linux errors.
Only for the sake of getting you ahead, you can't use apt installed Python working with the DelphiFMX library.
We need a Python executable linked statically or dynamically with the interpreter shared library, or an executable that exposes interpreter's symbols some how. The is not the case of Python available by apt.

[peardox]

I was using an Embedded Python 3.9 - which is why the issue is listed here

[lmbelo]

Our embeddables should work fine on any of the provided platforms.

[viniciusfbb]

@lmbelo, you didn't understand my post. The P4D-Data-Sciences code is wrong and python4delphi's MaskFPUExceptions implementation should be changed as well. I will explain it better.

MaskFPUExceptions code from python4delphi:

python4delphi/Source/PythonEngine.pas

Lines 9288 to 9306 in 19192ad

	procedure MaskFPUExceptions(ExceptionsMasked : boolean;
	MatchPythonPrecision : Boolean);
	begin
	{$IF Defined(CPUX86) or Defined(CPUX64)}
	if ExceptionsMasked then
	SetExceptionMask([exInvalidOp, exDenormalized, exZeroDivide,
	exOverflow, exUnderflow, exPrecision])
	else
	SetExceptionMask([exDenormalized, exUnderflow, exPrecision]);
	{$WARN SYMBOL_PLATFORM OFF}
	{$IF Defined(FPC) or Defined(MSWINDOWS)}
	if MatchPythonPrecision then
	SetPrecisionMode(pmDouble)
	else
	SetPrecisionMode(pmExtended);
	{$WARN SYMBOL_PLATFORM ON}
	{$IFEND}
	{$IFEND}
	end;

Note that the code above arbitrarily changes the exception mask to:

[exInvalidOp, exDenormalized, exZeroDivide, exOverflow, exUnderflow, exPrecision] when the ExceptionsMasked parameter is True
[exDenormalized, exUnderflow, exPrecision] when the ExceptionsMasked parameter is False

Now, look at one of the P4D-Data-Sciences import code:

procedure TTorchVision.ImportModule;
begin
  MaskFPUExceptions(true);
  try
    inherited;
  finally
    MaskFPUExceptions(false);
  end;
end;

As it stands, if the exception mask before the code above is empty [], after import it will be [exDenormalized, exUnderflow, exPrecision]. Likewise, if the exception mask is exAllArithmeticExceptions (all exceptions), which is what happens when using OpenGL, D2D, GDIP, Skia, among others, after import the mask will be [exDenormalized, exUnderflow, exPrecision]. And there is no way to predict the mask that will be configured in the application when it arrives at the P4D-Data-Sciences import code, there are many variants, there are several other libraries that change like the Delphi XML lib itself, and it can even vary according to the application (VCL or FMX).

So, the correct thing is: save the mask value, change it and after importing the P4D-Data-Sciences, restore the previous mask value and not arbitrarily change it to [exDenormalized, exUnderflow, exPrecision].

I made a possible fix for both codes. First for python4delphi's MaskFPUExceptions:

procedure MaskFPUExceptions(ExceptionsMasked: Boolean;
  var StoredMask: TArithmeticExceptionMask; MatchPythonPrecision: Boolean);
begin
  {$IF Defined(CPUX86) or Defined(CPUX64)}
  if ExceptionsMasked then
    StoredMask := SetExceptionMask(GetExceptionMask + [exInvalidOp, exDenormalized, exZeroDivide,
      exOverflow, exUnderflow, exPrecision])
  else
    SetExceptionMask(StoredMask + [exDenormalized, exUnderflow, exPrecision]); // Maybe a SetExceptionMask(StoredMask); would be better, but I'm not sure if removing the " + [exDenormalized, exUnderflow, exPrecision]" would cause problems in python4delphi
  {$WARN SYMBOL_PLATFORM OFF}
  {$IF Defined(FPC) or Defined(MSWINDOWS)}
  if MatchPythonPrecision then
      SetPrecisionMode(pmDouble)
    else
      SetPrecisionMode(pmExtended);
  {$WARN SYMBOL_PLATFORM ON}
  {$IFEND}
  {$IFEND}
end;

And now one of the P4D-Data-Sciences import codes:

procedure TTorchVision.ImportModule;
var
  StoredMask: TArithmeticExceptionMask;
begin
  MaskFPUExceptions(true, StoredMask);
  try
    inherited;
  finally
    MaskFPUExceptions(false, StoredMask);
  end;
end;

[lmbelo]

@lmbelo, you didn't understand my post. The P4D-Data-Sciences code is wrong and python4delphi's MaskFPUExceptions implementation should be changed as well. I will explain it better.

MaskFPUExceptions code from python4delphi:

python4delphi/Source/PythonEngine.pas

Lines 9288 to 9306 in 19192ad

	procedure MaskFPUExceptions(ExceptionsMasked : boolean;
	MatchPythonPrecision : Boolean);
	begin
	{$IF Defined(CPUX86) or Defined(CPUX64)}
	if ExceptionsMasked then
	SetExceptionMask([exInvalidOp, exDenormalized, exZeroDivide,
	exOverflow, exUnderflow, exPrecision])
	else
	SetExceptionMask([exDenormalized, exUnderflow, exPrecision]);
	{$WARN SYMBOL_PLATFORM OFF}
	{$IF Defined(FPC) or Defined(MSWINDOWS)}
	if MatchPythonPrecision then
	SetPrecisionMode(pmDouble)
	else
	SetPrecisionMode(pmExtended);
	{$WARN SYMBOL_PLATFORM ON}
	{$IFEND}
	{$IFEND}
	end;

Note that the code above arbitrarily changes the exception mask to:

[exInvalidOp, exDenormalized, exZeroDivide, exOverflow, exUnderflow, exPrecision] when the ExceptionsMasked parameter is True [exDenormalized, exUnderflow, exPrecision] when the ExceptionsMasked parameter is False

Now, look at one of the P4D-Data-Sciences import code:

procedure TTorchVision.ImportModule;
begin
  MaskFPUExceptions(true);
  try
    inherited;
  finally
    MaskFPUExceptions(false);
  end;
end;

As it stands, if the exception mask before the code above is empty [], after import it will be [exDenormalized, exUnderflow, exPrecision]. Likewise, if the exception mask is exAllArithmeticExceptions (all exceptions), which is what happens when using OpenGL, D2D, GDIP, Skia, among others, after import the mask will be [exDenormalized, exUnderflow, exPrecision]. And there is no way to predict the mask that will be configured in the application when it arrives at the P4D-Data-Sciences import code, there are many variants, there are several other libraries that change like the Delphi XML lib itself, and it can even vary according to the application (VCL or FMX).

So, the correct thing is: save the mask value, change it and after importing the P4D-Data-Sciences, restore the previous mask value and not arbitrarily change it to [exDenormalized, exUnderflow, exPrecision].

I made a possible fix for both codes. First for python4delphi's MaskFPUExceptions:

procedure MaskFPUExceptions(ExceptionsMasked: Boolean;
  var StoredMask: TArithmeticExceptionMask; MatchPythonPrecision: Boolean);
begin
  {$IF Defined(CPUX86) or Defined(CPUX64)}
  if ExceptionsMasked then
    StoredMask := SetExceptionMask(GetExceptionMask + [exInvalidOp, exDenormalized, exZeroDivide,
      exOverflow, exUnderflow, exPrecision])
  else
    SetExceptionMask(StoredMask + [exDenormalized, exUnderflow, exPrecision]); // Maybe a SetExceptionMask(StoredMask); would be better, but I'm not sure if removing the " + [exDenormalized, exUnderflow, exPrecision]" would cause problems in python4delphi
  {$WARN SYMBOL_PLATFORM OFF}
  {$IF Defined(FPC) or Defined(MSWINDOWS)}
  if MatchPythonPrecision then
      SetPrecisionMode(pmDouble)
    else
      SetPrecisionMode(pmExtended);
  {$WARN SYMBOL_PLATFORM ON}
  {$IFEND}
  {$IFEND}
end;

And now one of the P4D-Data-Sciences import codes:

procedure TTorchVision.ImportModule;
var
  StoredMask: TArithmeticExceptionMask;
begin
  MaskFPUExceptions(true, StoredMask);
  try
    inherited;
  finally
    MaskFPUExceptions(false, StoredMask);
  end;
end;

@viniciusfbb does Skia changes the exception mask for the whole app life cycle?