Knights Landing support

[loveshack]

cpuid(1) on a Knights Landing system shows:

   version information (1/eax):
      processor type  = primary processor (0)
      family          = Intel Pentium Pro/II/III/Celeron/Core/Core 2/Atom, AMD Athlon/Duron, Cyrix M2, VIA C3 (6)
      model           = 0x7 (7)
      stepping id     = 0x1 (1)
      extended family = 0x0 (0)
      extended model  = 0x5 (5)
      (simple synth)  = Intel Xeon Phi x200 (Knights Landing), 14nm

So this patch detects it as HASWELL:

*** OpenBLAS-0.2.19/cpuid_x86.c~    2016-09-08 17:46:18.122229447 +0100
--- OpenBLAS-0.2.19/cpuid_x86.c 2016-10-20 16:50:14.003283533 +0100
***************
*** 1192,1197 ****
--- 1192,1198 ----
            else
        return CPUTYPE_NEHALEM;
    case 5:
+   case 7:         // Knights Landing
          case 14:
      // Skylake
            if(support_avx())
@@ -1700,6 +1701,7 @@
           else
        return CORE_NEHALEM;
    case 5:
+   case 7:         // Knights Landing
    case 14:
      // Skylake
           if(support_avx())
*** OpenBLAS-0.2.19/driver/others/dynamic.c~    2016-09-01 04:58:42.000000000 +0100
--- OpenBLAS-0.2.19/driver/others/dynamic.c 2016-10-20 16:57:02.108394771 +0100
***************
*** 286,291 ****
--- 286,300 ----
        return &gotoblas_NEHALEM; //OS doesn't support AVX. Use old kernels.
      }
    }
+   //Intel Knights Landing
+   if (model == 7) {
+     if(support_avx())
+       return &gotoblas_HASWELL;
+     else{
+       openblas_warning(FALLBACK_VERBOSE, NEHALEM_FALLBACK);
+       return &gotoblas_NEHALEM; //OS doesn't support AVX. Use old kernels.
+     }
+   }
    return NULL;
        }
        case 0xf:

Is there any chance of AVX512 support? (Sorry I couldn't fund it.) Unfortunately, with HASWELL on KNL, OpenBLAS dgemm is about three times slower than MKL.

[brada4]

Xeon phi is not a general purpose CPU, it does not support some normal instructions available since i386, and from your measurement it looks that many others are emulated in firmware..

BTW can you point to any reference that MIC is based or related to Haswell in any way?

[martin-frbg]

As far as I know, the Knights Landing generation of Phi is binary compatible with Haswell - even if some or even most of that is likely to be microcode emulation trickery, Haswell is probably still a better OpenBLAS target for it than the Atom architecture it is somewhat distantly related to.
I suspect AVX512 support will become important once it is availabe in more mainstream Skylake EP or Kabi Lake systems expected for next year. Alas modifying the current haswell gemm kernels to make use of AVX512 looks to be a bit more complicated than just changing the instruction names.
@loveshack, while you cannot provide funds, would you be able to provide access to your system if and when a capable developer becomes available ?

[brada4]

CPUID says no MMX or SSE (With other hand intel writes you have to cjeck cpuid bits before emitting special instructions)
https://software.intel.com/sites/default/files/forum/278102/327364001en.pdf#section.B.8
Would be interesting to at least measure GEMM to find if SSE2 or AVX or AVX2 works best)

[loveshack]

Martin Kroeker notifications@github.com writes:

As far as I know, the Knights Landing generation of Phi is binary
compatible with Haswell - even if some or even most of that is likely
to be microcode emulation trickery, Haswell is probably still a better
OpenBLAS target for it than the Atom architecture it is somewhat
distantly related to.

Indeed. [As far as I know, all Xeons are microcoded anyhow.]

I suspect AVX512 support will become important once it is availabe in
more mainstream Skylake EP or Kabi Lake systems expected for next
year. Alas modifying the current haswell gemm kernels to make use of
AVX512 looks to be a bit more complicated than just changing the
instruction names.

For what it's worth, ATLAS has Knights Corner (I think) support, but I
haven't managed to build the development version.

@loveshack, while you cannot provide funds, would you be able to
provide access to your system if and when a capable developer becomes
available ?

Unfortunately not, but I could suggest another site to try. Also, there
is an Intel proprietary simulator for KNL, but I don't know how useful
it would be for this.

There is avx512 support for small matrix multiplication in
https://github.com/hfp/libxsmm, but I guess that won't help with the
general case.

[loveshack]

CPUID says no MMX or SSE

Please stop talking rubbish about it, not that those are relevant. # cpuid -1 | egrep 'MMX|SSE |simple synth' (simple synth) = Intel Xeon Phi x200 (Knights Landing), 14nm MMX Technology = true SSE extensions = true XCR0 supported: SSE state = true I've no idea in what way that it's not "general purpose" either, running that vanilla CentOS 7 like any other server.

(With other hand intel writes you have to cjeck cpuid bits before emitting special instructions)

Yes, I modified dynamic.c.

https://software.intel.com/sites/default/files/forum/278102/327364001en.pdf#section.B.8 Would be interesting to at least measure GEMM to find if SSE2 or AVX or AVX2 works best)

It wouldn't; we know how they compare.

[brada4]

How do we know? That spec explicitly says there is not even MMX in CPUID.... While I dont really believe that .... At least can you check Sandybridge vs Haswell (With seconds, not 2-5x faster, and we know it)

Emulator will not help with cache timings...

[martin-frbg]

My knowledge may well be outdated, but isn't ATLAS "just" plain C with some clever functions to tune loop unrolling etc. for the build target (making it easier to "port", but harder to match well-written machine code for a given target) ? And thanks for the pointer to libxsmm - the goals and license seem similar enough, but at first glance the code organization looks too complicated to even suggest trying a quick-and-dirty replacement of one of OpenBLAS' Haswell-optimized functions with its AVX512-using equivalent.

[loveshack]

You wrote:

My knowledge may well be outdated, but isn't ATLAS "just" plain C with
some clever functions to tune loop unrolling etc. for the build target
(making it easier to "port", but harder to match well-written machine
code for a given target) ?

It has assembler kernels, if that's what you mean. Knights Corner is
called "avxz" there, so presumably the relevant bits are
tune/blas/gemm/AMMCASES/*avxz.S in the development version (looking at
3.11.39) but I don't know anything much about it. It seems unlikely to
be useful for KNL, unfortunately, per Clint's initial response in
https://sourceforge.net/p/math-atlas/support-requests/1038/ and my
later build attempt.

For what it's worth, I just got a pointer to some level of support for
larger-size GEMM in libxsmm, with current status in
https://github.com/hfp/libxsmm/issues/99#issuecomment-255314392

[loveshack]

You wrote:

How do we know? That spec explicitly says there is not even MMX in
CPUID.... While I dont really believe that .... At least can you check
Sandybridge vs Haswell (With seconds, not 2-5x faster, and we know it)

Emulator will not help with cache timings...

Please stop responding like this; it's only likely to drive people away
from OpenBLAS, which I don't want.

[brada4]

OK, on MIC emulator fastest kernel is one that matches host CPU.
Does it work for you too?

[jeffhammond]

• Knights Landing (KNL) aka Xeon Phi 72xx is binary-compatible with Haswell (Xeon v3) except for TSX (transactional memory). This was documented long ago (here).
• Knights Landing: Second-Generation Intel Xeon Phi Product discusses the microarchitecture in detail.
• Mixing SSE and AVX instructions is a bad idea but there is no good reason to do this.
• LIBXSMM is written by some of the smartest people at Intel and is another great resource for microkernel insight.
• BLIS already supports KNL (code) well and should be consulted for BLAS implementation insight. The author of that code knows what he is doing and has been in frequent contact with some of the authors of LIBXSMM.
• Some of the comments in this thread are factually incorrect.

I work for Intel.

[jeffhammond]

@brada4

Linux has no trouble figuring out the cpuid bits associated with MMX, etc.

$ cat /proc/cpuinfo  | more 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 87
model name	: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz
stepping	: 1
microcode	: 0x12b
cpu MHz		: 1396.793
cache size	: 1024 KB
physical id	: 0
siblings	: 272
core id		: 0
cpu cores	: 68
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm const
ant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni 
pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm sse4_1 sse4_2
x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 
3dnowprefetch ida arat epb pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase 
tsc_adjust bmi1 avx2 smep bmi2 erms avx512f rdseed adx avx512pf avx512er avx512cd xsaveopt
bogomips	: 2793.58
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

[martin-frbg]

Thanks for your insight. (Actually I am not sure we still need to argue about Haswell compatibility or the chance to deduct it from cpuid features.) Dave Love's patch was committed as part of PR #1010 on november 7 and from similar open issues I consider it likely that not all of the observed performance difference to MKL comes from not using avx512.

[jeffhammond]

@martin-frbg Certainly not using AVX-512 is the leading order reason that performance is behind MKL, but one needs to consider the KNL cache hierarchy (no L3, L2 for each tile of 2 cores) carefully. Finally, using instruction encodings wider than 8-bytes reduces performance. The Intel manuals and Agner Fog's website have details.

[brada4]

(Already good info that Knights Landing can run Haswell)
Currently huge numbers of cores will not be very well handled. One core until some arbitrary threshold, then all.
Any memory access encoding exceeds 8 bytes - is it just observation that RAM is slower than registers or there is some extra care needed around lines of FMA?