Add basic check for bhyve support. #153

skarekrow · 2016-04-29T09:11:17Z

Not guaranteed, but a bit better then nothing.

Closes #152

…hen nothing.

moogle19 · 2016-04-29T09:21:36Z

Are these flags the same for AMD processors?

skarekrow · 2016-04-29T09:23:44Z

No, iirc AMD has RVI.

skarekrow · 2016-04-29T09:23:56Z

But I don't have an AMD processor to confirm, so I didn't add that to this commit.

skarekrow · 2016-04-29T09:24:52Z

But...this says The easiest way to tell if a processor supports bhyve is to run dmesg or look in /var/run/dmesg.boot for the POPCNT processor feature flag on the Features2 line for AMD® processors or EPT and UG on the VT-x line for Intel® processors.

pr1ntf · 2016-04-29T09:29:28Z

Let's wait on confirmation from an AMD user.

moogle19 · 2016-04-29T09:42:05Z

Do we have to check for EPT and UG or does EPT imply UG? (What is UG even?)

skarekrow · 2016-04-29T09:43:43Z

They are not exclusive, UG allows you to give a host more then one virtual CPU. Read that link, it has tons of good information ;)

EpiJunkie · 2016-04-29T15:33:13Z

UG stands for Unrestricted Guest.

The FreeBSD handbook does not mention UEFI support, so another thing to keep in mind is that UEFI is dependent on UG.

Speaking strictly in regards to the Xeon line of processors, only the 5500 series CPUs lack the UG CPU feature but they do have EPT|POPCNT support. The next generation 5600 series and all Xeons since have both features.

~~In addition to the dmesg output you can install devel/cpuflags to print out a slightly more verbose table of the CPU features available:~~ Apparently cpuflags does not show all the CPU flags consistently, post updated to remove incorrect output excerpts and replace with valid ones. My apologies to everyone for the mix up.

A fairly recent desktop Intel CPU with both POPCNT and UG support:

Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.3-RELEASE #0 4b75b72(releng/10.3): Thu Mar 31 10:32:11 EDT 2016
    root@avenger:/usr/obj/tmp/build/L51u/pcbsd/git/freebsd/sys/GENERIC amd64
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
VT(vga): resolution 640x480
info: [drm] Initialized drm 1.1.0 20060810
CPU: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz (3591.76-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x306c3  Family=0x6  Model=0x3c  Stepping=3
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Structured Extended Features=0x2fbb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics

An older 5500 series Xeon with the POPCNT feature but without the UG feature:

Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010
    xxx@xxx.xxx:/usr/obj/usr/src/sys/GENERIC amd64
Timecounter i8254 frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz (2261.02-MHz K8-class CPU)
  Origin = GenuineIntel  Id = 0x106a5  Family = 6  Model = 1a  Stepping = 5
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x9ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant

EpiJunkie · 2016-04-29T17:56:57Z

(For the email watchers, I updated my previous post because cpuflags was not displaying all the CPU flags. It did seem strange not to see the VT-x line for my example. My apologies to everyone. Lesson learned, dmesg is the best source of truth.)

My AMD findings:

There is no "RVI" CPU flag to indicate "Rapid Virtualization Indexing" support, also known as Extended Page Tables in the Intel realm of CPU features. Instead AMD CPUs also use POPCNT to indicate RVI support.

The other technology, which is most akin to VT-x in the Intel realm, is AMD-V. AMD-V is indicated by the SVM CPU flag under "AMD Features=" in the dmesg output. An interesting unappreciated fact is that AMD actually developed these hardware virtualized CPU extensions before Intel.

You can read more about it here.

There does not seem to be an UG equivalent in the AMD realm. I speculate that it was implemented as part of Rapid Virtualization Indexing from the onset.

Thanks to NYC*BUG's dmesgd repository I was able to find dmesg output from a POPCNT and SVM supported AMD Processor:

Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-STABLE #3: Fri May 24 02:46:03 WIT 2013
    Badi@DATABASE-10:/usr/obj/usr/src/sys/MYSQL amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: AMD Opteron(tm) Processor 6128 (2000.18-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x100f91  Family = 0x10  Model = 0x9  Stepping = 1
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x837ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,NodeId>
  TSC: P-state invariant
real memory  = 42949672960 (40960 MB)
avail memory = 41395609600 (39477 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <032311 APIC1603>
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
FreeBSD/SMP: 2 package(s) x 8 core(s)

Here is output from an AMD CPU with AMD-V SVM but without POPCNT:

Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.3-RELEASE #0: Wed Jan 16 01:43:02 UTC 2008
    xxx@xxx.xxx:/usr/obj/usr/src/sys/SMP
ACPI APIC Table: <PTLTD      APIC  >
Timecounter i8254 frequency 1193182 Hz quality 0
CPU: Dual-Core AMD Opteron(tm) Processor 1212 (2009.27-MHz K8-class CPU)
  Origin = AuthenticAMD  Id = 0x40f33  Stepping = 3
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x2001<SSE3,CX16>
  AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x1f<LAHF,CMP,SVM,ExtAPIC,CR8>
  Cores per package: 2
real memory  = 535756800 (510 MB)
avail memory = 503631872 (480 MB)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1

So the long winded conclusion I have drawn from this is it might be better to grep for POPCNT for all CPUs and UG for Intel based machines.

Thoughts?

pr1ntf · 2016-04-30T01:16:54Z

POPCNT and UG sound good to me.

However, let's still just warn, let's not stop the user from using iohyve, just in case.

skarekrow · 2016-04-30T01:36:57Z

I'm not sure I agree. I think POPCNT and EPT should be required. If they can't support those extensions, it won't work for them anyways. UG is only if they want multiple cpu. It's not a requirement for 1, so I don't think we should rely on that.

I misread with VT-x. EPT is on that line. So I will change this soon to look for POPCNT or EPT.

pr1ntf · 2016-04-30T01:46:26Z

So essentially we are back to the original plan.

skarekrow · 2016-04-30T02:02:37Z

Not really. Adding checks for those two with an exit I think is the right move. If they don't support those extensions, they couldn't use iohyve anyways. We saved them a setup essentially.

Brandon
On Fri, Apr 29, 2016 at 8:46 PM, pr1ntf notifications@github.com wrote:
So essentially we are back to the original plan.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub [https://github.com//pull/153#issuecomment-215923085]

pr1ntf · 2016-04-30T02:33:14Z

Be sure to put changes in the lib/setup file to fix conflict.

skarekrow · 2016-04-30T02:51:33Z

Roger Roger.

Brandon
On Fri, Apr 29, 2016 at 9:33 PM, pr1ntf notifications@github.com wrote:
Be sure to put changes in the lib/setup file to fix conflict.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub [https://github.com//pull/153#issuecomment-215926581]

pr1ntf · 2016-04-30T02:52:59Z

Don't call me Roger

skarekrow · 2016-04-30T02:57:38Z

Hahaha even as a joke EP 1 misses!

Brandon
On Fri, Apr 29, 2016 at 9:53 PM, pr1ntf notifications@github.com wrote:
Don't call me Roger

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub [https://github.com//pull/153#issuecomment-215927508]

This is my take on how this should be implemented. This is in response to: pr1ntf/iohyve#153 This checks to see if the necessary CPU features are available to run bhyve, limits Intel guests if lacking the UG feature, and exits if EPT/RVI missing.

pr1ntf · 2016-05-06T08:33:44Z

I've implemented this, didn't know how to edit the PR directly. womp

🐙

EpiJunkie · 2016-05-06T12:50:31Z

@pr1ntf I do not see what you changed from @skarekrow's original PR (except a leading comment) which does not address AMD CPUs or Intel CPUs missing UG.

This does not allow for AMD CPUs to be used even when they have the proper CPU features. AMD CPUs do no have the EPT or VT-x CPU feature flags in their dmesg output. I documented this above. The proper string to grep for would be grep 'POPCNT'. BUT then this would leave the Intel 5500 series out in the cold because they are really limited due to the lack of the unrestricted guest UG CPU feature. Without the UG feature a guest can not have more than one CPU, can not start UEFI guests, nor can a guest start in any other mode than 64-bit (aka only FreeBSD OSes are able to do this).

In addition to greping for the right CPU feature, I suggest another if statement to handle the Intel CPUs:

if ! $( cat /var/run/dmesg.boot | grep 'POPCNT' ) ; then

if [ -n "$( cat /var/run/dmesg.boot | grep 'CPU:' | grep 'Intel' )" ] && [ -z "$( cat /var/run/dmesg.boot | grep 'VT-x:' | grep 'UG' )" ]; then
    echo "Missing UG feature on Intel CPU"
    <insert handling>

James Lodge did an excellent job documenting the limited feature set of the Intel 5500 CPUs in his blog post here.

pr1ntf · 2016-05-06T13:08:20Z

do not see what you changed from @skarekrow's original PR (except a leading comment) which does not address AMD CPUs or Intel CPUs missing UG

Yeah.....keen observation.

Jumped the gun. I thought he had already added your suggestions. Got lost in babble. What was in there I just yanked out. I guess now would be the time to google how to submit changes to another person's PR.

EpiJunkie · 2016-05-06T21:19:30Z

414fa74 adapts @skarekrow's work.

EpiJunkie · 2016-05-06T21:36:28Z

I guess now would be the time to google how to submit changes to another person's PR.

@pr1ntf here is the simplified process I did to maintain the history of @skarekrow commit. If you go to Graphs and then Network it will demonstrate this.

git fetch upstream pull/153/head:MYLOCALBRANCHNAME
git checkout MYLOCALBRANCHNAME
Merged in current code from head
Made my additions
git push

Because you have write access to the repo, the work flow is going to be a little different. This article from Jennifer Mack goes through the process.

EpiJunkie · 2016-05-07T17:57:27Z

Opened #164 as my changes can not integrate with this one due to a lack of write access.

pr1ntf · 2016-05-10T00:48:25Z

Closing in favor of #164

Addendum to Skarekrow's #153 CPU feature check

Add basic check for bhyve support. Not guaranteed, but a bit better t…

9f9eb99

…hen nothing.

skarekrow force-pushed the master branch from e82ad8b to 9f9eb99 Compare April 29, 2016 09:13

pr1ntf added the enhancement label Apr 29, 2016

pr1ntf mentioned this pull request Apr 30, 2016

Check if CPU is supported by bhyve #152

Closed

pr1ntf pushed a commit that referenced this pull request May 6, 2016

implement rotting #153 cuz skarekrow is taking too long lolol

fa35a11

pr1ntf closed this May 6, 2016

pr1ntf reopened this May 6, 2016

pr1ntf closed this May 10, 2016

pr1ntf pushed a commit that referenced this pull request May 15, 2016

Merge pull request #164 from EpiJunkie/skarekrow-cpu-feature-check

058a8dd

Addendum to Skarekrow's #153 CPU feature check

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add basic check for bhyve support. #153

Add basic check for bhyve support. #153

skarekrow commented Apr 29, 2016

moogle19 commented Apr 29, 2016

skarekrow commented Apr 29, 2016

skarekrow commented Apr 29, 2016

skarekrow commented Apr 29, 2016

pr1ntf commented Apr 29, 2016

moogle19 commented Apr 29, 2016

skarekrow commented Apr 29, 2016 •

edited

Loading

EpiJunkie commented Apr 29, 2016 •

edited

Loading

EpiJunkie commented Apr 29, 2016

pr1ntf commented Apr 30, 2016

skarekrow commented Apr 30, 2016

pr1ntf commented Apr 30, 2016

skarekrow commented Apr 30, 2016

pr1ntf commented Apr 30, 2016

skarekrow commented Apr 30, 2016

pr1ntf commented Apr 30, 2016

skarekrow commented Apr 30, 2016

pr1ntf commented May 6, 2016

EpiJunkie commented May 6, 2016

pr1ntf commented May 6, 2016

EpiJunkie commented May 6, 2016

EpiJunkie commented May 6, 2016

EpiJunkie commented May 7, 2016

pr1ntf commented May 10, 2016

Add basic check for bhyve support. #153

Add basic check for bhyve support. #153

Conversation

skarekrow commented Apr 29, 2016

moogle19 commented Apr 29, 2016

skarekrow commented Apr 29, 2016

skarekrow commented Apr 29, 2016

skarekrow commented Apr 29, 2016

pr1ntf commented Apr 29, 2016

moogle19 commented Apr 29, 2016

skarekrow commented Apr 29, 2016 • edited Loading

EpiJunkie commented Apr 29, 2016 • edited Loading

EpiJunkie commented Apr 29, 2016

My AMD findings:

pr1ntf commented Apr 30, 2016

skarekrow commented Apr 30, 2016

pr1ntf commented Apr 30, 2016

skarekrow commented Apr 30, 2016

pr1ntf commented Apr 30, 2016

skarekrow commented Apr 30, 2016

pr1ntf commented Apr 30, 2016

skarekrow commented Apr 30, 2016

pr1ntf commented May 6, 2016

EpiJunkie commented May 6, 2016

pr1ntf commented May 6, 2016

EpiJunkie commented May 6, 2016

EpiJunkie commented May 6, 2016

EpiJunkie commented May 7, 2016

pr1ntf commented May 10, 2016

skarekrow commented Apr 29, 2016 •

edited

Loading

EpiJunkie commented Apr 29, 2016 •

edited

Loading