Skip to content
This repository has been archived by the owner on Sep 9, 2024. It is now read-only.

Release v1.3.0 ACPI Driver #12

Merged
merged 6 commits into from
Jul 31, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ This library is currently used as part of the
# Prerequisites

- Linux based OS
- P-States
- ``intel_pstates`` enabled - no entries in kernel cmdline disabling it
- P-States or acpi-cpufreq scaling driver enabled
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- P-States or acpi-cpufreq scaling driver enabled
- P-State or acpi-cpufreq scaling driver enabled

- C-States
- ``intel_cstates`` kernel module loaded
- Uncore frequency
Expand Down Expand Up @@ -128,7 +127,7 @@ err := perofmancePool.Remove()

All CPUs in the removed pool will be moved back to the Shared Pool.

### P-States
### Profiles

Power profiles can be associated with any Exclusive Pool or the Shared Pool

Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.20
require (
github.com/go-logr/logr v1.2.4
github.com/hashicorp/go-multierror v1.1.1
github.com/stretchr/testify v1.8.2
github.com/stretchr/testify v1.8.4
)

require (
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
Expand Down
13 changes: 9 additions & 4 deletions pkg/power/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,17 +189,22 @@ C4E/C5 Enhanced Deeper Sleep
C6 Deep Power Down
````

## P-State Governor
## Scaling Driver

The P-state governor feature allows the user to check if the P-state driver is enabled on the system. If the P-state
driver is enabled while using the Kubernetes Power Manger, users may select a P-state governor per core, which are
described as "performance" and "powersave" governors in the Power Profiles.
### P-state
The P-state governor feature allows the user to check if the P-state driver is enabled on the system. If the P-state
driver is enabled while using the Kubernetes Power Manager, users may select a P-state governor per core, which are
described as "performance" and "powersave" governors in the Power Profiles.

* Performance governor - The CPUfreq governor "performance" sets the CPU statically to the highest frequency within the
borders of scaling_min_freq and scaling_max_freq.
* Powersave governor - The CPUfreq governor "powersave" sets the CPU statically to the lowest frequency within the
borders of scaling_min_freq and scaling_max_freq.

### acpi-cpufreq
The acpi-cpufreq driver operates much like the P-state driver but has a number of both static and dynamic governors. For more information see [here](https://www.kernel.org/doc/html/v4.12/admin-guide/pm/cpufreq.html).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The acpi-cpufreq driver operates much like the P-state driver but has a number of both static and dynamic governors. For more information see [here](https://www.kernel.org/doc/html/v4.12/admin-guide/pm/cpufreq.html).
The acpi-cpufreq driver setting operates much like the P-state driver but has a different set of available governors. For more information see [here](https://www.kernel.org/doc/html/v4.12/admin-guide/pm/cpufreq.html).

One thing to note is that acpi-cpufreq reports the base clock as the frequency hardware limits however the P-state driver uses turbo frequency limits.
Both drivers can make use of turbo frequency however acpi-cpufreq will exceed its hardware frequency limits when using turbo frequency. This is important to take into account when setting frequencies for profiles.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Both drivers can make use of turbo frequency however acpi-cpufreq will exceed its hardware frequency limits when using turbo frequency. This is important to take into account when setting frequencies for profiles.
Both drivers can make use of turbo frequency; however, acpi-cpufreq can exceed hardware frequency limits when using turbo frequency. This is important to take into account when setting frequencies for profiles.

## Topology

Topology discovery is done via reading /sys/devices/system/cpuN/topology/{physical_package_id,die_id,core_id}. Based on
Expand Down
2 changes: 1 addition & 1 deletion pkg/power/c_states.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ func mapAvailableCStates() error {
cStatesNamesMap[stateName] = stateNumber
defaultCStates[stateName] = true
}
log.V(3).Info("mapped C-states","map", cStatesNamesMap)
log.V(3).Info("mapped C-states", "map", cStatesNamesMap)
return nil
}

Expand Down
3 changes: 2 additions & 1 deletion pkg/power/c_states_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,12 @@ package power

import (
"fmt"
"github.com/stretchr/testify/assert"
"os"
"path/filepath"
"strings"
"testing"

"github.com/stretchr/testify/assert"
)

func setupCpuCStatesTests(cpufiles map[string]map[string]map[string]string) func() {
Expand Down
23 changes: 13 additions & 10 deletions pkg/power/cpu_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@ package power

import (
"fmt"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
)

type cpuMock struct {
Expand Down Expand Up @@ -45,7 +46,7 @@ func (m *cpuMock) SetPool(pool Pool) error {
return m.Called(pool).Error(0)
}

func setupCpuPStatesTests(cpufiles map[string]map[string]string) func() {
func setupCpuScalingTests(cpufiles map[string]map[string]string) func() {
origBasePath := basePath
basePath = "testing/cpus"
defaultDefaultPowerProfile := defaultPowerProfile
Expand All @@ -56,7 +57,7 @@ func setupCpuPStatesTests(cpufiles map[string]map[string]string) func() {
getNumberOfCpus = func() uint { return uint(len(cpufiles)) }

// "initialise" P-States feature
featureList[PStatesFeature].err = nil
featureList[FreqencyScalingFeature].err = nil

// if cpu0 is here we set its values to temporary defaultPowerProfile
if cpu0, ok := cpufiles["cpu0"]; ok {
Expand Down Expand Up @@ -91,14 +92,16 @@ func setupCpuPStatesTests(cpufiles map[string]map[string]string) func() {
os.WriteFile(filepath.Join(cpudir, scalingMinFile), []byte(value+"\n"), 0644)
os.WriteFile(filepath.Join(cpudir, cpuMinFreqFile), []byte(value+"\n"), 0644)
case "package":
os.WriteFile(filepath.Join(cpudir,packageIdFile),[]byte(value+"\n"), 0644)
os.WriteFile(filepath.Join(cpudir, packageIdFile), []byte(value+"\n"), 0644)
case "die":
os.WriteFile(filepath.Join(cpudir,dieIdFile),[]byte(value+"\n"), 0644)
os.WriteFile(filepath.Join(cpudir,coreIdFile),[]byte(cpuName[3:]+"\n"), 0644)
os.WriteFile(filepath.Join(cpudir, dieIdFile), []byte(value+"\n"), 0644)
os.WriteFile(filepath.Join(cpudir, coreIdFile), []byte(cpuName[3:]+"\n"), 0644)
case "epp":
os.WriteFile(filepath.Join(cpudir, eppFile), []byte(value+"\n"), 0644)
case "governor":
os.WriteFile(filepath.Join(cpudir, scalingGovFile), []byte(value+"\n"), 0644)
case "available_governors":
os.WriteFile(filepath.Join(cpudir, availGovFile), []byte(value+"\n"), 0644)
}
}
}
Expand All @@ -110,7 +113,7 @@ func setupCpuPStatesTests(cpufiles map[string]map[string]string) func() {
// revert get number of system cpus function
getNumberOfCpus = origGetNumOfCpusFunc
// revert p-states feature to un initialised state
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably want to update this comment.

featureList[PStatesFeature].err = uninitialisedErr
featureList[FreqencyScalingFeature].err = uninitialisedErr
// revert default powerProfile
defaultPowerProfile = defaultDefaultPowerProfile
}
Expand All @@ -124,7 +127,7 @@ func TestNewCore(t *testing.T) {
"epp": "some",
},
}
defer setupCpuPStatesTests(cpufiles)()
defer setupCpuScalingTests(cpufiles)()

// happy path - ensure values from files are read correctly
core := &cpuCore{}
Expand All @@ -140,7 +143,7 @@ func TestNewCore(t *testing.T) {
}, cpu)

// now "break" P-States by setting a feature error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably want to update this comment.

featureList[PStatesFeature].err = fmt.Errorf("some error")
featureList[FreqencyScalingFeature].err = fmt.Errorf("some error")

cpu, err = newCpu(0, nil)

Expand Down
3 changes: 1 addition & 2 deletions pkg/power/host.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,8 @@ func initHost(nodeName string) (Host, error) {
host := &hostImpl{
name: nodeName,
exclusivePools: PoolList{},
featureStates: &featureList,
}

host.featureStates = &featureList
Comment on lines 43 to +45
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small question: why is this needed? It seems like we're doing the same thing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't remember exactly why I did this but I think it was because I was getting some weird behavior where the host was using a snapshot of the pointers' value instead of the pointer itself. Think this was just for the purposes of debugging to see the field go from unassigned to assigned but probably best to leave as is right now just in case and revert it for next time around

// create predefined pools
host.reservedPool = &reservedPoolType{poolImpl{
name: reservedPoolName,
Expand Down
6 changes: 3 additions & 3 deletions pkg/power/host_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -317,13 +317,13 @@ func (s *hostTestsSuite) TestUpdateProfile() {
//pool.On("Name").Return("powah")
host := hostImpl{
sharedPool: new(poolMock),
featureStates: &FeatureSet{PStatesFeature: &featureStatus{err: nil}},
featureStates: &FeatureSet{FreqencyScalingFeature: &featureStatus{err: nil}},
}
origFeatureList := featureList
featureList = map[featureID]*featureStatus{
PStatesFeature: {
FreqencyScalingFeature: {
err: nil,
initFunc: initPStates,
initFunc: initScalingDriver,
},
CStatesFeature: {
err: nil,
Expand Down
2 changes: 1 addition & 1 deletion pkg/power/pool.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ type poolImpl struct {
name string
cpus CpuList
host Host
// P-States
// Scaling-Driver
PowerProfile Profile
// C-States
CStatesProfile *CStates
Expand Down
3 changes: 2 additions & 1 deletion pkg/power/pool_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@ package power

import (
"fmt"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"testing"
)

type poolMock struct {
Expand Down
64 changes: 56 additions & 8 deletions pkg/power/power.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@ package power

import (
"fmt"
"github.com/go-logr/logr"
"github.com/hashicorp/go-multierror"
"os"
"path"
"runtime"
"strconv"
"strings"

"github.com/go-logr/logr"
"github.com/hashicorp/go-multierror"
)

var basePath = "/sys/devices/system/cpu"
Expand All @@ -18,19 +20,30 @@ const (
sharedPoolName = "sharedPool"
reservedPoolName = "reservedPool"

PStatesFeature featureID = iota
FreqencyScalingFeature featureID = iota
EPPFeature
CStatesFeature
UncoreFeature
)

type LibConfig struct {
CpuPath string
ModulePath string
Cores uint
}

// initialized with null logger, can be set to proper logger with SetLogger
var log = logr.Discard()

// default declaration of defined features, defined to uninitialized state
var featureList FeatureSet = map[featureID]*featureStatus{
PStatesFeature: {
EPPFeature: {
err: uninitialisedErr,
initFunc: initEpp,
},
FreqencyScalingFeature: {
err: uninitialisedErr,
initFunc: initPStates,
initFunc: initScalingDriver,
},
CStatesFeature: {
err: uninitialisedErr,
Expand All @@ -42,7 +55,7 @@ var featureList FeatureSet = map[featureID]*featureStatus{
},
}
var uninitialisedErr = fmt.Errorf("feature uninitialized")
var undefinedErr = fmt.Errorf("feature undefined")
var undefinederr = fmt.Errorf("feature undefined")

// featureStatus stores feature name, driver and if feature is not supported, error describing the reason
type featureStatus struct {
Expand Down Expand Up @@ -108,7 +121,7 @@ func (set *FeatureSet) isFeatureIdSupported(id featureID) bool {
func (set *FeatureSet) getFeatureIdError(id featureID) error {
feature, exists := (*set)[id]
if !exists {
return undefinedErr
return undefinederr
}
return feature.err
}
Expand All @@ -130,10 +143,45 @@ func CreateInstance(hostName string) (Host, error) {
}
return host, allErrors.ErrorOrNil()
}
func CreateInstanceWithConf(hostname string, conf LibConfig) (Host, error) {
if conf.CpuPath != "" {
basePath = conf.CpuPath
}
if conf.ModulePath != "" {
kernelModulesFilePath = conf.ModulePath
}
getNumberOfCpus = func () uint { return conf.Cores}
return CreateInstance(hostname)
}

// getNumberOfCpus defined as var so can be mocked by the unit test
var getNumberOfCpus = func() uint {
return uint(runtime.NumCPU())
// First, try to get CPUs from sysfs. If the sysfs isn't available
// return Number of CPUs from runtime
cpusAvailable, err := readStringFromFile(path.Join(basePath, "online"))
if err != nil {
return uint(runtime.NumCPU())
}

// Delete \n character and split the string to get
// first and last element
cpusAvailable = strings.Replace(cpusAvailable, "\n", "", -1)
cpuSlice := strings.Split(cpusAvailable, "-")
if len(cpuSlice) < 2 {
return uint(runtime.NumCPU())
}

// Calculate number of CPUs, if an error occurs
// return the number of CPUs from runtime
firstElement, err := strconv.Atoi(cpuSlice[0])
if err != nil {
return uint(runtime.NumCPU())
}
secondElement, err := strconv.Atoi(cpuSlice[1])
if err != nil {
return uint(runtime.NumCPU())
}
return uint((secondElement - firstElement) + 1)
}

// reads a file from a path, parses contents as an int a returns the value
Expand Down
28 changes: 20 additions & 8 deletions pkg/power/power_profile.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ type profileImpl struct {
// todo classification
}

// Profile is a P-states power profile
// requires P-states feature
// Profile contains scaling driver information
type Profile interface {
Name() string
Epp() string
Expand All @@ -23,22 +22,26 @@ type Profile interface {
Governor() string
}

var availableGovs []string

// todo add simple constructor that determines frequencies automagically?

// NewPowerProfile creates a power P-States power profile,
// NewPowerProfile creates a power profile,
func NewPowerProfile(name string, minFreq uint, maxFreq uint, governor string, epp string) (Profile, error) {
if !featureList.isFeatureIdSupported(PStatesFeature) {
return nil, featureList.getFeatureIdError(PStatesFeature)
if !featureList.isFeatureIdSupported(FreqencyScalingFeature) {
return nil, featureList.getFeatureIdError(FreqencyScalingFeature)
}

if minFreq > maxFreq {
return nil, fmt.Errorf("max Freq can't be lower than min")
}

if governor != cpuPolicyPerformance && governor != cpuPolicyPowersave { //todo determine by reading available governors, its different for acpi Driver
return nil, fmt.Errorf("governor can only be set to '%s' or '%s'", cpuPolicyPerformance, cpuPolicyPowersave)
if governor == "" {
governor = defaultGovernor
}
if !checkGov(governor) { //todo determine by reading available governors, its different for acpi Driver
return nil, fmt.Errorf("governor can only be set to the following %v", availableGovs)

}
if epp != "" && governor == cpuPolicyPerformance && epp != cpuPolicyPerformance {
return nil, fmt.Errorf("only '%s' epp can be used with '%s' governor", cpuPolicyPerformance, cpuPolicyPerformance)
}
Expand Down Expand Up @@ -72,3 +75,12 @@ func (p *profileImpl) Name() string {
func (p *profileImpl) Governor() string {
return p.governor
}

func checkGov(governor string) bool {
for _, element := range availableGovs {
if element == governor {
return true
}
}
return false
}
Loading