Skip to content

Commit

Permalink
in zcl-loader-dotdot.js:
Browse files Browse the repository at this point in the history
	* making sure discriminator ref is set correctly for all strings

	* making sure is_signed is set correctly for all numbers

	* dealing with subatomics (by processing them last)

	* picking up atomic size from length property

fixing incorrect value in zcl-loader-consecutive.test.js and adding string count test in zcl-loader.test.js
  • Loading branch information
dhchandw committed Sep 25, 2024
1 parent 2b71747 commit 3326112
Show file tree
Hide file tree
Showing 4 changed files with 245 additions and 30 deletions.
104 changes: 104 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -19614,9 +19614,11 @@ This module provides the APIs for dotdot Loading
* [~prepareCommands(commands, side, types)](#module_Loader API_ Loader APIs..prepareCommands) ⇒
* [~prepareCluster(cluster, isExtension, types)](#module_Loader API_ Loader APIs..prepareCluster) ⇒
* [~prepareAtomic(type)](#module_Loader API_ Loader APIs..prepareAtomic) ⇒
* [~prepareSubAtomic(type, atomics)](#module_Loader API_ Loader APIs..prepareSubAtomic) ⇒
* [~prepareBitmap(type, isContained)](#module_Loader API_ Loader APIs..prepareBitmap) ⇒
* [~prepareEnum(type)](#module_Loader API_ Loader APIs..prepareEnum) ⇒
* [~prepareStruct(type)](#module_Loader API_ Loader APIs..prepareStruct) ⇒
* [~prepareSubAtomicTypes(types)](#module_Loader API_ Loader APIs..prepareSubAtomicTypes)
* [~prepareTypes(zclTypes, types)](#module_Loader API_ Loader APIs..prepareTypes)
* [~prepareAttributeType(attribute, types, cluster)](#module_Loader API_ Loader APIs..prepareAttributeType)
* [~prepareDeviceType(deviceType)](#module_Loader API_ Loader APIs..prepareDeviceType) ⇒
Expand Down Expand Up @@ -19881,6 +19883,19 @@ Parses xml type into the atomic object for insertion into the DB
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the atomic format in the dotdot xml |

<a name="module_Loader API_ Loader APIs..prepareSubAtomic"></a>

### Loader API: Loader APIs~prepareSubAtomic(type, atomics) ⇒
Parses xml type into the sub-atomic object for insertion into the DB

**Kind**: inner method of [<code>Loader API: Loader APIs</code>](#module_Loader API_ Loader APIs)
**Returns**: object ready for insertion into the DB

| Param | Type | Description |
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the sub-atomic format in the dotdot xml |
| atomics | <code>\*</code> | an array of atomic types |

<a name="module_Loader API_ Loader APIs..prepareBitmap"></a>

### Loader API: Loader APIs~prepareBitmap(type, isContained) ⇒
Expand Down Expand Up @@ -19918,6 +19933,17 @@ Parses xml type into the struct object for insertion into the DB
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the struct format in the dotdot xml |

<a name="module_Loader API_ Loader APIs..prepareSubAtomicTypes"></a>

### Loader API: Loader APIs~prepareSubAtomicTypes(types)
Parses array of xml objects that conform to the sub-atomic format in the dotdot xml

**Kind**: inner method of [<code>Loader API: Loader APIs</code>](#module_Loader API_ Loader APIs)

| Param | Type |
| --- | --- |
| types | <code>\*</code> |

<a name="module_Loader API_ Loader APIs..prepareTypes"></a>

### Loader API: Loader APIs~prepareTypes(zclTypes, types)
Expand Down Expand Up @@ -21414,9 +21440,11 @@ This module provides the APIs for new data model loading
* [~prepareCommands(commands, side, types)](#module_Loader API_ Loader APIs..prepareCommands) ⇒
* [~prepareCluster(cluster, isExtension, types)](#module_Loader API_ Loader APIs..prepareCluster) ⇒
* [~prepareAtomic(type)](#module_Loader API_ Loader APIs..prepareAtomic) ⇒
* [~prepareSubAtomic(type, atomics)](#module_Loader API_ Loader APIs..prepareSubAtomic) ⇒
* [~prepareBitmap(type, isContained)](#module_Loader API_ Loader APIs..prepareBitmap) ⇒
* [~prepareEnum(type)](#module_Loader API_ Loader APIs..prepareEnum) ⇒
* [~prepareStruct(type)](#module_Loader API_ Loader APIs..prepareStruct) ⇒
* [~prepareSubAtomicTypes(types)](#module_Loader API_ Loader APIs..prepareSubAtomicTypes)
* [~prepareTypes(zclTypes, types)](#module_Loader API_ Loader APIs..prepareTypes)
* [~prepareAttributeType(attribute, types, cluster)](#module_Loader API_ Loader APIs..prepareAttributeType)
* [~prepareDeviceType(deviceType)](#module_Loader API_ Loader APIs..prepareDeviceType) ⇒
Expand Down Expand Up @@ -21681,6 +21709,19 @@ Parses xml type into the atomic object for insertion into the DB
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the atomic format in the dotdot xml |

<a name="module_Loader API_ Loader APIs..prepareSubAtomic"></a>

### Loader API: Loader APIs~prepareSubAtomic(type, atomics) ⇒
Parses xml type into the sub-atomic object for insertion into the DB

**Kind**: inner method of [<code>Loader API: Loader APIs</code>](#module_Loader API_ Loader APIs)
**Returns**: object ready for insertion into the DB

| Param | Type | Description |
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the sub-atomic format in the dotdot xml |
| atomics | <code>\*</code> | an array of atomic types |

<a name="module_Loader API_ Loader APIs..prepareBitmap"></a>

### Loader API: Loader APIs~prepareBitmap(type, isContained) ⇒
Expand Down Expand Up @@ -21718,6 +21759,17 @@ Parses xml type into the struct object for insertion into the DB
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the struct format in the dotdot xml |

<a name="module_Loader API_ Loader APIs..prepareSubAtomicTypes"></a>

### Loader API: Loader APIs~prepareSubAtomicTypes(types)
Parses array of xml objects that conform to the sub-atomic format in the dotdot xml

**Kind**: inner method of [<code>Loader API: Loader APIs</code>](#module_Loader API_ Loader APIs)

| Param | Type |
| --- | --- |
| types | <code>\*</code> |

<a name="module_Loader API_ Loader APIs..prepareTypes"></a>

### Loader API: Loader APIs~prepareTypes(zclTypes, types)
Expand Down Expand Up @@ -23214,9 +23266,11 @@ This module provides the APIs for ZCL/Data-Model loading.
* [~prepareCommands(commands, side, types)](#module_Loader API_ Loader APIs..prepareCommands) ⇒
* [~prepareCluster(cluster, isExtension, types)](#module_Loader API_ Loader APIs..prepareCluster) ⇒
* [~prepareAtomic(type)](#module_Loader API_ Loader APIs..prepareAtomic) ⇒
* [~prepareSubAtomic(type, atomics)](#module_Loader API_ Loader APIs..prepareSubAtomic) ⇒
* [~prepareBitmap(type, isContained)](#module_Loader API_ Loader APIs..prepareBitmap) ⇒
* [~prepareEnum(type)](#module_Loader API_ Loader APIs..prepareEnum) ⇒
* [~prepareStruct(type)](#module_Loader API_ Loader APIs..prepareStruct) ⇒
* [~prepareSubAtomicTypes(types)](#module_Loader API_ Loader APIs..prepareSubAtomicTypes)
* [~prepareTypes(zclTypes, types)](#module_Loader API_ Loader APIs..prepareTypes)
* [~prepareAttributeType(attribute, types, cluster)](#module_Loader API_ Loader APIs..prepareAttributeType)
* [~prepareDeviceType(deviceType)](#module_Loader API_ Loader APIs..prepareDeviceType) ⇒
Expand Down Expand Up @@ -23481,6 +23535,19 @@ Parses xml type into the atomic object for insertion into the DB
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the atomic format in the dotdot xml |

<a name="module_Loader API_ Loader APIs..prepareSubAtomic"></a>

### Loader API: Loader APIs~prepareSubAtomic(type, atomics) ⇒
Parses xml type into the sub-atomic object for insertion into the DB

**Kind**: inner method of [<code>Loader API: Loader APIs</code>](#module_Loader API_ Loader APIs)
**Returns**: object ready for insertion into the DB

| Param | Type | Description |
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the sub-atomic format in the dotdot xml |
| atomics | <code>\*</code> | an array of atomic types |

<a name="module_Loader API_ Loader APIs..prepareBitmap"></a>

### Loader API: Loader APIs~prepareBitmap(type, isContained) ⇒
Expand Down Expand Up @@ -23518,6 +23585,17 @@ Parses xml type into the struct object for insertion into the DB
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the struct format in the dotdot xml |

<a name="module_Loader API_ Loader APIs..prepareSubAtomicTypes"></a>

### Loader API: Loader APIs~prepareSubAtomicTypes(types)
Parses array of xml objects that conform to the sub-atomic format in the dotdot xml

**Kind**: inner method of [<code>Loader API: Loader APIs</code>](#module_Loader API_ Loader APIs)

| Param | Type |
| --- | --- |
| types | <code>\*</code> |

<a name="module_Loader API_ Loader APIs..prepareTypes"></a>

### Loader API: Loader APIs~prepareTypes(zclTypes, types)
Expand Down Expand Up @@ -25014,9 +25092,11 @@ This module provides the APIs for for common functionality related to loading.
* [~prepareCommands(commands, side, types)](#module_Loader API_ Loader APIs..prepareCommands) ⇒
* [~prepareCluster(cluster, isExtension, types)](#module_Loader API_ Loader APIs..prepareCluster) ⇒
* [~prepareAtomic(type)](#module_Loader API_ Loader APIs..prepareAtomic) ⇒
* [~prepareSubAtomic(type, atomics)](#module_Loader API_ Loader APIs..prepareSubAtomic) ⇒
* [~prepareBitmap(type, isContained)](#module_Loader API_ Loader APIs..prepareBitmap) ⇒
* [~prepareEnum(type)](#module_Loader API_ Loader APIs..prepareEnum) ⇒
* [~prepareStruct(type)](#module_Loader API_ Loader APIs..prepareStruct) ⇒
* [~prepareSubAtomicTypes(types)](#module_Loader API_ Loader APIs..prepareSubAtomicTypes)
* [~prepareTypes(zclTypes, types)](#module_Loader API_ Loader APIs..prepareTypes)
* [~prepareAttributeType(attribute, types, cluster)](#module_Loader API_ Loader APIs..prepareAttributeType)
* [~prepareDeviceType(deviceType)](#module_Loader API_ Loader APIs..prepareDeviceType) ⇒
Expand Down Expand Up @@ -25281,6 +25361,19 @@ Parses xml type into the atomic object for insertion into the DB
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the atomic format in the dotdot xml |

<a name="module_Loader API_ Loader APIs..prepareSubAtomic"></a>

### Loader API: Loader APIs~prepareSubAtomic(type, atomics) ⇒
Parses xml type into the sub-atomic object for insertion into the DB

**Kind**: inner method of [<code>Loader API: Loader APIs</code>](#module_Loader API_ Loader APIs)
**Returns**: object ready for insertion into the DB

| Param | Type | Description |
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the sub-atomic format in the dotdot xml |
| atomics | <code>\*</code> | an array of atomic types |

<a name="module_Loader API_ Loader APIs..prepareBitmap"></a>

### Loader API: Loader APIs~prepareBitmap(type, isContained) ⇒
Expand Down Expand Up @@ -25318,6 +25411,17 @@ Parses xml type into the struct object for insertion into the DB
| --- | --- | --- |
| type | <code>\*</code> | an xml object which conforms to the struct format in the dotdot xml |

<a name="module_Loader API_ Loader APIs..prepareSubAtomicTypes"></a>

### Loader API: Loader APIs~prepareSubAtomicTypes(types)
Parses array of xml objects that conform to the sub-atomic format in the dotdot xml

**Kind**: inner method of [<code>Loader API: Loader APIs</code>](#module_Loader API_ Loader APIs)

| Param | Type |
| --- | --- |
| types | <code>\*</code> |

<a name="module_Loader API_ Loader APIs..prepareTypes"></a>

### Loader API: Loader APIs~prepareTypes(zclTypes, types)
Expand Down
84 changes: 75 additions & 9 deletions src-electron/zcl/zcl-loader-dotdot.js
Original file line number Diff line number Diff line change
Expand Up @@ -456,10 +456,11 @@ function prepareCluster(cluster, types, isExtension = false) {
*/
function prepareAtomic(type) {
let desc = type.$.name
let length = type.restriction?.[0]?.['type:length']?.[0]?.$?.value
return {
name: type.$.short,
id: parseInt(normalizeHexValue(type.$.id)),
size: getNumBytesFromShortName(type.$.short),
size: length ?? getNumBytesFromShortName(type.$.short),
description: desc,
isDiscrete: type.$.discrete == 'true' ? true : false,
isSigned: desc.includes('Signed'),
Expand All @@ -469,6 +470,36 @@ function prepareAtomic(type) {
}
}

/**
* Parses xml type into the sub-atomic object for insertion into the DB
*
* @param {*} type an xml object which conforms to the sub-atomic format in the dotdot xml
* @param {*} atomics an array of atomic types
* @returns object ready for insertion into the DB
*/
function prepareSubAtomic(subAtomic, atomics) {
let relatedAtomic = atomics.find(
(atomic) => atomic.name === subAtomic.inheritsFrom
)
if (relatedAtomic) {
return {
name: subAtomic.short,
id: parseInt(normalizeHexValue(subAtomic.id)),
size: relatedAtomic.size,
description: relatedAtomic.description,
isDiscrete: relatedAtomic.isDiscrete,
isSigned: relatedAtomic.isSigned,
isString: relatedAtomic.isString,
isLong: relatedAtomic.isLong,
isChar: relatedAtomic.isChar
}
} else {
env.log(
`Could not find related atomic type for sub-atomic type ${subAtomic.short} - Dropping this type.`
)
}
}

/**
*
* Parses xml type into the bitmap object for insertion into the DB
Expand Down Expand Up @@ -573,6 +604,17 @@ function prepareStruct(type) {
return ret
}

/**
* Parses array of xml objects that conform to the sub-atomic format in the dotdot xml
*
* @param {*} types
*/
function prepareSubAtomicTypes(types) {
types.subAtomics = types.subAtomics
.map((subAtomic) => prepareSubAtomic(subAtomic, types.atomics))
.filter((subAtomic) => subAtomic != null || subAtomic != undefined)
}

/**
*
* Parses xml types into the types object for insertion into the DB
Expand All @@ -593,14 +635,11 @@ function prepareTypes(zclTypes, types) {
} else if (type.$.inheritsFrom === undefined) {
types.atomics.push(prepareAtomic(type))
} else {
// TODO: Need to handle sub-atomic types, these are types that impose restrictions
// and inherit from an atomic type but are not a struct, bitmap or enum
droppedTypes.push(type.$.name)
// Sub-atomic types - these are types that impose restrictions and inherit from an atomic type but are not a struct, bitmap or enum
// Will be processed after all other types are processed
types.subAtomics.push(type.$)
}
})
if (droppedTypes.length > 0) {
env.logDebug(`Dropped types in DotDot loader: ${droppedTypes}`)
}
}

/**
Expand Down Expand Up @@ -721,6 +760,8 @@ function prepareDataType(a, dataType, typeMap) {
a.name.toLowerCase().includes(dbEnum.zclType.string)
) {
dataTypeRef = typeMap.get(dbEnum.zclType.string)
} else if (!dataType && a.isString !== undefined && a.isString == 1) {
dataTypeRef = typeMap.get(dbEnum.zclType.string)
} else if (
!dataType &&
a.name.toLowerCase().includes(dbEnum.zclType.struct)
Expand Down Expand Up @@ -818,7 +859,14 @@ async function processDataType(db, filePath, packageId, data, dataType) {
function prepareNumber(a, dataType) {
return {
size: a.size,
is_signed: a.name.endsWith('u') || !a.name.includes('int') ? 0 : 1,
is_signed:
a.isSigned !== undefined
? a.isSigned
: a.name.startsWith('u') ||
a.name.endsWith('u') ||
!a.name.includes('int')
? 0
: 1,
name: a.name,
cluster_code: a.cluster ? a.cluster : null,
discriminator_ref: dataType
Expand Down Expand Up @@ -1177,7 +1225,13 @@ async function loadZclData(db, ctx) {
env.logDebug(
`Starting to load Dotdot ZCL data in to DB for: ${ctx.metadataFile}, clusters length=${ctx.zclClusters.length}`
)
let types = { atomics: [], enums: [], bitmaps: [], structs: [] }
let types = {
atomics: [],
enums: [],
bitmaps: [],
structs: [],
subAtomics: []
}
prepareTypes(ctx.zclTypes, types)
prepareTypes(ctx.zclGlobalTypes, types)
let preparedClusters = []
Expand Down Expand Up @@ -1219,6 +1273,8 @@ async function loadZclData(db, ctx) {
await queryLoader.insertAtomics(db, ctx.packageId, types.atomics)
ctx.ZCLDataTypes = ['ARRAY', 'BITMAP', 'ENUM', 'NUMBER', 'STRING', 'STRUCT']
await processDataTypeDiscriminator(db, ctx.packageId, ctx.ZCLDataTypes)
// prepare sub-atomics after all other types are processed
prepareSubAtomicTypes(types)
await processDataType(
db,
ctx.metadataFile,
Expand Down Expand Up @@ -1247,6 +1303,14 @@ async function loadZclData(db, ctx) {
types.structs,
dbEnum.zclType.struct
)
// sub-atomics are in the same format as atomics, so we can use the same prepare function
await processDataType(
db,
ctx.metadataFile,
ctx.packageId,
types.subAtomics,
dbEnum.zclType.atomic
)

await processEnumsFromAtomics(
db,
Expand All @@ -1262,6 +1326,8 @@ async function loadZclData(db, ctx) {
)
await processNumber(db, ctx.metadataFile, ctx.packageId, types.atomics)
await processString(db, ctx.metadataFile, ctx.packageId, types.atomics)
await processNumber(db, ctx.metadataFile, ctx.packageId, types.subAtomics)
await processString(db, ctx.metadataFile, ctx.packageId, types.subAtomics)
await processEnums(db, ctx.metadataFile, ctx.packageId, types.enums)
await processBitmaps(db, ctx.metadataFile, ctx.packageId, types.bitmaps)
await processStruct(db, ctx.metadataFile, ctx.packageId, types.structs)
Expand Down
Loading

0 comments on commit 3326112

Please sign in to comment.