From a9c8aa3b8ab8a93198bfc79ba2a62b987821be04 Mon Sep 17 00:00:00 2001 From: Raphael Kubo da Costa Date: Tue, 21 Nov 2023 17:19:14 +0100 Subject: [PATCH] Add automation support via WebDriver This PR integrates with the automation concepts defined in https://w3c.github.io/sensors/#automation to allow providing motion or orientation readings via virtual sensors through the WebDriver extension commands defined there. IMPORTANT: This does not mean that this specification requires implementations to support the Generic Sensor API specification and its derived specifications. Only the Automation section is being referenced, and when necessary some algorithms and definitions are being duplicated here, especially for Device Motion automation. The definitions of the "accelerometer", "linear-acceleration", "gyroscope", "absolute-orientation", and "relative-orientation" virtual sensor types have been moved here from their original specifications. As suggested in #124 and similarly to what has been discussed in PRs #121 and #123, since this specification is further along the Rec track and is implemented by more engines, it makes sense to have the exported definitions here and reference them from the other specs instead. Device Motion: - Readings are controlled via the "accelerometer", "linear-acceleration" and "gyroscope" virtual sensor types. Device Orientation: - The "absolute-orientation" and "relative-orientation" virtual sensor types are defined in this specification, along with a parsing algorithm that reads alpha, beta and gamma doubles. Compared to device motion, however, we have an additional problem in that device orientation data is provided in Euler angles and Orientation Sensor uses quaternions. The idea is to parse readings by reading alpha/beta/gamma keys from the WebDriver extension command and use them in the "fire orientation event" algorithm and skip using quaternions altogether. The Orientation Sensor spec can then augment the "parse orientation data reading" algorithm with the required steps to both 1) accept a "quaternion" key in the WebDriver extension command (and convert it to Euler angles as well) and 2) derive a quaternion from the alpha/beta/gamma Euler angles. Fixes #122. --- index.bs | 193 ++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 164 insertions(+), 29 deletions(-) diff --git a/index.bs b/index.bs index b41dbd8..efc3484 100644 --- a/index.bs +++ b/index.bs @@ -19,6 +19,15 @@ Include MDN Panels: if possible Issue Tracking: DeviceOrientation Event Specification Issues Repository https://github.com/w3c/deviceorientation/issues Markup Shorthands: css no +
+urlPrefix: https://w3c.github.io/sensors/; spec: GENERIC-SENSOR
+    type: dfn
+        text: can provide readings flag; url: virtual-sensor-can-provide-readings-flag
+        text: virtual sensor mapping; url: virtual-sensor-mapping
+urlPrefix: https://w3c.github.io/webdriver/; spec: WEBDRIVER2
+    type: dfn
+        text: get a property; url: dfn-getting-properties
+
Conformance requirements {#conformance-requirements} ==================================================== @@ -320,12 +329,23 @@ The requestPermission(absolute To fire an orientation event given a event name (a string), window (a {{Window}}) and absolute (a boolean): + 1. Let orientation be null. + 1. Let topLevelTraversable be window's navigable's top-level traversable. + 1. Let virtualSensorType be "relative-orientation" if absolute is false, and "absolute-orientation" otherwise. + 1. If topLevelTraversable's virtual sensor mapping contains virtualSensorType: + 1. Let virtualSensor be topLevelTraversable's virtual sensor mapping[virtualSensorType]. + 1. If virtualSensor's can provide readings flag is true: + 1. Set orientation to the latest readings provided to virtualSensor with the "alpha", "beta", and "gamma" keys. + 1. Otherwise: + 1. If absolute is false: + 1. Set orientation to the device's relative orientation in the tridimensional plane. + 1. Otherwise: + 1. Set orientation to the device's absolute orientation in the tridimensional plane. + 1. Let permissions be null. 1. If absolute is false: - 1. Let orientation be the device's relative orientation in the tridimensional plane. - 1. Let permissions be « "accelerometer", "gyroscope" ». + 1. Set permissions to « "accelerometer", "gyroscope" ». 1. Otherwise: - 1. Let orientation be the device's absolute orientation in the tridimensional plane. - 1. Let permissions be « "accelerometer", "gyroscope", "magnetometer" ». + 1. Set permissions to « "accelerometer", "gyroscope", "magnetometer" ». 1. Let environment be window's relevant settings object. 1. Run these steps in parallel: 1. For each permission name in permissions: @@ -344,6 +364,8 @@ To fire an orientation event given a event name (a string) A significant change in orientation indicates a difference in orientation values compared to the previous ones that warrants the firing of a deviceorientation or deviceorientationabsolute event. The process of determining whether a significant change in orientation has occurred is implementation-defined, though a maximum threshold for change of 1 degree is recommended. Implementations may also consider that it has occurred if they have reason to believe that the page does not have sufficiently fresh data. +Note: Implementations must take [[#automation]] into account to determine whether a significant change in orientation has occurred, so that a virtual sensor reading update causes it to be assessed. +
Whenever a significant change in orientation occurs, the user agent must execute the following steps on a navigable's active window window: @@ -528,40 +550,59 @@ At an implementation-defined interval interval, the user agent 1. Let document be window's associated Document. 1. For each policy of « "accelerometer", "gyroscope" »: - 1. If document is not allowed to use the policy-controlled feature named policy, return. + 1. If document is not allowed to use the policy-controlled feature named policy, return. + 1. Let topLevelTraversable be window's navigable's top-level traversable. + 1. Let platformLinearAcceleration be null. + 1. If topLevelTraversable's virtual sensor mapping contains "linear-acceleration": + 1. Let virtualSensor be topLevelTraversable's virtual sensor mapping["linear-acceleration"]. + 1. If virtualSensor's can provide readings flag is true, then set platformLinearAcceleration to the latest readings provided to virtualSensor. + 1. Otherwise, if the implementation is able to provide linear acceleration: + 1. Set platformLinearAcceleration to the device's linear acceleration along the X, Y and Z axes. 1. Let acceleration be null. + 1. If platformLinearAcceleration is not null: + 1. Set acceleration to a new {{DeviceMotionEventAcceleration}} created in window's realm. + 1. Set acceleration's x axis acceleration to platformLinearAcceleration's value along the X axis, or null if it cannot be provided. + 1. If acceleration's x axis acceleration is not null, limit its precision to no more than 0.1 m/s2. + 1. Set acceleration's y axis acceleration to platformLinearAcceleration's value along the Y axis, or null if it cannot be provided. + 1. If acceleration's y axis acceleration is not null, limit its precision to no more than 0.1 m/s2. + 1. Set acceleration's z axis acceleration to platformLinearAcceleration's value along the Z axis, or null if it cannot be provided. + 1. If acceleration's z axis acceleration is not null, limit its precision to no more than 0.1 m/s2. + 1. Let platformAccelerationIncludingGravity be null. + 1. If topLevelTraversable's virtual sensor mapping contains "accelerometer": + 1. Let virtualSensor be topLevelTraversable's virtual sensor mapping["accelerometer"]. + 1. If virtualSensor's can provide readings flag is true, then set platformAccelerationIncludingGravity to the latest readings provided to virtualSensor. + 1. Otherwise, if the implementation is able to provide acceleration with gravity: + 1. Set platformAccelerationIncludingGravity to the device's linear acceleration along the X, Y and Z axes. 1. Let accelerationIncludingGravity be null. + 1. If platformAccelerationIncludingGravity is not null: + 1. Set accelerationIncludingGravity to a new {{DeviceMotionEventAcceleration}} created in window's realm. + 1. Set accelerationIncludingGravity's x axis acceleration to platformAccelerationIncludingGravity's value along the X axis, or null if it cannot be provided. + 1. If accelerationIncludingGravity's x axis acceleration is not null, limit its precision to no more than 0.1 m/s2. + 1. Set accelerationIncludingGravity's y axis acceleration to platformAccelerationIncludingGravity's value along the Y axis, or null if it cannot be provided. + 1. If accelerationIncludingGravity's y axis acceleration is not null, limit its precision to no more than 0.1 m/s2. + 1. Set accelerationIncludingGravity's z axis acceleration to platformAccelerationIncludingGravity's value along the Z axis, or null if it cannot be provided. + 1. If accelerationIncludingGravity's z axis acceleration is not null, limit its precision to no more than 0.1 m/s2. + 1. Let platformRotationRate be null. + 1. If topLevelTraversable's virtual sensor mapping contains "gyroscope": + 1. Let virtualSensor be topLevelTraversable's virtual sensor mapping["gyroscope"]. + 1. If virtualSensor's can provide readings flag is true, then set platformRotationRate to the latest readings provided to virtualSensor. + 1. Otherwise, if the implementation is able to provide rotation rate: + 1. Set platformRotationRate to the device's rotation rate along the X, Y and Z axes. 1. Let rotationRate be null. + 1. If platformRotationRate is not null: + 1. Set rotationRate to a new {{DeviceMotionEventRotationRate}} created in window's realm. + 1. Set rotationRate's x axis rotation rate to platformRotationRate's value about the X axis, or null if it cannot be provided. + 1. If rotationRate's x axis rotation rate is not null, limit its precision to no more than 0.1 deg/s. + 1. Set rotationRate's y axis rotation rate to platformRotationRate's value about the Y axis, or null if it cannot be provided. + 1. If rotationRate's y axis rotation rate is not null, limit its precision to no more than 0.1 deg/s. + 1. Set rotationRate's z axis rotation rate to platformRotationRate's value about the Z axis, or null if it cannot be provided. + 1. If rotationRate's z axis rotation rate is not null, limit its precision to no more than 0.1 deg/s. 1. Let environment be window's relevant settings object. 1. Run these steps in parallel: 1. For each permission name in « "accelerometer", "gyroscope" »: 1. Let state be the result of getting the current permission state with permission name and environment. 1. If state is not "granted", return. 1. Queue a global task on the device motion and orientation task source given window to run the following steps: - 1. If the implementation is able to provide linear acceleration: - 1. Set acceleration to a new {{DeviceMotionEventAcceleration}} created in window's realm. - 1. Set acceleration's x axis acceleration to the device's linear acceleration along the X axis, or null if it cannot be provided. - 1. If acceleration's x axis acceleration is not null, limit its precision to no more than 0.1 m/s2. - 1. Set acceleration's y axis acceleration to the device's linear acceleration along the Y axis, or null if it cannot be provided. - 1. If acceleration's y axis acceleration is not null, limit its precision to no more than 0.1 m/s2. - 1. Set acceleration's z axis acceleration to the device's linear acceleration along the Z axis, or null if it cannot be provided. - 1. If acceleration's z axis acceleration is not null, limit its precision to no more than 0.1 m/s2. - 1. If the implementation is able to provide acceleration with gravity: - 1. Set accelerationIncludingGravity to a new {{DeviceMotionEventAcceleration}} created in window's realm. - 1. Set accelerationIncludingGravity's x axis acceleration to the device's acceleration with gravity along the X axis, or null if it cannot be provided. - 1. If accelerationIncludingGravity's x axis acceleration is not null, limit its precision to no more than 0.1 m/s2. - 1. Set accelerationIncludingGravity's y axis acceleration to the device's acceleration with gravity along the Y axis, or null if it cannot be provided. - 1. If accelerationIncludingGravity's y axis acceleration is not null, limit its precision to no more than 0.1 m/s2. - 1. Set accelerationIncludingGravity's z axis acceleration to the device's acceleration with gravity along the Z axis, or null if it cannot be provided. - 1. If accelerationIncludingGravity's z axis acceleration is not null, limit its precision to no more than 0.1 m/s2. - 1. If the implementation is able to provide rotation rate: - 1. Set rotationRate to a new {{DeviceMotionEventRotationRate}} created in window's realm. - 1. Set rotationRate's x axis rotation rate to the device's rotation rate about the X axis, or null if it cannot be provided. - 1. If rotationRate's x axis rotation rate is not null, limit its precision to no more than 0.1 deg/s. - 1. Set rotationRate's y axis rotation rate to the device's rotation rate about the Y axis, or null if it cannot be provided. - 1. If rotationRate's y axis rotation rate is not null, limit its precision to no more than 0.1 deg/s. - 1. Set rotationRate's z axis rotation rate to the device's rotation rate about the Z axis, or null if it cannot be provided. - 1. If rotationRate's z axis rotation rate is not null, limit its precision to no more than 0.1 deg/s. 1. Fire an event named devicemotion at window, using {{DeviceMotionEvent}}, with the {{DeviceMotionEvent/acceleration}} attribute initialized to acceleration, the {{DeviceMotionEvent/accelerationIncludingGravity}} attribute initialized to accelerationIncludingGravity, the {{DeviceMotionEvent/rotationRate}} attribute initialized to rotationRate, and the {{DeviceMotionEvent/interval}} attribute initialized to interval.
@@ -622,6 +663,100 @@ A mapping Web application uses the device's orientation to correctly align the m * The specification must be agnostic to the underlying sources of orientation and motion data. * The specification must use the existing DOM event framework. +Automation {#automation} +========== + +This specification can pose a challenge to test authors, as the events defined here depend on the presence of physical hardware whose readings cannot be easily controlled. + +To address this challenge, this document builds upon the [[WEBDRIVER2]] extension commands and infrastructure laid out by [[GENERIC-SENSOR#automation]]. This was chosen over the option of developing completely new and independent infrastructure with separate extension commands because there is significant overlap between the two specifications: not only does testing the [[GENERIC-SENSOR]] specification present similar challenges, but many derived APIs (e.g. [[GYROSCOPE]]) obtain and provide similar information. + +This specification only requires implementations to support the [[GENERIC-SENSOR#automation]] section of the [[GENERIC-SENSOR]] specification, not its interfaces and events. + +Device Orientation Automation {#device-orientation-automation} +----------------------------- + +Automation support for the deviceorientation event is built upon virtual sensors that represent accelerometers, gyroscopes and, optionally, magnetometers. + +Orientation data retrieved from the platform by the user agent comes from accelerometers, gyroscopes and, optionally, magnetometers. Contrary to motion data, however, these lower-level readings must be transformed into Euler angles in the formation described in [[#device-orientation-model]]. Furthermore, the platform might provide extra APIs to the user agent that already perform some of those conversions from raw acceleration and rotation data. + +Therefore, instead of requiring implementations (and automation users) to provide orientation readings via lower-level virtual sensors which use different units of measurement, this specification defines extra virtual sensor types for relative and orientation data in the format used by this specification. + +### Parse orientation reading data algorithm ### {#parse-orientation-data-reading-algorithm} + +
+To perform the parse orientation data reading algorithm, given a JSON {{Object}} parameters: + + 1. Let alpha be the result of invoking get a property from parameters with "alpha". + 1. If alpha is not a {{Number}}, or its value is NaN, +∞, or −∞, return undefined. + 1. If alpha is not in the range [0, 360), then return undefined. + 1. Let beta be the result of invoking get a property from parameters with "beta". + 1. If beta is not a {{Number}}, or its value is NaN, +∞, or −∞, return undefined. + 1. If beta is not in the range [-180, 180), then return undefined. + 1. Let gamma be the result of invoking get a property from parameters with "gamma". + 1. If gamma is not a {{Number}}, or its value is NaN, +∞, or −∞, return undefined. + 1. If gamma is not in the range [-90, 90), then return undefined. + 1. Let reading be a new map. + 1. Set reading["alpha"] to alpha. + 1. Set reading["beta"] to beta. + 1. Set reading["gamma"] to gamma. + 1. Return reading. + +Note: reading is defined as a map in the algorithm above to prevent a dependency on the sensor reading concept from the [[GENERIC-SENSOR]] specification. They should be interchangeable for the purposes of the algorithm above. + +
+ +### The "absolute-orientation" virtual sensor type ### {#absolute-orientation-virtual-sensors} + +The per-type virtual sensor metadata map must have the following entry: +: +: key +:: "absolute-orientation" +: value +:: A virtual sensor metadata whose reading parsing algorithm is parse orientation data reading. + +### The "relative-orientation" virtual sensor type ### {#relative-orientation-virtual-sensors} + +The per-type virtual sensor metadata map must have the following entry: +: +: key +:: "relative-orientation" +: value +:: A virtual sensor metadata whose reading parsing algorithm is parse orientation data reading. + +Device Motion Automation {#device-motion-automation} +------------------------ + +The motion data retrieved from the platform by the user agent comes from accelerometers and gyroscopes. This specification defines certain per-type virtual sensor metadata entries that are shared with the [[ACCELEROMETER]] and [[GYROSCOPE]] specifications. + +Accelerometer virtual sensors are used to provide acceleration with gravity data to the platform. Linear Acceleration virtual sensors are used to provide linear acceleration data to the platform. Gyroscope virtual sensors are used to provide rotation rate data to the platform. + +### The "accelerometer" virtual sensor type ### {#accelerometer-virtual-sensors} + +The per-type virtual sensor metadata map must have the following entry: + +: key +:: "accelerometer" +: value +:: A virtual sensor metadata whose reading parsing algorithm is parse xyz reading. + +### The "linear-acceleration" virtual sensor type ### {#linear-acceleration-virtual-sensors} + +The per-type virtual sensor metadata map must have the following entry: + +: key +:: "linear-acceleration" +: value +:: A virtual sensor metadata whose reading parsing algorithm is parse xyz reading. + +### The "gyroscope" virtual sensor type ### {#gyroscope-virtual-sensors} + +The per-type virtual sensor metadata map must have the following entry: + +: key +:: "gyroscope" +: value +:: A virtual sensor metadata whose reading parsing algorithm is parse xyz reading. +

A Examples

This section is non-normative.