|
| 1 | +# Architecture |
| 2 | + |
| 3 | +This document aims to give a high level overview of the |
| 4 | +Glean JavaScript SDK (aka Glean.js) architecture. |
| 5 | + |
| 6 | +This is a great place to start for newcomers to the project. |
| 7 | + |
| 8 | +For in-depth documentation on specific implementation details of the library, refer to |
| 9 | +the [Glean.js developer documentation](docs/README.md). |
| 10 | + |
| 11 | +## Context |
| 12 | + |
| 13 | +The Glean JavaScript SDK is part of [the Glean project](https://docs.telemetry.mozilla.org/concepts/glean/glean.html). |
| 14 | +An end-to-end data collection platform developed by Mozilla and primarily targeting Mozilla products |
| 15 | +across multiple platforms. |
| 16 | + |
| 17 | +Glean provides multiple client SDKs for different programming languages and platforms. |
| 18 | +One of the aspects that guide Glean SDK development is cross-platform consistency and the Glean |
| 19 | +JavaScript SDK is no exception to that. It is built to work on multiple JavaScript platforms -- |
| 20 | +websites, web extensions, Node.js and QML as of the time of writing -- and to be easily extendable |
| 21 | +to other platforms as well. |
| 22 | + |
| 23 | +The Glean JavaScript SDK is the latest addition to the family of Glean SDKs. The other Glean SDKs, |
| 24 | +namely Kotlin, Swift, Python, Rust and Firefox Desktop SDKs are not housed in this repository, but |
| 25 | +in the [`mozilla/glean`](https://github.com/mozilla/glean) repository and for the case of the |
| 26 | +Firefox Desktop SDK in the [mozilla-central](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean) repository. |
| 27 | + |
| 28 | +## Overview |
| 29 | + |
| 30 | +On a very high level a Glean SDK is a library which exposes APIs for users to record |
| 31 | +structured data and submit that data in an (again) structured format to a specific endpoint. |
| 32 | + |
| 33 | +Users cannot simply record arbitrary data though. The Glean SDKs expose specialized metrics APIs for |
| 34 | +different collection needs. The SDK is responsible to validating that the data given by a user is in |
| 35 | +the correct format and only then recording it (or recording an error in case the data provided is |
| 36 | +not correct). |
| 37 | + |
| 38 | +When data is submitted, the Glean SDK is responsible for assembling the correct group (aka ping) of data points |
| 39 | +(aka metrics) in the format expected by the Glean pipeline, uploading the data and managing the local |
| 40 | +storage. Each metric can have different [lifetimes](https://mozilla.github.io/glean/book/user/metrics/adding-new-metrics.html#a-lifetime-example) |
| 41 | +and the SDK will manage its storage so that data does not remain in storage after it's lifetime is expired. |
| 42 | + |
| 43 | +The Glean SDK tries to do all of this is the least disruptive way possible to users. All of the |
| 44 | +SDKs tasks are queued and executed asynchronously. The APIs exposed by the Glean SDK will only do |
| 45 | +the en-queuing of tasks, a quick synchronous operation. Internally, the Glean SDK will handle the |
| 46 | +queued tasks asynchronously, catching any errors thrown along the way so that Glean never |
| 47 | +crashes a users application. |
| 48 | + |
| 49 | +## Code Map |
| 50 | + |
| 51 | +The Glean JavaScript SDK source code lives under the `glean/src` folder. |
| 52 | + |
| 53 | +``` |
| 54 | +├── core/ # Contains the bulk of the library's implementation |
| 55 | +├── entry/ # Contains the different general API entry points for each platform |
| 56 | +├── platform/ # Contains platforms specific implementations for specific modules |
| 57 | +├── plugins/ # Contains plugins implementations |
| 58 | +├── cli.ts # Contains the Glean CLI implementation |
| 59 | +├── metrics.yaml |
| 60 | +└── pings.yaml |
| 61 | +``` |
| 62 | + |
| 63 | +### `core/` |
| 64 | + |
| 65 | +The `core/` folder is where developers will probably spend most of their time |
| 66 | +when working on Glean.js. This folder contains the majority of the library implementation. |
| 67 | + |
| 68 | +All of the code under this folder is expected to work across all platforms. Platform specific code |
| 69 | +is centralized on the `platform/` folder which is covered in a later section of this document. |
| 70 | + |
| 71 | +`glean.ts` contains the Glean [general API](https://mozilla.github.io/glean/book/reference/general/index.html) |
| 72 | +implementation where Glean state is initialized and managed. Due to its centralizer aspect, |
| 73 | +this file inevitably imports many of the other files on the library. As such, |
| 74 | +developers should avoid importing this file in any of the other files on the `core/` folder, |
| 75 | +because that can quickly cause a circular dependency mess. |
| 76 | + |
| 77 | +`context.ts` is where circular dependencies go to die. When Glean.js was first implemented, |
| 78 | +`glean.ts` not only managed and initialized Glean state, but it also exposed Glean state to the rest |
| 79 | +of the library. That caused a huge issue with circular dependencies. The `context.ts` file was |
| 80 | +created to mitigate that issue. This file houses the `Context` structure, a singleton that holds |
| 81 | +setters and getters to all of Glean's internal state. This file should avoid any imports that are |
| 82 | +not `import type`. |
| 83 | + |
| 84 | +`dispatcher.ts` is where Glean's dispatcher implementation lives. The dispatcher is the structure |
| 85 | +that manages Glean's task queue. |
| 86 | + |
| 87 | +`metrics/types` is where all of the specific metric type implementations are. Each metric type |
| 88 | +should be implemented in a single file and no other file types should be added to this folder, |
| 89 | +because the files under this folder are exposed through the `@mozilla/glean/private/metrics/*` |
| 90 | +entry point. |
| 91 | + |
| 92 | +To see all the exposed entry points, check out Glean.js' `package.json` file. |
| 93 | + |
| 94 | +### `entry/` |
| 95 | + |
| 96 | +The `entry/` folder contains the main entry points for the Glean.js package per platform. |
| 97 | +For example, when a user does `import Glean from @mozilla/glean/webext` it's the `entry/webext.ts` |
| 98 | +file that they are getting and not `core/glean.ts`. |
| 99 | + |
| 100 | +The main difference between each platform's file is that a different `Platform` implementation is |
| 101 | +imported per file. |
| 102 | + |
| 103 | +The Qt/QML entry point is the different one here. QML packages cannot be easily consumed through npm, |
| 104 | +so the QML entry point imports all of Glean's modules and exposes it through this entry point. That is |
| 105 | +what is done on the `qt.ts` file. The `qt.js` file is the file QML users actually interact with, |
| 106 | +it includes specific QML semantics and is copied to the final destination folder of the QML package |
| 107 | +as-is after Glean.js is compiled for the QML target. (See `bin/prepare-qml-module.sh` for more |
| 108 | +context.) |
| 109 | + |
| 110 | +### `platform/` |
| 111 | + |
| 112 | +Some modules such as storage and uploader, cannot be written in such a way that works |
| 113 | +for all platforms, because each platform provides different APIs for these tasks. In order |
| 114 | +for useless platform specific code not to bloat the size of the library on each platform, |
| 115 | +the `platform/` module contains implementations of identical interfaces in different platforms. |
| 116 | + |
| 117 | +This allows the pattern of only importing the necessary implementation of these modules on each |
| 118 | +platform. It also makes testing easier, because the exact same suite of tests can run for each of |
| 119 | +the platform specific implementation, thus guaranteeing that each modules works exactly the same |
| 120 | +in all platforms. |
| 121 | + |
| 122 | +### `plugins/` |
| 123 | + |
| 124 | +The `plugins/` folder contains the Glean.js' plugins code. |
| 125 | + |
| 126 | +Each plugin is expected to be implemented in a single file, because these files are |
| 127 | +exposed through the `@mozilla/glean/plugins/*` entry point. |
| 128 | + |
| 129 | +### `cli.ts` |
| 130 | + |
| 131 | +This file is what gets executed when a user that has installed Glean through npm invokes the `glean` |
| 132 | +command. It serves as a wrapper to call `glean_parser` commands more easily from JavaScript projects. |
0 commit comments