Skip to content

Commit a9bc3c9

Browse files
tilenkavcicgithub-actions[bot]
tilenkavcic
and
github-actions[bot]
authored
chore: Move chef-cli docs to fern (#186)
* Fix bash code * Make diagram look better * Adjust colors * Merge step-by-step with existing * Add missing things to metadata extraction * Add schema and validation info * Make faq accordion group * Improve metadata extraction * Add IDM and the rest from chef * Fix path * Revert diagram change * Update faq * Proofread IDM * Proofread fax and metadata extraction * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Apply suggestions from EkLine Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix wrapping * Fix title * Refine styling * Add code highlight * Small visual change * Refine rich text docs * Don't use ellipsis * Address comments * Fix getting started overview * Apply EkLine suggestions Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Address comments * Adjust code indent, address comments --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1 parent f8b1c2b commit a9bc3c9

File tree

9 files changed

+733
-68
lines changed

9 files changed

+733
-68
lines changed

‎fern/docs/pages/airdrop/data-extraction.mdx

Lines changed: 25 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
21
In the data extraction phase, the extractor is expected to call the external system's APIs
32
to retrieve all the items that were updated since the start of the last extraction.
43
If there was no previous extraction (the current run is an initial import),
@@ -43,12 +42,11 @@ Each artifact is submitted with an `item_type`, defining a separate domain objec
4342
external system and matching the `record_type` in the provided metadata.
4443
Item types defined when uploading extracted data must validate the declarations in the metadata file.
4544

46-
Extracted data must be normalized.
47-
45+
Extracted data must be normalized:
4846
- Null values: All fields without a value should either be omitted or set to null.
49-
For example, if an external system provides values such as "", -1 for missing values,
47+
For example, if an external system provides values such as "", –1 for missing values,
5048
those must be set to null.
51-
- Timestamps: Full-precision timestamps should be formatted as RFC3999 (`1972-03-29T22:04:47+01:00`),
49+
- Timestamps: Full-precision timestamps should be formatted as RFC3339 (`1972-03-29T22:04:47+01:00`),
5250
and dates should be just `2020-12-31`.
5351
- References: references must be strings, not numbers or objects.
5452
- Number fields must be valid JSON numbers (not strings).
@@ -58,7 +56,7 @@ Each line of the file contains an `id` and the optional `created_date` and `modi
5856
in the beginning of the record.
5957
All other fields are contained within the `data` attribute.
6058

61-
```json
59+
```json {2-4}
6260
{
6361
"id": "2102e01F",
6462
"created_date": "1972-03-29T22:04:47+01:00",
@@ -74,9 +72,28 @@ All other fields are contained within the `data` attribute.
7472
}
7573
```
7674

75+
## Validating extracted data
76+
77+
Extracted artifacts can be validated with the chef-cli using the following command:
78+
79+
```bash
80+
chef-cli validate-data -m external_domain_metadata.json -r issue < extractor_issues_2.json
81+
```
7782

78-
Extracted artifacts can be validated with the `chef-cli` using the following command:
83+
You can also generate example data to show the format the data has to be normalized to, using:
7984

8085
```bash
81-
$ chef-cli validate-metadata -m external_domain_metadata.json -r issue < extractor_issues_2.json
86+
echo '{}' | chef-cli fuzz-extracted -r issue -m external_domain_metadata.json > example_issues.json
8287
```
88+
89+
## Deploying and testing the snap-in
90+
91+
Once you have implemented data extraction, you should deploy your snap-in to your test organization and run an import.
92+
93+
To deploy the snap-in, run `make auth` and `make deploy` in the snap-in repository.
94+
Then, activate the snap-in by running `devrev snap_in activate`.
95+
96+
After activation, you can create an import in the DevRev UI, which will initially reach the 'waiting for user input' stage.
97+
During this phase, you can verify your data extraction implementation is working correctly.
98+
99+
Relevant documentation can be found in the [Snap-in development](/snapin-development/locally-testing-snap-ins) section.

‎fern/docs/pages/airdrop/data-model/mapping-reasons.mdx

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
Airdrop uses transformation methods to map data from one data model to another.
2+
These methods operate at the individual field level, meaning they transform data field-by-field rather than transforming entire records at once.
3+
Each transformation method has specific requirements regarding the field types and data formats it can work with.
4+
If the source data field does not meet these requirements for a particular transformation method, that mapping option is not available for selection during the mapping configuration process.
5+
6+
There are several reasons why some mappings might be unavailable.
7+
8+
1. A common reason is mismatch of types. For example, if a DevRev field is expected to be `rich_text`,
9+
but the field is set as `text` mapping to some fields is unavailable.
10+
Refer to the [supported types](../metadata-extraction.mdx#declare-fields-for-each-record-type)
11+
section and the general DevRev documentation for more information.
12+
2. Only references can be mapped to references. Ensure that source system fields are correctly
13+
mapped to reference fields in DevRev.
14+
3. Support for the `struct` type is limited. Marking a field as a struct in the metadata schema will
15+
make it unavailable for mapping outside of using the custom jq transformation method.
16+
Refer to the [metadata tips](../faq.mdx#metadata-extraction) for more information.
17+
4. Links are supported only on works and conversations.
18+
19+
## Transformation methods
20+
The following tables outline the available transformation methods categorized by their applicable field types:
21+
22+
### Custom fields transformation methods
23+
24+
| Method | Description | Requirements |
25+
|--------|-------------|-------------|
26+
| `make_constrained_simple_value` | Propagates validation constraints from the external system and enforces those in DevRev | - |
27+
| `make_enum` | Produces an enum field | External field must be of type enum |
28+
| `make_uenum` | Produces an enum field | External field must be of type enum |
29+
| `reference_custom_field` | Produces a reference field | External field must be of type reference |
30+
31+
### Metadata extraction transformation methods
32+
33+
| Method | Description | Requirements |
34+
|--------|-------------|-------------|
35+
| `make_custom_stages` | Makes custom stages | Requires stage diagram data in the domain metadata |
36+
| `map_enum` | Produces an enum field | External field must be of type enum |
37+
| `map_roles` | Maps permission roles from external to DevRev format | - |
38+
| `use_as_array_value` | Produces an array field | External field must be a scalar (single-value) |
39+
| `use_devrev_record` | Enables use of a fixed reference to something in DevRev | DON should be of the right type |
40+
| `use_directly` | Identity operator that returns exactly the input | External field must be of the same type as the DevRev field; if external field is an array, internal field must also be an array |
41+
| `use_fixed_value` | Produces a fixed value in DevRev | Only available for boolean or enum DevRev fields |
42+
43+
### Custom and stock fields transformation methods
44+
45+
| Method | Description | Requirements |
46+
|--------|-------------|-------------|
47+
| `use_rich_text` | Produces a rich text field | External field must be of type `rich_text` |
48+
49+
### Constructed custom fields transformation methods
50+
51+
| Method | Description | Requirements |
52+
|--------|-------------|-------------|
53+
| `construct_text_field` | Produces a text field | External field must be of type text |
54+
55+
### Universal transformation method
56+
57+
| Method | Description | Requirements |
58+
|--------|-------------|-------------|
59+
| `use_raw_jq` | Enables the use of `jq` to transform data | Can be used on all fields, but should be used sparingly, only if no other transformation method is available |

‎fern/docs/pages/airdrop/data-model/permissions.mdx

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
DevRev uses the concept of permissions to control access to records.
2+
Permissions are associated with users or groups and define the level of access they have to leaf types and their fields.
3+
4+
## Article permissions
5+
6+
{/* TODO: more on article permissions */}
7+
Articles can be shared with users or groups.
8+
9+
The `shared_with` field allows to specify the permission level for each user or group using the `permission` type.
10+
The `permission` type is a special structure associating a reference to an user-like record type (the field `member_id`)
11+
with an `enum` value that provides the user with the role or permission level.
12+
## Platform groups
13+
14+
Platform groups are groups automatically created by the DevRev platform for every organization.
15+
For example, DevRev provides *Dev users* and *Rev users* groups that contain all the Dev and Rev users respectively.
16+
17+
We allow developers to map their external default groups to platform groups.
18+
This functionality allows for flexible integration with the platform's permission system while allowing for re-mapping by the end user.
19+
20+
To implement this mapping, developers need to define a new object in external metadata with one enum field containing the possible values of default groups available in the external system. During the initial domain mapping, developers can map this object to the *platform groups* object in DevRev. The platform groups object can then be referenced in other objects, such as in the `shared_with` field of articles.
21+
22+
{/* ### Authorization policy */}

‎fern/docs/pages/airdrop/data-model/rich-text.mdx

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
Rich text is used in DevRev to represent text fields that can be formatted and can contain mentions, for example, description of an issue or body of a conversation.
2+
3+
A simple rich text looks like one markdown string wrapped in an array: `["Hello **world**!"]`.
4+
Markdown must conform to [CommonMark Spec v0.30](https://spec.commonmark.org/0.30).
5+
6+
## Rich text mentions
7+
8+
To support mentions, `rich_text` can be formatted as an array of strings and mention objects like so:
9+
```json
10+
[
11+
"Hello ",
12+
{"ref_type":"external_user_type", "id":"1...", "fallback_record_name": "John Doe"},
13+
"how are you?"
14+
]
15+
```
16+
17+
18+
Mention represents any mention (user, issue, etc.) in rich text and is defined as:
19+
| Field | Type | Required | Description |
20+
| ---------------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------ |
21+
| `id` | String | Yes | Identifier of the item being mentioned. This could be a user ID or any other identifier, in the format used by the source system. |
22+
| `ref_type` | String | Yes | Type of the item being mentioned. Examples include "issue", "comment", etc. The recipe converts this according to user mappings. |
23+
| `fallback_record_name` | String | No | The text to display if the mention cannot be resolved. This could be a user's display name or a ticket title, for instance. |
24+
25+
In reverse, the loader should expect the following structure:
26+
```json
27+
{
28+
"type": "rich_text",
29+
"content": [
30+
"Hello ",
31+
{
32+
"ref_type": "external_user_type",
33+
"id": "don:identity:dvrv-us-1:devo/xyz:devu/1",
34+
"fallback_record_name": "John Smith"
35+
},
36+
"how are you?"
37+
]
38+
}
39+
```
40+
41+
## Importing articles
42+
43+
An article is a document containing essential information about the company's products, services, and processes.
44+
Articles support Markdown as well as HTML.
45+
46+
Articles support mentions to artifacts and articles. An inline attachment must be mapped to an artifact.
47+
A link to another article must be mapped to an article.
48+
49+
### Managing permissions
50+
51+
You can manage article permissions in the `shared_with` field. Permissions can reference users, groups, and [platform groups](./platform_groups.md).
52+
53+
### Inline attachments
54+
If an inline attachment is hosted in the source system, it must be created as an artifact in DevRev.
55+
The same link cannot be used as the attachment is deleted in the source system when our customers deactivate the account.
56+
However, creating an artifact is not enough. The artifact must be linked in the appropriate place in the article content.
57+
58+
The following HTML example shows an inline attachment:
59+
```html
60+
<img src="don:core:dvrv-us-1:devo/0:artifact/1" alt="Alt Text"/>
61+
```
62+
63+
The following Markdown example shows an inline attachment:
64+
```markdown
65+
![Alt Text](don:core:dvrv-us-1:devo/0:artifact/1)
66+
```
67+
68+
Let's say the content of your external system looks like this:
69+
```html
70+
<p>
71+
This is an article with one image.
72+
</p>
73+
<p>
74+
<img src="https://devrev.zendesk.com/hc/article_attachments/29908544740244" alt="download.jpeg">
75+
</p>
76+
```
77+
78+
The content in DevRev should look like this:
79+
```html
80+
<p>
81+
This is an article with one image.
82+
</p>
83+
<p>
84+
<img src="don:core:dvrv-us-1:devo/0:artifact/1" alt="download.jpeg">
85+
</p>
86+
```
87+
88+
`don:core:dvrv-us-1:devo/0:artifact/1` is the ID of the artifact created in DevRev corresponding to the attachment with ID `29908544740244` in the external source system.
89+
To achieve this, you need to transform the content of the article to the following JSON:
90+
```json
91+
[
92+
"<p>This is an article with one image.</p><p><img src=\"",
93+
{
94+
"ref_type": "artifact",
95+
"id": "29908544740244",
96+
"fallback_record_name": "<fallback link>"
97+
},
98+
"\" alt=\"download.jpeg\"></p>"
99+
]
100+
```
101+
102+
The `ref_type` should be set to artifact and the ID should be the ID of the attachment in the external source system.
103+
The platform replaces the mention block with the ID of the corresponding artifact.
104+
The resolved value is not wrapped in double quotes.
105+
106+
### Links to other articles
107+
If there is a link to another article in the content of the article, you need to create a mention to the article.
108+
The link must be to an article that was either created in previous syncs or is created in the current sync.
109+
At the extractor stage, it is impossible to predict the ID of the article that is created in DevRev.
110+
This is why the platform handles this. This feature is only available for the HTML format.
111+
However, since Markdown can contain HTML, you can use the same approach for Markdown as well.
112+
113+
The following HTML example shows a link to another article:
114+
```html
115+
<a data-article-id="don:core:dvrv-us-1:devo/0:article/10" href="/ART-10" target="_self">
116+
Contact our Support Team
117+
</a>
118+
```
119+
120+
Let's say the content of your external system looks like this:
121+
```html
122+
<p>
123+
You can create an account and log-in
124+
<a href="https://devrev.zendesk.com/hc/en-us/articles/360059607772" target="_self">
125+
only
126+
</a>
127+
with the company email.
128+
</p>
129+
```
130+
131+
The content in DevRev should look like this:
132+
```html
133+
<p>
134+
You can create an account and log-in
135+
<a data-article-id="don:core:dvrv-us-1:devo/0:article/10" href="/ART-10" target="_self">
136+
only
137+
</a>
138+
with the company email.
139+
</p>
140+
```
141+
142+
`don:core:dvrv-us-1:devo/0:article/10` is the ID of the article created in DevRev corresponding to the article with ID `360059607772` in the external source system.
143+
To achieve this, you need to transform the content of the article to the following JSON:
144+
```json
145+
[
146+
"You can create an account and log-in <a data-article-id=\"",
147+
{
148+
"ref_type": "article",
149+
"id": "360059607772",
150+
"fallback_record_name": "<fallback article ID>"
151+
},
152+
"\" target=\"_self\"> only</a> with the company email."
153+
]
154+
```
155+
156+
The `ref_type` should be set to the item type in the external system that is being mapped to articles.
157+
For example, if you're importing documents from the external system as articles,
158+
the `ref_type` should be set to documents. The ID should be the ID of the item in the external source system.
159+
160+
The platform replaces the mention block with the ID of the corresponding article in DevRev as well as adds the href attribute with the appropriate value.

0 commit comments

Comments
 (0)