Skip to content

Commit 6b87ad4

Browse files
tilenkavcictilenkavcic
committed
Add IDM and the rest from chef
1 parent 03e97fd commit 6b87ad4

File tree

6 files changed

+428
-18
lines changed

6 files changed

+428
-18
lines changed

‎fern/docs/pages/airdrop/data-model/mapping_reasons.mdx

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
2+
Airdrop uses transformation methods to map data from one format to another.
3+
These methods work on field scope and each have their own requirements.
4+
If the data does not meet these requirements, the mapping will not be available.
5+
6+
There are several reasons why some mappings might be unavailable.
7+
8+
1. A common reason is mismatch of types. For example, if a DevRev field is expected to be `rich_text`,
9+
but the field is set as `text` mapping to some fields will be unavailable.
10+
Refer to the [supported types](../metadata-extraction.mdx#declare-fields-for-each-record-type)
11+
section and the general DevRev documentation for more information.
12+
2. Only references can be mapped to references. Ensure that source system fields are correctly
13+
mapped to reference fields in DevRev.
14+
3. Currently, support for the `struct` type is limited. Marking a field as a struct in the metadata schema will
15+
make it unavailable for mapping outside of using the custom jq transformation method.
16+
Refer to the [metadata tips](../faq.mdx#metadata-extraction) for more information.
17+
4. Links are supported only on works and conversations.
18+
19+
We provide the following transformation methods for **custom fields**:
20+
1. `make_constrained_simple_value` a simple method that along with mapping allows you to
21+
propagate validation constraints from the external system and enforces those in DevRev
22+
2. `make_enum` produces an enum field, where external field should be of type enum
23+
3. `make_uenum` produces an enum field, where external field should be of type enum
24+
4. `reference_custom_field` produces a reference field, where external field should be of type reference
25+
26+
We provide the following transformation methods for Metadata extraction:
27+
1. `make_custom_stages` makes custom stages in accordance with the stage diagram data provided in the domain metadata
28+
2. `map_enum` produces an enum field, where external field should be of type enum
29+
3. `map_roles` maps permission roles from external to DevRev format
30+
4. `use_as_array_value` produces an array field from a scalar (single-value) external field
31+
5. `use_devrev_record` enables use of a fixed reference to something in DevRev, where Don should be of right type
32+
6. `use_directly` is the identity operator, it returns exactly the input, where external field should be of the same
33+
type as the DevRev field. If external field is an array (is marked as collection) internal field has to be an array as well.
34+
7. `use_fixed_value` produces a fixed value in DevRev, which is useful to map required DevRev fields if source system doesn't have an equivalent field. Only available for boolean or enum DevRev fields
35+
36+
We provide the following transformation methods for both **custom and stock fields**:
37+
1. `use_rich_text` produces a rich text field, where external field should be of type `rich_text`
38+
39+
We provide the following transformation methods for **constructed custom fields**:
40+
1. `construct_text_field` produces a text field, where external field should be of type text
41+
42+
`use_raw_jq` can be used on all fields. It enables the use of `jq` to transform data and should be used sparingly
43+
if no other transformation method is available.

‎fern/docs/pages/airdrop/data-model/permissions.mdx

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
DevRev uses the concept of permissions to control access to records.
2+
Permissions are associated with users or groups and define the level of access they have to leaf types and their fields.
3+
4+
## Article permissions
5+
6+
{/* TODO: more on article permissions */}
7+
Articles can be shared with users or groups.
8+
9+
The `shared_with` field allows to specify the permission level for each user or group using the `permission` type.
10+
The `permission` type is a special structure associating a reference to an user-like record type (the field `member_id`)
11+
with an `enum` value that provides the user with the role or permission level.
12+
13+
## Platform Groups
14+
15+
Platform groups are groups automatically created by the DevRev platform for every organization.
16+
For example, DevRev provides *Dev users* and *Rev users* groups that contain all the Dev and Rev users respectively.
17+
18+
We allow developers to map their external default groups to platform groups.
19+
This functionality allows for flexible integration with the platform's permission system while allowing for re-mapping by the end user.
20+
21+
### Implementation Guide
22+
23+
1. Developers need to create a mapping between their external default groups and the platform's
24+
internal default groups using a new mapping object.
25+
- To do this define a new object in external metadata.
26+
- The object should have one enum field with the possible values being the default groups available in external system.
27+
2. Developers can use initial domain mapping to map their object to the *platform groups* object in devrev.
28+
3. The platform groups object can be referenced in other objects, for example in the `shared_with` field of articles.
29+
30+
{/* ### Authorization policy */}

‎fern/docs/pages/airdrop/data-model/rich-text.mdx

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
2+
Rich text is used in DevRev to represent text fields that can be formatted and can contain mentions. Eg.: description of an issue, body of a conversation, etc.
3+
4+
A simple rich text looks like one markdown string wrapped in an array: `["Hello **world**!"]`.
5+
Markdown should be compatible with [CommonMark Spec v0.30](https://spec.commonmark.org/0.30).
6+
7+
To support mentions `rich_text` can be formatted as an array of strings and mention objects like so:
8+
```json
9+
[
10+
"Hello ",
11+
{"ref_type":"external_user_type", "id":"1...", "fallback_record_name": "John Doe"},
12+
"how are you?"
13+
]
14+
```
15+
16+
17+
Mention represents any mention (user, issue, etc.) in rich text and is defined as:
18+
| Field | Type | Required | Description |
19+
| ---------------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------ |
20+
| `id` | String | Yes | Identifier of the item being mentioned. This could be a user ID or any other identifier, in the format used by the source system. |
21+
| `ref_type` | String | Yes | Type of the item being mentioned. Examples include "issue", "comment", etc. The recipe will convert this according to user mappings. |
22+
| `fallback_record_name` | String | No | The text to display if the mention cannot be resolved. This could be a user's display name or a ticket title, for instance. |
23+
24+
In reverse loaders should expect the following structure:
25+
```json
26+
{
27+
"type": "rich_text",
28+
"content": [
29+
"Hello ",
30+
{
31+
"ref_type": "external_user_type",
32+
"id": "don:...",
33+
"fallback_record_name": "John Smith"
34+
},
35+
"how are you?"
36+
]
37+
}
38+
```
39+
40+
41+
## Importing articles
42+
43+
Articles support Markdown as well as HTML.
44+
45+
### Article mentions
46+
47+
We support mentions to artifacts and articles. An inline attachment must be mapped to an artifact. A link to another article must be mapped to an article.
48+
49+
#### Inline attachments
50+
If an inline attachment is hosted in the source system, it must be created as an artifact in DevRev. The same link cannot be used as the attachment will be deleted in the source system when our customers deactivate the account. However, creating an artifact is not enough. The artifact must linked in the appropriate place in the article content.
51+
52+
##### HTML
53+
```html
54+
<img src="don:core:dvrv-us-1:devo/0:artifact/1" alt="Alt Text"/>
55+
```
56+
57+
##### Markdown
58+
```markdown
59+
![Alt Text](don:core:dvrv-us-1:devo/0:artifact/1)
60+
```
61+
62+
##### Example
63+
Let's say the content of your external system looks like this:
64+
```html
65+
<p>This is an article with one image.</p>
66+
<p><img src="https://devrev.zendesk.com/hc/article_attachments/29908544740244" alt="download.jpeg"></p>
67+
```
68+
69+
The content in DevRev should look like this:
70+
```html
71+
<p>This is an article with one image.</p>
72+
<p><img src="don:core:dvrv-us-1:devo/0:artifact/1" alt="download.jpeg"></p>
73+
```
74+
75+
`don:core:dvrv-us-1:devo/0:artifact/1` is the ID of the artifact created in DevRev corresponding to the attachment with ID `29908544740244` in the external source system.
76+
To achieve this, you need to transform the content of the article to this:
77+
```json
78+
[
79+
"<p>This is an article with one image.</p><p><img src=\"",
80+
{
81+
"ref_type": "artifact",
82+
"id": "29908544740244",
83+
"fallback_record_name": "<fallback link>"
84+
},
85+
"\" alt=\"download.jpeg\"></p>"
86+
]
87+
```
88+
The ref_type should be set to artifact and the ID should be the ID of the attachment in the external source system. The platform simply replaces the mention block with the ID of the corresponding artifact. The resolved value is not wrapped in double quotes.
89+
90+
#### Links to other articles
91+
If there is a link to another article in the content of the article, you need to create a mention to the article. The link must be to an article that was either created in previous syncs or will be created in the current sync. At the extractor stage, it is impossible to predict the ID of the article that will be created in DevRev. So, this must be handled by the platform. This feature is only available for HTML format. However, since Markdown can contain HTML, you can use the same approach for Markdown as well.
92+
93+
##### HTML
94+
```html
95+
<a data-article-id="don:core:dvrv-us-1:devo/0:article/10" href="/ART-10" target="_self">Contact our Support Team</a>
96+
```
97+
98+
##### Example
99+
Let's say the content of your external system looks like this:
100+
```html
101+
<p>You can create an account and log-in <a href="https://devrev.zendesk.com/hc/en-us/articles/360059607772" target="_self">only</a> with the company email.
102+
```
103+
104+
The content in DevRev should look like this:
105+
```html
106+
<p>You can create an account and log-in <a data-article-id="don:core:dvrv-us-1:devo/0:article/10" href="/ART-10" target="_self">only</a> with the company email.
107+
```
108+
109+
`don:core:dvrv-us-1:devo/0:article/10` is the ID of the article created in DevRev corresponding to the article with ID `360059607772` in the external source system.
110+
To achieve this, you need to transform the content of the article to this:
111+
```json
112+
[
113+
"You can create an account and log-in <a data-article-id=\"",
114+
{
115+
"ref_type": "article",
116+
"id": "360059607772",
117+
"fallback_record_name": "<fallback article ID>"
118+
},
119+
"\" target=\"_self\"> only</a> with the company email."
120+
]
121+
```
122+
The ref_type should be set to the item type in the external system that is being mapped to articles. For example, if you're importing documents from the external system as articles, the `ref_type` should be set to documents. The ID should be the ID of the item in the external source system. The platform replaces the mention block with the ID of the corresponding article in DevRev as well as adds the href attribute with the appropriate value.
123+
124+
## Managing Permissions
125+
126+
You can manage permissions in the `shared_with` field. Permissions can reference users, groups and [platform groups](./platform_groups.md).

‎fern/docs/pages/airdrop/faq.mdx

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,5 +24,125 @@
2424
</AccordionGroup>
2525

2626
## Metadata extraction
27+
<AccordionGroup>
28+
<Accordion title="Tips for defining record types">
29+
The main purpose of metadata is to define record types. Each record type should correspond to a homogenous set of records in the external system: a domain object that has a well-defined schema.
30+
31+
In some cases this means simply declaring one record type for API endpoints like '/comments' of the external system, but in other cases, external systems can have configurable custom types or subtypes (for example issuetypes in Jira). In these cases the snap-in will need to query an API for the list of types, and produce a dynamic list of record_types in the metadata.
32+
</Accordion>
33+
34+
<Accordion title="Understanding record type categories">
35+
Record types don't have hierarchy; each is a leaf type corresponding to concrete records in files marked with that itemtype. Record type categories can be used to group them, serving two purposes:
36+
37+
1. To define mapping rules that apply to a dynamic set of record types, unknown at the time the snap-in is created
38+
2. To tell the recipe system that a record can transition between two record types while preserving its identity
39+
</Accordion>
40+
41+
<Accordion title="Working with integer fields">
42+
The field type 'int' is used to represent integer numeric values. In certain external systems, identifiers of records or enum values are also stored as integers. These are however not 'conceptually integers' in Airdrop's perspective.
43+
44+
The natural format of integers is `null` | JSON numbers without decimals.
45+
Numbers encoded to strings (e.g., `"2112"`), or empty strings should not be used.
46+
</Accordion>
47+
48+
<Accordion title="Handling record IDs and metadata">
49+
The primary key (id) of the record in the external system doesn't need to be declared as a field in the record type. Instead, id, created_date, and modified_date should be provided at the top level, with all other fields inside the data field. Example:
50+
51+
```json
52+
{
53+
"id": "2102e01F",
54+
"operation": "created",
55+
"created_date": "1970-01-01T01:00:04+01:00",
56+
"modified_date": "1972-03-29T22:04:47+01:00",
57+
"data": {
58+
"actual_close_date": "1970-01-01T02:33:18+01:00",
59+
"created_date": "1970-01-01T01:08:25+01:00",
60+
"modified_date": "1970-01-01T01:00:08+01:00",
61+
"owner": "3A",
62+
"priority": "P1",
63+
"target_close_date": null,
64+
"title": "Lorem ipsum dolor sit amet"
65+
}
66+
}
67+
```
68+
</Accordion>
69+
70+
<Accordion title="Working with collections">
71+
All logical data types can be modified to be a collection instead.
72+
The natural format of a collection is a JSON array (or null), containing the natural format of its elements. For example:
73+
74+
```json
75+
{
76+
"reporter_ids": [
77+
{
78+
"ref_type": "user",
79+
"id": "2103232131",
80+
"fallback_record_name": "John Doe"
81+
},
82+
{
83+
"ref_type": "contact",
84+
"id": "2103232144",
85+
"fallback_record_name": "Jane Doe"
86+
}
87+
],
88+
"tags": ["bug", "good-first-issue"]
89+
}
90+
```
91+
92+
Some systems provide collections as enum values or references in a string, separated by some separator (comma or semicolon):
93+
94+
```json
95+
{
96+
"reporter_ids": "2103232131,2103232144",
97+
"tags": "bug;good-first-issue"
98+
}
99+
```
27100

101+
This format should be avoided, and the data normalized to the natural format in the extractor.
102+
</Accordion>
103+
104+
<Accordion title="Using struct fields">
105+
Structs are embedded JSON objects inside a given field. They represent data that consists of multiple elements naturally belonging together (like a phone number or address) but doesn't form its own record with identity.
106+
107+
These are helpful when the whole struct is optional/nullable, but some of its fields are required, providing a cleaner representation than flattening it. Example:
108+
109+
```json
110+
{
111+
"address": {
112+
"country": "US",
113+
"state": "TX",
114+
"city": "Austin",
115+
"address_line": "Rocket Road 1"
116+
},
117+
"phone_number": null
118+
}
119+
```
120+
121+
Many systems resolve references to embedded structs:
122+
123+
```json
124+
{
125+
"creator": {
126+
"userId": "2103232144",
127+
"name": "Lara Croft",
128+
"role": "Adventurer",
129+
"email": "tomb@raider.com"
130+
}
131+
}
132+
```
133+
134+
Such 'detailed references' should not be declared as structs in Airdrop. They should be transformed to either a simple ID reference or a dynamically typed reference with exactly the fields `id`, `fallback_record_name`, and `ref_type`. Using references and structs inside structs is currently not supported.
135+
</Accordion>
136+
137+
<Accordion title="Custom fields">
138+
Any fields in the metadata not otherwise mapped will be offered to the end user as custom fields, with no additional effort needed.
139+
</Accordion>
140+
</AccordionGroup>
141+
142+
## Troubleshooting
143+
<AccordionGroup>
144+
<Accordion title="Chef-CLI version">
145+
Check if chef-cli version is up to date. You can find the version you are using with `chef-cli --version` and the latest available version under project releases.
146+
</Accordion>
147+
</AccordionGroup>
28148

0 commit comments

Comments
 (0)