Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support write/query via OpenTSDB API #904

Open
1 of 2 tasks
jiacai2050 opened this issue May 18, 2023 · 5 comments
Open
1 of 2 tasks

Support write/query via OpenTSDB API #904

jiacai2050 opened this issue May 18, 2023 · 5 comments
Assignees
@jiacai2050
Labels
contributor friendly Good for contribution feature New feature or request

Comments

@jiacai2050
Copy link
Contributor

jiacai2050 commented May 18, 2023 •
edited
Loading

Describe This Problem

Currently there are three query language supported in CeresDB: SQL/InfluxQL/PromQL. It would be great to also support OpenTSBD query API.

Proposal

There are many API in OpenTSDB, the following two API are most important, so we can first implement those.

Additional Context

No response
@jiacai2050 jiacai2050 added feature New feature or request contributor friendly Good for contribution labels May 18, 2023
@zouxiang1993 zouxiang1993 mentioned this issue Jun 27, 2023
Merged
jiacai2050 pushed a commit that referenced this issue Jun 29, 2023
@zouxiang1993
feat: support opentsdb put api (#1037)
3c113b5 
## Rationale
Part of #904 

## Detailed Changes


## Test Plan
1. write some data points 
```
curl --location --request POST 'http://127.0.0.1:5440/opentsdb/api/put' --data-ascii '
[
{
    "metric": "sys.cpu.nice",
    "timestamp": 1687935743000,
    "value": 18,
    "tags": {
       "host": "web01",
       "dc": "lga"
    }
},
{
    "metric": "sys.cpu.nice",
    "timestamp": 1687935743000,
    "value": 18,
    "tags": {
       "host": "web01"
    }
}
]
'
```

2. select 
```
curl --location --request POST 'http://127.0.0.1:5440/sql' --data-ascii '
SELECT * from "sys.cpu.nice"
'
```

the response: 
```
{
  "rows": [
    {
      "tsid": 1890867319031064034,
      "timestamp": 1687935743000,
      "dc": null,
      "host": "web01",
      "value": 18.0
    },
    {
      "tsid": 7054964577922029584,
      "timestamp": 1687935743000,
      "dc": "lga",
      "host": "web01",
      "value": 18.0
    }
  ]
}
```
@parkma99
Copy link

parkma99 commented Aug 2, 2023

I would like to work query api, I am newer fo this project, Could you give me some suggestion to start?😊

@jiacai2050
Copy link
Contributor Author

jiacai2050 commented Aug 2, 2023 •
edited
Loading

Thanks for you interests, let us know when you have any problems.

Generally speaking, you need to convert opentsdb query syntax to datafusion plan, you can refer how InfluxQL is implemented here:

@parkma99
Copy link

parkma99 commented Aug 2, 2023

I find it's very hard to me current 😢😢，I give up to others this issue.

@parkma99 parkma99 removed their assignment Aug 2, 2023
@jiacai2050
Copy link
Contributor Author

@parkma99 😅 Thanks anyway, you can try other issue first.

dust1 pushed a commit to dust1/ceresdb that referenced this issue Aug 9, 2023
@zouxiang1993 @dust1
feat: support opentsdb put api (apache#1037)
510ea32 
## Rationale
Part of apache#904 

## Detailed Changes


## Test Plan
1. write some data points 
```
curl --location --request POST 'http://127.0.0.1:5440/opentsdb/api/put' --data-ascii '
[
{
    "metric": "sys.cpu.nice",
    "timestamp": 1687935743000,
    "value": 18,
    "tags": {
       "host": "web01",
       "dc": "lga"
    }
},
{
    "metric": "sys.cpu.nice",
    "timestamp": 1687935743000,
    "value": 18,
    "tags": {
       "host": "web01"
    }
}
]
'
```

2. select 
```
curl --location --request POST 'http://127.0.0.1:5440/sql' --data-ascii '
SELECT * from "sys.cpu.nice"
'
```

the response: 
```
{
  "rows": [
    {
      "tsid": 1890867319031064034,
      "timestamp": 1687935743000,
      "dc": null,
      "host": "web01",
      "value": 18.0
    },
    {
      "tsid": 7054964577922029584,
      "timestamp": 1687935743000,
      "dc": "lga",
      "host": "web01",
      "value": 18.0
    }
  ]
}
```
@jiacai2050
Copy link
Contributor Author

jiacai2050 commented Sep 4, 2023 •
edited
Loading

This docs explain how opentsdb query is executed:

Filtering

Grouping

Downsampling

Interpolation

Aggregation

Rate Conversion

Functions

Expressions

Understanding the order of operations is important. When returning query results the following is the order in which processing takes place.

@jiacai2050 jiacai2050 self-assigned this Sep 4, 2023
@jiacai2050 jiacai2050 mentioned this issue Sep 7, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jiacai2050 jiacai2050

Labels
contributor friendly Good for contribution feature New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jiacai2050 @parkma99