Skip to content

Commit

Permalink
fix: storage location (GreptimeTeam#1095)
Browse files Browse the repository at this point in the history
Co-authored-by: tison <wander4096@gmail.com>
Co-authored-by: Yiran <cuiyiran3@gmail.com>
  • Loading branch information
3 people committed Jul 25, 2024
1 parent ca24088 commit 3de6936
Show file tree
Hide file tree
Showing 8 changed files with 78 additions and 22 deletions.
10 changes: 10 additions & 0 deletions docs/nightly/en/user-guide/concepts/features-that-you-concern.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Features that You Concern

## Does GreptimeDB support logs or events?

Yes. Since v0.9.0, GreptimeDB treats all time series as contextual events with timestamps, and thus unifies the processing of metrics, logs, and events. It supports analyzing metrics, logs, and events with SQL, PromQL, and streaming with continuous aggregation.

Please read the [log user guide](/user-guide/logs/overview).

## Does GreptimeDB support updates?

Sort of, Please refer to the [update data](/user-guide/write-data/overview.md#update-data) for more information.
Expand Down Expand Up @@ -56,3 +62,7 @@ Yes, GreptimeDB's data access layer is based on [OpenDAL](https://github.com/apa
The data can be stored in cost-effective cloud storage services such as AWS S3 or Azure Blob Storage, please refer to storage configuration guide [here](./../operations/configuration.md#storage-options).

GreptimeDB also offers a fully-managed cloud service [GreptimeCloud](https://greptime.com/product/cloud) to help you manage data in the cloud.

## Does GreptimeDB have disaster recovery solutions?

Yes. Please refer to [disaster recovery](/user-guide/operations/disaster-recovery/overview).
9 changes: 6 additions & 3 deletions docs/nightly/en/user-guide/concepts/storage-location.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,18 @@ The storage file structure of GreptimeDB includes of the following:
└── LOCK
├── data
│   ├── greptime
│   └── system
│   └── public
├── logs
├── index_intermediate
│   └── staging
└── wal
├── raftlog
├── rewrite
└── LOCK
```

- `metadata`: The internal metadata directory that keeps catalog/table info, procedure states, etc. In cluster mode, this directory does not exist in Datanodes or Frontends, because all those states including region route info are saved in Metasrv.
- `data`: The files in data directory store time series data of GreptimeDB. To customize this path, please refer to [Storage option](../operations/configuration.md#storage-options).
- `metadata`: The internal metadata directory that keeps catalog, database and table info, procedure states, etc. In cluster mode, this directory does not exist, because all those states including region route info are saved in `Metasrv`.
- `data`: The files in data directory store time series data and index files of GreptimeDB. To customize this path, please refer to [Storage option](../operations/configuration.md#storage-options). The directory is organized in a two-level structure of catalog and schema.
- `logs`: The log files contains all the logs of operations in GreptimeDB.
- `wal`: The wal directory contains the write-ahead log files.
- `index_intermediate`: the temporary intermediate data while indexing.
Expand All @@ -32,6 +33,8 @@ The storage file structure of GreptimeDB includes of the following:

The `data` directory in the file structure can be stored in cloud storage. Please refer to [Storage option](../operations/configuration.md#storage-options) for more details.

Please note that only storing the data directory in object storage is not sufficient to ensure data reliability and disaster recovery. The `wal` and `metadata` also need to be considered for disaster recovery. Please refer to the [disaster recovery documentation](/user-guide/operations/disaster-recovery/overview).

## Multiple storage engines

Another powerful feature of GreptimeDB is that you can choose the storage engine for each table. For example, you can store some tables on the local disk, and some tables in Amazon S3 or Google Cloud Storage, see [create table](/reference/sql/create#create-table).
10 changes: 10 additions & 0 deletions docs/nightly/zh/user-guide/concepts/features-that-you-concern.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# 关键特性

## GreptimeDB 支持处理日志或事件吗?

是的,从 v0.9.0 版本开始,GreptimeDB 将所有时间序列视为具有时间戳的上下文事件,从而统一了指标、日志和事件的处理。它支持使用 SQL、PromQL 进行指标和事件分析,并支持通过连续聚合进行流式处理。

请阅读[日志处理使用指南](/user-guide/logs/overview)

## GreptimeDB 支持更新数据吗?

支持,请参考[更新数据](/user-guide/write-data/overview.md#更新数据)获取更多信息。
Expand Down Expand Up @@ -55,3 +61,7 @@ GreptimeDB 通过以下方式解决这个问题:
数据可以存储在如 AWS S3 或 Azure Blob Storage 等性价比高的云存储服务中,请参考这里的存储[配置指南](./../operations/configuration.md#storage-options)

GreptimeDB 还提供一个完全托管的云服务 [GreptimeCloud](https://greptime.cn/product/cloud) 来帮助您管理云中的数据。

## GreptimeDB 有灾难恢复解决方案吗?

有的,请参阅 [灾难恢复文档](/user-guide/operations/disaster-recovery/overview)
21 changes: 13 additions & 8 deletions docs/nightly/zh/user-guide/concepts/storage-location.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,29 +7,34 @@ GreptimeDB 支持将数据存储在本地文件系统、AWS S3 及其兼容服
GreptimeDB 的存储文件结构包括以下内容:

```cmd
├── cluster
│ └── dn-0
├── metadata
├── raftlog
├── rewrite
└── LOCK
├── data
│   ├── greptime
│   └── system
│   └── public
├── logs
├── index_intermediate
│   └── staging
└── wal
├── raftlog
├── rewrite
└── LOCK
```

- `cluster`:集群目录包含了内部数据,并按数据节点的 ID 组织数据
- `data`:data 目录下的文件存储 GreptimeDB 的时序数据。要定制这个路径,请参考[存储选项](../operations/configuration.md#存储选项)
- `logs`:日志文件包含 GreptimeDB 中所有的操作日志
- `wal`:wal 目录包含了预写日志文件
- `index_intermediate`: 索引相关的临时文件目录
- `metadata`: 内部元数据目录,保存 catatalog、数据库以及表的元信息、procedure 状态等内部状态。在集群模式下,此目录不存在,因为所有这些状态(包括区域路由信息)都保存在 `Metasrv`
- `data`: 存储 GreptimeDB 的实际的时间序列数据和索引文件。如果要自定义此路径,请参阅 [存储选项](../operations/configuration.md#storage-options)。该目录按照 catalog 和 schema 的两级结构组织
- `logs`: GreptimeDB 日志文件目录
- `wal`: 预写日志文件目录
- `index_intermediate`: 索引构建和查询相关的的临时中间数据目录

## 云存储

文件结构中的 `cluster``data` 目录可以存储在云存储中。请参考[存储选项](../operations/configuration.md#存储选项)了解更多细节。

请注意,仅将 `data` 目录存储在对象存储中不足以确保数据可靠性和灾难恢复,`wal``metadata` 也需要考虑灾难恢复,更详细地请参阅[灾难恢复文档](/user-guide/operations/disaster-recovery/overview)

## 多存储引擎支持

GreptimeDB 的另一个强大功能是可以为每张表单独选择存储引擎。例如,您可以将一些表存储在本地磁盘上,将另一些表存储在 Amazon S3 或 Google Cloud Storage 中,请参考 [create table](/reference/sql/create#create-table)
10 changes: 10 additions & 0 deletions docs/v0.9/en/user-guide/concepts/features-that-you-concern.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Features that You Concern

## Does GreptimeDB support logs or events?

Yes. Since v0.9.0, GreptimeDB treats all time series as contextual events with timestamps, and thus unifies the processing of metrics, logs, and events. It supports analyzing metrics, logs, and events with SQL, PromQL, and streaming with continuous aggregation.

Please read the [log user guide](/user-guide/logs/overview).

## Does GreptimeDB support updates?

Sort of, Please refer to the [update data](/user-guide/write-data/overview.md#update-data) for more information.
Expand Down Expand Up @@ -56,3 +62,7 @@ Yes, GreptimeDB's data access layer is based on [OpenDAL](https://github.com/apa
The data can be stored in cost-effective cloud storage services such as AWS S3 or Azure Blob Storage, please refer to storage configuration guide [here](./../operations/configuration.md#storage-options).

GreptimeDB also offers a fully-managed cloud service [GreptimeCloud](https://greptime.com/product/cloud) to help you manage data in the cloud.

## Does GreptimeDB have disaster recovery solutions?

Yes. Please refer to [disaster recovery](/user-guide/operations/disaster-recovery/overview).
9 changes: 6 additions & 3 deletions docs/v0.9/en/user-guide/concepts/storage-location.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,18 @@ The storage file structure of GreptimeDB includes of the following:
└── LOCK
├── data
│   ├── greptime
│   └── system
│   └── public
├── logs
├── index_intermediate
│   └── staging
└── wal
├── raftlog
├── rewrite
└── LOCK
```

- `metadata`: The internal metadata directory that keeps catalog/table info, procedure states, etc. In cluster mode, this directory does not exist in Datanodes or Frontends, because all those states including region route info are saved in Metasrv.
- `data`: The files in data directory store time series data of GreptimeDB. To customize this path, please refer to [Storage option](../operations/configuration.md#storage-options).
- `metadata`: The internal metadata directory that keeps catalog, database and table info, procedure states, etc. In cluster mode, this directory does not exist, because all those states including region route info are saved in `Metasrv`.
- `data`: The files in data directory store time series data and index files of GreptimeDB. To customize this path, please refer to [Storage option](../operations/configuration.md#storage-options). The directory is organized in a two-level structure of catalog and schema.
- `logs`: The log files contains all the logs of operations in GreptimeDB.
- `wal`: The wal directory contains the write-ahead log files.
- `index_intermediate`: the temporary intermediate data while indexing.
Expand All @@ -32,6 +33,8 @@ The storage file structure of GreptimeDB includes of the following:

The `data` directory in the file structure can be stored in cloud storage. Please refer to [Storage option](../operations/configuration.md#storage-options) for more details.

Please note that only storing the data directory in object storage is not sufficient to ensure data reliability and disaster recovery. The `wal` and `metadata` also need to be considered for disaster recovery. Please refer to the [disaster recovery documentation](/user-guide/operations/disaster-recovery/overview).

## Multiple storage engines

Another powerful feature of GreptimeDB is that you can choose the storage engine for each table. For example, you can store some tables on the local disk, and some tables in Amazon S3 or Google Cloud Storage, see [create table](/reference/sql/create#create-table).
10 changes: 10 additions & 0 deletions docs/v0.9/zh/user-guide/concepts/features-that-you-concern.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# 关键特性

## GreptimeDB 支持处理日志或事件吗?

是的,从 v0.9.0 版本开始,GreptimeDB 将所有时间序列视为具有时间戳的上下文事件,从而统一了指标、日志和事件的处理。它支持使用 SQL、PromQL 进行指标和事件分析,并支持通过连续聚合进行流式处理。

请阅读[日志处理使用指南](/user-guide/logs/overview)

## GreptimeDB 支持更新数据吗?

支持,请参考[更新数据](/user-guide/write-data/overview.md#更新数据)获取更多信息。
Expand Down Expand Up @@ -55,3 +61,7 @@ GreptimeDB 通过以下方式解决这个问题:
数据可以存储在如 AWS S3 或 Azure Blob Storage 等性价比高的云存储服务中,请参考这里的存储[配置指南](./../operations/configuration.md#storage-options)

GreptimeDB 还提供一个完全托管的云服务 [GreptimeCloud](https://greptime.cn/product/cloud) 来帮助您管理云中的数据。

## GreptimeDB 有灾难恢复解决方案吗?

有的,请参阅 [灾难恢复文档](/user-guide/operations/disaster-recovery/overview)
21 changes: 13 additions & 8 deletions docs/v0.9/zh/user-guide/concepts/storage-location.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,29 +7,34 @@ GreptimeDB 支持将数据存储在本地文件系统、AWS S3 及其兼容服
GreptimeDB 的存储文件结构包括以下内容:

```cmd
├── cluster
│ └── dn-0
├── metadata
├── raftlog
├── rewrite
└── LOCK
├── data
│   ├── greptime
│   └── system
│   └── public
├── logs
├── index_intermediate
│   └── staging
└── wal
├── raftlog
├── rewrite
└── LOCK
```

- `cluster`:集群目录包含了内部数据,并按数据节点的 ID 组织数据
- `data`:data 目录下的文件存储 GreptimeDB 的时序数据。要定制这个路径,请参考[存储选项](../operations/configuration.md#存储选项)
- `logs`:日志文件包含 GreptimeDB 中所有的操作日志
- `wal`:wal 目录包含了预写日志文件
- `index_intermediate`: 索引相关的临时文件目录
- `metadata`: 内部元数据目录,保存 catatalog、数据库以及表的元信息、procedure 状态等内部状态。在集群模式下,此目录不存在,因为所有这些状态(包括区域路由信息)都保存在 `Metasrv`
- `data`: 存储 GreptimeDB 的实际的时间序列数据和索引文件。如果要自定义此路径,请参阅 [存储选项](../operations/configuration.md#storage-options)。该目录按照 catalog 和 schema 的两级结构组织
- `logs`: GreptimeDB 日志文件目录
- `wal`: 预写日志文件目录
- `index_intermediate`: 索引构建和查询相关的的临时中间数据目录

## 云存储

文件结构中的 `cluster``data` 目录可以存储在云存储中。请参考[存储选项](../operations/configuration.md#存储选项)了解更多细节。

请注意,仅将 `data` 目录存储在对象存储中不足以确保数据可靠性和灾难恢复,`wal``metadata` 也需要考虑灾难恢复,更详细地请参阅[灾难恢复文档](/user-guide/operations/disaster-recovery/overview)

## 多存储引擎支持

GreptimeDB 的另一个强大功能是可以为每张表单独选择存储引擎。例如,您可以将一些表存储在本地磁盘上,将另一些表存储在 Amazon S3 或 Google Cloud Storage 中,请参考 [create table](/reference/sql/create#create-table)

0 comments on commit 3de6936

Please sign in to comment.