Skip to content

Commit

Permalink
docs: update biweekly report to latest (GreptimeTeam#572)
Browse files Browse the repository at this point in the history
Co-authored-by: Wenjie0329 <Wenjie0329@users.noreply.github.com>
  • Loading branch information
xtang and Wenjie0329 authored Oct 11, 2023
1 parent 6d96f25 commit 0aa46ea
Show file tree
Hide file tree
Showing 3 changed files with 112 additions and 85 deletions.
197 changes: 112 additions & 85 deletions docs/en/v0.4/changelog/biweekly-reports/latest-biweekly-report.md
Original file line number Diff line number Diff line change
@@ -1,112 +1,139 @@
# Biweekly Report (Aug.14 - Aug.27) – Support starting GreptimeDB clusters with KubeBlocks
# Biweekly Report (Aug.28 - Sep.17) – Welcome GreptimeDB's First Contributor from Africa

August 30, 2023
September 20, 2023

## Summary
Together with all our contributors worldwide, we are glad to see GreptimeDB making remarkable progress for the better.

Together with all our contributors worldwide, we are glad to see GreptimeDB making remarkable progress for the better. In the past two weeks, a total of **112 PRs** were merged for our program. Below are some highlights:
Over the past three weeks, we have been working intensively to prepare for the release of GreptimeDB v0.4, and we are thrilled for its official launch later this week.

- Support starting GreptimeDB clusters with KubeBlocks
- Enrich sqlness test scenarios
- Support gRPC row format write protocol
- Optimize `DESC TABLE` experience
**We will host an online Meetup at 19:00 PDT on October 10, 2023. During this event, we will unveil the latest details of the GreptimeDB v0.4 Benchmark and provide insights into the technical intricacies of the new engine.** Everyone is welcome to join the Meetup to stay updated on the latest developments of GreptimeDB v0.4.

## Contributor list: (in alphabetical order)
Register now for the Zoom invite: https://m0k0y6ku50y.typeform.com/to/m5G87XDI

For the past two weeks, our community has been super active with a total of **4 PRs from 2 contributors** merged successfully and lots pending to be merged.
Congrats on becoming our most active contributors in the past 2 weeks:
Below are some highlights for the past three weeks:

- [@NiwakaDev](https://github.com/NiwakaDev) ([db#2150](https://github.com/GreptimeTeam/greptimedb/pull/2150) [db#2209](https://github.com/GreptimeTeam/greptimedb/pull/2209))
- [@Taylor-lagrange](https://github.com/Taylor-lagrange) ([db#2138](https://github.com/GreptimeTeam/greptimedb/pull/2138) [db#2229](https://github.com/GreptimeTeam/greptimedb/pull/2229))
- Optimized SQL data types.
- Added support for adding primary key columns through `ALTER TABLE`.
- Refined the heartbeat protocol within the distributed architecture.
- Restricted the TIME INDEX type.

A big THANK YOU for the generous and brilliant contributions! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.
## Contributors
Over the last three weeks, our community has successfully merged a total of 189 PRs. Notably, 2 of these PRs came from 2 distinct external contributors, with several more awaiting approval.
A special shout-out to our standout contributors from the past three weeks:

## Highlights of Recent PR

### [Support starting GreptimeDB clusters with KubeBlocks](https://github.com/apecloud/kubeblocks/pull/4822)

We now support starting GreptimeDB clusters with KubeBlocks.

### [Enrich sqlness test scenarios](https://github.com/GreptimeTeam/greptimedb/pull/2073)

We have enriched the sqlness test scenarios by adding test cases for the following three data types: `timestamp`, `float`, and `string`, with reference to DuckDB's test cases.

### [Support gRPC row format write protocol](https://github.com/GreptimeTeam/greptimedb/pull/2188)

Until now, GreptimeDB has only provided support for the gRPC protocol in columnar format. Constructing data in columnar format is less client (SDK) friendly than in row format, which poses an additional burden on the development of cross-multilingual SDKs.

At the same time, the GreptimeDB-compatible InfluxDB Line Protocol and Prometheus Write Protocol both build data based on "rows". Based on these factors, we decided to support for the gRPC row format write protocol. We will provide a more user-friendly data format without compromising performance. Related PRs also include [#2189](https://github.com/GreptimeTeam/greptimedb/pull/2189), [#2231](https://github.com/GreptimeTeam/greptimedb/pull/2231) and [#2263](https://github.com/GreptimeTeam/greptimedb/pull/2263).
- [@NiwakaDev](https://github.com/NiwakaDev0) ([db#2267](https://github.com/GreptimeTeam/greptimedb/pull/2267))
- [@Lilit0x](https://github.com/Lilit0x) ([db#2400](https://github.com/GreptimeTeam/greptimedb/pull/2400))

### [Optimize `DESC TABLE` experience](https://github.com/GreptimeTeam/greptimedb/pull/2256)
👏 Let's welcome [@Lilit0x](https://github.com/Lilit0x) as the new contributors to join our community with the first PR merged.

We optimized the `DESC TABLE` experience in [#2256](https://github.com/GreptimeTeam/greptimedb/pull/2256) and [#2272](https://github.com/GreptimeTeam/greptimedb/pull/2272):
A big THANK YOU to all our members and contributors! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

- Add a `Key` column to describe the primary key information (both `Tag` and `Timestamp` are considered as primary key).

In the current output, the primary key information is stored in the Semantic Type column, which is inconsistent with the Semantic Type (Tag, Field, Timestamp) in the gRPC Protocol, so the optimization proposes a separate column `Key` to describe the primary key, and the Semantic Type column remains the same as in the gRPC Protocol.

Before:
## Highlights of Recent PR
[**#2331 Optimized SQL data types**](https://github.com/GreptimeTeam/greptimedb/pull/2331)

Before this PR, the data types displayed in CREATE TABLE and DESC TABLE were different, which was confusing for users. For instance:

```sql
CREATE TABLE data_types (
s string,
tint tinyint,
sint smallint,
i int,
bint bigint,
v varchar,
f float,
d double,
b boolean,
vb varbinary,
dt date,
dtt datetime,
ts0 timestamp(0),
ts3 timestamp(3),
ts6 timestamp(6),
ts9 timestamp(9) DEFAULT CURRENT_TIMESTAMP TIME INDEX,
PRIMARY KEY(s));
```

```rust
+-----------+----------------------+------+---------+---------------+
| Field | Type | Null | Default | Semantic Type |
+-----------+----------------------+------+---------+---------------+
| ts | TimestampMillisecond | NO | | TIME INDEX |
| collector | String | YES | | PRIMARY KEY |
| host | String | YES | | PRIMARY KEY |
| val | Float64 | YES | | FIELD |
+-----------+----------------------+------+---------+---------------+
```bash
+--------+----------------------+------+------+---------------------+---------------+
| Column | Type | Key | Null | Default | Semantic Type |
+--------+----------------------+------+------+---------------------+---------------+
| s | String | PRI | YES | | TAG |
| tint | Int8 | | YES | | FIELD |
| sint | Int16 | | YES | | FIELD |
| i | Int32 | | YES | | FIELD |
| bint | Int64 | | YES | | FIELD |
| v | String | | YES | | FIELD |
| f | Float32 | | YES | | FIELD |
| d | Float64 | | YES | | FIELD |
| b | Boolean | | YES | | FIELD |
| vb | Binary | | YES | | FIELD |
| dt | Date | | YES | | FIELD |
| dtt | DateTime | | YES | | FIELD |
| ts0 | TimestampSecond | | YES | | FIELD |
| ts3 | TimestampMillisecond | | YES | | FIELD |
| ts6 | TimestampMicrosecond | | YES | | FIELD |
| ts9 | TimestampNanosecond | PRI | NO | current_timestamp() | TIMESTAMP |
+--------+----------------------+------+------+---------------------+---------------+
16 rows in set (0.00 sec)
```

After:

```rust
+-----------+----------------------+-----+------+---------+---------------+
| Field | Type | Key | Null | Default | Semantic Type |
+-----------+----------------------+-----+------+---------+---------------+
| ts | TimestampMillisecond | PRI | NO | | TIMESTAMP |
| collector | String | PRI | YES | | TAG |
| host | String | PRI | YES | | TAG |
| val | Float64 | | YES | | FIELD |
+-----------+----------------------+-----+------+---------+---------------+
This PR adds aliases for SQL data types, as shown below:
- TimestampSecond, Timestamp_s, Timestamp_sec for Timestamp(0).
- TimestampMillisecond, Timestamp_ms for Timestamp(3).
- TimestampMicrosecond, Timestamp_us for Timestamp(6).
- TimestampNanosecond, Timestamp_ns for Timestamp(9).
- INT8 for tinyint
- INT16 for smallint
- INT32 for int
- INT64 for bigint
- And UINT8, UINT16 etc. for UnsignedTinyint etc.

Now, you can also use GreptimeDB types to create tables in SQL:

```sql
CREATE TABLE data_types (
s String,
tint Int8,
sint Int16,
i Int32,
bint Int64,
v String,
f Float32,
d Float64,
b Boolean,
vb Varbinary,
dt Date,
dtt DateTime,
ts0 TimestampSecond,
ts3 TimestampMillisecond,
ts6 TimestampMicrosecond,
ts9 TimestampNanosecond DEFAULT CURRENT_TIMESTAMP TIME INDEX,
PRIMARY KEY(s));
```

- Field to Column
[**#2310 Added support for adding primary key columns through ALTER TABLE**](https://github.com/GreptimeTeam/greptimedb/pull/2310)

In the output of the `DESC TABLE`, we use the name Field (following the MySQL specification), but our time series data model also contains Field (Tag, Field, Timestamp), which can be confusing for users. So we change it to Column.
In time-series data model, Label or Tag corresponds to the primary key in the Table model in GreptimeDB. Therefore, it's necessary to support adding primary key columns through the `ALTER TABLE` statement.

Before:
[**#96 Refined the heartbeat protocol within the distributed architecture**](https://github.com/GreptimeTeam/greptime-proto/pull/96)

```rust
+-----------+----------------------+-----+------+---------+---------------+
| Field | Type | Key | Null | Default | Semantic Type |
+-----------+----------------------+-----+------+---------+---------------+
| ts | TimestampMillisecond | PRI | NO | | TIMESTAMP |
| collector | String | PRI | YES | | TAG |
| host | String | PRI | YES | | TAG |
| val | Float64 | | YES | | FIELD |
+-----------+----------------------+-----+------+---------+---------------+
```
In the upcoming 0.4 release, we went through an in-depth restructuring and upgrade of the database architecture. The first to undergo remodelling was the heartbeat protocol that serves as the foundation of distributed coordination communication. The intent behind this overhaul was to manage larger clusters and a higher number of regions more efficiently. To that end, we trimmed the size of the heartbeat package to alleviate the communication layer's load. For instance, we removed the unnecessary NodeStat, eliminated information such as CPU and Load, and only retained W/RCU as the sole reference for load balancing and scheduling. At the same time, we also removed unnecessary information like table names and database names.

After:

```rust
+-----------+----------------------+-----+------+---------+---------------+
| Column | Type | Key | Null | Default | Semantic Type |
+-----------+----------------------+-----+------+---------+---------------+
| ts | TimestampMillisecond | PRI | NO | | TIMESTAMP |
| collector | String | PRI | YES | | TAG |
| host | String | PRI | YES | | TAG |
| val | Float64 | | YES | | FIELD |
+-----------+----------------------+-----+------+---------+---------------+
```
[**#2281 Restricted the TIME INDEX type**](https://github.com/GreptimeTeam/greptimedb/pull/2281)

## New Things
The TIME INDEX type is restricted, only allowing timestamp types to serve as TIME INDEX to prevent potential ambiguities.

### Vector 0.32 has been released
## New Things
New Features in Dashboard
- The tables list on the left supports shortcut quick input, allowing for one-click insertion of preset SQL query statements.
https://github.com/GreptimeTeam/dashboard/pull/323

A built-in GreptimeDB sink is now available, and visit <https://vector.dev/releases/0.32.0/> for more details.
<p><img src="/biweekly-images/image1.png" alt="Shortcut Demonstration" style="width: 70%; margin: 0 auto;" /></p>

### GreptimeDB gets 3000 stars on GitHub
- The editor offers input suggestions. Both SQL and PromQL editors can automatically suggest potential table names and column names based on entered keywords.
https://github.com/GreptimeTeam/dashboard/pull/329
https://github.com/GreptimeTeam/dashboard/pull/330

[GreptimeDB](https://github.com/GreptimeTeam/greptimedb) has reached a significant milestone of 3000 stars and 191 forks on GitHub! In the past few months, GreptimeDB has released v0.1 through v0.3, with v0.4 coming soon, and GreptimeCloud has been officially announced in public beta, with an updated release on the horizon. We also have got two external committers. All of these achievements would be impossible without the hard work of our team members and the support of our external contributors, and we would like to express our sincere thanks to all of you once again for your contributions!
<p><img src="/biweekly-images/image2.png" alt="Hint Demonstration" style="width: 70%; margin: 0 auto;" /></p>
Binary file added docs/public/biweekly-images/image1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/public/biweekly-images/image2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 0aa46ea

Please sign in to comment.