Skip to content

Commit 23cb64c

Browse files
authored
[IOTDB-4121] logical optimizations for ImportCsv (apache#6998)
1 parent 76c01dc commit 23cb64c

File tree

3 files changed

+23
-14
lines changed

3 files changed

+23
-14
lines changed

cli/src/main/java/org/apache/iotdb/tool/ImportCsv.java

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -331,10 +331,12 @@ private static void writeDataAlignedByTime(
331331

332332
Set<String> devices = deviceAndMeasurementNames.keySet();
333333
String devicesStr = StringUtils.join(devices, ",");
334-
try {
335-
queryType(devicesStr, headerTypeMap, "Time");
336-
} catch (IoTDBConnectionException e) {
337-
e.printStackTrace();
334+
if (headerTypeMap.isEmpty()) {
335+
try {
336+
queryType(devicesStr, headerTypeMap, "Time");
337+
} catch (IoTDBConnectionException e) {
338+
e.printStackTrace();
339+
}
338340
}
339341

340342
List<String> deviceIds = new ArrayList<>();
@@ -501,7 +503,9 @@ record -> {
501503
// query the data type in iotdb
502504
if (!typeQueriedDevice.contains(deviceName.get())) {
503505
try {
504-
hasResult = queryType(deviceName.get(), headerTypeMap, "Device");
506+
if (headerTypeMap.isEmpty()) {
507+
hasResult = queryType(deviceName.get(), headerTypeMap, "Device");
508+
}
505509
typeQueriedDevice.add(deviceName.get());
506510
} catch (IoTDBConnectionException e) {
507511
e.printStackTrace();
@@ -829,7 +833,7 @@ private static Object typeTrans(String value, TSDataType type) {
829833
if (value.startsWith("\"") && value.endsWith("\"")) {
830834
return value.substring(1, value.length() - 1);
831835
}
832-
return null;
836+
return value;
833837
case BOOLEAN:
834838
if (!"true".equals(value) && !"false".equals(value)) {
835839
return null;

docs/UserGuide/Write-And-Delete-Data/CSV-Tool.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -142,12 +142,14 @@ Time,root.test.t1.str,root.test.t2.str,root.test.t2.int
142142
1970-01-01T08:00:00.002+08:00,"123",,
143143
```
144144

145-
The data aligned by time, and headers with data type.
145+
The data aligned by time, and headers with data type.(Text type data supports double quotation marks and no double quotation marks)
146146

147147
```sql
148148
Time,root.test.t1.str(TEXT),root.test.t2.str(TEXT),root.test.t2.int(INT32)
149149
1970-01-01T08:00:00.001+08:00,"123hello world","123\,abc",100
150-
1970-01-01T08:00:00.002+08:00,"123",,
150+
1970-01-01T08:00:00.002+08:00,123,hello world,123
151+
1970-01-01T08:00:00.003+08:00,"123",,
152+
1970-01-01T08:00:00.004+08:00,123,,12
151153
```
152154

153155
The data aligned by device, and headers without data type.
@@ -159,13 +161,13 @@ Time,Device,str,int
159161
1970-01-01T08:00:00.001+08:00,root.test.t2,"123\,abc",100
160162
```
161163

162-
The data aligned by device, and headers with data type.
164+
The data aligned by device, and headers with data type.(Text type data supports double quotation marks and no double quotation marks)
163165

164166
```sql
165167
Time,Device,str(TEXT),int(INT32)
166168
1970-01-01T08:00:00.001+08:00,root.test.t1,"123hello world",
167-
1970-01-01T08:00:00.002+08:00,root.test.t1,"123",
168-
1970-01-01T08:00:00.001+08:00,root.test.t2,"123\,abc",100
169+
1970-01-01T08:00:00.002+08:00,root.test.t1,hello world,123
170+
1970-01-01T08:00:00.003+08:00,root.test.t1,,123
169171
```
170172

171173
### Syntax

docs/zh/UserGuide/Write-And-Delete-Data/CSV-Tool.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -142,12 +142,14 @@ Time,root.test.t1.str,root.test.t2.str,root.test.t2.int
142142
1970-01-01T08:00:00.002+08:00,"123",,
143143
```
144144

145-
通过时间对齐,并且header中包含数据类型的数据。
145+
通过时间对齐,并且header中包含数据类型的数据。(Text类型数据支持加双引号和不加双引号)
146146

147147
```sql
148148
Time,root.test.t1.str(TEXT),root.test.t2.str(TEXT),root.test.t2.int(INT32)
149149
1970-01-01T08:00:00.001+08:00,"123hello world","123\,abc",100
150-
1970-01-01T08:00:00.002+08:00,"123",,
150+
1970-01-01T08:00:00.002+08:00,123,hello world,123
151+
1970-01-01T08:00:00.003+08:00,"123",,
152+
1970-01-01T08:00:00.004+08:00,123,,12
151153
```
152154

153155
通过设备对齐,并且header中不包含数据类型的数据。
@@ -159,13 +161,14 @@ Time,Device,str,int
159161
1970-01-01T08:00:00.001+08:00,root.test.t2,"123\,abc",100
160162
```
161163

162-
通过设备对齐,并且header中包含数据类型的数据。
164+
通过设备对齐,并且header中包含数据类型的数据。(Text类型数据支持加双引号和不加双引号)
163165

164166
```sql
165167
Time,Device,str(TEXT),int(INT32)
166168
1970-01-01T08:00:00.001+08:00,root.test.t1,"123hello world",
167169
1970-01-01T08:00:00.002+08:00,root.test.t1,"123",
168170
1970-01-01T08:00:00.001+08:00,root.test.t2,"123\,abc",100
171+
1970-01-01T08:00:00.002+08:00,root.test.t1,hello world,123
169172
```
170173

171174
### 运行方法

0 commit comments

Comments
 (0)