Skip to content

Conversation

ferrirW
Copy link

@ferrirW ferrirW commented Jun 2, 2023

support hologres connectors (both source and sink)

link: #513

Data flow build, send and receive data success in hologres 1.3.42 version.
But the data must be structured, Otherwise, the type conversion will fail, such as: fileSource -> holoSink

Alternatively, perhaps fileSource could be modified to support JSON-type structured data instead of being transmitted in rows

@ferrirW ferrirW force-pushed the support-hologres-connector-dev branch from e67758f to e10e266 Compare June 2, 2023 05:01
@ferrirW ferrirW force-pushed the support-hologres-connector-dev branch from e10e266 to 44b80d8 Compare June 7, 2023 03:29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gitignore already exists, is this necessary?

- **connector-class**: connector的类名
- **tasks.num**: 启动的task数目

##### parameter configuration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

感觉可以标注一下是source还是sink。比如使用三级标题

return null;
}

if (++count % 1000 == 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

或许count需要reset一下,避免溢出

} catch (Exception e) {
log.error("Error while polling data from Hologres", e);
}
return records;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

每次poll只拉一个record下来吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants