feat(QTDI-1060): db mappings #1017

undx · 2025-04-07T07:11:43Z

Issues

framework: https://qlik-dev.atlassian.net/browse/QTDI-1060
studio: https://qlik-dev.atlassian.net/browse/QTUP-1717

Related pull requests

feat(QTDI-1060): db mappings #1017 (component-runtime)
https://github.com/Talend/tdi-studio-se/pull/11197 (master)
https://github.com/Talend/tcommon-studio-se/pull/8008 (master)

build pr

https://github.com/Talend/tdi-studio-se/pull/11042 (maintenance/8.0.2)
https://github.com/Talend/tcommon-studio-se/pull/7982 (maintenance/8.0.2)

...st/java/org/talend/sdk/component/design/extension/repository/RepositoryModelBuilderTest.java

...runtime-manager/src/main/java/org/talend/sdk/component/runtime/manager/ComponentManager.java

...nent-server-model/src/main/java/org/talend/sdk/component/server/front/model/ComponentId.java

component-tools/src/main/java/org/talend/sdk/component/tools/validator/ActionValidator.java

documentation/src/main/antora/modules/ROOT/pages/_partials/generated_actions.adoc

...src/main/java/org/talend/sdk/component/sample/feature/databasemapping/service/UIService.java

# Conflicts: # component-tools/src/main/java/org/talend/sdk/component/tools/validator/ActionValidator.java # sample-parent/sample-features/pom.xml

ozhelezniak-talend

I'm worried that we put such specific action inside the core of Component.

Copilot

Pull Request Overview

This PR adds support for dynamic database schema mappings throughout the component tooling, runtime, and server layers.

Introduces the @DatabaseSchemaMapping service annotation with validation logic in ActionValidator and corresponding unit tests.
Extends the TaCoKit schema guesser to capture an ORIGIN_TYPE property and updates tests to assert sourceType in output columns.
Propagates database mapping metadata to the component index via ComponentSchemaEnricher and adds server‐side tests for mapping metadata exposure.

Reviewed Changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
component-tools/.../ActionValidator.java	Added `findDatabaseMappingsErrors` to validate annotated methods
component-tools/.../ActionValidatorTest.java	New unit tests for valid/invalid database mapping actions
component-runtime-di/.../TaCoKitGuessSchema.java	Emit `sourceType` from `ORIGIN_TYPE` schema property
component-runtime-di/.../TaCoKitGuessSchemaTest.java	Updated test to expect `sourceType` in JSON output
component-runtime-manager/.../ComponentSchemaEnricher.java	Enrich component metadata with database mapping keys
component-runtime-manager/.../ComponentSchemaEnricherTest.java	Tests for mapping metadata on components
component-server/.../ComponentResourceImplTest.java	Test for retrieving mapping metadata via REST API
component-api/.../DatabaseSchemaMapping.java	New service annotation definition
component-api/.../DatabaseMapping.java	Added to component API for static mapping declarations

Comments suppressed due to low confidence (2)

component-tools/src/main/java/org/talend/sdk/component/tools/validator/ActionValidator.java:259

[nitpick] Variable name optionParameter is ambiguous. Consider renaming to optionParameterErrors to clarify this stream holds validation error messages.

final Stream<String> optionParameter = finder

component-tools/src/test/java/org/talend/sdk/component/tools/validator/ActionValidatorTest.java:101

This test uses Arrays.asList and Collectors.toList() but the imports for java.util.Arrays, java.util.List, and java.util.stream.Collectors may be missing. Add them to ensure the test compiles.

final List<String> errors = validator.validate(finder, Arrays.asList(ActionDatabaseMappingKO.class))

component-tools/src/main/java/org/talend/sdk/component/tools/validator/ActionValidator.java

...rc/main/java/org/talend/sdk/component/runtime/manager/extension/ComponentSchemaEnricher.java

...ent-api/src/main/java/org/talend/sdk/component/api/service/schema/DatabaseSchemaMapping.java

...est/java/org/talend/sdk/component/runtime/manager/extension/ComponentSchemaEnricherTest.java

ozhelezniak-talend · 2025-06-10T08:53:25Z

The implementation is still too specific to the studio. It uses Studio's implementation of file mapping as a core element.
During the meetings that I've heard it was introduced differently.
Clearly some miscommunication.
Maybe QLIK cloud can use this RAW DB type thing as well, so we might need to have some kind of mapping there too. It's just a thought.
@ypiel-talend what do you think (as proxy-po)

sonar-eks · 2025-06-11T09:47:24Z

Analysis Details

1 Issue

0 Bugs
0 Vulnerabilities
1 Code Smell

Coverage and Duplications

97.90% Coverage (56.50% Estimated after merge)
1.49% Duplicated Code (1.40% Estimated after merge)

Project ID: org.talend.sdk.component:component-runtime

View in SonarQube

ozhelezniak-talend

Contacted Yves.
Looks like the implementation approach was agreed.
It's studio specific.
Maybe if we find smth better in future, we would be able to deprecate this one and replace it with a new one.
Conceptually this project becomes less and less public and universal =)

feat(QTDI-1060): db mappings

a4afd1f

This comment has been minimized.

Sign in to view

feat(QTDI-1060): add validations

9d02776

This comment has been minimized.

Sign in to view

undx added 2 commits April 7, 2025 14:43

Merge branch 'master' into undx/QTDI-1060-dbtype

c6bc580

feat(QTDI-1060): add sample features for studio

2c3ae0d

This comment has been minimized.

Sign in to view

feat(QTDI-1060): add mapping to FamilyMeta

dbd1d72

This comment has been minimized.

Sign in to view

undx added 2 commits April 10, 2025 15:32

Merge remote-tracking branch 'origin/master' into undx/QTDI-1060-dbtype

a264fbc

feat(QTDI-1060): fix sonar comments

03dcb69

This comment has been minimized.

Sign in to view

undx requested a review from ozhelezniak-talend April 10, 2025 14:36

ozhelezniak-talend reviewed Apr 11, 2025

View reviewed changes

...st/java/org/talend/sdk/component/design/extension/repository/RepositoryModelBuilderTest.java Outdated Show resolved Hide resolved

ozhelezniak-talend reviewed Apr 11, 2025

View reviewed changes

...runtime-manager/src/main/java/org/talend/sdk/component/runtime/manager/ComponentManager.java Outdated Show resolved Hide resolved

ozhelezniak-talend reviewed Apr 11, 2025

View reviewed changes

...runtime-manager/src/main/java/org/talend/sdk/component/runtime/manager/ComponentManager.java Outdated Show resolved Hide resolved

ozhelezniak-talend reviewed Apr 11, 2025

View reviewed changes

...nent-server-model/src/main/java/org/talend/sdk/component/server/front/model/ComponentId.java Outdated Show resolved Hide resolved

ozhelezniak-talend reviewed Apr 11, 2025

View reviewed changes

component-tools/src/main/java/org/talend/sdk/component/tools/validator/ActionValidator.java Outdated Show resolved Hide resolved

ozhelezniak-talend reviewed Apr 11, 2025

View reviewed changes

documentation/src/main/antora/modules/ROOT/pages/_partials/generated_actions.adoc Outdated Show resolved Hide resolved

ozhelezniak-talend reviewed Apr 11, 2025

View reviewed changes

...src/main/java/org/talend/sdk/component/sample/feature/databasemapping/service/UIService.java Outdated Show resolved Hide resolved

ozhelezniak-talend previously approved these changes Apr 11, 2025

View reviewed changes

feat(QTDI-1060): fix pr comments

92f94c7

undx dismissed ozhelezniak-talend’s stale review via 92f94c7 April 11, 2025 08:46

This comment has been minimized.

Sign in to view

undx added 2 commits April 11, 2025 13:51

Merge branch 'master' into undx/QTDI-1060-dbtype

5b24505

# Conflicts: # component-tools/src/main/java/org/talend/sdk/component/tools/validator/ActionValidator.java # sample-parent/sample-features/pom.xml

feat(QTDI-1060): reformat code merged from master

dafb83a

This comment has been minimized.

Sign in to view

ozhelezniak-talend previously approved these changes Apr 11, 2025

View reviewed changes

undx added 2 commits May 5, 2025 10:49

Merge branch 'master' into undx/QTDI-1060-dbtype

9ed4056

Merge branch 'refs/heads/master' into undx/QTDI-1060-dbtype

1c8a4df

undx added 2 commits May 21, 2025 12:19

Merge branch 'master' into undx/QTDI-1060-dbtype

70959fc

feat(QTDI-1060): bump artifact version to master

3b62f31

This comment has been minimized.

Sign in to view

undx added 2 commits May 27, 2025 17:25

feat(QTDI-1060): add SchemaInfo to sample

154cdb2

feat(QTDI-1060): add sourceType to Column from guess schema

bd05fb8

This comment has been minimized.

Sign in to view

undx added 2 commits June 2, 2025 13:54

Merge branch 'master' into undx/QTDI-1060-dbtype

9996926

feat(QTDI-1060): change service signature

b4e918b

This comment has been minimized.

Sign in to view

feat(QTDI-1060): add new processor for extended test

15f209a

This comment has been minimized.

Sign in to view

undx added 2 commits June 4, 2025 16:51

Merge branch 'master' into undx/QTDI-1060-dbtype

d466744

feat(QTDI-1060): add datastore rule on action

0e61788

This comment has been minimized.

Sign in to view

undx added 2 commits June 6, 2025 13:45

feat(QTDI-1060): add datastore in docrule on action

69c03b7

feat(QTDI-1060): add datastore in docrule on action

e75ed77

This comment has been minimized.

Sign in to view

undx added 2 commits June 6, 2025 15:20

feat(QTDI-1060): correct db/2 mapping name

84dacb7

feat(QTDI-1060): correct db/2 mapping name

565c6b7

undx requested a review from ozhelezniak-talend June 6, 2025 15:04

ozhelezniak-talend requested a review from Copilot June 10, 2025 08:04

Copilot AI reviewed Jun 10, 2025

View reviewed changes

feat(QTDI-1060): pull request comments

6d3e91e

ozhelezniak-talend approved these changes Jun 11, 2025

View reviewed changes

undx merged commit d750cd8 into master Jun 30, 2025
7 of 8 checks passed

undx deleted the undx/QTDI-1060-dbtype branch June 30, 2025 08:36

feat(QTDI-1060): db mappings #1017

feat(QTDI-1060): db mappings #1017

Conversation

undx commented Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

ozhelezniak-talend left a comment

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ozhelezniak-talend commented Jun 10, 2025

Uh oh!

sonar-eks bot commented Jun 11, 2025

Analysis Details

1 Issue

Coverage and Duplications

Uh oh!

ozhelezniak-talend left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

undx commented Apr 7, 2025 •

edited

Loading