Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exposing SSL-only version of Postgres Source #6362

Merged
merged 12 commits into from
Sep 27, 2021
Merged

Exposing SSL-only version of Postgres Source #6362

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
9aaae36
first pass
cgardens Sep 22, 2021
9f352a4
checkpoint for feedback
cgardens Sep 22, 2021
71b2183
hack
cgardens Sep 22, 2021
d567fdc
fix image name in acceptance tests
cgardens Sep 22, 2021
b5a556c
enforce ssl for postgres normalization
cgardens Sep 27, 2021
5f4b68a
various clean
cgardens Sep 27, 2021
50ad203
fix test
cgardens Sep 27, 2021
72069d4
so it goes
cgardens Sep 27, 2021
275096c
go
cgardens Sep 27, 2021
36e11d7
fix main method
cgardens Sep 27, 2021
aaf3eaa
fix the noise
cgardens Sep 27, 2021
72ba74b
add java doc comments
cgardens Sep 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
so it goes
  • Loading branch information
@cgardens
cgardens committed Sep 27, 2021
commit 72069d4e33ef66dc37430a65385e2cd1e6bbc52a
4 changes: 3 additions & 1 deletion ...ava/src/main/java/io/airbyte/integrations/base/spec_modification/SpecModifyingSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@
import io.airbyte.protocol.models.ConnectorSpecification;

/**
* In some cases we want to prune or mutate the spec for an existing source. The common case is that we want to remove features that are not appropriate for some reason. e.g. In cloud, we do not want to allow users to send data unencrypted.
* In some cases we want to prune or mutate the spec for an existing source. The common case is that
* we want to remove features that are not appropriate for some reason. e.g. In cloud, we do not
* want to allow users to send data unencrypted.
*/
public abstract class SpecModifyingSource implements Source {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if all we're doing is modify the spec is a whole new superclass worth it? why not just extend the source directly and modify the spec method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair. I think we can reuse this class for all the db sources that this applies to so it gives us a nice convention to work from if we want to expand it later. i can just document it once. otherwise we're just going to extend a bunch of sources and have to explain it in a bunch of places.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree with @cgardens the decoration pattern makes the feature of modifying the spec more composable


Expand Down
1 change: 0 additions & 1 deletion .../testFixtures/java/io/airbyte/integrations/source/jdbc/test/JdbcSourceAcceptanceTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@
import java.util.Set;
import java.util.function.Function;
import java.util.stream.Collectors;
import org.apache.commons.lang3.tuple.ImmutablePair;
import org.hamcrest.Matchers;
import org.junit.jupiter.api.Test;

Expand Down
3 changes: 2 additions & 1 deletion ...e/src/test/java/io/airbyte/integrations/source/oracle/OracleJdbcSourceAcceptanceTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,8 @@ void cleanUpTables() throws SQLException {
ORACLE_DB.getUsername(),
ORACLE_DB.getPassword());
for (final String schemaName : TEST_SCHEMAS) {
final ResultSet resultSet = conn.createStatement().executeQuery(String.format("SELECT TABLE_NAME FROM ALL_TABLES WHERE OWNER = '%s'", schemaName));
final ResultSet resultSet =
conn.createStatement().executeQuery(String.format("SELECT TABLE_NAME FROM ALL_TABLES WHERE OWNER = '%s'", schemaName));
while (resultSet.next()) {
final String tableName = resultSet.getString("TABLE_NAME");
final String tableNameProcessed = tableName.contains(" ") ? SourceJdbcUtils
Expand Down
2 changes: 1 addition & 1 deletion ...o/airbyte/integrations/source/postgres/PostgresStrictEncryptJdbcSourceAcceptanceTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ public JdbcSource getJdbcSource() {

@Override
public Source getSource() {
return new PostgresSourceStrictEncrypt();
return new PostgresSourceStrictEncrypt();
}

@Override
Expand Down
4 changes: 3 additions & 1 deletion ...source-postgres/src/main/java/io/airbyte/integrations/source/postgres/PostgresSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,9 @@ public JsonNode toDatabaseConfig(final JsonNode config) {
return toDatabaseConfigStatic(config);
}

// todo (cgardens) - restructure AbstractJdbcSource so to take this function in the constructor. the current structure forces us to declarehave a bunch of pure function methods as instance members when they could be static.
// todo (cgardens) - restructure AbstractJdbcSource so to take this function in the constructor. the
// current structure forces us to declarehave a bunch of pure function methods as instance members
// when they could be static.
public JsonNode toDatabaseConfigStatic(final JsonNode config) {
final List<String> additionalParameters = new ArrayList<>();

Expand Down