Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bmoric/convert destination controller micronaut #20269

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
b42d814
tmp
benmoriceau Nov 23, 2022
5da2868
Fix build
benmoriceau Nov 28, 2022
83c3edc
tmp
benmoriceau Nov 28, 2022
27ccea0
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Nov 28, 2022
e3323ee
Tmp
benmoriceau Nov 29, 2022
648494e
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Nov 30, 2022
9f4fee5
tmp
benmoriceau Dec 1, 2022
2391852
tmp
benmoriceau Dec 1, 2022
453f2fd
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 5, 2022
9032d55
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 5, 2022
212dd86
Tmp
benmoriceau Dec 5, 2022
8fd0939
tmp
benmoriceau Dec 6, 2022
d378aab
tmp
benmoriceau Dec 6, 2022
0ff5a15
Clean up
benmoriceau Dec 6, 2022
8dcef2e
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 7, 2022
4ad415e
tmp
benmoriceau Dec 7, 2022
58d51db
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 7, 2022
c982867
Convert Connection Api Controller
benmoriceau Dec 7, 2022
8de361a
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 7, 2022
3580902
Tmp
benmoriceau Dec 8, 2022
fccb967
PR Comments
benmoriceau Dec 8, 2022
e0730b3
convert openapiapicontroller to micronaut (#20258)
colesnodgrass Dec 8, 2022
8855858
Fix bean
benmoriceau Dec 8, 2022
9f0da43
Add JsonSchemaValidator as a Bean
benmoriceau Dec 8, 2022
df5a30e
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 8, 2022
0024dff
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 8, 2022
4d8d7c2
Fix build
benmoriceau Dec 8, 2022
abbf357
Format
benmoriceau Dec 8, 2022
395c900
Merge brach 'bmoric/convert-connection-micronaut' of github.com:airby…
benmoriceau Dec 8, 2022
50a7bb6
Format
benmoriceau Dec 9, 2022
0cbc790
Test fix
benmoriceau Dec 9, 2022
67b69e5
Pr comments
benmoriceau Dec 9, 2022
c053aa7
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 9, 2022
7490f23
Remove media type
benmoriceau Dec 9, 2022
556e7d3
Format
benmoriceau Dec 9, 2022
b320000
Merge branch 'bmoric/convert-connection-micronaut' of github.com:airb…
benmoriceau Dec 9, 2022
f7698c5
Remove media type
benmoriceau Dec 9, 2022
f143e2a
Format
benmoriceau Dec 9, 2022
ef2b4a3
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 9, 2022
9fa00ff
format
benmoriceau Dec 9, 2022
bd5caee
Add missing airbyte api client
benmoriceau Dec 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ public class AirbyteApiClient {
public AirbyteApiClient(final ApiClient apiClient, final ApiClient micronautApiClient) {
connectionApi = new ConnectionApi(micronautApiClient);
destinationDefinitionApi = new DestinationDefinitionApi(apiClient);
destinationApi = new DestinationApi(apiClient);
destinationApi = new DestinationApi(micronautApiClient);
destinationSpecificationApi = new DestinationDefinitionSpecificationApi(apiClient);
jobsApi = new JobsApi(apiClient);
logsApi = new PatchedLogsApi(apiClient);
Expand Down
4 changes: 2 additions & 2 deletions airbyte-proxy/nginx-auth.conf.template
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ http {
}
}

location ~ ^/api/v1/(connections|operations)/.* {
location ~ ^/api/v1/(connections|destinations|operations)/.* {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down Expand Up @@ -93,7 +93,7 @@ http {
}
}

location ~ ^/api/v1/(connections|operations)/.* {
location ~ ^/api/v1/(connections|destinations|operations)/.* {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down
4 changes: 2 additions & 2 deletions airbyte-proxy/nginx-no-auth.conf.template
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ http {
proxy_pass "${PROXY_PASS_MICRONAUT_API}";
}

location ~ ^/api/v1/(connections|operations)/.* {
location ~ ^/api/v1/(connections|destinations|operations)/.* {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down Expand Up @@ -48,7 +48,7 @@ http {
proxy_pass "${PROXY_PASS_MICRONAUT_API}";
}

location ~ ^/api/v1/(connections|operations)/.* {
location ~ ^/api/v1/(connections|destinations|operations)/.* {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
import io.airbyte.config.persistence.SecretsRepositoryWriter;
import io.airbyte.db.Database;
import io.airbyte.persistence.job.JobPersistence;
import io.airbyte.server.apis.DestinationApiController;
import io.airbyte.server.apis.DestinationDefinitionApiController;
import io.airbyte.server.apis.DestinationDefinitionSpecificationApiController;
import io.airbyte.server.apis.DestinationOauthApiController;
Expand All @@ -30,7 +29,6 @@
import io.airbyte.server.apis.StateApiController;
import io.airbyte.server.apis.WebBackendApiController;
import io.airbyte.server.apis.WorkspaceApiController;
import io.airbyte.server.apis.binders.DestinationApiBinder;
import io.airbyte.server.apis.binders.DestinationDefinitionApiBinder;
import io.airbyte.server.apis.binders.DestinationDefinitionSpecificationApiBinder;
import io.airbyte.server.apis.binders.DestinationOauthApiBinder;
Expand All @@ -45,7 +43,6 @@
import io.airbyte.server.apis.binders.StateApiBinder;
import io.airbyte.server.apis.binders.WebBackendApiBinder;
import io.airbyte.server.apis.binders.WorkspaceApiBinder;
import io.airbyte.server.apis.factories.DestinationApiFactory;
import io.airbyte.server.apis.factories.DestinationDefinitionApiFactory;
import io.airbyte.server.apis.factories.DestinationDefinitionSpecificationApiFactory;
import io.airbyte.server.apis.factories.DestinationOauthApiFactory;
Expand Down Expand Up @@ -160,8 +157,6 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul
final WebBackendCheckUpdatesHandler webBackendCheckUpdatesHandler) {
final Map<String, String> mdc = MDC.getCopyOfContextMap();

DestinationApiFactory.setValues(destinationApiHandler, schedulerHandler, mdc);

DestinationDefinitionApiFactory.setValues(destinationDefinitionsHandler);

DestinationDefinitionSpecificationApiFactory.setValues(schedulerHandler);
Expand Down Expand Up @@ -192,7 +187,6 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul

// server configuration
final Set<Class<?>> componentClasses = Set.of(
DestinationApiController.class,
DestinationDefinitionApiController.class,
DestinationDefinitionSpecificationApiController.class,
DestinationOauthApiController.class,
Expand All @@ -211,7 +205,6 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul
WorkspaceApiController.class);

final Set<Object> components = Set.of(
new DestinationApiBinder(),
new DestinationDefinitionApiBinder(),
new DestinationDefinitionSpecificationApiBinder(),
new DestinationOauthApiBinder(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,61 +16,72 @@
import io.airbyte.api.model.generated.WorkspaceIdRequestBody;
import io.airbyte.server.handlers.DestinationHandler;
import io.airbyte.server.handlers.SchedulerHandler;
import javax.ws.rs.Path;
import io.micronaut.http.annotation.Body;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Post;
import lombok.AllArgsConstructor;

@Path("/v1/destinations")
@Controller("/api/v1/destinations")
@AllArgsConstructor
public class DestinationApiController implements DestinationApi {

private final DestinationHandler destinationHandler;
private final SchedulerHandler schedulerHandler;

@Post(uri = "/check_connection")
@Override
public CheckConnectionRead checkConnectionToDestination(final DestinationIdRequestBody destinationIdRequestBody) {
public CheckConnectionRead checkConnectionToDestination(@Body final DestinationIdRequestBody destinationIdRequestBody) {
return ApiHelper.execute(() -> schedulerHandler.checkDestinationConnectionFromDestinationId(destinationIdRequestBody));
}

@Post(uri = "/check_connection_for_update")
@Override
public CheckConnectionRead checkConnectionToDestinationForUpdate(final DestinationUpdate destinationUpdate) {
public CheckConnectionRead checkConnectionToDestinationForUpdate(@Body final DestinationUpdate destinationUpdate) {
return ApiHelper.execute(() -> schedulerHandler.checkDestinationConnectionFromDestinationIdForUpdate(destinationUpdate));
}

@Post(uri = "/clone")
@Override
public DestinationRead cloneDestination(final DestinationCloneRequestBody destinationCloneRequestBody) {
public DestinationRead cloneDestination(@Body final DestinationCloneRequestBody destinationCloneRequestBody) {
return ApiHelper.execute(() -> destinationHandler.cloneDestination(destinationCloneRequestBody));
}

@Post(uri = "/create")
@Override
public DestinationRead createDestination(final DestinationCreate destinationCreate) {
public DestinationRead createDestination(@Body final DestinationCreate destinationCreate) {
return ApiHelper.execute(() -> destinationHandler.createDestination(destinationCreate));
}

@Post(uri = "/delete")
@Override
public void deleteDestination(final DestinationIdRequestBody destinationIdRequestBody) {
public void deleteDestination(@Body final DestinationIdRequestBody destinationIdRequestBody) {
ApiHelper.execute(() -> {
destinationHandler.deleteDestination(destinationIdRequestBody);
return null;
});
}

@Post(uri = "/get")
@Override
public DestinationRead getDestination(final DestinationIdRequestBody destinationIdRequestBody) {
public DestinationRead getDestination(@Body final DestinationIdRequestBody destinationIdRequestBody) {
return ApiHelper.execute(() -> destinationHandler.getDestination(destinationIdRequestBody));
}

@Post(uri = "/list")
@Override
public DestinationReadList listDestinationsForWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) {
public DestinationReadList listDestinationsForWorkspace(@Body final WorkspaceIdRequestBody workspaceIdRequestBody) {
return ApiHelper.execute(() -> destinationHandler.listDestinationsForWorkspace(workspaceIdRequestBody));
}

@Post(uri = "/search")
@Override
public DestinationReadList searchDestinations(final DestinationSearch destinationSearch) {
public DestinationReadList searchDestinations(@Body final DestinationSearch destinationSearch) {
return ApiHelper.execute(() -> destinationHandler.searchDestinations(destinationSearch));
}

@Post(uri = "/update")
@Override
public DestinationRead updateDestination(final DestinationUpdate destinationUpdate) {
public DestinationRead updateDestination(@Body final DestinationUpdate destinationUpdate) {
return ApiHelper.execute(() -> destinationHandler.updateDestination(destinationUpdate));
}

Expand Down

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import io.airbyte.config.Configs.DeploymentMode;
import io.airbyte.config.Configs.TrackingStrategy;
import io.airbyte.config.persistence.ConfigRepository;
import io.airbyte.config.persistence.split_secrets.JsonSecretsProcessor;
import io.airbyte.persistence.job.JobPersistence;
import io.airbyte.persistence.job.WebUrlHelper;
import io.airbyte.persistence.job.tracker.JobTracker;
Expand Down Expand Up @@ -85,6 +86,13 @@ public Path workspaceRoot(@Value("${airbyte.workspace.root}") final String works
return Path.of(workspaceRoot);
}

@Singleton
public JsonSecretsProcessor jsonSecretsProcessor(final FeatureFlags featureFlags) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note, should we explicitly disable the JsonSecretProcessor for DATA_PLANE?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gosusnp I believe that this was added because originally the code loaded when in data-plane mode did not require this singleton. I'm not sure if that is still the case. If that has changed and it is now required, we should remove the @Requires annotation. If it is not used by anything in data-plane mode, we should leave this to avoid instantiating a singleton that we don't need. cc: @benmoriceau

return JsonSecretsProcessor.builder()
.copySecrets(false)
.build();
}

private <T> T convertToEnum(final String value, final Function<String, T> creatorFunction, final T defaultValue) {
return StringUtils.isNotEmpty(value) ? creatorFunction.apply(value.toUpperCase(Locale.ROOT)) : defaultValue;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,11 @@
import io.airbyte.config.persistence.split_secrets.JsonSecretsProcessor;
import io.airbyte.protocol.models.ConnectorSpecification;
import io.airbyte.validation.json.JsonValidationException;
import jakarta.inject.Singleton;
import java.io.IOException;
import java.util.UUID;

@Singleton
public class ConfigurationUpdate {

private final ConfigRepository configRepository;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,13 @@
import io.airbyte.server.converters.ConfigurationUpdate;
import io.airbyte.validation.json.JsonSchemaValidator;
import io.airbyte.validation.json.JsonValidationException;
import jakarta.inject.Singleton;
import java.io.IOException;
import java.util.List;
import java.util.UUID;
import java.util.function.Supplier;

@Singleton
public class DestinationHandler {

private final ConnectionsHandler connectionsHandler;
Expand Down