Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bmoric/convert destination definition controller micronaut #20277

Merged
merged 47 commits into from
Dec 9, 2022
Merged

Bmoric/convert destination definition controller micronaut #20277

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
b42d814
tmp
benmoriceau Nov 23, 2022
5da2868
Fix build
benmoriceau Nov 28, 2022
83c3edc
tmp
benmoriceau Nov 28, 2022
27ccea0
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Nov 28, 2022
e3323ee
Tmp
benmoriceau Nov 29, 2022
648494e
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Nov 30, 2022
9f4fee5
tmp
benmoriceau Dec 1, 2022
2391852
tmp
benmoriceau Dec 1, 2022
453f2fd
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 5, 2022
9032d55
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 5, 2022
212dd86
Tmp
benmoriceau Dec 5, 2022
8fd0939
tmp
benmoriceau Dec 6, 2022
d378aab
tmp
benmoriceau Dec 6, 2022
0ff5a15
Clean up
benmoriceau Dec 6, 2022
8dcef2e
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 7, 2022
4ad415e
tmp
benmoriceau Dec 7, 2022
58d51db
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 7, 2022
c982867
Convert Connection Api Controller
benmoriceau Dec 7, 2022
8de361a
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 7, 2022
3580902
Tmp
benmoriceau Dec 8, 2022
fccb967
PR Comments
benmoriceau Dec 8, 2022
e0730b3
convert openapiapicontroller to micronaut (#20258)
colesnodgrass Dec 8, 2022
8855858
Fix bean
benmoriceau Dec 8, 2022
9f0da43
Add JsonSchemaValidator as a Bean
benmoriceau Dec 8, 2022
df5a30e
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 8, 2022
0024dff
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 8, 2022
4d8d7c2
Fix build
benmoriceau Dec 8, 2022
9d6c512
Merge branch 'bmoric/convert-destination-controller-micronaut' of git…
benmoriceau Dec 8, 2022
abbf357
Format
benmoriceau Dec 8, 2022
395c900
Merge brach 'bmoric/convert-connection-micronaut' of github.com:airby…
benmoriceau Dec 8, 2022
b63bf4b
Merge branch 'bmoric/convert-destination-controller-micronaut' of git…
benmoriceau Dec 9, 2022
50a7bb6
Format
benmoriceau Dec 9, 2022
0cbc790
Test fix
benmoriceau Dec 9, 2022
67b69e5
Pr comments
benmoriceau Dec 9, 2022
c053aa7
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 9, 2022
7490f23
Remove media type
benmoriceau Dec 9, 2022
556e7d3
Format
benmoriceau Dec 9, 2022
b320000
Merge branch 'bmoric/convert-connection-micronaut' of github.com:airb…
benmoriceau Dec 9, 2022
f7698c5
Remove media type
benmoriceau Dec 9, 2022
f143e2a
Format
benmoriceau Dec 9, 2022
fc595d8
Merge branch 'bmoric/convert-destination-controller-micronaut' of git…
benmoriceau Dec 9, 2022
fa20dc1
Remove media type
benmoriceau Dec 9, 2022
f84305f
Format
benmoriceau Dec 9, 2022
e67f012
Merge branch 'bmoric/convert-health-micronaut' of github.com:airbyteh…
benmoriceau Dec 9, 2022
3fec796
api client
benmoriceau Dec 9, 2022
e9ad7a9
missing annotation
benmoriceau Dec 9, 2022
a8165f4
format
benmoriceau Dec 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-api/src/main/java/io/airbyte/api/client/AirbyteApiClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ public class AirbyteApiClient {

public AirbyteApiClient(final ApiClient apiClient, final ApiClient micronautApiClient) {
connectionApi = new ConnectionApi(micronautApiClient);
destinationDefinitionApi = new DestinationDefinitionApi(apiClient);
destinationDefinitionApi = new DestinationDefinitionApi(micronautApiClient);
destinationApi = new DestinationApi(micronautApiClient);
destinationSpecificationApi = new DestinationDefinitionSpecificationApi(apiClient);
jobsApi = new JobsApi(apiClient);
Expand Down
4 changes: 2 additions & 2 deletions airbyte-proxy/nginx-auth.conf.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ http {
}
}

location ~ ^/api/v1/(connections|destinations|operations)/.* {
location ~ ^/api/v1/(connections|destinations|destination_definitions|operations)/.* {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down Expand Up @@ -93,7 +93,7 @@ http {
}
}

location ~ ^/api/v1/(connections|destinations|operations)/.* {
location ~ ^/api/v1/(connections|destinations|destination_definitions|operations)/.* {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down
4 changes: 2 additions & 2 deletions airbyte-proxy/nginx-no-auth.conf.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ http {
proxy_pass "${PROXY_PASS_MICRONAUT_API}";
}

location ~ ^/api/v1/(connections|destinations|operations)/.* {
location ~ ^/api/v1/(connections|destinations|destination_definitions|operations)/.* {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down Expand Up @@ -48,7 +48,7 @@ http {
proxy_pass "${PROXY_PASS_MICRONAUT_API}";
}

location ~ ^/api/v1/(connections|destinations|operations)/.* {
location ~ ^/api/v1/(connections|destinations|destination_definitions|operations)/.* {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down
7 changes: 0 additions & 7 deletions airbyte-server/src/main/java/io/airbyte/server/ServerFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
import io.airbyte.config.persistence.SecretsRepositoryWriter;
import io.airbyte.db.Database;
import io.airbyte.persistence.job.JobPersistence;
import io.airbyte.server.apis.DestinationDefinitionApiController;
import io.airbyte.server.apis.DestinationDefinitionSpecificationApiController;
import io.airbyte.server.apis.DestinationOauthApiController;
import io.airbyte.server.apis.JobsApiController;
Expand All @@ -29,7 +28,6 @@
import io.airbyte.server.apis.StateApiController;
import io.airbyte.server.apis.WebBackendApiController;
import io.airbyte.server.apis.WorkspaceApiController;
import io.airbyte.server.apis.binders.DestinationDefinitionApiBinder;
import io.airbyte.server.apis.binders.DestinationDefinitionSpecificationApiBinder;
import io.airbyte.server.apis.binders.DestinationOauthApiBinder;
import io.airbyte.server.apis.binders.JobsApiBinder;
Expand All @@ -43,7 +41,6 @@
import io.airbyte.server.apis.binders.StateApiBinder;
import io.airbyte.server.apis.binders.WebBackendApiBinder;
import io.airbyte.server.apis.binders.WorkspaceApiBinder;
import io.airbyte.server.apis.factories.DestinationDefinitionApiFactory;
import io.airbyte.server.apis.factories.DestinationDefinitionSpecificationApiFactory;
import io.airbyte.server.apis.factories.DestinationOauthApiFactory;
import io.airbyte.server.apis.factories.JobsApiFactory;
Expand Down Expand Up @@ -157,8 +154,6 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul
final WebBackendCheckUpdatesHandler webBackendCheckUpdatesHandler) {
final Map<String, String> mdc = MDC.getCopyOfContextMap();

DestinationDefinitionApiFactory.setValues(destinationDefinitionsHandler);

DestinationDefinitionSpecificationApiFactory.setValues(schedulerHandler);

DestinationOauthApiFactory.setValues(oAuthHandler);
Expand Down Expand Up @@ -187,7 +182,6 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul

// server configuration
final Set<Class<?>> componentClasses = Set.of(
DestinationDefinitionApiController.class,
DestinationDefinitionSpecificationApiController.class,
DestinationOauthApiController.class,
JobsApiController.class,
Expand All @@ -205,7 +199,6 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul
WorkspaceApiController.class);

final Set<Object> components = Set.of(
new DestinationDefinitionApiBinder(),
new DestinationDefinitionSpecificationApiBinder(),
new DestinationOauthApiBinder(),
new JobsApiBinder(),
Expand Down
24 changes: 20 additions & 4 deletions airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionApiController.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,27 @@
import io.airbyte.api.model.generated.PrivateDestinationDefinitionReadList;
import io.airbyte.api.model.generated.WorkspaceIdRequestBody;
import io.airbyte.server.handlers.DestinationDefinitionsHandler;
import javax.ws.rs.Path;
import lombok.AllArgsConstructor;
import io.micronaut.context.annotation.Context;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Post;

@Path("/v1/destination_definitions")
@AllArgsConstructor
@Controller("/api/v1/destination_definitions")
@Context
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as on the Source PR, is this annotation necessary?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colesnodgrass I believe the intent is to force this singleton to be created on startup for performance reasons. Otherwise, the first call to this endpoint has to wait for all of the related beans to be initialized. We should look into why it is slow (maybe we are doing something in a constructor of a singleton that should be deferred, etc). I think that the use of @Context here is okay in this context because we are using it on endpoints/controllers to avoid a first request impact.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I answered in the other PR

public class DestinationDefinitionApiController implements DestinationDefinitionApi {

private final DestinationDefinitionsHandler destinationDefinitionsHandler;

public DestinationDefinitionApiController(final DestinationDefinitionsHandler destinationDefinitionsHandler) {
this.destinationDefinitionsHandler = destinationDefinitionsHandler;
}

@Post(uri = "/create_custom")
@Override
public DestinationDefinitionRead createCustomDestinationDefinition(final CustomDestinationDefinitionCreate customDestinationDefinitionCreate) {
return ApiHelper.execute(() -> destinationDefinitionsHandler.createCustomDestinationDefinition(customDestinationDefinitionCreate));
}

@Post(uri = "/delete")
@Override
public void deleteDestinationDefinition(final DestinationDefinitionIdRequestBody destinationDefinitionIdRequestBody) {
ApiHelper.execute(() -> {
Expand All @@ -37,42 +44,50 @@ public void deleteDestinationDefinition(final DestinationDefinitionIdRequestBody
});
}

@Post(uri = "/get")
@Override
public DestinationDefinitionRead getDestinationDefinition(final DestinationDefinitionIdRequestBody destinationDefinitionIdRequestBody) {
return ApiHelper.execute(() -> destinationDefinitionsHandler.getDestinationDefinition(destinationDefinitionIdRequestBody));
}

@Post(uri = "/get_for_workspace")
@Override
public DestinationDefinitionRead getDestinationDefinitionForWorkspace(final DestinationDefinitionIdWithWorkspaceId destinationDefinitionIdWithWorkspaceId) {
return ApiHelper.execute(() -> destinationDefinitionsHandler.getDestinationDefinitionForWorkspace(destinationDefinitionIdWithWorkspaceId));
}

@Post(uri = "/grant_definition")
@Override
public PrivateDestinationDefinitionRead grantDestinationDefinitionToWorkspace(final DestinationDefinitionIdWithWorkspaceId destinationDefinitionIdWithWorkspaceId) {
return ApiHelper
.execute(() -> destinationDefinitionsHandler.grantDestinationDefinitionToWorkspace(destinationDefinitionIdWithWorkspaceId));
}

@Post(uri = "/list")
@Override
public DestinationDefinitionReadList listDestinationDefinitions() {
return ApiHelper.execute(destinationDefinitionsHandler::listDestinationDefinitions);
}

@Post(uri = "/list_for_workspace")
@Override
public DestinationDefinitionReadList listDestinationDefinitionsForWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) {
return ApiHelper.execute(() -> destinationDefinitionsHandler.listDestinationDefinitionsForWorkspace(workspaceIdRequestBody));
}

@Post(uri = "/list_latest")
@Override
public DestinationDefinitionReadList listLatestDestinationDefinitions() {
return ApiHelper.execute(destinationDefinitionsHandler::listLatestDestinationDefinitions);
}

@Post(uri = "/list_private")
@Override
public PrivateDestinationDefinitionReadList listPrivateDestinationDefinitions(final WorkspaceIdRequestBody workspaceIdRequestBody) {
return ApiHelper.execute(() -> destinationDefinitionsHandler.listPrivateDestinationDefinitions(workspaceIdRequestBody));
}

@Post(uri = "/revoke_definition")
@Override
public void revokeDestinationDefinitionFromWorkspace(final DestinationDefinitionIdWithWorkspaceId destinationDefinitionIdWithWorkspaceId) {
ApiHelper.execute(() -> {
Expand All @@ -81,6 +96,7 @@ public void revokeDestinationDefinitionFromWorkspace(final DestinationDefinition
});
}

@Post(uri = "/update")
@Override
public DestinationDefinitionRead updateDestinationDefinition(final DestinationDefinitionUpdate destinationDefinitionUpdate) {
return ApiHelper.execute(() -> destinationDefinitionsHandler.updateDestinationDefinition(destinationDefinitionUpdate));
Expand Down
21 changes: 0 additions & 21 deletions ...e-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionApiBinder.java
View file
Open in desktop

This file was deleted.

29 changes: 0 additions & 29 deletions ...erver/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionApiFactory.java
View file
Open in desktop

This file was deleted.

5 changes: 4 additions & 1 deletion airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationDefinitionsHandler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
import io.airbyte.server.scheduler.SynchronousSchedulerClient;
import io.airbyte.server.services.AirbyteGithubStore;
import io.airbyte.validation.json.JsonValidationException;
import jakarta.inject.Singleton;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
Expand All @@ -53,6 +54,7 @@
import java.util.stream.Collectors;

@SuppressWarnings("PMD.AvoidCatchingNPE")
@Singleton
public class DestinationDefinitionsHandler {

private final ConfigRepository configRepository;
Expand Down Expand Up @@ -286,7 +288,8 @@ public void deleteDestinationDefinition(final DestinationDefinitionIdRequestBody
configRepository.writeStandardDestinationDefinition(persistedDestinationDefinition);
}

private ConnectorSpecification getSpecForImage(final String dockerRepository, final String imageTag, boolean isCustomConnector) throws IOException {
private ConnectorSpecification getSpecForImage(final String dockerRepository, final String imageTag, final boolean isCustomConnector)
throws IOException {
final String imageName = DockerUtils.getTaggedImageName(dockerRepository, imageTag);
final SynchronousResponse<ConnectorSpecification> getSpecResponse = schedulerSynchronousClient.createGetSpecJob(imageName, isCustomConnector);
return SpecFetcher.getSpecFromJob(getSpecResponse);
Expand Down