Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

editing oauth connector fields fails silently #7864

Closed
jrhizor opened this issue Nov 11, 2021 · 21 comments · Fixed by #8063
Closed

editing oauth connector fields fails silently #7864

jrhizor opened this issue Nov 11, 2021 · 21 comments · Fixed by #8063
Assignees
Labels
airbyte-cloud area/connectors Connector related issues area/platform issues related to the platform connectors/source/salesforce connectors/sources-api priority/high High priority type/bug Something isn't working

Comments

@jrhizor
Copy link
Contributor

jrhizor commented Nov 11, 2021

To reproduce with Salesforce on cloud:

  1. Create a source with Salesforce with OAuth
  2. Try to edit the start date.
  3. Get this error in the console:
instrument.ts:129 TypeError: Cannot read properties of undefined (reading 'value')
    at children (MainInfo.tsx:141)
    at createFormattedComponent.js:46
    at Vs (react-dom.production.min.js:274)
    at ju (react-dom.production.min.js:250)
    at Su (react-dom.production.min.js:250)
    at Eu (react-dom.production.min.js:250)
    at mu (react-dom.production.min.js:243)
    at react-dom.production.min.js:123
    at t.unstable_runWithPriority (scheduler.production.min.js:18)
    at qi (react-dom.production.min.js:122)
    at $i (react-dom.production.min.js:123)
    at Vi (react-dom.production.min.js:122)
    at fu (react-dom.production.min.js:237)
    at To (react-dom.production.min.js:170)
    at SourceSettings.tsx:67
    at c (runtime.js:63)
    at Generator._invoke (runtime.js:293)
    at Generator.throw (runtime.js:118)
    at r (asyncToGenerator.js:3)
    at s (asyncToGenerator.js:29)
  1. Get instantly redirected to the source page.
  2. Go back to the source settings and see the old value (due to the error).

I'm also getting a 500 error when clicking on the "Authenticate your Salesforce account" button in the edit page. Not sure if that's related or not.

I think there are up to three things to fix here:

  1. Editing fields for OAuth connectors
  2. Reauthenticating Oauth on the source page
  3. Making sure any errors like this TypeError are caught and shown as errors when submitting the form instead of silently failing in the future.
@jrhizor jrhizor added type/bug Something isn't working area/frontend priority/high High priority airbyte-cloud labels Nov 11, 2021
@sherifnada sherifnada added the area/connectors Connector related issues label Nov 15, 2021
@jrhizor jrhizor added area/platform issues related to the platform and removed area/frontend labels Nov 16, 2021
@jrhizor
Copy link
Contributor Author

jrhizor commented Nov 16, 2021

Error I'm seeing in the check_connection_for_update response:

{"message":"The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. \nErrors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required \nSchema: \n{\n  \"type\" : \"object\",\n  \"title\" : \"Salesforce Source Spec\",\n  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",\n  \"required\" : [ \"client_id\", \"client_secret\", \"refresh_token\", \"start_date\", \"api_type\" ],\n  \"properties\" : {\n    \"api_type\" : {\n      \"enum\" : [ \"BULK\", \"REST\" ],\n      \"type\" : \"string\",\n      \"default\" : \"BULK\",\n      \"description\" : \"Unless you know that you are transferring a very small amount of data, prefer using the BULK API. This will help avoid using up all of your API call quota with Salesforce. Valid values are BULK or REST.\"\n    },\n    \"client_id\" : {\n      \"type\" : \"string\",\n      \"description\" : \"The Consumer Key that can be found when viewing your app in Salesforce\"\n    },\n    \"is_sandbox\" : {\n      \"type\" : \"boolean\",\n      \"default\" : false,\n      \"description\" : \"Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false. We provide more info on this field in the <a href=\\\"https://docs.airbyte.io/integrations/destinations/salesforce#is_sandbox\\\">docs</a>.\"\n    },\n    \"start_date\" : {\n      \"type\" : \"string\",\n      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",\n      \"examples\" : [ \"2021-07-25T00:00:00Z\" ],\n      \"description\" : \"UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. This field uses the \\\"updated\\\" field if available, otherwise the \\\"created\\\" fields if they are available for a stream.\"\n    },\n    \"client_secret\" : {\n      \"type\" : \"string\",\n      \"description\" : \"The Consumer Secret that can be found when viewing your app in Salesforce\",\n      \"airbyte_secret\" : true\n    },\n    \"refresh_token\" : {\n      \"type\" : \"string\",\n      \"description\" : \"Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don't know what this is, follow this <a href=\\\"https://medium.com/@bpmmendis94/obtain-access-refresh-tokens-from-salesforce-rest-api-a324fe4ccd9b\\\">guide</a> to retrieve it.\",\n      \"airbyte_secret\" : true\n    }\n  },\n  \"additionalProperties\" : false\n}","exceptionClassName":"io.airbyte.server.errors.BadObjectSchemaKnownException","exceptionStack":["io.airbyte.server.errors.BadObjectSchemaKnownException: The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. ","Errors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required ","Schema: ","{","  \"type\" : \"object\",","  \"title\" : \"Salesforce Source Spec\",","  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",","  \"required\" : [ \"client_id\", \"client_secret\", \"refresh_token\", \"start_date\", \"api_type\" ],","  \"properties\" : {","    \"api_type\" : {","      \"enum\" : [ \"BULK\", \"REST\" ],","      \"type\" : \"string\",","      \"default\" : \"BULK\",","      \"description\" : \"Unless you know that you are transferring a very small amount of data, prefer using the BULK API. This will help avoid using up all of your API call quota with Salesforce. Valid values are BULK or REST.\"","    },","    \"client_id\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Key that can be found when viewing your app in Salesforce\"","    },","    \"is_sandbox\" : {","      \"type\" : \"boolean\",","      \"default\" : false,","      \"description\" : \"Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false. We provide more info on this field in the <a href=\\\"https://docs.airbyte.io/integrations/destinations/salesforce#is_sandbox\\\">docs</a>.\"","    },","    \"start_date\" : {","      \"type\" : \"string\",","      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",","      \"examples\" : [ \"2021-07-25T00:00:00Z\" ],","      \"description\" : \"UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. This field uses the \\\"updated\\\" field if available, otherwise the \\\"created\\\" fields if they are available for a stream.\"","    },","    \"client_secret\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Secret that can be found when viewing your app in Salesforce\",","      \"airbyte_secret\" : true","    },","    \"refresh_token\" : {","      \"type\" : \"string\",","      \"description\" : \"Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don't know what this is, follow this <a href=\\\"https://medium.com/@bpmmendis94/obtain-access-refresh-tokens-from-salesforce-rest-api-a324fe4ccd9b\\\">guide</a> to retrieve it.\",","      \"airbyte_secret\" : true","    }","  },","  \"additionalProperties\" : false","}","\tat io.airbyte.cloud.auth.AuthHelper.execute(AuthHelper.java:107)","\tat io.airbyte.cloud.auth.AuthHelper.callForWorkspace(AuthHelper.java:78)","\tat io.airbyte.server.wrapped.ConfigurationApiWrapped.checkConnectionToSourceForUpdate(ConfigurationApiWrapped.java:429)","\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)","\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)","\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)","\tat java.base/java.lang.reflect.Method.invoke(Method.java:564)","\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)","\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)","\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)","\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)","\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)","\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)","\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)","\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)","\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)","\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)","\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)","\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)","\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)","\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)","\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)","\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)","\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)","\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)","\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)","\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)","\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)","\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)","\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)","\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)","\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)","\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)","\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)","\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)","\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)","\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)","\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)","\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)","\tat org.eclipse.jetty.server.Server.handle(Server.java:501)","\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)","\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)","\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)","\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)","\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)","\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)","\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)","\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)","\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)","\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)","\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)","\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)","\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)","\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)","\tat java.base/java.lang.Thread.run(Thread.java:832)","Caused by: io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. ","Errors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required ","Schema: ","{","  \"type\" : \"object\",","  \"title\" : \"Salesforce Source Spec\",","  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",","  \"required\" : [ \"client_id\", \"client_secret\", \"refresh_token\", \"start_date\", \"api_type\" ],","  \"properties\" : {","    \"api_type\" : {","      \"enum\" : [ \"BULK\", \"REST\" ],","      \"type\" : \"string\",","      \"default\" : \"BULK\",","      \"description\" : \"Unless you know that you are transferring a very small amount of data, prefer using the BULK API. This will help avoid using up all of your API call quota with Salesforce. Valid values are BULK or REST.\"","    },","    \"client_id\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Key that can be found when viewing your app in Salesforce\"","    },","    \"is_sandbox\" : {","      \"type\" : \"boolean\",","      \"default\" : false,","      \"description\" : \"Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false. We provide more info on this field in the <a href=\\\"https://docs.airbyte.io/integrations/destinations/salesforce#is_sandbox\\\">docs</a>.\"","    },","    \"start_date\" : {","      \"type\" : \"string\",","      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",","      \"examples\" : [ \"2021-07-25T00:00:00Z\" ],","      \"description\" : \"UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. This field uses the \\\"updated\\\" field if available, otherwise the \\\"created\\\" fields if they are available for a stream.\"","    },","    \"client_secret\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Secret that can be found when viewing your app in Salesforce\",","      \"airbyte_secret\" : true","    },","    \"refresh_token\" : {","      \"type\" : \"string\",","      \"description\" : \"Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don't know what this is, follow this <a href=\\\"https://medium.com/@bpmmendis94/obtain-access-refresh-tokens-from-salesforce-rest-api-a324fe4ccd9b\\\">guide</a> to retrieve it.\",","      \"airbyte_secret\" : true","    }","  },","  \"additionalProperties\" : false","}","\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:68)","\tat io.airbyte.server.handlers.SchedulerHandler.checkSourceConnectionFromSourceIdForUpdate(SchedulerHandler.java:172)","\tat io.airbyte.server.wrapped.ConfigurationApiWrapped.lambda$checkConnectionToSourceForUpdate$25(ConfigurationApiWrapped.java:432)","\tat io.airbyte.cloud.auth.AuthHelper.execute(AuthHelper.java:101)","\t... 54 more"],"rootCauseExceptionClassName":"java.lang.Class","rootCauseExceptionStack":["io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. ","Errors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required ","Schema: ","{","  \"type\" : \"object\",","  \"title\" : \"Salesforce Source Spec\",","  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",","  \"required\" : [ \"client_id\", \"client_secret\", \"refresh_token\", \"start_date\", \"api_type\" ],","  \"properties\" : {","    \"api_type\" : {","      \"enum\" : [ \"BULK\", \"REST\" ],","      \"type\" : \"string\",","      \"default\" : \"BULK\",","      \"description\" : \"Unless you know that you are transferring a very small amount of data, prefer using the BULK API. This will help avoid using up all of your API call quota with Salesforce. Valid values are BULK or REST.\"","    },","    \"client_id\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Key that can be found when viewing your app in Salesforce\"","    },","    \"is_sandbox\" : {","      \"type\" : \"boolean\",","      \"default\" : false,","      \"description\" : \"Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false. We provide more info on this field in the <a href=\\\"https://docs.airbyte.io/integrations/destinations/salesforce#is_sandbox\\\">docs</a>.\"","    },","    \"start_date\" : {","      \"type\" : \"string\",","      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",","      \"examples\" : [ \"2021-07-25T00:00:00Z\" ],","      \"description\" : \"UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. This field uses the \\\"updated\\\" field if available, otherwise the \\\"created\\\" fields if they are available for a stream.\"","    },","    \"client_secret\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Secret that can be found when viewing your app in Salesforce\",","      \"airbyte_secret\" : true","    },","    \"refresh_token\" : {","      \"type\" : \"string\",","      \"description\" : \"Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don't know what this is, follow this <a href=\\\"https://medium.com/@bpmmendis94/obtain-access-refresh-tokens-from-salesforce-rest-api-a324fe4ccd9b\\\">guide</a> to retrieve it.\",","      \"airbyte_secret\" : true","    }","  },","  \"additionalProperties\" : false","}","\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:68)","\tat io.airbyte.server.handlers.SchedulerHandler.checkSourceConnectionFromSourceIdForUpdate(SchedulerHandler.java:172)","\tat io.airbyte.server.wrapped.ConfigurationApiWrapped.lambda$checkConnectionToSourceForUpdate$25(ConfigurationApiWrapped.java:432)","\tat io.airbyte.cloud.auth.AuthHelper.execute(AuthHelper.java:101)","\tat io.airbyte.cloud.auth.AuthHelper.callForWorkspace(AuthHelper.java:78)","\tat io.airbyte.server.wrapped.ConfigurationApiWrapped.checkConnectionToSourceForUpdate(ConfigurationApiWrapped.java:429)","\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)","\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)","\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)","\tat java.base/java.lang.reflect.Method.invoke(Method.java:564)","\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)","\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)","\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)","\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)","\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)","\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)","\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)","\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)","\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)","\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)","\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)","\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)","\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)","\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)","\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)","\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)","\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)","\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)","\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)","\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)","\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)","\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)","\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)","\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)","\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)","\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)","\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)","\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)","\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)","\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)","\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)","\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)","\tat org.eclipse.jetty.server.Server.handle(Server.java:501)","\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)","\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)","\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)","\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)","\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)","\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)","\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)","\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)","\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)","\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)","\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)","\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)","\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)","\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)","\tat java.base/java.lang.Thread.run(Thread.java:832)"]}

@cgardens
Copy link
Contributor

{"message":"The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. \nErrors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required \nSchema: \n{\n  \"type\" : \"object\",\n  \"title\" : \"Salesforce Source Spec\",\n  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",\n  \"required\" : [ \"client_id\", \"client_secret\", \"refresh_token\", \"start_date\", \"api_type\" ],\n  \"properties\" : {\n    \"api_type\" : {\n      \"enum\" : [ \"BULK\", \"REST\" ],\n      \"type\" : \"string\",\n      \"default\" : \"BULK\",\n      \"description\" : \"Unless you know that you are transferring a very small amount of data, prefer using the BULK API. This will help avoid using up all of your API call quota with Salesforce. Valid values are BULK or REST.\"\n    },\n    \"client_id\" : {\n      \"type\" : \"string\",\n      \"description\" : \"The Consumer Key that can be found when viewing your app in Salesforce\"\n    },\n    \"is_sandbox\" : {\n      \"type\" : \"boolean\",\n      \"default\" : false,\n      \"description\" : \"Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false. We provide more info on this field in the <a href=\\\"https://docs.airbyte.io/integrations/destinations/salesforce#is_sandbox\\\">docs</a>.\"\n    },\n    \"start_date\" : {\n      \"type\" : \"string\",\n      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",\n      \"examples\" : [ \"2021-07-25T00:00:00Z\" ],\n      \"description\" : \"UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. This field uses the \\\"updated\\\" field if available, otherwise the \\\"created\\\" fields if they are available for a stream.\"\n    },\n    \"client_secret\" : {\n      \"type\" : \"string\",\n      \"description\" : \"The Consumer Secret that can be found when viewing your app in Salesforce\",\n      \"airbyte_secret\" : true\n    },\n    \"refresh_token\" : {\n      \"type\" : \"string\",\n      \"description\" : \"Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don't know what this is, follow this <a href=\\\"https://medium.com/@bpmmendis94/obtain-access-refresh-tokens-from-salesforce-rest-api-a324fe4ccd9b\\\">guide</a> to retrieve it.\",\n      \"airbyte_secret\" : true\n    }\n  },\n  \"additionalProperties\" : false\n}","exceptionClassName":"io.airbyte.server.errors.BadObjectSchemaKnownException","exceptionStack":["io.airbyte.server.errors.BadObjectSchemaKnownException: The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. ","Errors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required ","Schema: ","{","  \"type\" : \"object\",","  \"title\" : \"Salesforce Source Spec\",","  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",","  \"required\" : [ \"client_id\", \"client_secret\", \"refresh_token\", \"start_date\", \"api_type\" ],","  \"properties\" : {","    \"api_type\" : {","      \"enum\" : [ \"BULK\", \"REST\" ],","      \"type\" : \"string\",","      \"default\" : \"BULK\",","      \"description\" : \"Unless you know that you are transferring a very small amount of data, prefer using the BULK API. This will help avoid using up all of your API call quota with Salesforce. Valid values are BULK or REST.\"","    },","    \"client_id\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Key that can be found when viewing your app in Salesforce\"","    },","    \"is_sandbox\" : {","      \"type\" : \"boolean\",","      \"default\" : false,","      \"description\" : \"Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false. We provide more info on this field in the <a href=\\\"https://docs.airbyte.io/integrations/destinations/salesforce#is_sandbox\\\">docs</a>.\"","    },","    \"start_date\" : {","      \"type\" : \"string\",","      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",","      \"examples\" : [ \"2021-07-25T00:00:00Z\" ],","      \"description\" : \"UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. This field uses the \\\"updated\\\" field if available, otherwise the \\\"created\\\" fields if they are available for a stream.\"","    },","    \"client_secret\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Secret that can be found when viewing your app in Salesforce\",","      \"airbyte_secret\" : true","    },","    \"refresh_token\" : {","      \"type\" : \"string\",","      \"description\" : \"Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don't know what this is, follow this <a href=\\\"https://medium.com/@bpmmendis94/obtain-access-refresh-tokens-from-salesforce-rest-api-a324fe4ccd9b\\\">guide</a> to retrieve it.\",","      \"airbyte_secret\" : true","    }","  },","  \"additionalProperties\" : false","}",
"\tat io.airbyte.cloud.auth.AuthHelper.execute(AuthHelper.java:107)",
"\tat io.airbyte.cloud.auth.AuthHelper.callForWorkspace(AuthHelper.java:78)",
"\tat io.airbyte.server.wrapped.ConfigurationApiWrapped.checkConnectionToSourceForUpdate(ConfigurationApiWrapped.java:429)",
"\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
"\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
"\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
"\tat java.base/java.lang.reflect.Method.invoke(Method.java:564)",
"\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)",
"\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)",
"\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)",
"\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)",
"\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)",
"\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)",
"\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)",
"\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)",
"\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)",
"\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)",
"\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)",
"\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)",
"\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)",
"\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)",
"\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)",
"\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)",
"\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)",
"\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)",
"\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)",
"\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)",
"\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)",
"\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)",
"\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)",
"\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)",
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)",
"\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)",
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)",
"\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)",
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)",
"\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)",
"\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)",
"\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)",
"\tat org.eclipse.jetty.server.Server.handle(Server.java:501)",
"\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)",
"\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)",
"\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)",
"\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)",
"\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)",
"\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)",
"\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)",
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)",
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)",
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)",
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)",
"\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)",
"\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)",
"\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)",
"\tat java.base/java.lang.Thread.run(Thread.java:832)","Caused by: io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. ","Errors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required ","Schema: ","{","  \"type\" : \"object\",","  \"title\" : \"Salesforce Source Spec\",","  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",","  \"required\" : [ \"client_id\", \"client_secret\", \"refresh_token\", \"start_date\", \"api_type\" ],","  \"properties\" : {","    \"api_type\" : {","      \"enum\" : [ \"BULK\", \"REST\" ],","      \"type\" : \"string\",","      \"default\" : \"BULK\",","      \"description\" : \"Unless you know that you are transferring a very small amount of data, prefer using the BULK API. This will help avoid using up all of your API call quota with Salesforce. Valid values are BULK or REST.\"","    },","    \"client_id\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Key that can be found when viewing your app in Salesforce\"","    },","    \"is_sandbox\" : {","      \"type\" : \"boolean\",","      \"default\" : false,","      \"description\" : \"Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false. We provide more info on this field in the <a href=\\\"https://docs.airbyte.io/integrations/destinations/salesforce#is_sandbox\\\">docs</a>.\"","    },","    \"start_date\" : {","      \"type\" : \"string\",","      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",","      \"examples\" : [ \"2021-07-25T00:00:00Z\" ],","      \"description\" : \"UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. This field uses the \\\"updated\\\" field if available, otherwise the \\\"created\\\" fields if they are available for a stream.\"","    },","    \"client_secret\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Secret that can be found when viewing your app in Salesforce\",","      \"airbyte_secret\" : true","    },","    \"refresh_token\" : {","      \"type\" : \"string\",","      \"description\" : \"Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don't know what this is, follow this <a href=\\\"https://medium.com/@bpmmendis94/obtain-access-refresh-tokens-from-salesforce-rest-api-a324fe4ccd9b\\\">guide</a> to retrieve it.\",","      \"airbyte_secret\" : true","    }","  },","  \"additionalProperties\" : false","}",
"\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:68)",
"\tat io.airbyte.server.handlers.SchedulerHandler.checkSourceConnectionFromSourceIdForUpdate(SchedulerHandler.java:172)",
"\tat io.airbyte.server.wrapped.ConfigurationApiWrapped.lambda$checkConnectionToSourceForUpdate$25(ConfigurationApiWrapped.java:432)",
"\tat io.airbyte.cloud.auth.AuthHelper.execute(AuthHelper.java:101)","\t... 54 more"],"rootCauseExceptionClassName":"java.lang.Class","rootCauseExceptionStack":["io.airbyte.validation.json.JsonValidationException: json schema validation failed when comparing the data to the json schema. ","Errors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required ","Schema: ","{","  \"type\" : \"object\",","  \"title\" : \"Salesforce Source Spec\",","  \"$schema\" : \"http://json-schema.org/draft-07/schema#\",","  \"required\" : [ \"client_id\", \"client_secret\", \"refresh_token\", \"start_date\", \"api_type\" ],","  \"properties\" : {","    \"api_type\" : {","      \"enum\" : [ \"BULK\", \"REST\" ],","      \"type\" : \"string\",","      \"default\" : \"BULK\",","      \"description\" : \"Unless you know that you are transferring a very small amount of data, prefer using the BULK API. This will help avoid using up all of your API call quota with Salesforce. Valid values are BULK or REST.\"","    },","    \"client_id\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Key that can be found when viewing your app in Salesforce\"","    },","    \"is_sandbox\" : {","      \"type\" : \"boolean\",","      \"default\" : false,","      \"description\" : \"Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false. We provide more info on this field in the <a href=\\\"https://docs.airbyte.io/integrations/destinations/salesforce#is_sandbox\\\">docs</a>.\"","    },","    \"start_date\" : {","      \"type\" : \"string\",","      \"pattern\" : \"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$\",","      \"examples\" : [ \"2021-07-25T00:00:00Z\" ],","      \"description\" : \"UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. This field uses the \\\"updated\\\" field if available, otherwise the \\\"created\\\" fields if they are available for a stream.\"","    },","    \"client_secret\" : {","      \"type\" : \"string\",","      \"description\" : \"The Consumer Secret that can be found when viewing your app in Salesforce\",","      \"airbyte_secret\" : true","    },","    \"refresh_token\" : {","      \"type\" : \"string\",","      \"description\" : \"Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don't know what this is, follow this <a href=\\\"https://medium.com/@bpmmendis94/obtain-access-refresh-tokens-from-salesforce-rest-api-a324fe4ccd9b\\\">guide</a> to retrieve it.\",","      \"airbyte_secret\" : true","    }","  },","  \"additionalProperties\" : false","}",
"\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:68)",
"\tat io.airbyte.server.handlers.SchedulerHandler.checkSourceConnectionFromSourceIdForUpdate(SchedulerHandler.java:172)",
"\tat io.airbyte.server.wrapped.ConfigurationApiWrapped.lambda$checkConnectionToSourceForUpdate$25(ConfigurationApiWrapped.java:432)",
"\tat io.airbyte.cloud.auth.AuthHelper.execute(AuthHelper.java:101)",
"\tat io.airbyte.cloud.auth.AuthHelper.callForWorkspace(AuthHelper.java:78)",
"\tat io.airbyte.server.wrapped.ConfigurationApiWrapped.checkConnectionToSourceForUpdate(ConfigurationApiWrapped.java:429)",
"\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
"\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
"\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
"\tat java.base/java.lang.reflect.Method.invoke(Method.java:564)",
"\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)",
"\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)",
"\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)",
"\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)",
"\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)",
"\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)",
"\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)",
"\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)",
"\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)",
"\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)",
"\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)",
"\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)",
"\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)",
"\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)",
"\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)",
"\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)",
"\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)",
"\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)",
"\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)",
"\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)",
"\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)",
"\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)",
"\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)",
"\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)",
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)",
"\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)",
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)",
"\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)",
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)",
"\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)",
"\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)",
"\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)",
"\tat org.eclipse.jetty.server.Server.handle(Server.java:501)",
"\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)",
"\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)",
"\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)",
"\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)",
"\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)",
"\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)",
"\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)",
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)",
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)",
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)",
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)",
"\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)",
"\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)",
"\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)",
"\tat java.base/java.lang.Thread.run(Thread.java:832)"]}

@lmossman
Copy link
Contributor

This is the request body that gets sent when trying to edit the connector settings:

{
  connectionConfiguration: {
    api_type: "REST"
    is_sandbox: false
    refresh_token: "<redacted>"
    start_date: "2021-11-16T00:00:00Z"
  }
  name: "Salesforce Test"
  sourceId: "315c854a-5220-445e-9de6-37cdf3bf1d64"
}

this doesn't include client_id and client_secret, which is consistent with the stacktrace posted above.

The 500 error from clicking Authenticate your Salesforce account is due to the refresh_token query param missing from the complete_oauth request that is made.

This appears to be an issue with the Oauth implementation for when a connector is updated. @ChristopheDuong @jamakase could you take a look?

@ChristopheDuong
Copy link
Contributor

ChristopheDuong commented Nov 17, 2021

With OAuth feature on cloud, client_id and client_secret are instance-wide parameters and are not supposed to be filled up by the user through UI (the backend would inject them into the connector config at runtime).

In the past, this was handled on a special web_backend/sources/create endpoint to bypass this when creating the source.
I am guessing, we should have done it when editing or checking connections for such sources too (we were missing those)?

However, in #7917 I recently removed the web_backend injection logic and made it so these global parameters are returned along with outputs from OAuth flow (masked)

Masked Instance-wide params (client_id/client_secret) are also returned in the completeOAuth API call along with OAuth output results (such as refresh_token), so it is not necessary to use the web_backend/sources/create anymore to do that when creating a source anymore.

So, I hope we shouldn't see this issue anymore for connector created after upgrading to that 0.32.1-alpha+ version (connector created before the upgrade probably would need to go through the oauth flow)

Unless there is some extra logic on the Front-end to handle this? @jamakase

@jamakase
Copy link
Contributor

I believe that will work as is as soon as they are returned from backend. Let's keep it open, I will verify that it works fine on new version.

@ChristopheDuong
Copy link
Contributor

ChristopheDuong commented Nov 17, 2021

Cloud is on 0.32.0 version now and is throwing an error when going through oauth:

POST /api/v1/source_oauths/get_consent_url HTTP/1.1
Host: cloud.airbyte.io
'{"workspaceId":"XXXXX","sourceDefinitionId":"b117307c-14b6-41aa-9422-947e34922962","redirectUrl":"https://cloud.airbyte.io/auth_flow","inputParams":{}}'

Which returns:

{"message":"Invalid json input. Unrecognized field \"inputParams\" (class io.airbyte.api.model.SourceOauthConsentRequest), not marked as ignorable (3 known properties: \"workspaceId\", \"sourceDefinitionId\", \"redirectUrl\"])\n at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 181] (through reference chain: 
io.airbyte.api.model.SourceOauthConsentRequest[\"inputParams\"]) Unrecognized field \"inputParams\" (class io.airbyte.api.model.SourceOauthConsentRequest), not marked as ignorable"

I am guessing it's due to #7325 not being fully reverted in the front-end somehow?

cc @avida

@avida
Copy link
Contributor

avida commented Nov 17, 2021

@ChristopheDuong It was reverted by #7968 Also there were following commit on updating package-lock.json file.

@ChristopheDuong
Copy link
Contributor

ChristopheDuong commented Nov 17, 2021

It was reverted by #7968 Also there were following commit on updating package-lock.json file.

It seems like we need to deploy 0.32.1-alpha then

@lmossman
Copy link
Contributor

lmossman commented Nov 17, 2021

@ChristopheDuong @jamakase @avida It looks like we have deployed cloud to 0.32.1-alpha; I just tried creating a Salesforce source connector and when I clicked Authenticate your Salesforce account, my browser received a 500 error in the complete_oauth call:

{"message":"Internal Server Error: java.io.IOException: Missing 'refresh_token' in query params from https://login.salesforce.com/services/oauth2/token","exceptionClassName":"java.lang.RuntimeException",...

and if I then try to click Set up Source, the create call fails with the error

{"message":"The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. \nErrors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required

and I get redirected to a page that just says Unknown error occurred

So it seems like oauth might be broken even worse now. Can someone take a look?

@ChristopheDuong
Copy link
Contributor

@ChristopheDuong @jamakase @avida It looks like we have deployed cloud to 0.32.1-alpha; I just tried creating a Salesforce source connector and when I clicked Authenticate your Salesforce account, my browser received a 500 error in the complete_oauth
So it seems like oauth might be broken even worse now. Can someone take a look?

Yes, I've noticed that too, maybe it's related to #7650 ?

@jamakase
Copy link
Contributor

@ChristopheDuong i don't think that issue could be a root cause of this behavior. refresh_token will be saved after first successful request.

Working on it.

@lmossman
Copy link
Contributor

Maybe related, but in that issue it looks like the first complete_oauth call succeeds and the second one fails, but in my case both calls failed with the 500

@ChristopheDuong
Copy link
Contributor

ChristopheDuong commented Nov 17, 2021

This is what i get on cloud atm:

First complete_oauth call succeeds and provides refresh_token, while the second one fails:

Screenshot 2021-11-17 at 19 50 36

Then when I save the connection I get:

Screenshot 2021-11-17 at 19 53 41

@lmossman
Copy link
Contributor

Interesting, both of my complete_oauth calls fail with the same {"message":"Internal Server Error: java.io.IOException: Missing 'refresh_token' in query params from https://login.salesforce.com/services/oauth2/token","exceptionClassName":"java.lang.RuntimeException",... error
Screen Shot 2021-11-17 at 10 58 37 AM

@ChristopheDuong
Copy link
Contributor

ChristopheDuong commented Nov 17, 2021

Interesting, both of my complete_oauth calls fail with the same `{"message":"Internal Server Error: java.io.IOException: Missing 'refresh_token' in query params

That's because you have a third one that succeeded!

@jamakase
Copy link
Contributor

@ChristopheDuong problem is in this PR #7917.

We can't just remove web_backend. We do not send client_secret and client_id as based on initial requirements we shouldn't send them. Also keep in mind that we can't just send to the backend everything that is returned from complete_oauth call.

Based on the requirements everything within oauthFlowInitParameters section is totally ignored. Everything within oauthFlowOutputParameters is merged back. Nothing has changed since then, so I assume this issue is on either wrong spec or backend implementation

@jamakase
Copy link
Contributor

jamakase commented Nov 17, 2021

Just verified that number of complete_oauth calls doesn't affect it in any way.

Is it correct that from now, the output of complete_oauth_flow call will always include oauthFlowInitParameters ( maybe in masked way )? If yes, I could add a qfix to treat oauthFlowInitParameters as oauthFlowOutputParametersand just hide them but always send to the backend.

@ChristopheDuong
Copy link
Contributor

ChristopheDuong commented Nov 17, 2021

Is it correct that from now, the output of complete_oauth_flow call will always include oauthFlowInitParameters

yes, it will always include them masked from now on. It's also in line with how it works with the new protocol changes here #7915

it makes the backend code easier that way too

@lmossman
Copy link
Contributor

@jamakase did you test if this fixes the oauth issue? If so should we release OSS so that we can use this fix in cloud?

@jamakase
Copy link
Contributor

Yep, I verified it numerous times for Salesforce connector and the backend that is currently deployed in dev env.

@ChristopheDuong
Copy link
Contributor

ChristopheDuong commented Nov 18, 2021

I tested on cloud 0.32.2-alpha and it works for me too when:

  • creating new source connector (salesforce)
  • editing existing connectors using oauth

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
airbyte-cloud area/connectors Connector related issues area/platform issues related to the platform connectors/source/salesforce connectors/sources-api priority/high High priority type/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants