-
Notifications
You must be signed in to change notification settings - Fork 4.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
🎉 Source Okta: add GroupMembers stream (#14380)
* add Group_Members stream to okta source - Group_Members return a list of users, the same schema of Users stream. - Create a shared schema users, and both group_members and users sechema use it as a reference. - Add Group_Members stream to source connector * add tests and fix logs schema - fix the test error: None is not one of enums though the enum type includes both string and null, it comes from json schema validator https://github.com/python-jsonschema/jsonschema/blob/ddb87afad8f5d5c40600b5ede0ab96e4d4bdf7d3/jsonschema/_validators.py#L279-L285 - change grouop_members to use id as the cursor field since `filter` is not supported in the query string - fix the abnormal state test on logs stream, when since is abnormally large, until has to defined, an equal or a larger value - remove logs stream from full sync test, because 2 full sync always has a gap -- at least a new log about users or groups api. * last polish before submit the PR - bump docker version - update changelog - add the right abnormal value for logs stream - correct the sample catalog * address comments:: - improve comments for until parameter under the logs stream - add use_cache on groupMembers * add use_cache to Group_Members * change configured_catalog to test * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
- Loading branch information
3 people
authored
Jul 12, 2022
1 parent
43f905b
commit 92b2d9c
Showing
15 changed files
with
336 additions
and
245 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 3 additions & 1 deletion
4
airbyte-integrations/connectors/source-okta/integration_tests/abnormal_state.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,6 @@ | ||
| { | ||
| "users": { "lastUpdated": "3021-09-08T07:04:28.000Z" }, | ||
| "groups": { "lastUpdated": "3021-09-08T07:04:28.000Z" } | ||
| "groups": { "lastUpdated": "3021-09-08T07:04:28.000Z" }, | ||
| "group_members": { "id": "00uzzzzzzzzzzzzzzzzz" }, | ||
| "logs": { "published": "3021-09-08T07:04:28.000Z" } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 4 additions & 0 deletions
4
airbyte-integrations/connectors/source-okta/sample_files/config.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "base_url": "https://myorg.okta.com", | ||
| "token": "xyz123foo325a.fbar" | ||
| } |
3 changes: 3 additions & 0 deletions
3
airbyte-integrations/connectors/source-okta/source_okta/schemas/group_members.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| { | ||
| "$ref": "users.json" | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
230 changes: 230 additions & 0 deletions
230
airbyte-integrations/connectors/source-okta/source_okta/schemas/shared/users.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,230 @@ | ||
| { | ||
| "properties": { | ||
| "_links": { | ||
| "additionalProperties": { | ||
| "type": ["object", "null"] | ||
| }, | ||
| "type": ["object", "null"] | ||
| }, | ||
| "activated": { | ||
| "format": "date-time", | ||
| "type": ["string", "null"] | ||
| }, | ||
| "created": { | ||
| "format": "date-time", | ||
| "type": "string" | ||
| }, | ||
| "credentials": { | ||
| "properties": { | ||
| "password": { | ||
| "properties": { | ||
| "hash": { | ||
| "properties": { | ||
| "algorithm": { | ||
| "enum": ["BCRYPT", "SHA-512", "SHA-256", "SHA-1", "MD5"], | ||
| "type": ["string", "null"] | ||
| }, | ||
| "salt": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "saltOrder": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "value": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "workFactor": { | ||
| "type": ["integer", "null"] | ||
| } | ||
| }, | ||
| "type": ["object", "null"] | ||
| }, | ||
| "hook": { | ||
| "properties": { | ||
| "type": { | ||
| "type": ["string", "null"] | ||
| } | ||
| }, | ||
| "type": ["object", "null"] | ||
| }, | ||
| "value": { | ||
| "format": "password", | ||
| "type": ["string", "null"] | ||
| } | ||
| }, | ||
| "type": ["object", "null"] | ||
| }, | ||
| "provider": { | ||
| "properties": { | ||
| "name": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "type": { | ||
| "enum": [ | ||
| "ACTIVE_DIRECTORY", | ||
| "FEDERATION", | ||
| "LDAP", | ||
| "OKTA", | ||
| "SOCIAL", | ||
| "IMPORT" | ||
| ], | ||
| "type": ["string", "null"] | ||
| } | ||
| }, | ||
| "type": ["object", "null"] | ||
| }, | ||
| "recovery_question": { | ||
| "properties": { | ||
| "answer": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "question": { | ||
| "type": ["string", "null"] | ||
| } | ||
| }, | ||
| "type": ["object", "null"] | ||
| } | ||
| }, | ||
| "type": "object" | ||
| }, | ||
| "id": { | ||
| "type": "string" | ||
| }, | ||
| "lastLogin": { | ||
| "format": "date-time", | ||
| "type": ["string", "null"] | ||
| }, | ||
| "lastUpdated": { | ||
| "format": "date-time", | ||
| "type": "string" | ||
| }, | ||
| "passwordChanged": { | ||
| "format": "date-time", | ||
| "type": ["string", "null"] | ||
| }, | ||
| "profile": { | ||
| "properties": { | ||
| "city": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "costCenter": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "countryCode": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "department": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "displayName": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "division": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "email": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "employeeNumber": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "firstName": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "honorificPrefix": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "honorificSuffix": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "lastName": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "locale": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "login": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "manager": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "managerId": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "middleName": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "mobilePhone": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "nickName": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "organization": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "postalAddress": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "preferredLanguage": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "primaryPhone": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "profileUrl": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "secondEmail": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "state": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "streetAddress": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "timezone": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "title": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "userType": { | ||
| "type": ["string", "null"] | ||
| }, | ||
| "zipCode": { | ||
| "type": ["string", "null"] | ||
| } | ||
| }, | ||
| "type": "object" | ||
| }, | ||
| "status": { | ||
| "enum": [ | ||
| "ACTIVE", | ||
| "DEPROVISIONED", | ||
| "LOCKED_OUT", | ||
| "PASSWORD_EXPIRED", | ||
| "PROVISIONED", | ||
| "RECOVERY", | ||
| "STAGED", | ||
| "SUSPENDED" | ||
| ], | ||
| "type": "string" | ||
| }, | ||
| "statusChanged": { | ||
| "format": "date-time", | ||
| "type": ["string", "null"] | ||
| }, | ||
| "type": { | ||
| "properties": { | ||
| "id": { | ||
| "type": ["string", "null"] | ||
| } | ||
| }, | ||
| "type": "object" | ||
| } | ||
| }, | ||
| "type": "object" | ||
| } |
Oops, something went wrong.