Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PXD-2916: implement uniqueKeys #259

Merged
merged 6 commits into from
Apr 18, 2019
Merged

PXD-2916: implement uniqueKeys #259

merged 6 commits into from
Apr 18, 2019

Conversation

fantix
Copy link
Contributor

@fantix fantix commented Feb 15, 2019

Auto migration is in uc-cdis/datamodelutils#6.

uc-cdis/gen3datamodel#12 replaced indexes on these columns with unique ones:

  • program.name
  • project.code

also added unique indexes for:

  • (project_id, submitter_id) of other nodes

Improvements

  • added unique indexes for uniqueKeys

Deployment changes

@fantix
Copy link
Contributor Author

fantix commented Feb 15, 2019

tests to fix 😞

@PlanXCyborg
Copy link

PlanXCyborg commented Feb 19, 2019

The style in this PR agrees with black. ✔️

This formatting comment was generated automatically by a script in uc-cdis/wool.

@fantix fantix changed the title [WIP] PXD-2468: implement uniqueKeys PXD-2468: implement uniqueKeys Feb 20, 2019
@fantix fantix force-pushed the feat/uniqueKeys branch 2 times, most recently from d592d99 to 17410b7 Compare February 20, 2019 21:29
@fantix
Copy link
Contributor Author

fantix commented Feb 20, 2019

PyLint bug fixed in pylint-dev/pylint#2760

@fantix fantix force-pushed the feat/uniqueKeys branch 2 times, most recently from 156a6c5 to f3ce970 Compare February 21, 2019 06:02
@fantix
Copy link
Contributor Author

fantix commented Feb 25, 2019

Closing in favor of #266

@fantix fantix closed this Feb 25, 2019
@fantix fantix deleted the feat/uniqueKeys branch February 25, 2019 04:02
@fantix fantix restored the feat/uniqueKeys branch April 17, 2019 16:01
@fantix fantix reopened this Apr 17, 2019
@fantix fantix force-pushed the feat/uniqueKeys branch 2 times, most recently from 8431d42 to 76ab247 Compare April 17, 2019 19:50
@PlanXCyborg
Copy link

Jenkins Build 1 : time taken 45 min
Check the https://jenkins.planx-pla.net/job/perf-sheepdog/job/PR-259/1/display/redirect

Test results:
Passed: 109, Failed: 0, Skipped: 0

Test Time (PR) Time (master) Diff
@ExportPerformanceTests @performance: Export all nodes of type: aliquot 0.302 0.274 0.03
@ExportPerformanceTests @performance: Export all nodes of type: case 0.354 0.222 0.13
@ExportPerformanceTests @performance: Export all nodes of type: experiment 0.304 0.261 0.04
@ExportPerformanceTests @performance: Export all nodes of type: project 0.083 0.042 0.04
@ExportPerformanceTests @performance: Export all nodes of type: read_group 0.509 0.428 0.08
@ExportPerformanceTests @performance: Export all nodes of type: sample 0.444 0.359 0.09
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: aliquot 0.174 0.146 0.03
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: case 0.238 0.214 0.02
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: experiment 0.404 0.174 0.23
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group_qc 0.072 0.056 0.02
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: sample 0.102 0.094 0.01
@SubmissionPerformanceTests @performance: Submission demographic 1000 30.078 26.739 3.34
@SubmissionPerformanceTests @performance: Submission publication 1000 20.839 19.284 1.56

@fantix fantix changed the title PXD-2468: implement uniqueKeys PXD-2916: implement uniqueKeys Apr 17, 2019
@PlanXCyborg
Copy link

Jenkins Build 2 : time taken 37 min
Check the https://jenkins.planx-pla.net/job/perf-sheepdog/job/PR-259/2/display/redirect

Test results:
Passed: 109, Failed: 0, Skipped: 0

Test Time (PR) Time (master) Diff
@ExportPerformanceTests @performance: Export all nodes of type: aliquot 0.296 0.274 0.02
@ExportPerformanceTests @performance: Export all nodes of type: experiment 0.397 0.261 0.14
@ExportPerformanceTests @performance: Export all nodes of type: program 0.186 0.068 0.12
@ExportPerformanceTests @performance: Export all nodes of type: project 0.046 0.042 0.00
@ExportPerformanceTests @performance: Export all nodes of type: read_group_qc 0.194 0.084 0.11
@ExportPerformanceTests @performance: Export all nodes of type: sample 0.492 0.359 0.13
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: aliquot 0.282 0.146 0.14
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: case 0.277 0.214 0.06
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: experiment 0.201 0.174 0.03
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group 0.165 0.064 0.10
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group_qc 0.131 0.056 0.07
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: sample 0.278 0.094 0.18
@QueryPerformanceTests @performance: Executing bottomUp query # 0 0.884 0.615 0.27
@QueryPerformanceTests @performance: Executing topDown query # 0 0.168 0.072 0.10

@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@uc-cdis uc-cdis deleted a comment from PlanXCyborg Apr 17, 2019
@fantix fantix requested a review from zflamig April 17, 2019 22:01
@PlanXCyborg
Copy link

Jenkins Build 3 : time taken 49 min
Check the https://jenkins.planx-pla.net/job/perf-sheepdog/job/PR-259/3/display/redirect

Test results:
Passed: 109, Failed: 0, Skipped: 0

Test Time (PR) Time (master) Diff
@ExportPerformanceTests @performance: Export all nodes of type: aliquot 0.296 0.274 0.02
@ExportPerformanceTests @performance: Export all nodes of type: experiment 0.397 0.261 0.14
@ExportPerformanceTests @performance: Export all nodes of type: program 0.186 0.068 0.12
@ExportPerformanceTests @performance: Export all nodes of type: project 0.046 0.042 0.00
@ExportPerformanceTests @performance: Export all nodes of type: read_group_qc 0.194 0.084 0.11
@ExportPerformanceTests @performance: Export all nodes of type: sample 0.492 0.359 0.13
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: aliquot 0.282 0.146 0.14
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: case 0.277 0.214 0.06
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: experiment 0.201 0.174 0.03
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group 0.165 0.064 0.10
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group_qc 0.131 0.056 0.07
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: sample 0.278 0.094 0.18
@QueryPerformanceTests @performance: Executing bottomUp query # 0 0.884 0.615 0.27
@QueryPerformanceTests @performance: Executing topDown query # 0 0.168 0.072 0.10

@PlanXCyborg
Copy link

Jenkins Build 5 : time taken 1 hr 13 min
Check the https://jenkins.planx-pla.net/job/perf-sheepdog/job/PR-259/5/display/redirect

Test results:
Passed: 109, Failed: 0, Skipped: 0

Test Time (PR) Time (master) Diff
@ExportPerformanceTests @performance: Export all nodes of type: experiment 0.276 0.261 0.01
@ExportPerformanceTests @performance: Export all nodes of type: project 0.056 0.042 0.01
@ExportPerformanceTests @performance: Export all nodes of type: read_group 0.635 0.428 0.21
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: case 0.23 0.214 0.02
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: experiment 0.226 0.174 0.05
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group 0.109 0.064 0.04
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group_qc 0.063 0.056 0.01
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: sample 0.11 0.094 0.02
@SubmissionPerformanceTests @performance: Submission aligned_reads_index 100 17.817 16.87 0.95
@SubmissionPerformanceTests @performance: Submission case 100 2.395 2.235 0.16
@SubmissionPerformanceTests @performance: Submission demographic 100 2.605 2.254 0.35
@SubmissionPerformanceTests @performance: Submission experimental_metadata 100 18.217 16.176 2.04
@SubmissionPerformanceTests @performance: Submission exposure 10 0.388 0.367 0.02
@SubmissionPerformanceTests @performance: Submission exposure 100 2.279 2.169 0.11
@SubmissionPerformanceTests @performance: Submission family_history 100 2.638 2.018 0.62
@SubmissionPerformanceTests @performance: Submission keyword 100 2.167 1.94 0.23
@SubmissionPerformanceTests @performance: Submission publication 1000 22.626 19.284 3.34
@SubmissionPerformanceTests @performance: Submission read_group_qc 100 3.959 3.603 0.36
@SubmissionPerformanceTests @performance: Submission slide_image 10 1.913 1.796 0.12
@SubmissionPerformanceTests @performance: Submission slide_image 100 17.47 14.74 2.73
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 100 17.301 16.181 1.12
@SubmissionPerformanceTests @performance: Submission submitted_methylation 100 20.026 17.149 2.88
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 100 17.699 15.312 2.39
@SubmissionPerformanceTests @performance: Submission submitted_unaligned_reads 100 17.862 16.488 1.37

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants