Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove signpost #135

Merged
merged 5 commits into from
Jun 19, 2019
Merged

Remove signpost #135

merged 5 commits into from
Jun 19, 2019

Conversation

vpsx
Copy link
Contributor

@vpsx vpsx commented Jun 17, 2019 •
edited
Loading

Remove signpost from peregrine dependencies and code

Improvements

  • looks for INDEX_CLIENT instead of SIGNPOST in app.config and gets INDEX_CLIENT_HOST env var instead of SIGNPOST_HOST--you must therefore use the latest cloud automation or manually recreate your peregrine secrets
  • stops importing and using signpost in testing, and stops calling index client signpost where index client is already being used instead of signpost

Dependency updates

  • point gen3datamodel to temporary branch until sqlalchemy and psqlgraph get updated
  • bump indexclient to 1.5.7
  • bump flaks-postgres-session to 0.1.4
  • remove signpost from dev-requirements

Deployment changes

  • look for INDEX_CLIENT instead of SIGNPOST in app.config
  • get INDEX_CLIENT_HOST env var instead of SIGNPOST_HOST

@PlanXCyborg
Copy link

Jenkins Build 2 : time taken 1 hr 29 min
Check the https://jenkins.planx-pla.net/job/Performance%20tests%20for%20uc-cdis/job/peregrine/job/PR-135/2/display/redirect

Test results:
Passed: 109, Failed: 0, Skipped: 0

Test Time (PR) Time (master) Diff
@ExportPerformanceTests @performance: Export all nodes of type: aliquot 0.279 0.222 0.06
@ExportPerformanceTests @performance: Export all nodes of type: case 0.334 0.251 0.08
@ExportPerformanceTests @performance: Export all nodes of type: experiment 0.319 0.421 -0.10
@ExportPerformanceTests @performance: Export all nodes of type: program 0.058 0.089 -0.03
@ExportPerformanceTests @performance: Export all nodes of type: project 0.048 0.08 -0.03
@ExportPerformanceTests @performance: Export all nodes of type: read_group 0.38 0.453 -0.07
@ExportPerformanceTests @performance: Export all nodes of type: sample 0.376 0.336 0.04
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: case 0.278 0.254 0.02
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: experiment 0.372 0.281 0.09
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group 0.175 0.06 0.11
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group_qc 0.118 0.079 0.04
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: sample 0.144 0.183 -0.04
@QueryPerformanceTests @performance: Executing bottomUp query # 0 0.9 0.535 0.36
@QueryPerformanceTests @performance: Executing bottomUp query # 1 0.701 0.498 0.20
@QueryPerformanceTests @performance: Executing bottomUp query # 2 0.606 0.508 0.10
@QueryPerformanceTests @performance: Executing bottomUp query # 3 0.606 0.403 0.20
@QueryPerformanceTests @performance: Executing bottomUp query # 4 0.581 0.3 0.28
@QueryPerformanceTests @performance: Executing bottomUp query # 5 0.398 0.235 0.16
@QueryPerformanceTests @performance: Executing bottomUp query # 6 0.201 0.144 0.06
@QueryPerformanceTests @performance: Executing bottomUp query # 7 0.157 0.061 0.10
@QueryPerformanceTests @performance: Executing topDown query # 0 0.1 0.05 0.05
@QueryPerformanceTests @performance: Executing topDown query # 1 0.087 0.067 0.02
@QueryPerformanceTests @performance: Executing topDown query # 2 0.191 0.096 0.09
@QueryPerformanceTests @performance: Executing topDown query # 3 0.211 0.149 0.06
@QueryPerformanceTests @performance: Executing topDown query # 4 0.394 0.238 0.16
@QueryPerformanceTests @performance: Executing topDown query # 5 0.388 0.308 0.08
@QueryPerformanceTests @performance: Executing topDown query # 6 0.687 0.478 0.21
@QueryPerformanceTests @performance: Executing topDown query # 7 0.699 0.613 0.09
@SubmissionPerformanceTests @performance: Submission acknowledgement 100 2.383 1.988 0.39
@SubmissionPerformanceTests @performance: Submission acknowledgement 1000 19.318 0.07 19.25
@SubmissionPerformanceTests @performance: Submission aligned_reads_index 10 1.782 2.267 -0.49
@SubmissionPerformanceTests @performance: Submission aligned_reads_index 100 16.811 19.605 -2.79
@SubmissionPerformanceTests @performance: Submission aliquot 10 0.365 0.341 0.02
@SubmissionPerformanceTests @performance: Submission case 10 0.444 0.344 0.10
@SubmissionPerformanceTests @performance: Submission case 100 2.015 2.282 -0.27
@SubmissionPerformanceTests @performance: Submission clinical_test 10 0.336 0.359 -0.02
@SubmissionPerformanceTests @performance: Submission clinical_test 100 2.496 2.328 0.17
@SubmissionPerformanceTests @performance: Submission core_metadata_collection 10 0.484 0.798 -0.31
@SubmissionPerformanceTests @performance: Submission core_metadata_collection 100 2.223 2.116 0.11
@SubmissionPerformanceTests @performance: Submission demographic 10 0.342 0.405 -0.06
@SubmissionPerformanceTests @performance: Submission demographic 1000 23.212 0.145 23.07
@SubmissionPerformanceTests @performance: Submission diagnosis 10 0.374 0.342 0.03
@SubmissionPerformanceTests @performance: Submission diagnosis 100 2.961 2.479 0.48
@SubmissionPerformanceTests @performance: Submission experiment 10 0.479 0.389 0.09
@SubmissionPerformanceTests @performance: Submission experimental_metadata 10 1.98 2.166 -0.19
@SubmissionPerformanceTests @performance: Submission experimental_metadata 100 16.692 19.811 -3.12
@SubmissionPerformanceTests @performance: Submission exposure 10 0.452 0.296 0.16
@SubmissionPerformanceTests @performance: Submission exposure 100 2.423 2.045 0.38
@SubmissionPerformanceTests @performance: Submission exposure 1000 20.694 0.074 20.62
@SubmissionPerformanceTests @performance: Submission family_history 10 0.321 0.372 -0.05
@SubmissionPerformanceTests @performance: Submission family_history 100 2.364 2.131 0.23
@SubmissionPerformanceTests @performance: Submission keyword 100 2.298 1.95 0.35
@SubmissionPerformanceTests @performance: Submission keyword 1000 19.398 0.037 19.36
@SubmissionPerformanceTests @performance: Submission publication 10 0.383 0.321 0.06
@SubmissionPerformanceTests @performance: Submission publication 100 2.299 1.954 0.35
@SubmissionPerformanceTests @performance: Submission publication 1000 20.302 15.404 4.90
@SubmissionPerformanceTests @performance: Submission read_group 10 0.362 0.326 0.04
@SubmissionPerformanceTests @performance: Submission read_group 100 2.236 2.402 -0.17
@SubmissionPerformanceTests @performance: Submission read_group_qc 10 0.612 0.469 0.14
@SubmissionPerformanceTests @performance: Submission read_group_qc 100 4.563 3.526 1.04
@SubmissionPerformanceTests @performance: Submission sample 10 0.351 0.392 -0.04
@SubmissionPerformanceTests @performance: Submission sample 100 2.493 2.354 0.14
@SubmissionPerformanceTests @performance: Submission slide 10 0.377 0.433 -0.06
@SubmissionPerformanceTests @performance: Submission slide 100 2.412 2.199 0.21
@SubmissionPerformanceTests @performance: Submission slide_count 10 0.375 0.321 0.05
@SubmissionPerformanceTests @performance: Submission slide_count 100 2.073 2.215 -0.14
@SubmissionPerformanceTests @performance: Submission slide_image 10 1.817 2.162 -0.34
@SubmissionPerformanceTests @performance: Submission slide_image 100 16.892 18.957 -2.07
@SubmissionPerformanceTests @performance: Submission slide_image 1000 44.816 0.079 44.74
@SubmissionPerformanceTests @performance: Submission submitted_aligned_reads 10 2.381 2.17 0.21
@SubmissionPerformanceTests @performance: Submission submitted_aligned_reads 100 17.242 22.228 -4.99
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 10 1.707 2.084 -0.38
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 100 15.9 18.403 -2.50
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 1000 45.216 0.131 45.08
@SubmissionPerformanceTests @performance: Submission submitted_methylation 10 1.699 2.438 -0.74
@SubmissionPerformanceTests @performance: Submission submitted_methylation 100 16.053 21.872 -5.82
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 10 1.753 2.188 -0.43
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 100 16.36 19.055 -2.69
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 1000 45.016 0.049 44.97
@SubmissionPerformanceTests @performance: Submission submitted_unaligned_reads 10 1.989 2.118 -0.13
@SubmissionPerformanceTests @performance: Submission submitted_unaligned_reads 100 17.497 20.352 -2.85
@SubmissionPerformanceTests @performance: Submission treatment 100 2.302 2.063 0.24
@SubmissionPerformanceTests @performance: Submission treatment 1000 21.315 20.171 1.14

@vpsx vpsx force-pushed the fix/rm-signpost branch from 9081ab1 to bf06b9c Compare June 18, 2019 20:22
@PlanXCyborg
Copy link

Jenkins Build 3 : time taken 1 hr 20 min
Check the https://jenkins.planx-pla.net/job/Performance%20tests%20for%20uc-cdis/job/peregrine/job/PR-135/3/display/redirect

Test results:
Passed: 109, Failed: 0, Skipped: 0

Test Time (PR) Time (master) Diff
@ExportPerformanceTests @performance: Export all nodes of type: case 0.213 0.251 -0.04
@ExportPerformanceTests @performance: Export all nodes of type: experiment 0.272 0.421 -0.15
@ExportPerformanceTests @performance: Export all nodes of type: program 0.061 0.089 -0.03
@ExportPerformanceTests @performance: Export all nodes of type: project 0.044 0.08 -0.04
@ExportPerformanceTests @performance: Export all nodes of type: read_group 0.4 0.453 -0.05
@ExportPerformanceTests @performance: Export all nodes of type: read_group_qc 0.271 0.1 0.17
@ExportPerformanceTests @performance: Export all nodes of type: sample 0.507 0.336 0.17
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: aliquot 0.172 0.132 0.04
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: case 0.196 0.254 -0.06
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: experiment 0.188 0.281 -0.09
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group 0.071 0.06 0.01
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: sample 0.093 0.183 -0.09
@QueryPerformanceTests @performance: Executing bottomUp query # 0 0.794 0.535 0.26
@QueryPerformanceTests @performance: Executing bottomUp query # 1 0.775 0.498 0.28
@QueryPerformanceTests @performance: Executing bottomUp query # 2 0.769 0.508 0.26
@QueryPerformanceTests @performance: Executing bottomUp query # 3 0.572 0.403 0.17
@QueryPerformanceTests @performance: Executing bottomUp query # 4 0.489 0.3 0.19
@QueryPerformanceTests @performance: Executing bottomUp query # 5 0.486 0.235 0.25
@QueryPerformanceTests @performance: Executing bottomUp query # 6 0.272 0.144 0.13
@QueryPerformanceTests @performance: Executing bottomUp query # 7 0.177 0.061 0.12
@QueryPerformanceTests @performance: Executing topDown query # 0 0.178 0.05 0.13
@QueryPerformanceTests @performance: Executing topDown query # 1 0.187 0.067 0.12
@QueryPerformanceTests @performance: Executing topDown query # 2 0.259 0.096 0.16
@QueryPerformanceTests @performance: Executing topDown query # 3 0.194 0.149 0.05
@QueryPerformanceTests @performance: Executing topDown query # 4 0.29 0.238 0.05
@QueryPerformanceTests @performance: Executing topDown query # 5 0.577 0.308 0.27
@QueryPerformanceTests @performance: Executing topDown query # 6 0.699 0.478 0.22
@QueryPerformanceTests @performance: Executing topDown query # 7 0.798 0.613 0.19
@SubmissionPerformanceTests @performance: Submission acknowledgement 1000 19.219 0.07 19.15
@SubmissionPerformanceTests @performance: Submission aligned_reads_index 10 1.766 2.267 -0.50
@SubmissionPerformanceTests @performance: Submission aligned_reads_index 100 16.398 19.605 -3.21
@SubmissionPerformanceTests @performance: Submission aliquot 100 1.918 2.109 -0.19
@SubmissionPerformanceTests @performance: Submission aliquot 1000 18.915 21.108 -2.19
@SubmissionPerformanceTests @performance: Submission case 10 0.375 0.344 0.03
@SubmissionPerformanceTests @performance: Submission clinical_test 10 0.385 0.359 0.03
@SubmissionPerformanceTests @performance: Submission core_metadata_collection 10 0.724 0.798 -0.07
@SubmissionPerformanceTests @performance: Submission demographic 10 0.345 0.405 -0.06
@SubmissionPerformanceTests @performance: Submission demographic 1000 21.776 0.145 21.63
@SubmissionPerformanceTests @performance: Submission diagnosis 10 0.479 0.342 0.14
@SubmissionPerformanceTests @performance: Submission experiment 10 0.454 0.389 0.06
@SubmissionPerformanceTests @performance: Submission experiment 100 2.531 2.155 0.38
@SubmissionPerformanceTests @performance: Submission experimental_metadata 10 1.848 2.166 -0.32
@SubmissionPerformanceTests @performance: Submission experimental_metadata 100 16.495 19.811 -3.32
@SubmissionPerformanceTests @performance: Submission exposure 10 0.365 0.296 0.07
@SubmissionPerformanceTests @performance: Submission exposure 1000 19.827 0.074 19.75
@SubmissionPerformanceTests @performance: Submission family_history 10 0.336 0.372 -0.04
@SubmissionPerformanceTests @performance: Submission keyword 1000 18.525 0.037 18.49
@SubmissionPerformanceTests @performance: Submission publication 10 0.286 0.321 -0.03
@SubmissionPerformanceTests @performance: Submission publication 1000 18.837 15.404 3.43
@SubmissionPerformanceTests @performance: Submission read_group_qc 1000 24.672 22.955 1.72
@SubmissionPerformanceTests @performance: Submission sample 10 0.363 0.392 -0.03
@SubmissionPerformanceTests @performance: Submission sample 100 2.173 2.354 -0.18
@SubmissionPerformanceTests @performance: Submission slide 10 0.365 0.433 -0.07
@SubmissionPerformanceTests @performance: Submission slide 1000 20.507 22.332 -1.83
@SubmissionPerformanceTests @performance: Submission slide_count 100 1.968 2.215 -0.25
@SubmissionPerformanceTests @performance: Submission slide_image 10 1.961 2.162 -0.20
@SubmissionPerformanceTests @performance: Submission slide_image 100 16.586 18.957 -2.37
@SubmissionPerformanceTests @performance: Submission slide_image 1000 44.966 0.079 44.89
@SubmissionPerformanceTests @performance: Submission submitted_aligned_reads 10 1.831 2.17 -0.34
@SubmissionPerformanceTests @performance: Submission submitted_aligned_reads 100 16.822 22.228 -5.41
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 10 1.644 2.084 -0.44
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 100 16.104 18.403 -2.30
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 1000 45.107 0.131 44.98
@SubmissionPerformanceTests @performance: Submission submitted_methylation 10 2.06 2.438 -0.38
@SubmissionPerformanceTests @performance: Submission submitted_methylation 100 16.612 21.872 -5.26
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 10 1.751 2.188 -0.44
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 100 16.475 19.055 -2.58
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 1000 45.008 0.049 44.96
@SubmissionPerformanceTests @performance: Submission submitted_unaligned_reads 10 1.757 2.118 -0.36
@SubmissionPerformanceTests @performance: Submission submitted_unaligned_reads 100 16.387 20.352 -3.97
@SubmissionPerformanceTests @performance: Submission treatment 10 0.306 0.323 -0.02
@SubmissionPerformanceTests @performance: Submission treatment 100 2.169 2.063 0.11

@vpsx vpsx merged commit 32142ed into master Jun 19, 2019
@vpsx vpsx deleted the fix/rm-signpost branch June 19, 2019 15:24
This was referenced Feb 10, 2020
Merged
Merged
@PlanXCyborg PlanXCyborg mentioned this pull request Mar 17, 2020
Merged
This was referenced Mar 31, 2020
Merged
Closed
Merged
@PlanXCyborg PlanXCyborg mentioned this pull request Oct 21, 2022
Closed
This was referenced Dec 18, 2023
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Avantol13 Avantol13 Avantol13 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vpsx @PlanXCyborg @Avantol13