Skip to content

Commit 7a5b4a8

Browse files
alanbchristieAlan Christiekaliif
authored
API authentication and security changes (#635)
* style: Reorder functions in the module * refactor: Moved non-view functions to viewer.utils * fix: Removed unused save_pdb_zip and minor refactoring * feat: Removed circular import * feat: Fix get_open_targets (also get_open_proposals now not _private_) * feat: Fix get_open_proposals reference * refactor: ISpyB -> ISPyB * docs: Updated for use of mixins * feat: More API security migrations * feat: More security migrations * feat: Security migrations for hotspots and hypothesis * feat: More security fixes * feat: More security changes * feat: More security changes (and get_params -> get_img_from_smiles with default w/h) * fix: Attempt to fix calls to /xcdb/fragspect/ 500 errors * feat: Another attempot to fix ISPyB * feat: Use of new user_is_member_of_target() * feat: Experiment with validator * feat: Better serializer log * feat: Even more work on the serializer * feat: Minor error message tweak * feat: Add support for TEST_RESTRICTED_TAS_LIST (#614) Co-authored-by: Alan Christie <alan.christie@matildapeak.com> * target permission validation mixin pattern implemented for Pose * feat: Fix restricted logic * most endpoints secured with VaildateTargetMixin * fix: Removed unused endpoint * fix: secure SessionActions serializer * fix: Removed pset_download * fix: Design set upload now unsupported (404) * fix: Snapshots now open again * fix: CompoundIdentifierTypeView & TagCategoryView now read-only views * fix: Discourse POST now requires login * feat: User now needs to be a member of CSET target to download it * fix: secured TaskStatus endpoint * feat: Removal of unsed xcdb app * feat: Add log to use of dicttocsv * feat: More secure DictToCsv * feat: More consistent use of _ISPYB_SAFE_QUERY_SET * feat: Stricter UploadCSet class inheritance * feat: Fix isort issues * feat: Fix ListAPIView * feat: Remove references to xcdb * fix: secure UploadTaskView and ValidateTaskView TODO: secure UpdateTaskView (if used) * Align 1247 with latest staging code (#616) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> * Align 1247 from staging (#619) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> * Align 1247 with latest staging (#620) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: remove version field from ComputedMolecule Field was added in error, explicit version tracking is not necessary * fix: forgot to stage migration file.. * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. * fix: add warning to logs about overwriting ComputedMolecule metadata --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> * feat: Rstore CSetUpload post() * feat: Revert UploadCSet inheritance * fix: Another attempt to fix UploadCSet * fix: Another attmept to fix the view * fix: Anotehr attempt to get UploadCSet * feat: Fix UploadCSet view * feat: Fix JobRequest GET (restrict to members of the project) * feat: Enhanced logging for membership check failures * docs: Improve docs relating to security * docs: Minor typo * fix: Remove TEST_RESTRICTED_TAS_LIST feature * Align 1247 with staging (#621) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: remove version field from ComputedMolecule Field was added in error, explicit version tracking is not necessary * fix: forgot to stage migration file.. * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. * fix: add warning to logs about overwriting ComputedMolecule metadata --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> * Align 1247 with latest staging (#623) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: remove version field from ComputedMolecule Field was added in error, explicit version tracking is not necessary * fix: forgot to stage migration file.. * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. * fix: add warning to logs about overwriting ComputedMolecule metadata * fix: add additional ccp4 files to download (issue 1448) --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> * fix: Attempt to fix PoseView (now Pose) * fix: Attempt to debug Pose failure * fix: Another patch to Pose * fix: Fix log typo * fix: Attempt to fix permission on create * fix: Fix for ValidateTargetMixin? * fix: Better Mixin (renamed and copes with shortest filter string) * fix: Fix some project mixin views (includes some renaming) * refactor: View name consistency * fix: Fix for targetdownload mixin (and extra log) * fix: Better file handling * fix: Now serches ExpUpload for first matching record * fix: Better experiment download (use of only ExpUpload record) * fix: ExpDownload now inspects Project * fix: More naming consistency changes * fix: Attempt to fix 'ManyRelatedManager' is not iterable * fix: Use of correct download path * Align 1247 with staging (#624) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: remove version field from ComputedMolecule Field was added in error, explicit version tracking is not necessary * fix: forgot to stage migration file.. * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. * fix: add warning to logs about overwriting ComputedMolecule metadata * fix: add additional ccp4 files to download (issue 1448) --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> * Align 1247 with staging (#627) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: remove version field from ComputedMolecule Field was added in error, explicit version tracking is not necessary * fix: forgot to stage migration file.. * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. * fix: add warning to logs about overwriting ComputedMolecule metadata * fix: add additional ccp4 files to download (issue 1448) --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> * docs: Tweak messages * fix: Better file handling * docs: Doc tweak * Merge compound fix to 1247 (#628) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: remove version field from ComputedMolecule Field was added in error, explicit version tracking is not necessary * fix: forgot to stage migration file.. * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. * fix: add warning to logs about overwriting ComputedMolecule metadata * fix: add additional ccp4 files to download (issue 1448) * fix: Branch for project reference fix * fix: Projects copied from Target (during RHS cset-upload) --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> Co-authored-by: Alan Christie <alan.christie@matildapeak.com> * refactor: restrict_to_membership now restrict_pubic_to_membership * fix: ValidateProjectMixin does not insist on public proposal membership for GET * fix: Apply conflict from staging * Align 1247 with staging (#631) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: remove version field from ComputedMolecule Field was added in error, explicit version tracking is not necessary * fix: forgot to stage migration file.. * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. * fix: add warning to logs about overwriting ComputedMolecule metadata * fix: add additional ccp4 files to download (issue 1448) * fix: updates to tag generation Changed how some of the tags are generated as per the comment here: m2ms/fragalysis-frontend#1482 (comment) * feat: added centroid_res field to CanonSite model Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that now it's being added to meta_aligner.yaml, so reading an additional file is not necessary. I hope... * feat: added new fields to metadata.csv Experiment code and centroid res * feat: added tag aliases to metadata.csv * Copies Target proposals to new (RHS) Compounds (#629) * fix: Branch for project reference fix * fix: Projects copied from Target (during RHS cset-upload) * fix: Add save before copying projects * fix: Remove unnecessary save() * ci: Attempt to fix docker-compose problem * ci: Fix staging and production builds (docker compose) --------- Co-authored-by: Alan Christie <alan.christie@matildapeak.com> --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> Co-authored-by: Alan Christie <alan.christie@matildapeak.com> * fix: Fix project_id type (aligns with staging) * Align 1247 with staging (#633) * fix: updated crystalform site tag generation scheme * fix: update tag generation scheme * fix: remove version field from ComputedMolecule Field was added in error, explicit version tracking is not necessary * fix: forgot to stage migration file.. * fix: fix querysets in target_loader.py Came up with pose generation - SiteObservation querysets were called over the entire table not by the target they should have been associated with. * fix: add warning to logs about overwriting ComputedMolecule metadata * fix: add additional ccp4 files to download (issue 1448) * fix: updates to tag generation Changed how some of the tags are generated as per the comment here: m2ms/fragalysis-frontend#1482 (comment) * feat: added centroid_res field to CanonSite model Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that now it's being added to meta_aligner.yaml, so reading an additional file is not necessary. I hope... * feat: added new fields to metadata.csv Experiment code and centroid res * feat: added tag aliases to metadata.csv * Copies Target proposals to new (RHS) Compounds (#629) * fix: Branch for project reference fix * fix: Projects copied from Target (during RHS cset-upload) * fix: Add save before copying projects * fix: Remove unnecessary save() * ci: Attempt to fix docker-compose problem * ci: Fix staging and production builds (docker compose) --------- Co-authored-by: Alan Christie <alan.christie@matildapeak.com> * fix: Fix typo accessing Target projects (#632) --------- Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com> Co-authored-by: Alan Christie <alan.christie@matildapeak.com> --------- Co-authored-by: Alan Christie <alan.christie@matildapeak.com> Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>
1 parent 97a56a5 commit 7a5b4a8

28 files changed

+1194
-991
lines changed

api/security.py

+51-19
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ def ping_configured_connector() -> bool:
185185
return conn is not None
186186

187187

188-
class ISpyBSafeQuerySet(viewsets.ReadOnlyModelViewSet):
188+
class ISPyBSafeQuerySet(viewsets.ReadOnlyModelViewSet):
189189
"""
190190
This ISpyBSafeQuerySet, which inherits from the DRF viewsets.ReadOnlyModelViewSet,
191191
is used for all views that need to yield (filter) view objects based on a
@@ -216,7 +216,7 @@ def get_queryset(self):
216216
q_filter = self._get_q_filter(proposal_list)
217217
return self.queryset.filter(q_filter).distinct()
218218

219-
def _get_open_proposals(self):
219+
def get_open_proposals(self):
220220
"""
221221
Returns the set of proposals anybody can access.
222222
These consist of any Projects that are marked "open_to_public"
@@ -226,6 +226,7 @@ def _get_open_proposals(self):
226226
Project.objects.filter(open_to_public=True).values_list("title", flat=True)
227227
)
228228
open_proposals.update(settings.PUBLIC_TAS_LIST)
229+
# End Temporary Test Code (1247)
229230
return open_proposals
230231

231232
def _get_proposals_for_user_from_django(self, user):
@@ -341,36 +342,67 @@ def _get_proposals_from_connector(self, user, conn):
341342
)
342343
CachedContent.set_content(user.username, prop_id_set)
343344

344-
def user_is_member_of_any_given_proposals(self, user, proposals):
345+
def user_is_member_of_target(
346+
self, user, target, restrict_public_to_membership=True
347+
):
348+
"""
349+
Returns true if the user has access to any proposal the target belongs to.
350+
"""
351+
target_proposals = [p.title for p in target.project_id.all()]
352+
user_proposals = self.get_proposals_for_user(
353+
user, restrict_public_to_membership=restrict_public_to_membership
354+
)
355+
is_member = any(proposal in user_proposals for proposal in target_proposals)
356+
if not is_member:
357+
logger.warning(
358+
"Failed membership check user='%s' target='%s' target_proposals=%s",
359+
user.username,
360+
target.title,
361+
target_proposals,
362+
)
363+
return is_member
364+
365+
def user_is_member_of_any_given_proposals(
366+
self, user, proposals, restrict_public_to_membership=True
367+
):
345368
"""
346369
Returns true if the user has access to any proposal in the given
347-
proposals list.Only one needs to match for permission to be granted.
348-
We 'restrict_to_membership' to only consider proposals the user
370+
proposals list. Only one needs to match for permission to be granted.
371+
We 'restrict_public_to_membership' to only consider proposals the user
349372
has explicit membership.
350373
"""
351-
user_proposals = self.get_proposals_for_user(user, restrict_to_membership=True)
352-
return any(proposal in user_proposals for proposal in proposals)
374+
user_proposals = self.get_proposals_for_user(
375+
user, restrict_public_to_membership=restrict_public_to_membership
376+
)
377+
is_member = any(proposal in user_proposals for proposal in proposals)
378+
if not is_member:
379+
logger.warning(
380+
"Failed membership check user='%s' proposals=%s",
381+
user.username,
382+
proposals,
383+
)
384+
return is_member
353385

354-
def get_proposals_for_user(self, user, restrict_to_membership=False):
386+
def get_proposals_for_user(self, user, restrict_public_to_membership=False):
355387
"""
356388
Returns a list of proposals that the user has access to.
357389
358-
If 'restrict_to_membership' is set only those proposals/visits where the user
390+
If 'restrict_public_to_membership' is set only those proposals/visits where the user
359391
is a member of the visit will be returned. Otherwise the 'public'
360-
proposals/visits will also be returned. Typically 'restrict_to_membership' is
392+
proposals/visits will also be returned. Typically 'restrict_public_to_membership' is
361393
used for uploads/changes - this allows us to implement logic that (say)
362394
only permits explicit members of public proposals to add/load data for that
363-
project (restrict_to_membership=True), but everyone can 'see' public data
364-
(restrict_to_membership=False).
395+
project (restrict_public_to_membership=True), but everyone can 'see' public data
396+
(restrict_public_to_membership=False).
365397
"""
366398
assert user
367399

368400
proposals = set()
369401
ispyb_user = settings.ISPYB_USER
370402
logger.debug(
371-
"ispyb_user=%s restrict_to_membership=%s (DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP=%s)",
403+
"ispyb_user=%s restrict_public_to_membership=%s (DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP=%s)",
372404
ispyb_user,
373-
restrict_to_membership,
405+
restrict_public_to_membership,
374406
settings.DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP,
375407
)
376408
if ispyb_user:
@@ -384,10 +416,10 @@ def get_proposals_for_user(self, user, restrict_to_membership=False):
384416
# We have all the proposals where the user has authority.
385417
# Add open/public proposals?
386418
if (
387-
not restrict_to_membership
419+
not restrict_public_to_membership
388420
or settings.DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP
389421
):
390-
proposals.update(self._get_open_proposals())
422+
proposals.update(self.get_open_proposals())
391423

392424
# Return the set() as a list()
393425
return list(proposals)
@@ -411,9 +443,9 @@ def _get_q_filter(self, proposal_list):
411443
return Q(title__in=proposal_list) | Q(open_to_public=True)
412444

413445

414-
class ISpyBSafeStaticFiles:
446+
class ISPyBSafeStaticFiles:
415447
def get_queryset(self):
416-
query = ISpyBSafeQuerySet()
448+
query = ISPyBSafeQuerySet()
417449
query.request = self.request
418450
query.filter_permissions = self.permission_string
419451
query.queryset = self.model.objects.filter()
@@ -459,7 +491,7 @@ def get_response(self):
459491
raise Http404 from exc
460492

461493

462-
class ISpyBSafeStaticFiles2(ISpyBSafeStaticFiles):
494+
class ISPyBSafeStaticFiles2(ISPyBSafeStaticFiles):
463495
def get_response(self):
464496
logger.info("+ get_response called with: %s", self.input_string)
465497
# it wasn't working because found two objects with test file name

api/urls.py

+14-17
Original file line numberDiff line numberDiff line change
@@ -3,20 +3,18 @@
33
from rest_framework.authtoken import views as drf_views
44
from rest_framework.routers import DefaultRouter
55

6-
# from xcdb import views as xchem_views
76
from hotspots import views as hostpot_views
87
from hypothesis import views as hypo_views
98
from scoring import views as score_views
109
from viewer import views as viewer_views
11-
from xcdb import views as xcdb_views
1210

1311
router = DefaultRouter()
1412
# Register the basic data
1513
router.register("compounds", viewer_views.CompoundView, "compounds")
1614
router.register("targets", viewer_views.TargetView, "targets")
1715
router.register("projects", viewer_views.ProjectView)
18-
router.register("session-projects", viewer_views.SessionProjectsView)
19-
router.register("snapshots", viewer_views.SnapshotsView)
16+
router.register("session-projects", viewer_views.SessionProjectView)
17+
router.register("snapshots", viewer_views.SnapshotView)
2018
router.register("action-type", viewer_views.ActionTypeView)
2119
router.register("session-actions", viewer_views.SessionActionsView)
2220
router.register("snapshot-actions", viewer_views.SnapshotActionsView)
@@ -26,7 +24,7 @@
2624
# Compounds sets
2725
router.register("compound-sets", viewer_views.ComputedSetView)
2826
router.register("compound-molecules", viewer_views.ComputedMoleculesView)
29-
router.register("numerical-scores", viewer_views.NumericalScoresView)
27+
router.register("numerical-scores", viewer_views.NumericalScoreValuesView)
3028
router.register("text-scores", viewer_views.TextScoresView)
3129
router.register("compound-scores", viewer_views.CompoundScoresView, "compound-scores")
3230
router.register(
@@ -65,16 +63,13 @@
6563
# Get the information
6664
router.register("siteobservationannotation", score_views.SiteObservationAnnotationView)
6765

68-
# fragspect
69-
router.register("fragspect", xcdb_views.FragspectCrystalView)
70-
7166
# discourse posts
7267
router.register(
7368
"discourse_post", viewer_views.DiscoursePostView, basename='discourse_post'
7469
)
7570

7671
# Take a dictionary and return a csv
77-
router.register("dicttocsv", viewer_views.DictToCsv, basename='dicttocsv')
72+
router.register("dicttocsv", viewer_views.DictToCSVView, basename='dicttocsv')
7873

7974
# tags
8075
router.register("tag_category", viewer_views.TagCategoryView, basename='tag_category')
@@ -92,36 +87,38 @@
9287
# Download a zip file of the requested contents
9388
router.register(
9489
"download_structures",
95-
viewer_views.DownloadStructures,
90+
viewer_views.DownloadStructuresView,
9691
basename='download_structures',
9792
)
9893

9994
# Experiments and Experiment (XChemAlign) upload support
10095
router.register(
10196
"upload_target_experiments",
102-
viewer_views.UploadTargetExperiments,
97+
viewer_views.UploadExperimentUploadView,
10398
basename='upload_target_experiments',
10499
)
105100
router.register(
106101
"download_target_experiments",
107-
viewer_views.DownloadTargetExperiments,
102+
viewer_views.DownloadExperimentUploadView,
108103
basename='download_target_experiments',
109104
)
110105

111106

112107
router.register(
113108
"target_experiment_uploads",
114-
viewer_views.TargetExperimentUploads,
109+
viewer_views.ExperimentUploadView,
115110
basename='target_experiment_uploads',
116111
)
117112
router.register(
118-
"site_observations", viewer_views.SiteObservations, basename='site_observations'
113+
"site_observations", viewer_views.SiteObservationView, basename='site_observations'
114+
)
115+
router.register("canon_sites", viewer_views.CanonSiteView, basename='canon_sites')
116+
router.register(
117+
"canon_site_confs", viewer_views.CanonSiteConfView, basename='canon_site_confs'
119118
)
120-
router.register("canon_sites", viewer_views.CanonSites, basename='canon_sites')
121119
router.register(
122-
"canon_site_confs", viewer_views.CanonSiteConfs, basename='canon_site_confs'
120+
"xtalform_sites", viewer_views.XtalformSiteView, basename='xtalform_sites'
123121
)
124-
router.register("xtalform_sites", viewer_views.XtalformSites, basename='xtalform_sites')
125122
router.register("poses", viewer_views.PoseView, basename='poses')
126123

127124
# Squonk Jobs

api/utils.py

+4-8
Original file line numberDiff line numberDiff line change
@@ -305,20 +305,16 @@ def parse_xenons(input_smi):
305305
return bond_ids, bond_colours, e_mol.GetMol()
306306

307307

308-
def get_params(smiles, request):
308+
def get_img_from_smiles(smiles, request):
309309
# try:
310310
smiles = canon_input(smiles)
311311
# except:
312312
# smiles = ""
313-
height = None
314313
mol = None
315314
bond_id_list = []
316315
highlightBondColors = {}
317-
if "height" in request.GET:
318-
height = int(request.GET["height"])
319-
width = None
320-
if "width" in request.GET:
321-
width = int(request.GET["width"])
316+
height = int(request.GET.get("height", "128"))
317+
width = int(request.GET.get("width", "128"))
322318
if "atom_indices" in request.GET:
323319
mol = Chem.MolFromSmiles(smiles)
324320
bond_id_list, highlightBondColors, mol = parse_atom_ids(
@@ -360,7 +356,7 @@ def get_highlighted_diffs(request):
360356
def mol_view(request):
361357
if "smiles" in request.GET:
362358
smiles = request.GET["smiles"].rstrip(".svg")
363-
return get_params(smiles, request)
359+
return get_img_from_smiles(smiles, request)
364360
else:
365361
return HttpResponse("Please insert SMILES")
366362

0 commit comments

Comments
 (0)