Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enumerate frontend fixes needed to reduce request processing time #1438

Open
mwinokan opened this issue May 21, 2024 · 4 comments
Open

Enumerate frontend fixes needed to reduce request processing time #1438

mwinokan opened this issue May 21, 2024 · 4 comments
Assignees
Labels
2024-03-15 indigo Data dissemination loose ends

Comments

@mwinokan
Copy link
Collaborator

mwinokan commented May 21, 2024

@boriskovar-m2ms please investigate what the quick wins, and high-priority bigger fixes that will help reduce the response processing time for API endpoints in #1427

@mwinokan mwinokan added 2024-04-25 pink Stack maintenance/monitoring frontend labels May 21, 2024
@mwinokan mwinokan moved this to FragTech in Fragalysis May 29, 2024
@boriskovar-m2ms boriskovar-m2ms moved this from FragTech to In Progress (DEV) in Fragalysis Jun 5, 2024
@boriskovar-m2ms
Copy link
Collaborator

boriskovar-m2ms commented Jun 6, 2024

After the restoration of bigger snapshots was fixed in #1440 now I had opportunity to analyze snapshot performance because of many complaints in the past especially of big lag between pressing the SHARE button and dialog showing up with link to new snapshot.

Length of this lag is a function of number of actions performed during the lifetime of a session (for undo and redo functionality) and also number of actions needed to restore a snapshots (it's two different sets). There are several and severe inefficiencies as it can be seen on a case study below.

Snapshot performance:

saveAndShareSnapshot

orig snapshot: https://fragalysis.xchem.diamond.ac.uk/viewer/react/projects/47/49
created snapshot: https://fragalysis.xchem.diamond.ac.uk/viewer/react/projects/49/51

  1. https://fragalysis.xchem.diamond.ac.uk/api/session-actions/?session_project=47
    GET
    10.4 MB
    gets all the project actions (what's difference between project action list and actions list - presumably it's snapshot actions?) - getTrackingActions and we want them copied to new project. Why? Don't we already have them? I pretty sure we do.
    Why don't we need only snapshot actions which we definitely have so I don't understand this step.

  2. https://fragalysis.xchem.diamond.ac.uk/api/session-actions/
    POST
    10.4 MB
    copyActionsToProject - here using the sendTrackingActions we are sending copied actions to new project.

  3. https://fragalysis.xchem.diamond.ac.uk/api/session-actions/?session_project=49
    GET
    10.4 MB
    we are downloading project (new project) actions again... which we already have because just moments ago we sent them there...

  4. https://fragalysis.xchem.diamond.ac.uk/api/session-actions/197/
    PUT
    10.4 MB
    using
    updateTrackingActions - because it's just JSON we don't have granular control over updates so we need to update whole list so we need to send whole 10.4 MB payload

  5. https://fragalysis.xchem.diamond.ac.uk/api/session-actions/
    POST
    2.6 MB
    saveActionsList - saveSnapshotAction - seems fine

  6. https://fragalysis.xchem.diamond.ac.uk/api/snapshot-actions/
    POST
    3 MB
    saveActionsList - saveTrackingActions - seems fine

Fixes:

  1. For sharing snapshot I think we just need step 6 because we don't need the undo redo list of creator of the snapshot.

  2. For creating new project I think we should be able to skip steps 1 and 3 because we should already have this information.

There is also inefficiency in step 4. Here all the actions are actually just one field that stores JSON string so if we want to update an action or and new action we need to send whole new string which in this case is not insignificant 10.4 MB. This is also significant because if you are working on a session project then every 5 actions are sent to B/E and every time we need to send all previous actions + 5 new actions.

@mwinokan
Copy link
Collaborator Author

mwinokan commented Jun 6, 2024

Thanks @boriskovar-m2ms, please implement the three fixes above.

I will talk to @phraenquex regarding using the state vs actions:

@boriskovar-m2ms says that previously snapshots used to work by transmitting the state rather than the list of actions needed to restore the state but it was changed due to the instability of ID's. Now that the ID's are more stable maybe we should revisit this.

@boriskovar-m2ms
Copy link
Collaborator

All of the fixes are now implemented not only for snapshot saving but also restoring.

@mwinokan
Copy link
Collaborator Author

mwinokan commented Jun 11, 2024

@boriskovar-m2ms says it is definitely faster but there may be outstanding broken issues. Please push it to staging so we can test it in anger.

Regarding encoding state vs list of actions in the snapshot: Boris says that using the redux store should be relatively simple.

@boriskovar-m2ms please briefly investigate the work involved and the scale of the performance gain. @phraenquex also suggests using compression for the JSON payload

@mwinokan mwinokan added 2024-03-15 indigo Data dissemination loose ends and removed 2024-04-25 pink Stack maintenance/monitoring labels Jun 11, 2024
@boriskovar-m2ms boriskovar-m2ms moved this from In Progress (DEV) to In staging - assess function vs spec in Fragalysis Jun 13, 2024
boriskovar-m2ms added a commit that referenced this issue Jul 2, 2024
* #1403 fixed timer when services are not available

* #1389 applied tag.hidden logic, reduced autohide time for tags toasts to 5 seconds

* Squashed commit of the following:

commit 7511b57
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Apr 23 08:27:24 2024 +0200

    - finished #1348

commit 6074d79
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Wed Apr 17 09:50:06 2024 +0200

    - #1348 - fixed cases when all pdbs are missing

commit 49f0ae8
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Wed Apr 10 13:41:50 2024 +0200

    - checkpoint

commit 13a315e
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Wed Apr 3 12:30:06 2024 +0200

    - checkpoint

commit a5b8cca
Merge: 34b75f7 756a25b
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Mar 25 12:24:01 2024 +0100

    Merge branch '#1348' of https://github.com/m2ms/fragalysis-frontend into #1348

commit 34b75f7
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 12 12:35:21 2024 +0100

    - latest backend

commit 1308cb2
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 1ee15e7
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit a033b68
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 7149081
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 623de4f
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 12 09:48:51 2024 +0100

    - using official backend version

commit eec204c
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 12:16:45 2024 +0100

    - checkpoint

commit dbc93a7
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 10:27:08 2024 +0100

    - checkpoint

commit b933243
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 4a0e171
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 634ed03
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 6e65fae
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 1bff7bf
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 07f21b6
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 021366e
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 36f87b4
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 87e765a
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 12 12:35:21 2024 +0100

    - latest backend

commit 72c94f0
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 224ca23
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 36f03f8
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 8072920
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit e319056
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 12 09:48:51 2024 +0100

    - using official backend version

commit 187e7f3
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 12:16:45 2024 +0100

    - checkpoint

commit b6aa0a4
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 10:27:08 2024 +0100

    - checkpoint

commit fccf976
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 965be5e
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit b01cdbf
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit e04fbf2
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 5d64c26
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit dbb155c
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit b1c0f3c
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Thu Feb 29 10:32:27 2024 +0100

    - checkpoint

commit abc8cc9
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 2ef004a
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 756a25b
Merge: f686ef4 f474ff6
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Thu Mar 14 10:00:02 2024 +0100

    Merge branch '#1348' of https://github.com/m2ms/fragalysis-frontend into #1348

commit f686ef4
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 12 12:35:21 2024 +0100

    - latest backend

commit df6209c
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit f5a842f
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 83ffccc
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 0ecff35
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit d3a90c7
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 12 09:48:51 2024 +0100

    - using official backend version

commit 50800b9
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 12:16:45 2024 +0100

    - checkpoint

commit 2af18fd
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 10:27:08 2024 +0100

    - checkpoint

commit cd75a27
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit c485389
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 96296e9
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 760152b
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 8a30204
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 6235e9c
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit e2e46cb
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Thu Feb 29 10:32:27 2024 +0100

    - checkpoint

commit 8176514
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 6aa5108
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit f474ff6
Merge: 56c0b85 e728f49
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Thu Mar 14 08:30:00 2024 +0100

    Merge branch '#1348' of https://github.com/m2ms/fragalysis-frontend into #1348

    # Conflicts:
    #	js/components/preview/tags/details/tagDetails.js

commit 56c0b85
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit a185933
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 09c97b5
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit c6d0738
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 72f36b0
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Thu Feb 29 10:32:27 2024 +0100

    - checkpoint

commit 971799d
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 1bb5326
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit e728f49
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 12 12:35:21 2024 +0100

    - latest backend

commit 125b8d4
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit 0e41551
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 8cf43e0
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit d336352
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

commit 2fb59dc
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 12 09:48:51 2024 +0100

    - using official backend version

commit 9439f5d
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Thu Mar 7 13:59:03 2024 +0100

    - partial fix for purple release for #1370

commit 4ff5f94
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Thu Mar 7 10:10:41 2024 +0100

    - implemented #1376
    - also fixed bug in download structures dialog where only first download got added to the dropdown menu

commit 66a98f3
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Wed Mar 6 09:47:12 2024 +0100

    - implemented #1361
    - improvements for #1354

commit 8a193a5
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 5 13:24:11 2024 +0100

    - cleanup for #1354

commit bb3bf3b
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 5 13:21:32 2024 +0100

    - updated message for #1354

commit a3dc819
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 5 13:05:12 2024 +0100

    - first implementation of #1354

commit 4121519
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Mar 5 08:17:49 2024 +0100

    - implemented #1357

commit 4d96ff4
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Mar 4 10:07:06 2024 +0100

    - implemented changes requested in #1326

commit 03158be
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Mar 4 08:45:22 2024 +0100

    - fixed problem with tags not visually updating on observations

commit 8765a3c
Author: matej <matej.vavrek@m2ms.sk>
Date:   Mon Feb 26 21:36:12 2024 +0100

    #1341 "pin" edit tag row outside of tag list

commit 6db8b22
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Thu Feb 29 10:32:27 2024 +0100

    - checkpoint

commit 027eaf0
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 12:16:45 2024 +0100

    - checkpoint

commit 7407dbd
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 10:27:08 2024 +0100

    - checkpoint

commit c13628b
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Tue Feb 27 09:23:04 2024 +0100

    - checkpoint

commit e39721c
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Feb 26 08:42:49 2024 +0100

    - checkpoint

* - using official backend now

* Squashed commit of the following:

commit 8fe886a
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Wed May 15 09:00:04 2024 +0200

    Squashed commit of the following:

    commit 6813219
    Author: matej <matej.vavrek@m2ms.sk>
    Date:   Mon Apr 29 07:38:30 2024 +0200

        #1261 added icon for "Manage grouping" button

    commit a665ed5
    Author: matej <matej.vavrek@m2ms.sk>
    Date:   Sun Apr 28 23:23:25 2024 +0200

        #1261 added buttons into observation dialog for later poses handling

commit 70a5e70
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Apr 29 09:20:46 2024 +0200

    - #1261 - initial implementation

* - now FE point to local dev env

* - pointing to correct backend

* - #1440 implemented fix

* fixed tag reset in edit modal

* added better default values handling in tag edit modal

* Squashed commit of the following:

commit 6813219
Author: matej <matej.vavrek@m2ms.sk>
Date:   Mon Apr 29 07:38:30 2024 +0200

commit a665ed5
Author: matej <matej.vavrek@m2ms.sk>
Date:   Sun Apr 28 23:23:25 2024 +0200

* #1393 - small bug fix for csv name

* - implementation of #1441

* Squashed commit of the following:

commit e68fab1
Author: Boris Kovar <boris.kovar@m2ms.sk>
Date:   Mon Jun 10 13:56:06 2024 +0200

    - optimized snapshot creation
    - optimized snapshot restoration
    - fixed bugs when switching between snapshots in one project

---------

Co-authored-by: matej <matej.vavrek@m2ms.sk>
@mwinokan mwinokan moved this from In staging - assess function vs spec to In production (Done) in Fragalysis Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2024-03-15 indigo Data dissemination loose ends
Projects
Status: In production (Done)
Development

No branches or pull requests

3 participants