fix(caa upload): compatibility with Greasemonkey #224

ROpdebee · 2021-11-07T22:54:50Z

Closes #123.

Use correct run-at value
Migrate to GM.* API by default and polyfill them when the userscript engine doesn't support them.
- We request grants for both the GM.* and GM_* version, one of the two (or both) will be granted. ~~If the GM_ version is granted, we polyfill the GM.* version based on that. This is done through the gm4-polyfill requirement.~~
- gm4-polyfill is injected into the script itself rather than @required so that we can transpile it for compatibility with older browsers. It's currently included as a submodule, so if you want to build it locally, you may have to init and update the submodule first. The build process is a bit hack-y since gm4-polyfill requires this, which isn't available in a rolled-up IIFE, so we patch it to have one. Eventually, I'd like to separate that out of the IIFE (perhaps as a separate @require).
- Added a compatibility layer to dispatch to the correct API, depending on which one exists.
- The switch to GM.* APIs also means we have to make some changes for IMU. Most importantly, we now have to deal with IMU's exports possibly being defined asynchronously, because GM.getValue and GM.setValue are asynchronous.
The main changes are in handling situations where we might be running in the content context instead of the page context. Under Greasemonkey, userscripts run in the content context and therefore have no access to window globals defined by the page itself. This includes window.MB and jQuery. Although we can access those globals through unsafeWindow, we can't provide our content globals to the page without cloning it first.
- fetch doesn't support relative URLs, so we're using absolute ones now.
- We can't easily use MB.CoverArt.validate_file in content scripts, since we provide content context functions to be called by page content, which leads to permission errors. Therefore, we duplicate MB's implementation.
- Similarly, triggering the jQuery event to enqueue images leads to permission errors, so we clone the event into the page content and trigger the handler manually to prevent jQuery from calling into content context functions.
- NB: We might be able to still have the page context call our content context functions if we clone those too.
- NB: Because content scripts cannot access page script globals, any polyfills injected by MB won't be visible to the userscript in Greasemonkey, so old Firefox + Greasemonkey likely still won't work (although it seemed to work properly with FF82).

Tested with:

Firefox 95.0a1 Violentmonkey, Tampermonkey, Greasemonkey
Chromium 97.0.4672.0 Violentmonkey, Tampermonkey
Firefox 82 Greasemonkey (I can't test on any older versions because they don't want to run on my OS...)
Waterfox Classic 2021.10 w/ Greasemonkey 3.17

`document-load` doesn't exist. I meant `document-end`...

Greasemonkey scripts run in the content context, not the page context, so the `fetch` calls don't properly resolve relative URLs. Easiest way to resolve this is by providing absolute URLs.

Mostly typing changes

…hanges `GM_xmlhttpRequest` -> `GM.xmlHttpRequest` `GM_getResourceURL` -> `GM.getResourceUrl`

These changes ensure the GM API is polyfilled only after the core-js polyfills are added and that they are transpiled by babel.

We cannot access the MB object defined in `window` since we're not running in a page context in Greasemonkey. Although we could circumvent this using `unsafeWindow`, this leads to issues with permission errors because we're crossing context boundaries with callback functions.

Greasemonkey scripts that use granted APIs run in the content context, not the page context, and therefore have no access to the jQuery object defined by MB. We "escape" this through `unsafeWindow` and `cloneInto` to trigger the event. We also trigger the handler manually rather than triggering the event via jQuery, since jQuery tries to call some functions defined on the event but the page scripts don't have that permission.

ImageMaxUrl only exports after all `GM.setValue` and `GM.getValue` calls have been done. With `GM.*` APIs, these are asynchronous, but there's no way for us to await them. Therefore, we retry on a timeout until IMU is exported.

ROpdebee · 2021-11-07T22:55:16Z

/deploy-preview

codecov · 2021-11-07T22:56:18Z

Codecov Report

Merging #224 (fe4a7e0) into main (8eb249c) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main      #224   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           39        39           
  Lines          857       860    +3     
  Branches       158       158           
=========================================
+ Hits           857       860    +3

Impacted Files	Coverage Δ
src/lib/MB/EditNote.ts	`100.00% <100.00%> (ø)`
src/lib/MB/URLs.ts	`100.00% <100.00%> (ø)`
src/lib/util/blob.ts	`100.00% <100.00%> (ø)`
src/lib/util/xhr.ts	`100.00% <100.00%> (ø)`
src/mb_enhanced_cover_art_uploads/fetch.ts	`100.00% <100.00%> (ø)`
src/mb_enhanced_cover_art_uploads/form.ts	`100.00% <100.00%> (ø)`
src/mb_enhanced_cover_art_uploads/maximise.ts	`100.00% <100.00%> (ø)`
.../mb_enhanced_cover_art_uploads/providers/amazon.ts	`100.00% <100.00%> (ø)`
...rc/mb_enhanced_cover_art_uploads/providers/base.ts	`100.00% <100.00%> (ø)`
...c/mb_enhanced_cover_art_uploads/providers/qobuz.ts	`100.00% <100.00%> (ø)`
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8eb249c...fe4a7e0. Read the comment docs.

ROpdebee · 2021-11-07T23:05:58Z

Deploying a preview is currently broken because it doesn't clone the submodules (will be fixed when PR is merged though), so here's one I made manually: https://github.com/ROpdebee/mb-userscripts/raw/dist-preview-224/mb_enhanced_cover_art_uploads.user.js

@Freso could you give the preview above a try and see if it works for you?

kellnerd

Looking good, I also tested with GM, TM and VM in Firefox 94 and everything I tried is working as it should.

build/plugin-userscript.ts

build/rollup.ts

src/mb_enhanced_cover_art_uploads/form.ts

Freso · 2021-11-08T16:16:43Z

@Freso could you give the preview above a try and see if it works for you?

Seems like it’s working for me now! :)

ROpdebee · 2021-11-08T22:30:22Z

It still doesn't work in GM <4, see https://chatlogs.metabrainz.org/libera/musicbrainz/msg/4904321/. Tested with Waterfox Classic 2021.10, running GM3.17.

Root cause is another piece of context switching between page and content scripts, more specifically in the derivation of the mime type. It seems to be related to greasemonkey/greasemonkey#2034. The script freezes once it gets to this line:

mb-userscripts/mb_enhanced_cover_art_uploads.user.js

Line 4228 in 351a4ce

if ((uint32view[0] & 0x00FFFFFF) === 0x00FFD8FF) {

For @CatQuest, it throws a (surprisingly helpful) error about TypedArrays and Xrays, for me personally, it just freezes without any error and doesn't get past that line.

In short: FileReader is W3C, Uint32Array is ES. Therefore, Uint32Array is defined in the userscript sandbox, whereas FileReader comes from the page itself and is Xrayed (proxied, sort of). Accessing the Uint32Array contents leads to xraying to some underlying ArrayBuffer which exists in the page context, which Firefox (Waterfox) refuses to do because of performance issues. Reference: https://bugzilla.mozilla.org/show_bug.cgi?id=1122687#c8

Solution: use unsafeWindow.Uint32Array instead of bare Uint32Array. This prevents the freeze for me at least. Will apply it later. Should also be applied to hexEncode since that also uses FileReader with Uint8Array

For some reason, under GM4.1 in the same Waterfox version, it works just fine. I guess it uses a different implementation of the sandbox.

build/plugin-add-imports.ts

build/rollup.ts

We're using the compat layer now.

They exist to ensure compatibility between different userscript engines and browsers, so they belong there.

Under GM3, the script wasn't properly switching contexts when processing `Uint*Array` instances created from `ArrayBuffer`s. See #224 (comment)

ROpdebee · 2021-11-11T19:51:28Z

/deploy-preview should work again now because we're not using git submodules anymore

github-actions · 2021-11-11T19:52:27Z

This PR changes 1 built userscript(s):

mb_enhanced_cover_art_uploads (install preview, changes: f54ec25)

See all changes

fix(caa upload): compatibility with Greasemonkey (#224)

kellnerd

Looking good without the unnecessary polyfills and hacks to make those work, thanks.

src/compat.ts

Under GM3, the script wasn't properly switching contexts when processing `Uint*Array` instances created from `ArrayBuffer`s. See #224 (comment)

github-actions · 2021-11-13T18:11:21Z

🚀 Released 1 new userscript version(s):

mb_enhanced_cover_art_uploads 2021.11.13 in 28eb59b

fix(caa upload): compatibility with Greasemonkey (#224)

ROpdebee added 10 commits November 7, 2021 15:14

fix(caa upload): use proper @run-at value

d090747

`document-load` doesn't exist. I meant `document-end`...

fix(caa upload): fix fetch calls in Greasemonkey

be12598

Greasemonkey scripts run in the content context, not the page context, so the `fetch` calls don't properly resolve relative URLs. Easiest way to resolve this is by providing absolute URLs.

fix(caa upload): support GMv4 API

9ce536a

fix(caa upload): switch to GM.* APIs

d7857ba

Mostly typing changes

fix(caa upload): properly handle @grant of GM functions with name c…

fe486fd

…hanges `GM_xmlhttpRequest` -> `GM.xmlHttpRequest` `GM_getResourceURL` -> `GM.getResourceUrl`

fix(caa upload): transpile GM4 polyfills

a5cf9a2

These changes ensure the GM API is polyfilled only after the core-js polyfills are added and that they are transpiled by babel.

fix(caa upload): wait for IMU to export under GM.* APIs

9b028c4

ImageMaxUrl only exports after all `GM.setValue` and `GM.getValue` calls have been done. With `GM.*` APIs, these are asynchronous, but there's no way for us to await them. Therefore, we retry on a timeout until IMU is exported.

build(deps): enable Dependabot on git submodules

38a1043

ROpdebee added bug Something isn't working mb_enhanced_cover_art_uploads labels Nov 7, 2021

ROpdebee requested a review from kellnerd November 7, 2021 22:54

ROpdebee added a commit that referenced this pull request Nov 7, 2021

preview for #224

351a4ce

ROpdebee mentioned this pull request Nov 7, 2021

[cover art uploader] Uncaught (in promise) TypeError: e.split(...).at is not a function #214

Closed

kellnerd reviewed Nov 8, 2021

View reviewed changes

build/plugin-userscript.ts Show resolved Hide resolved

build/rollup.ts Outdated Show resolved Hide resolved

src/mb_enhanced_cover_art_uploads/form.ts Outdated Show resolved Hide resolved

kellnerd reviewed Nov 10, 2021

View reviewed changes

build/plugin-add-imports.ts Outdated Show resolved Hide resolved

build/rollup.ts Outdated Show resolved Hide resolved

ROpdebee force-pushed the ecau-greasemonkey branch from 3b9ed95 to f0a76c7 Compare November 11, 2021 18:55

ROpdebee added 4 commits November 11, 2021 20:29

refactor(caa upload): use a compatibility layer for GM functions

23bc888

refactor: revert including GM4 polyfills

6414769

We're using the compat layer now.

refactor: move context switching functions into compat

761af1c

They exist to ensure compatibility between different userscript engines and browsers, so they belong there.

fix(caa upload): compatibility with GM3

8991394

Under GM3, the script wasn't properly switching contexts when processing `Uint*Array` instances created from `ArrayBuffer`s. See #224 (comment)

ROpdebee force-pushed the ecau-greasemonkey branch from f0a76c7 to 8991394 Compare November 11, 2021 19:32

refactor(caa upload): simplify drop event trigger

b341a25

ROpdebee force-pushed the ecau-greasemonkey branch from dd9f16d to b341a25 Compare November 11, 2021 19:50

github-actions bot added a commit that referenced this pull request Nov 11, 2021

🤖 mb_enhanced_cover_art_uploads 2021.11.11

f54ec25

fix(caa upload): compatibility with Greasemonkey (#224)

kellnerd approved these changes Nov 11, 2021

View reviewed changes

src/compat.ts Show resolved Hide resolved

fix(build): also grant VM's spelling of getResourceUrl

fe4a7e0

ROpdebee merged commit 1a35ef5 into main Nov 13, 2021

ROpdebee added a commit that referenced this pull request Nov 13, 2021

fix(caa upload): compatibility with GM3

bd63783

Under GM3, the script wasn't properly switching contexts when processing `Uint*Array` instances created from `ArrayBuffer`s. See #224 (comment)

ROpdebee deleted the ecau-greasemonkey branch November 13, 2021 18:09

github-actions bot added the deploy:success label Nov 13, 2021

github-actions bot added a commit that referenced this pull request Nov 13, 2021

🤖 mb_enhanced_cover_art_uploads 2021.11.13

28eb59b

fix(caa upload): compatibility with Greasemonkey (#224)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(caa upload): compatibility with Greasemonkey #224

fix(caa upload): compatibility with Greasemonkey #224

ROpdebee commented Nov 7, 2021 •

edited

Loading

ROpdebee commented Nov 7, 2021

codecov bot commented Nov 7, 2021 •

edited

Loading

ROpdebee commented Nov 7, 2021 •

edited

Loading

kellnerd left a comment

Freso commented Nov 8, 2021

ROpdebee commented Nov 8, 2021

ROpdebee commented Nov 11, 2021

github-actions bot commented Nov 11, 2021

kellnerd left a comment

github-actions bot commented Nov 13, 2021

fix(caa upload): compatibility with Greasemonkey #224

fix(caa upload): compatibility with Greasemonkey #224

Conversation

ROpdebee commented Nov 7, 2021 • edited Loading

ROpdebee commented Nov 7, 2021

codecov bot commented Nov 7, 2021 • edited Loading

Codecov Report

ROpdebee commented Nov 7, 2021 • edited Loading

kellnerd left a comment

Choose a reason for hiding this comment

Freso commented Nov 8, 2021

ROpdebee commented Nov 8, 2021

ROpdebee commented Nov 11, 2021

github-actions bot commented Nov 11, 2021

kellnerd left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 13, 2021

ROpdebee commented Nov 7, 2021 •

edited

Loading

codecov bot commented Nov 7, 2021 •

edited

Loading

ROpdebee commented Nov 7, 2021 •

edited

Loading