Migrate to Manifest V3

[b0o]

To use chrome.tabGroups APIs, the extension needs to migrate to Manifest V3.
But using Manifest V3 will break the feature that Surfingkeys allows to execute user scripts, which is important for customization.

— gkzhb on #1445 (comment)

Bolding mine - need to investigate this claim that V3 will break the ability to execute user scripts.

---

Support for Manifest V2 will be removed from Chrome in January 2023, which is 4 months away, so Surfingkeys needs to decide how to handle that quite urgently anyway - I don't think "wait and see" is a viable option any more. Should there be a separate issue for tracking MV3 support?

— aspiers on #1445 (comment)

---

Update 09/28/22

MV2 deprecation in Chrome Stable has been postponed until June 2023.

• Starting in January in Chrome 112, Chrome may run experiments to turn off support for Manifest V2 extensions in Canary, Dev, and Beta channels.
• Starting in June in Chrome 115, Chrome may run experiments to turn off support for Manifest V2 extensions in all channels, including stable channel.
• In June 2023, the Chrome Web Store will no longer allow Manifest V2 items to be published with visibility set to Public. All existing Manifest V2 items with visibility set to Public at that time will have their visibility changed to Unlisted.
• In January 2024, following the expiration of the Manifest V2 enterprise policy, the Chrome Web Store will remove all remaining Manifest V2 items from the store.

See: More details on the transition to Manifest V3

[b0o]

SurfingKeys is a complex extension and will likely require substantial changes to support Manifest V3. I'll be collecting resources for the migration in this comment.

Cross-browser

• Forming the WebExtensions Community Group | WebExtensions Community Group
• w3c/webextensions: Charter and administrivia for the WebExtensions Community Group (WECG)
• WebExtensions Community Group Meeting @ TPAC, 15 September 2022
  • webextensions/2022-09-15-wecg-tpac.md at meeting-2022-09-15-tpac · w3c/webextensions
  • User scripts in Manifest V3 · Issue #279 · w3c/webextensions
  • Publish minutes of 2022-09-15 TPAC meeting by Rob--W · Pull Request #277 · w3c/webextensions

Chrome

Discussions

• [Chrome] Manifest V3: examine the effects · Issue #644 · Tampermonkey/tampermonkey
• Remotely hosted code in MV3
• 1054624 - Dynamic content script support - chromium
• 1135492 - How to use proxy + onAuthRequired in manifest V3 - chromium
• 1207006 - MV3 content scripts need to add js in main world synchronously - chromium
• 1239976 - Circumventing MV3 restriction on RCE via 'fetch' SW event - chromium
• Google postpones MV2 shutoff in Chrome stable to June 2023 | Hacker News
• Pausing Manifest V2 phase-out changes

Official articles

• Manifest V2 support timeline - Chrome Developers
• Manifest V3 migration checklist - Chrome Developers
• More details on the transition to Manifest V3 - Chrome Developers
• Known issues when migrating to Manifest V3 - Chrome Developers

Working Documents

• [Public] Manifest V3 Dynamic Content Scripts - Google Docs

Comments

[...]

Since I've seen some confusion here and elsewhere, I want to quickly reaffirm that the Chrome team is still planning to support power user tools that inject user-written scripts or styles in Maniest V3, but we have not yet begun implementation work. I expect we're likely still a few months out on this. @derjanb, I also expect I'll be following up with you when we have a more substantial update.

— dotproto (Developer Advocate for Chrome Extensions) on Tampermonkey/tampermonkey#644 (comment)

It seems that, as of now, there is still no official guidance from the Chrome team on this.

Firefox

Official articles

• Manifest v3 update | Mozilla Add-ons Community Blog
• Manifest v3 in Firefox: Recap & Next Steps | Mozilla Add-ons Community Blog
• Manifest V3 Firefox Developer Preview — how to get involved | Mozilla Add-ons Community Blog
• Manifest V3 migration guide | Firefox Extension Workshop

Safari

Official articles

• New WebKit Features in Safari 15.4 > Safari Web Extensions | WebKit

[brookhong]

Some API changes are needed for manifest upgrade, see Migrate chrome extension from manifest v2 to v3: How to update these APIs? - Stack Overflow.

[brookhong]

I had looked into this, now the blocker is that background service worker just cannot start, it remains Inactive. To reload the extension does not help either. The worse thing is that there is no error or warning message. It just failed silently.

Looks like the bug just got fixed 4 days ago, will try that again after the fix released.

[brookhong]

Pending on w3c/webextensions#279 (comment)

[EvanCarroll]

This deadline is quickly approaching. The new dates are,

• In June 2023, the Chrome Web Store will no longer allow Manifest V2 items to be published with visibility set to Public. All existing Manifest V2 items with visibility set to Public at that time will have their visibility changed to Unlisted.
• In January 2024, following the expiration of the Manifest V2 enterprise policy, the Chrome Web Store will remove all remaining Manifest V2 items from the store.

[asknet]

@brookhong @b0o Appreciate it if the team could share insights on the current state. Thanks!

[b0o]

I myself don’t have any new info, and unfortunately I currently don’t have the bandwidth to work on this.

[b0o]

This deadline is quickly approaching. The new dates are,

• In June 2023, the Chrome Web Store will no longer allow Manifest V2 items to be published with visibility set to Public. All existing Manifest V2 items with visibility set to Public at that time will have their visibility changed to Unlisted.
• In January 2024, following the expiration of the Manifest V2 enterprise policy, the Chrome Web Store will remove all remaining Manifest V2 items from the store.

I believe the Chrome team pushed back these dates indefinitely. That’s not to say deprecation won’t happen, but there are no official dates as far as I can tell.

We're still working on the timeline of the MV2 phase-out plan, so keep an eye out for it in the coming months. We will provide at least 6 months between a timeline announcement and any experiments deprecating MV2 features.
Before announcing a new timeline, we are addressing prioritized platform gaps and closing critical bugs. You can track these on the Known Issues page.

https://developer.chrome.com/docs/extensions/migrating/mv2-sunset/

[asknet]

@b0o Thank you for the update

[EvanCarroll]

I don't read that the same way, so be cautious here. Those are the updated deadlines as I understand it, and as reported by third parties. From the article titled, "Google delays execution of doomed Chrome extensions"

New MV2 extensions could no longer be added to the Chrome Web Store in June 2022, and that remains unchanged under the new roadmap; MV2 extensions already available the Chrome Web Store can still be downloaded and can still receive updates.

As of June 2023, MV2 extensions will no longer be visible in the store (so they can't be newly installed, but can still be updated for existing users).

Come January 2024, nothing will be left to chance: the Chrome Web Store will stop accepting updates to MV2 extensions, all MV2 extensions will be removed from the store, and the MV2 usage in enterprises will end.

https://www.theregister.com/2022/09/30/google_manifestv3_chrome_extensions/

[b0o]

@EvanCarroll That article is dated 30 Sep 2022, it's about the first time Google delayed MV3. They later delayed it again, here is an article dated 01 Apr 2023:

"We're still working on the timeline of the MV2 phase-out plan, so keep an eye out for it in the coming months. We will provide sufficient migration time for developers – at least six months of heads-up – before beginning any experiments to turn off MV2 in the browser next year."

— Google halts purge of legacy Chrome Extensions, again • The Register

That article is based on this post on Google Groups:

Oliver Dunk
Mar 29, 2023, 10:44:50 AM
to Chromium Extensions, Simeon Vincent
Hey everyone,

As promised, we wanted to share an update on how things are going. Over the last few months, we’ve made progress on a number of improvements based on developer community feedback, such as allowing extension events to extend a service worker’s lifetime, increasing in-memory session storage from 1 MB to 10 MB after cross-browser discussions, and adding more reasons for offscreen documents. We're continuing to improve the platform by making service workers easier to use (including more flexible lifetimes with messaging and other APIs), introducing more service worker APIs in general, and working with other browser vendors to find ways we can align our extension platforms.

We plan to continue reviewing feedback, making changes and improving documentation to ensure the transition from Manifest V2 to Manifest V3 is smooth and successful. We're still working on the timeline of the MV2 phase-out plan, so keep an eye out for it in the coming months. We will provide sufficient migration time for developers - at least 6 months of heads-up - before beginning any experiments to turn off MV2 in the browser next year.

We remain committed to the rollout of MV3 to improve security, privacy, and performance for our users around the world. We also want to ensure that we phase out MV2 in a timely manner, and we’re continuing to listen to feedback from the developer community to help inform our improvements and timelines.

Thanks for your patience while we continue to work on moving the extensions ecosystem forward in a way that supports the needs of users and developers.

Chrome Extensions Team

— https://groups.google.com/a/chromium.org/g/chromium-extensions/c/zQ77HkGmK9E/m/HjaaCIG-BQAJ (If you're interested in this I recommend subscribing to that thread)

As far as I know, there haven't been any additional updates, so I think it should be at least 6+ months away. That's also what the official sunset page says:

We're still working on the timeline of the MV2 phase-out plan, so keep an eye out for it in the coming months. We will provide at least 6 months between a timeline announcement and any experiments deprecating MV2 features.

— https://developer.chrome.com/docs/extensions/migrating/mv2-sunset/

[EvanCarroll]

Thanks, that Mar 29th post is certainly the newest so I'm convinced. Talk about a butchered roll out. They really googled that one up.

[brookhong]

Yes, google must have delayed this again.

As for this extension, I had created a branch https://github.com/brookhong/Surfingkeys/tree/mv3 last December, it roughly works(I haven't verified every function), so MV3 should not be a big problem as long as a string parameter code could be supported by the scripting.executeScript() to inject user-script into context of the extension itself, otherwise we will lose the function to use user snippets to customize settings -- creating users' own bindings, which would definitely be a big loss.

I check w3c/webextensions#279 (comment) occasionally, there is no update this year except more and more extensions coming and requesting for the same support.

[newRoland]

@brookhong I submitted a formal proposal for it. They should hopefully discuss it next Thursday.

w3c/webextensions#477

[b0o]

Another update from Google: https://developer.chrome.com/blog/resuming-the-transition-to-mv3/

[newRoland]

It's unknown whether the userScripts.execute() method will be supported before the deadline, or even at all. Chrome suggests using a wrapped userScript approach for now.

This means adding a userScript that applies to all relevant URLs. The code should be wrapped in a function. You will call that function when needed using window.postMessage and window.onMessage.

[asknet]

@newRoland I don't know the full architecture. With this workaround, what are we losing?

[EvanCarroll]

Thanks @b0o for dropping the link. This is such a botched migration. Google needs help, badly. Here is the tldr of this release, Google will be disabling v2 in Dev, Canary, and Beta in June 2024, and they'll adapt that to stifle backlash until they're comfortable and then at some undetermined point they intend to graduate that functionality to production, in stages.

So with the branch and work already done in some form, I think this project just holds off until June.

[newRoland]

@asknet Surfingkeys allows users to execute user scripts. No loss if you migrate and use the wrapped userScript approach to maintain this feature. The wrapped userScript approach needs to be implemented with care as it's performance isn't great. The user script should be scoped to a particular website because you're pretty much pre-loading. If not scoped to a particular URL, you're pretty much loading it on all websites, even if it's irrelevant and will never be called.

[newRoland]

I have a great update! It looks like the proposal is moving forward.

w3c/webextensions#540

[macintacos]

Heya folks, sorry, it's not clear to me what's supposed to be happening next. The proposal linked above seems to have been merged a few months ago, what are the next steps here? Really would be sad if I was suddenly unable to use Surfingkeys 😅

[brookhong]

I have been working on this for some while, here are some updates

1. The biggest challenge now is to make setting snippet work as in MV2. The setting snippet is just a piece of user script that needs to be executed in the world/context of extension, MV3 does not allow that. Though chrome.userScripts.register can help to inject any user script, the script can only be injected into a USER_SCRIPT world or MAIN world, both of which don't have the necessary functions/objects that need to be used in setting snippet to enable users to create their own mappings/customize settings. Thus we need implement a mechanism to relay those function calls in setting snippet to the extension world, fortunately it is possible, but it still take some efforts to cover all functions/objects that need to be available in setting snippet.
2. Above solution may not work for non-chromium based browsers, so I will probably use mv3 for Chrome release, and merge it back to mainline after MV3 becomes fully ready for other browsers.

[brookhong]

1.17.0 is released(just for Chrome) to migrate to MV 3, please expect that there might be something broken for you though I tested it completely in my Chrome.

[petobens]

Hi @brookhong . In 1.17 I cannot seem to load Load Settings by pointing to a file. Is this expected?

[brookhong]

It works for me after Allow access to file URLs is turned on.

[laphylee]

@brookhong I met the same problems: can't set the customized file path even after I turn on "Allow access to file URLs".

[petobens]

@brookhong I met the same problems: can't set the customized file path even after I turn on "Allow access to file URLs".

Same here

[fecet]

I can read config file from local or remote, but it keep report

[SurfingKeys] Error found in settings: TypeError: Cannot read properties of undefined (reading 'local')`\

[clarkwang]

1.17.0 is released(just for Chrome) to migrate to MV 3, please expect that there might be something broken for you though I tested it completely in my Chrome.

I hit this error with v1.17:

[SurfingKeys] Error found in settings: TypeError: api.Hints.setCharacters is not a function

Seems like api.Hints.setCharacters is now a var and this works for me:

api.Hints.setCharacters = 'abcdefhjklmnoprstuvwxyz';

[brookhong]

@fecet please check your config file and find where local is being referred.

@clarkwang yes, the api setCharacters for Hint is missed to be exported. Though your fix suppresses the error, but it does not actually do the job. This will be fixed soon. Thanks for reporting.

Looks like you guys can load your config file, while @petobens and @laphylee can't.

@petobens Could you try file:///home/<login>/.surfingkeysrc?

[petobens]

@petobens Could you try file:///home//.surfingkeysrc

No success. For the record I'm tried this both in Brave Browser 128.1.69.168 and Microsoft Edge 130.0.2808.0 dev

[fecet]

@fecet please check your config file and find where local is being referred.

I don't find any local in my config except chrome.storage.local, does it matter? And I find

Uncaught (in promise) Error: Could not establish connection. Receiving end does not exist.

at chrome://extensions/?errors=gfbliohnnapiefjpjlpjnehglfpaknnc

[jackmac92]

Getting this error after the update [SurfingKeys] Error found in settings: TypeError: api.vmapkey is not a function

edit: but the rest of my config still works which is awesome!! thanks for the work on this @brookhong

[brookhong]

@fecet please check your config file and find where local is being referred.

I don't find any local in my config except chrome.storage.local, does it matter? And I find

Uncaught (in promise) Error: Could not establish connection. Receiving end does not exist.

at chrome://extensions/?errors=gfbliohnnapiefjpjlpjnehglfpaknnc

Yeah, that's the problem, chrome.storage is not allowed in user script, which is a restriction from Chrome.

[brookhong]

Getting this error after the update [SurfingKeys] Error found in settings: TypeError: api.vmapkey is not a function

edit: but the rest of my config still works which is awesome!! thanks for the work on this @brookhong

Thanks, vmapkey is missed from exporting, will be fixed soon.

[laphylee]

@brookhong I tried both "d:\OneDriveShare\Setting\Surfingkeys.js" and "file:///d:/OneDriveShare/Setting/Surfingkeys.js" but not successful.
I got this error on console log:

[brookhong]

For anyone who is unable to load his/her config file, please try @matko031's solution #2149 (comment), just to remove the extension then re-add it.

[deadcyclo]

For anyone who is unable to load his/her config file, please try @matko031's solution #2149 (comment), just to remove the extension then re-add it.

I tried this workaround both in chrome and in vivaldi and it didn't work in either.

Edit: Getting the following error message when trying to save configuration:



options.js:1 Uncaught TypeError: Cannot read properties of null (reading 'getValue')
at HTMLInputElement.A (options.js:1:3849)

[paulexyz]

Same here, just tried to install Surfingkeys in a new Chrome profile (Version 1.17.0)
And I can't access a file with my settings: Cannot read properties of null (reading 'getValue')

Chrome version 128.0.6613.84 (Official Build) (64-bit)

[lost22git]

Same error info when save configuration:

and also cannot add proxy pair.

Extension seems no error info hints:

[bojanjovicic]

Same error info when save configuration:

and also cannot add proxy pair.

Extension seems no error info hints:

Same here:

[brookhong]

Closing this now, and please report your broken use cases here with #2159.