Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: maybe it's time to rethink the possibility of porting to Safari #52

Closed
2 of 8 tasks
dminca opened this issue Jan 6, 2021 · 64 comments
Closed
2 of 8 tasks
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@dminca
Copy link

dminca commented Jan 6, 2021

Just asking if it's possible to make this wonderful extension available for Safari now, given the following article published by Apple Developers in regards to converting web extensions for Safari

https://developer.apple.com/news/?id=qiz0arxc

Prerequisites

  • I verified that this is not a filter issue
  • This is not a support issue or a question
  • I performed a cursory search of the issue tracker to avoid opening a duplicate issue
    • Your issue may already be reported.
  • I tried to reproduce the issue when...
    • uBlock Origin is the only extension
    • uBlock Origin with default lists/settings
    • using a new, unmodified browser profile
  • I am running the latest version of uBlock Origin
  • I checked the documentation to understand that the issue I report is not a normal behavior

Description

[Description of the bug or feature]

A specific URL where the issue occurs

[A specific URL is MANDATORY for issue happening on a web page, even if it happens "everywhere"]

Steps to Reproduce

  1. [First Step]
  2. [Second Step]
  3. [and so on...]

Expected behavior:

[What you expected to happen]

Actual behavior:

[What actually happened]

Your environment

  • uBlock Origin version:
  • Browser Name and version:
  • Operating System and version:
@uBlock-user uBlock-user added the invalid This doesn't seem right label Jan 6, 2021
@emikaadeo-git
Copy link

Hi,
maybe we can reconsider this with new experimental uBOLite MV3 API-based content blocker ?
Safari have support for MV3 extensions since version 15.4 https://webkit.org/blog/12445/new-webkit-features-in-safari-15-4/#safari-web-extensions

@gorhill
Copy link
Member

gorhill commented Sep 20, 2022

I don't have MacOS, this would have to be undertaken by a another volunteer developer.

Anyways, the implementation of declarativeNetRequest API on Safari is missing key capabilities for the current state of uBOL, it does not support conditions such as initiatorDomains, excludedInitiatorDomains, requestDomains, excludedRequestDomains. It also does not support action types redirect which will be needed once uBOL starts to use this capability.

Additionally, the Safari scripting API is also missing key capabilities, namely registeredContentScript, getRegisteredContentScript, unregisterContentScripts, updateContentScripts.

@emikaadeo-git
Copy link

@gorhill
Hi,
thanks for clarification.

@henryhchchc
Copy link

Just noticed that Apple added support for redirect 10 months ago as mentioned in https://webkit.org/blog/12193/release-notes-for-safari-technology-preview-139/

Added support for the redirect rule type in declarativeNetRequest, which requires host permissions to be granted for the host of the URL being redirected

I think this feature is available in Safari 16 but Apple haven't updated their documentation.

@abarisain
Copy link

It looks like safari 16.4, in beta, adds support for a lot of what is missing
image

That said, there is still this ridiculous requirement for it to be wrapped in an app, so someone will have to do that.

@wavded
Copy link

wavded commented Apr 3, 2023

Safari 16.4 is out of beta now, not sure how close this gets us.

@dminca
Copy link
Author

dminca commented Apr 3, 2023

I’ve posted on Mastodon to Jen and Jon from the WebKit team, let’s see if they’re considering the missing capabilities mentioned by Raymond https://mastodontech.de/@dminca/110135906549832299

@helvetiica
Copy link

Any updates for Safari 17 (macOS Sonoma)?

@gorhill gorhill transferred this issue from uBlockOrigin/uBlock-issues Jun 27, 2023
@gorhill
Copy link
Member

gorhill commented Jun 27, 2023

Transferring the issue to uBO Lite issues repo because if ever there is a port to Safari, that will be MV3 uBO Lite, not uBO, which requires MV2.

@gorhill gorhill reopened this Jun 27, 2023
@MasterKia MasterKia added enhancement New feature or request and removed invalid This doesn't seem right labels Jun 27, 2023
@Knaledge
Copy link

Knaledge commented Jul 5, 2023

Are there currently any other barriers to entry for restarting development of the Safari Extension for uBOL? Perhaps there is an opportunity to engage the broader community to alleviate some of those impediments (if there are any)

@stephenhawk8054
Copy link
Member

Are there currently any other barriers to entry for restarting development of the Safari Extension for uBOL?

Maybe the only barrier is someone willing to fork and develop it for Safari?

@hadig
Copy link

hadig commented Aug 23, 2023

Converting the firefox variant with the safari-web-extension-converter seems to work somehow out of the box on macOS Sonoma.
The dashboard is blank, but extension badge is updating blocked ads count.
Ad blocking seems to work, except youtube video ads.

@arjpar
Copy link

arjpar commented Sep 7, 2023

@hadig I'm also willing to work on porting UBO Lite with whatever code you have. Since UBO is such a highly used extension I'm sure there are others (even if few) that may be willing to work on porting it (UBO Lite) to Safari.

@khesed
Copy link

khesed commented Sep 8, 2023

Made a PR: #71

@aayushbhatia06
Copy link

I know uBlock team is famous for not taking donations (for all the right reasons I might add) but just wanted to say if another interested developer would like to develop but has monetary hurdles myself and a LOT of other Safari users would be happy to contribute.

@ghost
Copy link

ghost commented Mar 9, 2024

Is there any update on this issue/suggestion?
And also want to re-iterate I would also gladly donate just for starting uBlock Origin for Safari development again :)

@arjpar
Copy link

arjpar commented Mar 9, 2024

@Webtapper There's no update that I know of. A PR was made and rejected because Safari's implementation of DNR (declarativeNetRequest) lacks some specific features that uBO-Lite on Chrome/(Firefox?) requires to function properly. There was no minimal viable product in the PR. I'd be more than happy to develop a port, without donations, but I need to check how much support for DNR the Safari team has added. The WebKit devs have moved the project to GitHub, and they're willing to engage with the developer community. I'm sure I/someone else can look at Safari's documentation for DNR and see what is missing/implemented for an MVP for Safari to come to fruition.

In the PR I mentioned above, @gorhill stated that specific APIs weren't supported by Safari's DNR implementation, and an investigation was/is needed to figure out which APIs are and are not supported.

Like I said above, I'd be more than happy to port uBO-Lite to safari, I just haven't had the time. The requirement for a Safari extension to also be published on the app store and fulfilling the requirements for that ($100 for a dev license) is also one thing that may put a strain on a port.

Hopefully this answered your question!

@stephenhawk8054
Copy link
Member

Just FYI that any volunteers who want to take a look can check the table comparison of Safari's MV3 ability here: https://github.com/uBlockOrigin/uBOL-home/wiki/Availability-of-MV3-APIs-used-by-uBOL

I think it's better to PR and complete the table with clear sources first to have a better picture of Safari's compatibility with uBOL.

@arjpar
Copy link

arjpar commented Mar 9, 2024

For/to all those concerned, I'll be starting work on a port here, and then eventually pushing upstream.

@arjpar
Copy link

arjpar commented Jun 24, 2024

@Handwork5615 I read it, and I'm happy many people would support the project. I was thinking about writing my own Network Filtering Engine, thats on par or even better than uBlock Origin's, but I'm not sure if it would be allowed by Apple's App Store policies or if the APIs allow it (I don't believe so). Regardless I want to try to support the following:

  • Dynamic URL Filtering
  • Dynamic Filtering
  • Static Filtering
  • Pattern based filtering
  • Cosmetic filtering
  • Scriptlet injection filtering
  • Pop-up blocking
  • Removing tracking parameters
  • Show number of blocked requests
  • Suspend network activity until all filter lists are loaded
  • Custom filters
  • Custom rules
  • Trusted sites
  • Enable/disable blocker for current website
  • Element zapper

It will also be written entirely in Swift, SwiftUI, be a Catalyst & Universal App. Since it will use the latest technologies provided, it will only be available on recent versions of iOS, iPadOS, macOS & visionOS. These are just my aspirational goals. I think the Apple Documentation page for Content Blocking might be out of date, or maybe I'm just not looking in the right place. Again, these are just my idealistic/aspirational goals, I need to do further investigating to see if its feasible. I'm also still learning Swift, SwiftUI, & Apple Development, but I want to implement features in the most optimal way possible.

@abarisain
Copy link

I am very, very sorry to be the party pooper but it may be time to move this discussion elsewhere, please. I for one am satisfied with wipr and only interested in ublock.

@arjpar
Copy link

arjpar commented Jun 24, 2024

@dminca @gorhill This issue should be closed, since it's not possible to port uBlock Origin Lite to Safari currently. I have created an organization & repository to try to "re-create" uBlock Origin as much as we can in Safari using the Safari-relevant APIs. Further discussion can take place there. I'm currently in the design phase of the app, rather than just moving fast & breaking things.

@dminca
Copy link
Author

dminca commented Jun 25, 2024

@dminca @gorhill This issue should be closed, since it's not possible to port uBlock Origin Lite to Safari currently. I have created an organization & repository to try to "re-create" uBlock Origin as much as we can in Safari using the Safari-relevant API. Further discussion can take place there. I'm currently in the design phase of the app, rather than just moving fast & breaking things.

fair enough. Appreciate the effort of trying to integrate it.

Unfortunately the Safari team from Apple was not so communicative 😔

@gorhill
Copy link
Member

gorhill commented Aug 5, 2024

This issue should be closed, since it's not possible to port uBlock Origin Lite to Safari currently

Closing as suggested. Maybe in some future this will be possible, in such case the best action is for volunteer(s) to submit a pull request to bring support to Safari -- opening an issue asking to bring support to Safari is not really the way to go. Opening a discussion about this is fine.

@gorhill gorhill closed this as not planned Won't fix, can't repro, duplicate, stale Aug 5, 2024
@Dacatzgoescodin5267

This comment was marked as off-topic.

@vitorgalvao
Copy link

The only thing such an attitude accomplishes is forcing maintainers to block the disruptor and lock the thread. Everyone loses and it does nothing to advance the cause. On the contrary, it makes people more unwilling to help.

Please don’t do that. Volunteers are giving away their time and expertise for free so everyone else can enjoy. A little courtesy and respect is the minimum.

I’m going to take the opportunity to thank the maintainers of the uBlock Origin organisation for all their work, with a special mention to @gorhill for starting the project.

It’s a shame we can’t make it work in Safari—hopefully that is just a “yet”—but thank you all for looking into it and answering questions.

@arjpar
Copy link

arjpar commented Aug 14, 2024

It’s a shame we can’t make it work in Safari

@vitorgalvao To clarify, while it’s not possible to directly run uBlock Origin Lite (uBO Lite) on Safari as a WebExtension, there is a viable alternative that achieves similar functionality. Although we don’t share any code with uBO Lite, we’ve developed a solution that emulates its behavior from the end-user’s perspective.

We’ve achieved this by leveraging AdGuard’s filter lists, which are specifically optimized for mobile and Safari. The result is a tool that falls between uBlock Origin Lite and the full uBlock Origin (as used in Firefox) in terms of effectiveness. We’ve intentionally avoided naming it uBO Lite since it doesn’t share the same codebase, but it does match and even surpass uBO Lite in certain areas, such as support for custom filter lists and improved performance.

Our approach involves converting everything to Safari’s content-blocking format, including transforming extended CSS and scriptlets to JSON, which we then execute in the respective files. Although it’s currently in an alpha-beta stage, it works well for the vast majority of users (around 99%).

If anyone is interested, I’m happy to provide further details, though I recognize that this might be beyond the scope of the current discussion. The key takeaway is that while a direct port of uBO Lite to Safari isn’t feasible, a comparable solution is indeed possible using Safari’s APIs and other existing technologies. It might not be uBO Lite by name, but functionally, it delivers a very similar experience, often with added benefits.

@vitorgalvao
Copy link

If you (or anyone) are/is interested I can provide more details.

Yes, please. I’m curious in what you mean specifically.

@arjpar
Copy link

arjpar commented Aug 14, 2024

If you (or anyone) are/is interested I can provide more details.

Yes, please. I’m curious in what you mean specifically.

To elaborate, we utilize AdGuard’s ConverterLib to transform optimized filter lists for Safari, including AdGuard Base and others that are already comparable to uBlock Origin’s (uBO) lists, and surpass uBO Lite’s lists due to customization options (such as toggling filters on/off and adding custom lists which is yet to be fully implemented in WebShield). These lists are converted to JSON: static filters and rules supported by Safari’s Content Blocking API become a content-blocking JSON file, while scriptlets and extended CSS filters are converted into a single JSON file and executed separately by a dedicated scriptlet and extended CSS JavaScript file.

We’re planning to fork and enhance ConverterLib to further optimize performance and efficiency, though it’s already quite robust. The primary performance bottleneck comes from scriptlet injection, so we’re also working on improving their scriptlet library, with the aim of contributing these improvements upstream for the benefit of everyone.

The reason we don’t use uBlock Origin’s lists is that there aren’t optimized filter lists for Safari/mobile like there are with AdGuard. With AdGuard’s lists, we achieve nearly all the benefits (around 96%) while using fewer than ~80k rules, which is fewer than (or on par with) uBO’s default set.

The project I’m working on, WebShield, is designed to be more efficient and performant than both uBO Lite and possibly even uBO. Our content blocking (static and some cosmetic filtering) consumes minimal CPU and memory resources—less than 1% of CPU and a few kilobytes of RAM, as far as I recall. Scriptlet injection and extended CSS use around 40MB of memory. WebShield also allows for custom filter lists, enabling/disabling defaults, and even an element zapper mode (which is yet to be implemented but feasible).

The reason the content blocking is so lightweight is that it’s converted into native WebKit rules. I’m working on optimizing the entire pipeline, from our current code to the conversion process (via SafariConverterLib, also known as BlockKit), and scriptlet/extended CSS injection. I’m also considering using Protobuf instead of JSON for scriptlet/extended CSS injection, as it’s faster, though I need to weigh the trade-offs.

In my opinion, WebShield already outperforms alternatives like Wipr 2, which lacks customisation and relies on some manual filter list maintenance, and 1Blocker, which is paid and seems to use custom-curated lists that don’t block as effectively as Wipr 2, AdGuard, or WebShield. Moreover, AdGuard requires separate purchases for macOS and iOS to unlock all features, whereas WebShield will always be free (in both freedom and cost) and open-source. I’m currently evaluating different licensing options to best serve the community and Safari users.

If you or anyone else has further questions, feel free to ask, though it might be more appropriate to discuss this on the WebShield discussions page, as this conversation might be beyond the scope of a uBO issue tracker. We also have a sister project, wBlock, which is developed in parallel with WebShield and shares much of its codebase. Questions about wBlock can be directed to its issue tracker, though I’m not sure if @0xCUB3 will open GitHub discussions for that repository.

@0xCUB3
Copy link

0xCUB3 commented Aug 15, 2024

If you (or anyone) are/is interested I can provide more details.

Yes, please. I’m curious in what you mean specifically.

To elaborate, we utilize AdGuard’s ConverterLib to transform optimized filter lists for Safari, including AdGuard Base and others that are already comparable to uBlock Origin’s (uBO) lists, and surpass uBO Lite’s lists due to customization options (such as toggling filters on/off and adding custom lists which is yet to be fully implemented in WebShield). These lists are converted to JSON: static filters and rules supported by Safari’s Content Blocking API become a content-blocking JSON file, while scriptlets and extended CSS filters are converted into a single JSON file and executed separately by a dedicated scriptlet and extended CSS JavaScript file.

We’re planning to fork and enhance ConverterLib to further optimize performance and efficiency, though it’s already quite robust. The primary performance bottleneck comes from scriptlet injection, so we’re also working on improving their scriptlet library, with the aim of contributing these improvements upstream for the benefit of everyone.

The reason we don’t use uBlock Origin’s lists is that there aren’t optimized filter lists for Safari/mobile like there are with AdGuard. With AdGuard’s lists, we achieve nearly all the benefits (around 96%) while using fewer than ~80k rules, which is fewer than (or on par with) uBO’s default set.

The project I’m working on, WebShield, is designed to be more efficient and performant than both uBO Lite and possibly even uBO. Our content blocking (static and some cosmetic filtering) consumes minimal CPU and memory resources—less than 1% of CPU and a few kilobytes of RAM, as far as I recall. Scriptlet injection and extended CSS use around 40MB of memory. WebShield also allows for custom filter lists, enabling/disabling defaults, and even an element zapper mode (which is yet to be implemented but feasible).

The reason the content blocking is so lightweight is that it’s converted into native WebKit rules. I’m working on optimizing the entire pipeline, from our current code to the conversion process (via SafariConverterLib, also known as BlockKit), and scriptlet/extended CSS injection. I’m also considering using Protobuf instead of JSON for scriptlet/extended CSS injection, as it’s faster, though I need to weigh the trade-offs.

In my opinion, WebShield already outperforms alternatives like Wipr 2, which lacks customisation and relies on some manual filter list maintenance, and 1Blocker, which is paid and seems to use custom-curated lists that don’t block as effectively as Wipr 2, AdGuard, or WebShield. Moreover, AdGuard requires separate purchases for macOS and iOS to unlock all features, whereas WebShield will always be free (in both freedom and cost) and open-source. I’m currently evaluating different licensing options to best serve the community and Safari users.

If you or anyone else has further questions, feel free to ask, though it might be more appropriate to discuss this on the WebShield discussions page, as this conversation might be beyond the scope of a uBO issue tracker. We also have a sister project, wBlock, which is developed in parallel with WebShield and shares much of its codebase. Questions about wBlock can be directed to its issue tracker, though I’m not sure if @0xCUB3 will open GitHub discussions for that repository.

To this point, wBlock is ready for use as an alpha (I'm the dev btw). You just have to build it with your own Xcode. I think porting ublock is not only feasible, but would be very beneficial.

@abarisain
Copy link

Can we please take this elsewhere?

The explanation was interesting, but I think that this is not the place to discuss this anymore

All that will end up with is subscribers never being able to know of a safari uBO light via this issue because this thread ends up closed.

@leo60228
Copy link

Due to the issues explained here which explains the problems I've run into, a port of uBlock Origin Lite to Safari is currently not possible.

@arjpar The linked post was deleted. What were the issues described?

@arjpar
Copy link

arjpar commented Oct 25, 2024

@leo60228

To answer your question: it's because Safari (at the time, not sure about recent versions of WebKit) didn't support the required DNR & other APIs to produce a MVP for uBO Lite.

While Safari's limited APIs prevent a direct port of uBlock Origin Lite, I'm developing a full content blocking solution (like uBlock Origin) specifically for Safari that's showing promising results. Here's what's been accomplished:

Key Features:

  • Ad blocking functionality
  • Custom filter list support
  • Safari's Content Blocking API integration, which converts filter lists into native WebKit rules
  • Pre-page-load processing for efficient blocking
  • JavaScript scriptlet/extended-CSS implementation for advanced features

Current Status:

  • Core functionality exceeds uBlock Origin Lite's capabilities
  • Preliminary testing shows performance matching or potentially surpassing uBlock Origin
  • UI refinement and code stabilization in progress
  • Expected TestFlight release in a few months

Technical Notes:

  • Uses AdGuard's technology for filter list processing and conversion
  • We chose AdGuard for their optimized Safari and mobile filter lists
  • Completely separate codebase from uBlock Origin
  • Built as an alternative solution due to Safari's lack of DNR and other MV3 APIs required by uBlock Origin Lite

The project leverages Safari's native capabilities to deliver a robust content blocking solution, despite the platform's API limitations. Instead of compromising with a lighter version, we're creating a full-featured alternative optimized for Safari's architecture.

It will be (and is) free & open source.

@antdude
Copy link

antdude commented Oct 30, 2024

When will the first public compiled prerelease version be able for us to try?

@0xCUB3
Copy link

0xCUB3 commented Oct 30, 2024

When will the first public compiled prerelease version be able for us to try?

My version, wBlock, already has a public beta. It does not have all the features mentioned but can block scripts that serve youtube ads and more: https://github.com/0xCUB3/wBlock

@antdude
Copy link

antdude commented Oct 30, 2024

Thanks.

@gorhill
Copy link
Member

gorhill commented Jan 2, 2025

At this point it seems latest Safari 18 supports all the APIs required by uBO Lite.

I have access to an iPad, I have created an Apple developer account, a distribution certificate, and an app id. I used a workflow to convert to a safari webextensions using xcrun safari-web-extension-converter. So now I have a uBOLite.xcodeproj. From this point I need help into how I could setup a GitHub Actions workflow (which supports macos) to build, create and a limited distribution of an early beta version of uBO Lite.

I have been trying to sort out all of this on my own but I am currently stuck, I have no experience with xcodebuild-based development, and I can't pass this hurdle on my own. I welcome any help to pass this hurdle.

@arjpar
Copy link

arjpar commented Jan 2, 2025

@gorhill Have you posted the source code? I could help.


@0xCUB3 and I have created wBlock and WebShield respectively that pretty much re-creates uBlock Origin (full like Firefox) in Safari, but it works with AdGuard filter list syntax and requires "optimized" lists with fewer rules because of Safari's ~160k (iirc) rule limit for content blocking, but has no limit on the amount of scriptlets. The only thing preventing me from publishing to TestFlight is the $100/yr for an Apple Developer license.


I can help in any way I can, but seeing the source code would allow me to see if I could create the GitHub action. I do everything in Xcode, not through the CLI, but I could certainly try to make it work through the cli using xcodebuild.

@gorhill
Copy link
Member

gorhill commented Jan 2, 2025

@arjpar Thanks for offering to help.

I created a public repo which is essentially the result of running the conversion tool using latest Chromium version of uBO Lite: https://github.com/gorhill/uBOLite-safari

Eventually this will be generated using a script like it's done for Chromium and Firefox (make mv3-chromium, make mv3-firefox), but for now I want to understand the steps needed to build and publish a version once we have the source files as seen in https://github.com/gorhill/uBOLite-safari.

I have gotten to this so far:

xcodebuild build -scheme "${{ env.PROJNAME }} (iOS)" -sdk iphoneos -destination platform="platform=iOS,id=${{ secrets.DEVICE_ID }}"

Where ${{ env.PROJNAME }} is uBOLite (there was a space originally but I thought this could be a reason the build was erroring).

But I get the following errors:

/Users/runner/work/xc-experiment/xc-experiment/safari/uBOLite.xcodeproj: error: Signing for "uBOLite Extension (iOS)" requires a development team. Select a development team in the Signing & Capabilities editor. (in target 'uBOLite Extension (iOS)' from project 'uBOLite')
/Users/runner/work/xc-experiment/xc-experiment/safari/uBOLite.xcodeproj: error: Signing for "uBOLite (iOS)" requires a development team. Select a development team in the Signing & Capabilities editor. (in target 'uBOLite (iOS)' from project 'uBOLite')
Error: Process completed with exit code 65.

Note that I tried to make sense of a lot of articles about building from command line, but I just do not yet have the big picture in my head about how all this works and it just raises more questions. The first steps I did was to follow this Youtube guide on how to create a certificate etc. using openssl on the command line. This went well (I guess).

But after this, I can't find a guide with all the detailed steps to be able to build and publish from the command line, most assume you have access to Xcode on MacOS, for example https://tarikdahic.com/posts/build-ios-apps-from-the-command-line-using-xcodebuild/, which is what I do not have.

@Avakining
Copy link

Avakining commented Jan 2, 2025

But I get the following errors:

/Users/runner/work/xc-experiment/xc-experiment/safari/uBOLite.xcodeproj: error: Signing for "uBOLite Extension (iOS)" requires a development team. Select a development team in the Signing & Capabilities editor. (in target 'uBOLite Extension (iOS)' from project 'uBOLite')
/Users/runner/work/xc-experiment/xc-experiment/safari/uBOLite.xcodeproj: error: Signing for "uBOLite (iOS)" requires a development team. Select a development team in the Signing & Capabilities editor. (in target 'uBOLite (iOS)' from project 'uBOLite')
Error: Process completed with exit code 65.

Assuming you're using Xcode, you need to select the top level of the project in the sidebar, select the target, then "Signing & Capabilities", then select a team (see screenshot)

Screenshot 2025-01-02 at 11 16 20

@gorhill
Copy link
Member

gorhill commented Jan 2, 2025

Assuming you're using Xcode

That's the issue, I don't have Xcode or access to MacOS beside the macos virtual machine through GitHub Actions -- that is actually how I use Safari's webextensions converter to get the code at https://github.com/gorhill/uBOLite-safari.

I can find articles about how to build from command line, but all of these assumes you can access Xcode on MacOS so they skip the details on all the required steps from start to finish from command line.

@arjpar
Copy link

arjpar commented Jan 2, 2025

@gorhill
Since you did the app developer account, app id, distribution cert, thats good.

For iOS or Safari on iOS you’ll also need a provisioning profile. This is the .mobileprovision file you get from the Apple Developer portal IIRC. Then place it in:

~/Library/MobileDevice/Provisioning Profiles/<UUID>.mobileprovision

If you’re using iOS or Apple Silicon (macOS) distribution signing, you typically have:

  • A Development provisioning profile (for internal builds).
  • A Distribution provisioning profile (for App Store or Ad Hoc).
xcodebuild \
  -project "uBOLite.xcodeproj" \
  -scheme "uBOLite (iOS)" \
  -sdk iphoneos \
  -destination 'generic/platform=iOS' \
  -configuration Release \
  DEVELOPMENT_TEAM="YOURTEAMID" \
  CODE_SIGN_IDENTITY="Apple Development: Your Name (TEAMID)" \
  PROVISIONING_PROFILE_SPECIFIER="Your Provisioning Profile Name" \
  OTHER_CODE_SIGN_FLAGS="--verbose" \
  build

Or try using a custom xcconfig file.

DEVELOPMENT_TEAM = YOURTEAMID
CODE_SIGN_IDENTITY = Apple Distribution: Your Name (TEAMID)
PROVISIONING_PROFILE_SPECIFIER = Your-Profile-Name

And then run.

xcodebuild \
  -project "uBOLite.xcodeproj" \
  -scheme "uBOLite (iOS)" \
  -configuration Release \
  -sdk iphoneos \
  -destination 'generic/platform=iOS' \
  -xcconfig CLI-Release.xcconfig \
  build

For an .app (macOS) or test build for iOS you can do:

xcodebuild \
  -project "uBOLite.xcodeproj" \
  -scheme "uBOLite (iOS)" \
  -sdk iphoneos \
  -destination 'generic/platform=iOS' \
  -configuration Release \
  DEVELOPMENT_TEAM="YOURTEAMID" \
  CODE_SIGN_IDENTITY="Apple Development: Your Name (TEAMID)" \
  PROVISIONING_PROFILE_SPECIFIER="YourProfileName" \
  build

If you plan to distribute your build, you often do a 2-step process: archive then exportArchive.

Archive:

xcodebuild archive \
  -project "uBOLite.xcodeproj" \
  -scheme "uBOLite (iOS)" \
  -sdk iphoneos \
  -destination 'generic/platform=iOS' \
  -configuration Release \
  -archivePath "./build/uBOLite.xcarchive" \
  DEVELOPMENT_TEAM="YOURTEAMID" \
  CODE_SIGN_IDENTITY="Apple Distribution: Your Name (TEAMID)" \
  PROVISIONING_PROFILE_SPECIFIER="YourProfileName"

For ExportArchive (generates an ipa) it'll require an ExportOptions.plist, specifying method (app-store, ad-hoc, etc.) and more signing specifics. For example create a file ExportOptions.plist:

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" 
                      "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>method</key>
  <string>app-store</string>
  <key>teamID</key>
  <string>YOURTEAMID</string>
  <key>signingStyle</key>
  <string>manual</string>
  <key>provisioningProfiles</key>
  <dict>
    <key>com.your.bundle.id</key>
    <string>YourProfileName</string>
  </dict>
  <key>compileBitcode</key>
  <true/>
</dict>
</plist>

Then run:

xcodebuild -exportArchive \
  -archivePath "./build/uBOLite.xcarchive" \
  -exportPath "./build/ipa" \
  -exportOptionsPlist "./ExportOptions.plist"

Since you are working on a Safari Web Extension, you have a “container” iOS/macOS app + the Safari Web Extension target. The steps remain the same: all targets in the project must be associated with signing details. Often that means you must specify a Development Team for:

  • The container app target,
  • The extension target,
  • (Optionally) any additional frameworks if present.

If any target is missing or has incorrect signing settings you’ll get the dreaded “requires a development team” or code‐signing errors.

Apple’s toolchain (xcodebuild, codesign, security etc) is designed to support CI/CD pipelines on macOS build agents (which is exactly what GitHub Actions Mac runners are). Many large iOS/macOS apps are built and distributed daily without ever manually opening the Xcode GUI. But you need to:

  1. Have a valid Developer certificate in the Keychain.
  2. Have the provisioning profile(s) installed on that machine.
  3. Set all signing info in your command‐line invocations or .xcconfig.
  4. Archive and export if you are distributing/testflight. Or just build if you only want a local .app.

Hopefully this GitHub Actions workflow is a decent starting point:

name: Build-uBOLite
on:
  push:
    branches: [ "main" ]

jobs:
  build-ios:
    runs-on: macos-latest

    steps:
      - uses: actions/checkout@v3

      # 1. Install certs & provisioning profile
      - name: Install Certificate
        run: |
          security create-keychain -p "" build.keychain
          security import certs/YourPrivateKey.p12 -k ~/Library/Keychains/build.keychain -P "your_password" -T /usr/bin/codesign
          security import certs/YourAppleCertificate.cer -k ~/Library/Keychains/build.keychain -T /usr/bin/codesign
          # Possibly add the .mobileprovision to ~/Library/MobileDevice/Provisioning Profiles

      # 2. xcodebuild for archive
      - name: Build Archive
        run: |
          xcodebuild archive \
            -project "uBOLite.xcodeproj" \
            -scheme "uBOLite (iOS)" \
            -sdk iphoneos \
            -destination 'generic/platform=iOS' \
            -configuration Release \
            -archivePath "./build/uBOLite.xcarchive" \
            DEVELOPMENT_TEAM="YOURTEAMID" \
            CODE_SIGN_IDENTITY="Apple Distribution: Your Name (TEAMID)" \
            PROVISIONING_PROFILE_SPECIFIER="YourProfileName"

      # 3. Export .ipa
      - name: Export Archive
        run: |
          xcodebuild -exportArchive \
            -archivePath "./build/uBOLite.xcarchive" \
            -exportPath "./build/ipa" \
            -exportOptionsPlist "./ExportOptions.plist"

For all of above:

  • You must manually handle all aspects of provisioning, code signing, and project settings via command‐line arguments, environment variables, or .xcconfig overrides—these replace the “Signing & Capabilities” GUI.

I'll admit I've never built a project from the command line purely, as I've always worked on Xcode, so the above commands might not work. I also don't have an Apple developer license to test things. I just hope the above is a good starting point. If I made any mistakes, please let me know. I'd be happy to further assist when and if I can.

Relevant documentation:

@gorhill
Copy link
Member

gorhill commented Jan 3, 2025

@arjpar Thanks for all the details, I will try to do some progress on this in the coming days and will provide feedback on how it's going.

@ExtRIELICi
Copy link

ExtRIELICi commented Jan 3, 2025

and I have created wBlock and WebShield respectively that pretty much re-creates uBlock Origin (full like Firefox) in Safari, but it works with AdGuard filter list syntax and requires "optimized" lists with fewer rules because of Safari's ~160k (iirc) rule limit for content blocking, but has no limit on the amount of scriptlets. The only thing preventing me from publishing to TestFlight is the $100/yr for an Apple Developer license.

Probably a stupid question, so sorry in advance, but why can’t the original uBlock Origin be ported to Safari, requiring the use of uBlock Origin Lite, which is based on Manifest V3?

AdGuard is still based on Manifest V2 and works fine, right?

Does uBlock Origin exceed the 150,000-rule limit per extension set by Apple, while uBO Lite does not?

I’ve also noticed that AdGuard splits its extension into multiple extensions to address this issue:

image

Can't the same be done for uBO?

@arjpar
Copy link

arjpar commented Jan 3, 2025

Can't the same be done for uBO?

@ExtRIELICi No. I don't believe uBlock Origin can be ported over to Safari because it doesn't support all the required MV2 APIs. Even Chromium didn't support all the APIs that uBO used for full gains, which is why Firefox is the best option for running uBlock Origin if you want to exploit its full power. uBlock Origin uses MV2 webRequest APIs which to my knowledge Safari doesn't support, and probably won't.


AdGuard is still based on Manifest V2 and works fine, right?

No. To my knowledge they don't use WebExtensions for Safari on macOS. They convert filter list rules to Safari Content Blocking API, and parse the scriptets/extended-css into its own file, which runs separately through a javascript file. WebShield does the same thing (so far), but in a single app for macOS/iOS/iPadOS/visionOS, built in SwiftUI, using SwiftData. AdGuard for Safari (macOS) uses Electron. Perusing the repo for AdGuard for iOS looks like they use a combination of app extensions and web extensions (web extensions for sure). But in both repositories, AdGuard for Safari & iOS, the code is not as "uniform(?)" as WebShield or wBlock. Furthermore, I designed WebShield's UI to be as similar to uBlock Origin as I could, in line with Apple HIG (to my knowledge).

Converting filter list rules to Safari Content Blocker rules gives more overhead when converting, because once converted those content blocker rules are kind of "embedded" into WebKit and get run before the page is fully rendered (iirc). Basically, it's very very very performant.

IIRC (a long time ago) I once did some perf testing, and I think the Content Blockers used like 0.1% of CPU usage, probably even less. Wipr 2 uses content blocking and it uses 0% of CPU or GPU and ~3.3MB of memory.

The main performance issues comes from scriptlets/extended-css (being run through js) but AdGuard has the same problem, and so does uBO, since they have to run scriptlets through js.


The difference between uBlock Origin Lite (for Safari) and WebShield, is that WebShield allows you to do many things that you can do in uBlock Origin, like importing custom filter lists, being able to manually toggle exactly which default lists (not just one huge "ads, trackers, miners, etc" list) you want on or off. You can view the information for a filter list, when it was last updated, its homepage, view the source of the filter list, like in uBO. I'll also be adding a content blocker for each category of filter list (ads, privacy, etc) so each category will have the ~160k rule limit for content blocking. But this rule limit only applies to content blockers. Scriptlets have no limit to my knowledge, and that's where most of the power of ad blockers come from. It's what blocks your YouTube ads.

In other words, uBO Lite uses WebExtensions MV3 DNR APIs, whereas WebShield uses Safari App Extensions & Content Blocking API (while running scriptlets in js like uBO to my knowledge).

Literally the only thing preventing me from releasing WebShield to TestFlight is the $100/yr developer fee, which I think is crazy because Firefox nor Chrome require you to pay $100/yr to post an extension on their stores. I understand Apple uses App Store for distribution and they're very paranoid about security and privacy, but at least allow users to sideload.


  • Safari's Content Blocking API is declarative so you specify what to block rather than how to block it
  • Rules are compiled into a native, optimized format (I believe?) for very fast matching
  • Network requests are intercepted early (allegedly before page rendering) reducing latency
  • Declarative rules eliminate the need for runtime decision-making, minimizing CPU and memory overhead. (Not sure about how DNR specifically works, but I can assure you that Content Blockers are more performant, though I'm not sure about if it's more powerful, but Scriptlets are for power)
  • Rules run within Safari/WebKit which avoids exposure to third-party scripts or processing

Relevant documentation: https://developer.apple.com/documentation/safariservices/creating-a-content-blocker

@ExtRIELICi
Copy link

Thank you for your detailed and excellent reply. I truly appreciate it! @arjpar

@stevenya97
Copy link

The main performance issues comes from scriptlets/extended-css (being run through js) but AdGuard has the same problem, and so does uBO, since they have to run scriptlets through js.

@arjpar This might interest you, seems AdGuard is trying to address the overhead from scriptlets during the conversion phase before it hits the Content Blocker loader phase. Need to peruse the repo myself to see exactly what optimizations they did but the demo seems legit and testing the new update rule conversion is much quicker now, basically instant.

https://github.com/AdguardTeam/AdguardForiOS/releases/tag/v4.5.8.1026
https://adguard.com/en/blog/adguard-v4-5-8-for-ios.html

@arjpar
Copy link

arjpar commented Jan 4, 2025

@stevenya97 Thanks for this! This is the first time I heard of this, I should probably read AdGuard's blog. I'll try implementing this soon, and test it on my end. Currently I'm converting everything asynchronously and storing it using SwiftData, but there are probably some optimizations I can do/better ways of doing it, and clearly can and should do given that AdGuard has found a solution to "nearly instantly" convert filter lists.

Edit: I don't have to do anything on my end because it's all in SafariConverterLib, which I updated. Now I'm able to download & convert 119k rules in ~8 seconds (M1 Pro).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests