Using CFBundleDocumentTypes breaks Spyder application

[mrclary]

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

When using CFBundleDocumentTypes for the osx platform for Spyder, the info.plist file is created correctly, but the application bundle does not launch. The resulting bundle structure is as follows:

/Users/rclary/Applications/Spyder 6.app
└── Contents
    ├── Info.plist  # This file correctly has CFBundleDocumentTypes
    ├── MacOS
    │   ├── python -> /Users/rclary/Library/spyder-6/envs/spyder-runtime/bin/python
    │   └── spyder-6
    ├── PkgInfo
    └── Resources
        ├── Spyder 6.app
        │   └── Contents
        │       ├── Info.plist  # This file does not have CFBundleDocumentTypes
        │       ├── MacOS
        │       │   ├── spyder-6
        │       │   └── spyder-6-script
        │       ├── PkgInfo
        │       └── Resources
        │           └── spyder.icns
        └── spyder.icns

For Spyder, I don't believe that the sub application in Resources is necessary. If I omit CFBundleDocumentTypes from the json file, but later manually update the info.plist file, then Spyder launches and behaves as expected.

spyder-menu.json

{
    "$schema": "https://json-schema.org/draft-07/schema",
    "$id": "https://schemas.conda.io/menuinst-1.schema.json",
    "menu_name": "{{ DISTRIBUTION_NAME }} spyder",
    "menu_items": [
        {
            "name": "Spyder __PKG_MAJOR_VER__ ({{ ENV_NAME }})",
            "description": "Scientific PYthon Development EnviRonment",
            "icon": "{{ MENU_DIR }}/spyder.{{ ICON_EXT }}",
            "activate": false,
            "terminal": false,
            "command": [""],
            "platforms": {
                "win": {
                    "desktop": true,
                    "app_user_model_id": "spyder.Spyder",
                    "command": ["{{ PREFIX }}/pythonw.exe", "{{ PREFIX }}/Scripts/spyder-script.py"]
                },
                "linux": {
                    "Categories": [
                        "Development",
                        "Science"
                    ],
                    "command": ["{{ PREFIX }}/bin/spyder", "$@"],
                    "StartupWMClass": "Spyder"
                },
                "osx": {
                    "precommand": "pushd \"$(dirname \"$0\")\" &>/dev/null",
                    "command": ["./python", "{{ PREFIX }}/bin/spyder", "$@"],
                    "link_in_bundle": {
                        "{{ PREFIX }}/bin/python": "{{ MENU_ITEM_LOCATION }}/Contents/MacOS/python"
                    },
                    "CFBundleName": "Spyder __PKG_MAJOR_VER__",
                    "CFBundleIdentifier": "org.spyder-ide.Spyder",
                    "CFBundleVersion": "__PKG_VERSION__",
                    "CFBundleDocumentTypes": [
                        {
                            "CFBundleTypeName": "text document",
                            "CFBundleTypeRole": "Editor",
                            "LSHandlerRank": "Default",
                            "CFBundleTypeIconFile": "spyder.icns",
                            "LSItemContentTypes": [
                                "com.apple.applescript.text",
                                "com.apple.ascii-property-list",
                                "com.apple.audio-unit-preset",
                                "com.apple.binary-property-list",
                                "com.apple.configprofile",
                                "com.apple.crashreport",
                                "com.apple.dashcode.css",
                                "com.apple.dashcode.javascript",
                                "com.apple.dashcode.json",
                                "com.apple.dashcode.manifest",
                                "com.apple.dt.document.ascii-property-list",
                                "com.apple.dt.document.script-suite-property-list",
                                "com.apple.dt.document.script-terminology-property-list",
                                "com.apple.property-list",
                                "com.apple.rez-source",
                                "com.apple.scripting-definition",
                                "com.apple.structured-text",
                                "com.apple.traditional-mac-plain-text",
                                "com.apple.xcode.ada-source",
                                "com.apple.xcode.apinotes",
                                "com.apple.xcode.bash-script",
                                "com.apple.xcode.configsettings",
                                "com.apple.xcode.csh-script",
                                "com.apple.xcode.entitlements-property-list",
                                "com.apple.xcode.fortran-source",
                                "com.apple.xcode.glsl-source",
                                "com.apple.xcode.ksh-script",
                                "com.apple.xcode.lex-source",
                                "com.apple.xcode.make-script",
                                "com.apple.xcode.mig-source",
                                "com.apple.xcode.pascal-source",
                                "com.apple.xcode.strings-text",
                                "com.apple.xcode.tcsh-script",
                                "com.apple.xcode.yacc-source",
                                "com.apple.xcode.zsh-script",
                                "com.apple.xml-property-list",
                                "com.netscape.javascript-source",
                                "com.scenarist.closed-caption",
                                "com.sun.java-source",
                                "com.sun.java-web-start",
                                "net.daringfireball.markdown",
                                "org.khronos.glsl-source",
                                "org.oasis-open.xliff",
                                "public.ada-source",
                                "public.assembly-source",
                                "public.bash-script",
                                "public.c-header",
                                "public.c-plus-plus-header",
                                "public.c-plus-plus-source",
                                "public.c-source",
                                "public.case-insensitive-text",
                                "public.comma-separated-values-text",
                                "public.csh-script",
                                "public.css",
                                "public.delimited-values-text",
                                "public.dylan-source",
                                "public.filename-extension",
                                "public.fortran-77-source",
                                "public.fortran-90-source",
                                "public.fortran-95-source",
                                "public.fortran-source",
                                "public.html",
                                "public.json",
                                "public.ksh-script",
                                "public.lex-source",
                                "public.log",
                                "public.m3u-playlist",
                                "public.make-source",
                                "public.mig-source",
                                "public.mime-type",
                                "public.module-map",
                                "public.nasm-assembly-source",
                                "public.objective-c-plus-plus-source",
                                "public.objective-c-source",
                                "public.opencl-source",
                                "public.pascal-source",
                                "public.patch-file",
                                "public.perl-script",
                                "public.php-script",
                                "public.plain-text",
                                "public.python-script",
                                "public.rss",
                                "public.ruby-script",
                                "public.script",
                                "public.shell-script",
                                "public.source-code",
                                "public.tcsh-script",
                                "public.text",
                                "public.utf16-external-plain-text",
                                "public.utf16-plain-text",
                                "public.utf8-plain-text",
                                "public.utf8-tab-separated-values-text",
                                "public.xhtml",
                                "public.xml",
                                "public.yacc-source",
                                "public.yaml",
                                "public.zsh-script"
                            ]
                        }
                    ]
                }
            }
        }
    ]
}

Conda Info

mamba version : 1.5.6
     active environment : base
    active env location : /Users/rclary/Library/spyder-6
            shell level : 1
       user config file : /Users/rclary/.condarc
 populated config files : /Users/rclary/Library/spyder-6/.condarc
                          /Users/rclary/.condarc
          conda version : 23.11.0
    conda-build version : not installed
         python version : 3.10.13.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=skylake
                          __conda=23.11.0=0
                          __osx=14.3=0
                          __unix=0=0
       base environment : /Users/rclary/Library/spyder-6  (writable)
      conda av data dir : /Users/rclary/Library/spyder-6/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/label/spyder_kernels_rc/osx-64
                          https://conda.anaconda.org/conda-forge/label/spyder_kernels_rc/noarch
                          https://conda.anaconda.org/conda-forge/label/spyder_dev/osx-64
                          https://conda.anaconda.org/conda-forge/label/spyder_dev/noarch
                          https://conda.anaconda.org/conda-forge/osx-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /Users/rclary/Library/spyder-6/pkgs
                          /Users/rclary/.conda/pkgs
       envs directories : /Users/rclary/.conda/envs
                          /Users/rclary/Library/spyder-6/envs
               platform : osx-64
             user-agent : conda/23.11.0 requests/2.31.0 CPython/3.10.13 Darwin/23.3.0 OSX/14.3 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.6
                UID:GID : 784632031:1061179201
             netrc file : None
           offline mode : False

Conda Config

==> /Users/rclary/Library/spyder-6/.condarc <==
auto_update_conda: False
notify_outdated_conda: False
env_prompt: [spyder]({default_env}) 
channel_priority: flexible
channels:
  - conda-forge/label/spyder_kernels_rc
  - conda-forge/label/spyder_dev
  - conda-forge
repodata_fns:
  - repodata.json

==> /Users/rclary/.condarc <==
auto_activate_base: False
envs_dirs:
  - ~/.conda/envs
channel_priority: flexible
channels:
  - conda-forge/label/spyder_kernels_rc
  - conda-forge
  - defaults
show_channel_urls: True
conda-build:
  root-dir: ~/.conda/conda-bld

Conda list

# packages in environment at /Users/rclary/Library/spyder-6:
#
# Name                    Version                   Build  Channel
archspec                  0.2.2              pyhd8ed1ab_0    conda-forge
boltons                   23.1.1             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py310h9e9d8ca_1    conda-forge
bzip2                     1.0.8                h10d778d_5    conda-forge
c-ares                    1.25.0               h10d778d_0    conda-forge
ca-certificates           2023.11.17           h8857fd0_0    conda-forge
certifi                   2023.11.17         pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py310hdca579f_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     23.11.0         py310h2ec42d9_1    conda-forge
conda-libmamba-solver     23.12.0            pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
fmt                       10.1.1               h7728843_1    conda-forge
icu                       73.2                 hf5e326d_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               2.4             py310h2ec42d9_3    conda-forge
krb5                      1.21.2               hb884880_0    conda-forge
libarchive                3.7.2                hd35d340_1    conda-forge
libcurl                   8.5.0                h726d00d_0    conda-forge
libcxx                    16.0.6               hd57cbcb_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 h10d778d_2    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libiconv                  1.17                 hd75f5a5_2    conda-forge
libmamba                  1.5.6                ha449628_0    conda-forge
libmambapy                1.5.6           py310hd168405_0    conda-forge
libnghttp2                1.58.0               h64cf6d3_1    conda-forge
libsolv                   0.7.27               hf4d7fad_0    conda-forge
libsqlite                 3.44.2               h92b6c6a_0    conda-forge
libssh2                   1.11.0               hd019ec5_0    conda-forge
libxml2                   2.12.4               hc0ae0f7_1    conda-forge
libzlib                   1.2.13               h8a1eda9_5    conda-forge
lz4-c                     1.9.4                hf0c8a7f_0    conda-forge
lzo                       2.10              haf1e3a3_1000    conda-forge
mamba                     1.5.6           py310h6bde348_0    conda-forge
menuinst                  2.0.2           py310h2ec42d9_0    conda-forge
ncurses                   6.4                  h93d8f39_2    conda-forge
openssl                   3.2.0                hd75f5a5_1    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pip                       23.3.2             pyhd8ed1ab_0    conda-forge
platformdirs              4.1.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py310h6729b98_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.10.13         h00d2728_1_cpython    conda-forge
python_abi                3.10                    4_cp310    conda-forge
readline                  8.2                  h9e318b2_1    conda-forge
reproc                    14.2.4.post0         h10d778d_1    conda-forge
reproc-cpp                14.2.4.post0         h93d8f39_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
ruamel.yaml               0.18.5          py310hb372a2b_0    conda-forge
ruamel.yaml.clib          0.2.7           py310h6729b98_2    conda-forge
setuptools                69.0.3             pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h1abcd95_1    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
truststore                0.8.0              pyhd8ed1ab_0    conda-forge
tzdata                    2023d                h0c530f3_0    conda-forge
urllib3                   2.1.0              pyhd8ed1ab_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h775f41a_0    conda-forge
yaml-cpp                  0.8.0                he965462_0    conda-forge
zstandard                 0.22.0          py310hd88f66e_0    conda-forge
zstd                      1.5.5                h829000d_0    conda-forge

Additional Context

No response

[jaimergp]

macOS file type association is a bit special. When you launch an application through an associated document, the OS triggers an event which the registered app must be able to listen to. In contrast, in Linux and Windows, the kernel simply launches {program} %s through the command line arguments.

What we have done here is to implement a simple universal listener that will take the OS event and pass it to the handler via the designated command. In other words, you need event_handler. I'm now realizing that the docs are missing this detail in the example. The tests showcase this a bit better.

[mrclary]

macOS file type association is a bit special. When you launch an application through an associated document, the OS triggers an event which the registered app must be able to listen to. In contrast, in Linux and Windows, the kernel simply launches {program} %s through the command line arguments.

What we have done here is to implement a simple universal listener that will take the OS event and pass it to the handler via the designated command. In other words, you need event_handler. I'm now realizing that the docs are missing this detail in the example. The tests showcase this a bit better.

I still don't understand why this is necessary. Following is a screencast illustrating that file associations work just fine with only the specifications in info.plist. Is there something else that I'm missing here? What should be specified in the event_handler? Just the same call to the application used for command? Will this still result in a nested application bundle? Nevertheless, this breaks Spyder without even trying to open files.

This shows

• 2 plist files: one with and one without CFBundleDocumentTypes
• Without CFBundleDocumentTypes, Spyder is not a recommended application for Python files
• With CFBundleDocumentTypes, Spyder is a recommended application for Python files
• After setting Spyder as the default application, a Python file can be opened from the Finder with the expected behavior if Spyder is closed or already open.
• Note that moving the application from, and back to, the Applications folder is necessary in order to trigger macOS to re-evaluate the new info.plist for Launch Services. This is only necessary for this demonstration because the info.plist file is changed. Under normal circumstances, the info.plist will be evaluated when the bundle is created in the Applications folder.

[jaimergp]

Hm, when I was researching this I couldn't get it to work. Then @aganders3 gave us a hand with the listener/launcher. Is Spyder registering a special listening method in Qt? What version of macOS is this? Which Qt?

I think we can change the logic to inject the launcher (e.g. only when event_handler is defined) if it works fine in some contexts.

[aganders3]

Hm, I'm super interested. I thought maybe this could be done with Qt and/or pyobjc but I couldn't figure it out at the time. It would be great to not need this hack.

Edit: just peeked at the Spyder source. Maybe it's because Spyder uses itself as both the listener and handler?

Ah no there's some more customization here I need to understand

https://github.com/spyder-ide/spyder/blob/582251bc6930ea37fc8b060d51775167bc486a2f/spyder/utils/qthelpers.py#L749

[aganders3]

I think we can change the logic to inject the launcher (e.g. only when event_handler is defined) if it works fine in some contexts.

This is probably a reasonable way to do it, and would just need to document its then on the app itself to listen for the macOS events.

[jaimergp]

Yea, it looks like you are using applelaunchservices to handle this via Qt.

[mrclary]

Yea, it looks like you are using applelaunchservices to handle this via Qt.

applaunchservices is not actually being used in this conda-based implementation; we explicitly do not import it if Spyder is in our constructor-made environment. So the screencast I showed above does not use applaunchservices. See here. We also bypass its import for our 5.x branch standalone application built with py2app.

applaunchservices was developed by one of Spyder's core developers long before I started working on standalone applications for it. It was intended to do just what you think it does, but for Spyder launched from command-line without an app bundle.

What I think may actually be going on is that this issue was resolved when we added the Python symbolic link to the app bundle, i.e. link_in_bundle. (@jaimergp, I can't locate where we had that conversation in order to link back to it...) I think without the Python symbolic link, macOS sends the events to Python and not to the intended application (Spyder); hence the need for applaunchservices or an event listener. With the Python symbolic link (and perhaps the executable stub?), macOS sends the events to the intended application and applaunchservices or an event listener are not needed.

[mrclary]

Hm, I'm super interested. I thought maybe this could be done with Qt and/or pyobjc but I couldn't figure it out at the time. It would be great to not need this hack.

Agreed.

Edit: just peeked at the Spyder source. Maybe it's because Spyder uses itself as both the listener and handler?

I don't believe so. See my previous comments about applaunchservices.

Ah no there's some more customization here I need to understand

https://github.com/spyder-ide/spyder/blob/582251bc6930ea37fc8b060d51775167bc486a2f/spyder/utils/qthelpers.py#L749

There is nothing magical going on here. We do handle FileOpen events etc., but these are just the events that Qt receives. macOS must still send those events to Spyder before Qt can pick it up. If there is anything special that Qt is doing, then it does it regardless of whether Spyder is launched via app bundle or command-line and so I don't think is relevant to this discussion.

[aganders3]

Thanks for the extra context. I'm not very familiar with Spyder but will take a closer look. I'm happy to arrange some time with either/both of you to chat live as well.

[aganders3]

Okay I think I understand a bit more now. Qt is definitely doing whatever needs to be done here, and the only thing Spyder is doing is subclassing QApplication to handle the QFileOpenEvent (which Qt provides based on the OS event). This is great and I think will work for napari! It's just embarrassing that I missed it before and clearly overthought the problem 🤦.

Still I think the appkit-launcher we have may be useful for certain circumstances. Basically it would let you write a Python app without Qt and without importing any pyobjc stuff. I'm not sure how common that might be. I'm even less familiar with other GUI toolkits so not sure if they can similarly translate the macOS file open events. If it's not useful we can just toss that separate launcher and the corresponding logic for the nested app bundle.

Anyway for now let's try #183 (though I haven't looked closely at it) to see if that at least gets Spyder working.

[jaimergp]

The PR is ready for review. Let me know if it works or if the docs are clear! Thanks 🙏