PEP: 730 Title: Adding iOS as a supported platform Author: Russell Keith-Magee Sponsor: Ned Deily Discussions-To: https://discuss.python.org/t/pep730-adding-ios-as-a-supported-platform/35854 Status: Final Type: Standards Track Content-Type: text/x-rst Created: 09-Oct-2023 Python-Version: 3.13 Resolution: https://discuss.python.org/t/pep-730-adding-ios-as-a-supported-platform/35854/66 Abstract ======== This PEP proposes adding iOS as a supported platform in CPython. The initial goal is to achieve Tier 3 support for Python 3.13. This PEP describes the technical aspects of the changes that are required to support iOS. It also describes the project management concerns related to adoption of iOS as a Tier 3 platform. Motivation ========== Over the last 15 years, mobile platforms have become increasingly important parts of the computing landscape. iOS is one of two operating systems that control the vast majority of these devices. However, there is no official support for iOS in CPython. The `BeeWare Project `__ and `Kivy `__ have both supported iOS for almost 10 years. This support has been able to generate applications that have been accepted for publication in the iOS App Store. This demonstrates the technical feasibility of iOS support. It is important for the future of Python as a language that it is able to be used on any hardware or OS that has widespread adoption. If Python cannot be used a on a platform that has widespread use, adoption of the language will be impacted as potential users will adopt other languages that *do* provide support for these platforms. Rationale ========= Development landscape --------------------- iOS provides a single API, but 2 distinct ABIs - ``iphoneos`` (physical devices), and ``iphonesimulator``. Each of these ABIs can be provided on multiple CPU architectures. At time of writing, Apple officially supports ``arm64`` on the device ABI, and ``arm64`` and ``x86_64`` are supported on the simulator ABI. As with macOS, iOS supports the creation of "fat" binaries that contains multiple CPU architectures. However, fat binaries *cannot* span ABIs. That is, it is possible to have a fat *simulator* binary, and a fat *device* binary, but it is not possible to create a single fat "iOS" binary that covers both simulator and device needs. To support distribution of a single development artefact, Apple uses an "XCframework" structure - a wrapper around multiple ABIs that implement a common API. iOS runs on a Darwin kernel, similar to macOS. However, there is a need to differentiate between macOS and iOS at an implementation level, as there are significant platform differences between iOS and macOS. iOS code is compiled for compatibility against a minimum iOS version. Apple frequently refers to "iPadOS" in their marketing material. However, from a development perspective, there is no discernable difference between iPadOS and iOS. A binary that has been compiled for the ``iphoneos`` or ``iphonesimulator`` ABIs can be deployed on iPad. Other Apple platforms, such as tvOS, watchOS, and visionOS, use different ABIs, and are not covered by this PEP. POSIX compliance ---------------- iOS is broadly a POSIX platform. However, similar to WASI/Emscripten, there are POSIX APIs that exist on iOS, but cannot be used; and POSIX APIs that don't exist at all. Most notable of these is the fact that iOS does not provide any form of multiprocess support. ``fork`` and ``spawn`` both *exist* in the iOS API; however, if they are invoked, the invoking iOS process stops, and the new process doesn't start. Unlike WASI/Emscripten, threading *is* supported on iOS. There are also significant limits to socket handling. Due to process sandboxing, there is no availability of interprocess communication via socket. However, sockets for network communication *are* available. Dynamic libraries ----------------- The iOS `App Store guidelines `__ allow apps to be written in languages other than Objective C or Swift. However, they have very strict guidelines about the structure of apps that are submitted for distribution. iOS apps can use dynamically loaded libraries; however, there are very strict requirements on how dynamically loaded content is packaged for use on iOS: * Dynamic binary content must be compiled as dynamic libraries, not shared objects or binary bundles. * They must be packaged in the app bundle as Frameworks. * Each Framework can only contain a single dynamic library. * The Framework *must* be contained in the iOS App's ``Frameworks`` folder. * A Framework may not contain any non-library content. This imposes some constraints on the operation of CPython. It is not possible store binary modules in the ``lib-dynload`` and/or ``site-packages`` folders; they must be stored in the app's Frameworks folder, with each module wrapped in a Framework. This also means that the common assumption that a Python module can construct the location of a binary module by using the ``__file__`` attribute of the Python module no longer holds. As with macOS, compiling a binary module that is accessible from a statically-linked build of Python requires the use of the ``--undefined dynamic_lookup`` option to avoid linking ``libpython3.x`` into every binary module. However, on iOS, this compiler flag raises a deprecation warning when it is used. A warning from this flag has been observed on macOS as well - however, responses from Apple staff suggest that they `do not intend to break the CPython ecosystem by removing this option `__. As Python does not currently have a notable presence on iOS, it is difficult to judge whether iOS usage of this flag would fall under the same umbrella. Console and interactive usage ----------------------------- Distribution of a traditional CPython REPL or interactive "python.exe" should not be considered a goal of this work. Mobile devices (including iOS) do not provide a TTY-style console. They do not provide ``stdin``, ``stdout`` or ``stderr``. iOS provides a system log, and it is possible to install a redirection so that all ``stdout`` and ``stderr`` content is redirected to the system log; but there is no analog for ``stdin``. In addition, iOS places restrictions on downloading additional code at runtime (as this behavior would be functionally indistinguishable from trying to work around App Store review). As a result, a traditional "create a virtual environment and pip install" development experience will not be viable on iOS. It is *possible* to build an native iOS application that provides a REPL interface. This would be closer to an IDLE-style user experience; however, Tkinter cannot be used on iOS, so any app would require a ground-up rewrite. The iOS app store already contains several examples of apps in this category (e.g., `Pythonista `__ and `Pyto `__). The focus of this work would be to provide an embedded distribution that IDE-style native interfaces could utilize, not a user-facing "app" interface to iOS on Python. Specification ============= Platform identification ----------------------- ``sys`` ''''''' ``sys.platform`` will identify as ``"ios"`` on both simulator and physical devices. ``sys.implementation._multiarch`` will describe the ABI and CPU architecture: * ``"arm64-iphoneos"`` for ARM64 devices * ``"arm64-iphonesimulator"`` for ARM64 simulators * ``"x86_64-iphonesimulator"`` for x86_64 simulators ``platform`` '''''''''''' ``platform`` will be modified to support returning iOS-specific details. Most of the values returned by the ``platform`` module will match those returned by ``os.uname()``, with the exception of: * ``platform.system()`` - ``"iOS"`` or ``iPadOS`` (depending on the hardware in use), instead of ``"Darwin"`` * ``platform.release()`` - the iOS version number, as a string (e.g., ``"16.6.1"``), instead of the Darwin kernel version. In addition, a ``platform.ios_ver()`` method will be added. This mirrors ``platform.mac_ver()``, which can be used to provide macOS version information. ``ios_ver()`` will return a namedtuple that contains the following: * ``system`` - the OS name (``iOS`` or ``iPadOS``, depending on hardware) * ``release`` - the iOS version, as a string (e.g., ``"16.6.1"``). * ``model`` - the model identifier of the device, as a string (e.g., ``"iPhone13,2"``). On simulators, this will return ``"iPhone"`` or ``"iPad"``, depending on the simulator device. * ``is_simulator`` - a boolean indicating if the device is a simulator. ``os`` '''''' ``os.uname()`` will return the raw result of a POSIX ``uname()`` call. This will result in the following values: * ``sysname`` - ``"Darwin"`` * ``release`` - The Darwin kernel version (e.g., ``"22.6.0"``) This approach treats the ``os`` module as a "raw" interface to system APIs, and ``platform`` as a higher-level API providing more generally useful values. ``sysconfig`` ''''''''''''' The ``sysconfig`` module will use the minimum iOS version as part of ``sysconfig.get_platform()`` (e.g., ``"ios-12.0-arm64-iphoneos"``). The ``sysconfigdata_name`` and Config makefile will follow the same patterns as existing platforms (using ``sys.platform``, ``sys.implementation._multiarch`` etc.) to construct identifiers. Subprocess support ------------------ iOS will leverage the pattern for disabling subprocesses established by WASI/Emscripten. The ``subprocess`` module will raise an exception if an attempt is made to start a subprocess, and ``os.fork`` and ``os.spawn`` calls will raise an ``OSError``. Dynamic module loading ---------------------- To accommodate iOS dynamic loading, the ``importlib`` bootstrap will be extended to add a metapath finder that can convert a request for a Python binary module into a Framework location. This finder will only be installed if ``sys.platform == "ios"``. This finder will convert a Python module name (e.g., ``foo.bar._whiz``) into a unique Framework name by using the full module name as the framework name (i.e., ``foo.bar._whiz.framework``). A framework is a directory; the finder will look for a binary named ``foo.bar._whiz`` in that directory. Compilation ----------- The only binary format that will be supported is a dynamically-linkable ``libpython3.x.dylib``, packaged in an iOS-compatible framework format. While the ``--undefined dynamic_lookup`` compiler option currently works, the long-term viability of the option cannot be guaranteed. Rather than rely on a compiler flag with an uncertain future, binary modules on iOS will be linked with ``libpython3.x.dylib``. This means iOS binary modules will not be loadable by an executable that has been statically linked against ``libpython3.x.a``. Therefore, a static ``libpython3.x.a`` iOS library will not be supported. This is the same pattern used by CPython on Windows. Building CPython for iOS requires the use of the cross-platform tooling in CPython's ``configure`` build system. A single ``configure``/``make``/``make install`` pass will produce a ``Python.framework`` artefact that can be used on a single ABI and architecture. Additional tooling will be required to merge the ``Python.framework`` builds for multiple architectures into a single "fat" library. Tooling will also be required to merge multiple ABIs into the ``XCframework`` format that Apple uses to distribute multiple frameworks for different ABIs in a single bundle. An Xcode project will be provided for the purpose of running the CPython test suite. Tooling will be provided to automate the process of compiling the test suite binary, start the simulator, install the test suite, and execute it. Distribution ------------ Adding iOS as a Tier 3 platform only requires adding support for compiling an iOS-compatible build from an unpatched CPython code checkout. It does not require production of officially distributed iOS artefacts for use by end-users. If/when iOS is updated to Tier 2 or 1 support, the tooling used to generate an ``XCframework`` package could be used to produce an iOS distribution artefact. This could then be distributed as an "embedded distribution" analogous to the Windows embedded distribution, or as a CocoaPod or Swift package that could be added to an Xcode project. CI resources ------------ `Anaconda `__ has offered to provide physical hardware to run iOS buildbots. GitHub Actions is able to host iOS simulators on their macOS machines, and the iOS simulator can be controlled by scripting environments. The free tier currently only provides x86_64 macOS machines; however ARM64 runners `recently became available on paid plans `__. However, in order to avoid exhausting macOS runner resources, a GitHub Actions run for iOS will not be added as part of the standard CI configuration. Packaging --------- iOS will not provide a "universal" wheel format. Instead, wheels will be provided for each ABI-arch combination. iOS wheels will use tags: * ``ios_12_0_arm64_iphoneos`` * ``ios_12_0_arm64_iphonesimulator`` * ``ios_12_0_x86_64_iphonesimulator`` In these tags, "12.0" is the minimum supported iOS version. As with macOS, the tag will incorporate the minimum iOS version that is selected when the wheel is compiled; a wheel compiled with a minimum iOS version of 15.0 would use the ``ios_15_0_*`` tags. At time of writing, iOS 12.0 exposes most significant iOS features, while reaching near 100% of devices; this will be used as a floor for iOS version matching. These wheels can include binary modules in-situ (i.e., co-located with the Python source, in the same way as wheels for a desktop platform); however, they will need to be post-processed as binary modules need to be moved into the "Frameworks" location for distribution. This can be automated with an Xcode build step. PEP 11 Update ------------- :pep:`11` will be updated to include two of the iOS ABIs: * ``arm64-apple-ios`` * ``arm64-apple-ios-simulator`` Ned Deily will serve as the initial core team contact for these ABIs. The ``x86_64-apple-ios-simulator`` target will be supported on a best-effort basis, but will not be targeted for tier 3 support. This is due to the impending deprecation of x86_64 as a simulation platform, combined with the difficulty of commissioning x86_64 macOS hardware at this time. Backwards Compatibility ======================= Adding a new platform does not introduce any backwards compatibility concerns to CPython itself. There may be some backwards compatibility implications on the projects that have historically provided CPython support (i.e., BeeWare and Kivy) if the final form of any CPython patches don't align with the patches they have historically used. Although not strictly a backwards compatibility issue, there *is* a platform adoption consideration. Although CPython itself may support iOS, if it is unclear how to produce iOS-compatible wheels, and prominent libraries like cryptography, Pillow, and NumPy don't provide iOS wheels, the ability of the community to adopt Python on iOS will be limited. Therefore, it will be necessary to clearly document how projects can add iOS builds to their CI and release tooling. Adding iOS support to tools like `crossenv `__ and `cibuildwheel `__ may be one way to achieve this. Security Implications ===================== Adding iOS as a new platform does not add any security implications. How to Teach This ================= The education needs related to this PEP mostly relate to how end-users can add iOS support to their own Xcode projects. This can be accomplished with documentation and tutorials on that process. The need for this documentation will increase if/when support raises from Tier 3 to Tier 2 or 1; however, this transition should also be accompanied with simplified deployment artefacts (such as a Cocoapod or Swift package) that are integrated with Xcode development. Reference Implementation ======================== The BeeWare `Python-Apple-support `__ repository contains a reference patch and build tooling to compile a distributable artefact. `Briefcase `__ provides a reference implementation of code to execute test suites on iOS simulators. The `Toga Testbed `__ is an example of a test suite that is executed on the iOS simulator using GitHub Actions. Rejected Ideas ============== Simulator identification ------------------------ Earlier versions of this PEP suggested the inclusion of ``sys.implementation._simulator`` attribute to identify when code is running on device, or on a simulator. This was rejected due to the use of a protected name for a public API, plus the pollution of the ``sys`` namespace with an iOS-specific detail. Another proposal during discussion was to include a generic ``platform.is_emulator()`` API that could be implemented by any platform - for example to differentiate running on x86_64 code on ARM64 hardware, or when running in QEMU or other virtualization methods. This was rejected on the basis that it wasn't clear what a consistent interpretation of "emulator" would be, or how an emulator would be detected outside of the iOS case. The decision was made to keep this detail iOS-specific, and include it on the ``platform.ios_ver()`` API. GNU compiler triples -------------------- ``autoconf`` requires the use of a GNU compiler triple to identify build and host platforms. However, the ``autoconf`` toolchain doesn't provide native support for iOS simulators, so we are left with the task of working out how to squeeze iOS hardware into GNU's naming regimen. This can be done (with some patching of ``config.sub``), but it leads to 2 major sources of naming inconsistency: * ``arm64`` vs ``aarch64`` as an identifier of 64-bit ARM hardware; and * What identifier is used to represent simulators. Apple's own tools use ``arm64`` as the architecture, but appear to be tolerant of ``aarch64`` in some cases. The device platform is identified as ``iphoneos`` and ``iphonesimulator``. Rust toolchains uses ``aarch64`` as the architecture, and use ``aarch64-apple-ios`` and ``aarch64-apple-ios-sim`` to identify the device platform; however, they use ``x86_64-apple-ios`` to represent iOS *simulators* on x86_64 hardware. The decision was made to use ``arm64-apple-ios`` and ``arm64-apple-ios-simulator`` because: 1. The ``autoconf`` toolchain already contains support for ``ios`` as a platform in ``config.sub``; it's only the simulator that doesn't have a representation. 2. The third part of the host triple is used as ``sys.platform``. 3. When Apple's own tools reference CPU architecture, they use ``arm64``, and the GNU tooling usage of the architecture isn't visible outside the build process. 4. When Apple's own tools reference simulator status independent of the OS (e.g., in the naming of Swift submodules), they use a ``-simulator`` suffix. 5. While *some* iOS packages will use Rust, *all* iOS packages will use Apple's tooling. The initially accepted version of this document used the ``aarch64`` form as the PEP 11 identifier; this was corrected during finalization. "Universal" wheel format ------------------------ macOS currently supports 2 CPU architectures. To aid the end-user development experience, Python defines a "universal2" wheel format that incorporates both x86_64 and ARM64 binaries. It would be conceptually possible to offer an analogous "universal" iOS wheel format. However, this PEP does not use this approach, for 2 reasons. Firstly, the experience on macOS, especially in the numerical Python ecosystem, has been that universal wheels can be exceedingly difficult to accommodate. While native macOS libraries maintain strong multi-platform support, and Python itself has been updated, the vast majority of upstream non-Python libraries do not provide multi-architecture build support. As a result, compiling universal wheels inevitably requires multiple compilation passes, and complex decisions over how to distribute header files for different architectures. As a result of this complexity, many popular projects (including NumPy and Pillow) do not provide universal wheels at all, instead providing separate ARM64 and x86_64 wheels. Secondly, historical experience is that iOS would require a much more fluid "universal" definition. In the last 10 years, there have been *at least* 5 different possible interpretations of "universal" that would apply to iOS, including various combinations of armv6, armv7, armv7s, arm64, x86 and x86_64 architectures, on device and simulator. If defined right now, "universal-iOS" would likely include x86_64 and arm64 on simulator, and arm64 on device; however, the pending deprecation of x86_64 hardware would add another interpretation; and there may be a need to add arm64e as a new device architecture in the future. Specifying iOS wheels as single-platform-only means the Python core team can avoid an ongoing standardization discussion about the updated "universal" formats. It also means wheel publishers are able to make per-project decisions over which platforms are feasible to support. For example, a project may choose to drop x86_64 support, or adopt a new architecture earlier than other parts of the Python ecosystem. Using platform-specific wheels means this decision can be left to individual package publishers. This decision comes at cost of making deployment more complicated. However, deployment on iOS is already a complicated process that is best aided by tools. At present, no binary merging is required, as there is only one on-device architecture, and simulator binaries are not considered to be distributable artefacts, so only one architecture is needed to build an app for a simulator. Supporting static builds ------------------------ While the long-term viability of the ``--undefined dynamic_lookup`` option cannot be guaranteed, the option does exist, and it works. One option would be to ignore the deprecation warning, and hope that Apple either reverses the deprecation decision, or never finalizes the deprecation. Given that Apple's decision-making process is entirely opaque, this would be, at best, a risky option. When combined with the fact that the broader iOS development ecosystem encourages the use of frameworks, there are no legacy uses of a static library to consider, and the only benefit to a statically-linked iOS ``libpython3.x.a`` is a very slightly reduced app startup time, omitting support for static builds of ``libpython3.x`` seems a reasonable compromise. It is worth noting that there has been some discussion on `an alternate approach to linking on macOS `__ that would remove the need for the ``--undefined dynamic_lookup`` option, although discussion on this approach appears to have stalled due to complications in implementation. If those complications were to be overcome, it is highly likely that the same approach *could* be used on iOS, which *would* make a statically linked ``libpython3.x.a`` plausible. The decision to link binary modules against ``libpython3.x.dylib`` would complicate the introduction of static ``libpython3.x.a`` builds in the future, as the process of moving to a different binary module linking approach would require a clear way to differentate "dynamically-linked" iOS binary modules from "static-compatible" iOS binary modules. However, given the lack of tangible benefits of a static ``libpython3.x.a``, it seems unlikely that there will be any requirement to make this change. Interactive/REPL mode --------------------- A traditional ``python.exe`` command line experience isn't really viable on mobile devices, because mobile devices don't have a command line. iOS apps don't have a stdout, stderr or stdin; and while you can redirect stdout and stderr to the system log, there's no source for stdin that exists that doesn't also involve building a very specific user-facing app that would be closer to an IDLE-style IDE experience. Therefore, the decision was made to only focus on "embedded mode" as a target for mobile distribution. x86_64 simulator support ------------------------ Apple no longer sells x86_64 hardware. As a result, commissioning an x86_64 buildbot can be difficult. It is possible to run macOS binaries in x86_64 compatibility mode on ARM64 hardware; however, this isn't ideal for testing purposes. Therefore, the x86_64 Simulator (``x86_64-apple-ios-simulator``) will not be added as a Tier 3 target. It is highly likely that iOS support will work on the x86_64 without any modification; this only impacts on the *official* Tier 3 status. On-device testing ----------------- CI testing on simulators can be accommodated reasonably easily. On-device testing is much harder, as availability of device farms that could be configured to provide Buildbots or Github Actions runners is limited. However, on device testing may not be necessary. As a data point - Apple's Xcode Cloud solution doesn't provide on-device testing. They rely on the fact that the API is consistent between device and simulator, and ARM64 simulator testing is sufficient to reveal CPU-specific issues. Ordering of ``_multiarch`` tags ------------------------------- The initially accepted version of this document used ``-`` ordering (e.g., ``iphoneos-arm64``) for ``sys.implementation._multiarch`` (and related values, such as wheel tags). The final merged version uses the ``-`` ordering (e.g., ``arm64-iphoneos``). This is for consistency with compiler triples on other platforms (especially Linux), which specify the architecture before the operating system. Values returned by ``platform.ios_ver()`` ----------------------------------------- The initially accepted version of this document didn't include a ``system`` identifier. This was added during the implementation phase to support the implementation of ``platform.system()``. The initially accepted version of this document also described that ``min_release`` would be returned in the ``ios_ver()`` result. The final version omits the ``min_release`` value, as it is not significant at runtime; it only impacts on binary compatibility. The minimum version *is* included in the value returned by ``sysconfig.get_platform()``, as this is used to define wheel (and other binary) compatibility. Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.