python-peps/peps/pep-0730.rst

PEP: 730
Title: Adding iOS as a supported platform
Author: Russell Keith-Magee <russell@keith-magee.com>
Sponsor: Ned Deily <nad@python.org>
Discussions-To: https://discuss.python.org/t/pep730-adding-ios-as-a-supported-platform/35854
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 09-Oct-2023
Python-Version: 3.13

Abstract
========

This PEP proposes adding iOS as a supported platform in CPython. The initial
goal is to achieve Tier 3 support for Python 3.13. This PEP describes the
technical aspects of the changes that are required to support iOS. It also
describes the project management concerns related to adoption of iOS as a Tier 3
platform.

Motivation
==========

Over the last 15 years, mobile platforms have become increasingly important
parts of the computing landscape. iOS is one of two operating systems that
control the vast majority of these devices. However, there is no official
support for iOS in CPython.

The `BeeWare Project <https://beeware.org>`__ and `Kivy <https://kivy.org>`__
have both supported iOS for almost 10 years. This support has been able to
generate applications that have been accepted for publication in the iOS App
Store. This demonstrates the technical feasibility of iOS support.

It is important for the future of Python as a language that it is able to be
used on any hardware or OS that has widespread adoption. If Python cannot be
used a on a platform that has widespread use, adoption of the language will be
impacted as potential users will adopt other languages that *do* provide support
for these platforms.

Rationale
=========

Development landscape
---------------------

iOS provides a single API, but 2 distinct ABIs - ``iphoneos`` (physical
devices), and ``iphonesimulator``. Each of these ABIs can be provided on
multiple CPU architectures. At time of writing, Apple officially supports
``arm64`` on the device ABI, and ``arm64`` and ``x86_64`` are supported on the
simulator ABI.

As with macOS, iOS supports the creation of "fat" binaries that contains
multiple CPU architectures. However, fat binaries *cannot* span ABIs. That is,
it is possible to have a fat *simulator* binary, and a fat *device* binary, but
it is not possible to create a single fat "iOS" binary that covers both
simulator and device needs. To support distribution of a single development
artefact, Apple uses an "XCframework" structure - a wrapper around multiple ABIs
that implement a common API.

iOS runs on a Darwin kernel, similar to macOS. However, there is a need to
differentiate between macOS and iOS at an implementation level, as there are
significant platform differences between iOS and macOS.

iOS code is compiled for compatibility against a minimum iOS version.

Apple frequently refers to "iPadOS" in their marketing material. However, from a
development perspective, there is no discernable difference between iPadOS and
iOS. A binary that has been compiled for the ``iphoneos`` or ``iphonesimulator``
ABIs can be deployed on iPad.

Other Apple platforms, such as tvOS, watchOS, and visionOS, use different ABIs,
and are not covered by this PEP.

POSIX compliance
----------------

iOS is broadly a POSIX platform. However, similar to WASI/Emscripten, there are
POSIX APIs that exist on iOS, but cannot be used; and POSIX APIs that don't
exist at all.

Most notable of these is the fact that iOS does not provide any form of
multiprocess support. ``fork`` and ``spawn`` both *exist* in the iOS API;
however, if they are invoked, the invoking iOS process stops, and the new
process doesn't start.

Unlike WASI/Emscripten, threading *is* supported on iOS.

There are also significant limits to socket handling. Due to process sandboxing,
there is no availability of interprocess communication via socket. However,
sockets for network communication *are* available.

Dynamic libraries
-----------------

The iOS `App Store guidelines
<https://developer.apple.com/app-store/review/guidelines>`__ allow apps to be
written in languages other than Objective C or Swift. However, they have very
strict guidelines about the structure of apps that are submitted for
distribution.

iOS apps can use dynamically loaded libraries; however, there are very strict
requirements on how dynamically loaded content is packaged for use on iOS:

* Dynamic binary content must be compiled as dynamic libraries, not shared
  objects or binary bundles.

* They must be packaged in the app bundle as Frameworks.

* Each Framework can only contain a single dynamic library.

* The Framework *must* be contained in the iOS App's ``Frameworks`` folder.

* A Framework may not contain any non-library content.

This imposes some constraints on the operation of CPython. It is not possible
store binary modules in the ``lib-dynload`` and/or ``site-packages`` folders;
they must be stored in the app's Frameworks folder, with each module wrapped in
a Framework. This also means that the common assumption that a Python module can
construct the location of a binary module by using the ``__file__`` attribute of
the Python module no longer holds.

As with macOS, compiling a binary module that is accessible from a
statically-linked build of Python requires the use of the ``--undefined
dynamic_lookup`` option to avoid linking ``libpython3.x`` into every binary
module. However, on iOS, this compiler flag raises a deprecation warning when it
is used. A warning from this flag has been observed on macOS as well - however,
responses from Apple staff suggest that they `do not intend to break the CPython
ecosystem by removing this option
<https://github.com/python/cpython/issues/97524#issuecomment-1458855301>`__. As
Python does not currently have a notable presence on iOS, it is difficult to
judge whether iOS usage of this flag would fall under the same umbrella.

Console and interactive usage
-----------------------------

Distribution of a traditional CPython REPL or interactive "python.exe" should
not be considered a goal of this work.

Mobile devices (including iOS) do not provide a TTY-style console. They do not
provide ``stdin``, ``stdout`` or ``stderr``. iOS provides a system log, and it
is possible to install a redirection so that all ``stdout`` and ``stderr``
content is redirected to the system log; but there is no analog for ``stdin``.

In addition, iOS places restrictions on downloading additional code at runtime
(as this behavior would be functionally indistinguishable from trying to work
around App Store review). As a result, a traditional "create a virtual
environment and pip install" development experience will not be viable on iOS.

It is *possible* to build an native iOS application that provides a REPL
interface. This would be closer to an IDLE-style user experience; however,
Tkinter cannot be used on iOS, so any app would require a ground-up rewrite. The
iOS app store already contains several examples of apps in this category (e.g.,
`Pythonista <http://www.omz-software.com/pythonista/>`__ and `Pyto
<https://pyto.readthedocs.io/>`__). The focus of this work would be to provide
an embedded distribution that IDE-style native interfaces could utilize, not a
user-facing "app" interface to iOS on Python.

Specification
=============

Platform identification
-----------------------

``sys``
'''''''

``sys.platform`` will identify as ``"ios"`` on both simulator and physical
devices.

``sys.implementation._multiarch`` will describe the ABI and CPU architecture:

* ``"iphoneos-arm64"`` for ARM64 devices
* ``"iphonesimulator-arm64"`` for ARM64 simulators
* ``"iphonesimulator-x86_64"`` for x86_64 simulators

``platform``
''''''''''''

``platform`` will be modified to support returning iOS-specific details. Most of
the values returned by the ``platform`` module will match those returned by
``os.uname()``, with the exception of:

* ``platform.system()`` - ``"iOS"``, instead of the default ``"Darwin"``

* ``platform.release()`` - the iOS version number, as a string (e.g.,
  ``"16.6.1"``), instead of the Darwin kernel version.

In addition, a ``platform.ios_ver()`` method will be added. This mirrors
``platform.mac_ver()``, which can be used to provide macOS version information.
``ios_ver()`` will return a namedtuple that contains the following:

* ``release`` - the iOS version, as a string (e.g., ``"16.6.1"``).
* ``min_release`` - the minimum supported iOS version, as a string (e.g.,
  ``"12.0"``)
* ``model`` - the model identifier of the device, as a string (e.g.,
  ``"iPhone13,2"``). On simulators, this will return ``"iPhoneSimulator"``.
* ``is_simulator`` - a boolean indicating if the device is a simulator.

``os``
''''''

``os.uname()`` will return the raw result of a POSIX ``uname()`` call. This will
result in the following values:

* ``sysname`` - ``"Darwin"``

* ``release`` - The Darwin kernel version (e.g., ``"22.6.0"``)

This approach treats the ``os`` module as a "raw" interface to system APIs, and
``platform`` as a higher-level API providing more generally useful values.

``sysconfig``
'''''''''''''

The ``sysconfig`` module will use the minimum iOS version as part of
``sysconfig.get_platform()`` (e.g., ``"ios-12.0-iphoneos-arm64"``). The
``sysconfigdata_name`` and Config makefile will follow the same patterns as
existing platforms (using ``sys.platform``, ``sys.implementation._multiarch``
etc.) to construct identifiers.

Subprocess support
------------------

iOS will leverage the pattern for disabling subprocesses established by
WASI/Emscripten. The ``subprocess`` module will raise an exception if an attempt
is made to start a subprocess, and ``os.fork`` and ``os.spawn`` calls will raise
an ``OSError``.

Dynamic module loading
----------------------

To accommodate iOS dynamic loading, the ``importlib`` bootstrap will be extended
to add a metapath finder that can convert a request for a Python binary module
into a Framework location. This finder will only be installed if ``sys.platform
== "ios"``.

This finder will convert a Python module name (e.g., ``foo.bar._whiz``) into a
unique Framework name by using the full module name as the framework name (i.e.,
``foo.bar._whiz.framework``). A framework is a directory; the finder will look
for ``_whiz.dylib`` in that directory.

Compilation
-----------

The only binary format that will be supported is a dynamically-linkable
``libpython3.x.dylib``, packaged in an iOS-compatible framework format. While
the ``--undefined dynamic_lookup`` compiler option currently works, the
long-term viability of the option cannot be guaranteed. Rather than rely on a
compiler flag with an uncertain future, binary modules on iOS will be linked
with ``libpython3.x.dylib``. This means iOS binary modules will not be loadable
by an executable that has been statically linked against ``libpython3.x.a``.
Therefore, a static ``libpython3.x.a`` iOS library will not be supported. This
is the same pattern used by CPython on Windows.

Building CPython for iOS requires the use of the cross-platform tooling in
CPython's ``configure`` build system. A single ``configure``/``make``/``make
install`` pass will produce a ``Python.framework`` artefact that can be used on
a single ABI and architecture.

Additional tooling will be required to merge the ``Python.framework`` builds for
multiple architectures into a single "fat" library. Tooling will also be
required to merge multiple ABIs into the ``XCframework`` format that Apple uses
to distribute multiple frameworks for different ABIs in a single bundle.

An Xcode project will be provided for the purpose of running the CPython test
suite. Tooling will be provided to automate the process of compiling the test
suite binary, start the simulator, install the test suite, and execute it.

Distribution
------------

Adding iOS as a Tier 3 platform only requires adding support for compiling an
iOS-compatible build from an unpatched CPython code checkout. It does not
require production of officially distributed iOS artefacts for use by end-users.

If/when iOS is updated to Tier 2 or 1 support, the tooling used to generate an
``XCframework`` package could be used to produce an iOS distribution artefact.
This could then be distributed as an "embedded distribution" analogous to the
Windows embedded distribution, or as a CocoaPod or Swift package that could be
added to an Xcode project.

CI resources
------------

`Anaconda <https://anaconda.com>`__ has offered to provide physical hardware to
run iOS buildbots.

GitHub Actions is able to host iOS simulators on their macOS machines, and the
iOS simulator can be controlled by scripting environments. The free tier
currently only provides x86_64 macOS machines; however ARM64 runners `recently
became available on paid plans <https://github.blog/
2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/>`__.
However, in order to avoid exhausting macOS runner resources, a GitHub Actions
run for iOS will not be added as part of the standard CI configuration.

Packaging
---------

iOS will not provide a "universal" wheel format. Instead, wheels will be
provided for each ABI-arch combination.

iOS wheels will use tags:

* ``ios_12_0_iphoneos_arm64``
* ``ios_12_0_iphonesimulator_arm64``
* ``ios_12_0_iphonesimulator_x86_64``

In these tags, "12.0" is the minimum supported iOS version. As with macOS, the
tag will incorporate the minimum iOS version that is selected when the wheel is
compiled; a wheel compiled with a minimum iOS version of 15.0 would use the
``ios_15_0_iphone*`` tags. At time of writing, iOS 12.0 exposes most significant
iOS features, while reaching near 100% of devices; this will be used as a floor
for iOS version matching.

These wheels can include binary modules in-situ (i.e., co-located with the
Python source, in the same way as wheels for a desktop platform); however, they
will need to be post-processed as binary modules need to be moved into the
"Frameworks" location for distribution. This can be automated with an Xcode
build step.

PEP 11 Update
-------------

:pep:`11` will be updated to include the three iOS ABIs:

* ``aarch64-apple-ios``
* ``aarch64-apple-ios-simulator``
* ``x86_64-apple-ios-simulator``

Ned Deily will serve as the initial core team contact for these ABIs.

Backwards Compatibility
=======================

Adding a new platform does not introduce any backwards compatibility concerns to
CPython itself.

There may be some backwards compatibility implications on the projects that have
historically provided CPython support (i.e., BeeWare and Kivy) if the final form
of any CPython patches don't align with the patches they have historically used.

Although not strictly a backwards compatibility issue, there *is* a platform
adoption consideration. Although CPython itself may support iOS, if it is
unclear how to produce iOS-compatible wheels, and prominent libraries like
cryptography, Pillow, and NumPy don't provide iOS wheels, the ability of the
community to adopt Python on iOS will be limited. Therefore, it will be
necessary to clearly document how projects can add iOS builds to their CI and
release tooling. Adding iOS support to tools like `crossenv
<https://crossenv.readthedocs.io/>`__ and `cibuildwheel
<https://cibuildwheel.readthedocs.io/>`__ may be one way to achieve this.

Security Implications
=====================

Adding iOS as a new platform does not add any security implications.

How to Teach This
=================

The education needs related to this PEP mostly relate to how end-users can add
iOS support to their own Xcode projects. This can be accomplished with
documentation and tutorials on that process. The need for this documentation
will increase if/when support raises from Tier 3 to Tier 2 or 1; however, this
transition should also be accompanied with simplified deployment artefacts (such
as a Cocoapod or Swift package) that are integrated with Xcode development.

Reference Implementation
========================

The BeeWare `Python-Apple-support
<https://github.com/beeware/Python-Apple-support>`__ repository contains a
reference patch and build tooling to compile a distributable artefact.

`Briefcase <https://briefcase.readthedocs.org>`__ provides a reference
implementation of code to execute test suites on iOS simulators. The `Toga
Testbed <https://github.com/beeware/toga/tree/main/testbed>`__ is an example of
a test suite that is executed on the iOS simulator using GitHub Actions.

Rejected Ideas
==============

Simulator identification
------------------------

Earlier versions of this PEP suggested the inclusion of
``sys.implementation._simulator`` attribute to identify when code is running on
device, or on a simulator. This was rejected due to the use of a protected name
for a public API, plus the pollution of the ``sys`` namespace with an
iOS-specific detail.

Another proposal during discussion was to include a generic
``platform.is_emulator()`` API that could be implemented by any platform - for
example to differentiate running on x86_64 code on ARM64 hardware, or when
running in QEMU or other virtualization methods. This was rejected on the basis
that it wasn't clear what a consistent interpretation of "emulator" would be, or
how an emulator would be detected outside of the iOS case.

The decision was made to keep this detail iOS-specific, and include it on the
``platform.ios_ver()`` API.

GNU compiler triples
--------------------

``autoconf`` requires the use of a GNU compiler triple to identify build and
host platforms. However, the ``autoconf`` toolchain doesn't provide native
support for iOS simulators, so we are left with the task of working out how to
squeeze iOS hardware into GNU's naming regimen.

This can be done (with some patching of ``config.sub``), but it leads to 2 major
sources of naming inconsistency:

* ``arm64`` vs ``aarch64`` as an identifier of 64-bit ARM hardware; and
* What identifier is used to represent simulators.

Apple's own tools use ``arm64`` as the architecture, but appear to be tolerant
of ``aarch64`` in some cases. The device platform is identified as ``iphoneos``
and ``iphonesimulator``.

Rust toolchains uses ``aarch64`` as the architecture, and use
``aarch64-apple-ios`` and ``aarch64-apple-ios-sim`` to identify the device
platform; however, they use ``x86_64-apple-ios`` to represent iOS *simulators*
on x86_64 hardware.

The decision was made to use ``arm64-apple-ios`` and
``arm64-apple-ios-simulator`` because:

1. The ``autoconf`` toolchain already contains support for ``ios`` as a platform
   in ``config.sub``; it's only the simulator that doesn't have a representation.
2. The third part of the host triple is used as ``sys.platform``.
3. When Apple's own tools reference CPU architecture, they use ``arm64``, and
   the GNU tooling usage of the architecture isn't visible outside the build
   process.
4. When Apple's own tools reference simulator status independent of the OS
   (e.g., in the naming of Swift submodules), they use a ``-simulator`` suffix.
5. While *some* iOS packages will use Rust, *all* iOS packages will use Apple's
   tooling.

"Universal" wheel format
------------------------

macOS currently supports 2 CPU architectures. To aid the end-user development
experience, Python defines a "universal2" wheel format that incorporates both
x86_64 and ARM64 binaries.

It would be conceptually possible to offer an analogous "universal" iOS wheel
format. However, this PEP does not use this approach, for 2 reasons.

Firstly, the experience on macOS, especially in the numerical Python ecosystem,
has been that universal wheels can be exceedingly difficult to accommodate.
While native macOS libraries maintain strong multi-platform support, and Python
itself has been updated, the vast majority of upstream non-Python libraries do
not provide multi-architecture build support. As a result, compiling universal
wheels inevitably requires multiple compilation passes, and complex decisions
over how to distribute header files for different architectures. As a result of
this complexity, many popular projects (including NumPy and Pillow) do not
provide universal wheels at all, instead providing separate ARM64 and x86_64
wheels.

Secondly, historical experience is that iOS would require a much more fluid
"universal" definition. In the last 10 years, there have been *at least* 5
different possible interpretations of "universal" that would apply to iOS,
including various combinations of armv6, armv7, armv7s, arm64, x86 and x86_64
architectures, on device and simulator. If defined right now, "universal-iOS"
would likely include x86_64 and arm64 on simulator, and arm64 on device;
however, the pending deprecation of x86_64 hardware would add another
interpretation; and there may be a need to add arm64e as a new device
architecture in the future. Specifying iOS wheels as single-platform-only means
the Python core team can avoid an ongoing standardization discussion about the
updated "universal" formats.

It also means wheel publishers are able to make per-project decisions over which
platforms are feasible to support. For example, a project may choose to drop
x86_64 support, or adopt a new architecture earlier than other parts of the
Python ecosystem. Using platform-specific wheels means this decision can be left
to individual package publishers.

This decision comes at cost of making deployment more complicated. However,
deployment on iOS is already a complicated process that is best aided by tools.
At present, no binary merging is required, as there is only one on-device
architecture, and simulator binaries are not considered to be distributable
artefacts, so only one architecture is needed to build an app for a simulator.

Supporting static builds
------------------------

While the long-term viability of the ``--undefined dynamic_lookup`` option
cannot be guaranteed, the option does exist, and it works. One option would be
to ignore the deprecation warning, and hope that Apple either reverses the
deprecation decision, or never finalizes the deprecation.

Given that Apple's decision-making process is entirely opaque, this would be, at
best, a risky option. When combined with the fact that the broader iOS
development ecosystem encourages the use of frameworks, there are no legacy uses
of a static library to consider, and the only benefit to a statically-linked iOS
``libpython3.x.a`` is a very slightly reduced app startup time, omitting support
for static builds of ``libpython3.x`` seems a reasonable compromise.

It is worth noting that there has been some discussion on `an alternate approach
to linking on macOS <https://github.com/python/cpython/issues/103306>`__ that
would remove the need for the ``--undefined dynamic_lookup`` option, although
discussion on this approach appears to have stalled due to complications in
implementation. If those complications were to be overcome, it is highly likely
that the same approach *could* be used on iOS, which *would* make a statically
linked ``libpython3.x.a`` plausible.

The decision to link binary modules against ``libpython3.x.dylib`` would
complicate the introduction of static ``libpython3.x.a`` builds in the future,
as the process of moving to a different binary module linking approach would
require a clear way to differentate "dynamically-linked" iOS binary modules from
"static-compatible" iOS binary modules. However, given the lack of tangible
benefits of a static ``libpython3.x.a``, it seems unlikely that there will be
any requirement to make this change.

Interactive/REPL mode
---------------------

A traditional ``python.exe`` command line experience isn't really viable on
mobile devices, because mobile devices don't have a command line. iOS apps don't
have a stdout, stderr or stdin; and while you can redirect stdout and stderr to
the system log, there's no source for stdin that exists that doesn't also
involve building a very specific user-facing app that would be closer to an
IDLE-style IDE experience. Therefore, the decision was made to only focus on
"embedded mode" as a target for mobile distribution.

Open Issues
===========

x86_64 buildbot availability
----------------------------

Apple no longer sells x86_64 hardware. As a result, commissioning an x86_64
buildbot may not be possible. It is possible to run macOS binaries in x86_64
compatibility mode on ARM64 hardware; however, this isn't ideal for testing
purposes.

If native x86_64 Mac hardware cannot be sourced for buildbot purposes, it may be
necessary to exclude the x86_64 simulator platform in Tier 3. Given the
anticipated deprecation of x86_64 as a macOS development platform, this doesn't
pose a significant impediment to adoption or long term maintenance.

On-device testing
-----------------

CI testing on simulators can be accommodated reasonably easily. On-device
testing is much harder, as availability of device farms that could be configured
to provide Buildbots or Github Actions runners is limited.

However, on device testing may not be necessary. As a data point - Apple's Xcode
Cloud solution doesn't provide on-device testing. They rely on the fact that the
API is consistent between device and simulator, and ARM64 simulator testing is
sufficient to reveal CPU-specific issues.

Copyright
=========

This document is placed in the public domain or under the CC0-1.0-Universal
license, whichever is more permissive.