From 4d14f7a1f975100b53a862d039b9a1913496a579 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Tue, 24 Oct 2023 08:19:00 +0800 Subject: [PATCH] PEP 730: Revisions and clarifications following community discussion (#3506) --- peps/pep-0730.rst | 201 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 160 insertions(+), 41 deletions(-) diff --git a/peps/pep-0730.rst b/peps/pep-0730.rst index e759aabad..50ebc9369 100644 --- a/peps/pep-0730.rst +++ b/peps/pep-0730.rst @@ -124,7 +124,7 @@ Distribution ------------ Adding iOS as a Tier 3 platform only requires adding support for compiling an -iOS-compatibile code with an unpatched CPython code checkout. It does not +iOS-compatible build from an unpatched CPython code checkout. It does not require production of officially distributed iOS artefacts for use by end-users. If/when iOS is updated to Tier 2 or 1 support, there should be a process for @@ -176,30 +176,27 @@ devices. * ``"iphonesimulator-arm64"`` for ARM64 simulators * ``"iphonesimulator-x86_64"`` for x86_64 simulators -``sys.implementation`` will also have an additional attribute - ``_simulator`` - -storing a Boolean that is ``True`` if the device running the app is a simulator. -This attribute would not exist on non-iOS platforms. - ``platform`` '''''''''''' -Platform will be used as the primary mechanism for retrieving OS and device -details. +``platform`` will be modified to support returning iOS-specific details. Most of +the values returned by the ``platform`` module will match those returned by +``os.uname()``, with the exception of: -* ``platform.system()`` - ``"iOS"`` +* ``platform.system()`` - ``"iOS"``, instead of the default ``"Darwin"`` -* ``platform.node()`` - the user-provided name of the device, as returned by the - ``[[UIDevice currentDevice] systemName]`` system call (e.g., - ``"Janes-iPhone"``). For simulated devices, this will be the name of the - development computer running the simulator. +* ``platform.release()`` - the iOS version number, as a string (e.g., + ``"16.6.1"``), instead of the Darwin kernel version. -* ``platform.release()`` - the iOS version number, as a string (e.g., ``"16.6.1"``) +In addition, a ``platform.ios_ver()`` method will be added. This mirrors +``platform.mac_ver()``, which can be used to provide macOS version information. +``ios_ver()`` will return a namedtuple that contains the following: -* ``platform.machine()`` - The device model returned by ``[[UIDevice - currentDevice] model]`` (e.g., ``"iPhone13,2"``); or ``"iPhoneSimulator"`` for - simulated devices. - -All other values will be as returned by ``os.uname()`` +* ``release`` - the iOS version, as a string (e.g., ``"16.6.1"``). +* ``min_release`` - the minimum supported iOS version, as a string (e.g., ``"12.0"``) +* ``model`` - the model identifier of the device, as a string (e.g., ``"iPhone13,2"``). + On simulators, this will return ``"iPhoneSimulator"``. +* ``is_simulator`` - a boolean indicating if the device is a simulator. ``os`` '''''' @@ -211,11 +208,14 @@ result in the following values: * ``release`` - The Darwin kernel version (e.g., ``"22.6.0"``) +This approach treats the ``os`` module as a "raw" interface to system APIs, and +``platform`` as a higher-level API providing more generally useful values. + ``sysconfig`` ''''''''''''' The ``sysconfig`` module will use the minimum iOS version as part of -``sysconfig.get_platform()`` identifier (e.g., ``"iOS-12.0-iphoneos-arm64"``). +``sysconfig.get_platform()`` (e.g., ``"ios-12.0-iphoneos-arm64"``). The ``sysconfigdata_name`` and Config makefile will follow the same patterns as existing platforms (using ``sys.platform``, ``sys.implementation._multiarch`` etc.) to construct identifiers. @@ -237,41 +237,42 @@ into a Framework location. This finder will only be installed if ``sys.platform == "ios"``. This finder will convert a Python module name (e.g., ``foo.bar._whiz``) into a -unique Framework name by replacing the dots with underscores (i.e., -``foo_bar__whiz.framework``). A framework is a directory; the finder will look +unique Framework name by using the full module name as the framework name (i.e., +``foo.bar._whiz.framework``). A framework is a directory; the finder will look for ``_whiz.dylib`` in that directory. CI resources ------------ +`Anaconda `__ has offered to provide physical hardware to +run iOS buildbots. + GitHub Actions is able to host iOS simulators on their macOS machines, and the iOS simulator can be controlled by scripting environments. The free tier -currently only provides x86_64 macOS machines; however ARM64 runners `have -recently become available on paid plans `__. - -If GitHub Actions resources are insufficient or not viable for cost reasons, -Anaconda has offered to provide resources to support CI requirements. +However, in order to avoid exhausting macOS runner resources, a GitHub Actions +run for iOS will not be added as part of the standard CI configuration. Packaging --------- iOS will not provide a "universal" wheel format. Instead, wheels will be -provided for each ABI-arch combination. At present, no binary merging is -required. There is only one on-device architecture; and simulator binaries are -not considered to be distributable artefacts, so only one architecture is needed -to build a simulator. +provided for each ABI-arch combination. iOS wheels will use tags: -* ``iOS_12_0_iphoneos_arm64`` -* ``iOS_12_0_iphonesimulator_arm64`` -* ``iOS_12_0_iphonesimulator_x86_64`` +* ``ios_12_0_iphoneos_arm64`` +* ``ios_12_0_iphonesimulator_arm64`` +* ``ios_12_0_iphonesimulator_x86_64`` -In these tags, "12.0" is the minimum supported iOS version. The choice of -minimum supported iOS version is a decision of whoever compiles CPython for iOS. -At time of writing, iOS 12.0 exposes most significant iOS features, while -reaching near 100% of devices. +In these tags, "12.0" is the minimum supported iOS version. As with macOS, the +tag will incorporate the minimum iOS version that is selected when the wheel +is compiled; a wheel compiled with a minimum iOS version of 15.0 would use the +``ios_15_0_iphone*`` tags. At time of writing, iOS 12.0 exposes most significant +iOS features, while reaching near 100% of devices; this will be used as a floor +for iOS version matching. These wheels can include binary modules in-situ (i.e., co-located with the Python source, in the same way as wheels for a desktop platform); however, they @@ -340,15 +341,133 @@ a test suite that is executed on the iOS simulator using GitHub Actions. Rejected Ideas ============== -``sys.implementation._simulator`` availability ----------------------------------------------- +Simulator identification +------------------------ -The ``_simulator`` attribute could be provided on *all* platforms, returning -``False``. However, the attribute has no use outside of an iOS context. +Earlier versions of this PEP suggested the inclusion of +``sys.implementation._simulator`` attribute to identify when code is running on +device, or on a simulator. This was rejected due to the use of a protected name +for a public API, plus the pollution of the ``sys`` namespace with an +iOS-specific detail. + +Another proposal during discussion was to include a generic +``platform.is_emulator()`` API that could be implemented by any platform - for +example to differentiate running on x86_64 code on ARM64 hardware, or when +running in QEMU or other virtualization methods. This was rejected on the basis +that it wasn't clear what a consistent interpretation of "emulator" would be, or +how an emulator would be detected outside of the iOS case. + +The decision was made to keep this detail iOS-specific, and include it on the +``platform.ios_ver()`` API. + +GNU compiler triples +-------------------- + +``autoconf`` requires the use of a GNU compiler triple to identify build and +host platforms. However, the ``autoconf`` toolchain doesn't provide native +support for iOS simulators, so we are left with the task of working out how to +squeeze iOS hardware into GNU's naming regimen. + +This can be done (with some patching of ``config.sub``), but it leads to 2 major +sources of naming inconsistency: + +* ``arm64`` vs ``aarch64`` as an identifier of 64-bit ARM hardware; and +* What identifier is used to represent simulators. + +Apple's own tools use ``arm64`` as the architecture, but appear to be tolerant +of ``aarch64`` in some cases. The device platform is identified as ``iphoneos`` +and ``iphonesimulator``. + +Rust toolchains uses ``aarch64`` as the architecture, and use +``aarch64-apple-ios`` and ``aarch64-apple-ios-sim`` to identify the device +platform; however, they use ``x86_64-apple-ios`` to represent iOS *simulators* +on x86_64 hardware. + +The decision was made to use ``arm64-apple-ios`` and +``arm64-apple-ios-simulator`` because: + +1. The ``autoconf`` toolchain already contains support for ``ios`` as a platform + in ``config.sub``; it's only the simulator that doesn't have a representation. +2. The third part of the host triple is used as ``sys.platform``. +3. When Apple's own tools reference CPU architecture, they use ``arm64``, and the + GNU tooling usage of the architecture isn't visible outside the build process. +4. When Apple's own tools reference simulator status independent of the OS + (e.g., in the naming of Swift submodules), they use a ``-simulator`` suffix. +5. While *some* iOS packages will use Rust, *all* iOS packages will use Apple's + tooling. + +"Universal" wheel format +------------------------ + +macOS currently supports 2 CPU architectures. To aid the end-user development +experience, Python defines a "universal2" wheel format that incorporates both +x86_64 and ARM64 binaries. + +It would be conceptually possible to offer an analogous "universal" iOS wheel +format. However, this PEP does not use this approach, for 2 reasons. + +Firstly, the experience on macOS, especially in the numerical Python ecosystem, +has been that universal wheels can be exceedingly difficult to accommodate. While +native macOS libraries maintain strong multi-platform support, and Python itself +has been updated, the vast majority of upstream non-Python libraries do not +provide multi-architecture build support. As a result, compiling universal +wheels inevitably requires multiple compilation passes, and complex decisions +over how to distribute header files for different architectures. As a result of +this complexity, many popular projects (including NumPy and Pillow) do not +provide universal wheels at all, instead providing separate ARM64 and x86_64 +wheels. + +Secondly, historical experience is that iOS would require a much more fluid +"universal" definition. In the last 10 years, there have been *at least* 5 +different possible interpretations of "universal" that would apply to iOS, +including various combinations of armv6, armv7, armv7s, arm64, x86 and +x86_64 architectures, on device and simulator. If defined right now, +"universal-iOS" would likely include x86_64 and arm64 on simulator, and arm64 on +device; however, the pending deprecation of x86_64 hardware would add another +interpretation; and there may be a need to add arm64e as a new device +architecture in the future. Specifying iOS wheels as single-platform-only means +the Python core team can avoid an ongoing standardization discussion about the +updated "universal" formats. + +It also means wheel publishers are able to make per-project decisions over which +platforms are feasible to support. For example, a project may choose to drop +x86_64 support, or adopt a new architecture earlier than other parts of the +Python ecosystem. Using platform-specific wheels means this decision can be left +to individual package publishers. + +This decision comes at cost of making deployment more complicated. However, +deployment on iOS is already a complicated process that is best aided by tools. +At present, no binary merging is required, as there is only one on-device +architecture, and simulator binaries are not considered to be distributable +artefacts, so only one architecture is needed to build an app for a simulator. + +Interactive/REPL mode +--------------------- + +A traditional ``python.exe`` command line experience isn't really viable on +mobile devices, because mobile devices don't have a command line. iOS apps don't +have a stdout, stderr or stdin; and while you can redirect stdout and stderr to +the system log, there's no source for stdin that exists that doesn't also +involve building a very specific user-facing app that would be closer to an +IDLE-style IDE experience. Therefore, the decision was made to only focus on +"embedded mode" as a target for mobile distribution. Open Issues =========== +x86_64 buildbot availability +---------------------------- + +Apple no longer sells x86_64 hardware. As a result, commissioning an x86_64 +buildbot may not be possible. It is possible to run macOS binaries in x86_64 +compatibility mode on ARM64 hardware; however, this isn't ideal for testing +purposes. + +If native x86_64 Mac hardware cannot be sourced for buildbot purposes, it may be +necessary to exclude the x86_64 simulator platform in Tier 3. Given the +anticipated deprecation of x86_64 as a macOS development platform, this doesn't +pose a significant impediment to adoption or long term maintenance. + On-device testing -----------------