python-peps/peps/pep-0738.rst

PEP: 738
Title: Adding Android as a supported platform
Author: Malcolm Smith <smith@chaquo.com>
Sponsor: Petr Viktorin <encukou@gmail.com>
Discussions-To: https://discuss.python.org/t/pep-738-adding-android-as-a-supported-platform/40975
Status: Draft
Type: Standards Track
Created: 12-Dec-2023
Python-Version: 3.13


Abstract
========

This PEP proposes adding Android as a supported platform in CPython. The initial
goal is for Android to achieve Tier 3 support in Python 3.13.

This PEP is based on :pep:`730` – "Adding iOS as a supported platform" by
Russell Keith-Magee, and covers many of the same issues. Notable differences
between the two platforms can be found by searching for the word "iOS".


Motivation
==========

Over the last 15 years, mobile platforms have become increasingly important
parts of the computing landscape. Android is the operating system that runs on
about `70% of these devices
<https://gs.statcounter.com/os-market-share/mobile/worldwide>`__. However, there
is no official support for Android in CPython.

The `Chaquopy <https://chaquo.com/chaquopy/>`__, `BeeWare
<https://beeware.org>`__ and `Kivy <https://kivy.org>`__ projects
have all supported Android for many years, and they have all been used to
generate applications that have been accepted for publication in the Google Play
Store. This demonstrates the technical feasibility of Android support.

It is important for the future of Python as a language that it is able to be
used on any platform that has widespread adoption. Otherwise, potential users
will choose other languages that *do* provide support for these platforms. This
is especially true in education, where the next generation of developers is in
many cases already spending more time using mobile platforms than desktop ones.


Rationale
=========

General
-------

Android is broadly a POSIX platform, based on a Linux kernel and the
ELF binary format. It does not use glibc, instead providing its own C
library implementation called Bionic. As a result, it is generally not
binary-compatible with any other Linux distribution, even if the architecture
matches. It also has its own filesystem layout which doesn't resemble any other
Unix.

However, Android's source-compatibility with Linux is quite good. In its early years,
the C library was very incomplete, but most of the gaps were filled by around
2014. Since then, any C code that compiles for Linux can usually be compiled for
Android, unless it involves direct access to hardware devices or operating
system services.

This is also true of CPython. Although it has never officially supported
Android, recent versions (since 3.6) can already be compiled for Android with
minimal patching.

.. _738-os-versions:

OS versions
-----------

Each Android version can be identified in three ways:

* A conventional dotted version number (though recent versions have all used
  whole numbers)
* A sequential integer "API level" (the most common form in developer
  documentation)
* An alphabetic confectionery-themed code name (no longer used for marketing,
  but still appears in developer documentation)

There is no consistent pattern to link one of these to another; they must be
looked up in `a table <https://en.wikipedia.org/wiki/Android_version_history>`__.

A new major Android version is released each year, but the updates available to
each device are entirely under the control of its manufacturer. Unfortunately
many manufacturers stop sending updates to devices long before their users are
ready to dispose of them. For example, as of October 2023, the oldest Android
version still receiving security updates was API level 30, but according to
`Google's own statistics
<https://dl.google.com/android/studio/metadata/distributions.json>`__, only 60%
of devices were on that version or newer.

For Python 3.13 we therefore propose the minimum Android version to be 5.0
(API level 21), which was released in 2014. According to the statistics above,
this would cover 99% of active devices.


Development tools
-----------------

The Android development tools are equally supported on Linux (x86_64), Windows
(x86_64) and macOS (x86_64 and ARM64). For CPython, the most important tools
are:

* The NDK (native development kit) contains a C and C++ compiler (clang),
  linker (lld), and headers for all the system libraries.

  Binary compatibility between libraries compiled with different versions of the
  NDK is generally very good, but for reproducibility it would be best for each
  Python version to stick with one NDK version throughout its life. For Python
  3.13, this would be the current NDK long-term support version, r26.

  Each NDK version can be set to target any of a wide range of Android versions.
  For example, NDK r26 supports :ref:`API levels <738-os-versions>` 21 to 34.
  However, binaries compiled for an older Android version will usually keep on
  working indefinitely on newer versions; exceptions to this rule are only made
  for security reasons.

* Gradle is the tool used to build complete, deployable apps.

* The emulator, based on QEMU, is a simulated Android device running on a
  development machine. Unlike on iOS, an emulator uses the same ABI as a real
  device of the same architecture, and can run the same binaries.

These tools may all be used either from the command line, or through the Android
Studio IDE, which is based on IntelliJ IDEA.


Architectures
-------------

Android currently supports 4 architectures. Their names as used by the Android
tools are:

* ``armeabi-v7a``
* ``arm64-v8a``
* ``x86``
* ``x86_64``

Virtually all current physical devices use one of the ARM architectures. ``x86``
and ``x86_64`` are supported for use in the emulator.

For Python 3.13 we propose that Tier 3 support will only cover the 64-bit platforms
(``arm64-v8a`` and ``x86_64``):

* ``x86`` has not been supported as a development platform since 2020, and no
  new emulator images have been released since then.

* ``armeabi-v7a``'s proportion of active devices is now
  `less than 10% and steadily falling
  <https://github.com/chaquo/chaquopy/issues/709#issuecomment-1744541892>`__.

  It would also be more difficult to cover with a reliable buildbot, since there
  are no native hosts available for the emulator (ARM64 Macs don't have hardware
  support for ARM32 code). Although cross-architecture emulation is possible, it
  has much worse performance and stability, which is why the ``armeabi-v7a``
  emulator images have not been updated since 2016.

  However, it continues to be used for watches and ultra-low-cost phones. If
  this persists, we may need to consider adding it in a future Python version.

Even if 32-bit architectures are not officially supported, no changes should be
made which would impede any downstream projects which still wish to build them.


App lifecycle
-------------

The primary programming language in Android apps is Java, or its modern descendant
Kotlin. As such, an app does not provide its own executable file. Instead, all
apps start off as a Java virtual machine running an executable provided by the
operating system. The app's Java code can then add native code to the process by
loading dynamic libraries and calling them through JNI.

Unlike iOS, creating subprocesses *is* supported on Android. However apps may
only run executables in `certain locations
<https://issuetracker.google.com/issues/128554619#comment4>`__, none of which
are writable at runtime. Long-running subprocesses are `officially discouraged
<https://issuetracker.google.com/issues/128554619#comment4>`__, and are not
guaranteed to be supported in future Android versions.

Android does provide a command-line shell, but this is intended only for use by
developers, and is not available to the typical end user.

For these reasons, the recommended way of running Python on Android will be by
loading ``libpython3.x.so`` into the main app process. A ``python3.x``
executable will not be officially supported on this platform.


Specification
=============

Scope of work
-------------

The focus of this work will be to produce an Android equivalent to the existing
`Windows embeddable package
<https://docs.python.org/3/using/windows.html#the-embeddable-package>`__,
i.e. a set of compiled libraries which developers
can add to their apps. No installer will be required.

Adding Android as a Tier 3 platform only requires adding support for compiling
an Android-compatible build from the unpatched CPython source code. It does not
necessarily require there to be any officially distributed Android artifacts on
python.org, although these could be added in the future.

A Gradle project will be provided for the purpose of running the CPython test
suite. Tooling will be provided to automate the process of building the test
suite app, starting the emulator, installing the test suite, and executing
it.


Linkage
-------

For the reasons discussed in `App lifecycle`_, Python will be included in the
app as a dynamic ``libpython3.x.so`` library which can be loaded into an app
using ``dlopen``.

Unlike Linux, Android does not implicitly use a dlopened library to resolve
relocations in subsequently-loaded libraries, `even if RTLD_GLOBAL is used
<https://github.com/android/ndk/issues/1244#issuecomment-620310397>`__. All
Python extension modules must therefore be explicitly linked against
``libpython3.x.so`` when building for Android.

An extension module linked against ``libpython3.x.so`` cannot be loaded by an
executable that has been statically linked against ``libpython3.x.a``.
Therefore, a static ``libpython3.x.a`` library will not be supported on Android.
This is the same pattern used by CPython on Windows.

This approach also allows using the ``-Wl,--no-undefined`` option to detect
missing symbols at build time, which can be a significant time-saver.

Unlike iOS, Android allows dynamic libraries to be loaded from any location, so
a directory tree containing co-located .py, .pyc and .so files can be handled by
Python's standard importer.


Standard library
----------------

Unsupported modules
'''''''''''''''''''

A number of standard library modules will not be supported on Android because
the underlying C APIs are not available:

* ``curses`` and ``readline``
* ``dbm.gnu`` and ``dbm.ndbm``
* ``grp``
* ``multiprocessing`` – although subprocesses in general are allowed (see `App
  lifecycle`_), Android does not support any part of the `System V IPC API
  <https://man7.org/linux/man-pages/man7/sysvipc.7.html>`__.
* ``tkinter`` and ``turtle`` – these would require an Android build of Tk
  itself, which is not officially supported.

``sys``
'''''''

``sys.platform`` will return ``"android"``. Although Android is based on Linux,
it differs in enough significant ways that a separate name is justified.

When embedded in an Android app, the C-level stdio streams are not connected to
anything. So in this mode, ``sys.stdout`` and ``sys.stderr`` will be redirected
to the system `Logcat <https://developer.android.com/studio/debug/logcat>`__,
which can be viewed with the Android development tools. ``sys.stdin`` will
always return EOF.

``platform``
''''''''''''

Most of the values returned by the ``platform`` module will match those returned
by ``os.uname()``, with the exception of:

* ``platform.system()`` - ``"Android"``, instead of the default ``"Linux"``

* ``platform.release()`` - Android version number, as a string (e.g. ``"14"``),
  instead of the Linux kernel version

In addition, a ``platform.android_ver()`` method will be added, which returns a
namedtuple containing the following:

* ``release`` - Android version of the device, as a string (e.g. ``"14"``)
* ``api_level`` - :ref:`API level <738-os-versions>` of the device, as an
  integer (e.g. ``34``)
* ``min_api_level`` - Minimum API level this build of Python can run on, as
  an integer (e.g. ``23``). This is the same as ``sys.getandroidapilevel``.
* ``manufacturer`` - `manufacturer
  <https://developer.android.com/reference/android/os/Build#MANUFACTURER>`__ of
  the device, as a string (e.g. ``"Google"``)
* ``model`` - `model name
  <https://developer.android.com/reference/android/os/Build#MODEL>`__ of the
  device, as a string (e.g. ``"Pixel 7"``)
* ``device`` - `device name
  <https://developer.android.com/reference/android/os/Build#DEVICE>`__ of the
  device, as a string (e.g. ``"panther"``)

Which one of ``model`` and ``device`` is more likely to be unique, and which one
is more likely to resemble the marketing name, varies between different
manufacturers.

``os``
''''''

``os.uname()`` will return the raw result of a POSIX ``uname()`` call. This will
result in the following values:

* ``sysname`` - ``"Linux"``

* ``release`` - The Linux kernel version (e.g.
  ``"5.10.157-android13-4-00003-gdfb1120f912b-ab10994928"``)

This approach treats the ``os`` module as a "raw" interface to system APIs, and
``platform`` as a higher-level API providing more generally useful values.


CI resources
------------

Since Android emulators and physical devices use the same ABI, and come with
identical or very similar operating system binaries, testing on emulators will
be adequate. x86_64 emulators can be run on Linux, macOS or Windows, but ARM64
emulators are only supported on ARM64 Macs.

GitHub Actions is able to host Android emulators on their Linux and macOS
runners. The free tier currently only provides x86_64 machines; however ARM64
macOS runners `recently became available on paid plans <https://github.blog/
2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/>`__.

If necessary, `Anaconda <https://anaconda.com>`__ has also offered to provide
Android CI resources.


Packaging
---------

Android wheels will use tags in the format ``android_<api-level>_<abi>``. For
example:

* ``android_21_arm64_v8a``
* ``android_21_x86_64``

For the meaning of ``<api-level>``, see `OS versions`_. In the context of
the wheel tag, it indicates the minimum Android version that was selected when
the wheel was compiled. Installation tools such as pip should interpret this in
a similar way to the existing macOS tags, i.e. an app with a minimum API level
of N can incorporate wheels tagged with API level N or older.

This format originates from the Chaquopy project, which currently maintains a
`wheel repository <https://chaquo.com/pypi-13.1/>`__ with tags varying between
API levels 16 and 21.

However, relying on a small group of Android enthusiasts to build the whole
Python ecosystem is not a scalable solution. Until prominent libraries routinely
release their own Android wheels, the ability of the community to adopt
Python on Android will be limited.

Therefore, it will be necessary to clearly document how projects can add Android
builds to their CI and release tooling. Adding Android support to tools like
`crossenv <https://crossenv.readthedocs.io/>`__ and `cibuildwheel
<https://cibuildwheel.readthedocs.io/>`__ may be one way to achieve this.

The Android wheel tag format should also be added to the list of tags accepted
by PyPI.


PEP 11 Update
-------------

:pep:`11` will be updated to include the two supported Android ABIs. Autoconf
already identifies them with the following triplets:

* ``aarch64-linux-android``
* ``x86_64-linux-android``

Petr Viktorin will serve as the initial core team contact for these ABIs.


Backwards Compatibility
=======================

Adding a new platform does not introduce any backwards compatibility concerns to
CPython itself. However, there may be some backwards compatibility implications
on the projects that have historically provided CPython support (i.e., BeeWare
and Kivy) if the final form of any CPython patches don't align with the patches
they have historically used.


Security Implications
=====================

Adding a new platform does not add any new security implications.


How to Teach This
=================

The education needs related to this PEP relate to two groups of developers.

First, developers of *apps* need to know how to build Python into an Android
app, along with their own Python code and any supporting packages, and how to
use them all at runtime. The documentation will cover this in a similar form to
the existing `Windows embeddable package
<https://docs.python.org/3/using/windows.html#the-embeddable-package>`__.
However, it will recommend most developers to use higher-level tools such as
`Briefcase <https://briefcase.readthedocs.io/en/stable/>`__,
`Chaquopy <https://chaquo.com/chaquopy/>`__ and `Buildozer
<https://buildozer.readthedocs.io/en/latest/>`__, all of which already have
comprehensive documentation.

Second, developers of *packages* with binary components need to know how to
build and release them for Android (see `Packaging`_).


Reference Implementation
========================

The `Chaquopy repository
<https://github.com/chaquo/chaquopy/tree/master/target>`__ contains a reference
patch and build scripts. These will have to be decoupled from the other
components of Chaquopy before they can be upstreamed.

`Briefcase <https://briefcase.readthedocs.org>`__ provides a reference
implementation of code to execute test suites on Android devices and emulators.
The `Toga Testbed <https://github.com/beeware/toga/tree/main/testbed>`__ is an
example of a test suite that is executed on the Android emulator using GitHub
Actions.


Copyright
=========

This document is placed in the public domain or under the CC0-1.0-Universal
license, whichever is more permissive.