126 lines
4.1 KiB
Plaintext
126 lines
4.1 KiB
Plaintext
PEP: 277
|
|
Title: Unicode file name support for Windows NT
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: neilh@scintilla.org (Neil Hodgson)
|
|
Status: Final
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 11-Jan-2002
|
|
Python-Version: 2.3
|
|
Post-History:
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP discusses supporting access to all files possible on
|
|
Windows NT by passing Unicode file names directly to the system's
|
|
wide-character functions.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
Python 2.2 on Win32 platforms converts Unicode file names passed
|
|
to open and to functions in the ``os`` module into the 'mbcs' encoding
|
|
before passing the result to the operating system. This is often
|
|
successful in the common case where the script is operating with
|
|
the locale set to the same value as when the file was created.
|
|
Most machines are set up as one locale and rarely if ever changed
|
|
from this locale. For some users, locale is changed more often
|
|
and on servers there are often files saved by users using
|
|
different locales.
|
|
|
|
On Windows NT and descendent operating systems, including Windows
|
|
2000 and Windows XP, wide-character APIs are available that
|
|
provide direct access to all file names, including those that are
|
|
not representable using the current locale. The purpose of this
|
|
proposal is to provide access to these wide-character APIs through
|
|
the standard Python file object and posix module and so provide
|
|
access to all files on Windows NT.
|
|
|
|
|
|
Specification
|
|
=============
|
|
|
|
On Windows platforms which provide wide-character file APIs, when
|
|
Unicode arguments are provided to file APIs, wide-character calls
|
|
are made instead of the standard C library and posix calls.
|
|
|
|
The Python file object is extended to use a Unicode file name
|
|
argument directly rather than converting it. This affects the
|
|
file object constructor ``file(filename[, mode[, bufsize]])`` and also
|
|
the ``open`` function which is an alias of this constructor. When a
|
|
Unicode filename argument is used here then the ``name`` attribute of
|
|
the file object will be Unicode. The representation of a file
|
|
object, ``repr(f)`` will display Unicode file names as an escaped
|
|
string in a similar manner to the representation of Unicode
|
|
strings.
|
|
|
|
The ``posix`` module contains functions that take file or directory
|
|
names: ``chdir``, ``listdir``, ``mkdir``, ``open``, ``remove``, ``rename``,
|
|
``rmdir``, ``stat``, and ``_getfullpathname``. These will use Unicode
|
|
arguments directly rather than converting them. For the ``rename`` function, this
|
|
behaviour is triggered when either of the arguments is Unicode and
|
|
the other argument converted to Unicode using the default
|
|
encoding.
|
|
|
|
The ``listdir`` function currently returns a list of strings. Under
|
|
this proposal, it will return a list of Unicode strings when its
|
|
path argument is Unicode.
|
|
|
|
|
|
Restrictions
|
|
============
|
|
|
|
On the consumer Windows operating systems, Windows 95, Windows 98,
|
|
and Windows ME, there are no wide-character file APIs so behaviour
|
|
is unchanged under this proposal. It may be possible in the
|
|
future to extend this proposal to cover these operating systems as
|
|
the VFAT-32 file system used by them does support Unicode file
|
|
names but access is difficult and so implementing this would
|
|
require much work. The "Microsoft Layer for Unicode" could be a
|
|
starting point for implementing this.
|
|
|
|
Python can be compiled with the size of Unicode characters set to
|
|
4 bytes rather than 2 by defining ``PY_UNICODE_TYPE`` to be a 4 byte
|
|
type and ``Py_UNICODE_SIZE`` to be 4. As the Windows API does not
|
|
accept 4 byte characters, the features described in this proposal
|
|
will not work in this mode so the implementation falls back to the
|
|
current 'mbcs' encoding technique. This restriction could be lifted
|
|
in the future by performing extra conversions using
|
|
``PyUnicode_AsWideChar`` but for now that would add too much
|
|
complexity for a very rarely used feature.
|
|
|
|
|
|
Reference Implementation
|
|
========================
|
|
|
|
The implementation is available at [2]_.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] Microsoft Windows APIs
|
|
http://msdn.microsoft.com/
|
|
|
|
.. [2] https://bugs.python.org/issue594001
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
fill-column: 70
|
|
End:
|
|
|