After much discussion and preparation, a change has now been made to the Python programming language that allows the Global Interpreter Lock (GIL) to be deactivated. This can be used to remove a brake block for multi-threading applications.
Advertisement
The implementation of the new flags PYTHON_GIL
and -X gil=0
has been added via pull request to the main branch of the CPython project, which is currently at Alpha 5 for Python 3.13.
Thread safety at the expense of efficiency
Python relies on the GIL to guarantee thread safety and prevent race conditions. The latter occur when several processes of a program running in parallel access the same data in order to change it, which can result in different results depending on the process.
The Global Interpreter Lock ensures that only one thread is actively running at a time. Although this prevents race conditions, it also means that Python cannot efficiently use the potential of multiprocessor systems or multi-core processors.
Two PEPs and a survey
Discussions about ending the GIL have been going on for a long time. Python 3.12 allows the implementation of Python Enhancement Proposal (PEP) 684 “A Per-Interpreter GIL” to create sub-interpreters that have independent locks.
In January 2023, Sam Gross from Metas Facebook AI Research, who is involved in the development of PyTorch, among other things, finally launched the Python Enhancement Proposal (PEP) 703 “Making the Global Interpreter Lock Optional in CPython”.
The Python Steering Council conducted a survey in the summer of 2023 to find out whether the majority of the core development team wanted a better multithreading variant and were willing to endure the effort and problems associated with PEP 703 are to be expected in the transition phase. The results of the survey show clear agreement.
Optional waiver
A pull request has now been accepted for the CPython project that introduces flags for the Global Interpreter Lock: PYTHON_GIL=0
or -X gil=0
deactivate the GIL at runtime.
The prerequisite for this is that Python itself has the flag Py_GIL_DISABLED
is created, as stated in the issue “Add a mechanism to disable the GIL”.
In Python 3.13, the Global Interpreter Lock will still be enabled by default and can be disabled manually using the flag. One reason for this is that the development team is expecting some bugs during the transition. Once the issues are resolved, Python should run without GIL by default.
(rme)