Mailing List Archive

[3.10] bpo-45250: fix docs regarding `__iter__` and iterators being inconsistently required by CPython (GH-29170) (GH-29650)
https://github.com/python/cpython/commit/99aad31b7ad493d4feea04064bcd6b04061477f9
commit: 99aad31b7ad493d4feea04064bcd6b04061477f9
branch: 3.10
author: Brett Cannon <brett@python.org>
committer: brettcannon <brett@python.org>
date: 2021-11-22T15:09:15-08:00
summary:

[3.10] bpo-45250: fix docs regarding `__iter__` and iterators being inconsistently required by CPython (GH-29170) (GH-29650)

It is now considered a historical accident that e.g. `for` loops and the `iter()` built-in function do not require the iterators they work with to define `__iter__`, only `__next__`.
(cherry picked from commit be36e0634060c7d5dee8e8876fb888bbb53d992a)

Co-authored-by: Brett Cannon <brett@python.org>

files:
A Misc/NEWS.d/next/Documentation/2021-10-22-12-09-18.bpo-45250.Iit5-Y.rst
M Doc/c-api/iter.rst
M Doc/c-api/typeobj.rst
M Doc/glossary.rst
M Doc/library/functions.rst
M Doc/library/stdtypes.rst
M Doc/reference/datamodel.rst

diff --git a/Doc/c-api/iter.rst b/Doc/c-api/iter.rst
index f7106f4ef0990..3e388bb917a02 100644
--- a/Doc/c-api/iter.rst
+++ b/Doc/c-api/iter.rst
@@ -9,8 +9,8 @@ There are two functions specifically for working with iterators.

.. c:function:: int PyIter_Check(PyObject *o)

- Return non-zero if the object *o* supports the iterator protocol, and ``0``
- otherwise. This function always succeeds.
+ Return non-zero if the object *o* can be safely passed to
+ :c:func:`PyIter_Next`, and ``0`` otherwise. This function always succeeds.

.. c:function:: int PyAIter_Check(PyObject *o)

@@ -21,10 +21,11 @@ There are two functions specifically for working with iterators.

.. c:function:: PyObject* PyIter_Next(PyObject *o)

- Return the next value from the iteration *o*. The object must be an iterator
- (it is up to the caller to check this). If there are no remaining values,
- returns ``NULL`` with no exception set. If an error occurs while retrieving
- the item, returns ``NULL`` and passes along the exception.
+ Return the next value from the iterator *o*. The object must be an iterator
+ according to :c:func:`PyIter_Check` (it is up to the caller to check this).
+ If there are no remaining values, returns ``NULL`` with no exception set.
+ If an error occurs while retrieving the item, returns ``NULL`` and passes
+ along the exception.

To write a loop which iterates over an iterator, the C code should look
something like this::
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index faef6ab5fed50..e491c26f981b5 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -1520,9 +1520,9 @@ and :c:type:`PyType_Type` effectively act as defaults.)

.. c:member:: getiterfunc PyTypeObject.tp_iter

- An optional pointer to a function that returns an iterator for the object. Its
- presence normally signals that the instances of this type are iterable (although
- sequences may be iterable without this function).
+ An optional pointer to a function that returns an :term:`iterator` for the
+ object. Its presence normally signals that the instances of this type are
+ :term:`iterable` (although sequences may be iterable without this function).

This function has the same signature as :c:func:`PyObject_GetIter`::

@@ -1535,8 +1535,8 @@ and :c:type:`PyType_Type` effectively act as defaults.)

.. c:member:: iternextfunc PyTypeObject.tp_iternext

- An optional pointer to a function that returns the next item in an iterator.
- The signature is::
+ An optional pointer to a function that returns the next item in an
+ :term:`iterator`. The signature is::

PyObject *tp_iternext(PyObject *self);

@@ -2428,8 +2428,8 @@ Async Object Structures

PyObject *am_await(PyObject *self);

- The returned object must be an iterator, i.e. :c:func:`PyIter_Check` must
- return ``1`` for it.
+ The returned object must be an :term:`iterator`, i.e. :c:func:`PyIter_Check`
+ must return ``1`` for it.

This slot may be set to ``NULL`` if an object is not an :term:`awaitable`.

diff --git a/Doc/glossary.rst b/Doc/glossary.rst
index 40b57bd41e8c6..1bbd05a5bcecd 100644
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -669,6 +669,11 @@ Glossary

More information can be found in :ref:`typeiter`.

+ .. impl-detail::
+
+ CPython does not consistently apply the requirement that an iterator
+ define :meth:`__iter__`.
+
key function
A key function or collation function is a callable that returns a value
used for sorting or ordering. For example, :func:`locale.strxfrm` is
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
index 47ebbd0b509a5..307d67978559c 100644
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -66,9 +66,6 @@ are always available. They are listed here in alphabetical order.
Return an :term:`asynchronous iterator` for an :term:`asynchronous iterable`.
Equivalent to calling ``x.__aiter__()``.

- ``aiter(x)`` itself has an ``__aiter__()`` method that returns ``x``,
- so ``aiter(aiter(x))`` is the same as ``aiter(x)``.
-
Note: Unlike :func:`iter`, :func:`aiter` has no 2-argument variant.

.. versionadded:: 3.10
@@ -929,8 +926,8 @@ are always available. They are listed here in alphabetical order.
Return an :term:`iterator` object. The first argument is interpreted very
differently depending on the presence of the second argument. Without a
second argument, *object* must be a collection object which supports the
- iteration protocol (the :meth:`__iter__` method), or it must support the
- sequence protocol (the :meth:`__getitem__` method with integer arguments
+ :term:`iterable` protocol (the :meth:`__iter__` method), or it must support
+ the sequence protocol (the :meth:`__getitem__` method with integer arguments
starting at ``0``). If it does not support either of those protocols,
:exc:`TypeError` is raised. If the second argument, *sentinel*, is given,
then *object* must be a callable object. The iterator created in this case
@@ -1060,7 +1057,7 @@ are always available. They are listed here in alphabetical order.

.. function:: next(iterator[, default])

- Retrieve the next item from the *iterator* by calling its
+ Retrieve the next item from the :term:`iterator` by calling its
:meth:`~iterator.__next__` method. If *default* is given, it is returned
if the iterator is exhausted, otherwise :exc:`StopIteration` is raised.

diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
index d6eff144cce54..1b18abcd6c95c 100644
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -772,21 +772,21 @@ using two distinct methods; these are used to allow user-defined classes to
support iteration. Sequences, described below in more detail, always support
the iteration methods.

-One method needs to be defined for container objects to provide iteration
+One method needs to be defined for container objects to provide :term:`iterable`
support:

.. XXX duplicated in reference/datamodel!

.. method:: container.__iter__()

- Return an iterator object. The object is required to support the iterator
- protocol described below. If a container supports different types of
- iteration, additional methods can be provided to specifically request
+ Return an :term:`iterator` object. The object is required to support the
+ iterator protocol described below. If a container supports different types
+ of iteration, additional methods can be provided to specifically request
iterators for those iteration types. (An example of an object supporting
multiple forms of iteration would be a tree structure which supports both
breadth-first and depth-first traversal.) This method corresponds to the
- :c:member:`~PyTypeObject.tp_iter` slot of the type structure for Python objects in the Python/C
- API.
+ :c:member:`~PyTypeObject.tp_iter` slot of the type structure for Python
+ objects in the Python/C API.

The iterator objects themselves are required to support the following two
methods, which together form the :dfn:`iterator protocol`:
@@ -794,18 +794,19 @@ methods, which together form the :dfn:`iterator protocol`:

.. method:: iterator.__iter__()

- Return the iterator object itself. This is required to allow both containers
- and iterators to be used with the :keyword:`for` and :keyword:`in` statements.
- This method corresponds to the :c:member:`~PyTypeObject.tp_iter` slot of the type structure for
- Python objects in the Python/C API.
+ Return the :term:`iterator` object itself. This is required to allow both
+ containers and iterators to be used with the :keyword:`for` and
+ :keyword:`in` statements. This method corresponds to the
+ :c:member:`~PyTypeObject.tp_iter` slot of the type structure for Python
+ objects in the Python/C API.


.. method:: iterator.__next__()

- Return the next item from the container. If there are no further items, raise
- the :exc:`StopIteration` exception. This method corresponds to the
- :c:member:`~PyTypeObject.tp_iternext` slot of the type structure for Python objects in the
- Python/C API.
+ Return the next item from the :term:`iterator`. If there are no further
+ items, raise the :exc:`StopIteration` exception. This method corresponds to
+ the :c:member:`~PyTypeObject.tp_iternext` slot of the type structure for
+ Python objects in the Python/C API.

Python defines several iterator objects to support iteration over general and
specific sequence types, dictionaries, and other more specialized forms. The
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
index f480915eeb50b..111417d937fbb 100644
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -648,13 +648,13 @@ Callable types

A function or method which uses the :keyword:`yield` statement (see section
:ref:`yield`) is called a :dfn:`generator function`. Such a function, when
- called, always returns an iterator object which can be used to execute the
- body of the function: calling the iterator's :meth:`iterator.__next__`
- method will cause the function to execute until it provides a value
- using the :keyword:`!yield` statement. When the function executes a
- :keyword:`return` statement or falls off the end, a :exc:`StopIteration`
- exception is raised and the iterator will have reached the end of the set of
- values to be returned.
+ called, always returns an :term:`iterator` object which can be used to
+ execute the body of the function: calling the iterator's
+ :meth:`iterator.__next__` method will cause the function to execute until
+ it provides a value using the :keyword:`!yield` statement. When the
+ function executes a :keyword:`return` statement or falls off the end, a
+ :exc:`StopIteration` exception is raised and the iterator will have
+ reached the end of the set of values to be returned.

Coroutine functions
.. index::
@@ -674,7 +674,7 @@ Callable types
A function or method which is defined using :keyword:`async def` and
which uses the :keyword:`yield` statement is called a
:dfn:`asynchronous generator function`. Such a function, when called,
- returns an asynchronous iterator object which can be used in an
+ returns an :term:`asynchronous iterator` object which can be used in an
:keyword:`async for` statement to execute the body of the function.

Calling the asynchronous iterator's :meth:`aiterator.__anext__` method
@@ -2464,12 +2464,10 @@ through the object's keys; for sequences, it should iterate through the values.

.. method:: object.__iter__(self)

- This method is called when an iterator is required for a container. This method
- should return a new iterator object that can iterate over all the objects in the
- container. For mappings, it should iterate over the keys of the container.
-
- Iterator objects also need to implement this method; they are required to return
- themselves. For more information on iterator objects, see :ref:`typeiter`.
+ This method is called when an :term:`iterator` is required for a container.
+ This method should return a new iterator object that can iterate over all the
+ objects in the container. For mappings, it should iterate over the keys of
+ the container.


.. method:: object.__reversed__(self)
diff --git a/Misc/NEWS.d/next/Documentation/2021-10-22-12-09-18.bpo-45250.Iit5-Y.rst b/Misc/NEWS.d/next/Documentation/2021-10-22-12-09-18.bpo-45250.Iit5-Y.rst
new file mode 100644
index 0000000000000..0c2bf18b20010
--- /dev/null
+++ b/Misc/NEWS.d/next/Documentation/2021-10-22-12-09-18.bpo-45250.Iit5-Y.rst
@@ -0,0 +1,2 @@
+Update the documentation to note that CPython does not consistently
+require iterators to define ``__iter__``.

_______________________________________________
Python-checkins mailing list
Python-checkins@python.org
https://mail.python.org/mailman/listinfo/python-checkins