EP 9 Consider Generator Expressions for Large Comprehensions

LIst comprehensions can cause problems for large inputs by using too much memory.

Generator expressions avoid memory issues by producing outputs one at a time as an iterator.
Generator expressions can be composed by passing the iterator from one generator expression into the for subexpression of another.
Generator expressions execute very quickly when chained together.

Effective python

Generator is lazy evaluation and comprehension is eager evaluation.

>>> comprehension = [x ** x for x in range(10)]
>>> comprehension
[1, 1, 4, 27, 256, 3125, 46656, 823543, 16777216, 387420489]

>>> generator = (x ** x for x in range(10))
>>> generator
<generator object <genexpr> at 0x10afeea98>
>>> next(generator)
1
>>> next(generator)
1
>>> next(generator)
4
>>> next(generator)
27
>>> next(generator)
256
>>> next(generator)
3125
>>> next(generator)
46656
>>> next(generator)
823543
>>> next(generator)
16777216
>>> next(generator)
387420489
>>> next(generator)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

yakshaving Different type of evaluation strategy

Eager evaluation or greedy evaluation
eager evaluation, an expression is evaluated as soon as it is bound to a variable.
Lazy evaluation
delays the evaluation of an expression until its value is needed (non-strict evaluation) and which also avoids repeated evaluations (sharing).
Partial evaluation
Remote evaluation
a general term for any technology that involves the transmission of executable software code from a client computer to a server computer for subsequent execution at the server.
e.g. RPC
Short-circuit evaluation
the semantics of some Boolean operators in some programming languages in which the second argument is executed or evaluated only if the first argument does not suffice to determine the value of the expression Benefit of lazy evaluation
Performance increases by avoiding needless calculations, and error conditions in evaluating compound expressions
The ability to construct potentially infinite data structures
The ability to define control flow (structures) as abstractions instead of primitives

Generator can make a chian of iterator but be careful when you execute it more than once

>>> it = (x * x for x in range(10))
>>> roots = ((x, x ** 0.5) for x in it)
>>> next(it)
0
>>> next(it)
1
>>> next(it)
4
>>> next(it)
9
>>> next(roots)
(16, 4.0)
>>> next(roots)
(25, 5.0)
>>> next(roots)
(36, 6.0)
>>> next(roots)
(49, 7.0)
>>> next(it)
64
>>> next(roots)
(81, 9.0)

Copy iter: itertools.tee