Styjun

I have a tuple, say, atup = (1,3,4,5,6,6,7,78,8) and produced dynamically by list of tuples when iterated (generator yield). Each tuple needs to get converted to list so each elements of tuple can be transformed further and used in a method. While doing this, I was surprised to learn that just doing list(atup) is much faster than using list comprehension like this [i for i in atup]. Here is what I did:

Performance Test 1:

timeit.timeit('list((1,3,4,5,6,6,7,78,8))', number=100000)
0.02268475245609025

Performance Test 2:

timeit.timeit('[i for i in (1,3,4,5,6,6,7,78,8)]', number=100000)
0.05304025196801376

Can you please explain this ?

The list comprehension has to iterate over the tuple at the Python level:

>>> dis.dis("[i for i in (1,2,3)]")
 1 0 LOAD_CONST 0 (<code object <listcomp> at 0x1075c0c90, file "<dis>", line 1>)
 2 LOAD_CONST 1 ('<listcomp>')
 4 MAKE_FUNCTION 0
 6 LOAD_CONST 5 ((1, 2, 3))
 8 GET_ITER
 10 CALL_FUNCTION 1
 12 RETURN_VALUE

list iterates over the tuple itself, and uses the C API to do it without going through (as much of) the Python data model.

>>> dis.dis("list((1,2,3))")
 1 0 LOAD_NAME 0 (list)
 2 LOAD_CONST 3 ((1, 2, 3))
 4 CALL_FUNCTION 1
 6 RETURN_VALUE

The Python-level iteration is more clearly seen in Python 2, which implements list comprehensions in a different fashion.

>>> def f():
... return [i for i in (1,2,3)]
...
>>> dis.dis(f)
 2 0 BUILD_LIST 0
 3 LOAD_CONST 4 ((1, 2, 3))
 6 GET_ITER
 >> 7 FOR_ITER 12 (to 22)
 10 STORE_FAST 0 (i)
 13 LOAD_FAST 0 (i)
 16 LIST_APPEND 2
 19 JUMP_ABSOLUTE 7
 >> 22 RETURN_VALUE

As @blhsing points out, you can get disassemble the code object generated by the list comprehension in Python 3 to see the same thing.

>>> code = compile('[i for i in (1,2,3)]', '', 'eval')
>>> dis(code.co_consts[0])
 1 0 BUILD_LIST 0
 2 LOAD_FAST 0 (.0)
 >> 4 FOR_ITER 8 (to 14)
 6 STORE_FAST 1 (i)
 8 LOAD_FAST 1 (i)
 10 LIST_APPEND 2
 12 JUMP_ABSOLUTE 4
 >> 14 RETURN_VALUE

The list constructor is implemented purely in C and has therefore minimal overhead, while with a list comprehension the Python compiler has to build a temporary function, build an iterator, store the iterator's output as variable i, and load the variable i to append to a list, which are a lot more Python byte codes to execute than simply loading a tuple and calling the list constructor.