RUF017 Avoid quadratic list summation: consider itertools.chain.from_iterable

[RonnyPfannschmidt]

currently RUF017

takes

mess = [[1], [1,2],[1], [1,2],[1], [1,2]]

tags = sorted(sum([x for x in mess], []))

and turns it into

import functools
import operator
mess = [[1], [1,2],[1], [1,2],[1], [1,2]]

tags = sorted(functools.reduce(operator.iadd, [x for x in mess], []))

instead of the more idiomatic

import itertools

mess = [[1], [1,2],[1], [1,2],[1], [1,2]]

tags = sorted(itertools.chain.from_iterable(mess))

based on the applications of the pattern i have seen so far, i would almost always recommend to replace
the sum with a list(chain.from_iterable(...))

with the additional "optimization" of in-lining list comprehensions and/or generator expressions when applicable

[charliermarsh]

This makes sense... We could at least do this for cases in which it's directly within a sorted or similar.

[charliermarsh]

Interestingly, though, the functools.reduce variant does seem to be quite a bit faster.

import timeit

code1 = """
tags = sorted(itertools.chain.from_iterable(mess))
"""

code2 = """
tags = sorted(functools.reduce(operator.iadd, [x for x in mess], []))
"""

repeats = 10000

time1 = timeit.timeit(stmt=code1, setup='import itertools\nmess = [[i, i + 1] for i in range(10000)]', number=repeats)
print(f"{time1} seconds")

time2 = timeit.timeit(stmt=code2, setup='import itertools\nimport functools\nimport operator\nmess = [[i] for i in range(10000)]', number=repeats)
print(f"{time2} seconds")

Yields:

4.019917957950383 seconds
2.7585187909426168 seconds

[RonnyPfannschmidt]

I suspect this relates to different optimization levels of the Code paths

[hauntsaninja]

Here's some more benchmarking from when we added this: #5073 (comment)

[Skylion007]

This would be covered under a REFURB/FURB (FURB179) rule when we choose to implement it. #1348

[hauntsaninja]

I'd recommend closing this issue, since the functools variant is meaningfully faster: #5073 (comment)

[charliermarsh]

I agree.