[Grim Archon]

One of the key attractions of most of Python is it's potential for clarity.

As worthy as they are, it is worrying to me that the sexy/cool features that are being added in 2.4 (applies to some of 2.2/2.3 as well) are hard for even the gurus to describe in clear Joe-public terms. By all means study and use them but this is a bit of a plea to avoid them in examples posted here unless you know your audience's knowledge level can cope.

It is for that reasons I generally don't use list comprehensions as I believe they are the exact opposite when it comes to my own code reviews or helping others understand a principle.

Perhaps this is trivial but here is an example of what I mean:
lines = file("myfile").readlines()
This is a very convenient short hand tha we all use, that relies on defaults and the writers (hopefully) full understanding of the file IO process. Unfortunately, it is used too often in examples posted here and elsewhere (I'm guilty too) where the audience is often a complete beginner.

As useful as it is - if it doesn't work as expected for the person who uses the code where do they start debugging?

1. It uses a hidden read access mode 'r'.
2. The file must exist
3. We must have the correct permissions
4. We must be able to read the whole file in one go.
6. The file is opened and closed automatically.


IMO For example code then this line should would only be used is two case:
1. Where you are explicitly comparing the methods (implicit versus explicit)
2. Where it is not relevent to the problem except as a means of supplying data.

As long-winded (old fashioned?) as it may seem:

myfile = file("mytext","r")
lines = myfile.readlines()
myfile.close()

tells the newbie much more about objects,methods and process and the error reports are likely to be more helpful too.
In real production code - it's still good practice - you can control the process with exceptions and error checking to a much finer degree.

Any counter thoughts?

[Strike]

Originally Posted by Grim Archon

It is for that reasons I generally don't use list comprehensions as I believe they are the exact opposite when it comes to my own code reviews or helping others understand a principle.

List comprehensions are actually pretty clear when used intelligently. When used for "python golf", they are nothing but obfuscating.

>>> evens = [x for x in range(100) if x % 2 == 0]
>>> evens
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98]
>>> evens2 = []
>>> for x in range(100):
...     if x % 2 == 0:
...             evens2.append(x)
...
>>> evens2
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98]

The former is pretty succinct to me (without a loss of clarity).

List comprehensions can easily be (and often are) abused for the sake of brevity, but they actually can make things clearer in some cases as well.

[Grim Archon]

evens = [x for x in range(100) if x % 2 == 0]

possibly one of the simplest list comprehension examples - but I think it illustrates my point well.

From the Python documentation:

Each list comprehension consists of an expression followed by a for clause, then zero or more for or if clauses. The result will be a list resulting from evaluating the expression in the context of the for and if clauses which follow it.

not exactly Python 101.

To comprehend the comprehension ( ) I (you?) have to first mentally re-write it in the traditional way. This gets easier the more you do it. In the meantime we have left most of the readers behind. [Of course there are many cool points to be had for this].

You need to understand Python well to understand a comprehension. This sort of cancels them out as a useful method for examples except for advanced use. And since at least for me it requires an internal translation step there is a potential for more errors.

And how do I debug a non-trivial comprehension?
Simple -
1. I re-write it in the old fashioned way.
2. Sort out the problems, (sprinkle a few print statements or breakpoints).
3. Re-write again it as the new improved list comprehension.

Do that a few times and you wonder why you bother

Given time I shall probably grow to love these new features.

[Strike]

Originally Posted by Grim Archon

To comprehend the comprehension ( ) I (you?) have to first mentally re-write it in the traditional way. This gets easier the more you do it. In the meantime we have left most of the readers behind. [Of course there are many cool points to be had for this].

Actually, to comprehend it, I just read it out loud as is. It reads pretty much like normal English to me. The only "extra" part you have to do is prepend "a list containing" (which isn't a big stretch, since it's in []s). So,

[x for x in range(100) if x % 2 == 0]

reads as:

A list containing - the surrounding []s
x, for (every) x in range(100) if x modulo 2 equals zero - x for x in range(100) if x % 2 == 0

For people with any symbolic math background, that should be as plain as it gets, and it should be fairly obvious even for those who don't have such a background.

[Grim Archon]

lol - this is hardly math.

>>> [x
     for x in range(100)
	 if x % 2 == 0]

I can do the above to add emphasis and aid readability but it is a compromise. And it still has that nasty little x at the front

Perhaps I should propose Python's LC format should allow optional punctuation as per your example.

[rebbit]

I agree with Strike, and in fact do the same thing when it comes to understanding list comprehensions - just read it out. your extra formatting doesn't really seem necessary to me, list comprehensions are fairly straight forward so long as they're not being used in an attempt to golf code. As Strike said, they read pretty much like english.

I think code examples should be kept pretty simple, but if a list comprehension seems to fit the bill then use it. if the person you're explaining the concept to asks how it works then explain it, or even pre-emptively explain it, but I don't see a great need to shelter people from the little "tricks" of the Python language. you might get lucky and not only teach someone about a fundamental concept but also how list comprehensions work.

[Grim Archon]

I think code examples should be kept pretty simple, but if a list comprehension seems to fit the bill then use it. if the person you're explaining the concept to asks how it works then explain it, or even pre-emptively explain it, but I don't see a great need to shelter people from the little "tricks" of the Python language. you might get lucky and not only teach someone about a fundamental concept but also how list comprehensions work.

Fair comment

Time for me to "get with the program" I guess

Grim

[Strike]

Originally Posted by Grim Archon

lol - this is hardly math.

>>> [x
     for x in range(100)
	 if x % 2 == 0]

Apparently you haven't done any higher math.

The python list comprehension syntax is more or less a direct translation of set notation.

In set notation, the above would be:

{x: x ∈ [0,100) | x % 2 = 0}

[Grim Archon]

Strike,

I am ashamed to admit I don't remember much about sets - think that might have been in my 'O'-levels in '76 lol. I don't think it was considered higher math then tho

I guess - if you don't use it you lose it.

grim

[rycamor]

While I have to admit that I am relatively new to Python, it is exactly this sort of capability (list comprehension) and other functional features which is winning me over from PHP and Perl. These mean more to me than the object-oriented capabilities. I just love the application of declarative, set-oriented thinking to programming problems. I also tend to like short-hand notations as long as they are somewhat understandable, and I think lines = file("myfile").readlines() fits the bill.

As a side note (since Python is one of the languages that can be used for stored procedures in PostgreSQL), these kinds of features are a real boon to database programming, since the relational model is based on set-oriented thinking. I'm looking forward to using Python 2.4 with PostgreSQL .

[lazy_yogi]

Originally Posted by Strike

Apparently you haven't done any higher math.

Well alteast you're polite
</sarcasm>

[Strike]

As if directly contradicting someone about something when you aren't well-versed on the subject matter is polite ... (i.e., saying something is "hardly math" when you aren't very math-savvy)

[Grim Archon]

For "hardly" read "not very challenging" - an observation rather than a contradiction.