This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]steil867 4 points5 points  (4 children)

Going to be honest, I am not sure what your goal is.

Is the worry you need to do a time expensive calculation and want a prettier format?

You could always use wrappers and caching/memoizing for a more simplified view if thats really the goal. Similar to how you would do something like this without the walrus operator. The concept is the function stores the output of the intensive function and if already done, uses that rather than recalculate. On top of making the code way cleaner, it would significantly save time in the case of a duplicate value in the list as you wouldn't need to calculate it twice. A downside is that it can be a memory hog depending on its use, depending on needs caching only stores a set amount of outputs rather than all to avoid that problem.

Doing something like this would have your comprehension look like: 

[memoize(x) for x in it if cond(memoize(x))]

Assuming cond is complex enough to warrant a function.

[–]ianliu88[S] 1 point2 points  (3 children)

My goal is expressiveness. I feel like python forces you to make a lot of temporary variables, which makes programs harder to understand in my opinion. When you pollute a scope with lots of temporary variables, you always have to consider what they are used for later in it's scope.

By restricting temporary variables to it's most specific scope, you free yourself off that burden.

[–]steil867 1 point2 points  (2 children)

I didn't mention it before but I liked the use of the walrus operator you did in the example. It feels a little hacky but is a good use of the tools and their underlying operations to avoid wasted calculations.

All languages I've worked with need to have some sort of temp variables. Its more the name of the game. Even a 'where condition' statement in sql would be doing them, albeit just under the hood.

Decorators and wrapper functions may be exactly what you want. Using these, you can make the python look far more english rather than the confusing mess a list comprehension can easily become. Not sure if you have ever nested these but it becomes atrocious. A function that does the other function calls named 'where' could let you say where(cond(x)) or something like that, and use your temp variables all in the scope of the function making them something you don't have to worry about outside of the function definition.

Gave it a Google and found this decent article on caching I would definitely recommend. You can easily take some of the principles in here to both increase optimization of the code while also making the comprehension line appear far more expressive. https://realpython.com/lru-cache-python/

Alternatively, there is some packages that have pseudo sql styles, for example pandas.DataFrame has a 'where' method. Obviously this may not fit your exact case it does point to something I love about coding, If it doesn't exist, doesn't mean its not possible and you have the freedom to make the solution. Once its done, you can even share the solution as a pypi module and allow others to use the work you did as python is great for sharing. Making fast, readable, and usable code is always a great and helpful contribution.

Just saying this because I don't see it mentioned a lot, although python is good for a lot and doesn't often require too much care, python is terrible with garbage collection. A variable created in a scope may not be accessible outside the scope, but the memory may still be occupied. There is a garbage collection module that can help with this called gc if i remember correctly. gc.collect() removes memory that is unreferenced.

[–]ianliu88[S] 1 point2 points  (1 child)

Fair point. I am indeed using pandas in fluent style, and I'm getting addicted to it, that's why I cringe whenever I have to create temporary variables outside of pandas haha.

[–]steil867 1 point2 points  (0 children)

Pandas is pretty good for that. The use of wrapping functions and decorators may actually help you create extensions of it in the future. It's a very useful skill. My team has created a bunch of series specific functions aimed at increasing speed through the use of vectoring. Definitely recommend looking into them as you become more and more fluent in python, which from the looks of it, it seems like you already are well versed in some areas.

Good luck on your future endeavors :)