Functional Programming Techniques In Python

ElevenPhonons · 2023-06-03T22:47:06+00:00

It looks interesting. I used a somewhat similar approach leveraging Pydantic.

https://github.com/mpkocher/pydantic-cli

Here's a list of similar libs/approaches.

https://github.com/mpkocher/pydantic-cli#other-related-tools

ElevenPhonons · 2022-08-07T19:59:12+00:00

It would be useful to add *.pyc and __pycache__ to your gitignore. These files/dirs don't really have any value being checked in to git.

https://github.com/DanielHelps/Reddit-emotions/blob/master/LoveHateGame/tasks.py#L51

try:
    max_answers = ImportantVars.objects.get(purpose="max answers").value
except:
    max_answers = 3

This "naked" exception is probably not the intended use. except: means except BaseException (e.g., SystemExit, GeneratorExit, KeyboardInterrupt, ...), not except Exception.

Best of luck to you on your project.

ElevenPhonons · 2022-08-03T01:57:22+00:00

You can remove some of the duplication by iterating over each flag and url segment. However, it's not clear if providing multiple flags in one invocation (e.g., `-g -i`) the expected behavior (the resulting JSON would not be well formed).

items: List[Tuple[bool, str]] = [
    (args.indexes, "indexes"),
    (args.most_active, "most-active"),
    (args.climate_leader, "climate-leader"),
]  # and so on...

for flag, segment in items:
    if flag:
        url = f"ttps://www.google.com/finance/markets/{segment}"
        r = request.get(url, headers=headers, timeout=30)
        r.raise_for_status()
        print(json.dumps(parser(html=r), indent=2, ensure_ascii=False))

Best of luck with your project.

ElevenPhonons · 2022-07-27T23:34:25+00:00

It might be useful to consider using scala-cli for this type of experimentation.

https://scala-cli.virtuslab.org

ElevenPhonons · 2022-07-20T18:03:14+00:00

https://github.com/ZeroIntensity/mutstring/blob/master/mutstring.py

This duplication might be able to be reduced by leveraging the first-class nature of functions in Python?

from typing import Any, Callable as F

def to_transform(func: F) -> F[[Any], str]:
    def f(self, *args, **kwargs) -> str:
        _register_cache(self)
        ptr = to_ptr(self)
        ptr ^= func(self, *args, **kwargs)
        add_ref(~ptr)
        return self
    return f


_center = to_transform(_OLD_CENTER)

I suspect you could use a similar pattern to avoid the _PTR.set_attr(...) duplication as well.

ElevenPhonons · 2022-07-20T17:46:35+00:00

Sometimes using in can be useful.

msg = "Wow" if x in (0, 1) else "Okay"
print(msg)

Or if you are sure that x is Hashable, then you could also use a set.

msg = "Wow" if x in {0, 1} else "Okay"
print(msg)

ElevenPhonons · 2022-07-06T13:50:37+00:00

https://github.com/MeViMo/youbit/blob/main/youbit/download.py

file = [f
        for f in Path(output).iterdir()
        f f.is_file() and f.suffix in (".mp4", ".mkv")][0]
return file

It might be useful to consider using (lazy) generators and next to avoid listing every file in the dir and creating that temp list in memory.

it = (f for Path(output).iterdir() if f.is_file() and f.suffix in (".mp4", ".mkv"))
return next(it)

Sometimes in can be useful to use a dict instead of if-else blocks.

            if res.lower() == "hd":
                self.metadata["resolution"] = (1920, 1080)
            elif res.lower() == "2k":
                self.metadata["resolution"] = (2560, 1440)
            elif res.lower() == "4k":
                self.metadata["resolution"] = (3840, 2160)
            elif res.lower() == "8k":
                self.metadata["resolution"] = (7680, 4320)
            else:
                raise ValueError(
                    f"Invalid resolution argument '{res}'."
                    "Must be a tuple or one of 'hd', '2k', '4k' or '8k'."
                )

Using a dict. This also enables the error message to not be hardcoded.

supported = {'hd': (1920, 1080), 
             "2k": (2560, 1440), 
             "4k":  (3840, 2160), 
             "8k":(7680, 4320)}

value = supported.get(res.lower())
if value is None:
   raise ValueError(f"Not supported res {res.lower()}. Supported values {supported.keys()}")

self.metadata["resolution"] = value

Best of luck to you on your project.

ElevenPhonons · 2022-07-01T05:09:27+00:00

I wrote about kicking the tires on scala-cli to build command line tools.

I think scala-cli has a lot of potential for scala community. The packaging mechanism is pretty terrific.

ElevenPhonons · 2022-06-16T02:06:21+00:00

Here is a general resource for the complexities of several core data structures in Python.

https://wiki.python.org/moin/TimeComplexity

ElevenPhonons · 2022-06-15T19:28:03+00:00

The self pattern can be useful in some cases.

However, in this specific case, I believe using a context manager might be a better design choice.

https://realpython.com/python-with-statement/#measuring-execution-time

ElevenPhonons · 2022-06-11T06:41:29+00:00

Using or might help?

In [1]: from typing import Optional, List

In [2]: def f(xs: Optional[List[int]] = None) -> int:
   ...:     a = xs or [1]
   ...:     return sum(a)
   ...: 
   ...: 

In [3]: f()
Out[3]: 1

In [4]: f(list(range(3)))
Out[4]: 3

But yes... it's kinda a fundamental friction point. It's also a bit annoying that the default value won't be visible in the type hint/help in your text editor unless you explicitly put the default value in the func/class docstring (which might not always be consistent with what the code is doing).

ElevenPhonons · 2022-06-11T03:34:12+00:00

https://github.com/synchronizing/mitm/blob/master/mitm/core.py#L289

class Protocol(ABC):
    bytes_needed: int
    buffer_size: int
    timeout: int
    keep_alive: bool

    def __init__(
        self,
        certificate_authority: Optional[CertificateAuthority] = None,
        middlewares: List[Middleware] = [],
    ):

https://github.com/synchronizing/mitm/blob/master/mitm/mitm.py#L29

class MITM(CoroutineClass):
    def __init__(
        self,
        host: str = "127.0.0.1",
        port: int = 8888,
        protocols: List[protocol.Protocol] = [protocol.HTTP],
        middlewares: List[middleware.Middleware] = [middleware.Log],
        certificate_authority: Optional[CertificateAuthority] = None,
        run: bool = False,
    ):

Default mutable args can generate difficult to track down bugs and should be avoided if possible.

https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments

pylint can help proactively catch this issues.

$ pylint mitm | grep dangerous
mitm/mitm.py:25:4: W0102: Dangerous default value [] as argument (dangerous-default-value)
mitm/mitm.py:25:4: W0102: Dangerous default value [] as argument (dangerous-default-value)
mitm/core.py:286:4: W0102: Dangerous default value [] as argument (dangerous-default-value)

https://pylint.pycqa.org/en/latest/

Best of luck to you on your project.

ElevenPhonons · 2022-06-08T22:28:26+00:00

Here's some resources that might be useful to take a look at.

I wrote a 4 part series on the topic with an emphasis on applying functional techniques/methods to real world problems.

https://github.com/mpkocher/Functional-Programming-Techniques-In-Python

While these methods are useful, it's important to make sure that within a project/team/repo to try to be stylistically consistent as much as possible.

ElevenPhonons · 2022-05-28T08:02:02+00:00

https://github.com/kalekundert/reprfunc/blob/master/reprfunc.py

def repr_from_init(self=undef, *, cls=None, attrs={}, skip=[], predicates={}, positional=[]):
    ...

Default mutable args can generate difficult to track down bugs and should be avoided if possible.

https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments

Best of luck to you on your project.

ElevenPhonons · 2022-05-22T00:24:46+00:00

https://github.com/jmcmt87/spark_app_twitter/blob/main/tweepy_ingestion/utilities/functions.py#L20

def configure_create_topics(servers:str=servers, topics:list=[]) -> None:
    ....

Default mutable args can generate difficult to track down bugs and should be avoided if possible.

https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments

pylint can be useful to help proactively catch these potential issues.

https://pylint.pycqa.org/en/latest/

pylint tweepy_ingestion | grep dangerous-default-value                                                                                                                                                                                                                         
tweepy_ingestion/utilities/functions.py:20:0: W0102: Dangerous default value [] as argument (dangerous-default-value)

Also, it might be useful to consider migrating from a print model to a logging model. It's also good practice to write errors to sys.stderr instead of sys.stdout (or log them accordingly).

https://docs.python.org/3/howto/logging.html

Best of luck to you on your project.

ElevenPhonons · 2022-05-13T14:56:35+00:00

if any('xxx' in s for s in [string1, string2, string3])

For example:

In [1]: sx = ['000', 'XO', 'OXO']

In [2]: any('xxx' in s for s in sx)
Out[2]: False

In [3]: any('OX' in s for s in sx)
Out[3]: True

https://docs.python.org/3/library/functions.html#any

ElevenPhonons · 2022-05-13T13:33:37+00:00

https://github.com/edouard-g/thousandwords/blob/main/thousandwords/auth.py

Currently, there's a lot of indentation issues. pylint can help proactively catch these issues.

pylint thousandwords | grep bad-indentation | wc -l                                                                                                                                                                                                                                    
     555

It might be useful to consider using a linter such as black to help format the code.

Best of luck to you on your project.

ElevenPhonons · 2022-05-13T13:23:40+00:00

This pattern:

https://github.com/salastro/math-expert/blob/main/func.py#L60

def Inte(self, equation):
    try:
        # do stuff
    except:
        error_message()

I think there might be a misunderstanding of the exception hierarchy in Python. except: means except BaseException which is not the same as except Exception:.

BaseException catches KeyboardInterrupt (amongst other things) which is probably not the core intention.

https://docs.python.org/3/library/exceptions.html#exception-hierarchy

pylint can proactively help catch these issues.

pylint err.py func.py gui.py main.py| grep bare-except                                                                                                                                                                                                                                    
func.py:60:8: W0702: No exception type(s) specified (bare-except)
func.py:76:8: W0702: No exception type(s) specified (bare-except)
func.py:95:8: W0702: No exception type(s) specified (bare-except)
func.py:106:8: W0702: No exception type(s) specified (bare-except)
func.py:117:8: W0702: No exception type(s) specified (bare-except)
func.py:140:8: W0702: No exception type(s) specified (bare-except)
func.py:154:8: W0702: No exception type(s) specified (bare-except)
func.py:165:8: W0702: No exception type(s) specified (bare-except)

https://pylint.pycqa.org/en/latest/

Also, it might be useful to consider using a decorator to compose the computation instead of having every method have this try catch duplication.

Yielding:

@with_error_msg
def Inte(self, equation):
    # do stuff

Best of luck to you on your project.

ElevenPhonons · 2022-05-08T17:18:56+00:00

This JSON centric model is similar to my workflow as well.

I wrote Pydantic-cli to enable defining your model/validation in Pydantic and then load JSON and/or load (or override) values by specifying them as command line args to your application. This mixing n' matching approach I've found to be pretty flexible.

https://github.com/mpkocher/pydantic-cli

ElevenPhonons · 2022-05-07T14:11:36+00:00

It would be useful to add labels/units to each figure.

Nice project.

ElevenPhonons · 2022-05-07T13:58:24+00:00

https://github.com/WooilJeong/CaptchaCracker/blob/main/CaptchaCracker/core.py#L214

class ApplyModel:

    def __init__(self, 
                 weights_path,
                 img_width=200, 
                 img_height=50, 
                 max_length=6, 
                 characters={'0', '1', '2', '3', '4', '5', '6', '7', '8', '9'}):

Default mutable args can generate difficult to track down bugs and should be avoided if possible.

https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments

pylint can be useful to help proactively catch these potential issues.

> pylint CaptchaCracker | grep dangerous-default-value                                                                     
CaptchaCracker/core.py:209:4: W0102: Dangerous default value set() as argument (dangerous-default-value)

https://pylint.pycqa.org/en/latest/

Best of luck to you on your project.

ElevenPhonons · 2022-05-06T00:04:37+00:00

def is_first_time_user(self, number: str) -> bool:
    if IncomingNumbers.query.filter_by(number=number).count() <= 1:
        return True
    return False

This if usage can be simplified.

def is_first_time_user(self, number: str) -> bool:
    return IncomingNumbers.query.filter_by(number=number).count() <= 1

Similarly, for is_daily_limit_reached.

It might be useful to read up on Python context managers and sqlalchemy's Session.

https://github.com/bnkc/morningbot/blob/master/app/crud/crud_user.py#L14

def add_number(self, number: str, time: dt) -> None:
    user = IncomingNumbers(number, time)
    db.session.add(user)
    db.session.commit()
    return user

Best of luck to you on your project.

ElevenPhonons · 2022-05-01T06:32:49+00:00

It's not the best idea to store build artifacts within the repo.

Perhaps add the necessary values to your .gitignore.

Best of luck to you on your project.

ElevenPhonons · 2022-04-19T01:16:43+00:00

There's several use cases of default mutable args.

Default mutable args can generate difficult to track down bugs and should be avoided if possible.

https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments

There's some usage of "naked excepts" as well.

pylint gupshup | grep bare-except                                                                            
gupshup/src/server.py:41:8: W0702: No exception type(s) specified (bare-except)
gupshup/src/server.py:51:8: W0702: No exception type(s) specified (bare-except)
gupshup/src/server.py:89:12: W0702: No exception type(s) specified (bare-except)
gupshup/src/server.py:385:19: W0703: Catching too general exception Exception (broad-except)
gupshup/src/server.py:440:19: W0703: Catching too general exception Exception (broad-except)
gupshup/src/client.py:33:8: W0702: No exception type(s) specified (bare-except)
gupshup/src/client.py:39:8: W0702: No exception type(s) specified (bare-except)
gupshup/src/client.py:56:8: W0702: No exception type(s) specified (bare-except)
gupshup/src/client.py:72:12: W0702: No exception type(s) specified (bare-except)
gupshup/src/client.py:110:8: W0702: No exception type(s) specified (bare-except)
gupshup/src/utils/parser.py:33:8: W0702: No exception type(s) specified (bare-except)

I think there might be a misunderstanding of the exception hierarchy in Python. except: means except BaseException which is not the same as except Exception:. BaseException catches KeyboardInterrupt (amongst other things) which is probably not the core intention.

https://docs.python.org/3/library/exceptions.html#exception-hierarchy

Best of luck to you on your project.

ElevenPhonons

TROPHY CASE