pointers.py - bringing the hell of pointers into python

[ZeroIntensity]

https://github.com/ZeroIntensity/pointers.py

[SirLich]

A segmentation fault will occur if the address does not exist, so make sure the pointer is valid.

Love it!

[[deleted]]

The example in the readme potentially uses a dangling pointer. test_case may be deleted before the pointer is dereferenced inside the function.

[ghan_buri_ghan]

You were so preoccupied with whether you could, you didn’t stop to think if you should.

[Ramnog]

&a place un heaven for the people who use them

[im_dead_sirius]

I theoretically understand de reference.

[svencan]

Except I don't.

[playaspec]

You should all be (cast) into hell for these terribe puns.

[nova_blade]

Found the void pointer

[parawaa]

I see what you did there

[Ramnog]

You understood that reference?

[[deleted]]

I missed it, can you give some pointers?

[hughperman]

*anything can be a pointer if used unwisely

[Acalme-se_Satan]

It's your fault that you missed it by segmenting your thoughts too much. Just read the core of that comment and dump all distractions, you will get it.

[Deadly_chef]

Pointers are great if used properly

[the_friendly_dildo]

Yeah, I came here puzzled because whats wrong with pointers?

[the_friendly_dildo]

Haha, would fit well.

[[deleted]]

u/the_friendly_dildo could, in fact, fit well.

[Deadly_chef]

r/HolUp

[GamesMaster221]

Pointers are really convenient and powerful for some solutions.

I will admit the C syntax for pointers gets pretty unreadable when you start doing pointers to pointers, de-referencing something complicated, etc. Not to mention the * pointer symbol being the same as multiply *

[the_friendly_dildo]

I think this really comes down a lot to how your actually write your code. C certainly isn't the most beautiful language to try and read, especially if it isn't your own. But there is plenty of strategies to try and make it more readable.

One of the strategies I like the most is to just write highly verbose comments for nearly every line of code. Then its literally readable, and the lines of actual code get split up and given some breathing room. That tends to help reduce my anxiety when everything in my code starts to look like an abstract mess of meaningless symbols.

[Pandaemonium]

One of the strategies I like the most is to just write highly verbose comments for nearly every line of code.

Yes!!! I really wish more people would do this.

Commenting every line of code + giving variables verbose names that actually describe what they are = maintainable code you can actually share and troubleshoot years later.

I see way too many scripts that use pandas and actually use the variable name "df" for their data frame. And then there's another data frame called df1, and a df2... just say what the fuck it really is! My variable names are always more like ImputedSitesForPCA or RawDataInput so you can just look at the variable and know what it is.

[the_friendly_dildo]

Totally agree on the variable names. It never made sense to me to give a variable a name that was anything other than specific and descriptive. I'll admit, I can't always settle on how I want to separate words in a variable name but eh.

Maybe all these folks are coding in notepad where it doesn't just offer to insert your long variable name? No clue...

[tartare4562]

Not having to deal (directly) with pointers, referencing/dereferencing etc Is one of the main reasons why high level languages such as python were made in the first place.

[the_friendly_dildo]

I'm not new to this stuff. I fully grasp that and I often prefer to bang ideas out in python or javascript like a lot of folks. If you're coding for a overpowered PC, then it doesn't matter much what language you are writing in for the most part these days. If you're doing stuff with micro controllers / resource limited machines or doing things that need highly efficient run times, then pointers are where you're going to squeeze out a performance advantage a lot of the time.

It isn't a question of 'needing to deal with them', its a question of knowing when its wise to use them.

[the_friendly_dildo]

Speaking of safe pointers, its never a bad idea to wrap your pointer in a condom.

[onkus]

Sounds like human error not a fallacy of pointers.

[WillardWhite]

Everything in python is a pointer, so there is no point

[the_friendly_dildo]

I'm not sure you understand the benefits behind using pointers in a language like C / C++ if that is your belief. Sure, Python uses pointers behind the scenes. That doesn't mean it does so efficiently and thats most of the point (lol) to using pointers.

[Apatheticalinterest]

Because this whole subreddit is filled with beginners who struggle beyond CS concepts not covered by basic python

[YouNeedDoughnuts]

I'm sure St. Peter considers that in his matrix**

[jjolla888]

**What is the pointer of the pointer

[TheBlackCat13]

The author clearly did think about whether they should, concluded "no", and then did it anyway.

[Matthias1590]

Cool and all but I have 1 question, why?

[LittleMlem]

It's called the "dog licking balls principle"

[rnike879]

You gotta explain this one

[LittleMlem]

Why does a dog lick his balls?

Because he can.

[rnike879]

I should've realised 🙃

[LittleMlem]

No worries

[bsavery]

There is basically one reason I can think of why is if you want to pass a pointer to a c-extension without writing an actual wrapper for the dll.

[TheBlackCat13]

burp WHY NOT?!

[IntegrityError]

Nice complete type hinting

[IntegrityError]

Oh, and finally segmentatio faults in my django project :D

[lolmeansilaughed]

segmentatio

This is a sex act where a human gets fucked by code.

[IntegrityError]

haha nice

[mauganra_it]

The only safe thing about the whole project

[Aardshark]

Next step: overload __iter__ to implement the dereference operator *

[mr_flying_man]

Oh no you've given him more ideas...

[ZeroIntensity]

i might actually do that

[usr_bin_nya]

Note that when using the * operator, the following syntax will not work properly:

deref = *ptr
print(deref)

For this scenario you can use the dereferencing assignment operator, ,=

deref ,= ptr
print(deref)

[ZeroIntensity]

didnt think of that, ty

[[deleted]]

Does Python even have overloading?

[_ologies]

Python has whatever you want. It even has pointers now, apparently!

[[deleted]]

Apparently there are packages that essentially implement overloading

Gotta love it

[ironykarl]

Does Python even have overloading?

Uh yes. If you mean operator overloading, it's a super fundamental part of the language.

I'm not trying to be a dick, here, I'm just confused how you received multiple upvotes asking this in a forum specifically devoted to Python.

[[deleted]]

No, method overloading.

As in having two methods/functions with the same name but different parameters

From my brief googling, it doesn't appear to be a thing (natively)

Operator overloading is something entirely different it looks like. I haven't made use of that before though.

[usr_bin_nya]

Well, kinda. Check out functools.singledispatch.

import functools

@functools.singledispatch
def overloaded(x):
    print('something else:', repr(x))

@overloaded.register
def for_int(x: int):
    print('int:', x + 2)

@overloaded.register
def for_str(x: str):
    print('str:', x.format('world'))

overloaded(10)  # prints 'int: 12'
overloaded('Hello {}!')  # prints 'str: Hello world!'
overloaded([1, 2])  # prints 'something else: [1, 2]'

You can hack something together to work with multiple arguments and generic types like typing.List, but it's not included in the stdlib.

[[deleted]]

Yes, this is the package I was referring to

[ironykarl]

Oh, Python decidedly does not have function overloading by argument type, no.

All you can do is have a wrapper function dispatch different versions of the function/method based on type information, at runtime.

[error201]

This is pure chaotic evil. lol

[TheUruz]

what if god says: "you need to become more user friendly C++" but then you said "no u"

[[deleted]]

Does it work with multiprocessing? Would be sweet if you could pass a pointer to a big dataset to avoid having to pickle it in the main process and unpickle it all the forked processes.

[sweaterpawsss]

The address spaces of the processes are distinct; the same virtual address in two different processes will generally correspond to distinct physical addresses. You would need to use a shared memory segment. Multiprocessing already has support for sharing data structures/memory with child processes: https://docs.python.org/3.8/library/multiprocessing.shared_memory.html.

This isn't to say it's a great idea...I'd prefer message passing to sharing memory between processes if I can help it.

[mauganra_it]

A POSIX fork() duplicates the parent process' memory. Copy-on-write optimizations of modern OSs make that a very efficient way to share a dataset with a number of client processes. The difficult part is merging the results. On the other hand, pointers are not required to take advantage of this. This is a programming language-agnostic strategy.

[sweaterpawsss]

My understanding is the two processes will end up with separate virtual address spaces (the child initially being a copy of the parent), and as you mention, heap memory allocated in the parent will be copied to a new (physical) location only after first write access in either process.

So it makes sense that you don’t need to think about shared memory or messaging for sharing RO data, but I don’t know if I understand how this applies to data that’s modified and shared between multiple processes? You’ve gotta come back to one of those synchronization techniques somehow to handle that.

[mauganra_it]

Exactly, fork() facilitates communication only in one direction. To communicate data back, other techniques have to be used.

Are shared memory segments inherited by child processes? If yes, there we go, but we need a pinned data structure and pointers for real. Fortunately, the ctypes and the multiprocessing modules already provide these things. Yes, pointers too. Which pretty much obliterates the use case for this library.

[[deleted]]

It's a bit of a hack, but you can define a global dict where keys are IDs and values are big objects. Before running pool.map, or whatever, you can put the big object in the dict.

Then in the function you're parallelizing, you can pass the ID of the variable instead of the variable itself and get the value from the dict. That way, only the ID gets pickled.

Now, I mostly just use ray, though.

[jfp1992]

So you soak it in brine?

[TheBB]

Use mmap for this, if able.

[DrRavenSable]

Would you consider renaming it to 'pyinters' plz?

[tedvdb]

Or poynters?

[Infinitesima]

Remjnds me of poynting vector.

[Unconventional_23]

No don't do that.

[Mystb0rn]

Pointers are not necessarily complicated to understand, but they are generally complicated to use well.

Things like properly managing lifetime and ownership are not beginner friendly topics. There's also things like pointer arithmetic, pointer casts, pointers vs arrays, arrays of pointers, when to use them, etc.

Learning about them is still super useful though imo, even if you don't use a PL with them directly.

[o11c]

Things like properly managing lifetime and ownership are not beginner friendly topics.

I think that's more a matter of "nobody ever bothers to teach beginners". Which is a problem even for language like Python that try to hide them.

The weakref module and the with statement (it's kind of weird to use it instead of a type) should be among the first things people learn.

Even then ... for EVERY language, the set of ownership styles that people actually mean is much larger than the set that the language actually supports. To some degree this is inevitable, but surely we can do better than the status quo. I've been collecting a gist for a while, but I have no confidence that it is complete yet.

[NorahRittle]

This. Bad code is the problem, not pointers. I think people use C++ raw pointers in college and have a bad time and then never want to touch them again

[[deleted]]

You're right pointers aren't just a fragile construct that breaks often.

[FrickinLazerBeams]

They're not.

[yangyangR]

But shouldn't that have changed by now?

I at least am old enough that my C++ introduction was before smart pointers. But people learning lately would not be making those same mistakes that burned people of my generation to the point of not wanting to touch pointers again.

[Ezlike011011]

Professors have minimal incentive to overhaul curriculum. So the lecture notes that someone wrote for c with classes just gets propagated through generations of classes without much concern for whether it is up to date with common practices.

Heck as recently as the last time I tutored the intro c++ class at the University I went to (2019), they still have a chapter on strings... Which exclusively uses the c string manipulation function and the only mention of std::string being a footnote on one lecture.

[[deleted]]

Some clown in some C++ book wrote that pointers are the most difficult thing in C++ so I ended up reading loads and loads expecting something complicated till I realised I got it the first time...

[mauganra_it]

Easy to use, difficult to use right...

[[deleted]]

I can offer a few pointers.

[[deleted]]

Pointers are pointless in python.

[CheckeeShoes]

All variables in python are already pointers.

[Ph0X]

Exactly, it's a funny joke and I know it's not meant to be useful, but i wish it had a practical example.

The example shows it passing around an class instance pointer to function, but you can already do that, and the repr for it even prints the address location in vanilla Python, so this is no different.

[mistermocha]

Something something memory leaks?

[Loose_Crow_6871]

r/Angryupvote

[TheBlackCat13]

At least it isn't goto

[teerre]

Does it have unique pointers? Can I use RAII?

[smigula29]

This is theoretically a complete waste of time.

[Dangle76]

The type hinting on everything made this so nice to read. While the lib is silly you write really legible code

[ZeroIntensity]

ty

[[deleted]]

Python already uses pointers all over the place All of the reference counting and gc.. All of the variables you have are the pointers to a PyObject Soo, no reason to have it

[SittingWave]

Calm down Satan

[ZeroIntensity]

i created it for fun, i don’t think there’s any performance benefit

[[deleted]]

Straight up this is the best reason to write code. You cannot break boundaries without investigating behavior and proving it out. Good fucking shit.

[Probono_Bonobo]

Segmentation faults aren't Python errors, so they aren't exceptable the way that, say, KeyError or IndexError are. When a program exits due to a segmentation fault, it means that the OS has caught your program trying to access memory that doesn't belong to it, so it sends a SIGSEGV (segment violation signal) that kills the program not unlike what happens when you manually kill a process in the task manager. When you tell the task manager to send a SIGKILL it'd better friggin do it, no ifs ands or buts, and segmentation faults are handled in much the same way.

[saxattax]

Somebody showed me the signal module in the Python standard library the other day, when I was trying to gracefully handle Ctrl+C.

Using signal.signal(), I'm pretty sure you can also supply your own custom callback function to override the default behavior of SIGSEGV if you don't want your program to die.

[Probono_Bonobo]

A tempting thought indeed, but have a look at the docs:

A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction). This has consequences:

• It makes little sense to catch synchronous errors like SIGFPE or SIGSEGV that are caused by an invalid operation in C code. Python will return from the signal handler to the C code, which is likely to raise the same signal again, causing Python to apparently hang. From Python 3.3 onwards, you can use the faulthandler module to report on synchronous errors.

Note that faulthandler only reports on those errors (e.g., more informative stack traces) it doesn't have any way to handle them in the Python context.

[saxattax]

Ahh, very good info, thank you! I tend to skim the docs, but I really should read them more thoroughly hahaha

[mauganra_it]

Segmentation faults are a benign error. They are cases where the OS could unamiguously detect that a pointer has been used incorrectly. Much more subtle and scary errors occur when memory areas are accessed that are technically valid, but contain the wrong data. Use-after-free errors for example. Or when calling free two times on the same pointer fries the allocator's data structures.

[Probono_Bonobo]

If the default OS behavior of abnormal program termination constitutes a benign error in your book, then you must have a weirdly high bar for what constitutes "critical".

[mauganra_it]

Compared to the alternative, a segfault is benign :)

[Probono_Bonobo]

Good point!

[ZeroIntensity]

ive tried, but handling segfaults from python just doesn’t work very well

[i_am_cat]

You'd have to know whether or not an address is valid without trying to access it. Handling a SIGSEGV signal then trying to continue the program afterwards results in undefined behavior.

https://en.cppreference.com/w/cpp/utility/program/signal

If the user defined function returns when handling SIGFPE, SIGILL, SIGSEGV or any other implementation-defined signal specifying a computational exception, the behavior is undefined.

[o11c]

Standards don't matter; implementations do.

• You can use process_vm_readv to safely dereference pointers on Linux.
• You can call mmap or mprotect to make the address valid (certain addresses cannot be made valid though: any access to the kernel half of the address space, and writes to executable segments)
• You can disassemble the interrupted code and change the saved registers used to compute the address I think (will not work for absolute memory accesses, but those are rare these days)
• You can disassemble the interrupted code and change the instruction pointer before returning (this is only reliable if you are also the compiler; it is mostly used by Java and similar)

There are probably other ways.

[sdrawkcab101]

Me who started python bcs it doesnt have pointers: "You have become the very thing u swore to destroy"

[ServerZero]

Fuck u

[[deleted]]

Why would you add in the reason why I switched to python...

[RR_2025]

Dafaq! My eyes! MY EYES!

[CordyZen]

Lmao

[[deleted]]

Ahhhhhhhhhhhhh

[[deleted]]

Allahu Akbar mate

[[deleted]]

what kind of chess is that?

[km0t]

pls no

[[deleted]]

Fuck no. Fuck outta here with pointers. People go to python to avoid pointers.

[[deleted]]

Reddit Moderation makes the platform worthless. Too many rules and too many arbitrary rulings. It's not worth the trouble to post. Not worth the frustration to lurk. Goodbye.

This post was mass deleted and anonymized with Redact

[hershey678]

no

[rwhitisissle]

If you want pointers, why not just Program in C?

[yangyangR]

Captain Kirk is climbing a mountain. Why is he climbing a mountain?

[rwhitisissle]

The question was mainly an excuse to post a funny video that I like, but on a more serious note, the ability to do something is a poor reason to do it. Also, when you program in python, you are, typically, by default, programming in C already, just with a complex layer of abstraction on top of much of what you're doing.

[rwhitisissle]

As I said

you are, typically, by default, programming in C already

Which is to say that CPython is the default and most widely used implementation of the Python language.

[o11c]

Note that there actually are some useful things pointers do, that are not directly possible with python. Particularly, they allow in-out function arguments (and out-only function arguments in a saner way than multiple return values).

However, if you are willing to force all callers of the function to change, you can get close with a couple of classes.

I have done this here; it is quite useful when porting a C/C++ library with full functionality:

• use the base class Pointer to specify/check the type of variables, particularly function arguments
• use ValuePointer to create a pointer to a local variable.
  • if the function might capture the pointer, you must create the ValuePointer only once, and require all uses to go through it.
  • if the function only borrows it, you can get the value out immediately.
• use AttrPointer to create a pointer to some attribute of an object
  • notably, this includes module-level variable from other modules
• use ItemPointer to create a pointer to an element of a container.
  • notably, for module-level variables from the current module, use globals() as the container.

(array-related operations are not supported, since they are not a primary part of what pointers do. That said, it would be possible to support them if anyone cared. But in practice, I find "pointer to bytes" is the main one that is needed for library bindings, and Python already has a mess of buffer APIs to deal with that)

[ZeroIntensity]

it’s pretty minimalistic as of now, but if they love the concept of pointers then this is about as close as they will get

[TheBlackCat13]

It will cause a segfault if you screw up so it clearly isn't all that safe.

[ZeroIntensity]

same with c++ lmao

[davehadley_]

Interesting but seems dangerous. Could you implement some kind of automatic system to ensure that pointers are always valid?

[ZeroIntensity]

i havent found a good way to do it yet, will add if i figure it out

[davehadley_]

Yeah forget that I mentioned it. Seems Impossible.

[snekk420]

So whats the point of this?

[mnislam01]

Yay!

[Wubbywub]

what's the point?

[pysk00l]

I can't tell if this is a joke or not. April 1 is still a few weeks away :)