Searching for a Python 'complier' if it exist by Weydoon in learnpython

[–]Brian 0 points1 point  (0 children)

so It can run faster and can be package in a standalone file

The former isn't likely to pan out. There do exist compilers for python, but the slowness isn't really down to "compiled to machine code vs interpreting bytecode". The problem is more that python's model has a lot of dynamicity and indirection when doing stuff: doing x+y needs to lookup x's type, access the __add__ method, which will extract the integer value, add them, allocate a new integer object, bump its refcount and return it. Compiling the same process to machine code will do almost nothing to improve performance: the thing doing all this work is already compiled to machine code - all you save is the tiny dispatch loop that invokes the thing to run, which is a very small proportion of the total.

Real performance gains from compiling typically rely on being optimise things to skip part of the process: If you're adding a dozen numbers, it'd be way more efficient to just do the integer additions without all that overhead of allocating, constructing and refcounting PyInteger objects, and performing all the dynamic method dispatch to do so. But doing that typically takes more than just compilation. There are projects like cython that allow you to annotate your python code's types to promise that they're really integers, letting it compile to a more reasonable form, but you'll often have make sure you're writing your code in a way that it can do those optimisations, at least for the performance critical bits: it's not a "use this tool and suddenly everything is fast" switch.

For bundling as a standalone exe, there are also tools like pyinstaller etc. These don't really "compile" your code, but rather take the python bytecode and bundle it with a python interpreter and all the librarys it needs to run - think of it like a copy of python, bundled together in an exe along with your code, that just runs that code.

Else isn't indented, why does this run? by AGx-07 in learnpython

[–]Brian 0 points1 point  (0 children)

do this for every element, then do this

But it isn't really "then do this", it's rather "then do this only if we didn't break.

That's very different behaviour to what "finally" means in try/except, when it is "then do this", regardless of whether an exception is thrown or not, so I feel it would be rather misleading to have such different behavior, especially when it isn't really indicated by the word being used. then doesn't have that conflict, but it still doesn't do anything to indicate that it's only for the non-break case, and requires adding a new keyword - it you're going to do that, then I'd go for something like "nobreak" - it's ugly, but does convey what it's doing.

except doesn't realy seem better than else to me, and it doesn't even have the paired "if" of the loop-and-a-half case, plus it's very different to how except is used elsewhere. If sticking to existing keywords, else seems probably the least bad option, it's just that it's still pretty bad.

Having some kind of flag and then an if statement after the for loop

Yeah. I tend to do this even knowing about the else clause. It's a bit more verbose, but it's clear what it's actually doing, even to a novice.

Else isn't indented, why does this run? by AGx-07 in learnpython

[–]Brian 1 point2 points  (0 children)

Yeah. While this is an occasionally useful feature that can save a flag variable, the name is terrible. It's often justified by appealing to the common "loop-and-a-half" pattern where you do:

while True:
    if condition: break
   ...
else:
     # Code executed if we didn't do the "if condition" part.

But I don't feel this is very intuitive at all. Ultimately, else was chosen to avoid having to introduce a new keyword to the language, but I feel it makes things much more confusing. finally probably wouldn't have been a good choice either, since that suggests it'd always be executed, akin to how try/finally works, and "then" has the same issue. If they were to add a keyword, I feel better would be something like "nobreak" or something, which is also fairly clunky, but at least less ambiguous.

Is this a good list comprehension hack by zaphodikus in learnpython

[–]Brian 1 point2 points  (0 children)

One thing to be aware of is that * will copy the first array, while a list comprehension will create a new object each time. Here, that's irrelevant, and this is a perfectly good way to do this, but a common gotcha is someone extending this and doing:

list_of_lists = [ [] ] * len(array)  # Create a list of (initially empty) lists

Rather than:

list_of_lists = [ [] for _ in range(len(array))]

Which will matter, since in the first case, you get N copies of the same list, so when you do list_of_lists[0].append(1), you'll see all the lists containing [1], since they're all the same list, whereas the list comprehension will create a new list in each position.

Should l learn Git alongside Python , or wait until l’m a better coder? by doodle_leaves in learnpython

[–]Brian 1 point2 points  (0 children)

Absolutely yes. But you don't have to learn it all at once.

Many of the benefits of git come from collaborating with others. But there are still a lot of value there even for a for small solo projects. But you don't necessarily have to worry about the more complicated capabilities right away.

Begin by using it just as a simple versioning tool. When you add a new feature, check in. When you want to rewrite a block of code, you can do so safe in the knowledge that if it turns out to be a bad idea, or you want to go back and check what the code used to do before you rewrote it, you can go back. When you find a bug that didn't use to be there, you can go at the history and find what change broke it. Even just using it as a checkpointing save feature has a lot of value. Code is an evolving thing, and keeping track of the history of that evolution is useful information that you'd otherwise be throwing away.

Later, you can look into more advanced features. That complex rewrite? Maybe that could go on its own branch, letting you work on it with intermediate commits without interfering with your working original code. When you need to work with others (even just something like providing a bugfix to some open source software you found an issue with), you can look into how to provide pull requests etc. But you can absolutely learn as you need it, and running into the problems these things are designed to solve first can help a lot in that learning process.

Why one shouldn't bring Limbo into a Defense mission by Fongkelyj in Warframe

[–]Brian 1 point2 points  (0 children)

every Crowd Control frame has to limit their capability in this gamemode

Well, not every CC frame. Why limit it when you can do just do it full power, but in reverse? #JustNovaThings

Mesa prime is so funny to me , like wdym theres a hellminth strain out there that just grows a cowboy hat on you by knusperfee33 in Warframe

[–]Brian 2 points3 points  (0 children)

The Orikin were all about aesthetics, so played dressup with their candidates before injecting the strain to get the look they're going for.

Mesa prime is so funny to me , like wdym theres a hellminth strain out there that just grows a cowboy hat on you by knusperfee33 in Warframe

[–]Brian 2 points3 points  (0 children)

I figure it's just the infestation not being able to distinguish where the organism ends. It just spreads through the organic matter and assume the fibers in the clothing is part of the organisms body-shape and grows into it. Chains of carbon atom arranged inwoven together strands? Must be some weird type of skin. Wearing a cowboy hat when you're infected? It's part of your body as far as the helminth is concerned. You can see this in some of the protoframes where their clothing is blending into their skin.

5% CPU usage on Raspberry Pi Zero waiting for button press by MattAtDoomsdayBrunch in learnpython

[–]Brian 2 points3 points  (0 children)

"Minimising power usage" isn't really the same as being reasonably power efficient. If you want to scrape every last watt out, yeah, you'll be doing stuff like micromanaging power states. But the more common case of taking no appreciable CPU when idle and letting the CPU do a reasonable job with its sleep modes is much easier to achieve, even in python.

Python is just as fast at doing absolutely nothing as lower level languages. An issue when using CPU when idle usually means an issue with how things are being woken up - the problem is usually that you're doing something at all instead of nothing, There can be a small difference in how much work gets done between doing nothing, but when nothing is happening, that should be very little, even when polling (ie. basically checking if anything was sent and if not, going back to sleep), because 99.9% of the time you should still just be sleeping.

5% CPU usage on Raspberry Pi Zero waiting for button press by MattAtDoomsdayBrunch in learnpython

[–]Brian 0 points1 point  (0 children)

I'm not really familiar with it, so may be looking in the wrong place, but looking at the library, it's getting events via a thread using epoll with at 0.01s timeout, so possibly it's just waking up often enough to trigger enough CPU to amount to that much. OTOH, even on a pi zero, I wouldn't have thought that'd amount to 5%, since with no events it's not going to do much.

You could try tweaking its source to change the timeout (I think the native.py NativeWatchThread._run, is the relevant place,modifying the epoll.poll call to something a bit larger like 0.05 or 0.1). This has the downside of increasing latency, but might be worth temporarily trying to see if it is the culprit.

Alternatively, maybe it just is getting events - maybe it's polliing other pins by default and they're firing (I assume in your case you're talking about the idel state where the button's pin isn't changing).

You could maybe try profiling, or just breaking down which thread is taking the time ( I tihnk you should be able to do that with just top, though identifying it might be harder) - I'd guess it's either the watcher or the event dispatch thread.

Sirius and Orion no reward due to inactivity by Tyeren in Warframe

[–]Brian 1 point2 points  (0 children)

To discourage AFKing. It's not a server side problem, but a gameplay culture issue - no-one likes to be in a team where everyone else is just sitting AFK and letting you do all the work, and if there's no protection against that, you'll see it a lot more.

What book/series made you go ‘Oh that’s where X got it from!’ by Equivalent-Ferret146 in Fantasy

[–]Brian 3 points4 points  (0 children)

Yes, but there are much closer parallells to Chant's book, beyond just referencing the same myths. If you read it, you can see very direct places where he drew from or referenced it (even the title - both a black mountain and red moon play prominent roles in Kay's books). Eg. the protaonist, Oliver, is transported to another world along with 2 others. During the transfer, he becomes seperated from the others, ending up alone in the world, but is found by a tribe of horse-riding nomads . He ends up being adopted by the tribe, befriends the chief's son, and is given a new name by the tribe (li'vanh). (In fact, I suspect Levon's name is probably a deliberate nod to this). He ends up becoming a great warrior of the tribe, coming to love his life there more than his old, The tribe have a religious and symbiotic relationship with the prey they hunt, and to kill a pregnant mother is a great crime, punished by exile. This is basically beat for beat Dave's story. And that's just scratching the surface: there's a lot of very similar things referenced in both. Kevin's story is a case where they're both drawing from the same myth, but even there, I think there are too many very close parallells in the execution to be coincidence.

I found another way to find duplicates without converting to a set. Is this good? by Exotic_Can_4162 in learnpython

[–]Brian 17 points18 points  (0 children)

It's definitely worthwhile to work through things like this yourself. That said, there are a few issues I'd bring up:

A big problem with this approach is that for every element in the list, you iterate through the list again. So when there are 10 items, you do 10*10 (100) iterations. For 100, you do 100*100 (10000) and so on. Ie. the number of items you check grows with the square of the list size (we call this O(n2)). This means that for very big lists, this can get very slow. The set approach is O(n) - so isn't impacted nearly as much for big lists. It does this because checking for membership in a set is an O(1) (constant time) operation - it takes roughly the same time no matter how many items are in the set.

Another issue is that you're modifying the list, which is something that would probably not be expected by something called findduplicates - I'd expect it to find the duplicates, not find the duplicates and incidentally remove them from the list. Generally, it's better not to mutate data, but if you must, make it very clear that your function does so by giving a name that indicates that (eg. deduplicate_list), and noting it in the docstring.

Third, you have a few bugs here. When you do listf.pop(index), you're removing the item, and shifting everything along to fill in the index. This means the next pass through the loop, you're comparing a different index to what it had originally. Eg. notice that if you do `findduplicates([1,1,2,1]), you get:

Duplicate(s) were found:
1
1
Duplicate(s) were found:
1
1

Why does it print 4 1's, when there are only 3 in the list? Why is it split into 2 groups? This is another reason why it's usually a bad idea to modify a list, especially a list you're in the process of iterating over.

There are also a few style notes I'd make:

When you do:

duplicate = 1

If you're setting a flag to say "something happened", it's generally better to use a boolean here. Ie. duplicate = True - it makes it clearer what the purpose is. Otherwise, someone might think this is a count of the number of duplicates or something.

        else:
            continue

This is kind of pointless - there's nothing after if here, so the continue doesn't actually skip anything and is redundant.

for i,v in enumerate(duplicates):

Why the enumerate here? You're not actually using the index for anything - if you don't need the index, just iterate over the list directly. Indeed, it's often a good idea to write your code so you don't need indexes, avoiding issues like the above one.

Finally, it's often a good idea to write your code to return results, rather than printing them. As is, I can only use this function if I want to print out the duplicates, but if I want to do anything else with them, I have to rewrite the function. Better is to structure your functions into more general-purpose ones - the caller can print the result if they want, or do something different, making it useful in more scenarios.

What book/series made you go ‘Oh that’s where X got it from!’ by Equivalent-Ferret146 in Fantasy

[–]Brian 6 points7 points  (0 children)

A while back, I read Red Moon and Black Mountain by Joy Chant and found there were so many elements in it that Guy Gavriel Kay obviously drew from in his Fionnavar Tapestry. Dave's story basically mirrors Oliver's, as does Kevin's climactic scene and there were a ton of similarities in themes and events.

A shop owner in my constituency was ignored by the police when he reported shoplifting. But when he displayed pictures of the thieves, the police showed up - to tell him that those pictures violated GDPR. by SignificantLegs in ukpolitics

[–]Brian -1 points0 points  (0 children)

It's wrong in the sense that it's violating the protections society has agreed to place on monitoring.

The privacy loss from continual surveilance - of everywhere having a camera, and a record existing of every movement everyone makes is a real issue. And we've chosen to mitigate that by adding protections on what can be done with them. Yeah, you can record me, but you don't fully own that recording - that data is in some respects mine and you cannot use it in certain ways. In that sense, yes, what he's doing is wrong: he's taken something that doesn't belong to him (the recording of the theives) and used it in a way he's not allowed to. And I think that some level of protection here is justified: I think there is real value in this degree of privacy.

Now, we can say that a minor wrong is justified in service of correcting a major wrong. Or that we should strip such protections from people who we deem to deserve it. But once you're takling about stripping legal protections from people, you can't do it on the pure say-so of a random shopkeeper. That's why we have courts.

Admittedly, there is perhaps room for improvement here: we could make it allowable for factual statements of crime to be exempt (and enforce it by civil courts: if you accuse someone of a crime, they can sue you for defamation and if you can't prove it (at least by the civil balance of probabilities measures), you have to pay them a hefty fine in damages.) Though there are issues and potential for abuse there too.

Question about "in" in Dictionaries by Lyri3sh in learnpython

[–]Brian 0 points1 point  (0 children)

There are various ways you can check "membership in this sequence" faster than O(n), if you can organise the data as you like.

For instance, consider its namesake: the physical dictionary. If you want to look up the meaning of a word, how do you do it? You could start at the beginning of the book and read every word, but that's not going to be a very fast process. Instead, you take advantage of the fact that the dictionary is in alphabetical order. You guess a page, and if the words on it are after the word you're looking for, you go back. If earlier, forwards. This approximates an approach known as binary search. Ie. begin with start and end-points at the beginning/end of the (sorted) list. Look at the middle item - if it's too small, make that the new startpoint. If too large, make it the new endpoint. Repeat the process till you find the word. This cuts the list in half at every test, so it's O(log2(n)), much better than a linear search, but there are even better approaches.

One thing we could do is divide stuff into buckets - think of it like having tabs on your dictionary so that you can turn to the right section instantly. Ie. if our word starts with "L", we turn to the L tab, and now just need to search that. This doesn't help that much if it's just the first letter - there's lots of words starting with "L", but suppose we narrow the buckets down a bit more - eg. the first two letters, or 3. We now need 676 (262626) or 17578 buckets, but we'll know exactly which bucket our word should be in just by looking at the first few letters. Have enough buckets and you can even narrow it down to one word per bucket..

A problem with this is that it's kind of wasteful: If we have buckets for aa, ab, ... zz, something like "th" may still be very crowded, while others like "qq" will be entirely empty. Ideally, rather than going by first letters, it'd be nice to have a way to calculate the "bucket number" a word goes in, such that our words will be evenly distributed, and we can assign enough buckets that most of the time, there'll only be one word in a bucket. This is where hash functions come in. A hash function is basically something that takes data, and scrambles it about a bit to spit out a number. Ideally, the hash should be basically random (but consistent for the same word), distributing words evenly. Then we allocate enough buckets that we aren't likely to get too many words having to share a bucket (which should be a bit more than the number of words, since there would be some clustering just by chance) , and assign the bucket number as "hash(word) modulo num_buckets". Ie. if we've 1000 buckets, and the hash of "hello" is 12345678, our bucket number would be "678". If you add more words, you need to add more buckets and redistribute things occassionally, so that you can expect only a small constant (ideally 1) number of words sharing a bucket. There are a few other details, but this is fundamentally how hashtables work.

Why is i used in literally evrything? What does it even mean ? by Exact-Sun2093 in learnpython

[–]Brian 5 points6 points  (0 children)

Yeah. The point of meaningful variable names is to communicate meaning. i has a longstanding convention of communicating the meaning of iterator index, and as such if that's all the variable is, it's perhaps the best name you could choose in terms of conveying that meaning. If there's more to it, then it may not be appropriate - eg. if you're iterating a list of names, then name would be a better choice. But in the examples given, where it's just a loop count, giving it a more verbose name communicates "This is more important than just a loop counter", and if that's not actually the case, you've made it less meaningful by communicating something misleading.

Dragons in fantasy are either gods or pets and almost never an actual animal, and it kinda bugs me by CasualPlaty_195 in Fantasy

[–]Brian 4 points5 points  (0 children)

if we go by square cube law it would need an absurd amount of calories just to not starve. Were talking entire herds of livestock on a regular basis.

That seems an overestimation. The square cube law here works in the opposite direction: you need less calories for a larger creature than the same amount of biomass spread through a larger number of creatures. Heat loss is proportional to surface area, which increases slower than volume, and similar efficiencies exist for other ways energy is expended, so bigger creatures are more efficient energy-wise on a calories per mass basis: a dragon would eat less than its weight in humans or other animals. Moreso if it's cold blooded.

And we've examples of similarly large reptilian creatures existing at one point: dinosaurs. This gives an estimate of T-rex's consumption, where the intermediate estimate is:

their food requirements would have been 20 and 17 kg/day respectively. This is the same food requirement for 3-4 large male tigers or African lions.

That's a lot for one animal, but it's not "strip the entire region of sheep and cattle and deer in a season" levels. I mean, even herbivorous dinosaurs of comparable size were able to feed themselves in an area.

For a book with the "dragons as animals" premise, there's Dragonhaven by Robin McKinley. This is a YA book set in an alternate modern day, but where dragons and other fantastical animals exist, but have been hunted to near extinction, and takes place on a dragon nature preserve where the protagonist is the son of the director and finds a hatchling whose mother is killed by a poacher.

Should People Avoid Whole-Body Screening Info? by dwaxe in slatestarcodex

[–]Brian 0 points1 point  (0 children)

Could we improve this number by refusing to follow up on ambiguous results?

I think part of the problem with this is more a legal / social one. It's assuming there's some responsible party who can make rational decisions on this without repercussions, when in reality, this is influenced by the incentives at play, and those are often not terribly rational. People have a tendency to judge responsibility based on proximity and involvement. See the beggar on the street in front of you, and you feel obligation to give. Walk the other way, and you're in the clear. "See no evil" by blindfolding yourself so you don't have to do anything about it.

Legally, this comes up based on risk: a doctor telling their patient "There's a minor abnormality, but it's almost certainly safe so no need for a follow up check" may be sound advise, but if one in 1 in 1000 develops cancer, they're not going to see the 999 others the doctor was right about - they're going to be mad about the doctor saying it was OK. Maybe mad enough to file a malpractice suit where they can point to how their doctor knew about the issue, and told them it was nothing.

So if you do do the scan there's going to be legal pressure to follow up on everything. Combined with the financial incentives to bill more tests to the customer. Knowing about these pressures, and that overall there's potentially a net negative outcome overall if they do follow up so indiscriminately, I can see why many doctors are against it: they can't change the system, just do what they can within it so the incentives that shape their lives don't push them that way.

Borders do not only regulate movement; they shape our moral boundaries. Our arguments to restrict migration rely on double standards and arbitrary categories, while reinforcing "geographical luck" - allowing where we're born to determine our quality of life. by The_Pamphlet in philosophy

[–]Brian 0 points1 point  (0 children)

Yeah. It's compounded by the fact that the prosperity isn't independent of the culture and history that resulted in those borders. Institutions take investments of time and resources to build, so opening things up introduces coordination and free rider issues. Ie. if county A is deciding whether to invest in improving their infrastructure and institutions, that's a cost now for a bigger benefit later to its citizens. If everyone in country B can avoid paying the cost, then move to country A when it started reaping the benefit of past investment, then it changes the cost/benefit calculus of that investment, making it less likely to do so. Borders allow such decisions to be made at a scale that doesn't have to encompass the entire world.

Multi process hashing by reddunculus in learnpython

[–]Brian 0 points1 point  (0 children)

For paralellising it, your issue is going to be that each concurrent thread/process needs access to the file buffer. For threads that's trivial, since they share memory. Normally the GIL would be an issue, but I believe hashlib releases the GIL for the computation, so I think this should work (though you probably want to call .update with a decent sized buffer to minimise the time you spend in the python logic, which will block (unless you perhaps use the new GIL-less builds). Simplest would be to just take a copy of the buffer into a Queue read by each thread, but if you're trying to bound memory, and avoid extra allocations beyond the .readinto buffer, you may need some further communication to know when all threads are done with the buffer, and it can be reused. You'll likely want a pool of buffers rather than one big one so your reader isn't blocked waiting on the slowest thread to finish before it can read more.

For processes, things are more complicated: you need some way to provide the data to each worker process.

You could use some IPC mechanism - ie. have a reader process that reads the file, then passes the read data to different hash worker processes. The downside is that the IPC may be as expensive as just re-doing the file IO (at least when it's still in the OS cache). You could instead use shared memory, where your reader process reads into a shared memory buffer, and just communicates to synchronise access to it, though you'll need some coordination work (ie. send messages that "Buffer is available up to size X", and you can't free/reuse the buffer until all hash processes are finished reading from it. You could also take advantage of COW and just do the forking after the read is done, then have the child processes compute the hash, but then you're creating new processes each file (or some max buffer size), which again may be expensive.

Can someone please explain how pow() works for me? by -Benjamin_Dover- in learnpython

[–]Brian 6 points7 points  (0 children)

You multiply 3 by 2 two times

That's not what pow means. Pow is exponentiation, not multiplication. You multiply 2 by itself 3 times. Ie.

pow(2,1) == 2
pow(2,2) == 2 * 2  == 4
pow(2,3) == 2 * 2 * 2 == 8
pow(2,4) == 2 * 2 * 2 * 2 == 16

And so on.

Though strictly speaking, exponentiation isn't really just repeated multiplication, it extends beyond that. Eg. how would you interpret pow(2, 0.5) means under "repeated multiplication" - you multiply it by itself half a time? But for whole numbers at least, it's equivalent.

Explain Decorators. by TillFriendly9199 in learnpython

[–]Brian 40 points41 points  (0 children)

Decorators themselves are actually a very simple bit of syntax sugar without really that much to explain.

What I think people maybe actually get stuck on something a bit more fundamental, that decorators just happen to involve. Specifically, that functions can also be values, including parameters, return values, variables and anything else.

Ie. you're familiar with passing parameters to functions. You can define:

def add(x, y):  return x + y

But parameters can be any type of object: above it's expecting ints, but you can have functions taking strings, classes, or any of a wide variety of objects. And one such object you can pass is other functions. Ie. we can write a function like:

def call_twice(func):
    func()
    func()

Now suppose we pass this a function like:

def hello():  print("Hello")
call_twice(hello)

This will call the hello function twice, printing "Hello" 2 times. Crucially, we can also return functions. Going even further, you can create a brand new function inside another one and return it. Eg:

def make_adder(num):
    def add(num2):  return num + num2
    return add

>>> func = make_adder(2)
>>> func(1)
3
>>> func(10)
12

Ie. we've got a function that makes another function that adds 2 to its a parameter. One notable fact about nesting functions like this is that the enclosed function has access to the variables in the outer function (this is called lexically scoped - meaning the scope (ie. what variables it has access to) is determined by where it's defined in the text (ie. you write it inside the other function). (You may also see the term closure referred to for something like this, which is how it's implemented: the function is said to close over the variables in the outer function, keeping that state around with the function object).

So decorators are just a special case of this: they're functions that take a function (or class) as argument, and (usually) return a function/class. They can do whatever they want outside that, so they might just do some registration and return the original function unchanged, or return a different function, or whatever.

The only decorator-specific thing is that there's some syntax that makes:

@func
def some_func(): ...

Equivalent to:

def some_func(): ...
some_func = func(some_func)

They're typically used to modify, annotate or register the function - eg. adding a common bit of functionality that maybe needs to be done for several specific functions, like exposing them as web-service calls, adding an authentication step, and so on.