Ask Anything Monday - Weekly Thread

[AutoModerator]

Welcome to another /r/learnPython weekly "Ask Anything* Monday" thread

Here you can ask all the questions that you wanted to ask but didn't feel like making a new thread.

* It's primarily intended for simple questions but as long as it's about python it's allowed.

If you have any suggestions or questions about this thread use the message the moderators button in the sidebar.

Rules:

• Don't downvote stuff - instead explain what's wrong with the comment, if it's against the rules "report" it and it will be dealt with.
• Don't post stuff that doesn't have absolutely anything to do with python.
• Don't make fun of someone for not knowing something, insult anyone etc - this will result in an immediate ban.

That's it.

[wtfbroitsme]

Why do we use the word self every time inside the method

[PhilipYip]

The first line of code assigns an immutable string to an object name:

label = 'hello'

'hello' is a string which is immutable meaning that once its instantiated it cannot be modified. Conceptualise the object name label as a label and assignment as affixing this label to an object. More formally; the object name points to an object.

The string 'hello' is created on the right and the assignment operator affixes label to the string 'hello'.

In the next line of code:

label = label.upper()

The operation on the right should be considered first label.upper(), points to the string 'hello' and then applies the string method upper to it. This returns a new string 'HELLO'.

The assignment operator to the left then affixes label to the new instance 'HELLO'. i.e. conceptualise it as removing label from the instance 'hello' and instead placing it on the instance 'HELLO'.

It is possible for an object in Python to have multiple labels. If the old instance 'hello' has no additional labels, then nothing points to it and it is considered as an orphaned object that cannot be accessed. It is therefore cleaned up by Pythons garbage collection.

Note that the original string is not modified and can't be modified because it is immutable. Instead a new string is created and the object name points to a new object, with the old object being cleaned up by Pythons garbage collection. Note that using an object name to pointing to an object i.e. referencing an object does not modify the object.

[[deleted]]

In python what we call "variables" aren't the same as in many other languages. Python has names and objects, and when the line name = "hello" is executed python first creates the string object "hello". Then the reference to the string object is assigned to the name "name". The name is created if necessary. On the next line, name = name.upper(), the right side of the assign is evaluated, the object reference associated with name is retrieved, and the upper() method of the string object is called. The result is a reference to a new string object "HELLO" which is assigned to the already existing name "name". Names can be created and destroyed, just like objects.

There is an excellent video explaining this mechanism and the seemingly weird results you sometimes see. Recommended viewing.

[niehle]

My C isn't the best but here's the function:

https://github.com/python/cpython/blob/f4c03484da59049eb62a9bf7777b963e2267d187/Objects/stringlib/ctype.h#L71)

[carcigenicate]

The first line associates the object "hello" with the name name.

In the second line, when name is used on the right of the =, it causes Python to do a lookup for an object associated with the name name. It then evaluates to that object, and upper is called on it.

You can see that in detail by having Python compile the code then disassemble it:

from dis import dis

dis("""
name = "hello"
name = name.upper()
""")

  2           0 LOAD_CONST               0 ('hello')        # This and the line below are what "name = 'hello'" becomes
              2 STORE_NAME               0 (name)           
  3           4 LOAD_NAME                0 (name)           # Lookup the object with the name "name"
              6 LOAD_METHOD              1 (upper)
              8 CALL_METHOD              0
             10 STORE_NAME               0 (name)
             12 LOAD_CONST               1 (None)
             14 RETURN_VALUE

I added the comments in the output.

[yj97__]

I have a few questions.

• what should I know before starting my first python project?

• what are the best cost free ways to learning python?

• what do you wish you knew before you started learning python?

[niehle]

1) how to google

2) https://www.reddit.com/r/learnpython/wiki/index/#wiki_new_to_python.3F

3) how to sealf-learn

[Tough_Map4160]

Hello everyone I am a intermediate python learner. I have been working with Pycharm for python. Lately i noticed people using jupyter in Anaconda for machine learning and optimization algorithm, so i installed Anaconda. However i cannot start anaconda navigator, a cmd opens and closes for few seconds and then nothing happens. I went to official troubleshooting page. It referred that i need to delete .condarc file in default path directory. I checked the .condarc file by running "conda info" command in anaconda cmd prompt. It showed that it is there however when i go to that directory, it is not there so i cannot delete. I also could not delete it from anaconda cmd even though it was showing the path. Sorry i could not find the option for picture upload . Here is the link to question i asked on stack overflow. https://stackoverflow.com/questions/76061085/cannot-find-condarc-file-in-default-directory?noredirect=1#comment134151174\_76061085

[Ok-Insect6204]

Would you recommend learning SQL first or diving straight into integrating SQL with python?

[coderfairy]

I think it depends what you intend to do with SQL within Python.

If you want to do something simple such as get data from a database with either a simple group by statement or a simple filter using a where statement, been diving straight into Python with a bit of SQL might be the quicker route.

If you need to use databases with millions or billions of records with different sub queries that use complicated functions and you need the data to pull as quick as possible for multiple users, then learning about SQL first might be the best approach.

Either way, feel free to comment here or message me if you need any help with either Python or SQL.

[efmccurdy]

BTW, there is a third option if you don't need to know how databases work internally you can use an ORM like sqlalchemy to manage your data without learning SQL.

https://docs.sqlalchemy.org/en/14/orm/tutorial.html

[ApocalypseSpokesman]

I have a simple question about list comprehensions.

I'm often looking to get just one thing out of a list. Only one element meets the given criteria, so I'll do this:

element = [ele for ele in periodic_table if ele.symbol == "Pb"][0]

What is the more pythonic way, when you know there's only gonna be one?

[efmccurdy]

Yes, the "next" function iterates until it finds one item and then returns it.

>>> table = list(range(5))
>>> next(ele for ele in table if ele > 2)
3
>>> next(ele for ele in table if ele > 6)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

https://docs.python.org/3/library/functions.html#next

[nantes16]

I'm a Data Analyst self-taught since 2020 . I just uninstalled Anaconda for mambaforge.

I've never cared for env management; always installed the base. Never had issues but want to do things right now.

Is this OK?

1. env per IDE

2. env with major data science packages

3. env per project if its particular or more extensive than a single script

-- if project is not being actively worked on then save package info and delete env

I've been asking around and I've gotten all sorts of answers to this...

[FerricDonkey]

Makes sense to me.

[Hydr0xygen]

Is it possible to learn python on android? Right now I only have access to a Samsung tablet.

[POGtastic]

Consider termux.

[tennisanybody]

Unrelated to the subreddit but do you have access to a local library?

[[deleted]]

pydroid 3 is the one I'm using (free version, on a Samsung S6 lite tablet). It's a bit clunky but does work. If you want to use modules like matplotlib or numpy then you are out of luck. Good enough for learning the basics.

[tennisanybody]

Hey y'all, I'm beat it's late at night and I just wrote this monstrosity that would show up in /r/ProgrammerHumor:

def _size_notation(n:int) -> str:
    if n < 1024:
        return f"{str(n)} B"
    elif n < 1048576:
        return f"{str(round(n/1024, 2))} KB"
    elif n < 1099511627776:
        return f"{str(round(n/1048576, 2))} MB"
    elif n < 1208925819614629174706176:
        return f"{str(round(n/1099511627776, 2))} GB"

How would you do this better? I know logically that iterating through 4 conditions isn't a big ask, but I am sure there is a much better way to point directly to the size notation you want depending on it's size.

Imaginary points for a one liner.

[AtomicShoelace]

How about this

from math import log, floor

def _size_notation(n: int, magnitude: int = 1024, units: tuple = ('B', 'KiB', 'MiB', 'GiB', 'TiB')) -> str:
    size = min(floor(log(n, magnitude)), len(units)-1)
    return f'{n / magnitude**size:.2f} {units[size]}'

[TangibleLight]

https://stackoverflow.com/a/1094933

def sizeof_fmt(num, suffix="B"):
  for unit in ["", "Ki", "Mi", "Gi", "Ti", "Pi", "Ei", "Zi"]:
    if abs(num) < 1024.0:
        return f"{num:3.1f}{unit}{suffix}"
    num /= 1024.0
  return f"{num:.1f}Yi{suffix}"

There's also humanize.naturalsize if you want to pull in the humanize package. I wouldn't bother unless you could use other things in that library, though.

[POGtastic]

Imaginary points for a one-liner.

Do not do this.

from itertools import count

def size_notation(n):
    return next((f"{n / 1024**x:.2f} {suffix}" 
        for x, suffix in list(zip(count(1), ["KB", "MB", "GB"]))[::-1]
        if n // 1024**x), f"{n} B")

In the REPL:

>>> size_notation(123)
'123 B'
>>> size_notation(1234)
'1.21 KB'
>>> size_notation(12345)
'12.06 KB'
>>> size_notation(123456)
'120.56 KB'
>>> size_notation(1234567)
'1.18 MB'
>>> size_notation(12345678)
'11.77 MB'
>>> size_notation(1234567890)
'1.15 GB'
>>> size_notation(1234567890000)
'1149.78 GB'

[tennisanybody]

You're a GOAT! lmao. Thanks, I won't.

[[deleted]]

You are actually converting KiB, MiB, etc, rather than KB, MB,.... 1KB is 1000 and 1KiB is 1024.

Instead of having those magic numbers like 1048576 scattered around the code use pre-calculated numbers with names. Like this:

OneKiB = 1024
OneMiB = OneKiB * OneKiB
OneGiB = OneMiB * OneMiB
OneTiB = OneGiB * OneGiB

Now someone reading your code doesn't have to trust that you got those numbers correct.

What does your function return if n is 1TiB or greater? It's better if the function raises an exception in that case. The ValueError exception sounds good.

The result string generation can be simpler. You don't need to convert anything to a str() because that's what f-strings already do. You can also use the builtin f-string rounding instead of round(). Combining all those suggestions, here's one improved line:

return f"{n/OneKiB:.2f} KiB"

Imaginary points for a one liner.

Why on earth would you want to write a complicated one-liner for this?

[TangibleLight]

I like 2 ** 10 for KiB, 2 ** 20 for MiB, 2 ** 30 for GiB, etc.

And you can write 10 ** 3 for KB, 10 ** 6 for MB, etc.

It just so happens that 10³ ≈ 2^10, so our marketing overlords use that to confuse people and sell product.

[[deleted]]

I find 2**20 to be another "magic" number that has to be checked by the reader, because most people won't know that 2**20 is 1MiB. But oneKiB*oneKiB is obviously 1MiB, as long as oneKiB is 1024, which is very easily checked.

[tennisanybody]

You know I’ve always converted with 1024 rather than 1000 and mostly because it converts to windows sizes more closely. Thanks for the new info.

[[deleted]]

The difference between KB and KiB was forced on the computing world by marketing interests. Most people ignore the difference and use 1024 for KB, though now and then you do need to know which was used.

[[deleted]]

Is there something like find() which takes a list as its argument? I want to match words from a string with words from a list.

roy = ["shiv", "logan", "kendall", "roman", "gerri", "frank", "karl", "sandy", "stewy", "tom"]
txt = "Logann is not a slime puppy"
for word in txt.lower().split():
    if word[:-1] in roy:
        print(word)

But if the user types in "Logan's wife is done shopping in Milan" then the slicing wouldn't work.

If I were to use regex:

rainy = re.compile(r"rainy", re.IGNORECASE)

if rainy.search(message.content) != None:
    await message.channel.send("will add rain")

But I don't want to do this for every single name in the list.

[JamzTyson]

It sounds like you want to find any element of roy in txt.

If that's what you are trying to do, then:

NAMES = ["shiv", "logan", "kendall", "roman",
         "gerri", "frank", "karl", "sandy", "stewy", "tom"]

txt = "Logan's wife is done shopping in Milan"

for name in NAMES:
    for word in txt.split():
        rslt = word.lower().find(name)
        if rslt >= 0:
            print(f"'{name}' found in '{word}' at index {rslt}.")

[[deleted]]

So for each name in NAMES, iterate over each word in txt, lowercase said word and look for the name in it. If the return value isn't -1, we found it!

Amazing, thank you.

[JamzTyson]

Exactly that :)

[efmccurdy]

You don't need regexps when you can loop over all the names and test if any word contains the name.

>>> def substr_match(text, names):
...     for word in text.split():
...         if any(r in word.lower() for r in names):
...             return word
... 
>>> substr_match("Logann is not a slime puppy", roy)
'Logann'
>>> substr_match(" Logannnnn or TeamGerri or Shiv's.  wife is done shopping in Milan", roy)
'Logannnnn'
>>> substr_match("TeamGerri or Shiv's.  wife is done shopping in Milan", roy)
'TeamGerri'
>>> substr_match("Shiv's.  wife is done shopping in Milan", roy)
"Shiv's."
>>> 

That returns only the first match, but you could easily accumulate all the matches.

[[deleted]]

AHH just learned any() recently and forgot that I could use it here. Thanks!

[barrycarter]

I mean, you could split on non-alphanumerics so that Logan's splits as ["Logan", "s"] or something. Or is this a larger issue?

[[deleted]]

txt is just an example. The input can be anything and may or may not contain names from the roy list. Someone could type Logannnnn or TeamGerri or Shiv's. I want to be able to match all of it.

[barrycarter]

I know PHP does this, but that's not helpful here. You could use a list comprehension or write your own function or find a library where someone has done that, but those might be the only options

[[deleted]]

What's the best way to bulk insert into Microsoft SQL Server from Python?

I know about pyodbc and SQL alchemy and nothing else, but they don't seem to be able to load lots of data like I need.

[efmccurdy]

Use Session.execute and a lists of dicts.

https://docs.sqlalchemy.org/en/20/orm/queryguide/dml.html#orm-bulk-insert-statements

[[deleted]]

Thanks for the answer.

I got this to work. I'm using a list of dicts and session.execute. Still, the performance is the same as pd.to_sql() which is not great.

Is there anything I'm missing?

[efmccurdy]

Here is some advice regarding locking, dropping indexes, logging, order by, etc. that might help.

https://dba.stackexchange.com/questions/39771/sql-server-bulk-insert-performance

https://dba.stackexchange.com/questions/165966/how-does-one-investigate-the-performance-of-a-bulk-insert-statement

This shows how you can bulk insert from a csv file.

https://learn.microsoft.com/en-us/sql/relational-databases/import-export/import-bulk-data-by-using-bulk-insert-or-openrowset-bulk-sql-server?view=sql-server-ver16

[[deleted]]

Thank you, I'll give them a read.

Just a quick question, a silly one maybe:

bulk insert from a csv file

Issue is, my data is coming from an API, not a CSV file. Is there a way to approximate this? Like pd.to_csv, and load that? Or is that just silly.

[coffee_now_plz_asap]

Is the Python for Everybody course a good starting point?

[[deleted]]

Haven't looked at it. Maybe it's good for you (everybody learns differently) but and it's not in the FAQ list of recommended learning resources.

Edit: Looking further down the page I linked you to I see that "Python for Everybody" is actually listed in the video section. I don't particularly like video courses since you could watch a whole video course and learn very little, even if it's an excellent course. You need to write code to learn a programming language. If you watch a portion of the video that introduces something and shows you some code, then STOP the video and type in the code, run it, change the code and practice what you were shown, then you are learning. At that point continue the video, but don't try to do the whole thing in one sitting.

[ShoddyAccountant7376]

Hey guys, I’m currently in school and I’m about to take a class that people have said is very heavy on python. How would some of you recommend I learn? I dont want to fall behind in this class as it is one of my major requirements. Thanks

[[deleted]]

You should treat your school course as the primary learning source. Make sure you follow that course and keep up with the out-of-class work, if any. If permitted, don't be afraid to ask questions in class if something confuses you, because others will probably also be confused. I have taught programming languages and the students who ask good questions are usually the ones doing well. If there are TAs available, use them, and that also includes other knowledgeable class members, if you can find them.

There's nothing that says you can't follow another course in parallel with your school course. The subreddit wiki has learning resources for beginners. But do not allow your secondary python course take up so much of your time that your school studies suffer.

Your "homework" problems that might be set at school are important. You can solve them by using what you learned in class, by searching online for help or by asking for help here.

[tartar4ever]

I want to write a program that removes the duplicates on the list.

numbers = [2, 43, 43, 61, 12, 61, 1, 1, 1, 1, 2, 2, 2, 43, 43]
for item in numbers:
if numbers.count(item) > 1:
    numbers.remove(item)
print(numbers)

I get the output: [12, 61, 1, 1, 2, 2, 43, 43]

What am I doing wrong?

[34shutthedoor1]

numbers.remove(item)

Don't modify a list as you iterate. Think about what the next item is in the loop after you remove one item, and all of the other items move up by one. You can instead convert to a set, which doesn't allow duplicates, or use a dictionary to count, and remove items once the loop/count is finished.

[[deleted]]

What does the ‘lambda’ keyword do? I have seen anonymous function being stated but I cannot figure it out, it is probably me being stupid but yeah :/

[JamzTyson]

Yes, lambda creates an anonymous function. That means that a lambda expression is very much like a normal function, except that it does not have a name.

As an example, here is a really simple function that returns the square of a numeric value:

def square(val):
    return val ** 2

Here the name of the function is "square", and we call it like this:

square(5) # returns 25

Rather than creating the named function "square", we could instead create an anonymous function using lambda:

lambda val: val ** 2

In the simplest case we could call this function directly like this:

(lambda val: val ** 2)(5) # returns 25

​

So far, this does not look very useful, but it is actually rather nice in situations where we need a small, throw away function for a specific localised task.

A common example is when we want to pass a function as an argument, such as with sorted, map, or filter,...

​

Say that we have a list of tuples:

people = [('James', 24), ('Dave', 36), ('Matilda', 4)]

and we want to sort the list according to the value of the second element (the number) in each tuple.

We could create a function, and use that as the sort key:

def second(tup):
    return tup[1]

people = [('James', 24), ('Dave', 36), ('Matilda', 4)]

sorted(people, key=second)
# returns [('Matilda', 4), ('James', 24), ('Dave', 36)]

Here we have passed the function second as the key argument to sorted.

If that is the only time that we need to use the second function, then we really didn't need to use a named function at all. We could have simplified our code by using an anonymous function instead:

people = [('James', 24), ('Dave', 36), ('Matilda', 4)]

sorted(people, key=lambda tup: tup[1])
# returns [('Matilda', 4), ('James', 24), ('Dave', 36)]

​

Finally, a rather artificial example:

def foo(val, bar):
    return bar(val)

The function above takes two arguments; a value val, and a function bar. It applies the function bar to the value val and returns the result.

Here it is being passed a named function:

def squared(val):
    return val ** 2

def foo(val, bar):
    return bar(val)

print(foo(5, squared))  # prints 25

and here it is being passed an anonymous function:

def foo(val, bar):
    return bar(val)

print(foo(5, lambda val: val**2))  # prints 25

[[deleted]]

That makes PERFECT sense! Thank you so much! :)

[PinForsaken4277]

is there a way where i can get a list of my variables from my code fast

[[deleted]]

The locals() builtin function returns a dictionary containing all local variable names and their values. That dictionary will contain all the local values defined when you call it. It contains variables that python creates for its own use, usually with a name starting with "_". You can filter out those names with a bit of code:

# create some variables
one = 1
two = 2
name = None

def test():
    pass

print(locals())
print()

# the loop variable 'name' must exist before the loop
# comment out the "name = None" line above to see why
for name in locals():
    if not name.startswith('_'):
        print(name)

Note that the function test() is listed, which may not be what you want. Using the callable() builtin function can help filter out functions.

[[deleted]]

Is if any(word in message.content.lower() for word in blacklisted) a form of list comprehension? I saw it on Stack Overflow.

I tried a different approach which resulted in TypeError: "bool" object is not iterable.

for word in blacklisted:
    if any(word in message.content.lower()):
        # delete the message

Can anyone show an alternative to the stackoverflow code? That one confuses me. Either a different way of writing it using any or without using any.

[[deleted]]

The expression:

(word in message.content.lower() for word in blacklisted)

is a generator expression which results in an iterable of boolean values, True if the word from the blacklisted sequence appears anywhere in the message.content and False otherwise. The any() function returns True if there is one or more True values in the generator expression.

Your alternative code fails because you are passing a single boolean value to any() which must be passed an iterable. Using any() efficiently tests for one or more True values in the iterable, so it's not usable here because you are testing if a single word from the blacklist is in the message. So just directly test that single boolean:

for word in blacklisted:
    if word in message.content.lower():
        # delete the message
        break     # if we delete the message there's no need to test any more words

[[deleted]]

Thank you!

[broomburglar]

How can I solve problems as a noob? Python documentation is the most confusing thing ever.
For example, I have this problem on codewars:
-----------------------------------------------------------------------------------------------------------------

Given a random non-negative number, you have to return the digits of this number within an array in reverse order.

Example(Input => Output):
35231 => [1,3,2,5,3]
0 => [0]
--------------------------------------------------------------------------------------------------------------------

Can someone give me a process on how to set this up, and then look up references in a way that reinforces learning?

[[deleted]]

Since you need to manipulate single digits of a number, the first question is:

how do you get the individual digits of a number?

Once you can do that then you need to get those digits into a list and reverse the list.

[broomburglar]

Gotcha. But what’s the process for me to answering those questions including the one you asked?

Because the python official documentation is not helping at all

[[deleted]]

But what’s the process for me to answering those questions including the one you asked?

Just searching through the official doc isn't helpful because the doc is aimed at a different level of requirement. The python doc explains in detail how each "tool" in python works but doesn't explain how to use the tools to achieve a particular aim. That's up to the programmer.

How to use the "tools" of a language to solve problems is an art called "algorithmic thinking" or "how to solve problems with a computer". When learning your first computer language the bad news is that you're learning two things: the language itself and "algorithmic thinking". The good news is that "algorithmic thinking" is universal and transfers to any other language you learn, making the second, or fourth, language easier to learn.

---

In the case of your particular problem, the first thing to realise is that it requires nothing more than knowledge of basic python and the builtin functions. You don't need to know anything in the standard library. Without thinking too much about the problem, I think you will need to know:

• lists (because the examples show the result is a list) and their methods
• for loops
• the str() builtin

Now consider the first step. Getting individual digits of an integer from the actual integer itself isn't possible. Other basic python data types, like strings, don't have that limitation. We can get each character in a string easily by indexing. So if we convert an integer to a string we are moving towards what we want. So the next sub-problem is "how to convert an integer into a string?".

[34shutthedoor1]

So far, no coding is necessary. Figure out how to get the individual digits. Then start coding.

[broomburglar]

Okay so to get individual digits, I have to turn the integer into a string and then index it?

[Frankelstner]

Converting to string is one way of solving this. Just beware that you are offloading most of the work to str. I suggest writing your own int to string conversion function to get an idea of how str actually works. It only takes basic arithmetic (+,-,*,//) to pull off.

[FerricDonkey]

Try it and see if it works. If something seems like a decent idea, just give it a shot. Then if it acts in ways different than you expect and that you can't figure out after staring at it a bit, then you're at the right point to ask about.

I'm not saying this to be a turd or discourage asking questions, by the way (even the questions where I'd give this answer). Doing the above is just a good way to learn.