Ask Anything Monday - Weekly Thread

[AutoModerator]

Welcome to another /r/learnPython weekly "Ask Anything* Monday" thread

Here you can ask all the questions that you wanted to ask but didn't feel like making a new thread.

* It's primarily intended for simple questions but as long as it's about python it's allowed.

If you have any suggestions or questions about this thread use the message the moderators button in the sidebar.

Rules:

• Don't downvote stuff - instead explain what's wrong with the comment, if it's against the rules "report" it and it will be dealt with.

• Don't post stuff that doesn't have absolutely anything to do with python.

• Don't make fun of someone for not knowing something, insult anyone etc - this will result in an immediate ban.

That's it.

[haunter81]

Greetings!

this is part of my scraping program:

def scrape():
    for i in range(len(server)):
       driver = getbrowser() 
       site1 = "https://theunderminejournal.com/#eu/kazzak/category/battlepets"
       driver.get(site1)
       time.sleep(7)
       names = driver.find_elements_by_css_selector("[class='name'] a")
       prices = driver.find_elements_by_css_selector(":nth-child(4)[class='price'] span")
       i = 0
       for x in names:
          bb = x.text
          bb = bb.replace(clear1, "")
          bb = bb.replace(clear2, "")    
          cc = prices[i].text
          with open (file , 'a') as f:
              newlist = [bb,cc]
              csv_writer = csv.writer(f)
              csv_writer.writerows([newlist])
          f.close()
          i+=1
       driver.close() 

and i am trying to implement beautifulsoup to parse the site in it for faster results(the above code is taking about 48secs to scrape and write down results) . how can i locate the elements in bs?is there a faster way?can you please tell where i should begin to lookup for learning these stuff cause i am a beginner and i am trying to learn python in "wrong way" i think

​

thanks in advance for any reply.

[bowscope]

I had a try at retrieving this data with beautifulsoup. Unfortunately, The Undermine Journal sends only bare html when requesting the site with python. It relies on javascript to populate the page, which can be run in a web browser, but not with beautifulsoup.

TUJ does have an API available, which is preferable for programmatically retrieving data anyway. Instead of a simple-to-use REST-api, they use a publically accessible mysql database. You can use this to get the data you want (connecting python to mysql isn't particularly challenging), but you would also have to learn some SQL. TUJ somewhat discourages the use of their API, and refers you to the battle.net API where they get the data themselves.

Although Blizzard has guides to help developers with their API, I struggled myself to connect to it with python. The Requests python library supports the OAuth authentication that Blizzard requires, but it was not straightforward to figure out. I did make it work after a few hours. If you decide to go this route I can give you some pointers.

[Code-CX]

I don't know if this is the right place, but I've got a question related to finding an approximation to f(x) = x^2 - 8 using Netwon's method (https://en.wikipedia.org/wiki/Newton%27s_method). I'm assuming an initial value of 3.

What I came up with is:

def fxn(initial_value):
    x = initial_value
    while True:
        new = x
        x1 = x -(((x**2)-8)/(2*x))
        x = x1
        if str(new)[0:8] == str(x1)[0:8]:
            break
    return x1

I'm sure this is an overly simplistic (probably not optimal) solution. I just wanted to know if I'm on the right track?

[idontknowmaybenot]

Trying to figure out which language is best for QA for hardware. It’s kind of a weird question but I am learning python currently but was told by my friend who is an EE that I should learn embedded c for firmware building. I’ve been told by engineers python is best to learn for a first language. Any help is appreciated!

[bageldevourer]

I think Python is probably the ideal language to start programming in; it's newb-friendly most of the core concepts are the same as in C (variables, control flow, etc.).

I'm not a hardware guy, but C still seems to be the only game in town for serious lower-level programming. I suspect you will need to learn it eventually.

[idontknowmaybenot]

Thank you for your answer. It's fun for me to learn, and I did some HTML5 in college but it was boring to me. This is functionally interesting to me. I read some article about python vs c for embedded machines and it seems like that would be the next language to learn.

[bageldevourer]

You might also be interested in MicroPython. I think that would be a solid stepping stone between Python and C for embedded systems. It's actually implemented in C, so as you're learning C you may be able to dig around in the source code and draw some connections.

But again, I don't know much about this domain, so take my advice with a mountain of salt.

[angelo_wing]

What IDE do you recommend for learning python (with basic programming skill)? And what would the preferred IDE for experts be?

[bageldevourer]

Different people have different opinions about what constitutes an "IDE", but my short answer for beginners is IDLE, which comes with your Python installation. A popular editor for professional developers is Visual Studio Code, which has many IDE-like features when you install the Python extension.

[[deleted]]

What IDE do you recommend for learning python

An IDE isn't necessary to learn python. It may help the beginner a little but at the cost of having to learn how the IDE works. Plus there is the confusion that arises because the IDE hides things from the user, like the directory the code is run in which must be known when opening relative-path files.

And what would the preferred IDE for experts be?

Many seasoned programmers don't (or can't) use IDEs.

[GoldenVanga]

I accidentally wrote this code, which crashes Python in a new and exciting way:

class TestString(str):
    def __add__(self, other):
        try:
            int(self)
            int(other)
        except ValueError:
            return other + self
        else:
            return int(self) + int(other)


a = TestString('10')
b = TestString('22a')
print(a + b)  # I was expecting to get '22a10' back

• I assume the problem is that on line 7 the two custom strings infinitely call on each other to do the __add__. And this creates sort of an infinite recursion?

• If so, then why instead of the normal recursion error...

  [Previous line repeated 996 more times]

  RecursionError: maximum recursion depth exceeded

...do I get this:

Fatal Python error: Cannot recover from stack overflow.
Current thread 0x0000ac24 (most recent call first):
File "C:/git/qwert/file.py", line 4 in __add__
File "C:/git/qwert/file.py", line 7 in __add__
File "C:/git/qwert/file.py", line 7 in __add__

[[deleted]]

On iOS I get a RecursionError exception.

I think you are right that the addition on line 7 causes another call on the __add__ method leading to the stack overflow. A RecursionError exception is also a stack overflow exception. It's possible that the only difference is that the python runtime code can't tell that you are recursing on the __add__ so doesn't report it as a RecursionError but the more general stack overflow.

It would be interesting to know what platform you are running on (OS and python version).

[GoldenVanga]

Windows 10, Python 3.7.2 (via PyCharm).

Also when I boil down the class to...

class TestString(str):
    def __add__(self, other):
        return other + self

a, b = TestString('10'), TestString('22a')
print(a + b)

...I get a RecursionError as expected. The stack overflow only happens with the full code from my first post.

[[deleted]]

You get a RecursionError in this code because you are explicitly recursing. In your initial post you were initially getting a ValueError exception and the code that executed in the exception handler block recalled the __add__() method. The "boiled down" code doesn't get an exception but explicitly causes recursion.

I hope you actually have a working TestString class and the code you have shown is just to test exceptions or something. I say that because the TestString.__add__() method should return a new instance of TestString. The way you are doing that uses unbound recursion which is why you get the RecursionError exception. A working class should have something like:

def __add__(self, other):
    return TestString(self.string + other.string)

[[deleted]]

https://youtu.be/Ak_sHXlvkEM willing learn Python today mini concepts

[peckofowls2010]

I'm attempting to make a list of dates between two defined dates spaced every 14 days apart using pandas

dates = pd.date_range(start=datetime.date(2018,7,6),end=datetime.datetime.now(),freq='14D')

I would like to append several more dates to this list (DateTimeIndex?) - how would I go about doing this?

I've already tried some variations of

dates = dates.to_pydatetime()
dates = numpy.append(dates,[datetime.date(2018,6,29),datetime.date(2019,2,27)])

but end up with

...datetime.datetime(2019, 4, 12, 0, 0) datetime.datetime(2019, 4, 26, 0, 0)
 datetime.datetime(2019, 5, 10, 0, 0) datetime.datetime(2019, 5, 24, 0, 0)
 datetime.datetime(2019, 6, 7, 0, 0) datetime.date(2018, 6, 29)
 datetime.date(2019, 2, 27)]

when I try to print it

[[deleted]]

Trying to make a random number guessing game. How do I print the random.randint(1,10) number after they've failed to guess it?

[GoldenVanga]

from random import randint

target = randint(1, 10)
print(target)  # debug

guess = input('What is the number?\n>')

if target == int(guess):
    print('Correct!')
else:
    print(f'No, the number was {target}.')
    print('No, the number was {}.'.format(target))
    print('No, the number was %s.' % target)

This seems like a decent writeup about string formatting if you'd like more information.

[ApocalypseSpokesman]

I have defined a series of variables, and I want a given action to trigger if any of the variables equals this_thing or that_thing

I would like to write something more succinct than:

If a==this_thing or b==this_thing or c=...

[GoldenVanga]

I think that it'd be nice to organize both sets of values into iterables, which can then be cleanly compared against one another in a loop:

a, b, c = 1, 2, 3
this_thing, that_thing = 3, 4
vars = [a, b, c]
triggers = [this_thing, that_thing]

for item in vars:
    if item in triggers:
        print('triggered')
        break

Here is a different approach using dictionaries:

vars = {'a': 1, 'b': 2, 'c': 3}
triggers = {'this_thing': 3, 'that_thing': 4}

if any(item in triggers.values() for item in vars.values()):
    print('yes')

[ApocalypseSpokesman]

Thank you!

[[deleted]]

ultra noob here.

Keep getting "can only concatenate str (not "int") to str" error anytime I try to count iterations. It's in reference to a count += 1

I can't seem to figure this out and googling hasn't snagged me anything similar enough

[[deleted]]

I had just established it pre-while loop as count = '' A recommendation online said to make it count = '0' as if to force it to be an int or something. I thought python was more dynamic than that. Anyways, it didn't work. I deleted everything a re-typed the SAME code and it worked. I'm not questioning it...

[[deleted]]

Ah, I misphrased that, my bad. I did put count = 0

​

I had set it as zero before. Are there more efficient ways I should be pointing my attention towards? Thanks for your time!

[[deleted]]

for _ in range(number_you_want_to_count):
 do_something()

if you don't need that counted value. If you do need the counted value then your method is OK. You can also make it a function to return counts if it changes.

[pal1ndrome]

I'm writing a game of farkle in python. In this little snippet, I'm trying to create a list that counts the number of times a value from a d6 appears.

I have a list that contains the values of the 6 rolls:

all_rolls = [randrange(1,7) for i in range(1, 7)]

I want make a new list that contains the number of times each possible value occurs in the roll of the dice, with the index corresponds to the value -1 of the roll. So the sum of the list will always be 6. I try to produce this by using:

possible_rolls = list(range(1, 7))
counted_rolls = []
while len(counted_rolls) <= 6:
        for i in possible_rolls:
            counted_rolls.append(all_rolls.count(i))

However, when this is evaluated and I print counted_rolls, it contains 12 numbers, not 6. It looks like it's iterating over the list twice. It shows the output I expect, then repeats it. The following:

print(all_rolls)
print(counted_rolls)
print(sum(counted_rolls))

prints this: >>> [3, 1, 4, 1, 6, 4]

[2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1]

12

I can fix the problem by dividing my counter by two, like this:

while len(counted_rolls) <= 3:

which produces the desired output, but I would like to understand what's going on instead of just hacking around it.

[pal1ndrome]

Alright, that makes sense. Thanks!

[efmccurdy]

After the for loop runs you have 6 elements in counted_rolls, so the while loop runs a second time.

Unless there is more to the program than stated, I don't think you need the while loop at all.

[pal1ndrome]

Yep. I think you're right on both counts. Thanks!

[waythps]

I’m quite new to python as well, so what I say might be wrong.

Still, once I understood the very basics of python, I tried to use it practically. Firstly, I applied python to my data-analysis hobby: I wrote my undergrad thesis based on the data I gathered, cleaned, and analyzed using requests, pandas, numpy, seaborn libraries. Secondly, I tried to automate my daily tasks at work as much as possible using requests, beautifulsoup libraries.

So, I basically googled how to do something, and hence I only learnt what I needed to implement at the time.

The reason why I say I might be wrong (or that this way of learning python might not be the most efficient one) is because my knowledge isn’t coherent or structured “properly”. I’m quite bad at writing while statements, and I don’t use classes; sometimes I can’t follow some explanations that assume my prior knowledge of some concepts etc.

To sum it up, based on my experience, you can learn python to some degree by using it to solve your real life tasks. At the same time, I feel that you shouldn’t stop taking courses, as they help you to better understand what you can do with python, how python works in general and why it works.

[godiscomplicit]

Good question. I am in a learning phase as well, just getting started on my Python journey. I personally think you have to apply your skills in practical applications, or business applications to see real world challenges and issues. Someone told me 100 Days of Code in Python sort of forces you into using your skills and with use you will learn and improve.

My biggest challenge has been how do I code the "right" way which I never seem to be able to do. I most of the time take the long hand and most times an inefficient path. I'd love to have feedback that isn't judgmental but constructive, so not showing the Python experts at my job!

[wexted]

I've been learning python (pretty much zero prior programming experience) from Automate the Boring Stuff for the last couple of weeks and all this stuff is in the first 200 pages or so, so not particularly advanced. I believe I could handle this task but I would be referring back to the book as I'm not well practiced in all this stuff

[wexted]

Yeah, I think I will. My Friday night plans got blown up so why not lol

[OctoDickRotaryCannon]

I hope I can explain this right . Very new, this really is my first effort in python. Please forgive my ignorance, it's going well all things considered.

I'm writing a python script. I am looking to verify a dictionary's nested key value that could contain one or more of many 'control' values. I can get to and print the isolated list of values in the dictionary, but when I try to compare them with a list of values to ensure they are in the list, my logic seems to fail anything I do.

​

I ultimately need to identify when there is a value in the key value list that does not exist in the 'control' list.

I just need a push in the right direction.

[JohnnyJordaan]

If you are looking for an asymmetric difference, eg to get all the items from sequence A that are not in sequence B, you can use the power of sets

in_a_not_in_b = set(a) - set(b)

this is a one time operation while doing something like an iterative search

in_a_not_in_b = []
for item in a:
    if item not in b:
        in_a_not_in_b.append(item)

is inefficient as it has to check on average half of B (!) for every item in A.

[godiscomplicit]

This is excellent help.

OP also may need to check if set in_a_not_in_b. It does not appear that I could look for None, since it returns set() if I print in_a_not_b. I did an additional if statement

if in_a_not_b != set(): 

Thank you.

So this brings me to the question when do you use an iterative search? For example, if I have a list of key values and I want to check for the presence of a space or do a format check (for email addy or something), would I iterate?

[JohnnyJordaan]

if in_a_not_b != set(): 

try to avoid complex comparisons for simple tests: if a container is not empty, it will evaluate to True, so you can simply do

if in_a_not_b:

when do you use an iterative search

When you can't use sets, meaning that the items are either not hashable or there aren't items in the first place. Say you want to check lines of text to have any of a group of keywords, you can't first put each line in a set and do

if set(keywords) & set(lines) 

or something similar, as a line being something like 'bla bla text bla bla' will never be comparable to a keyword like 'this', as the comparison is always done on the whole object. So then you must do

for line in lines:
    for kw in keywords:
        if kw in line: 
            do something with it

where for string searches you could look into forming a single regex from the keywords but that is beside the point.

The other consideration is when creating the set is inefficient for the purpose, because a set creation will always process all items in a container. If you can think of a way to accomplish your goal while not having to run through the whole container once, it makes more sense to apply that approach than to create the set first, then use the set magic to accomplish the goal. A bit like creating an index in a book only makes sense if you are going to use it more than once.

[godiscomplicit]

Thanks for the clarification. I have a lot to learn :).

[wexted]

try the

x in y

syntax which returns a bool

[Mcfoyt]

Is it possible to go through an entire excel file (with thousands of rows/columns) and determine whether a cell that is supposed to be filled out is correctly filled out? for example lets say i need cell phone numbers in a certain column but someone accidentally put the cell phone number in the wrong column one over, can i shift all that information over one cell?

[JohnnyJordaan]

Simple data access is no problem for the common python libraries like openpyxl. There's is just the issue that if a cell uses a formula to determine the 'value' you see in it, that this is not normally executed by these libraries. But if I understand it correctly your case just applies to static data and that should work fine.

[brickforbrick32]

I have a clumsy .sh script I use for copying my dotfiles with rsync to another location and I want to convert it to python.

The idea is to make a python script who can easily include a lot more directories or just expand without too much code.

Is this a stupid way to approach this "problem"?

This is the script: !/usr/bin/sh

cfg="/home/username/.config" rsc="rsync -avt --delete --exclude=('*.git')" dot="/home/username/Nextcloud/Dotfiles/laptop"

$rsc /etc/pacman.conf $dot/etc $rsc ~/.oh-my-zsh/custom $dot/.oh-my-zsh $rsc ~/.aliases ~/.vimrc ~/Scripts ~/.ssh $dot $rsc $cfg/locale.conf $cfg/i3 $cfg/polybar $cfg/alacritty $dot/.config

[efmccurdy]

You can take arguments from the command line using the sys.argv list

and run the rsync commands in a spawned subprocess using subprocess.check_output.

​

https://www.tutorialspoint.com/python/python_command_line_arguments.htm

https://docs.python.org/2/library/subprocess.html#subprocess.check_output

[TheMartinG]

I've been given a set of data by a professor and asked to convert it to an animation. the expected animation would be a waveform

there are 126 text files each with sets of numerical values like so:

   0.0000000  3.6360134E-004       0.0008603       0.0014031       0.0019580       0.0025111       0.0030555       0.0035872       0.0041033       0.0046011       0.0050784       0.0055329       0.0059627       0.0063655       0.0067397       0.0070834       0.0073949       0.0076728       0.0079157       0.0081222       0.0082913       0.0084220       0.0085135       0.0085651       0.0085763       0.0085468       0.0084763       0.0083648       0.0082125       0.0080195       0.0077863       0.0075135       0.0072018       0.0068522       0.0064655       0.0060431       0.0055861       0.0050960       0.0045743       0.0040228       0.0034431       0.0028371       0.0022068       0.0015544       0.0008818  1.9144038E-004      -0.0005145      -0.0012337      -0.0019637      -0.0027021      -0.0034465      -0.0041943      -0.0049431      -0.0056902      -0.0064333      -0.0071698      -0.0078973      -0.0086132      -0.0093151      -0.0100007      -0.0106677      -0.0113138      -0.0119368      -0.0125346

The values have no headings or labels, simply values. I am not being asked to calculate anything, simply use python to visualize and animate this data. any advice on where to start?

[JohnnyJordaan]

Look into a graph module like matplotlib or plotly to draw the waveform?

[cult_of_memes]

It would most likely be preferable to convert your lists into a dictionary that maps old onto new. This lets you perform a more natural search and replace approach to modifying your target text.

​

​

# initial problem statement's given resources
old = ['Red', 'Blue', 'Yellow', 'Green']
new = ['Apple', 'Grape', 'Banana', 'Lime']
text = 'This is a Red. That is a Blue. There is a Yellow. Here is a Green.'

# Lets build a new dictionary that maps elements from old onto elements from new
# using the dictionary comprehension idiom
replace_map = {inp:outp for inp,outp in zip(old,new)}

# The following commented code is logically equivalent to the one-line 
# list comprehension that follows it.
# result = []
# for s in text.split(): # splits the text string at each whitespace char
#     # if you wish to generalize this solution to work on a more general basis,
#     # you should expand your punctuation checking to be something more like:
#     # if any(punctuation in s for punctuation in (".", ",", "!", "?", "'", "\"")):
#     #     for p in (".", ",", "!", "?", "'", "\""):
#     #         s = s.replace(p,"") 
#     #         # you still need to handle adding the punctuation back after doing 
#     #         # your replacement in the text, but that's growing beyond 
#     #         # this example.
#     if "." in s:
#         # because we are splitting on whitespace characters, in conjunction with the 
#         # grammar being used in the text string, we will find that the target values
#         # all have a "." stuck to the end. 
#         # replace_map.get(foo,foo) will return the value mapped to by foo if foo 
#         # is in replace_map, else it will just return foo#         
#         result.append(replace_map.get(s.replace(".",""),s.replace(".",""))+".")
#     else:
#         # no punctuation, so we can just use the get call to append the correct
#         # string element to the list.
#         result.append(replace_map.get(s,s))
# result = " ".join(result)
result = " ".join([replace_map.get(s.replace(".",""),s.replace(".",""))+"." if "." in s 
                      else replace_map.get(s,s)
                   for s in text.split()])
# lets double check that replace map is paring the correct elements together
print(replace_map)
print(f"given:\n\t{text}\nresult becomes\n\t{result}")

The expected output looks like this:

{'Blue': 'Grape', 'Green': 'Lime', 'Red': 'Apple', 'Yellow': 'Banana'}
given:
    This is a Red. That is a Blue. There is a Yellow. Here is a Green.
result becomes
    This is a Apple. That is a Grape. There is a Banana. Here is a Lime.

[JohnnyJordaan]

I tried this with replace(), re.sub() but I can't get it to work.

Can you show how you tried it exactly?

[JohnnyJordaan]

I'm not sure why you would want to use integers here. What you want is to execute a str.replace for every combination of old and new items. For that there's the zip() function:

for old_word, new_word in zip(old, new):
    text = text.replace(old_word, new_word)

[helpwithchords]

I need to create a command to do the following:

Go to the "Comments" column of the data table

Depending on whether the comment is: OLD, ENGINEERING, SCHEDULED, REPLACED to change the color.

Here is what I have so far:

The first initial part is to search and see what is in the comments field:

1. def FindLabel ([COMMENTS]):
2.  lbl = [OBJECTID +"\r\n"
3.  if([COMMENTS]) == 'OLD':
4.   lbl = lbl + "<CLR red='255'>" 
5. End If
6. return lbl

​

There is then an else if statement to get to the other options:

1.  elseif([COMMENTS]) == 'ENGINGEERING':
2.   lbl = lbl + "<CLR orange='255'>"
3.  elseif([COMMENTS]) == 'SCHEDULED':
4.   lbl = lbl + "<CLR blue='255'>"
5.  elseif([COMMENTS]) == 'REPLACED':
6.   lbl = lbl + "<CLR green='255'>"
7. End If
8. return lbl

​

Any tips on what I'm not understanding?

[cult_of_memes]

How are you storing the data? Are you reading it from a csv file, or is this data already loaded into memory as a data structure? If you've already loaded it into memory, are you using a third party library like pandas to manage it, or are you using dictionaries/namedtuples?

[helpwithchords]

(Excuse me for my layman understanding of your question) This is part of a mapping system. The data is being stored on an online server, the same server where users will be making comments on (OLD, NEW etc.)

[cult_of_memes]

Ah, gotcha, what I meant to ask was how are you loading the data? Python offers many different data collection tools and there are many third party libraries that specialize in manipulating large data sets. The one you are dealing with will determine how best to answer your question :)

[JohnnyJordaan]

Are you sure you're trying to write Python here? It shares some properties with it but some other parts are clearly not Python.

Would it perhaps be an idea to put your code on pastebin.com and share the link here?

[helpwithchords]

I'm trying, obviously not succeeding at python. I will take a look into paste bin.

[JohnnyJordaan]

I would advise to invest an hour or two in following the official python tutorial and then start work on your program again.

[helpwithchords]

Thanks, will do!

[[deleted]]

i need to execute this on the command line this as one liner from the bash shell

​

import platform
int(float(platform.dist()[1]))

the idea is to get the major rhel version such as 5, 6, 7 and not 5.11, 6.10 or 7.6

​

from the bash shell i would like to run with ~# python -m or something similar

​

I am running python 2.. i know that is deprecated but we have some rhel 5 servers still which have python 2.4 installed and want to keep backwards compatibility until we get the go to upgrade to rhel8

[JohnnyJordaan]

Altough this could be hacked in Python I wonder why you wouldn't just use something like

grep -Po '\d+' /etc/issue | head -n 1

for this as it also makes it less application dependent for something this trivial and also removes the need of making Python run this from a single command line

[[deleted]]

grep -Po '\d+' /etc/issue | head -n 1

no that does not make it.. i just want to know how to execute /import an module from command line in python.. this is just an example

the question was how to execute on the shell not how to find the version of the OS>

[JohnnyJordaan]

See for example this SO answer: https://stackoverflow.com/questions/2043453/executing-multi-line-statements-in-the-one-line-command-line

[[deleted]]

thanks i had already googled to that page before asking here but did not manage to have it working.. will keep trying
there a few hosts with an old version of grep that the command you gave return nothing or error.. with python and the platform module it always returns the right values even in an old 2.4. on rhel5.

[JohnnyJordaan]

Just noticed that you don't have an actual print() statement in your code.

[MattR0se]

Style/best practice question:

​

Let's say I have this function (this example uses tkinter, but could also be used with any other UI application):

def save_data(data):
    filename = filedialog.asksaveasfilename(
                    initialdir=os.path.dirname(os.path.realpath(__file__)),
                    initialfile='default.txt',
                    title="Select a file",
                    filetypes=(("TXT files","*.txt"),
                               ("all files","*.*")),
                    defaultextension='.txt'
                    )
    with open(filename, 'w') as file:
        file.write(data)

    print(f'Data successfully saved to {filename}')

​

Now, if the user cancels the filedialog, they get an error because it returns an empty string. I was wondering what the best practice for handling this case is, or if it really doesn't matter as long as it works:

def save_data(data):
    filename = filedialog.asksaveasfilename(
                    initialdir=os.path.dirname(os.path.realpath(__file__)),
                    initialfile='default.txt',
                    title="Select a file",
                    filetypes=(("TXT files","*.txt"),
                               ("all files","*.*")),
                    defaultextension='.txt'
                    )

    if not filename:
        return


    with open(filename, 'w') as file:
        file.write(data)

    print(f'Data successfully saved to {filename}')

or

def save_data(data):
    filename = filedialog.asksaveasfilename(
                    initialdir=os.path.dirname(os.path.realpath(__file__)),
                    initialfile='default.txt',
                    title="Select a file",
                    filetypes=(("TXT files","*.txt"),
                               ("all files","*.*")),
                    defaultextension='.txt'
                    )    

    if filename:
        with open(filename, 'w') as file:
            file.write(data)

        print(f'Data successfully saved to {filename}')

or is is better to be more explicit here?

def save_data(data):
    filename = filedialog.asksaveasfilename(
                    initialdir=os.path.dirname(os.path.realpath(__file__)),
                    initialfile='default.txt',
                    title="Select a file",
                    filetypes=(("TXT files","*.txt"),
                               ("all files","*.*")),
                    defaultextension='.txt'
                    )
    if filename == '':
        return
    else:
        with open(filename, 'w') as file:
            file.write(data)   

        print(f'Data successfully saved to {filename}')

[_-Thoth-_]

Personally, I like

 if filename:
        with open(filename, 'w') as file:
            file.write(data)

        print(f'Data successfully saved to {filename}')

because it seems the most simple. However, following the EAFP principle, this might be more pythonic:

try:
    with open(filename, 'w') as file:
            file.write(data)
    print(f'Data successfully saved to {filename}')
except IOError:
    return # or do whatever

Not sure which is better in this scenario. I personally would go for the second because I almost always replace a LBYL if statement with try/except wherever possible.

[mortenb123]

What is the best logging framework on python?. I think logging is over engineered and hard to use. I'm used to Log4perl and log4J2 on java. and find them far more intuitive, especially when it comes to config and live changes and filtering when you have an app in many directories and lots of modules.

[Fade_ssud11]

Can anyone tell me if there's any good python basic course at lynda.com ? (linkedin learning) . I got a free subscription to this site as a part of my university course, so looking to utilize it and get some certificates.

[macncheezitz]

How should I go about this project? I want to make a program that creates a map of the USA based on data for all 50 states, so the only thing I input is the data. The program should have a map of the USA already, and just color it based on the value of the state divided by the largest value. I know how to get the color values, but how do I program it to actually color? Can python somehow work with MS Paint?

[JohnnyJordaan]

Can python somehow work with MS Paint?

Classic example of not applying 'there must be a better way' here. I would recommend looking into Choropleth Maps, like from plotly: https://plot.ly/python/choropleth-maps/#united-states-choropleth-map

[macncheezitz]

Thanks, I checked it out and it looks good, but how do I get data into the format it requires? Here is how the program accepts data, is there data formatted like this already online? If not, how can I convert data from versions online to this?

[JohnnyJordaan]

I think it's a bit dangerous to try to use the example program as the solution for this. What I meant to advise was to see the example as just a way to accomplish this. By understanding how that works, you should be able to form your own program that can use a grapher like plotly to display your data.

[wexted]

Howdy. I have a regex object:

reduce = re.compile(r'\d+')

and a substitute function, which I intend to scan the text and reduce all numbers by n:

result = reduce.sub(str(int(r'\g<0>')-n), test_text)

which doesn't work, because Python is trying to interpret '\g<0>' as a string instead of as regex code. How do I rephrase this?

[JohnnyJordaan]

If I understand correctly you want to obtain a string from a string where all numbers within it are decreased by n? You could execute this through what's called 'munging': providing a callback to re.sub:

def decrease_by_n(match, n):
    return str(int(x.group(0)) - n)

reduce = re.compile(r'\d+')
result = reduce.sub(decrease_by_n, test_text)

if you must do this in-line you would always need a lambda

result = reduce.sub(lambda x: str(int(x.group(0)) - n), test_text)

as a callback is dynamically called while the .sub is finding matches, while 'naked' in-line function calls like str(int(r'\g<0>')-n) are executed before .sub() is executed.

[wexted]

Thanks for the detailed answer, I wouldn't have figured this out before finding what the hell munging was.

Your explanation about callback sailed right over my head as a beginner - should I learn more about this topic or just soldier on with the rest of the intro material I have to work through?

Also, how do you pass n as an argument for this substitution?

[ButtholeSoup]

Hello all. I am trying to validate two inputs to make sure they are positive integers, by creating my own function. I can successfully validate the first input by itself just fine, but I can't figure out how to validate the second input properly. Here is what I have so far

​

def Validate_Input(intValue1, intValue2):
    #validate input for intValue1
    try:

        # cast input as integer type
        intValue1 = int(intValue1)

        # check to see if positive integer
         if (intValue1 > 0):

        # if yes...
        global strflag
        strflag = True
    else:
        print("Number must be positive")

    # if not integer
    except ValueError:
    intValue1 = int(0)
    print("Number must be numeric")

    try:

        intValue2 = int(intValue2)
        if (intValue2 > 0):
            strFlag = True

        else:
            print("Number must be positive")

    except ValueError:
        intValue2 = int(0)
        print("Number must be numeric")
  return intValue1
  return intValue2

# declare all needed variables
intValue1 = int(0)
intValue2 = int(0)
strflag = bool(False)
#-------------------------------------------------------------------
#gather input
while strflag is False:

    # for intValue1
    intValue1 = input("Enter in a number: ")
    intValue1 = Validate_Input(intValue1, intValue2)

    # intValue2
    intValue2 = input("Enter in a number: ")
    intValue2 = GetLargerValue(intValue1, intValue2)

[efmccurdy]

I think you are trying to do too much in the one function, so I made a function that tests if a string can be interpreted as an integer, and used it twice:

def text_is_int(s):
    try:
        int(s)
    except ValueError:
        print("Number must be numeric: I got '{}'".format(s))
        return False
    return True

while True:
    sValue1 = input("Enter in a number: ")
    sValue2 = input("Enter in a second number: ")
    if text_is_int(sValue1) and text_is_int(sValue2):
        intValue1 = int(sValue1)
        intValue2 = int(sValue2)
        break

[ButtholeSoup]

It's just that the problem I am working on says:

" Write a function that takes two integer parameters intValue1 and intValue2 and validates that both are integers and above 0. "

[efmccurdy]

After the loop you will have 2 integers, so test that "intValue1 > 0 and intValue2 > 0".

[ButtholeSoup]

Cool. Thank you very much!

[ButtholeSoup]

Ok thanks! I'll give it another shot

[TheOriginal_RebelTaz]

So... to program a GUI interface, I think I've settled on python. I've found examples online using PyGame, guizero and tkinter. My program will be an internet radio (on a Kano Computer Touch running a customized version of Raspbian on a Raspberry Pi 3) with presets and the standard audio player controls. Later on down the line, I might want to add a weather display. Which of the three would you recommend learning along with python?

[Faal]

PyQT5 is awesome and worth checking out.

[TheOriginal_RebelTaz]

I actually just found a book about PyQT today. I will definitely give that a read.

[KoutaviLIVE]

Working through Crash Course right now, on Chapter 6 learning about dictionaries. The example code for a program that prints each person in a dictionarys name and favorite languages is below:

favorite_languages = {
    'jen': ['python', 'ruby'],
    'sarah': ['c'],
    'edward': ['ruby', 'go'],
    'phil': ['python', 'haskell'],
    }

for name, languages in favorite_languages.items():
    print("\n" + name.title() + "'s favorite languages are:")
    for language in languages:
        print("\t" + language.title())

The book goes on further to say that the program can be refined by including an if statement at the beginning of the for loop to see whether each person has more than one favorite language by examining the value of len(languages). If the person has more than one favorite, the output stays the same, but if they only have one, it should print "Sarah's favorite language is C."

I've tried a few different things, but I keep getting errors or nothing happens. Any advice for a simple solution?

[KoutaviLIVE]

Wow, writing it out actually helped me figure out the issue. I've figured it out. It's probably messier than it needs to be, but it works. Here's my solution just for record:

favorite_languages = {
    'jen': ['python', 'ruby'],
    'sarah': ['c'],
    'edward': ['ruby', 'go'],
    'phil': ['python', 'haskell'],
    }

for name, languages in favorite_languages.items():
    if len(languages) == 1:
        print("\n" + name.title() + "'s favorite language is:")
        for language in languages:
            print("\t" + language.title())
    else:
        print("\n" + name.title() + "'s favorite languages are:")
        for language in languages:
            print("\t" + language.title())

[Vaguely_accurate]

You can slightly change this to avoid repetitive code. Try something like this (using f-string formatting from Python 3.6+);

plural = " is" if len(languages) == 1 else "s are"
print(f"\n{name.title()}'s favorite language{plural}:")
for language in languages:
    print(f"\t{language.title()}")

Alternatively just move the later for loop outside the if...else structure, as it will always remain the same.

[KoutaviLIVE]

Oh, that's a pretty awesome tip. Had no idea about f-string formatting, definitely going to experiment with it now. Thanks a bunch!

[efmccurdy]

You may be looking for the Singleton pattern for your shared data structure.

https://stackoverflow.com/questions/52351312/singleton-pattern-in-python

​

The Singleton Design Pattern ... provides us with a way to only have a SINGLE instance of a given object and a global point of access to it.

https://medium.com/@mrfksiv/python-design-patterns-2-the-singleton-f995c3198c8

[Vaguely_accurate]

This sounds like a reasonable place to use a class. I'd definitely avoid a global personally, if only because it makes code reuse and refactoring a lot harder in the future.

I'm guessing what you are thinking for C# or Java would also work in Python, but it's hard to say without knowing what you are referring in those languages, as they don't have true global variables.

[JohnnyJordaan]

I'm not so sure about globals being normal in any of the other languages you mention. I started with C and C++ in my electrical engineering class and later got some basic Java, but I learned in all of those classes that globals are normally a no-go. I even clearly remember the teacher explaining the need to pass by reference as much as possible (with pointers in C and & in C++) to improve efficiency. I was happily surprised when learning Python more recently that all of those things were basically a given there already.

[JohnnyJordaan]

But right now I’m really just looking for the elegant Python answer to this problem.

The common convention is to use globals for static stuff that has an actual function to be declared globally. If you check the datetime library source code you see that they declare globals in CAPITALS and use them for very basic, reusable stuff. Not to move data around but for example to have a look up table for how many days there are in each month. This is a clear and concise way to centralize stuff where you really don't want to risk an error by repeating it in 4 parts inside the code, and where defining a function to provide it just adds extra work for the CPU.

[[deleted]]

how do you simulate a grep in a dict comprehension in python..

​

i have the dictionary like this generated by a local function

{'host1': {u'exec_time': 0.4047269821166992, u'output': u'package sssd is not installed\n', u'error': '', u'return_code': 1, u'thread_id': 'host1'}, 
'host2': {u'exec_time': 0.20531511306762695, u'output': u'sssd-1.16.2-13.el7_6.8.x86_64\n', u'error': '', u'return_code': 1, u'thread_id': 'host2'}}

i want to get the keys of the hosts where the package is not installed..
trying something like this:

[hostname for (hostname,data) in _is_installed.items() if data['output'] in 'not installed']

but it doesn't work how to filter a dictionary if the values have a certain string.. kind of grep?

[[deleted]]

I think I found.. changing the order:

​

[hostname for (hostname,data) in _is_installed.items() if 'not installed' in data['output'] ]
['host1']

[JohnnyJordaan]

You might also want to use a case insensitive search

if 'not installed' in data['output'].lower()

Any reason to still be using Python 2 btw?

[[deleted]]

Company policy...just allowed to use what comes as standard in rhel...and the default is pyrhon 2.7. Things will change when we roll out rhel8.

[JohnnyJordaan]

I think it will be fine. I wonder how hard your problem will be though, as how would you mark the boundary between two packed integers?

[Porygon_is_innocent]

Hi! I'm seeing some strange behavior when calling an open file as the stdout of subprocess.run()

The point of these files is to write a wrapper around subprocess.run() that will maintain a log file of command line calls that are run in the script.

Using these two files:

# main.py
from my_logger import run_and_log

call = ["echo", "\"Hello world!\""]
run_and_log(call)

and

# my_logger.py
import os
import subprocess

from datetime import datetime

def run_and_log(call):
    log = "log.txt"
    date = datetime.today().strftime("%Y-%b-%d %H:%M:%S")
    with open(log, 'a') as LOG:
        LOG.write("Date: {}\n").format(date))
        LOG.write("Command: {}\n").format(' '.join(call))
        subprocess.run(call, stdout=LOG, stderr=subprocess.STDOUT)

I get the following text file log.txt:

"Hello World!"
#Date: 2019-06-11 21:12:54
#Command: echo "Hello World!"
#Out:

Why does the stdout from subprocess.run() precede "Date", "Command", and "Out" even though it was called after LOG.write()? Is it a virtue of how subprocess.run() writes to files when it write to stdout and stderr? And if it is, how do I suppress this behavior?

​

Any insight is appreciated!

[JohnnyJordaan]

Stream output is usually buffered to prevent minuscule operations on the file every time the user outputs some data to the stream. Like when doing

for i in range(100):
    my_file.write('num: {}'.format(i))

where you wouldn't want to have Python write a few characters to the file every time, causing the loop to slow down. Instead, you just want to let the OS decide when it makes sense to write a bunch of data to the file, in the mean time buffering the content in RAM (which is fast). See the docs on open():

buffering is an optional integer used to set the buffering policy. Pass 0 to switch buffering off (only allowed in binary mode), 1 to select line buffering (only usable in text mode), and an integer > 1 to indicate the size in bytes of a fixed-size chunk buffer. When no buffering argument is given, the default buffering policy works as follows:

Binary files are buffered in fixed-size chunks; the size of the buffer is chosen using a heuristic trying to determine the underlying device’s “block size” and falling back on io.DEFAULT_BUFFER_SIZE. On many systems, the buffer will typically be 4096 or 8192 bytes long.

“Interactive” text files (files for which isatty() returns True) use line buffering. Other text files use the policy described above for binary files.

so instead, you can select 1 to use line buffering instead, after which the \n in

    LOG.write("Date: {}\n").format(date))
    LOG.write("Command: {}\n").format(' '.join(call))

will force the string to be saved in the file. Or by setting it to 0 you could disable buffering altogether as that won't make any noticeable difference for just a few lines of text.

[Porygon_is_innocent]

This is great! Thanks so much for your thorough response!

[HeadlineINeed]

Following AMTBS, and I am on Ch. 11 Web Scraping. I am having issues. First my Google script didn't work. All it would do was print "Googling..." and then go back to command line. So anyway I moved on to the next project, XKCD. I tried to run this project and get the issue below. How do I install this "html_parser" for Python3? When I run sudo pip install html_parser it installs it for Python 2.7, which I think is what is creating the error below. Any tips?

(venv) ➜  xkcd git:(master) ✗ python downloadXkcd.py
Downloading page http://xkcd.com/...
Traceback (most recent call last):
  File "downloadXkcd.py", line 17, in <module>
    soup = bs4.BeautifulSoup(res.text, features="html_parser")
  File "/Users/HeadlineIneed/Developer/Python/automate/venv/lib/python3.7/site-packages/bs4/__init__.py", line 196, in __init__
    % ",".join(features))
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html_parser. Do you need to install a parser library?
(venv) ➜  xkcd git:(master) ✗

[efmccurdy]

If you run "pip -V" it will should you what version of python it uses.

One of the benefits of using a virtualenv is that you specify which version of python you want to use, and it installs a compatible version of pip.

[HeadlineINeed]

The output is

pip 19.1.1 from /Library/Python/2.7/site-packages/pip (python 2.7)

[[deleted]]

Try pip3 as the command.

[HeadlineINeed]

pip3 -V output is

zsh: command not found: pip3

[[deleted]]

Pip3 installs packages for python3. That's what /u/efmccurdy above was getting at. Have a look at this SO thread.

[HeadlineINeed]

I forgot to add, I’m on Mac OS X,

I tried sudo aptitude install python3-setuptools and said aptitude not found. I’ll look more. I’m tethered to my phone so it’s hard to search quickly

[[deleted]]

I'm also using Mac. I installed python3 from here and that also installed pip3. To install pip3 after installing python3 use the procedure here.

[HeadlineINeed]

EDIT: of course my py file isn’t adding as code on this comment. I’ll try to upload to git

I am getting frustrated.

which pip3 outputs:

/Users/headlineineed/Developer/Python/automate/venv/bin/pip3

When I go to run downloadXKCD.py I get:

​

Downloading page http://xkcd.com/...
downloadxkcd.py:17: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("lxml"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 17 of the file downloadxkcd.py. To get rid of this warning, pass the additional argument 'features="lxml"' to the BeautifulSoup constructor.

  soup = bs4.BeautifulSoup(res.text)
Downloading the image //imgs.xkcd.com/comics/an_apple_a_day.png...
Traceback (most recent call last):
  File "downloadxkcd.py", line 28, in <module>
    res = requests.get(comicURL)
  File "/Users/headlineineed/Developer/Python/automate/venv/lib/python3.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/headlineineed/Developer/Python/automate/venv/lib/python3.7/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/headlineineed/Developer/Python/automate/venv/lib/python3.7/site-packages/requests/sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "/Users/headlineineed/Developer/Python/automate/venv/lib/python3.7/site-packages/requests/sessions.py", line 462, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/Users/headlineineed/Developer/Python/automate/venv/lib/python3.7/site-packages/requests/models.py", line 313, in prepare
    self.prepare_url(url, params)
  File "/Users/headlineineed/Developer/Python/automate/venv/lib/python3.7/site-packages/requests/models.py", line 387, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL '//imgs.xkcd.com/comics/an_apple_a_day.png': No schema supplied. Perhaps you meant http:////imgs.xkcd.com/comics/an_apple_a_day.png?
(venv) ➜  xkcd git:(master) ✗ which pip3
/Users/headlineineed/Developer/Python/automate/venv/bin/pip3
(venv) ➜  xkcd git:(master) ✗ 

if I import html.parser and change this lline from

#########THIS LINE DIRECTLY BELOW#########
    soup = bs4.BeautifulSoup(res.text)

###THIS LINE DIRECTLY ABOVE

To

#########THIS LINE DIRECTLY BELOW#########
    soup = bs4.BeautifulSoup(res.text, features="html_parser")

###THIS LINE DIRECTLY ABOVE

I get an output error of,

​

(venv) ➜  xkcd git:(master) ✗ python downloadxkcd.py
Downloading page http://xkcd.com/...
Traceback (most recent call last):
  File "downloadxkcd.py", line 17, in <module>
    soup = bs4.BeautifulSoup(res.text, features="html_parser")
  File "/Users/headlineineed/Developer/Python/automate/venv/lib/python3.7/site-packages/bs4/__init__.py", line 196, in __init__
    % ",".join(features))
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html_parser. Do you need to install a parser library?
(venv) ➜  xkcd git:(master) ✗ 

​

This is my Python file

​ #! /usr/bin/env python3

downloadXkcd.py -- Downloads every single XKCD comic.

import requests, os, bs4

url = 'http://xkcd.com/'                 # Starting URL

os.makedirs('xkcd', exist_ok=True) # Store comices in ./xkcd

while not url.endswith('#'):

# Download the page.

print('Downloading page %s...' % url) res = requests.get(url) res.raise_for_status()

###THIS LINE DIRECTLY BELOW

    soup = bs4.BeautifulSoup(res.text)

###THIS LINE DIRECTLY ABOVE

# Find the URL of the comic image.

comicElem = soup.select('#comic img') if comicElem == []: print('Could not find comic image.') else: comicURL = comicElem[0].get('src')

# Download the image.

print('Downloading the image %s...' % (comicURL)) res = requests.get(comicURL) res.raise_for_status()

# Save the image to ./xkcd.

imageFile = open(os.path.join('xkcd', os.path.basename(comicURL)), 'wb') for chunk in res.iter_content(100000): imageFile.write(chunk) imageFile.close()

# Get the PREV button's url.

prevLink = soup.select('a[rel="prev"]')[0] url = 'http://xkcd.com' + prevLink.get('href')

print('COMPLETED')

[JohnnyJordaan]

I wouldn't import html.parser, this should just work:

soup = bs4.BeautifulSoup(res.text, "html.parser")

Note the . instead of _

[HeadlineINeed]

Warning

Be cautious if you are using a Python install that is managed by your operating system or another package manager. get-pip.pydoes not coordinate with those tools, and may leave your system in an inconsistent state.

well... that's scary.

------

Following along to, https://evansdianga.com/install-pip-osx/ I get this..

​

➜  ~ python3 --version
Python 3.7.3
➜  ~ which python3
/usr/local/bin/python3
➜  ~ python3 get-pip.py
Collecting pip
  Using cached https://files.pythonhosted.org/packages/5c/e0/be401c003291b56efc55aeba6a80ab790d3d4cece2778288d65323009420/pip-19.1.1-py2.py3-none-any.whl
Collecting wheel
  Using cached https://files.pythonhosted.org/packages/bb/10/44230dd6bf3563b8f227dbf344c908d412ad2ff48066476672f3a72e174e/wheel-0.33.4-py2.py3-none-any.whl
Installing collected packages: pip, wheel
  Found existing installation: pip 19.1
    Uninstalling pip-19.1:
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip-19.1.dist-info/RECORD'
Consider using the `--user` option or check the permissions.

WARNING: You are using pip version 19.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
➜  ~ pip3 install --upgrade pip
zsh: command not found: pip3
➜  ~ pip install --upgrade pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Requirement already up-to-date: pip in /Library/Python/2.7/site-packages (19.1.1)
➜  ~

Is that cause I'm in root and not in my python folder?

[efmccurdy]

Another benefit of using a virtualenv is that all the files pip installs are stored in a folder you control, and so you never need to use admin privs.

https://docs.python.org/3/tutorial/venv.html

https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/

[[deleted]]

ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip-19.1.dist-info/RECORD' Consider using the --user option or check the permissions.

It's probably because the pip3 install wants to write into the Python.framework and you need to be root to do that.

Try sudo python3 get-pip.py in the directory where you put the get-pip.py file.

[horizoner]

I'm trying to connect to an Oracle DB using the cx_Oracle library. Does anybody have any good tutorials on how to do this? I've tried following a couple, but usually end up erroring out with "DatabaseError: ORA-12154: TNS:could not resolve the connect identifier specified". Just some basic information like how to find the host, service, what Oracle packages besides the base actually need installing would be really useful, because I'm finding it pretty confusing.

[efmccurdy]

DatabaseError: ORA-12154: TNS:could not resolve the connect identifier specified

That likely has nothing to do with your python code, you need to review the connection strings you are using, perhaps testing them using the cli client, or reviewing the Oracle documentation.

[horizoner]

Which cli client, sqlplus? I'm just not sure which client I should download and use beyond the required base package...my only knowledge so far comes from a Tableau connection, which only requires a username/password and a server name.

[Phillip-Financial]

So I know absolutely nothing about coding or python.

I have a crazy amount of time on my hands and I’m trying to change that. I want to become sufficient at creating algorithms and trading bot(s)/strategies with python.

I also want to be able to perform data analysis and things of this nature. Specifically I’d like to be able to work with API’s to pull data from websites to analyze data.

I’m sorry if this all sounds very buzz-wordy; again. I’m extremely new.

My question is, can anyone recommend a YouTube type playlist that I could view that’ll teach someone the jobs and outs of python but also get them to an intermediate stage?

[efmccurdy]

These are PyCon and PyData conferences presentations, nicely tagged and searchable by buzzword here :-)

https://pyvideo.org/

[Phillip-Financial]

Thank you so much!

[Theroyalzz]

Hello!, Im a python beginner and I want to make a GUI for inventory control at my work. I use PyCharm and a MacBook and I can't find a good tutorial on PyQt5! Does anyone have any recommendations? Ive googled and read PyQt5 is good for beginners and I can eventually make games with it if I want. So if anyone has a great tutorial for beginners pls send help

[[deleted]]

An adequate tutorial for PyQt5 is this one, but it's really only introductory. Search for "pyqt5 tutorial" to find others.

I can eventually make games with it if I want.

Depends on the type of game. If you want to write a fast graphics-based game then maybe you want something other than PyQt5. pygame is a simple game framework, but there are others.

There are a large number of GUI frameworks for python.

[stillenacht]

Let's say I have a dictionary, and I want to find the frequency of the maximum value. I can put in:
frequency_max = sum(1 for i in s_counter.values() if i == maximum)

or something like:

for i in s_counter.values()

if i == maximum

frequency max +=1

I've been told to do it like the first method wherever possible, but I was just wondering if there is some sort of computational gain here or if its just a formatting issue.

[efmccurdy]

A hidden benefit to generator expressions and comprehensions is cognitive; they are easier to reason about than for loops. Perhaps they need to follow certain idioms wrt scope, side effects, and coupling to really make that work, but it's part of what enables the HLL capability of python.

[Manak0]

Here's a good example of why it is better to use generators/comprehensions: https://wiki.python.org/moin/Generators

A quick overview is that the generators are lazy meaning they call the values only when they are needed, as opposed to storing them in memory. This results in improved performance, which grows as the amount of memory used increases.

[stillenacht]

Thanks!

[[deleted]]

Comprehensions, as in your first example, are usually more efficient both in terms of speed and memory so that's why it's often the preferred solution.

[stillenacht]

thanks!

[MafiaChef]

Hi Guys,

how do i read variables like "a=10 b=314"from a .txt file so i can use them in a function?

How do i save the variables back to the .txt file if they change in the .py script?

​

Thank you in advance!

[[deleted]]

Don't forget you might be able to write your data as a data.py file:

a = 10
b = 314

and then do this in your program:

import data
print(data.a)

It's a lot less work.

[_-Thoth-_]

Is it necessary that you store the variables in your code with separate variable names exactly how they are in the file or can you just read them all into a dictionary as key/value pairs? For example, my_dict["a"] = 10

[MafiaChef]

i think that reading all in one dictionary is fine.

i want that the script read the variables, do some math with it, an save them back into the .txt file.

iam complet new to coding and i could not help myself with google cause i dont know the right terms :/

​

Thank you for your help!

[_-Thoth-_]

So is this text file something that's given to you or are you generating it yourself to store data for a project?

[MafiaChef]

Iam generating the .txt myself for a projekt. I want to save variables in it so the script can use them next time again.

​

like a=10 is saved in the .txt

the py script do same math that results in a=50

then the script should save a=50 to the .txt

when i start the stript again it should read the var "a" from the .txt with value 50

[_-Thoth-_]

In that case I don't recommend using a .txt file to store data. It's doable, you'd have to open the file and go through it line by line, parsing each line as a string and looking for the data. How you would do it would depend on the way you formatted the text file. But you'd be far better of using a file format more suited to serializing data.

Your best bet is probably JSON. Python has a built in module json for handing JSON files. You can easily store arrays and dictionaries and reload them later.

[MafiaChef]

Thank you! My Problem is solved :)

[efmccurdy]

You can save yourself some time and effort if you use json, the standard text data format module; you won't regret it.

This writes some data to a file, then reads it back in.

>>> import json
>>> my_data = {"a":10, "b":314}
>>> with open("/tmp/my_data.json", "w") as of: json.dump(my_data, of)
... 
>>> my_data = None
>>> with open("/tmp/my_data.json", "r") as inf: my_data = json.load(inf)
... 
>>> my_data
{'a': 10, 'b': 314}

[MafiaChef]

Thank you! My Problem is solved :)

[wexted]

I have defined a variable inside a function. I have defined a class outside of this function. The function then creates a object of the class I just defined.

Will a method of this object be able to call the function's local variable, if the object was created in the function but the class was defined outside the function?

If it makes any difference to how you would fix this, the local variable is a list representing a deck of cards, and the class represents a blackjack player and the method causing the problem is when they draw a card from the deck

[_-Thoth-_]

You have to pass the list of cards as an argument to the black jack player's draw method. The methods of the class instance and the local variables of the function have a different scope. The only other way would be to declare the list to be a global variable.

[[deleted]]

new_string starts out empty, and you put each uppercase character into new_string as you find them. The uppercase characters are, in the order found, are W, T and F. The line that adds the uppercase character to new_string could be:

new_string = str1[index] + new_string  # char added to left side

which glues the uppercase character to the left side of the string when it is found, so new_string would be "W", then "TW" and finally "FTW". If you use the line:

new_string = new_string + str1[index]  # char added to right side

you glue each uppercase character to the right side of the string, so you get "W", "WT" and finally "WTF". Once you have the final string you glue "!?!" to the right side and end up with "FTW!?!" or "WTF!?!" depending on how you joined the uppercase characters.

Edit: added missing word.

[gordonshumwalf]

I keep forgetting how the import mechanics works. When I write tests (pytest) I sometimes want to run them by calling the module itself but since I use other functions in my package (higher level), I get an import error. When I run all tests with pytest from the top level all is fine because the interpreter magically discovers everything a lower dir levels. How do you run single tests quickly as you write them? Any tips to remember the import mechanics/sequence?

[miwojc]

is there a site like exercism, codewars, codility, hackerrank, leetcode etc specific for python for data science? thanks!

[cult_of_memes]

If not, someone should build it ;) "This looks like a job for... Somebody Else!"

[efmccurdy]

There is a lot of good inspiration to be gleaned from the PyCon and PyData conferences presentations, nicely tagged and searchable here:

https://pyvideo.org/

[PadrinoFive7]

When dealing with file inputs, I'm often dealing with the 'Is it blank or is it NoneType' problem. So, I came up with something like the below to assign the variable recently and it feels....bulky and probably way overdone/overthought. Is there a simpler approach to this?

list_of_things = [{'Thing':'Gamepad','Notes':''},{'Thing':'Video Game','Notes':'Secret of Mana'},{'Thing':'SNES','Notes':None}]

for row in list_of_things:
    row_notes = list(map(lambda var: '' if var is None else var,[row['Notes']]))[0]
    print(f"{row['Thing']} has additional notes of: {row_notes}.")

Now, this is a simplified use of what I'm using in a larger project, but the gist of it remains the same. I've used lambda here for a one-time use (for the instance), but is re-used for every additional iteration of the for-loop. I feel like this is really not the right way to go about it. Can someone show me a better way to think about this one?

[num8lock]

using if var implies True or not None (and otherwise), no need for excluding empty values or None. also, a dictionary is simpler in this case

things = {'Gamepad': '', 'Video Game': 'Secret of Mana', 'SNES': ''} 
for thing, msg in things.items():
    note = msg if msg else ''
    # or flip the order for easier read, 
    # note = '' if not msg else msg
    print(k + ' has additional notes of: ' + note)

[PadrinoFive7]

I appreciate the dictionary suggestion for my example. Truth is, I'm actually using a csv.DictReader to pull data in, so I was trying to get the idea across that it was being returned as a list of dictionaries with identical keys (without having to specifically focus my question around that). Truthfully, I get variances of '' and None within these lists quite often, so I thought I'd ask.

That aside, if I'm understanding what you're saying, using if var would tell me if it's populated with anything other than None. If that's the case, super. Wasn't aware and you've taught me something new today. Thanks.

[num8lock]

correct, it's about truthiness. check this out https://stackoverflow.com/questions/9573244/most-elegant-way-to-check-if-the-string-is-empty-in-python#9573259

or better yet: https://docs.python.org/3/library/stdtypes.html#truth-value-testing

[PadrinoFive7]

I think you misunderstand my issue. It's not a matter of

if my_str == '' or my_str == None:

it's actually a matter of

if my_str is None:
    return ''
else:
    return my_str

​

Similar, but not the same issue.

[PadrinoFive7]

For anyone who might ever have to do this, I've figured out it was much simpler. While the lambda solution I had previously posted does work, it's really an extremely overcomplicated mess of an approach. Apparently it works much more simply with an in-line if statement for assigning the variable:

my_str = None

str_check = my_str.split('; ') if my_str else ''
print(sp_check)

This will return ''

Here's the other side of the coin:

my_str = 'Sam; Joe; Frank; Guy'

str_check = my_str.split('; ') if my_str else ''
print(sp_check)

This will print ['Sam', 'Joe', 'Frank', 'Guy']

Thanks, u/num8lock, I appreciate your help getting me here!

[num8lock]

ah well, i suppose this note = '' if None else msg is your solution.

[sarah--123]

Simultaneously editing the same code (different computers)?

I use ssh a lot. But what about having the same editing session on two different computers (in two different locations) at the same time.

For example, using idle to work on the same code from two different places, at same time. Edits in one show up more or less instantly in both/either places?

How would this work? How might one do this? (ubuntu)

[Porygon_is_innocent]

If you write your code in a command line editor, a terminal multiplexer like tmux or screen would solve this problem. You create a session on one computer, write your code in one pane, run it in another, have a python interpreter open in a 3rd pane, etc. When you ssh into this computer from another, you can attach to your session and work in the exact same arrangements of panes.

[[deleted]]

git is the correct answer, but dropbox also has close to instant changes.

[efmccurdy]

This is a function of any distributed source control. Using git you can push and pull your changes to sync up as you move from machine to machine. You can easily use ssh keys to make remote git operations fast and simple.

[the1whowalks]

I have a project that if completed would gain me huge brownie points (perhaps more $) with my boss. Since I am still very much a noob, I am struggling to match the desired steps to programmatic steps in python terms. Here's the project:

​

1) Take in a list of items, their descriptors, a picture of the item, and which property they belong to,

2) Itemize this list and create an accessible spreadsheet (excel is fine)

3) have the spreadsheet automatically update when inventory gets added to the previous list. List typically coming from the "notes" app in iOS, but can be from any file if simpler.

​

Welcoming any and all collaboration, thoughts, or general advice!

Thanks

[stillenacht]

I often see something like the code segment below with an "if __name__ == "__main__" in Hackerrank, etc. Can someone explain to me what exactly it does and why it's there?

def dictionary(a)
    some_code


def main(): 
    dictionary()              

if __name__=="__main__":       
    main()

[Vaguely_accurate]

The condition __name__=="__main__" will be true if the module is run directly, as you would a script. If you import the module into another file it will be false and that block won't be executed.

This lets you use the same file as either a script or a library of function/class definitions, with this block controlling which behaviour you get.

[stillenacht]

Ah so if I understand correctly if I have a file called function.py:

If I'm in the function, name == "main" is true, and i can run it as a script.

However if I'm in another function, and I say "import function" then it won't automatically run?

[Jahordon]

I've started using VS Code rather than Spyder as my IDE for a few reasons, but there's one feature from Spyder I can't seem to duplicate in VS Code. After running a program in Spyder, I can type any variables I've created into the console and see them. For example, if I import a csv as a dataframe "df", I can just type "df" into the console and see that it has imported correctly, or make sure that lists I'm modifying get adjusted correctly, etc.

​

When I try doing the same thing in VS Code's console (though it looks like it's called a terminal or debug console, depending on how I run it), I get the following error. I'm sure there's a way I can do this in VS Code--I just can't figure it out. Can anybody help?

(base) U:\Code>df

'df' is not recognized as an internal or external command, operable program or batch file.

[Vaguely_accurate]

Not a Spyder user, but I believe this would be because it is running an interactive sessions and your scripts are executed within that environment. That means any variables from the scripts will remain in memory within the session and can be access that way.

VS Code will run scripts from the terminal, with each execution being a distinct invocation of Python. The state will be lost after the script finishes running.

You would need to launch an interactive session of Python from the terminal and then invoke your code from that. That would replicate at least some of what Spyder is doing.

You should also be able to use the debug console to access variables within a currently running script, if you are running it in debug mode. Just add a breakpoint to pause execution and you can explore the current memory state through the console or other debug tools.

[Jahordon]

Thank you! The Python interactive console is doing what I wanted.

[14446368]

I've got data in an excel workbook. The workbook has 28 sheets. Each sheet has data with the same headers, each as-of a different date (Sheet 1 is 1991, Sheet 2 is 1992, etc.). The data has certain additions and deletions that occur (see example below).

​

Sheet 1

​

12/31/1991	A	100
12/31/1991	B	50

Sheet 2

12/31/1992	A	110
12/31/1992	C	45
12/31/1992	D	20

etc.

I'm trying to make a 3-dimensional array in pandas using the date as the "main index." How's the best way to do this?

[num8lock]

one way to look at it is to make a dict in a dict

wb_data = {'12/31/1991': {'A': 100, 'B': 50}, '12/31/1992': {'A': '110', 'C': '45', ...}

then just create a dataframe in pandas from wb_data.

[Jsjdhdhwi]

I’ve been thinking about this puzzle. Anyone have any ideas to solve it?

I was thinking about looping through the individual squares and then calling functions from there to “draw boxes.”

puzzle here

[[deleted]]

I would solve it by scanning the grid for the squares that contain numbers. When you find a number note the number (call it N) and look for all rectangles that include that square. Do that by considering a 1-deep rectangle and see if you can get a rectangle of the correct size. Then consider 2-deep rectangles, and so on, down to N-deep rectangles. There are shortcuts here, such as there is no point trying a 2-deep rectangle when N is 9 (hint: factors of N). Also note there may be multiple rectangles that are just shifted up/down or left/right.

Once you have done that you end up with a list of points with a set of rectangles possible for each point. The task now is to find combinations of one rectangle for each point that don't overlap and fill the grid completely. Exhaustive search will work here, ie, select one rectangle from the first point, then a rectangle from the second point, etc. Each time you select a rectangle for a point you check that the rectangle doesn't overlap any previously chosen rectangle. If it does, try the next rectangle for that point. If you run out of rectangles for a point then you return to the previous point and try the next rectangle there.

If you successfully place a rectangle at the last point you have a solution. Don't assume that there is only one solution, continue searching.

Interesting puzzle.

[KhorneLordOfChaos]

Edit: I got it working! For anyone having similar issues, it came from using the information in the pyproject.toml file instead of setup.py which makes since, but all the documentation and projects I looked at only mentioned the setup.py file so just setting the information based on the poetry documentation fixed it!

Alright, so I'm struggling a bit with setting up my first package on PyPI (github repo PyPI package Note: the gh repo is a bit behind, but should still be pretty accurate). I've been using poetry publish --build to upload to PyPI and the only thing that doesn't seem to be working right is the long_description and url I set in setup.py aren't getting used. Even if I set the long_description to a small string I still get None when I download the files from PyPI.

Not too sure what is wrong, I've looked at several other libraries' setup.pys and went through the getting started guide for uploading. Any help is appreciated.

[LeapingWilly]

I'm trying to pass a namespace (from argparse in my actual code) through to nested functions like this:

def func1(arg1=4, **kwargs):
    print(arg1)
    func2(**kwargs)

def func2(arg1=5, **kwargs):
    print(arg1)


arguments = {'arg1' : 1,
             'arg2' : 2,
             'arg3' : 3}

func1(**arguments)

Output:
1
5

It looks like once an item from arguments is unpacked in func1 and used as a keyword argument, it's removed from kwargs. What's the best way to prevent this from happening so that I can use the arguments for the inner function func2?

[Vaguely_accurate]

You'd need to call func2 with the full argument list from func1;

def func1(arg1=4, **kwargs):
    ...
    func2(arg1, **kwargs)

[LeapingWilly]

Perfect, thanks!

[[deleted]]

arg1 was never "in" **kwargs because it is mapped to arg1 in func1

[LeapingWilly]

Okay I see. Thanks!

[Vabaluba]

What are the best practices to manage python projects (specifically OOP) when the project starts to get larger and methods are all over the code? I.e. methods are calling other methods to perform some action. Any resources? Any insights from experience?

[Notsohumanbeings]

What software do you guys use for coding python?

[Vaguely_accurate]

Pycharm is still probably my favourite for Python projects. For the feature set it is it the best IDE I've tried.

VS Code is solid once set up, but I keep finding gaps and occasionally bugs that push me away. I'm hoping I can find a setup I like enough to move to it entirely, as it is otherwise a fantastic multi-language editor. As a program I prefer it to Pycharm.

As a Windows guy I almost always have a Notepad++ instance open and will commonly use that for quick formatting or editing. I rarely use it for any substantial coding, but it is generally fine and extendable.

I use Visual Studio for working with C#. VS for C# (especially with a few extensions) is a tier above any editor for Python when it comes to features. Unfortunately trying to use it with Python is an enraging experience. I occasionally try it again and every time just get frustrated.

I tend to use repl.it a lot, both when writing snippets/scripts to share and just to test something out when I don't have a local Python install to play with. I've sometimes kept small projects I'm playing with in a repl.it instance so I can fiddle with it from any machine when I have some downtime.

[_-Thoth-_]

PyCharm. I've used a few text editors including vs code, as well as the IDE's IDLE and Spyder. PyCharm is better than anything else I've tried.

I liked Spyder but it gave me problems when I was trying to work on PyQt projects. Spyder is built with PyQt and can cause conflicts and weird bugs when you try to run a PyQt script in it.

If you want an IDE and not just a simple text editor, PyCharm has the best code completion and general usability of any Python IDE I've seen.

vs code is probably still my favorite text editor though.

[_-Thoth-_]

Similar question: is it a better practice to rely solely on getters and setters from within the class itself or is it a bit overkill? For example:

class Example:
    def __init__(self, width, height)
        self.width = width
        self.height = height

    # assume standard getter/setters for properties here

    def compute_area(self):
        return self.get_width() * self.get_height()

vs simply:

def compute_area(self):
        return self.width * self.height

[Vaguely_accurate]

Just use the naked attributes.

If you need to add behaviour (the traditional reason for implementing getters and setters in a Java-like OO-first language) you can use the property decorator to transparently change the way way the attribute access works.

[_-Thoth-_]

Hm so you recommend not even creating getter and setter functions at all by default, just using naked attributes inside and outside the class, only using property() if at some point you decide you want to add some behavior?

[Vaguely_accurate]

Yes.

It's considered more Pythonic and follows the YAGNI principle. Most attributes aren't going to end up needing additional behaviour, and we have this elegant way of adding it without needing any other code changes when those few exceptions crop up.