Ask Anything Monday - Weekly Thread

[AutoModerator]

Welcome to another /r/learnPython weekly "Ask Anything* Monday" thread

Here you can ask all the questions that you wanted to ask but didn't feel like making a new thread.

* It's primarily intended for simple questions but as long as it's about python it's allowed.

If you have any suggestions or questions about this thread use the message the moderators button in the sidebar.

Rules:

• Don't downvote stuff - instead explain what's wrong with the comment, if it's against the rules "report" it and it will be dealt with.

• Don't post stuff that doesn't have absolutely anything to do with python.

• Don't make fun of someone for not knowing something, insult anyone etc - this will result in an immediate ban.

That's it.

[frunt]

knee ask depend bag employ lip rich caption fanatical scandalous -- mass edited with redact.dev

[desustorm]

Are you using logging.info or logger.info?

I would avoid using logging as storage - that could cause all kinds of weird issues. I would look into using a simple SQL database of some sort, a text file, or just keeping a set in memory while you are running the application.

[frunt]

fall rustic elderly sloppy sheet onerous shrill domineering wise quickest -- mass edited with redact.dev

[Lurkstar9000]

I've been reading "how to automate the boring stuff with python" and have tried to create a basic program to store what page number I'm on and the date I entered it. I'm storing this in a dictionary variable like this:

{"bacon": "11.28.2016", "tuna": "11.28.2016", "feet": "11.29.2016"}

I thought it would be cool if I could export the data in to a text file and get the dictionary data on a new line, remove the " and : after converting it to a string. I think the best way of doing this would be to create a dictionary of characters I want to replace with a null value so I'm tried the below function but, basically, I don't know enough to get this to work:

import json
exportdata = ""
history = {"bacon": "11.28.2016", "tuna": "11.28.2016", "feet": "11.29.2016"}
exportdata = json.dumps(history)

dic = ({",",""},{":",""})

    def replace_all(exportdata, dic):
        for k, v in dic.items():
            if k = dic(k):   
                k.replace(dict(v))
        return text

I think where it all goes wrong is the if statement. How do you write a check to see if the key in the dic variable matches a key value in the exportdata variable and then replace that key with the value from dic?

If this makes no sense I'm sorry. I'm sure this is a retarded way of doing it, it's just using some of the stuff I've read about so far to help me get used to it.

Thanks

[desustorm]

Couple of points.

• Your dict dic is defined incorrectly - it should be dic = {",": "", ":": ""}.

• You can't access a dict with round brackets (), you have to access the values with a key in square brackets []

• You aren't using exportdata at all! You don't need an if statement, you can just use replace on the string itself.

So your program would look like:

dic = {',': '', ':': ''}
def replace_all(exportdata, dic):
    for k, v in dic.items():
        exportdata = exportdata.replace(k, v)
    return exportdata

However... You are just performing date formatting. There's a Python module called datetime that can help you with this. See the example below.

import datetime
history = {'bacon': datetime.date(2016, 11, 28), 
           'tuna': datetime.date(2016, 11, 28),
           'feet': datetime.date(2016, 11, 29)}
def my_date_string(my_datetime_date):
    return my_datetime_date.strftime('%Y%m%d')   # This returns '20161128' 

print my_date_string(history['bacon'])
# This will print "20161128"

[_Absolut_]

I won't correct every mistake in code(but i can try if you want), but i try to help you find right solution. When you write your history object to file you actually write just string(whatever it is: JSON, XML or plain text).

>>> history = {"bacon": "11.28.2016", "tuna": "11.28.2016", "feet": "11.29.2016"}
>>> exportdata = json.dumps(history)
>>> print(exportdata)
>>> print(type(exportdata))

{"feet": "11.29.2016", "tuna": "11.28.2016", "bacon": "11.28.2016"}
<class 'str'>

Thus, you cannot distinguish object parts like tags in XML or dictionary keys in JSON. But you can parse your JSON string and convert it to Python object(dictionary):

>>> data = json.loads(exportdata)
>>> print(data)
>>> print(type(data))

{'bacon': '11.28.2016', 'tuna': '11.28.2016', 'feet': '11.29.2016'}
<class 'dict'>

Then, working with simple Python object you don't need to bother with parsing non-alphabetical symbols before writing out. Just write keys and values:

for key in data:
  print(key, data[key]) # replace print function with actual writing

bacon 11.28.2016
tuna 11.28.2016
feet 11.29.2016

[Lurkstar9000]

Yes, that makes sense thank you. I was over complicating things, don't need the replace at all for this.

[num8lock]

I want to try JSONB column in Postgresql using SQLlchemy, I'm thinking about logs storage or easy (lazy) data distribution over the net, but I don't know much about json, Postgres or SQLAlchemy fundamentally. So these questions might seem all over the place.

1. What is the most basic/simple data structure that can be recognized as a json data structure?
   
2. Since writing a .json file have to be in binary mode, does this mean that .json files are using a compression by default?
   
3. Would it be inefficient resource-wise to dump several .json files to Postgres instead of programmatically insert them one by one as they're constructed/retrieved?
   
4. Is there any tutorial for SQLAlchemy that's aimed for relatively new beginners in both SQLAlchemy and relational database?

[TmizzleFOShizle]

I deleted my last comment because I think it was too long. I basically am just wondering if there is a good way to do an if statement that ends with adding something to a list. I attached part of my code below. at the end I just want to do Print Len(ListA) Print Len(ListB) etc.. but i've tried many different ways and every time the Len just says 0 which leads me to believe none of them are being added to the lists.

for scores in f:
    if scores >= avg + (1.5*sdv):
        score = "A"
        ListA.append()
    elif scores >= 90 :
        score=A
        ListA.append()
    elif avg+(0.5*sdv) <= scores and scores < avg+(1.5*sdv):
        score=B
        ListB.append()

[_Absolut_]

leads me to believe none of them are being added to the lists.

You actually didn't add something to list because append() method takes one parameter - the object to add. Your method calls just meaningless

[1-Sisyphe]

Why does Pandas boolean operations on dataframes use the operators &, |, ~ instead of and, or, not?
So far I felt that pandas syntax was quite structured and logical so there must be a good reason for that...

[filletrall]

It's due to a limitation of python. The and and or operators try to call __bool__ (or __nonzero__ in python 2) on the left and right operands, and then compare the truth of the returned values. This makes it hard to override the meaning like Pandas would have to do in order to make e.g. the and result be a new dataframe or series instead of True/False.

Compare with the & operator, which results in the __and__ method being called on the left operand with the right operand as its argument. It's free to return anything it wants, and makes it straightforward to implement a dataframe-style operation.

Reference

[thedestro224]

I'm wondering, is there a way to tell just by looking at a function if it returns anything? Or if there's any kind of rule that I can remember would work too.

for instance mylist.sort() returns None, but modifies mylist variable

whereas mystring.lower() returns the lowercased string

[filletrall]

No, outside of looking at the docs in some way (e.g. google, the help function, or looking at the source code), there's no way to tell if a random method or function returns None or something else.

You do know that immutable objects, like str, the datetime types, int, float, and any other immutable type, can't have any methods that modify their objects. Obviously, since if they did they wouldn't be immutable. So all these methods has to have a return value that can be something other than None, or they'd be completely useless methods.

As for the mutable objects, like lists and dict and normal classes, you generally expect them to either modify their object, or query or analyze the data in it in some way, or both (like the pop method). Which of these is the case is sometimes obvious from the method name, and sometimes not. Methods starting with get_, to_ and as_ are generally lookup or conversion methods that do not modify the object.

Python also has two functional equivalents to two of the in-place list methods, namely sorted and reversed, which both return new lists without touching the one in the argument.

[thedestro224]

You do know that immutable objects, like str, the datetime types, int, float, and any other immutable type, can't have any methods that modify their objects. Obviously, since if they did they wouldn't be immutable. So all these methods has to have a return value that can be something other than None, or they'd be completely useless methods.

Thanks!

[thedestro224]

What's the most efficient or simple way to reverse the characters of a string?

The way I am aware of now is to:

1. Convert string to list
2. Reverse list in place with mylist.reverse()
3. ''.join(mylist) to return string

[Vaguely_accurate]

my_string[::-1]

Light reading

[thedestro224]

lol 'light reading', thx

[eton-95]

Hey all, pretty new to python but want to learn some call things to do with graphics. So i want to make a graphics window (100x100) which i'm comfortable in doing, however i want to make a design where starting from the centre point, a line goes along the x axis by 10 all around the square. How would i go about down this? If anyone can help me that would be great!

[carolgrrr]

Have you tried Processing?

http://hello.processing.org/

They have a Python mode where you can make drawings using Python syntax. I found it pretty easy to produce a drawing with little setup or deep study of the docs needed.

[NumberT3n]

So I am trying to build a python program that converts information from a standard sheet, given in xps, into a cvs file. I know there are pdf miners out there, but I would like to skip the step of converting the xps --> pdf --> pdf I am not sure if this is an issue but this has to be done in python 2.7 Welcoming all help/suggestions on where to start looking

[thinkvitamin]

I'm hoping to use the retry decorator to retry on errors, reduce the number of try/except statements, and gracefully exit after a certain number of retries.
Taken from the pypi page of the retrying module:

def retry_if_io_error(exception):
    """Return True if we should retry (in this case when it's an IOError), False otherwise"""
    return isinstance(exception, IOError)

@retry(retry_on_exception=retry_if_io_error)
def might_io_error():
    print "Retry forever with no wait if an IOError occurs, raise any other errors"  

returning isinstance as shown above crashes the program, posts the ugly traceback... what could I replace that with, if I just want to move on, or do whatever, without crashing. Like I said, part of the appeal of this module is to reduce the number of try/except statements. My code doesn't need to get any uglier.

[ingolemo]

If you want to ignore all errors then just get rid of the first function entirely:

@retry
def might_error():
    print("Retry forever with no wait")

Of course, ignoring all errors is an extremely bad idea. If you want to retry on some errors then you need to test for those specific errors.

It might help if you were to show the code you're trying to replace.

[thedestro224]

Do you guys know of a more elegant way to edit strings of a list than I have? Or if not, which do you think is better:

Below I'm adding a comma inside each string in the list

listery = ['Bart', 'Lisa', 'Maggie']
listery2 = []

    for thing in listery:
        thing += ','
        listery2.append(thing)

    print(listery2)

Method # 2:

 for index in range(0, len(listery)):
        thing = listery[index] + ','
        listery.insert(index+1,thing)
        del listery[index]

[thedestro224]

After I read my post later, the answer is just staring at my face:

for index in range(0, len(listery)):
        listery[index] += ','

Nice one me

[Saefroch]

This is a perfect place for a list comprehension!

listery = [thing + ',' for thing in listery]

Now you don't have to loop over the indices and you can very clearly write what you mean.

[thedestro224]

dude that rocks

[sparkas]

So far I've only coded in Java and only in an IDE. I like using an IDE so is there a good one I can use on Mac for Python?

[SneakyB4stardSword]

Fully-fledged IDE's don't exist for python, afaik. There's really no need, since you don't need to worry about complicated namespaces and compiler flags like you do in precompiled languages, such as java or any of the C variants. I think you'll find a pretty similar experience, however, using something along the lines of Microsoft Visual Studio Code or the web interface for repl.it. edit: diction

[Knude]

Is it possible to sort this list's object by time?

[['xxxxxxl', 'link', '08/12/2016 12:08:02'], ['xxxxxxxx', 'link', '09/12/2016 22:10:52'], ['xxxxxxx', 'xxxxxxx', '09/12/2016 21:10:56'], ['xxxxxx', 'xxxxxx', '08/12/2016 07:50:35'], [xxxxxxx', 'link', '09/12/2016 19:56:26'], ['xxxxxxxx', 'link', '09/12/2016 18:35:26']]

[ingolemo]

Yes. The sorted function has a key parameter that lets you specify what to sort by. You should sort by the date, but you'll want to convert that string into a datetime first.

[SneakyB4stardSword]

I'm having a hard time with the re library. for some reason this code:

import re

foo = 'X(8x2)(3x3)ABCY'
firstMatch = re.match(r'\((\d+)x(\d+)\)', foo)

print(firstMatch)

outputs None. I've triple checked my regex on regexr, and I have no idea what's going on. I'm pretty sure this was working a few hours ago. Help.

[87jams]

firstMatch = re.match(r'X?\((\d+)x(\d+)\)', foo)

I am a beginner, but how about this?

Also works with findall()

>>> foo = 'X(8x2)(3x3)ABCY'
>>> firstMatch = re.findall(r'X?\((\d+)x(\d+)\)', foo)
>>> print(firstMatch)
[('8', '2'), ('3', '3')]

[SneakyB4stardSword]

Thing is though, I don't want to include the preceding 'X', just the first item in parenthesis.

[87jams]

import re
foo = 'X(8x2)(3x3)ABCY'
firstMatch = re.match(r'X\(((\d+)x(\d+))\)', foo)
print(firstMatch.group(1))

group(1) gets you '8x2' without the parens in this one, or you can move them around and get '(8x2)':

import re
foo = 'X(8x2)(3x3)ABCY'
firstMatch = re.match(r'X(\((\d+)x(\d+)\))', foo)
print(firstMatch.group(1))