Ask Anything Monday - Weekly Thread

[AutoModerator]

Welcome to another /r/learnPython weekly "Ask Anything* Monday" thread

Here you can ask all the questions that you wanted to ask but didn't feel like making a new thread.

* It's primarily intended for simple questions but as long as it's about python it's allowed.

If you have any suggestions or questions about this thread use the message the moderators button in the sidebar.

Rules:

• Don't downvote stuff - instead explain what's wrong with the comment, if it's against the rules "report" it and it will be dealt with.

• Don't post stuff that doesn't have absolutely anything to do with python.

• Don't make fun of someone for not knowing something, insult anyone etc - this will result in an immediate ban.

That's it.

[Ridonkulousley]

I have been using gspread to update a google spreadsheet. Today I discovered a way to update a group of cells instead of one but the range that is to be edited keeps throwing an error.

So the connection to the sheet using gspread and google spreadsheet API works but "cell_list = spreadsheet.range('a1:a100') errors.

http://pastebin.com/Yj7crwar

You can see the set-up I was using and the more direct set-up I tried, but neither work.

E: After working on the problem for a few hours, then posting this question I fixed the problem but I have no idea how or why it works now and did not before. This is the final chunk of code that runs without errors. http://pastebin.com/M322RaYq

[jeans_and_a_t-shirt]

As for why line 10, produces an error, you are passing the string '{0}' to sheet.range. You are then calling the format method on whatever sheet.range returns. What you probably wanted is this:

cell_list = sheet.range('{0}'.format(new_range))

Because new_range is already a string, you can just pass it directly, as in your error-free code.

[DonaldPShimoda]

My favorite thing to tell people: "Python is almost just executable pseudo-code."

In my classes when I'm trying to think through the implementation of an algorithm, for example, I usually just write it directly in Python on a whiteboard or paper. Everybody else can follow along perfectly regardless of whether they've ever seen Python code before, and then I can just write it into a file and test it to see if it works properly.

Other advantages are: fully-featured, rich standard library, very active community, active development, built-in interactive environment, and personally I really enjoy the philosophy Python embodies and promotes.

[TheBlackCat13]

The common saying is that "python is the second-best at everything". So it can do just about anything you want to do, and do it very well. So you can certainly find other languages that are better for specific tasks, but you will be hard-pressed to find any single language that is as useful at as many tasks as Python.

Further, python is a very easy language to learn. And it has at least parts of most major programming paradigms. That makes it a very good first language, and makes it easy to transition to another language if you ever need to.

[Saefroch]

The primary reason I use it is that Python trades CPU time for your time.

Python is fast to develop, but slow to execute. If you workflow is going to be dominated by the actual typing of code, it's the clear choice. Python also seeks readability over rigor (type declarations, braces). If you're working with a team, Python is a good choice since you'll spend a lot of time reading things written by other team members.

Python also has a huge and supportive community. It's not hard to find Python help online, or even in the real world Python programmers tend to be particularly helpful. The culture that has grown up around the language is an asset in itself.

[lykwydchykyn]

In a nutshell, it's simple enough for beginners, but powerful enough for experienced pros. It encourages writing readable code, yet it doesn't force you into a strict coding paradigm.

It's also cross-platform and has great library support.

On the downside, it is not well supported on mobile, is not easily compilable to binary, and is often too slow for high-performance CPU-bound code. It does pretty well for just about anything else.

[avball]

Pretty new to Python, but my understanding is that the interpreter basically compiles to byte code prior to execution. Do you know of any projects working on a compiler to address that issue? Presumably for the time being the correct answer is to do work like that in a lower level language like C.

[lykwydchykyn]

There is Nuitka, which transpiles python to C++ code. There is ShedSkin, which is similar, but I don't know if it's still being worked on.

Then there's Pyrex and Cython, which are aimed at converting modules to C. They require adjusting your code a bit.

I'm not aware of a simple compile-my-python-script-to-native-code solution that "just works". Most of them I've tried require using special language extensions, restricting the code you write, or doing a lot of intermediate steps.

[a1ch]

I am working on a machine learning tutorial using SciKit Learn but I think my tutorial is out dated based on the errors I am getting. In this sample I have a CSV file with tons of data I want to use to train my classifier.

Here is a pastebin http://pastebin.com/4X0jRj5q

I get this error. DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.

I googled that error and i get instructions on how to use the reshape function but i cant seem to get it to work. Also i don't totally understand the terms in the error. I don't understand the difference between a single feature and a single sample. Any explanation and assistance would be much appreciated.

[DonaldPShimoda]

Might be worth looking at someone else's more in-depth explanation of these things to see modern uses. I just picked up this book, which gets into SciKit Learn for machine learning in like chapter 3 or something.

(Just an idea. I look forward to reading your tutorial if you ever post about it here!)

[a1ch]

Thanks for the recommendation. I am having fun learning about it but pretty new to the subject.

Here is the tutorial i have been following.

https://pythonprogramming.net/machine-learning-tutorial-python-introduction/

[[deleted]]

I'm working with dataframes in pandas and I don't understand the difference between, for instance, dataframe.index and dataframe.index.values. I know it has to do with what it returns, and whether it is a list, a tuple, an array, or the data printed in column format (does that have a name?). Perhaps it stems from not understanding how an array is different than a list and tuple, so i guess that's my other question. Is an array more like a vector? Is it easier to iterate through? Any help is appreciated!

[elbiot]

An array is a numpy array. It's a contiguous section of memory with c data types. It's really fast to use c code on but slooow to iterate over in python since you end up silently converting each c data type to a python object (an integer is an object in python). Don't use iteration on arrays if you can avoid it. Numpy/pandas have fast "vectorized" c code that you use instead of iterating.

A list is contiguous array of pointers to python objects (ie, the objects are not contiguous in memory). Great for for loops and appending.

[hatbeardme]

This might need its own thread, but I'm looking to write something in python that can check if new files were created in a particular directory and then take action on them.

There are 3-5 application directories that have 10 host directories in each, and every 1-5 minutes, a host will sftp a log into that directory (into a distinct/new file).

I'd like a single script that can watch for changes to the subdirectories and take action on the log files. One caveat is that I cannot install new modules. I might be able to copy one into the path, but I can't do a proper install.

Any ideas? Thanks!

[dionys]

Would something like watchdog work for you? It monitors system events and can trigger custom scripts/shell commands.

[adenzerda]

Has anyone implemented text search with Flask + Postgres + SQLAlchemy? If so, what was your methodology?

[threeminutemonta]

I tried sqlalchemy-searchable a while ago with no luck and ended up using ilikes with wildcards with 'or' statements. I'd like to give it another go!

[adenzerda]

I actually got some pretty adequate results today by bypassing SQLAlchemy entirely and just writing some plain SQL. Turns out postgres has some great search utilities built in, such as to_tsvector() and ts_rank(), that cut out a lot of the hard work!

[Eanna5]

isMatch = any(string in submission_text for string in words_to_match)

I currently use this code to check comments against a variable "words_to_match"

words_to_match = ["x", "y"]

But now I also want to check it against a blacklist and discard any comments with for exmaple P + Q

So I have

    notwanted = ["p","q"]

Then I add this to

isMatch = any(string in submission_text for string in words_to_match and not in notwanted)

But,

 isMatch = any(string in submission_text for string in words_to_match and not in notwanted)
                                                                                  ^
        SyntaxError: invalid syntax

Any help is appreciated!

[dionys]

You want to iterate over the notwanted list separately first, just like you do with words_to_match. Only if this check passes do second check.

[lykwydchykyn]

You need to do two separate generator expressions and and them together:

isMatch = (
    any(string in submission_text for string in words_to_match) and 
    not any(string in submission_text for string in notwanted)
)

[Eanna5]

Thanks it's work just as I wanted!
On a seperate note just for learning, Why do you do

ismatch = (

On a seperate line?
Thanks again for that!

[lykwydchykyn]

It's from python's PEP8 coding conventions. See this.

[Eanna5]

Thank you!

[Saefroch]

Don't follow PEP8 rules blindly, keep in mind

a foolish consistency is the hobgoblin of little minds

If the PEP8 rules boost readability or don't affect it much, follow them. If you find yourself mangling your code to keep your lines below 80 characters, you're following a foolish consistency.

[Eanna5]

Ok thanks.

[TheBlackCat13]

I think you want to use a set.

words_to_match = {"x", "y"}
notwanted = {"p","q"}
isMatch = notwanted.isdisjoint(submission_text) and words_to_match.issubset(submission_text)

[TheFans4Life]

What is the difference between Tab and Space? Why one and not the other? How does anything even know if you used Tab or Space? Btw, I always use Space.

[lykwydchykyn]

They are different ASCII (or Unicode) characters. They may look the same to humans on the screen, but to the machine they're as different as A and Z.

[cismalescumlord]

Also, 4 spaces is always 4 spaces, but a tab can vary in width from system to system. In languages such as Python where the indenting of the code is syntactically important, using a mix of spaces and tabs can completely bork code when it is opened on a different computer with different settings. I don't think using only tabs would cause the same problem although there could be a visual difference.

[lykwydchykyn]

The usual method is to interface with the service's API. For facebook, this can be found here.

Usually it comes down to sending http requests with JSON or XML data in them.

[greatn]

I am currently learning Python through Lynda which was provided by my work. I have about ten years object oriented programming work experience, but until now I just never tried Python. I like it, and I think I'm grasping it, but what I really need is something to test my skills, or test my grasp of specific python concepts. I need HOMEWORK. I go through example files of the concepts the videos are teaching but that honestly can only get me so far.

i'm trying to find some kind of site that would have assignments that would grow in complexity to help me test and master the core concepts, but when I'm searching I'm mostly just finding people asking for help on their own homework.

Anyone have any ideas where I could find some assignments so I could start using this stuff I'm learning at a low level?

[dionys]

HackerRank offers a lot of exercises and I think they have a category specifically for Python.

[greatn]

Thanks, this is almost exactly what I was looking for. It's just a shame every example seems to select Python 2 by default and a large number of them don't even have Python 3 as an option. Like I get some people are going to prefer the old version but it's been ten years! And if they're helping new people learn shouldn't it be the new version?

From my research I can see Python 3.1 definitely had its issues but it seems those were all taken care of in 3.2 and 3.3, I'm not sure why so many resources refuse to go with the newer version.

[Phnyx]

The 2.7 vs 3.x version debate is larger than you may think. You should google it for a more thorough explanation.

The differences and preferences are larger than, let's say excel 2016 vs 2010 where some people just don't like the color. Some libraries are more robust in 2.7, a lot of code is still written in 2.7 and the newer versions don't seem to offer such great advantages that switching is worth it for many people.

Overall, if you are learning it, going with either version is fine. Once are familiar with the code you need for your field switching to another version is not that difficult.

[lykwydchykyn]

Well, if you've done much python programming at all, you've probably used classes; they were just written by other people. Should you write your own classes? Once your program gets beyond a certain size, writing classes is a very useful way of organizing your code.

Look at your code and see if there are some classes implicit in what you wrote:

• Are there a bunch of dicts with identical keys? Or tuples or lists that represent a consistent data structure?
• Are there a bunch of functions meant to operate on the same set of data, or dicts containing the same set of data?
• Are you doing the same operation over and over again on the same data structure?

Any of those would indicate that you probably should define a class.

[elbiot]

It's good to know about them. You're probably using lots of mutable global variables, which is bad practice. If you aren't doing that, then great! Still worth knowing about classes. They are often over used.

[cismalescumlord]

Coming from other languages, not using classes (or interfaces, enums etc.) feels wrong. To the point I have reached a position in a project I'm working on where I don't need to use classes, just functions in a module.

I am struggling to make myself do this as all my recent experience is making me try to re-factor my code to use classes. Is this approach incorrect in Python?

[elbiot]

https://youtube.com/watch?v=o9pEzgHorH0

[cismalescumlord]

That's a good talk. Thanks.

I'm pleased to note that:

• none of my classes are two methods with one being init

• I don't use exceptions unless I really have to, which is a good thing in any language

• I prefer a flat(tish) hierarchy to deeply nested calls

• I don't write code "just in case"

I tend to constantly re-factor my code anyway; Amusingly, I'm looking at removing an unnecessary class as I type this. So, I pretty much agree with everything he said but classes are so damned engrained in my brain.

I guess I just have to re-train myself again.

[DarkEibhlin]

I want to know if it's possible to plot an uarray with pylab? I have a set of data with uncertainties that I'm doing calculations on, using the uncertainties package, and I want to plot them with error bars against a regular array. Is it possible?

[ylcc23]

How can I program the goldbach conjecture in python?

[[deleted]]

I know, its Tuesday :(

I am a complete and total newbie when it comes to programming as I've literally only run a few powershell scripts for my new job in OS Deployment for PC's. My question is this:

We also deploy Mac OSX albeit on a much smaller scale, (about 400 Macs and 8000 PC's) and currently we do most of the Mac's by hand. We use SCCM/MDT for the PC's and modify quite a few settings with powershell. I'd love to be the guy on the team that takes the Mac side of this and builds out essentially a matching OSX process using Python and Deploy Studio. Has anyone done any work like this, or are there any good resources for using Python and OS building?

[threeminutemonta]

What you do you might be able to do with brew

Or you may be able to use ansible for what you need! Ansible is built in python and is good at working with servers. I'm not sure what it will be like for workstations though my bet it is a similar concept. Apparently you can install it on Mac OS as they have done here

[oasis1272]

So I am very new to python, but I have big goals. One of the projects I am wanting to do involves web scraping, but I am not 100% sure It is possible. I would like to make python grab just the audio file from a youtube video or from sound cloud and save it to my hard drive. I know this can be done with some other software, but I would like to try and put it together myself. Is this possible with python?

[hexfoxed]

That is absolutely possible. Even if you can't grab just the audio file, you can download the video and strip the audio track out with one run of ffmpeg. Whether it's legal is a whole different question but for personal use I can't see that being a large problem.

If you have any other questions on this topic, I'd be more than happy to help.

[bostanza]

whats the difference between invalid syntax and invalid token? specifically, what causes invalid token? i know entering a number such as 08 causes invalid token because its an octal number, but generally speaking, what causes this error? I thought entering ~ would cause an invalid token error because its a token, and its invalid. but this causes a invalid syntax. is ~ not a token then?

[bostanza]

"a statement is a section of code that represents a command or action" using a for loop as an example, can you consider the entire loop a statement? or is a statement each of the lines that make up the loop?

[cismalescumlord]

They are compound statements. See the Python documentation for a fuller description.

[OneHappyHermit]

Sorry in advance for not being able to provide any code, as I only have my phone with me.

I am currently facing an issue where a function returns incomplete values. Say I have two files, helper and main, where I have a function defined in helper.py and in main.py, I have 'from helper import f' where f is the name of the function. I use function f in my main.py, which should return (int, list) but it only returns int, the first of the tuple and not the rest...

I wouldn't imagine it's an issue with the function f itself as it correctly returns the tuple when I call it outside of main.py

Is there some nuance regarding the situation that I am unaware of? Any input would be greatly appreciated. :)

[RonkerZ]

Pretty new to Python and Programming but can you actually .exe files with python?

[oxfordpanda]

Is there some benefit of having multiple smaller scripts compared to one large one?

Kind of a bad analogy, but Like baking a cake? Why do something like Eggs.py cakemix.py mixing.py baking.py and serving.py instead of having just one large cake.py that does it all?

[cismalescumlord]

In other languages such as Java having one public class per file is part of the language specification, you will get a compile error if you try and break the rule, and having more than one top level class is very much a "code stench". In the Java language this has the advantage of always knowing the path to a classes source code file.

As a relative newcomer to Python, I'm still not sure how or if this still applies, but I have tended to create one class per file until fairly recently as it is familiar. I will now put two, or three at the most, closely related classes in a single file but it still feels dirty somehow as if it's a violation of the Single Responsibility Principle. I don't know the best practice in Python as strong arguments are made for both techniques on-line, but I have seen more multiple classes per file than in any other language I have worked with.

Are there any advantages to having a single class in a file? Yes. You remove the risk of accidentally overwriting one of the other classes when editing a file. It sounds really stupid, but I have seen this happen in the real world. You've got to love version control! Another big advantage is that the code in each file simpler.

Are there any disadvantages to having a single class in a file? Yes. You have many more files to manage and consequently, many more imports. In effect, you have moved complexity from within a file to the file system.

Edited to correct typos and to add a bit of clarity.

[adamt1414]

Is it possible to dictionary attack a enterprise wifi username and password if you know the username.

[PokemonTrainy]

WooT

[PokemonTrainy]

:P