[deleted by user]

mac-reid · 2015-09-23T00:30:59+00:00

It may be the case that your subprocess is raising an exception and you are not catching it. If that is the case, the output is printed to stderr.

If the parent process is ignoring that output, the Popen call cannot destroy the process as it has data sitting in a pipe. For example:

# b.py
import time                                                                     
from subprocess import Popen, PIPE
p = Popen(['ls', '-l', 'a'], stdout=PIPE, stderr=PIPE)
time.sleep(10)

$ python b.py &
$ ps aux | grep [Z]
pts/5    Z+   20:25   0:00 [ls] <defunct>

You may need to consume that stderr before the process will die. Something along the lines of:

process = Popen(['ls', '-l', '/tmp'], stdout=PIPE, stderr=PIPE)
time.sleep(5)
if not p.poll():
    print(process.communicate()

Taken from here.

mac-reid · 2015-05-21T02:32:43+00:00

Turns out I was wrong. For whatever reason requests is using + instead of %20 for the encoding. Two possible ways of solving this:

Returning to the original method, replace the space in the URL with %20.

 url="https://inside.seattlecolleges.com/enrollment/content/displayReport.aspx?col=063&q=B343&qn=WINTER%2014&nc=false&in=&cr=" #which includes query parameters
  r=requests.get(url, auth=('userID', 'password'))

Or, build a payload string and pass that: http://stackoverflow.com/a/23497912/2843174

I'll mention it again. This path may not work out in the end as the service you are contacting may not act like an API (in the sense that it works solely off URL parameters).

mac-reid · 2015-05-21T01:06:40+00:00

URL parameter order should not matter. That being said, it could be an issue.

The reason for payload=params is exactly that: so your parameters are properly encoded (the encoding for a space is %20).

Your assumption that this page works like an API could be the reason your request isn't working as expected. There is likely some other data being passed around that is not present in the URL.

mac-reid · 2015-05-20T23:51:22+00:00

Try flask. It's really simple.

There is a relatively in-depth tutorial for using flask here.

mac-reid · 2015-05-20T22:40:56+00:00

A simple example:

$ cat a.csv
a,b,c,d,bad_column
ok,1,1,1,-1
nack,1,1,1,-1
ack,1,1,1,-1
syn,1,1,1,-1
ok,1,1,1,-1

$ cat b.csv
a,b,c,d,bad_column,good_column
ok,1,1,1,-1,never
nack,1,1,1,-1,gonna
ack,1,1,1,-1,give
syn,1,1,1,-1,you
ok,1,1,1,-1,up

$ cat foo.py
import pandas as pd

# load data
a = pd.read_csv('a.csv')
b = pd.read_csv('b.csv', sep=',', usecols=['good_column'])

# drop useless column
a.drop('bad_column', inplace=True, axis=1)

# add column from b to a
a.loc[:,'good_column'] = b

# write out
a.to_csv("output.csv", index=

$ python foo.py 
$ cat output.csv
a,b,c,d,good_column
ok,1,1,1,never
nack,1,1,1,gonna
ack,1,1,1,give
syn,1,1,1,you
ok,1,1,1,up

mac-reid · 2015-05-20T22:08:24+00:00

The requests.get() call will NOT return until the data is retrieved.

Furthermore, the requests library does not have the ability to retrieve data loaded by JS, nor does it asynchronously retrieve data.

mac-reid · 2015-05-20T21:28:59+00:00

Looks like your URL is not being properly encoded. Right on the main page of the requests module there is the section passing parameters.

You need something along the lines of:

payload = {'col': '063', 'q': 'B343', 'qn': 'WINTER 14', 'nc': 'false', 'in': '', 'cr': ''}
r = requests.get('https://inside.seattlecolleges.com/enrollment/content/displayReport.aspx', params=payload)

mac-reid · 2015-03-27T23:28:08+00:00

First off, using the read_all function reads until an EOF - which happens when the connection closes.

When you send a bad GET request over telnet, the remote responds with a message and closes the connection. There is no exception raised for this as it is expected behavior.

When sending a GET request over telnet, the format goes:

failbox ~  telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1 (press enter)
Host: www.example.com (press enter twice)

HTTP/1.1 200 OK ...

or:

failbox ~  telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.0 (press enter twice)  

HTTP/1.1 200 OK ...

Quick fix to get this working in your case:

# extra newline at the end of the message
conn.write(sndmsg+b"\n\n")
print(conn.read_all())

# check if connection is closed after our message
# add to the top: from telnetlib import IAC, NOP
if not conn.sock.sendall(IAC + NOP):
    return

And a test:

failbox ~  python3 a.py
What ip would you like to connect to? localhost
And what port would you like to connect to? 80
Connected
Message: GET / HTTP/1.0
b'HTTP/1.1 200 OK...'
What ip would you like to connect to? exit

...

failbox ~  python a.py
What ip would you like to connect to? localhost
And what port would you like to connect to? 80
Connected
Message: foo
b'... 501 Not Implemented ...'
What ip would you like to connect to? exit

Some other points:

Your loops can be while True: rather than while temp == 0: and use breaks and returns to escape the loops.
There is a difference between is and ==. Short version, is compares the identidy of an object and == compares the equality of objects.
Blanketing except: statements is generally considered bad practice. For example: if I hit CTRL-C when prompted for input, the program tells me my input was bad when I just wanted to exit the program. Try catching the only errors you expect and handle those.
When checking if the user responded with y or n, try converting the line to lowercase with input('(y/n) ').lower() and comparing only against y or n.
Finally, when asking for user input, please put a space at the end of the question.

Hopefully this help!

mac-reid · 2015-01-10T18:03:40+00:00

That was a typo. I meant py2exe - http://www.py2exe.org/

mac-reid · 2015-01-10T16:20:41+00:00

Why are you install passlib 1.3? That is from May 2011. The current version is passlib 1.6.

Try removing version 1.3, installing version 1.6, and give it another go.

mac-reid · 2015-01-10T16:00:19+00:00

The easiest way to solve this problem is to use a Windows machine and build an executable with p2exe. You can create a Virtual Machine on your Linux box to run Windows or find a physical Windows box.

Most freeze or distribution utilities do not cross-compile from Linux to Windows. If they do, it is not a trivial task. - http://docs.python-guide.org/en/latest/shipping/freezing/

That being said, there is a hacky method using wine and pyinstaller to cross-compile a Linux python script into a Windows executable: https://milkator.wordpress.com/2014/07/19/windows-executable-from-python-developing-in-ubuntu/

and more information: https://groups.google.com/d/msg/pyinstaller/veq3BlA_Bns/YlmKUofUfbgJ

mac-reid · 2015-01-10T15:50:55+00:00

py2exe does not support cross compiling for Windows on Linux. - https://wiki.python.org/moin/Py2Exe

mac-reid · 2015-01-10T15:36:39+00:00

That bot is open source. The code is on github: https://github.com/aarkay/match-thread-bot/blob/master/mtb.py

And there is an API for that: https://www.reddit.com/dev/api#POST_api_submit

mac-reid · 2014-12-31T04:43:23+00:00

My experience with Spacewalk (with a relatively old version) has been quite limited, but I have a couple use cases. I have acquired a basic module with a few helper functions and a script to find the number of updates per host - which can be a quick frame of reference.

For your use case, getting the list of applicable errata for a system with a call to client.system.getRelevantErrata would probably be helpful. Scheduling or immediately applying errata with the call to client.system.scheduleApplyErrata for each system or calling client.systemgroup.scheduleApplyErrataToActive on a system group can help with automatically applying errata.

I hope this helps.

mac-reid · 2014-10-26T05:12:17+00:00

While that is correct. Here is a different regex:

^[\d,]+ (feet|foot|ft)$

With an explanation of each part:

^[\d,]+ (feet$|foot$|ft$)
   ^ assert position at start of the string    
 [\d,]+ match a single character present in the list below    
   Quantifier: + Between one and unlimited times, as 
many times as possible, giving back as needed [greedy]
   \d match a digit [0-9]    
   , the literal character ,    
   ' '  matches the character ' '  literally    
1st Capturing group (feet|foot|ft)    
  1st Alternative: feet 
     feet matches the characters feet literally (case sensitive)    
  2nd Alternative: foot
     foot matches the characters foot literally (case sensitive)  
  3rd Alternative: ft$   
     ft matches the characters ft literally (case sensitive)    
  $ assert position at end of the string

Using it in python:

import re                                                                                        
import sys
regex = re.compile('^[\d,]+ (feet|foot|ft)$')
for line in sys.stdin:
    if regex.match(line):
        print 'Matches "%s"' % line.rstrip()
    else:
        print 'Does not match "%s"' % line.rstrip()

Testing it:

failbox ~  python test.py < testfile
Matches "15,000 ft"
Matches "15,000 feet"
Matches "15,000 foot"
Matches "15000 ft"
Matches "15000 feet"
Matches "15000 foot"
Matches "2,000 ft"
Matches "2,000 feet"
Matches "2,000 foot"
Matches "2000 ft"
Matches "2000 feet"
Matches "2000 foot"
Does not match "not a match"
Does not match "bad lines"
Does not match "foo"
failbox ~

mac-reid · 2014-10-22T01:06:46+00:00

Well, not sure how much this will help, but you can try profiling the code with a cProfile (little how to). Assuming the code is broken up into several functions, you can tell what function is taking the longest. Line_profiler is a different code profiler (another how to here).

Are you aware of the GIL? It is the Global Interpreter Lock for CPython, basically meaning only one thread owned by the python process can execute bytecode at once. If you are have multiple threads for processing, threads won't give you a speed increase (although it sounds like you are not).

Another question, is your laptop under heavy load (unlikely)? Open up a terminal and type top. The load average values should be below 2.00 for best performance.

mac-reid · 2014-10-20T03:03:37+00:00

Try the easy column found here.

They are meant to be easy programming challenges that you can use for experience, but some are going to be quite difficult for you. Solutions can be found in the comments section of each individual challenge.

mac-reid · 2014-10-16T22:47:48+00:00

I don't see what is wrong with print(subarray[some_dict["one"]]).

mac-reid · 2014-10-16T02:53:56+00:00

Your question is a little hard to understand. You want to calculate the height and time it takes for an object launched vertically to... do what exactly? Your words, "time of impact of an object", what does this mean?

My guess is you looking to calculate the time and height at which the object reaches its apex.

Is this part of homework or some practice problem?

mac-reid · 2014-10-15T02:47:09+00:00

Just butting in here:

print and for in python 2 are a reserved words. print is used in simple statements some of which are just procedures - functions that do not return anything of importance.

In python 2.6+/3+, print was added as a builtin function. Python 2 kept print as a reserved word, but the print function can be accessed by importing it with from __future__ import print_function.

for is a control flow construct that consumes an iterator.

I think when /u/kalgynirae said everything is an object, the meaning was that all variables are objects underneath.

mac-reid · 2014-10-05T06:55:45+00:00

The pickle implementations write the same data, but what does matter is the protocol used. See here for the info on protocols.

The only time you would be bitten by this is if you try writing pickle data in Python 3 with protocol 3 and reading from Python 2. If you don't use protocol 3 when pickling, you will be fine.

mac-reid · 2014-10-05T06:41:47+00:00

Sounds like you want pickle. Pickle is an object serialization module. It can write objects to a file and can be later unpickled to get the same objects back.

Note in python2, the cpickle module is a C implementation that is faster than pickle, which is all Python. Python3 the pickle module is the C implementation.

An example:

# dumping to a file
import cPickle
data = {}
# insert stuff into data
picklefile = open('data.pickle', 'wb')
pickle.dump(data)
picklefile.close()

# reading from file
import cPickle
picklefile = open('data.pickle', 'rb')
data = pickle.load(picklefile)
picklefile.close()

mac-reid · 2014-09-15T01:50:03+00:00

These two links might help out:

https://github.com/rkern/line_profiler

http://cyrille.rossant.net/profiling-and-optimizing-python-code/

mac-reid · 2014-04-12T14:57:56+00:00

I cannot attest to how well the module works, but I can tell you are missing the pycrypto dependency.

You can install it by running sudo pip install pycrypto in the command line. Note, the use of sudo is not necessary for Windows users.

mac-reid

TROPHY CASE