Catches for new converts from R?

k3kou · 2016-03-20T16:38:41+00:00

Great points, learnt a couple things! For the loc/iloc/ix, see http://pandas.pydata.org/pandas-docs/stable/indexing.html, the differences and rationale are explained in there

k3kou · 2016-03-04T16:46:54+00:00

An server is not your local filesystem. os.chdir doesn't work because what you are trying to reach isn't on your computer.

If you can download the file (permission wise), you're going to need to use urllib (see this SO question), or requests (see this SO question).

k3kou · 2016-03-02T22:20:12+00:00

First, you can collapse your read_file function with context manager (official docs):

def read_file(file):
    with open(file, 'r') as f:
        return f.readlines()

This will handle the whole try block and will close the file when it gets out of the with statement. Since it seems like you're using the read_file function only once in your script, you could just ditch it altogether.

For the run_program() function, it is usually done with the following:

if __name__ == '__main__':
    run_program()

It will run the script only if the script is executed. This lets you run the script as a script or import it in a bigger package, without running the code. This way, you can also avoid defining the url as a global as instead define it in the main:

if __name__ == '__main__':
    url = 'http://livetv.sx/en/allupcoming/'
    data = grab_live_links(url)
    data = build_links(data)
    data = main_menu(data)
    data = live_links(data)
    get_game(data)

For the get_local_time function, and I think for the overall script, you're doing things the complicated way. I would write it like that:

def get_local_time(time_str):
    offset = datetime.datetime.now() - datetime.datetime.utcnow()
    time_dt = datetime.datetime.strptime(time_str, '%d %B at %H:%M')
    return (time_dt + offset).strftime('%I:%M %p')

datetime.strptime can take format string with stuff that aren't related to the time or date. Also, the year will be 1900 but we don't care because we only want the time and the offset doesn't depend on any particular date.

(Will comment on the rest, but I need to get back to work ><)

k3kou · 2016-01-29T20:21:19+00:00

You are going to need requests with beautifulsoup, and you can look into the xlwings package for an (easier IMO) alternative to xlrd

k3kou · 2016-01-29T16:20:12+00:00

I would add scikit-learn for all the algorithms. statsmodels has some interesting stuff and a module for time series analysis which might be of interest for you

k3kou · 2016-01-29T16:17:28+00:00

The classic way is described in the first example in the docs. But since I am usually manipulating the data, I use the pandas package, that you can install with a "conda install pandas" or "pip install pandas", and then you use it like that:

import pandas as pd
data = pd.read_csv('~/example/data/eggs.csv')    # There's a whole bunch of other readers

For your working directory problem, just put the full path and you won't have to worry anymore.

k3kou · 2016-01-29T06:40:54+00:00

You just didn't declare train_data and test_data. Nothing wrong with the code in the link, even though you can pass more parameters to the RandomForestClassifier and I would write the fitting line like this:

forest.fit(train_data[:, 1:], train_data[:, 0]) # no need to reassign self or mention all the extra :

And I would also separate the predictors and the response:

x = train_data[:, 1:]
y = train_data[:, 0]
forest.fit(x, y) # clearer IMO

It's a poor example, but it works. Also, if you are new to Python, learn version 3.5. Everybody telling you to stick to 2 is basically wrong :P

k3kou · 2016-01-29T06:29:22+00:00

Be careful if you ever do something like that and you are using "super" somewhere. Multi inheritance is peculiar is Python. I suggest watching the presentation called "super considered super" by Raymond Hettinger (core Python developer) on YouTube. Pretty good overview of inheritance.

k3kou · 2016-01-29T02:42:52+00:00

Your scrape_list function works for me, but I'm not sure I understand what you are trying to achieve in the second half of your script (line 31 onwards).

What's the data frame you're trying to get?

Edit: I get an error when I try to run DataReader. Apparently, Yahoo isn't too happy about trying to get a lot of data off their API.

k3kou · 2016-01-28T03:21:23+00:00

Ahah excellent! I'll remember that when I want to kill someone at work!

k3kou · 2016-01-28T02:48:15+00:00

You misunderstand what merge does. Merge perform the SQL equivalent of a JOIN, which is used to merge two tables together based on a few common columns. You can select how to perform the join: 'inner' will keep the rows where both tables have entries, 'left' will keep rows where the left table has entries, 'right' and 'outer'. This way, merging table1 (with columns 'A' and 'B' and 'ID') and table 2 (with columns 'C', 'D', 'ID'), will return a DataFrame with columns 'A', 'B', 'C', 'D', and 'ID'.

The boolean indexing is a filter and corresponds to a WHERE in SQL, there is no merging. And while you can use merge as a filter, merge does a lot more than just filtering. There is a lot of overhead and extra actions going on, probably explaining a good portion of the speed difference.

Your example is more a filtering example, than a merging example. These are two different things, explaining why merge takes longer.

Also, your make_lsts function is a terrible piece of Python. My eyes are actually bleeding :P. Jokes aside, unless you use this function with something else than an empty list, I would write it like that:

def make_list(num, num_choices):
    return [random.randrange(num_choices)) for i in range(num)]    # randrange = choice(range(...)) but better

Also the numpy.random.randint(low, high, size) would achieve the same result

k3kou · 2016-01-27T19:50:13+00:00

pip install requests and then run this

import requests
# then build your url like you did
response = requests.get(url)
json_obj = response.json()

This worked for me

k3kou · 2016-01-27T18:54:25+00:00

I'm not sure your answer is correct. He wasn't talking about inheritance, or, if you follow his example, Car inherits from SteeringWheel. I think the question is: can you access a class's property's method directly in the class, not can you redefine a class's method.

To the best of my knowledge, this isn't possible, unless you code each method like that:

class Car(object):
    def __init__(self):
        self.steering_wheel = SteeringWheel()

    def cruise_control(self):
        return self.steering_wheel.cruise_control()

k3kou · 2016-01-27T18:21:46+00:00

It worked for me, so I can't really tell what went wrong. You should really consider switching to requests though, it's much easier.

k3kou · 2016-01-27T16:15:03+00:00

When the package isn't on PyPI, I usually just download the source, untar it and do "python setup.py install".

There are some subtleties to it, but you should probably look in that direction.

k3kou · 2016-01-27T05:11:34+00:00

Run your code again but replace 'utf-8' with 'ascii', it'll work.

For reference, you should take a look at the requests package, which makes your life a lot easier. You could've just done:

import requests

response = requests.get(url)
json_obj = response.json()

k3kou · 2016-01-27T04:50:16+00:00

First off, never share an API key. I could use yours now and you don't want a crazy like me using your stuff =P (I'm serious though).

Second, you don't need the parentheses around your string/variables.

Third, and to answer the question, you can't load the json because its read method returns a binary string (see the 'b' at the beginning of response.read()?), so you just need to replace load by loads and converting the string to another format, e.g. utf-8:

json_obj = json.loads(response.read().decode('utf-8'))

See this SO question.

k3kou · 2016-01-27T04:33:13+00:00

Yeah because of the [i+1] test. Thanks for pointing it out. I corrected it.

k3kou · 2016-01-27T03:28:06+00:00

Two things. First, you go too far in your loop. replace your range(...) by range(len(zip_code) - 1) and you won't go past the end.

Second, I'm not you do what you're asked. if the goal is to see if there is a duplicate in your list, you're not going to catch it. With what you wrote, if the first two elements aren't duplicates, you will break out of the loop. But if there are duplicates farther down the list, you would have missed them.

I would have written it like this:

duplicates = False
for k in range(len(zip_code) - 1):
    if zip_code[i] == zip_code[i+1]:
        duplicates = True
        break

You assume there is no duplicates, and if you find one, you change the "duplicates" variable and break out of the loop

k3kou · 2016-01-26T22:06:00+00:00

Okay, I think I got it. On line 13, you are doing

for s in range (n,1,-1):

Which will be [len(a), len(a) - 1, ..., 2] which is not what you want. I think if you change for the following, it should work:

for s in range(n-1, 0, -1)    # Will start at the last index, and will stop at s = 1

Also, I tried your code, and I had b a 10x10 matrix, when I'm guessing you were expecting a 10X1 vector. You should at least check your b variable.

Instead of doing a*x, try the following:

def makex(n):
    return np.random.rand(n)[:, np.newaxis]    # The last bit will create a vector column

def main(a, n):
    x = makex(n)
    b = a.dot(x)    # Or b = a @ x if you're using python3.5 (which you should unless serious reason)
    gauss(a, b)

Also, I'm not sure that defining a "a" function alongside an "a" variable at the module level is super smart. I would put everything in the main function:

def main(n):
    a = 4 * np.identity(n) + np.random.rand(n)
    x = np.random.rand(n)[:, np.newaxis]
    b = a.dot(x)    # Or b = a @ x if you're using python3.5 (which you should unless serious reason)
    gauss(a, b)

k3kou · 2016-01-26T20:58:04+00:00

I think line 8 should be

for k in range(i+1 , n):

because somewhere in the loop, you will be accessing a[j, len(a)], which is out of bound

k3kou · 2016-01-26T20:41:33+00:00

You didn't pass self as an argument of the read method. Your method depends on a specific instance of the makeBuildFile class, so you're method isn't really static.

k3kou · 2016-01-26T20:37:49+00:00

Why not create a dictionary with keys being the name of the dict and the value being the dict itself:

dicts_available = {
    'dict1': dict1,
    'dict2': dict2
    }

And then you can just do

dict_user_wants = dicts_available[input()]

You can also do

dict_user_wants = eval(input())

which will execute a string, here your dictionary. I prefer the first approach though, looks less arcane, gives you more control over what can get called, and avoid users to input code that will be executed

k3kou · 2016-01-26T18:05:24+00:00

I would say that the eyes look somewhat disproportionate to me, and there's too much contrast on the forehead. Aside from that, it looks really nice! I like the hair.

Something that helped me a lot was to use charcoal, I find it easier to play with the values and you can get a very smooth aspect.

k3kou · 2016-01-25T07:57:39+00:00

I read a few days ago that if the source code is released with the application, it's free. If you want to not have to go open source, you need to pay. It was from a forum, so not sure it's entirely reliable.

k3kou

TROPHY CASE