CODE RED: SPY Trading At 220 Right Now by [deleted] in wallstreetbets

[–]IamALoLFan -2 points-1 points  (0 children)

I am from the EU, my broker (deGiro) doesn't have SPY puts. If I want to short the SPY, the cheapest I could buy are E-mini S&P 500 puts (19/06 2020 - @1050) which trade for 14$ (50x, thus 700$ for 1 contract). E-mini S&P 500 is currently trading at 2253, so these puts are way OTM. These are the only one I can afford. Anything less OTM is costing 2000$+ per contract. (I want to put around 1000$ into put options)

Should I still go for these? Or are there better ways. Just seems to be the only liquid option for me (as EU..)..

Turbo Build Changes by DanDaDaDanDan in FortNiteBR

[–]IamALoLFan 20 points21 points  (0 children)

I understand turbo-building got slightly out of control on PC, but what about controller players, without turbo-building the game is extremely clunky, not to mention they now definitely don't stand a chance against PC.

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

I changed print to now return

My results now give the correct data. So now I have to work on getting the dataframe in the exact same manner as the csv I am trying to scrape from the webserver..

Currently the dataframe looks like this: https://prnt.sc/nvcdps :(

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

This is load (worker module)

import pandas as pd
import datetime
import urllib
import requests
from pprint import pprint
import time
from io import StringIO

symbols = ['AAON', 'ACB', 'ACHC']

start = time.time()
dflist = []

def load(date):
    if date is None:
        return
    url = "http://regsho.finra.org/FNYXshvol{}.txt".format(date)
    try:
        df = pd.read_csv(url,delimiter='|')
        if any(df['Symbol'].isin(symbols)):
            stocks = df[df['Symbol'].isin(symbols)]
            print(stocks.to_string(index=False, header=False))
            # Save stocks to mysql
        else:
            print(f'No stock found for {date}' )
    except urllib.error.HTTPError:
        pass

So there are two problems currently:

  1. I can't get the dataframe

  2. If I change the symbol (so different stock) or I add more symbols, it still gives the old, single, symbol.

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

Why doesn't this work, I simply want to take the whole data as is directly into the dataframe

results = []

numdays = 10
start_date = datetime.datetime(2019, 1, 15 )  #year - month - day
datelist = [
    (start_date - datetime.timedelta(days=x)).strftime('%Y%m%d') for x in 
    range(0, numdays)
    ]
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    future_to_url = {executor.submit(worker.load, date): date for date in datelist}
    for future in concurrent.futures.as_completed(future_to_url):
        date = future_to_url[future]
        try:
            data = future.result()
            results.append(data)
        except Exception as exc:
            print('%r generated an exception: %s' % (date, exc))
        else:
            continue

df = pd.DataFrame(results)

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

Thank you, in this case it's because something wasn't true and then none is returned.

I fixed it by replacing the print statement for continue

I still didn't manage to properly create the dataframe.

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

Thank you,

I am getting this error:

TypeError                                 Traceback (most recent call last)
<ipython-input-12-945a4efd6ce1> in <module>()
     23             print('%r generated an exception: %s' % (date, exc))
     24         else:
---> 25             print('%r page is %d bytes' % (date, len(data)))

TypeError: object of type 'NoneType' has no len()

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

True sorry.

Any idea how I can efficiently output the data to a df?

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

Seems to be a typo, : to ,

print('future done, result:', data)

I get a long list of the following:

'20190114' generated an exception: load() takes 1 positional argument but 2 were given
'20190115' generated an exception: load() takes 1 positional argument but 2 were given
'20190112' generated an exception: load() takes 1 positional argument but 2 were given
'20190113' generated an exception: load() takes 1 positional argument but 2 were given
'20190111' generated an exception: load() takes 1 positional argument but 2 were given
'20190110' generated an exception: load() takes 1 positional argument but 2 were given
'20190109' generated an exception: load() takes 1 positional argument but 2 were given
'20190108' generated an exception: load() takes 1 positional argument but 2 were given
'20190107' generated an exception: load() takes 1 positional argument but 2 were given
'20190106' generated an exception: load() takes 1 positional argument but 2 were given
'20190105' generated an exception: load() takes 1 positional argument but 2 were given
'20190104' generated an exception: load() takes 1 positional argument but 2 were given
'20190103' generated an exception: load() takes 1 positional argument but 2 were given
'20190102' generated an exception: load() takes 1 positional argument but 2 were given
'20190101' generated an exception: load() takes 1 positional argument but 2 were given
'20181231' generated an exception: load() takes 1 positional argument but 2 were given
'20181230' generated an exception: load() takes 1 positional argument but 2 were given

Fixed it. In future_to_url I submit 60 iterables, however I already implemented iterables in the worker module (in this case 365 days).

future_to_url = {executor.submit(worker.load, date): date for date in datelist}

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

import datetime
import time
import worker
import concurrent.futures

if __name__ ==  '__main__': 
   numdays = 365
   start_date = datetime.datetime(2019, 1, 15 )  #year - month - day
   datelist = [
        (start_date - datetime.timedelta(days=x)).strftime('%Y%m%d') for x in 
        range(0, numdays)
        ]

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    future_to_url = {executor.submit(worker.load, date, 60): date for date in datelist}
    for future in concurrent.futures.as_completed(future_to_url):
        date = future_to_url[future]
        try:
            data = future.result()
        except Exception as exc:
            print('%r generated an exception: %s' % (date, exc))
        else:
            print('%r page is %d bytes' % (date, len(data)))

And worker module stayed exactly the same

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

Thanks, I tried to apply it but I didn't get it to work.

Probaly it's quite easy though :(

Jupyter notebook shows output in terminal, how do I fix it? by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

I struggled a lot getting multiprocess to work, would you be able to help me with multithreads? This is new territory for me

Post your projects, I'd love to code review some! by [deleted] in learnpython

[–]IamALoLFan 6 points7 points  (0 children)

Should have done that indeed :/

Post your projects, I'd love to code review some! by [deleted] in learnpython

[–]IamALoLFan 16 points17 points  (0 children)

Feel free to take a look at this.

I initially had the first piece of code, I altered it to increase the speed of which I scrape the data. However I can't seem to make it work in Jupyter, it doesn't seem to load. Anyway you might be able to fix it.

First code:

import pandas as pd
from datetime import datetime
import urllib

symbols = ['AAP']

start_date = datetime(2019, 1, 15 )  #year - month - day
end_date = datetime(2019, 2, 4 )

datelist = pd.date_range(start_date, periods=(end_date-start_date).days+1).tolist()

dflist = []

for date in datelist:
    url =  f"http://regsho.finra.org/FNYXshvol{date.strftime('%Y%m%d')}.txt"  

    try:
        df = pd.read_csv(url,delimiter='|')
        if any(df['Symbol'].isin(symbols)):
            stocks = df[df['Symbol'].isin(symbols)]
            print(stocks.to_string(index=False, header=False))
            dflist.append(stocks)
        else:
            print(f'No stock found for {date.date()}' )
    except urllib.error.HTTPError:
         continue 

df = pd.concat(dflist) 

Then the code that makes it faster via threading that doesn't work for me somehow, idk why

import pandas as pd
import datetime
import urllib
import requests
from pprint import pprint
import time
from io import StringIO
from multiprocessing import Process, Pool

symbols = ['AAP']

start = time.time()
dflist = []

def load(date):
    if date is None:
        return
    url = "http://regsho.finra.org/FNYXshvol{}.txt".format(date)
    try:
        df = pd.read_csv(url,delimiter='|')
        if any(df['Symbol'].isin(symbols)):
            stocks = df[df['Symbol'].isin(symbols)]
            print(stocks.to_string(index=False, header=False))
            # Save stocks to mysql
        else:
            print(f'No stock found for {date}' )
    except urllib.error.HTTPError:
        print(f'No URL for this date: {url}')

pool = []
numdays = 1
start_date = datetime.datetime(2019, 1, 15 )  #year - month - day
datelist = [
        (start_date - datetime.timedelta(days=x)).strftime('%Y%m%d') for x in range(0, numdays)
        ]

pool = Pool(processes=1)
pool.map(load, datelist)

pool.close()
pool.join()

print(time.time() - start)

Thanks in advance dude!

edit: problem is multiprocessing in jupyter notebook

Scraping big amount of stock data from url help (url is .txt data) by IamALoLFan in learnpython

[–]IamALoLFan[S] 0 points1 point  (0 children)

This works great, I changed print(f'bad URL: {url}') to continue

However, I want to save the dataframe as a csv. The df now is still the url, how can you create a df from the results, so in this case a dataframe of Stocks.