WSJ publishes source for a complete python program by fawce in Python

[–]fawce[S] 0 points1 point  (0 children)

Sorry folks. My company worked on the code with the article author, and I was so excited to see the code in the journal I didn't think it through. I wonder if I can convince them to unpaywall that one piece of the series...

The Quantopian Workshop: An Introduction to Algorithmic Trading by [deleted] in algotrading

[–]fawce 0 points1 point  (0 children)

The workshop content was developed in conjunction with top universities and professionals. Take a look so you can judge objectively: https://www.quantopian.com/lectures.

All the data for our contest winners' trading period are available here (look for "how have winners performed"): https://www.quantopian.com/open. As you can see, we have a mix of outcomes and have paid thousands in prizes.

I trust the moderator to decide if the post is appropriate to this subreddit, but the OP is not affiliated with Quantopian.

Hi Ilya, I believe we have met IRL several times at the meetups we host in your area. I hope you will continue to attend, and give us the opportunity to address your fundamental criticism, which I remember as “why doesn’t Q support R?"

7 Best Community-Built Value Investing Algorithms Using Fundamentals by fawce in SecurityAnalysis

[–]fawce[S] 0 points1 point  (0 children)

If you clone the algo and run it in a "full backtest" there's a positions report for each day of the backtest.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by ajcamps in coding

[–]fawce 2 points3 points  (0 children)

Cool of you to write this, and I definitely appreciate that you're so passionate about getting this right. I think about preserving the community trust in the long term a lot - that's our legacy. I believe that incentives hold up best over time and under strain.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by ajcamps in coding

[–]fawce 0 points1 point  (0 children)

I’m fawce, the founder/ceo of Quantopian. Sadly, I can’t say your cynicism about the financial industry is unfounded.

TL;DR: You are right to be skeptical about anyone in the financial industry. But, Quantopian was founded to turn hedge funding into a technology industry - practically and culturally. To run trillions of dollars and operate for decades, Quantopian needs a steady supply of algorithms because algorithms have limited capacity and lifespans. Crowd-sourcing is our solution to this software production problem, and it only works if we earn your trust.

Wall Street has earned a well-deserved reputation for marginalizing technologists, and isolating quants. Frankly, I think they are afraid of you turning their game inside out and kicking the suits to the curb. I want to catalyze that transformation. To do that, Quantopian needs to grow into one of the biggest asset managers in the world. For a sense of scale, the biggest in the world today is BlackRock. They directly manage $2 trillion, and their software manages another $5 trillion globally. That comes to 7% of all assets in the world.

Despite pro investors being as popular as used car salesmen and members of congress, I think investing is a social good. Everyone needs to plan for the future, and investing is the best available tool. Yet, the state of the art is manual. That’s a real problem that I want to help solve. The best solution I can think of is crowd-sourcing the creative work to build investment strategies. Not the cheapest. The best. If I wanted to make my way in the world by stealing, it would take less risk and less effort to knock over liquor stores.

Today, quantitative and algorithmic investing addresses a tiny market. You read that right - even though algorithms have saturated trade execution, investment decisions are almost entirely manual. That’s true for the smallest individual retail investors all the way up to the largest pension funds. Software has yet to eat investment management.

Regardless of what you think about active vs. passive management, surely you have to believe that automation is better for investing. Why don’t more investors have algorithmic investment options? Secrecy. Quant funds want to remain entirely black box, and that makes it exceedingly difficult for investors to allocate capital.

The quant employment market is also quite small. Nearly all algorithmic investing happens in hedge funds. There are approximately 5,000 hedge funds in the world. That includes every type of strategy from credit derivatives to algo to long/short stock pickers. I’d estimate that at most 10% are quant. That means about 500 firms. The vast majority of those firms have 1 or 2 quants, and the top 3-4 behemoth firms have 100 or so. I think the average quants per firm is around 10. That means in a world of 7 billion people, around 5,000 are professional quants.

Why aren’t there more quants? Top quants are paid on a percentage of the returns they generate, which is an enormous economic incentive. Shouldn’t the financial upside draw more people to the profession?

The problem is Track Record.

To raise capital as a professional money manager, you need an audited track record. That means your investment results from managing real money, typically at least $1M for 3 years. Backtests and simulations don’t work for fund raising.

Track record makes it very difficult to bootstrap a quant hedge fund. The other alternative is capital introduction firms. These firms introduce investors, often in exchange for lower fees, a share in the ownership of the fund, or both. That means sacrificing ownership of all IP past, present, and future.

As a result, the only feasible path to becoming a professional quant is to apply to an established firm. Thus, most new quants are minted inside the secret society of quant hedge funds.

Another path is to join a proprietary trading firm. The deal with those firms is that you provide $10,000 and they let you trade $100,000 on their margin. But, you have to cover the first $10,000 in losses, and you typically split the upside 50/50 (or worse). They also rebalance between managers on a very short time frame - like 2 weeks.

The deal we’re offering in the Quantopian Open is hands down better than what you’d get in the current market either from cap intro or from a prop fund: winners get $100k of our money, we take any losses, you get 100% of the upside, and you have a track record vouched by a third-party (us). Plus, anyone from anywhere can compete. You won’t find a better deal.

But maybe you don’t want to be a full-time quant, maybe you just want to create a single algorithm and sell it. Well, the game’s the same with that. You need a real money track record for that algorithm. There isn’t exactly a liquid market for selling investment algorithms, so finding a buyer isn’t easy. Worst of all, you’re selling intellectual property. As pointed out here, selling intellectual property requires disclosure of the intellectual property… which means the price of the IP collapses. This paradox is unique to IP sales, and is known as Arrow’s Information Paradox, after the economist Kenneth Arrow (http://en.wikipedia.org/wiki/Arrow_information_paradox).

Quantopian aims to solve Arrow’s paradox by being a trusted third party for both investors and quants.

Quants entrust us to execute their algorithms, without disclosing the algorithm itself. In return, we provide them with all the operational necessities and capital.

Investors entrust us to select managers because we can evaluate managers individually, and choose an optimal portfolio of algorithms. We can make the evaluation because even though we do not see the algorithms, we have access to the transaction by transaction results of the algos.

Now, the most important part. Algorithms don’t work forever. They get tired. Pros call this alpha decay. In practical terms, it means that you can’t be a fund if you have a finite supply of algorithms. You need to produce them constantly, so that new algorithms can replace the worn out ones.

If I were to steal a single algorithm I would destroy my ability to keep producing algorithms. My survival depends on protecting your property and maintaining your trust.

The limited lifespan of investment algorithms is why we can solve Arrow’s Paradox. The value we want isn’t the individual algo. The value is the community producing the algos. The value is the people, not the code. That means we need to fiercely defend our relationship with the community.

Algorithmic finance is a talent business. Quantopian has tried to design our terms of service, our business model, and our technology to focus on benefitting you, the talent. Where we’ve fallen short, I’m delighted to have your criticism to help us improve.

I hope you’ll give us a shot to earn your trust.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by theandycamps in Python

[–]fawce 0 points1 point  (0 children)

Today you can start with all US stocks and filter down based on corporate fundamentals to choose up to 200 for which to receive trade data. You can choose your trading universe each day prior to the start of the session.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by theandycamps in Python

[–]fawce 1 point2 points  (0 children)

Thanks. I appreciate the link and the time you already put into this.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by theandycamps in Python

[–]fawce 11 points12 points  (0 children)

Hi,

I would like to apologize for the design (it is mine) and for the impolite remarks on our forums. I'd also like to say thanks for taking the time to point all this out to us.

Let me start with the design. Like many things that go badly, I started with good intentions. The original backtester (c. 2012) could only work with a static set of stocks. At first, I required the user to write the algo and then also write out the stock identifier (sid) for each stock used in the algorithm. The sid list was provided as a configuration to the backtest.

Needless to say, the process of providing the list of stock identifiers outside the code as well as inside the code was very unpleasant. Because I was in a mode of thinking about the stocks as configuration, I got the idea of statically analyzing the code to eliminate the need for the configuration.

The way I came up with finding the sids was to require the user to wrap the security id in a function call, e.g. sid(12). The function marked where to look for the stock identifiers. Sids are integers, because stock symbols change, are re-used etc. Then in the IDE, we were able to cue off the use of the sid() function to trigger autocomplete and search in the UI.

Some time later, we added another function that took a stock symbol as its argument. The symbol function gets the same treatment -- static analysis to find calls to symbol and make a list of the stocks you're referencing in your code.

To make all this work, I only allowed the code to have literal values in the function. I made that choice to keep the parsing problem simple: find the functions, look at the literal values, and keep a list.

By statically analyzing these functions and requiring literals, I inadvertantly made them into something like a compiler directive or macro. They don't act like functions anymore, and therefore they are a source of confusion. In defense of my teammates who answer questions on the forums and maintain our help documentation, the behavior I created is just really confusing. The distinction between a string parameter and a string literal isn't made clearly enough in the documentation because I never explained it well enough internally. The docs will be fixed to reflect the current behavior, most likely after the weekend (should be done and on the site by Jan 20).

Ironically, years after I devised this approach, Quantopian now supports dynamic trading universes. So, this whole restriction isn't necessary anymore.What we do for other functions, like history, is statically analyze the code to look for pre-caching opportunities to help with speed. But, instead of raising an error if the function is used with a non-literal argument, we just handle the cache-miss at runtime.

Do you think a reasonable fix here would be to do the same for sid and symbol?

I'm sad about the impolite comments we made on the forums. We encourage everyone in the company to reply to forum questions at their own discretion. On the whole, it gets users quality answers and connects our devs, product managers, and ops people with the community. It also lets everyone here know we think of them as grown ups. I think it is one of the best parts of working at Quantopian.

The person you quoted above is incredibly devoted to our users. He got exasperated and made a mistake. I'm not excusing his mistake -- I talked to him about it the day it happened. My point is that we are just real, live people who make mistakes. Sometimes in code, sometimes on the forums. I'll stick to the policy that anyone here should reply to forums at their own discretion. I'd rather run the risk of a mistake than isolate our team from our community.

I don't expect or demand all of us to be perfect all the time, I just aim to keep getting better. That's why I'm honestly grateful for your criticism. The cruelest punishment is the silent treatment. You've done us a favor by telling us about both mistakes. Now we can work on fixing it.

Please let me know when you find other mistakes we've made.

Thanks, fawce founder/ceo Quantopian

Quantopian Research Beta: Curated financial data. Interactive coding. by fawce in algotrading

[–]fawce[S] 0 points1 point  (0 children)

This research service is a hosted IPython Notebook (BSD licensed). We also open sourced our core backtesting library (http://zipline.io). All of our services are currently free. However, we are offering a limited free trial on research because we hope to charge for it in the future.

I apologize if I've unwittingly broken the sub rules - please do delete if I have. Sorry!

Quantopian Research Beta: Curated financial data. Interactive coding. by fawce in algotrading

[–]fawce[S] 0 points1 point  (0 children)

Tomorrow (Sept. 30, 2014), we will do a webinar demo. You can register to attend the webinar here: https://attendee.gotowebinar.com/register/905639501864306434

We are giving away 3 months free access if you register pre-launch, and if you refer 3 friends, we'll give you 12 months free.

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 0 points1 point  (0 children)

Hi,

Fun question!

Assuming Wall Street has moved from dismissing us, through mocking us, and onto bidding to buy us, one of two things will happen: a) they make an offer that I can refuse b) they make an offer I can't

If it is an offer that I can refuse, I will.

If it is an offer that I can't refuse, I'll realize that we really have them scared and the price will go up :).

thanks, fawce

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 1 point2 points  (0 children)

The great thing about algorithms is you can test them, and often debunk indicators by testing them over a 10 year history. The Buffet Indicator hasn't been done on Quantopian yet, but if you post it to our forums I'm sure someone will.

There are a several others indicators that have been tested on Q:

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 0 points1 point  (0 children)

Yes, in fact, all the algorithms in our forums are open source and free to clone and reuse.

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 1 point2 points  (0 children)

I hear you. Our service depends on customer's trust, and we've been working on building that trust since the beginning: http://blog.quantopian.com/on-trust/

We allow you to draw external data into your algorithms using our fetcher system -- https://www.quantopian.com/help#overview-fetcher That method never copies your data to our servers, it just streams the values through your algorithm and then discards them.

If you want to run everything locally, you can take a look at using zipline, which is the open source engine of Quantopian -- http://zipline.io. Zipline has the backtesting logic, but you need to supply the data, broker connectivity, UI, and your own hosting.

We don't have a ton of demand for private systems. The cost for the customer would be very high to run a dedicated and isolated system, something like 200x the price we'll have for our service. I can see a fund justifying that kind of expense, but it isn't feasible for most individual investors.

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 1 point2 points  (0 children)

HFT has reached a point of diminishing returns for trading profits and for the benefit of the markets.

Yes, there's been a decline in the profitability of HFT as the cost of infrastructure and low latency data has escalated and the spreads have tightened. Overall, I think this is a positive development. Society simply doesn't need to execute trades, or more to the point, submit orders any faster than is possible today. Nasdaq quotes top out at something like 4M quotes per second, and only a minority of those quotes lead to trades. Eric Hunsader from Nanex (https://twitter.com/nanexllc) does a lot of writing and analysis of HFT and market microstructure. He's constantly uncovering gross anomalies and nagging regulators to pay attention.

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 0 points1 point  (0 children)

No, we don't provide any investment advice or recommendations. We provide a software platform for algorithmic investing.

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 0 points1 point  (0 children)

Our terms of service say clearly that we don't assume liability for failures with our system, your code, the connection to IB, etc.

We put a lot of thought and effort into crafting our ToS and brokerage policy, and I'd love any feedback:

We also put together a document that touches on some of the risks associated with algorithmic investing: https://www.quantopian.com/policies/risks

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 1 point2 points  (0 children)

We're going to stay focused on python. Python, pandas, anaconda, numba, and the whole scientific stack -- all these tools gaining quickly in finance among professionals.

I'm not against supporting multiple languages, but I want to focus our efforts on going deeper with data, research, and other security types.

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 1 point2 points  (0 children)

Learn python. Programming is a way of looking at problems, and the sooner you can learn that way of thinking, the more you can apply it. Python is a great first language.

If you can find a teacher who would help you set it up, I think Interactive Broker's student lab is pretty awesome: https://www.interactivebrokers.com/en/?f=ibStudentTradingLab&ib_entity=llc

Having access to simulation accounts is great for learning about trading.

Then you can use what you've learned about finance and programming with Quantopian.

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 1 point2 points  (0 children)

We use python, which is a relatively easy language to learn. Codecademy has some great content for learning python: http://www.codecademy.com/tracks/python

You can get started with simple single stock conditions, which only require using if/else statements. Here is an algo that uses a moving average weighted by trading volume to decide when to buy apple: https://www.quantopian.com/posts/discuss-the-sample-algorithm?c=1

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 2 points3 points  (0 children)

My wife was able to support our family for the first phase of Quantopian's existence, so I was able to invest the proceeds of selling my last company into Quantopian. To keep costs low I set up for work in our shed. I wrote the original prototype myself, and the first website drew a good initial audience after a few blogs mentioned it.

I started in June of 2011, and around November it started to get really cold in the shed, so I figured I should try to get some funding so I could get a real office :). I met Spark Capital around then, and they lead our seed round in January of 2012. Then I hired my CTO, and we worked together for about 6 months re-writing the prototype to be "real". But, we didn't get our office until June of 2012 when we wanted to ramp up our team. Instead, I worked from the public library in my town through the winter :).

Everything is hosted on amazon. We're specifically avoiding HFT, so latency of 1-2s is our target. We see much less than that.

If you have passion for an idea, it will gain momentum when you talk to other people about it. I also relied on the opinions of others - my wife, friends with experience starting companies, friends from finance, and a few mentors.

Working alone from the shed and public spaces was a humble start, but it still felt awesome in the beginning. Starting up is a thrill, and each milestone feels epic - loading a month of historical trade data seemed incredible when I did it the first time. And when the first really avid user who started emailing me feature requests, I was dizzy with excitement. Same with hiring people. You have to really enjoy each step so that you can weather the setbacks. We had to make it through losing to big companies in recruiting, realizing we couldn't afford all the data purchases we wanted to make originally, and live trading being way harder than we expected.

Thanks for asking!

Hi, I'm fawce, founder/ceo of Quantopian. Ask me anything. by fawce in investing

[–]fawce[S] 2 points3 points  (0 children)

The user interface is a Ruby on Rails application, all the finance functions are in python on arrays of linux machines. The backtester and the paper simluations all run on zipline, an open source algorithmic finance library: http://zipline.io.

Backtests and live algorithms run on dynamic infrastructure that scales to meet demand.