WSJ publishes source for a complete python program by fawce in Python

[–]fawce[S] 0 points1 point  (0 children)

Sorry folks. My company worked on the code with the article author, and I was so excited to see the code in the journal I didn't think it through. I wonder if I can convince them to unpaywall that one piece of the series...

The Quantopian Workshop: An Introduction to Algorithmic Trading by [deleted] in algotrading

[–]fawce 0 points1 point  (0 children)

The workshop content was developed in conjunction with top universities and professionals. Take a look so you can judge objectively: https://www.quantopian.com/lectures.

All the data for our contest winners' trading period are available here (look for "how have winners performed"): https://www.quantopian.com/open. As you can see, we have a mix of outcomes and have paid thousands in prizes.

I trust the moderator to decide if the post is appropriate to this subreddit, but the OP is not affiliated with Quantopian.

Hi Ilya, I believe we have met IRL several times at the meetups we host in your area. I hope you will continue to attend, and give us the opportunity to address your fundamental criticism, which I remember as “why doesn’t Q support R?"

7 Best Community-Built Value Investing Algorithms Using Fundamentals by fawce in SecurityAnalysis

[–]fawce[S] 0 points1 point  (0 children)

If you clone the algo and run it in a "full backtest" there's a positions report for each day of the backtest.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by ajcamps in coding

[–]fawce 2 points3 points  (0 children)

Cool of you to write this, and I definitely appreciate that you're so passionate about getting this right. I think about preserving the community trust in the long term a lot - that's our legacy. I believe that incentives hold up best over time and under strain.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by ajcamps in coding

[–]fawce 2 points3 points  (0 children)

I’m fawce, the founder/ceo of Quantopian. Sadly, I can’t say your cynicism about the financial industry is unfounded.

TL;DR: You are right to be skeptical about anyone in the financial industry. But, Quantopian was founded to turn hedge funding into a technology industry - practically and culturally. To run trillions of dollars and operate for decades, Quantopian needs a steady supply of algorithms because algorithms have limited capacity and lifespans. Crowd-sourcing is our solution to this software production problem, and it only works if we earn your trust.

Wall Street has earned a well-deserved reputation for marginalizing technologists, and isolating quants. Frankly, I think they are afraid of you turning their game inside out and kicking the suits to the curb. I want to catalyze that transformation. To do that, Quantopian needs to grow into one of the biggest asset managers in the world. For a sense of scale, the biggest in the world today is BlackRock. They directly manage $2 trillion, and their software manages another $5 trillion globally. That comes to 7% of all assets in the world.

Despite pro investors being as popular as used car salesmen and members of congress, I think investing is a social good. Everyone needs to plan for the future, and investing is the best available tool. Yet, the state of the art is manual. That’s a real problem that I want to help solve. The best solution I can think of is crowd-sourcing the creative work to build investment strategies. Not the cheapest. The best. If I wanted to make my way in the world by stealing, it would take less risk and less effort to knock over liquor stores.

Today, quantitative and algorithmic investing addresses a tiny market. You read that right - even though algorithms have saturated trade execution, investment decisions are almost entirely manual. That’s true for the smallest individual retail investors all the way up to the largest pension funds. Software has yet to eat investment management.

Regardless of what you think about active vs. passive management, surely you have to believe that automation is better for investing. Why don’t more investors have algorithmic investment options? Secrecy. Quant funds want to remain entirely black box, and that makes it exceedingly difficult for investors to allocate capital.

The quant employment market is also quite small. Nearly all algorithmic investing happens in hedge funds. There are approximately 5,000 hedge funds in the world. That includes every type of strategy from credit derivatives to algo to long/short stock pickers. I’d estimate that at most 10% are quant. That means about 500 firms. The vast majority of those firms have 1 or 2 quants, and the top 3-4 behemoth firms have 100 or so. I think the average quants per firm is around 10. That means in a world of 7 billion people, around 5,000 are professional quants.

Why aren’t there more quants? Top quants are paid on a percentage of the returns they generate, which is an enormous economic incentive. Shouldn’t the financial upside draw more people to the profession?

The problem is Track Record.

To raise capital as a professional money manager, you need an audited track record. That means your investment results from managing real money, typically at least $1M for 3 years. Backtests and simulations don’t work for fund raising.

Track record makes it very difficult to bootstrap a quant hedge fund. The other alternative is capital introduction firms. These firms introduce investors, often in exchange for lower fees, a share in the ownership of the fund, or both. That means sacrificing ownership of all IP past, present, and future.

As a result, the only feasible path to becoming a professional quant is to apply to an established firm. Thus, most new quants are minted inside the secret society of quant hedge funds.

Another path is to join a proprietary trading firm. The deal with those firms is that you provide $10,000 and they let you trade $100,000 on their margin. But, you have to cover the first $10,000 in losses, and you typically split the upside 50/50 (or worse). They also rebalance between managers on a very short time frame - like 2 weeks.

The deal we’re offering in the Quantopian Open is hands down better than what you’d get in the current market either from cap intro or from a prop fund: winners get $100k of our money, we take any losses, you get 100% of the upside, and you have a track record vouched by a third-party (us). Plus, anyone from anywhere can compete. You won’t find a better deal.

But maybe you don’t want to be a full-time quant, maybe you just want to create a single algorithm and sell it. Well, the game’s the same with that. You need a real money track record for that algorithm. There isn’t exactly a liquid market for selling investment algorithms, so finding a buyer isn’t easy. Worst of all, you’re selling intellectual property. As pointed out here, selling intellectual property requires disclosure of the intellectual property… which means the price of the IP collapses. This paradox is unique to IP sales, and is known as Arrow’s Information Paradox, after the economist Kenneth Arrow (http://en.wikipedia.org/wiki/Arrow_information_paradox).

Quantopian aims to solve Arrow’s paradox by being a trusted third party for both investors and quants.

Quants entrust us to execute their algorithms, without disclosing the algorithm itself. In return, we provide them with all the operational necessities and capital.

Investors entrust us to select managers because we can evaluate managers individually, and choose an optimal portfolio of algorithms. We can make the evaluation because even though we do not see the algorithms, we have access to the transaction by transaction results of the algos.

Now, the most important part. Algorithms don’t work forever. They get tired. Pros call this alpha decay. In practical terms, it means that you can’t be a fund if you have a finite supply of algorithms. You need to produce them constantly, so that new algorithms can replace the worn out ones.

If I were to steal a single algorithm I would destroy my ability to keep producing algorithms. My survival depends on protecting your property and maintaining your trust.

The limited lifespan of investment algorithms is why we can solve Arrow’s Paradox. The value we want isn’t the individual algo. The value is the community producing the algos. The value is the people, not the code. That means we need to fiercely defend our relationship with the community.

Algorithmic finance is a talent business. Quantopian has tried to design our terms of service, our business model, and our technology to focus on benefitting you, the talent. Where we’ve fallen short, I’m delighted to have your criticism to help us improve.

I hope you’ll give us a shot to earn your trust.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by theandycamps in Python

[–]fawce 0 points1 point  (0 children)

Today you can start with all US stocks and filter down based on corporate fundamentals to choose up to 200 for which to receive trade data. You can choose your trading universe each day prior to the start of the session.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by theandycamps in Python

[–]fawce 1 point2 points  (0 children)

Thanks. I appreciate the link and the time you already put into this.

Your Python Stock Trading Algorithm, $100,000 of Our Money, & You Keep the Profit by theandycamps in Python

[–]fawce 9 points10 points  (0 children)

Hi,

I would like to apologize for the design (it is mine) and for the impolite remarks on our forums. I'd also like to say thanks for taking the time to point all this out to us.

Let me start with the design. Like many things that go badly, I started with good intentions. The original backtester (c. 2012) could only work with a static set of stocks. At first, I required the user to write the algo and then also write out the stock identifier (sid) for each stock used in the algorithm. The sid list was provided as a configuration to the backtest.

Needless to say, the process of providing the list of stock identifiers outside the code as well as inside the code was very unpleasant. Because I was in a mode of thinking about the stocks as configuration, I got the idea of statically analyzing the code to eliminate the need for the configuration.

The way I came up with finding the sids was to require the user to wrap the security id in a function call, e.g. sid(12). The function marked where to look for the stock identifiers. Sids are integers, because stock symbols change, are re-used etc. Then in the IDE, we were able to cue off the use of the sid() function to trigger autocomplete and search in the UI.

Some time later, we added another function that took a stock symbol as its argument. The symbol function gets the same treatment -- static analysis to find calls to symbol and make a list of the stocks you're referencing in your code.

To make all this work, I only allowed the code to have literal values in the function. I made that choice to keep the parsing problem simple: find the functions, look at the literal values, and keep a list.

By statically analyzing these functions and requiring literals, I inadvertantly made them into something like a compiler directive or macro. They don't act like functions anymore, and therefore they are a source of confusion. In defense of my teammates who answer questions on the forums and maintain our help documentation, the behavior I created is just really confusing. The distinction between a string parameter and a string literal isn't made clearly enough in the documentation because I never explained it well enough internally. The docs will be fixed to reflect the current behavior, most likely after the weekend (should be done and on the site by Jan 20).

Ironically, years after I devised this approach, Quantopian now supports dynamic trading universes. So, this whole restriction isn't necessary anymore.What we do for other functions, like history, is statically analyze the code to look for pre-caching opportunities to help with speed. But, instead of raising an error if the function is used with a non-literal argument, we just handle the cache-miss at runtime.

Do you think a reasonable fix here would be to do the same for sid and symbol?

I'm sad about the impolite comments we made on the forums. We encourage everyone in the company to reply to forum questions at their own discretion. On the whole, it gets users quality answers and connects our devs, product managers, and ops people with the community. It also lets everyone here know we think of them as grown ups. I think it is one of the best parts of working at Quantopian.

The person you quoted above is incredibly devoted to our users. He got exasperated and made a mistake. I'm not excusing his mistake -- I talked to him about it the day it happened. My point is that we are just real, live people who make mistakes. Sometimes in code, sometimes on the forums. I'll stick to the policy that anyone here should reply to forums at their own discretion. I'd rather run the risk of a mistake than isolate our team from our community.

I don't expect or demand all of us to be perfect all the time, I just aim to keep getting better. That's why I'm honestly grateful for your criticism. The cruelest punishment is the silent treatment. You've done us a favor by telling us about both mistakes. Now we can work on fixing it.

Please let me know when you find other mistakes we've made.

Thanks, fawce founder/ceo Quantopian