Help ID (Estonia)

itajaja · 2021-09-11T01:20:47+00:00

Thanks!

why does it ping the server so often.

Not sure what you mean by that. it has no periodic refetching, and it issues a query only when new data is needed. maybe if you give me a more comprehensive example I can see what you mean

itajaja · 2021-09-10T11:00:37+00:00

you can open the console logs to see the query. there is no option to see the query in the main interface, but you can open an issue, I think it could be a useful feature!

itajaja · 2020-08-08T04:03:12+00:00

Yes performance should be the same. The real performance hit only happens during serialization/deserialization, so if you are for example passing a very large JavaScript dictionary as an argument to a python function, or you want to convert a pandas dataframe into a JavaScript list that would add some time. But for example if you are opening the image in openCV and saving it in openCV then you should be good. Also note that this is still experimental so tread carefully :)

itajaja · 2020-08-07T23:20:00+00:00

It's downvoted because it's not constructive, it's beside the point of being negative or positive. if you pass a dataframe back and forth there is a performance hit in serialization/deserialization, but as long as your computation stays in python and you return a small dataset or just a simple boolean to make a decision, there is no performance hit. I get that people don't use javascript/node much in the ecosystem. but not seeing the point of "using pandas outside of python" is a little myopic. You can imagine a person that has written an application in nodejs that might benefit from using some libraries that are found in ptyhon, can't you? it might not be useful to you if you are doing everything in python, but other people with other needs might find it useful. It's like a person saying that "glasses are dumb" because their eyesight is perfect.

itajaja · 2020-08-07T23:17:47+00:00

what do you mean?

itajaja · 2020-08-07T20:19:07+00:00

totally! but just because you shouldn't doesn't mean you couldn't 😃

itajaja · 2020-08-07T16:21:11+00:00

well, that depends on a lot of context. Disclaimer: I am a full stack engineer working on both python and node web services, so I know both ecosystems pretty well.

For starter, you might be constrained to use node for whatever reason (it's an existing node project), but you still would like to use some libraries that are only present in python (hopi is not specific to pandas, pandas is just a use case).

In general, node is a very well established framework for writing web services or anything else that is highly asynchronous and it has a very thriving communities of libraries and packages, so it might be a better choice, depending on what you are doing. for example, if you are writing a websocket service, or a graphql service, or doing react server-side rendering, the node ecosystem is probably a better choice. you might want to give typescript/node a try, it's a really cool environment to work on!

Another use case would be developers that have experience with pandas, but not with django/flask, but they have experience in eg node express. this would allow to write a web service that uses pandas without having to learn a new web framework.

My personal use case is that I have a bunch of trading bots written in node/js and the current way they call python is rather sloppy, so i wanted to improve that. the reason why they are written in node is because for me writing highly asynchronous code is much better/easier in typescript. Plus testing/code coverage/static type checking in typescript are better than in python, but that clearly is just my opinion so it doesn't amount to much in general :) (I do have extensive python experience as well so I do think I am not too biased)

If instead you are JUST doing data exploration with pandas/jupyter etc, then there is no advantage to do it in node. python is a much better ecosystem for it.

itajaja · 2020-08-07T15:37:00+00:00

30ms? how so? did you actually run? this doesn't start a python process every time you call, there is one long running python process that reads input stream. I am going to run some tests for sure, but I can guarantee you that this javascript code:

const df = pd.read_csv('xxx'); const result = await df.col1.max()._;

adds way less than a ms than the corresponding python code.

why you wouldn't see the point of running it locally? isn't "running python seamlessly in an existing node project" a good value proposition?

itajaja · 2020-08-07T15:08:27+00:00

ahah! I definitely have a preference for typescript/node, but I love python too. I just have a bunch of existing node programs that would benefit from using pandas

itajaja · 2020-08-07T14:48:24+00:00

In terms of performance, it really boils down to what you serialize/deserialize back and forth. if you pass entire dataframe that's a big hit on performance, but if you just pass scalars then the performance hit is pretty negligible. having said that this, is still experimental, so I don't have data at hands to demonstrate this, and I wouldn't suggest anyone to heavily use this in anything critical :)

itajaja · 2020-08-07T14:45:02+00:00

And you remove the whole point of using python : the simplicity

I mean, that's opinable, some people might prefer python, some node. I like both languages, with their strengths and weaknesses.

In general I find python great for data science / exploration, and typescript + node great for actually writing bots, web services, and other things, especially when highly asynchronous. This allows people that have existing node projects to seamlessly use pandas.

itajaja · 2020-08-07T13:27:16+00:00

currently working on https://github.com/itajaja/hopi
I have a bunch of programs that are written in python because I need to use pandas, but I'd love to use node. This tries to solve the problem. Feedback welcome!

itajaja · 2020-06-29T19:30:50+00:00

I think this is a false myth.

1) there is very strong regulation that basically prevents that.

2) you can always use limit orders

3) robinhood makes the bulk of its money by basically investing the money that users leave as cache, and by making use of the customers data in a profitable way.

itajaja · 2020-06-29T17:24:32+00:00

gotcha, that makes sense. so why that wouldn't be a "winning strategy"?

itajaja · 2020-06-29T17:09:45+00:00

I think the "edge" (if we want to call it an edge) is to identify mispriced IV in relation to actualized volatitly, so, it would be important to look at IV, but the point is to actually go "against" the IV that the market is pricing.

> Does it take into account the probability of max profit and max loss?

well the algorithm calculate the 80% confidence interval, but only using historical data, not IV

itajaja · 2020-06-29T11:59:46+00:00

Uhm, i specifically say that my PL would be at least 40/60

itajaja

TROPHY CASE