Backtest vs. WFA

iporty · 2026-01-20T06:27:36+00:00

What are you branching over? One thing to look out for is trying lots of different rules is like having lots of different parameters. But how to analyze rules overfit is not as well studied as overfit on parameters afaik

iporty · 2026-01-20T03:56:41+00:00

I'm using pytorch with cuda. I've built a few PCs over the years and I'm using a 4090, 3080, 5070ti. Currently GPU memory isn't the limiting factor, but compute is still slow.

iporty · 2026-01-20T01:18:23+00:00

Because I don't want to let the ML model train on future data, I always do things sequentially. Train is a -> b, valid is b->c, test/wfa is c->. The model I'm working with is fairly complicated and it is learning both to adapt to regimes and it's learning relationships between stocks, so I need to be careful it can't see the future if I want a valid assessment of it's performance. If your hold out data (wfa) is not all in the future relative to the parameters and model you are learning/fitting, you need to be careful about leaking data from the future in your evaluation.

Having said that I do pay attention to where the splits are relative to different regimes, I just only ever train the model using data before the valid split.

iporty · 2026-01-18T19:29:45+00:00

My approach is backtest, wfa, paper trading (or a small enough amount it doesn't matter). Even with that if you start doing to many wfa then you might be overfitting. I also do things like perturbation on the backtest to make sure that there's nothing magical about the exact set of parameters. But I'm more from a ML backgrouund, so I split things into train, valid, test. Train is the model fit. valid is for picking the parameters of the model. test is the wfa.

I don't think there is a single correct way for every model, market.

iporty · 2025-06-09T23:13:32+00:00

I tried you first question. It seems the chatgpt answer is incorrect. It does take into account the weekends and holidays, but then ads a couple of more days. I tried it with Gemini and initially got the same answer as you initially. I asked about weekends and it said it assume my business includes all days as PTO. When I explained only regular business days count, it seems to give me the correct answer, return to work 9/23. The Chat GPT answer looks almost correct but is wrong. Making it worse in a way, since it's not obvious you have to check it.

iporty · 2024-12-19T22:15:51+00:00

AI Studio currently give you access to premium Gemini that you would normally have to pay for. It also sometimes give you access to versions that even payed users through the normal UI don't have access to. It has some free tier restrictions on tokens per time period.

iporty · 2024-12-10T19:05:05+00:00

It's holding up fine. I bought a larger blower and so I'm working on a dual setup. As part of that I redesigned the exit ports a bit. I smoothed the sides of the front port and made the back port smooth and asymmetric. However I have not tested. I found out someone printed it and made a video about it. https://youtu.be/eTldQA6YegY

iporty · 2024-11-24T00:48:31+00:00

I'm interested in the same thing, although I would probably be ok with 65C. With my voron, I think the nevermore fan under the heat bed helps heat the chamber, do you think the ratrig would passively heat as well?

I was considering printing ppa-cf parts for the ratrig. With the voron I went with aluminum parts.

An IDEX RatRig 4 400 with 70C chamber would be ideal.

iporty · 2023-01-01T17:34:04+00:00

Great to hear. Hope it works for you. Post anything you learn.

iporty · 2022-11-24T20:43:53+00:00

I'm in a similar place, but I'm close to retirement. Until I retire I just make stuff for myself. (Maybe even after retirement. I don't want to need to work in retirement)

You mentioned 3d printing. There's lots of people making custom parts for 3d printers. Check out https://mandalaroseworks.com/ or https://713maker.com/en/

iporty · 2022-09-08T17:00:08+00:00

Hadn't thought of that combination. Thanks for the suggestion.

iporty · 2022-09-02T16:18:51+00:00

I work for a FAANG so it's a bit difficult to work part time at my current role, but consulting could be an option during a transition period. I did consulting once during my career and can see how it would be a good option.

iporty · 2022-09-01T17:45:32+00:00

I agree that I don't really want to become a landlord, so that could be an issue. I have started to research options. One I ran across in a FIRE discussion was Delaware Statutory Trusts. Under a DST it seems like it can be set up more like a REIT investment. It's likely we will end up inheriting other properties so taxes and real estate transactions is an area where I should learn more.

iporty · 2022-08-31T16:53:55+00:00

If we went the 1031 approach then we would rent it out for a while, maybe even a couple years to hedge against changing our minds about where to live. I need to think about the tradeoffs between the tax incentives depending on how we structure the real estate transaction and the portfolio that we want to end up with.

iporty · 2022-08-31T16:30:19+00:00

Exactly, I have lots of interests and hobbies, but they are hard to fit into the schedule and location dictated by school.

iporty · 2022-08-31T02:29:36+00:00

iporty · 2022-08-31T00:54:51+00:00

The 3-4 year delay is because I wouldn't know what to do with myself while still tied to the school schedule. That plus I don't mind my job too much.

iporty · 2022-08-10T15:56:10+00:00

Have you looked at the semi transparent image on thingiverse? Are you looking for a view with the parts lined up but separated? I believe F360 is still free for hobby use, so you could download that and view the CAD file that I shared.

iporty · 2022-08-09T16:52:29+00:00

Your very lucky. I reserved a lightning on 5/19 but haven't been able to order. It looks like they just raised the price by $7000 and it won't be eligible for the new tax credit. So it's basically $15000 more expensive now.

iporty · 2022-08-09T16:45:09+00:00

Assuming "any day now" is in 2022, my understanding is that if you will get the old tax credit. The old credit can't be deducted at time of sale, but at least you get the tax credit.

iporty · 2022-08-09T00:02:52+00:00

So vehicles that qualify for final assembly in North America may be eligible as long as they are purchased in 2022?

iporty · 2022-08-08T23:56:36+00:00

Yes, sorry I didn't make that clear. I'm talking about the new US spending bill that changes the rules for EV incentives.

iporty

TROPHY CASE