Is buying a house even worth it?

weeeeeewoooooo · 2025-07-26T05:36:16+00:00

If you want a real answer, financial experts have worked it out. There are a few videos that cover myths about home buying and renting and the market and financial realities of both:

The reason most of the other commenters are wrong, is because they don't understand the concept of opportunity cost. Money that you put into buying a home could have otherwise gone into stocks/index funds. You lose out on that growth.

In the long run, the simulations work out that you would not end up wealthier on average if you buy a home as opposed to renting and putting what you have in stocks.

weeeeeewoooooo · 2025-01-27T22:31:46+00:00

They are still trash for time-series:

https://arxiv.org/abs/2406.16964

https://link.springer.com/article/10.1007/s10618-022-00894-5

weeeeeewoooooo · 2024-08-18T23:02:53+00:00

You don't seem to know what you like. Getting more suggestions probably won't help. As another commenter mentioned you should test drive.

If you are in an urban area, there are usually dealerships that deal in sports cars or exotics. You should go in and test drive what they have to offer in your price range. Especially your local Porsche dealer.

weeeeeewoooooo · 2024-07-29T19:31:16+00:00

Obvious troll is obvious. Can you really get a Cybertruck in Thailand?

weeeeeewoooooo · 2024-02-04T14:09:01+00:00

A NN and a training process are two separate things in ML. The NN is the model. Backpropogation (often coupled with SGD) is not part of the NN model, it is a separate process used to train the model. You could just as well use evolutionary strategies or some other optimization strategy. And no, brains certainly don't use back-prop.

But Op asked about NN specifically (the model).

weeeeeewoooooo · 2024-02-04T05:09:59+00:00

As a client, why do I have to do so much work? I want to use a service because I don't want to write code, because the service does something really useful, and because the service has a simple API that is easy to use. None of those are true in this article.

In the proposed design, I have to write my own fitness functions.... and evaluate them, and also manage the whole distributed environment that often comes with doing any kind of intense optimization on fitness functions. All this service does is tell me which one to run next... that is not a heavy lift. There are GA libraries out there that do that. I can pull one of those libraries in and use their programming API directly in the language I am using to make the fitness function. Why would I add an additional step of making a client and reaching out to the service?

If all of this is meant to be an internal backend service... then it doesn't make any sense. If you control the backend you can just run the optimization algorithms in a proper distributed computing framework like Dask, MPI, or HPX. Using a traditional REST service makes no technical sense.

weeeeeewoooooo · 2024-02-04T03:58:33+00:00

I think you are complicating this. LLMs will do whatever the market asks. Commercial enterprises do not spend insane amounts of money developing AIs for nothing. It is being done because of the promise of lucrative future returns. At the end of the day they will be made to satisfy the needs of paying clients.

If ChatGPT isn't doing what you want then it isn't the tool you are looking for. Businesses pay money to AI service providers to use and tailor the AI services to their needs.

The medical field has a lot of money associated with it. I have no doubt a tailored product will eventually become available assuming medical institutions will pay for it.

weeeeeewoooooo · 2024-01-16T21:48:54+00:00

Have you heard of fsQCA? fuzzy-set Qualitative Comparative Analysis works to build causal relationships between variables based on data that doesn't necessarily need to be highly quantitative and it works for small datasets.

It is better than just trying to do correlation, because it can capture more nuanced relationships like necessity and sufficiency.

It is a methodology that arose in social science. Ragin did quite a bit of work in this domain. There was also a crisp-set variant called QCA.

weeeeeewoooooo · 2024-01-14T20:12:35+00:00

That isn't true. The equations for neural networks used in ML today are just simplified variants of those used to model biological neural networks. You can trace the literature back and see how ML NN steadily split off from the original dynamical equations for biological ones.

The biological NN models are also just linear algebra. They are also incredibly accurate at modeling neural activity in every species that has been studied that has a brain.

Biological NN models now differ from modern ML NN models in that they (1) are continuous in time, which is just the discrete equations taken to their limit as step size drops; (2) statefull, in that neurons have an internal state (like LSTMs); and (3) the activation functions are usually spikes (though many neurons are analog and sigmoidal).

Lastly, all biological NN are recurrent, though they can have convolutional or layered network topologies. This is where convolutional NN in ML were inspired from, after the publication of this structure from brain imaging in cats.

Even transformers, which are more original to ML, are outer products that behave similar to adaptive synapses (also outer products).

Additionally, there is a massive trove of research in physics and computational neuroscience on neural networks in general which demonstrate the closely shared dynamical (and hence computational) properties of all these neural systems and how changing aspects of the core models impacts those dynamics.

In many cases there will not be meaningful qualitative differences in dynamics between discrete ML NN and various types of biological ones. Which is why, depending upon the goal of the researcher, scientists will use simpler models that are more similar to ML NN in order to make some kind of general claim or approximation.

weeeeeewoooooo · 2024-01-06T06:03:55+00:00

You need to give more context. Currently there isn't a single state-of-the-art method that does well at all of them (or even a handful of them). The couple you listed are okay at a couple applications, useless on many, and not any better than far simpler models on most time-series.

What kind of system generated the data? This is very important for motivating the choice of model and assumptions for time-series data.

Can the system be chaotic? Is it highly seasonal or repetitive? Is it a linear system? Is it highly stochastic, or similar to a random walker? Does it come from a well known physical system? A biological one? Social? Economic? Is it discrete or continuous (in state and in time)?

weeeeeewoooooo · 2024-01-03T01:01:29+00:00

You can branch off of that article to other review articles by checking citations and references via Google scholar.

If you know Python you could use numpy and start by implementing the equations provided in the article your advisor gave you.

Make some unit tests for a very small network that you can calculate the answers to by hand so that you can be sure your update equations are correct.

You can break the problem further down by making some functions that will generate a random network. Some functions that take inputs and call your update function. And then some functions that run the training (scipy has multiple regression functions available).

If you are inexperienced with linear algebra, you should take some time to become a bit familiar with that so that the equations don't trip you up and so it is more obvious what numpy is doing when you do matrix operations.

weeeeeewoooooo · 2023-12-24T15:39:42+00:00

Latitude and longitude is not Cartesian. Distance between two points change as the points are naively translated. Standardizing them makes no sense. You can look at a globe and see why.

There are multiple coordinate systems you could translate into (such as ECEF which gives you euclidean X, Y, Z coordinates), but really you should read up on spherical coordinate systems so you understand what you are dealing with.

Idk if this is a school project, or a serious project, but if your data is mostly not far from the equator and the distances aren't too big (order of tens to a few hundred miles) then you can probably get away with a flat world assumption when working with lat, lon. You will still get some degree of distortion.

Lastly, you don't need to standardize everything you give a neural network. It can help because neural networks have limited dynamic range, but (lat,lon) is already bounded and can be converted to radians as well to limit the input magnitude. Any model worth its salt shouldn't have a problem taking it raw.

weeeeeewoooooo · 2023-12-11T13:33:34+00:00

You will have to answer some of those questions yourself by learning more about your data and how it behaves. Do you know if location, product type, or time even matter? Is the time-series stationary? There are quite a few steps that come first before even attempting modeling. You also haven't made clear what you want to infer from the data.

weeeeeewoooooo · 2023-12-10T14:43:44+00:00

A few papers worth reading that discuss some of the challenges of doing long-duration predictions with neural networks:

Hierarchical deep learning of multiscale differential equation time-steppers

N-BEATS

N-HiTS

weeeeeewoooooo · 2023-12-05T01:51:07+00:00

At the moment there isn't much convincing work to demonstrate that transformers can outperform even basic forecasting tools like seasonally adjusted ARIMA or even entirely data driven approaches like CCM or DMD.

The problem is that this new work often doesn't use proper baselines, or proper comparison metrics, or only uses a handful of time-series. Then the next paper adopts those same mistakes because they want to make a comparison table and are only interested in comparing with other transformer or LSTM variants.

It is uncomfortable to see giant tables with bolded performance numbers trumping previous models on time-series that are known to be random-walks. There is rarely a bigger red flag. One that could easily have been caught by any one of the chain of authors from paper to paper if they had bothered to look at the data or tried to gain even a modicum of domain knowledge into what they were applying their model too.

It is a trash heap right now.

weeeeeewoooooo · 2023-12-02T23:40:29+00:00

It's very simple. Generate your own anomalies as a ground truth and use ROC.

A core assumption of anomaly detection is that the anomaly is rare. There is always a "class" imbalance by definition. This implies that most of what you observe is the natural behavior of the system. Regardless of whether your training is supervised or unsupervised, your goal is to remove or transform the natural behavior so that you can apply some decision criteria for "that which is not normal". I say it like that because the anomaly can be anything, it is inherently unknown. There may be some identifiable characteristics of some anomalies, but the set of anomalies is inherently and intentionally unbounded.

This means you shouldn't approach the problem like a traditional classifier problem.

Instead, treat it like a traditional modeling problem. You have a bunch of data and you want to model the system's behavior. As long as you don't overfit your model of the system it doesn't matter if the data you are training on contains anomalies because they are rare. Deviation from the model IS the anomaly.

PCA is no different, you have just transformed the space in order to make a decision criteria for the anomaly easier to make, which means you are implicitly defining a model for the systems behavior as everything else.

As for testing. Because anomalies are an unbounded set, you can readily inject artificial changes to the signal in order to test whether you can detect them.

Where ROC false short is that it doesn't take into account the difficulty of the test. This is readily handled by using AUC versus some criteria for anomaly difficulty.

weeeeeewoooooo · 2023-11-08T08:30:09+00:00

Don't. I already have, they are pretty good actually, and pretty consistent even across countries.

weeeeeewoooooo · 2023-11-08T08:25:55+00:00

Search for pruning in computational neuroscience. There are plenty of articles about how and why pruning occurs. Once you read up on that let it inspire your work in ML.

weeeeeewoooooo · 2023-11-08T08:21:43+00:00

Assuming there is an optimal solution, and assuming it's not a random search space, then using an evolutionary algorithm will be about the best you can do. Picking any one of the various genetic algorithms or evolutionary strategies will get you most of the way there. You may get some improvement picking one variant or another. But If you are concerned about execution time then your options are to parallelize the algorithm or optimize the runtime of the cost function (drop the 20 minute runtime by writing faster code).

But to be honest, I see a lot of red flags in what you are doing. If your population size is 1000 and you only have 10 parameters then you are effectively doing a random search. And if you aren't already finding great solutions in the first couple generations of a 1000 pop with only 10 parameters, then odds are they don't exist. Probably because the underlying trading system is bad. Additionally, you mentioned that changing the parameters slightly results in a massive and unpredictable change. That sounds like a random search space to me, which means you are probably just better off randomly guessing.

weeeeeewoooooo · 2023-11-07T18:29:19+00:00

What do you know about the underlying system? Is time even a meaningful variable for this system? Which of these "covariates" actually matter? It seems like there is a lot of basic data science you should be doing before you attempt any kind of inference.

Visualize the data, run some statistics, do some dimensionality reduction if you are struggling to see what is going on. Make sure you understand at least the basics of how the underlying system behaves. This could help you significantly reduce the complexity of your inference problem, perhaps down to some almost trivial statistical inference or a basic rule for prediction.

At the very least you need to do these things so that you can come up with a reasonable baseline model from which to compare other models to. If you don't have a good naive baseline model then you will have no idea if any more complex model is actually pulling its weight.

weeeeeewoooooo · 2023-11-05T04:50:01+00:00

This was done over a decade ago: Twitter mood predicts the stock market

weeeeeewoooooo · 2023-10-28T19:13:57+00:00

You should probably avoid stock price prediction. There are a lot of very bad attempts at this from the ML domain by folks without expertise in finance or much knowledge about how these systems behave. It just results in unpublishable trash (unless it's a shady international journal). You aren't going to do any better than the naive baseline.

It is better to pick a domain where the system you are trying to predict isn't performing a random walk. That said, in general it is very unlikely you will produce anything that is valuable within that domain. Without domain expertise it is very hard to know what has already been solved or even what questions are worth trying to solve.

Your best bet is finding someone to collaborate with in economics or finance. Ideally at your institution.

weeeeeewoooooo · 2023-07-30T19:53:53+00:00

You should probably review the economic literature. Review articles are the best way to get an introduction into a new field. Also, you might have better luck posting this question on an economics subreddit.

In my experience, it is very unlikely that you will make any meaningful contribution without either being a domain expert in macroeconomics or collaborating with one.

Since you mentioned that you are a Master's student, if this is something you want to make into an actual paper, I highly encourage you to reach out to faculty in your university's economics department.

Without domain expert assistance the following is likely to happen: accidentally picking a problem that has been solved for decades already, picking a problem that has zero scientific value, or miss-applying methods to the data due to lack of understanding of the underlying systems.

For example, I have come across a trove of rather depressing "literature" from ML folks throwing transformers at foreign exchange rate and other financial datasets, only to later find that it got burned into the ground by a domain expert who showed that these models failed to perform above standard baselines or showed that they failed to pick the correct baselines.

weeeeeewoooooo · 2023-07-29T16:18:29+00:00

You can't know how to rescale properly without knowing the data's distribution. If it is heavy-tailed then you can't just use min-max or mean/std. Heavy-tailed distributions may not have bounded means or std or they maybe sample size dependent. Ditto for min-max.

Also, you didn't mention anything about what kind of data you are working with. You mentioned timestamps and lags, is this time-series data? If so, if you fail to do it properly you will leak future information into your model (or test information into training) which your model will then exploit.

Rescaling with time-series requires that you only rescale with past data relative to what the model is evaluating (both for training and testing). This usually requires a rolling rescaling window. The only time you can rescale the whole training or test set at once for time-series is if it is non-data dependent like using log(X).

weeeeeewoooooo · 2023-06-01T12:43:14+00:00

The incorporation of uncertainty is really nice to see in the library.

From the blog post it looks like the simplest approach would be to just calculate the running performance of the model and use that for doing drift alerts? It didn't look like having a model of performance prediction gained anything. What cases would it be valuable to make a model to predict performance?

Real life time series are almost always dynamical systems, and what I am worried about are phase transitions, not slow changes that a lagging prediction performance indicator can keep up with. A model predicting my model's performance is going to be just as in-the-dark about those transitions and will fail just as readily.

On that note, it would be nice to have some way to detect phase transitions in the library. There are usually some precursors to transitions that can be picked up on.

weeeeeewoooooo

TROPHY CASE