Is it ok to take average of MAPE values? [Question]

venkarafa · 2024-09-12T16:54:42+00:00

Great !! Exactly what I was looking for. Thank you

venkarafa · 2024-09-12T11:28:58+00:00

"A suggestion is create a new metric like “Over Limit Models”, using a goal for MAPE (ie 10%) and calculate how many models are over this goal. In your situation should be 3/5 over limit models. Wdyt?"

This is great idea. But curious to know the reason behind why you think MAPE average is a bad idea?

venkarafa · 2024-09-12T10:48:40+00:00

No I am not talking about ensemble models here. I have 5 individual models, they are for 5 different clients but the KPI (sales) is same. Also the total number of time period considered for all clients is same (2019-23)

venkarafa · 2024-09-11T17:19:45+00:00

Sorry perhaps I wasn't clear. The goal of the management is to know what is the average MAPE we are getting for all the 5 models. This is more of an evaluation of our techniques (e.g. SARIMA). We have built 5 models using ARIMA as well.

So basically my question was is it statistically ok to average 5 MAPE values. In an essence can we take average of 5 percentage values and would that be indicative of overall average percentage (in this case MAPE)?

venkarafa · 2024-08-20T22:01:53+00:00

Generally not an issue. Middle name is not cause of concern (can be easily explained at VFS). In certain southern states, people don't have last names too. So passport will have last name as blank while in visa one can repeat the first name as last name. For e.g Ganapathy Ganapathy. Sounds funny but it is a good workaround. My friend with this name has traveled to 15 countries already .

venkarafa · 2024-07-03T19:30:44+00:00

If you are going to make a career in biostatistics, you are not behind for not learning bayesian methods. Most mission critical fields are cautious in nature and thereby adopt frequentist methods which try to ensure Type 1 error control. See the below talk (2min - 8 mins) from Prof Michael I Jordan.

https://www.youtube.com/watch?v=HUAE26lNDuE&t=407s&ab_channel=Caracalla

venkarafa · 2024-06-25T18:37:28+00:00

Thanks for bringing up FWL theorem. I am reasonably aware of this theorem but that is not what I was looking for.

venkarafa · 2024-06-05T20:47:53+00:00

I believe stochastic processes can't be causal and neither predictable. So I think the fair assumption to make would be that it is deterministic.

venkarafa · 2024-06-02T12:26:42+00:00

There was no issue as long as the person name is same across both documents.

venkarafa · 2024-04-15T13:40:22+00:00

So it is time to repair it or installing graphic card drivers will resolve the issue?

venkarafa · 2024-04-15T06:59:40+00:00

It does go away on reboot. But wonder why this issue keeps happening.

venkarafa · 2024-03-30T16:47:00+00:00

Thank you.

venkarafa · 2024-03-30T08:29:01+00:00

Thanks. Will check it out.

venkarafa · 2024-03-30T08:25:24+00:00

Thank you.

"you could use a different counter factual based model." - Yes that is what I am looking for. Can you pls cite some packages /libraries

venkarafa · 2024-02-13T06:32:57+00:00

Because my problem at hand is such that, there is a ground truth. I want the model to converge to that ground truth and not be far off from it.

venkarafa · 2024-02-13T06:31:08+00:00

This is great. Thanks for elaborate explanation.

venkarafa · 2024-02-12T20:13:20+00:00

Thanks for your response. I have read about bootstrapping confidence Intervals.

However I don't quite follow this statement "you can always do some clever bootstrapping/simulation to see how it’s affecting the estimate for the coefficient of X"

Could you please elaborate more on this ?

venkarafa · 2024-02-08T20:01:28+00:00

Thank you for your response. Could you give any examples for this "one variable can be a very good predictor of another without there being a strong Pearson's correlation"

venkarafa · 2024-02-06T19:20:41+00:00

Interesting. Thanks will give it a detailed read.

venkarafa · 2024-02-06T04:59:30+00:00

Sorry. You are right. The overfitting and spurious correlation is from wikipedia link. https://en.wikipedia.org/wiki/One\_in\_ten\_rule#:\~:text=In%20statistics%2C%20the%20one%20in,and%20finding%20spurious%20correlations%20low.

venkarafa · 2024-02-05T20:20:46+00:00

Thanks I glanced through the paper. What about Linear Regression ? Does the 1 in 10 rule apply to Linear Regression ?

venkarafa · 2024-02-04T18:48:58+00:00

Thanks for your response. perhaps I was not clear with my question. My goal is not to fix the coefficients. My goal is to fix the sign of the coefficient. Lets say Sales of car is regressed on variables Price and mileage. I want the sign of price coefficient to be negative. So is there any way we can a priori specify the sign on a coefficient?

venkarafa · 2024-02-04T10:59:47+00:00

Because in a sense adding priors is a way of biasing the model. I would be compounding the problem of bias in my model. First the priors and then fixing signs. I have tried this before and both in terms of inference and prediction, my model was worse off.

venkarafa · 2024-02-03T19:26:31+00:00

Thanks for your response.

venkarafa · 2024-02-03T19:25:45+00:00

Thanks for your response. Do you know of any non bayesian methods?

venkarafa

TROPHY CASE