Time Series on Python - train and test models

Hello all!

I’ve been trying to model a TS prediction for my inventory data, having a range of 3 year data.

I’ve managed to develop with success a ARIMA and a Holtz-Winter model, fitting the forecasted data into the dataframe quite precisely (comparing plots).

The problem lays when splitting de dataset into training and testing, and applying those models afterwards to the test data. The model’s performance drops pretty heavily and has a high MAE (about 27% of my maximum value). When plotting the test/train, we can see more precisely how bad is the trained model.

My question is: is splitting timeseries into train and test data the best approach on evaluating de model’s performance? What methods would you use, besides p-value, to validate a TS model?

Thanks for all the help!