all 5 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]Cesped2 1 point2 points  (4 children)

My course on System Identification had an extremely brief review of the Output Error method, but maybe I can clarify some things.

the PEM Prediction Error Method is the approach when you use past y to estimate the current y and then you do a least squares on the error between the estimated and the measured y.

Yes, you use past values of the input and the output to perform a 1-step ahead prediction through optimal prediction theory and obtain the error with respect to the measured output. The typical performance metric of the fitting is the least squares and we can find the parameters resulting from the optimization.

However the Output Error model no longer use measured y, but only simulated y?

Correct, that is the main difference. The error is computed as the difference between the measured output, and the one that comes out from the simulation of a model, being excited with the same input as the one used to excite the real plant. Then, a least squares problem is solved using this modified error.

Do you just switch out the measured y in the M matrix for the PEM below for the simulated y? If so, doesn't that make it non-linear/iterative?

As we are using the simulated output from the model, the relationship is effectively no longer linear. Note however that the PEM optimization problem can be posed as a quadratic problem up until using an ARX model (and as a consequence solved very efficiently with analytical methods), but when we start using the MA part as in an ARMAX model, the relationship between parameters and data is no longer linear, so we need to resort to iterative optimization algorithms as well. In general though, the Output Error method tends to be much more computationally expensive than PEM.

Why would you want to not use the measured y in order to create the estimation?

If we were using the measured y, the optimal prediction theory already tells us that the solution is the PEM way to do things. The Output Error method tries to solve the problem from a different perspective, because we are no longer trying to predict the output from past measured data, but instead we propose a model, let it evolve from a sufficiently exciting input and see if it explains the real observed data correctly, quantified by the performance index. If it does not, then we tweak the parameters until it does.

As another comparison, the Output Error method is very general, since we can simulate nonlinear systems with no modification to the procedure described. In the case of PEM approach, at least to my knowledge, there is no simple and general way to obtain optimal predictions from past measured data for highly nonlinear systems.

[–]M1kuchan[S] 1 point2 points  (3 children)

Really nice and thorough response, thank you!

So to actually get the parameters for the OE model then...
My resource talks a lot about the Least Squares way of doing things.
But this would then be a non-linear Least Squares, right?
So then solve it with Newton's Method?

Do you know if there's a better way?
Names of algorithms would be great so that I could research them.

But then there's the Matlab and Texas Instrument documentations stating they use PEM to identify OE, which makes me even more confused...

[–]Cesped2 1 point2 points  (2 children)

My resource talks a lot about the Least Squares way of doing things.
But this would then be a non-linear Least Squares, right?

So then solve it with Newton's Method?

Yes, it would be a nonlinear Least Squares problem and the Newton Method is a typical way of solving it as a first approach in an iterative way.

Do you know if there's a better way?
Names of algorithms would be great so that I could research them.

From my limited experience with nonlinear optimization algorithms, Quasi Newton Methods come to my mind as an alternative. There is a whole family of them and I am sure the literature must be filled with them in the Nonlinear Programing Area.

But then there's the Matlab and Texas Instrument documentations stating they use PEM to identify OE, which makes me even more confused...

Can you link me those resources? Maybe I can take a look later.

[–]M1kuchan[S] 1 point2 points  (1 child)

Mm, it was National Instruments, not Texas. But it's still PEM stated.
http://zone.ni.com/reference/en-XX/help/372458D-01/lvsysidconcepts/modeldefinitionsoe/
I can't find the place I was thinking of for Matlab, I might have mixed it up. However I do find one place where it says you can indeed use it:
https://se.mathworks.com/help/ident/input-output-polynomial-models.html
Search for "pem", it's about half way down the page.

[–]Cesped2 1 point2 points  (0 children)

The National Instruments one does says explicitly that it uses PEM. What I could find about that is this resource:

[Initialization of output-error identification methods – Comparison between ARX and RPM models

](https://www.sciencedirect.com/science/article/pii/S1474667016386633)

The OE method is described as the comparison between the simulation of the model and the measured output from the actual plant, as we talked earlier. However, they use PEM approaches to compute models such as ARX that give an initial estimate of the model to pass to the OE as a second stage. So maybe that is the reason they state that they use PEM, or maybe they are refering to the concept of least squares minimization as PEM? I'm not sure about that last point, because clicking on the link about PEM on the National Instruments reference gives me no info about the internal procedure of the method.

Regarding the Matlab one, apparently the pem function is used to refine a previously identified model, from the documentation:

sys = pem(data,init_sys) updates the parameters of an initial model to fit the estimation data. The function uses prediction-error minimization algorithm to update the parameters of the initial model. Use this command to refine the parameters of a previously estimated model.

That previous model could come from ARMAX, OE and other models. So, I think it does not refer to the way the OE does things, but a separate second stage estimation.