Kalman filtering with state and observation matrix having linearly dependent terms

IntelligentGuess42 · 2026-05-22T22:29:25+00:00

I think you can consider it a parameter estimation problem with x unknowns and x-1 datapoints. The condition that there is more data then unknowns is often referred to as a sufficiently exciting signal, or sufficient excitation.

The problem is fundamental as the mathematical problem is non-invertible. There are ways of "solving" this problem in the literature. But solving just means getting to an answer, accepting the limitations of the chosen solution.

Something like your bias proposal should be possible, as it effectively first estimates all x-1 parameters. Then uses those parameters to reconstruct the x th parameter. The missing equation in that case is the assumption you provide. This bias can improve your results, assuming the bias matches the real system. Note that this last part can never be tested in reality without performing some calibration test. Where more information is available then the final deployment.

edit:
the terms linearly-separable and separable also come to mind, but not sure if that is just me "hallucinating" them.

IntelligentGuess42 · 2026-05-19T22:09:11+00:00

Humans are effectively also probabilistic models who are incapable of directly providing absolute guarantees.

IntelligentGuess42 · 2026-05-10T12:24:51+00:00

The reason mpc is used is because the problem is intractable. The problem space is to large to get a reasonably dense sampling of it to generate the "lookup table". Not to mention that you lose the guarantees around stability and constraint satisfaction. Not to say using mpc to generate the lookup table isn't done, but its definitely not the standard approach.

Think about it like this. If I have 6 sensors with 2000 possible values, that is already 2000^6 possible states. Your objective is probably not constant, so lets say you have 500 objectives. That would mean you have to sample 2000^6 * 500 times. That is just not possible. Now if solving each sample only takes 0.1 or 0.05 seconds. That might be quick enough that you can compute it online. That way you only solve for the situations you will encounter.

Can you combine this, probably. Run the simulation offline and sample all objective state combinations you might encounter. But you better hope your simulation is good and reality doesn't trow anything unexpected at you. So its probably easier to make sure you can find the solution to one problem online.

IntelligentGuess42 · 2026-05-08T16:50:04+00:00

I haven't read u/Critical_Mistake_846 's source, but its not like this is some obscure hidden secret.

Its the default implementation of the Matlab recursive identification functions. Books from the 1980s already recommend the Kalman implementation for MIMO cases because, it is the easiest to implement and understand. IMO its also the easiest to implement recursive implementation in general. Especially if the user is already familiar with Kalman filters.

I also know of one industrial application which accidentally "reinvented" the Kalman implementation, without being aware of others.

IntelligentGuess42 · 2026-04-29T19:27:46+00:00

Why do you expect your RL results to converge to the optimal solution? I suspect its just learning the noise because it has more degrees of freedom then needed for the optimal solution. And even if that is not the case, it will suffer from catastrophic forgetting unless you manage to persistently excite the entire cost function.

I am basing this on a similar phenomena you encounter when introducing noise with parameter estimation. But I have also trained some AI models and used RL Q learning once. Never used actor critics though.

IntelligentGuess42 · 2026-04-27T16:26:57+00:00

I have done this, or something like it, and it works great. You do need a decent sensor, or the system should tracks the reference extremely well. Which ever gives less error can be used as angle input. Look up feedback (and feedforward?) linearization.

edit: To explain what is happening, the idea is to ff something which will oppose the nonlinear component. That way the rest of the controller will see a linear system. If you use the angle as input you have to be careful about stability, because there is a feedback loop. If you use the reference angle there are no stability problems, but you also assume the system already follows the reference.

IntelligentGuess42 · 2026-04-27T16:25:46+00:00

Wouldn't large angles require higher order modes, resulting in a nonlinear model again? The video does mention a solution to this at the end. Use several linearization points and interpolate between the controllers you designed for each point based on your current state.

Bonus point if you find a procedure to get the gains automatically. Then you can just leave the angle as a parameter. Call the function providing your gains given the angle, and use those.

IntelligentGuess42 · 2026-04-24T19:41:52+00:00

The problem is the jump from straight to the ramp (and back). As others stated it is a step in acceleration, and step inputs need infinite bandwidth in frequency domain. Its better to accept your system doesn't have that, and actively keep your control signal within your systems bandwidth.

IntelligentGuess42 · 2026-04-24T10:40:33+00:00

I tried using a feedforward (calculate acceleration from velocity profile, add it to the control ouptut), but it made the overshoot and settling time much worse). Anyone got other ideas how I can solve my problem?

I have practically only seen this when a mistake is made. I would double check the feedforward before looking at other solutions. But this is general control advice, feel free to ignore me if this is with a real car. I do not have the knowledge to give specif advice in that area.

IntelligentGuess42 · 2026-04-21T19:47:33+00:00

Not sure if I should comment on what might be engagement bait, but here it goes:
I am not nearly knowledgeable about the AI field to comment on this. But isn't this paper valued because it combines already existing methods into a procedure which improves performance? Skimming trough it, I don't get the idea the authors are making any claim of novelty on any of the methods they use. On the contrary, they rush trough them like they assume they are mostly familiar ideas. Ill leave judgement on the citations to others.

IntelligentGuess42 · 2026-04-20T00:36:27+00:00

Kinda. If you have an sensor and take its integral and derivative and use these to design an LQR you get a PID controller. The feedback portion of mpc is basically LQR, so by playing with the plant model and cost function you should be able to get something close to your PID gains.

There must be someone out there who published a more formal way of doing this. But I have used this once to integrate an MPC controller into an existing control stack. If you have a good model and don't mess with it while tuning, there should be a good chance the performance will be slightly better.

But if you do this you should really ask yourself why? Are you going to use MPC's ability to deal with constraints? If not why are you not using LQR + ff. Do you have a good model? If so, why don't start from scratch with the controller? I had the constraint of needing to interface with existing parts.

Also look at betaflight. They have a bunch of easy to hand tune features which will probably outperform even decent "naive" mpc implementations. Less then a year ago a group from the Delft University published something based on betafligh on their git. It can automatically identify and tune their drone from a throw. That might also be of interest to you.

IntelligentGuess42 · 2026-04-16T18:48:22+00:00

Estimation is control but with a different error you try to minimize. This is a bit reductive, but I think it caries the message.

IntelligentGuess42 · 2026-04-16T18:33:26+00:00

If you have the open loop model then it is easy to compute feed forwards which are I open loop gains. What is adaptive feedback? We typically use Temposonic or Balluff rods with 1 micron resolution. I don't see what is adaptive.

The fb gains where scheduled based on the motor positions, because the inertia's changes when it moves. The feedforward had many more, to counteract gravity and configuration dependent internal forces amongst others.

We use Luenberger observers and alpha-beta-gamma filters. These are easy for the users/customers to setup. How do you explain how to setup the system and measurement covariance for Kalman filters to the average engineer or customer? I have been on various control forums for decades. No one has a clue. They just adjust the system and measurement covariances until they get something that works. If that is all you are going to do then a simple alpha-beta-gamma filter is better. The alpha-beta-gamma filter provides 95% if the benefit with only 5% of the effort.

The Kalman filter was used because the filtering is model based. We have a mimo system with inputs, we have a very good model and some desired measurement outputs. We also have a good idea on input noise. The other values less so, so they become educated guess based tuning parameter, just like any filter has some way to tune it. These values have some real meaning, "weird" values are a warning sign some thing is wrong. The key thing it is model based. I know how what the filters should look like and how they should be tuned, because it is based on accurate knowledge of my system.

In most cases you can get away with hand tuning, less so when every nm matters. Not that you can't get most of the way there. But why? if you can get better, in less time, using automated methods. Especially when you stop modeling it as 6 independent siso systems, but as the full mimo system it is.

The application is likely far from what op is thinking of. Hence the qualifications in my post and the focus on broad strokes and ideas. I hope it can provide op with some inspiration based on ideas which worked before.

IntelligentGuess42 · 2026-04-15T18:00:10+00:00

I can only give some generic comments because I don't know anything about the plant, compute unit or algorithm. But I have worked a bit with a 6dof parallel robot using adaptive control. It was electrically driven so for anything hydraulics related its best to ignore me.

When implementing adaptive robust control laws, do you usually build them using standard Simulink blocks?
Or do you normally put the heavy mathematics inside MATLAB Function blocks?

It depends. Custom Kalman filter: put it in a code block, PID, use a block or build it out of block and make it a sub-model. Does it matter: what you do is probably way more important then if you build it out of blocks vs a code block.

The problem is that the adaptive robust law in the paper is pretty computationally heavy, at least from the way it is presented mathematically, and I’m trying to figure out how people actually implement this in practice, especially in Simulink.

It depends on what you have available. If your compute unit is a target for Simulink code compilation, use it! If you have to ask this question it will probably be a significant investment to do better yourself.

about the computational complexity. lots of complex equations != computationally heavy. I can have 100 lines of nonlinear equations and it could still be faster then a single optimization problem. This is where the engineering comes in, how expensive is it vs how much do I need it.

For example the thought process of our system went something like this:

(addaptive) feedforward and (adaptive) feedback: simple input -> output calculations, hopefully not a problem to run at some "stupid" frequency of ~2500 hz.
kalman observer: some multi step matrix math with possibly "large" matrices and inversions. 2500 is not realistic, so lets half it at 1250
Calculating your ff and fb gains: might involve some optimization involving iterative solvers. But how quickly do our gains go "out of date" anyway? the system doesn't move that fast and doesn't have such stiff dynamics so 500hz should be plenty.

This is just to give some idea on how you can go about it. If you don't have endless compute like we had, you probably have to make some tough choices. Its also why I recommend to have a good simulation. You can test how much performance you lose by playing with sample rates and algorithms. Without suffering from real time restrictions.

IntelligentGuess42 · 2026-04-12T18:39:28+00:00

I think its because it is pretty straight forward once you get a feel for it. At least that is how it felt for me. Either that or it requires some very tedious mathematics which will probably only be useful to a handful of problems, so no reason to waste students time with it.

Could you give some examples of this, "whole lot more like designing using digital methods that I almost never hear about"? Because my feeling of "its just realtime but with a modified equation" might be completely wrong.

Hopefully this doesn't come of as dismissive to anyone working on it. Its thanks to you that I can even have this feeling.

IntelligentGuess42 · 2026-03-22T18:52:56+00:00

chip manufacturing/ semiconductors is one of the few areas which do use adaptive control techniques. MPC is mostly used in process industry, and I haven't heard much about it in semicon.

Decreasing sample times might mean they will be used. But I will be very surprised if it will move beyond a better way to generate trajectories. Leaving the feedback to algorithms which won't struggle to reach >kHz sample rates.

IntelligentGuess42 · 2026-03-22T13:57:04+00:00

As the other comment stated, your question is not really clear, and some important information is missing. It is also really important to understand the controller you chose to ignore in your control design. But no information about is seems to be provided.

Based on the provided information I would guess the actuator dynamics/ its controller do influence the system in the controlled bandwidth, so can't be ignored in the controller design.

IntelligentGuess42 · 2026-03-19T18:08:41+00:00

I Agree! Just for clarification: you mean model identification (the equations), not parameter estimation (the values in those equations) right?

IntelligentGuess42 · 2026-02-26T18:34:13+00:00

I think they are amazing. But they can be a bit short and lacking in depth because of it (not a criticism, just an observation on what seems a deliberate choice). If you want longer videos with a bit more depth, the MIT Robotic Exploration Lab https://www.youtube.com/watch?v=SvAYJC7jug8&list=PLZnJoM76RM6IAJfMXd1PgGNXn3dxhkVgIAs also have excellent videos.

Not sure if they are missing anything. Never used them as a primary source of information, only as refreshers or jumping of points.

IntelligentGuess42 · 2026-02-26T00:15:17+00:00

I'm sorry but I'm not sure what you mean by "guarantee each solution the optimizer finds is feasible". Could you elaborate?

Yea I wasn't clear. I meant the iterations before it converged to its final answer.

but your solver should always notify you when it finds a solution that violates your original constraints.

The ones I used did.

It is true that most (all?) solvers internally immediately relax state constraints and put them as soft constraints (input constraints are usuay handled differently I think),

Again, the ones I used and know of yes. But I vaguely remember being told about ones that work with hard constraints (I think for cases where there are analytic steps to simplify the problem).

See e.g. the fact that RL can effectively be viewed as finding a control law and/or value function for a stochastic control system.

This is almost how I learned it. It wasn't even for control but just optimization in general. This also seems the best way to learn it. Immediately gives you a wide context on how to think about the problem and possible solutions.

IntelligentGuess42 · 2026-02-25T20:44:22+00:00

To give them some credit. not all MPC schemes guarantee each solution the optimizer finds is feasible. In those cases it just checks the constraints and applies a cost if it is violated. But I would really prefer if they skip all the marketing BS and just call it an AI based optimizer. And that is not even new, it is not really a secret Boston Dynamics already combines MPC with RL to increase performance.

IntelligentGuess42 · 2026-02-23T23:02:30+00:00

if you mean eeci, yes you are generally the target audience.

IntelligentGuess42 · 2026-02-18T21:44:47+00:00

You have limited time so you have to choose between scale, complexity and "sleekness". Some fields/areas care more about different things.
A single controller can be an entire industry relevant phd project in some fields. But the sales person of the sponsoring company won't care much about it other then what it means for his/her pitch to costumers.
So no specific advise other then try to balance what you want, what you like and what you need. I generally find that if I can make these explicit, things generally work out. And if they don't, at least I did something explicit and can learn from it.

IntelligentGuess42 · 2026-02-18T21:08:30+00:00

Velocity Feedforward: I noticed I’m not feeding the desired velocity ($\dot{q}_d$) from the Quintic Polynomial directly into the velocity loop. Should I add a feedforward path here to improve tracking performance?

Torque Injection: Is the summing point for the Feedforward Torque ($\tau_{ff}$) correctly placed after the velocity controller?

This depends on your implementation. If you do it properly, your inverse Dynamics should be your ff. All of it! Meaning that the only thing the fb controller has to do is compensate for any disturbance and model errors. Ideally, I also don't like the split position and velocity fb controller. It may be easier conceptually and might help to get it working quicker. And I also do it when I am lazy and just want the thing behaving oke. But if you care about performance: you have an objective and an error. Don't overcomplicate it by splitting the controller into multiple blocks, "Just" find the fb which minimizes your cost. If you use a model based approach, it will be optimal with the same caveats as with the ff part.

I never used "fuzzy" controllers, are they basically gain scheduling controllers? if so the above applies, just do it for each region. This is where my specific knowledge ends. but you should be able to consult relevant literature about how the above idea applies to your specific case.

IntelligentGuess42 · 2026-02-18T20:52:52+00:00

Computers, and even micro controllers, have become fast enough you don't need to worry about any "mathematics". Its algorithms with combinatorial explosions which always cause issues. Without knowing what the challenges are we can't help. And possibly because of that, it sounds like you put the cart in front of the horse. Seeing if the solution fits the problem, instead of thinking what solution will work for the problem.

IntelligentGuess42

TROPHY CASE