Least Squares Estimation

The need for Estimation

An example robot

A system can be characterized by quantifying some of its physical properties which are relevant to the process it is undergoing and all the quantified values stacked together in a vector form is termed as state. Now to ‘’’uniquely’’’ determine the state of the system at a particular instant of time, ‘’’sufficient’’’ amount of ‘’’certain’’’ information (the information could be an indirect function of state variables) about the state is required, but often the source of the information are measurements provided by sensors which are almost always ‘’’uncertain’’’ in which case the state cannot be ‘’’deterministically calculated’’’. Still, controlling a system requires a good approximation or an estimate of the state if not exactly the true state, hence the need of ‘’’estimating’’’ it.
The point to be noted here is that the need of ‘’’sufficiency’’’ of the information is still there for state estimation although the exact definition of sufficiency can only be understood after delving deeper into the subject of estimation. The following example will provide more clarity to the above discussion. Consider a little robot that can wander around in the woods, and the robot needs to know exactly where it is so that it can navigate. Robot has a state [math]\vec{x} = \begin{bmatrix} \vec{p}\\ \vec{v} \end{bmatrix}[/math] where [math]\vec{p}[/math] is the position and [math]\vec{v} [/math] is the velocity of the bot.
The robot also has a GPS sensor, which is accurate to about 5 meters, which is good, but it needs to know its location more precisely than 5 meters. There are lots of gullies and cliffs in these woods, and if the robot is wrong by more than a few feet, it could fall off a cliff. So GPS by itself is not good enough. GPS also provides the bot with its velocity which will also be inaccurate. The bot also has information about its speed from a wheel-speed-sensor placed inside it which measures the angular speed of its wheels but of course the wheels might slip a little bit so the rate at which the wheels are turning might not exactly represent its linear speed and in fact the measured angular speed will itself have some inaccuracy because of sensor noise.
A measurement vector [math]\vec{y}[/math] can be constructed as follows: [math]\vec{y} = \begin{bmatrix} \vec{p}_{m,G}\\ \vec{v}_{m,G}\\ \omega \end{bmatrix}[/math], where [math]\vec{p}_{m,G}[/math] is the measured position given by GPS sensor, [math]\vec{v}_{m,G}[/math] is the measured velocity also given by GPS sensor and [math]\omega[/math] is the angular speed of the bot’s wheels given by the wheel-speed-sensor. Thus given [math]\vec{y}[/math] and the relation between the physical variables measured and the state [math]\vec{x}[/math], we can estimate [math]\vec{x}[/math].

Formulation of any state estimation problem

Let [math]\vec{X} \in \mathbb{R}^n[/math] represent the state of a system and [math]\vec{Y} \in \mathbb{R}^r[/math] represent the vector containing all the measured information about the state. Physical properties of the system being measured will be known beforehand and can be written as some function of state vector [math]\vec{X}[/math], say [math]\vec{f}(\vec{X}): \mathbb{R}^n \rightarrow \mathbb{R}^r[/math]. Ideally, without any measurement errors, [math]\vec{f}(\vec{X}): [/math] should be equal to [math]\vec{Y}[/math] but the presence of uncertainty makes them unequal, thus