HW2. Math 499. Spring 2007 Independent studies course Supervisor: Dr Angel R. Pineda, Assistant Professor Mathematics Department California State University, Fullerton

Student: Nasser Abbasi

Problem

question: Consider the solution of where is matrix, and is vector of i.i.d. Gaussian noise vector. (i.e. noise is white Gaussian noise). Determine the best solution

Answer:

Let us refer to the observed output (which includes the noise ) as , hence we write where is the uncontaminated output (what the observed output would be if there is no noise).

Since the noise is an additive noise to the output of the system, the since the noise has zero mean, then the mean of will be the same as the mean of . But is a deterministic signal which does not change, hence its mean is its value, hence the mean of is

Now, is described by a probability density function PDF as follows ( is in , hence it is long vector)

But since , then the above can be written as

Since is a constant matrix (system is assumed to be time-invariant), hence from the above we see that the expression gives the probability of observing for a given . Hence the best estimate of would be the one which maximizes this probability. Instead of maximizing the PDF directly, we maximize its natural logarithm (a mathematical convenience trick, no more).

Now find the natural logarithm of the above quantity, and find where the result is maximum.

Start with , hence the above becomes Note_1

Set the above to zero and solve for

Hence

This matches the least squares solution

Now I need to do this for and assuming that . We can start from equation (1) above, shown again below

Let

Then

so

and

Hence we obtain that

Hence

and

so, solve for . Write as and solve:

Solve for

so

and

Hence

is the least squares error. To validate

Compare equations (2) and (3) above, they are the same. Ok, confirmed.