Dtyjlui

I have an exercise which is quite trivial. However I got stuck and I'm not sure if this the end-result. I assume there has to be a way to get this result much quicker.

Given are two randomly distributed variables y and n with mean 0 and variance 1. Further we know that $E{yn} = 0.5$ We measure $$x=y+n$$

We look for linear-mean-square estimation of y as a function of x.

From the given facts we now x has zero mean two. The variance has to be $$var(x) = var(y)+var(n)+2cdot cov(yn)$$
because y and n are correlated. Therefor $var(x)$ becomes $$1+1+2cdot 0.5 = 3$$

y becomes $y = n-x$ and $hat{y}$ should have the form $hat{y}= ax+b$.
For a and b I already have the general formula:
$$a=frac{E{xy}-m_xm_y}{sigma_x^2}$$
$$b=frac{E{x^2}m_y-E{xy}m_x}{sigma_x^2}.$$
In my case y should be $n-x$ if i got this right. But solving the problem with those formulas for a and b leaves me with terms like $E{xn}$. How can I know what this mean should be? I can't assume they are uncorrelated, can I? Probably I'm running completely in the wrong direction, and the solution is much more obvious.

EDIT 1: From Fat32's input I get for a:
$$a = frac{E{x(n-x)}}{sigma_x^2}=frac{3-1.5}{3}=1/2$$
and b $$b=frac{E{x^2}m_y-E{x(n-x)}m_x}{sigma_x^2}=frac{3-1.5}{3}=frac{1}{2}.$$
The solution would therefor be $y=-frac{1}{2}x+frac{1}{2}=frac{1}{2}(1-x)$ Not sure if this is true. Have to test it with random samples.

EDIT 2:
I did a test with matlab:

N = 10000;             %// Number of samples in each vector

M = randn(N, 2);



R = [1 0.5; 0.5 1]; %// correlation matrix

M = M * chol(R); %used to calculated depended random variables



n = M(:, 1);

y = M(:, 2);



x = y+n;



y_hat = -0.5*x+0.5;



mean(y-y_hat)

y_hat is not even close to the real y. Has not even the same mean. I don't get it. I'm making definitely some mistakes here.

EDIT 3:
Found another formula which uses the a and b. Inserted the linear leas squares solution becomes $$hat{y}=rho_{xy}frac{sigma_y}{sigma_x}(x-m_x)+m_y.$$ When I insert my values I get:
$$hat{y}=rho_{xy}frac{1}{sqrt{3}}x.$$
Rho is $frac{1.5}{sqrt(3)}$ and so $hat{y}$ becomes 0.5x as @Fat32 pointed out. The error above was that b is zero because $m_x$ and $m_y$ are zero.

Now I wanted to show you how to get those minimum linear mean square estimator coefficients $a$ and $b$ for your given problem setup. The procedure is summarised from the book Statistical Digital Signal Processing_MonsonHayes.

Given two random variables $X$ and $Y$, we observe $X$ and want to estimate $Y$ using a linear estimator :

$$ hat{Y} = acdot X + b $$

which minimized the mean square error $$xi^2 = E{ (Y-hat{Y})^2 } $$.

The solution is:

$$boxed{ a = frac{ E{XY} - m_xm_y }{ sigma_x^2} } $$

$$boxed{ b = frac{ E{X^2} m_y - E{X Y} m_x }{ sigma_x^2} } $$

And a better simplification happens by recognizing the correlation coefficient $$rho_{xy} = frac{ E{XY} - m_xm_y }{ sigma_x sigma_y } $$

Then the optimal linear estimator of $Y$ is re-written as:

$$boxed{ hat{Y} = rho_{xy} frac{sigma_y}{sigma_x}(X-m_x) + m_y }$$

Note that the resulting mimimum mean square error is also given by :

$$ xi_o^2 = sigma_y^2(1-rho_{xy}^2) $$

And further note that the orthogonality principle, for the optimum estimator, requires that:

$$ E{Xcdot E} = E{ X (Y - hat{Y}) } = 0 $$.

Now coming to your problem,

We are given the observation $ X = Y + N $ with the following statistics:

$$ E{Y}= E{N} = E{X} = 0 ~~~,~~~ sigma_y^2 = 1, sigma_n^2 = 1, sigma_x^2 = 3$$

(you can compute $sigma_x^2 = 3$ from the givens) and further given $E{YN} = 0.5$. Now we shall compute $rho_{xy}$ which is:

$$rho_{xy} = frac{ E{XY} - m_x m_y }{ sigma_x sigma_y } = frac{ E{(Y+N)Y}}-0cdot 0 }{ sqrt{3} } = frac{ 1 + 1.5}{ sqrt{3} } = frac{ sqrt{3} }{2}$$

Then the optimal linear mse becomes:

$$ hat{Y} = frac{sqrt{3}}{2} frac{1}{ sqrt{3}}(X-0) + 0 = 0.5 X $$

From which you can also infer that $a = 0.5$ and $b=0$.

Note that you could also reach the same result by just computing $a$ and $b$ according to formulas as follows:

$$ a = frac{ E{XY} - m_xm_y }{ sigma_x^2} = frac{ 1.5 - 0 cdot 0 }{3} = 0.5$$

$$ b = frac{ E{X^2} m_y - E{X Y} m_x }{ sigma_x^2} = frac{ 3 cdot 0 - 1.5 cdot 0 }{ 3} = 0$$

pretty simple ?

So in your case doesn't the relation $x = n+y$ help ?

I mean, assuming your derivation for the mean square estimtor is right, then to compute $E{xn}$ you would look for $E{ (y+n)n}$ and using properties of $x$ and $n$ you would get

$$E{xn} = E{(y+n)n} = E{yn} + E{n^2} = 0.5 + 1 = 1.5 $$