Conserving variance of normally disributed vector in linear transformation












0












$begingroup$


I have normally distibuted vector $X$ with known variance $Var(X)$ and mean $overline X$ and then I take new vector $Y$ as $W*X$ where $W$ is normally distributed matrix. If I required $Y$ to have same variance and mean as $X$, what should $Var(W)$ and $overline W$ be ? More generally - given $overline X$, $Var(X)$, $overline W$ and $Var(W)$, what would $overline {W*X}$ and $Var(W*X)$ be ?



Note : For the desired application ( initialization in neural networks), the equality of means isn't really necessary as both means are likely to be almost zero anyway






statistics normal-distribution neural-networks







share|cite|improve this question









$endgroup$












  • $begingroup$
    Could you be clearer about the distribution assumptions? Since $X$ is a vector, is $text{Var}(X)$ the covariance matrix, or are you saying that the components of $X$ are i.i.d with common variance? Is $W$ independent of $X$? etc.
    $endgroup$
    – angryavian
    Jan 16 at 19:02










  • $begingroup$
    By $Var(X)$ i mean common variance. $X$ and $W$ are independent on each other.
    $endgroup$
    – Regedin
    Jan 16 at 19:05


















0












$begingroup$


I have normally distibuted vector $X$ with known variance $Var(X)$ and mean $overline X$ and then I take new vector $Y$ as $W*X$ where $W$ is normally distributed matrix. If I required $Y$ to have same variance and mean as $X$, what should $Var(W)$ and $overline W$ be ? More generally - given $overline X$, $Var(X)$, $overline W$ and $Var(W)$, what would $overline {W*X}$ and $Var(W*X)$ be ?



Note : For the desired application ( initialization in neural networks), the equality of means isn't really necessary as both means are likely to be almost zero anyway






statistics normal-distribution neural-networks







share|cite|improve this question









$endgroup$












  • $begingroup$
    Could you be clearer about the distribution assumptions? Since $X$ is a vector, is $text{Var}(X)$ the covariance matrix, or are you saying that the components of $X$ are i.i.d with common variance? Is $W$ independent of $X$? etc.
    $endgroup$
    – angryavian
    Jan 16 at 19:02










  • $begingroup$
    By $Var(X)$ i mean common variance. $X$ and $W$ are independent on each other.
    $endgroup$
    – Regedin
    Jan 16 at 19:05
















0












0








0





$begingroup$


I have normally distibuted vector $X$ with known variance $Var(X)$ and mean $overline X$ and then I take new vector $Y$ as $W*X$ where $W$ is normally distributed matrix. If I required $Y$ to have same variance and mean as $X$, what should $Var(W)$ and $overline W$ be ? More generally - given $overline X$, $Var(X)$, $overline W$ and $Var(W)$, what would $overline {W*X}$ and $Var(W*X)$ be ?



Note : For the desired application ( initialization in neural networks), the equality of means isn't really necessary as both means are likely to be almost zero anyway






statistics normal-distribution neural-networks







share|cite|improve this question









$endgroup$




I have normally distibuted vector $X$ with known variance $Var(X)$ and mean $overline X$ and then I take new vector $Y$ as $W*X$ where $W$ is normally distributed matrix. If I required $Y$ to have same variance and mean as $X$, what should $Var(W)$ and $overline W$ be ? More generally - given $overline X$, $Var(X)$, $overline W$ and $Var(W)$, what would $overline {W*X}$ and $Var(W*X)$ be ?



Note : For the desired application ( initialization in neural networks), the equality of means isn't really necessary as both means are likely to be almost zero anyway






statistics normal-distribution neural-networks




statistics normal-distribution neural-networks






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked Jan 16 at 19:00









RegedinRegedin

31




31












  • $begingroup$
    Could you be clearer about the distribution assumptions? Since $X$ is a vector, is $text{Var}(X)$ the covariance matrix, or are you saying that the components of $X$ are i.i.d with common variance? Is $W$ independent of $X$? etc.
    $endgroup$
    – angryavian
    Jan 16 at 19:02










  • $begingroup$
    By $Var(X)$ i mean common variance. $X$ and $W$ are independent on each other.
    $endgroup$
    – Regedin
    Jan 16 at 19:05




















  • $begingroup$
    Could you be clearer about the distribution assumptions? Since $X$ is a vector, is $text{Var}(X)$ the covariance matrix, or are you saying that the components of $X$ are i.i.d with common variance? Is $W$ independent of $X$? etc.
    $endgroup$
    – angryavian
    Jan 16 at 19:02










  • $begingroup$
    By $Var(X)$ i mean common variance. $X$ and $W$ are independent on each other.
    $endgroup$
    – Regedin
    Jan 16 at 19:05


















$begingroup$
Could you be clearer about the distribution assumptions? Since $X$ is a vector, is $text{Var}(X)$ the covariance matrix, or are you saying that the components of $X$ are i.i.d with common variance? Is $W$ independent of $X$? etc.
$endgroup$
– angryavian
Jan 16 at 19:02




$begingroup$
Could you be clearer about the distribution assumptions? Since $X$ is a vector, is $text{Var}(X)$ the covariance matrix, or are you saying that the components of $X$ are i.i.d with common variance? Is $W$ independent of $X$? etc.
$endgroup$
– angryavian
Jan 16 at 19:02












$begingroup$
By $Var(X)$ i mean common variance. $X$ and $W$ are independent on each other.
$endgroup$
– Regedin
Jan 16 at 19:05






$begingroup$
By $Var(X)$ i mean common variance. $X$ and $W$ are independent on each other.
$endgroup$
– Regedin
Jan 16 at 19:05












1 Answer
1






active

oldest

votes


















0












$begingroup$

So $X$ has i.i.d. normal entries, and $W$ has i.i.d. normal entries.



The entries of $WX$ will not be normal.



But you can answer your question about means and variances without any normality assumptions on $W$ and $X$.



Let $W$ have dimension $m times n$.



Since $$Y_i = sum_{j=1}^n W_{i,j} X_j$$
we can use independence to obtain
$$E[Y_i] = sum_{j=1}^n E[W_{i,j}] E[X_j] = n overline{W} overline{X}$$
and
$$text{Var}(Y_i) = sum_{j=1}^n text{Var}(W_{i,j} X_j) = n[
text{Var}(X) text{Var}(W) + text{Var}(X) overline{W}^2 + text{Var}(W) overline{X}^2
].$$



Setting these equal to $overline{X}$ and $text{Var}(X)$ respectively yields $overline{W} = 1/n$ and $$text{Var}(W) = frac{left(frac{1}{n} - frac{1}{n^2}right)text{Var}(X)}{text{Var}(X) + overline{X}^2}$$






share|cite|improve this answer









$endgroup$













  • $begingroup$
    Thanks a lot, thats exactly what i was looking for.
    $endgroup$
    – Regedin
    Jan 16 at 19:57











Your Answer





StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3076150%2fconserving-variance-of-normally-disributed-vector-in-linear-transformation%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









0












$begingroup$

So $X$ has i.i.d. normal entries, and $W$ has i.i.d. normal entries.



The entries of $WX$ will not be normal.



But you can answer your question about means and variances without any normality assumptions on $W$ and $X$.



Let $W$ have dimension $m times n$.



Since $$Y_i = sum_{j=1}^n W_{i,j} X_j$$
we can use independence to obtain
$$E[Y_i] = sum_{j=1}^n E[W_{i,j}] E[X_j] = n overline{W} overline{X}$$
and
$$text{Var}(Y_i) = sum_{j=1}^n text{Var}(W_{i,j} X_j) = n[
text{Var}(X) text{Var}(W) + text{Var}(X) overline{W}^2 + text{Var}(W) overline{X}^2
].$$



Setting these equal to $overline{X}$ and $text{Var}(X)$ respectively yields $overline{W} = 1/n$ and $$text{Var}(W) = frac{left(frac{1}{n} - frac{1}{n^2}right)text{Var}(X)}{text{Var}(X) + overline{X}^2}$$






share|cite|improve this answer









$endgroup$













  • $begingroup$
    Thanks a lot, thats exactly what i was looking for.
    $endgroup$
    – Regedin
    Jan 16 at 19:57
















0












$begingroup$

So $X$ has i.i.d. normal entries, and $W$ has i.i.d. normal entries.



The entries of $WX$ will not be normal.



But you can answer your question about means and variances without any normality assumptions on $W$ and $X$.



Let $W$ have dimension $m times n$.



Since $$Y_i = sum_{j=1}^n W_{i,j} X_j$$
we can use independence to obtain
$$E[Y_i] = sum_{j=1}^n E[W_{i,j}] E[X_j] = n overline{W} overline{X}$$
and
$$text{Var}(Y_i) = sum_{j=1}^n text{Var}(W_{i,j} X_j) = n[
text{Var}(X) text{Var}(W) + text{Var}(X) overline{W}^2 + text{Var}(W) overline{X}^2
].$$



Setting these equal to $overline{X}$ and $text{Var}(X)$ respectively yields $overline{W} = 1/n$ and $$text{Var}(W) = frac{left(frac{1}{n} - frac{1}{n^2}right)text{Var}(X)}{text{Var}(X) + overline{X}^2}$$






share|cite|improve this answer









$endgroup$













  • $begingroup$
    Thanks a lot, thats exactly what i was looking for.
    $endgroup$
    – Regedin
    Jan 16 at 19:57














0












0








0





$begingroup$

So $X$ has i.i.d. normal entries, and $W$ has i.i.d. normal entries.



The entries of $WX$ will not be normal.



But you can answer your question about means and variances without any normality assumptions on $W$ and $X$.



Let $W$ have dimension $m times n$.



Since $$Y_i = sum_{j=1}^n W_{i,j} X_j$$
we can use independence to obtain
$$E[Y_i] = sum_{j=1}^n E[W_{i,j}] E[X_j] = n overline{W} overline{X}$$
and
$$text{Var}(Y_i) = sum_{j=1}^n text{Var}(W_{i,j} X_j) = n[
text{Var}(X) text{Var}(W) + text{Var}(X) overline{W}^2 + text{Var}(W) overline{X}^2
].$$



Setting these equal to $overline{X}$ and $text{Var}(X)$ respectively yields $overline{W} = 1/n$ and $$text{Var}(W) = frac{left(frac{1}{n} - frac{1}{n^2}right)text{Var}(X)}{text{Var}(X) + overline{X}^2}$$






share|cite|improve this answer









$endgroup$



So $X$ has i.i.d. normal entries, and $W$ has i.i.d. normal entries.



The entries of $WX$ will not be normal.



But you can answer your question about means and variances without any normality assumptions on $W$ and $X$.



Let $W$ have dimension $m times n$.



Since $$Y_i = sum_{j=1}^n W_{i,j} X_j$$
we can use independence to obtain
$$E[Y_i] = sum_{j=1}^n E[W_{i,j}] E[X_j] = n overline{W} overline{X}$$
and
$$text{Var}(Y_i) = sum_{j=1}^n text{Var}(W_{i,j} X_j) = n[
text{Var}(X) text{Var}(W) + text{Var}(X) overline{W}^2 + text{Var}(W) overline{X}^2
].$$



Setting these equal to $overline{X}$ and $text{Var}(X)$ respectively yields $overline{W} = 1/n$ and $$text{Var}(W) = frac{left(frac{1}{n} - frac{1}{n^2}right)text{Var}(X)}{text{Var}(X) + overline{X}^2}$$







share|cite|improve this answer












share|cite|improve this answer



share|cite|improve this answer










answered Jan 16 at 19:21









angryavianangryavian

41.5k23381




41.5k23381












  • $begingroup$
    Thanks a lot, thats exactly what i was looking for.
    $endgroup$
    – Regedin
    Jan 16 at 19:57


















  • $begingroup$
    Thanks a lot, thats exactly what i was looking for.
    $endgroup$
    – Regedin
    Jan 16 at 19:57
















$begingroup$
Thanks a lot, thats exactly what i was looking for.
$endgroup$
– Regedin
Jan 16 at 19:57




$begingroup$
Thanks a lot, thats exactly what i was looking for.
$endgroup$
– Regedin
Jan 16 at 19:57


















draft saved

draft discarded




















































Thanks for contributing an answer to Mathematics Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


Use MathJax to format equations. MathJax reference.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3076150%2fconserving-variance-of-normally-disributed-vector-in-linear-transformation%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

MongoDB - Not Authorized To Execute Command

Image
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 0 I have successfully enabled authorization on MongoDB and I have created an account on the admin database and then I created an account for my database called test. The following connection string to connect to my test database works successfully: mongo --host 192.168.17.52 --port 27017 -u user1 -p password --authenticationDatabase test Only problem I have now is, I cannot execute commands such as: show dbs. I get the following error when I try to do so: "errmsg" : "not authorized on admin to execute command { listDatabases: 1.0, lsid: { id: UUID("a1d5bc0d-bc58-485e-b232-270758a89455") }, $db: "admin...
Read more

How to fix TextFormField cause rebuild widget in Flutter

Image
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 0 I've got the same issue at #9280. I tried all solutions from that topic with/without GlobalKey for Scaffold and Form but the problem wasn't fixed. I tried to build a authentication form with login and signup. But the keyboard doesn't show when I tap the TextFormField at the first time. Then I try to input some letter from physical keyboard, the widget is rebuilt. Like this: login screen Nothing to be log when I run flutter run --verbose, so I logged manually. Every tapping on field, the widget called dispose -> init -> build. [√] Flutter (Channel beta, v1.0.0, on Microsoft Windows [Version 10.0.17763.195], ...
Read more

in spring boot 2.1 many test slices are not allowed anymore due to multiple @BootstrapWith

Image
1 I tried to upgrade a yummy sandwich made of two test slices (@JsonTest and @JdbcTest in my case, crunchy test code in between) adding spring boot 2.1 flavour to it. But it seems it was not much of a success. I cannot annotate my tests with many @...Test since they are now each bringing their own XxxTestContextBootstrapper. It used to work when they all used same SpringBootTestContextBootstrapper. @RunWith(SpringRunner.class) @JdbcTest @JsonTest public class Test { @Test public void test() { System.out.printn("Hello, World !"); } } The error I get from BootstrapUtils is illegalStateException : Configuration error: found multiple declarations of @BootstrapWith for test class I understand I might be doing something wrong here but is there an easy way I could load both Json and Jdbc contex...
Read more