Mixing
What is the conditional probability density function of a statistic given its samples?
$begingroup$
I want to find a probability density function (pdf) of
a statistic $T:=t(X_1,dots, X_n)$ given its
samples $(X_1,dots, X_n)=: mathbf{X}$, where
$t(cdot)$ is a function such as
$t(x_1,dots, x_n) := frac{1}{n}sum_{i=1}^n x_i$ (sample mean).
In other words, I want to find the pdf $p_{T mid mathbf{X}}$.
When $mathbf{X} = (X_1,dots, X_n)$ are discrete random variables,
I think finding the counterparts of the probability math function (pmf) is easy.
The answer is as follows:
$$
p_{Tmid mathbf{X}}(tmid x_1,dots, x_n) =
begin{cases}
1, & text{if } t = t(x_1,dots, x_n), \
0, & text{otherwise}.
end{cases}
$$
What about when the samples are continuous?
Motivation:
I want to find the pdf $p_{Ymid mathbf{X}}$ given the following Markov chain:
$$
mathbf{X} to Tto Y,
$$
where $T:= t(mathbf{X})$.
In other words, I want to represent $p_{Ymid mathbf{X}}$ by
$p_mathbf{X}, t(cdot)$ and $p_{Ymid T}$.
I have tried the following calculation:
begin{align}
p_{Ymid mathbf{X}} (ymid mathbf{x})
&= frac{p_{mathbf{X}, Y}(mathbf{x}, y)}{p_{mathbf{X}}(mathbf{x})}\
&= frac{int p_{mathbf{X}}(mathbf{x})p_{Tmid mathbf{X}}(tmid mathbf{x})p_{Ymid T}(ymid t), dt}{intint p_{mathbf{X}}(mathbf{x})p_{Tmid mathbf{X}}(tmid mathbf{x})p_{Ymid T}(ymid t), dtdy}.
end{align}
Then I have realized that I need to calculate the pdf $p_{Tmid mathbf{X}}$.
When $Y = T + Z$ and $Zsim N(0, sigma^2)$, i.e.,
$p_{Ymid T}$ is a pdf of the distribution $N(t, sigma^2)$,
I suppose $p_{Ymid X}(ymid x)$ is equal to $p_{Ymid T}(y mid t(x))$.
probability markov-chains conditional-probability
asked Jan 9 at 18:58
ykkxyzykkxyz
83
$endgroup$
add a comment |
$begingroup$
I want to find a probability density function (pdf) of
a statistic $T:=t(X_1,dots, X_n)$ given its
samples $(X_1,dots, X_n)=: mathbf{X}$, where
$t(cdot)$ is a function such as
$t(x_1,dots, x_n) := frac{1}{n}sum_{i=1}^n x_i$ (sample mean).
In other words, I want to find the pdf $p_{T mid mathbf{X}}$.
When $mathbf{X} = (X_1,dots, X_n)$ are discrete random variables,
I think finding the counterparts of the probability math function (pmf) is easy.
The answer is as follows:
$$
p_{Tmid mathbf{X}}(tmid x_1,dots, x_n) =
begin{cases}
1, & text{if } t = t(x_1,dots, x_n), \
0, & text{otherwise}.
end{cases}
$$
What about when the samples are continuous?
Motivation:
I want to find the pdf $p_{Ymid mathbf{X}}$ given the following Markov chain:
$$
mathbf{X} to Tto Y,
$$
where $T:= t(mathbf{X})$.
In other words, I want to represent $p_{Ymid mathbf{X}}$ by
$p_mathbf{X}, t(cdot)$ and $p_{Ymid T}$.
I have tried the following calculation:
begin{align}
p_{Ymid mathbf{X}} (ymid mathbf{x})
&= frac{p_{mathbf{X}, Y}(mathbf{x}, y)}{p_{mathbf{X}}(mathbf{x})}\
&= frac{int p_{mathbf{X}}(mathbf{x})p_{Tmid mathbf{X}}(tmid mathbf{x})p_{Ymid T}(ymid t), dt}{intint p_{mathbf{X}}(mathbf{x})p_{Tmid mathbf{X}}(tmid mathbf{x})p_{Ymid T}(ymid t), dtdy}.
end{align}
Then I have realized that I need to calculate the pdf $p_{Tmid mathbf{X}}$.
When $Y = T + Z$ and $Zsim N(0, sigma^2)$, i.e.,
$p_{Ymid T}$ is a pdf of the distribution $N(t, sigma^2)$,
I suppose $p_{Ymid X}(ymid x)$ is equal to $p_{Ymid T}(y mid t(x))$.
probability markov-chains conditional-probability
asked Jan 9 at 18:58
ykkxyzykkxyz
83
$endgroup$
1
$begingroup$
Well, since $T=t(X_1,ldots,X_n)$ is a function of the data $X=(X_1,ldots,X_n)$, conditional distribution of $T$ given $X$ is just constant $T=t(X)$. Uncertainty disappears.
$endgroup$
– Song
Jan 9 at 20:49
$begingroup$
@Song "Uncertainty disappears." Intuitively, yes. Then, how should I formulate the intuition, like when samples are discrete random variables?
$endgroup$
– ykkxyz
Jan 10 at 3:58
add a comment |
$begingroup$
I want to find a probability density function (pdf) of
a statistic $T:=t(X_1,dots, X_n)$ given its
samples $(X_1,dots, X_n)=: mathbf{X}$, where
$t(cdot)$ is a function such as
$t(x_1,dots, x_n) := frac{1}{n}sum_{i=1}^n x_i$ (sample mean).
In other words, I want to find the pdf $p_{T mid mathbf{X}}$.
When $mathbf{X} = (X_1,dots, X_n)$ are discrete random variables,
I think finding the counterparts of the probability math function (pmf) is easy.
The answer is as follows:
$$
p_{Tmid mathbf{X}}(tmid x_1,dots, x_n) =
begin{cases}
1, & text{if } t = t(x_1,dots, x_n), \
0, & text{otherwise}.
end{cases}
$$
What about when the samples are continuous?
Motivation:
I want to find the pdf $p_{Ymid mathbf{X}}$ given the following Markov chain:
$$
mathbf{X} to Tto Y,
$$
where $T:= t(mathbf{X})$.
In other words, I want to represent $p_{Ymid mathbf{X}}$ by
$p_mathbf{X}, t(cdot)$ and $p_{Ymid T}$.
I have tried the following calculation:
begin{align}
p_{Ymid mathbf{X}} (ymid mathbf{x})
&= frac{p_{mathbf{X}, Y}(mathbf{x}, y)}{p_{mathbf{X}}(mathbf{x})}\
&= frac{int p_{mathbf{X}}(mathbf{x})p_{Tmid mathbf{X}}(tmid mathbf{x})p_{Ymid T}(ymid t), dt}{intint p_{mathbf{X}}(mathbf{x})p_{Tmid mathbf{X}}(tmid mathbf{x})p_{Ymid T}(ymid t), dtdy}.
end{align}
Then I have realized that I need to calculate the pdf $p_{Tmid mathbf{X}}$.
When $Y = T + Z$ and $Zsim N(0, sigma^2)$, i.e.,
$p_{Ymid T}$ is a pdf of the distribution $N(t, sigma^2)$,
I suppose $p_{Ymid X}(ymid x)$ is equal to $p_{Ymid T}(y mid t(x))$.
probability markov-chains conditional-probability
asked Jan 9 at 18:58
ykkxyzykkxyz
83
$endgroup$
I want to find a probability density function (pdf) of
a statistic $T:=t(X_1,dots, X_n)$ given its
samples $(X_1,dots, X_n)=: mathbf{X}$, where
$t(cdot)$ is a function such as
$t(x_1,dots, x_n) := frac{1}{n}sum_{i=1}^n x_i$ (sample mean).
In other words, I want to find the pdf $p_{T mid mathbf{X}}$.
When $mathbf{X} = (X_1,dots, X_n)$ are discrete random variables,
I think finding the counterparts of the probability math function (pmf) is easy.
The answer is as follows:
$$
p_{Tmid mathbf{X}}(tmid x_1,dots, x_n) =
begin{cases}
1, & text{if } t = t(x_1,dots, x_n), \
0, & text{otherwise}.
end{cases}
$$
What about when the samples are continuous?
Motivation:
I want to find the pdf $p_{Ymid mathbf{X}}$ given the following Markov chain:
$$
mathbf{X} to Tto Y,
$$
where $T:= t(mathbf{X})$.
In other words, I want to represent $p_{Ymid mathbf{X}}$ by
$p_mathbf{X}, t(cdot)$ and $p_{Ymid T}$.
I have tried the following calculation:
begin{align}
p_{Ymid mathbf{X}} (ymid mathbf{x})
&= frac{p_{mathbf{X}, Y}(mathbf{x}, y)}{p_{mathbf{X}}(mathbf{x})}\
&= frac{int p_{mathbf{X}}(mathbf{x})p_{Tmid mathbf{X}}(tmid mathbf{x})p_{Ymid T}(ymid t), dt}{intint p_{mathbf{X}}(mathbf{x})p_{Tmid mathbf{X}}(tmid mathbf{x})p_{Ymid T}(ymid t), dtdy}.
end{align}
Then I have realized that I need to calculate the pdf $p_{Tmid mathbf{X}}$.
When $Y = T + Z$ and $Zsim N(0, sigma^2)$, i.e.,
$p_{Ymid T}$ is a pdf of the distribution $N(t, sigma^2)$,
I suppose $p_{Ymid X}(ymid x)$ is equal to $p_{Ymid T}(y mid t(x))$.
probability markov-chains conditional-probability
probability markov-chains conditional-probability
asked Jan 9 at 18:58
ykkxyzykkxyz
83
asked Jan 9 at 18:58
ykkxyzykkxyz
83
edited Jan 10 at 3:38
ykkxyz
asked Jan 9 at 18:58
ykkxyzykkxyz
83
asked Jan 9 at 18:58
ykkxyzykkxyz
83
asked Jan 9 at 18:58
ykkxyzykkxyz
83
83
1
$begingroup$
Well, since $T=t(X_1,ldots,X_n)$ is a function of the data $X=(X_1,ldots,X_n)$, conditional distribution of $T$ given $X$ is just constant $T=t(X)$. Uncertainty disappears.
$endgroup$
– Song
Jan 9 at 20:49
$begingroup$
@Song "Uncertainty disappears." Intuitively, yes. Then, how should I formulate the intuition, like when samples are discrete random variables?
$endgroup$
– ykkxyz
Jan 10 at 3:58
add a comment |
1
$begingroup$
Well, since $T=t(X_1,ldots,X_n)$ is a function of the data $X=(X_1,ldots,X_n)$, conditional distribution of $T$ given $X$ is just constant $T=t(X)$. Uncertainty disappears.
$endgroup$
– Song
Jan 9 at 20:49
$begingroup$
@Song "Uncertainty disappears." Intuitively, yes. Then, how should I formulate the intuition, like when samples are discrete random variables?
$endgroup$
– ykkxyz
Jan 10 at 3:58
1
1
$begingroup$
Well, since $T=t(X_1,ldots,X_n)$ is a function of the data $X=(X_1,ldots,X_n)$, conditional distribution of $T$ given $X$ is just constant $T=t(X)$. Uncertainty disappears.
$endgroup$
– Song
Jan 9 at 20:49
$begingroup$
Well, since $T=t(X_1,ldots,X_n)$ is a function of the data $X=(X_1,ldots,X_n)$, conditional distribution of $T$ given $X$ is just constant $T=t(X)$. Uncertainty disappears.
$endgroup$
– Song
Jan 9 at 20:49
$begingroup$
@Song "Uncertainty disappears." Intuitively, yes. Then, how should I formulate the intuition, like when samples are discrete random variables?
$endgroup$
– ykkxyz
Jan 10 at 3:58
$begingroup$
@Song "Uncertainty disappears." Intuitively, yes. Then, how should I formulate the intuition, like when samples are discrete random variables?
$endgroup$
– ykkxyz
Jan 10 at 3:58
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
I'm not sure that OP is familiar with the measure-theoretic definition of conditional expectation and probability, but I'll present it anyway. (This is very roughly written assuming that you have little background on measure theory.) If $X,Y$ are random variables such that $E[|X|]<infty$, then the conditional expectation of $X$ given $Y$ is defined as the unique random variable $ E[X|Y] = Phi(Y)$ (almost surely) for some (Borel-measurable) function $Phi$ such that
$$E[Xf(Y)] = E[Phi(Y)f(Y)]tag{*}
$$ holds for all bounded (Borel-measurable) function $f$.
We often write $Phi(y)$ as $E[X|Y=y]$. If $Y$ is a discrete random variable, then the conditional expectation calculated by $$
E[X|Y=y] = frac{E[X1_{{Y=y}}]}{P(Y=y)},tag{**}
$$ coincides with the $Phi(y)$ in the measure theoretic definition $(*)$ (You may be able to check this.) If $X,Y$ are random variables for which their joint p.d.f. exists, it can also be calculated explicitly by
$$
E[X|Y=y]=frac{int xf_{XY}(x,y)dxdy}{f_Y(y)}.tag{***}
$$ One can also prove that the one obtained from $(***)$ satisfies the definition of $(*)$. In this sense, $(*)$ is a generalization of those elementary definitions. However, if it is not the either cases, it is hard to obtain explicit formula for $Phi$ and one should rely on the (abstract) definition $(*)$.
Now, the following seemingly obvious statement about conditional expectation can be shown directly by the definition: If $E|F(X)|<infty$, then (almost surely)
$$
E[F(X)|X] = F(X).
$$Proof: Obviously, $F(X)$ is a function of $X$. By uniqueness it is sufficient to check $(*)$ holds. This is true since
$$
E[F(X) f(X)] = E[E[F(X)|X] f(X)]=E[F(X)f(X)]
$$ for all bounded $f$. $blacksquare$
This is the formalization of "Uncertainty disappears". It remains to check if $$P(T(X)in B|X) = 1_{{T(X)in B}}=begin{cases} 1quadtext{if }; T(X)in B\0quadtext{otherwise}end{cases}.$$ (Here, $1_{A}(x) = 1_{xin A}$ denotes the indicator function.) It follows from
$$
P(T(X)in B|X)=E[1_{{T(X)in B}}|X]= E[1_{{T(cdot)in B}}(X)|X]=1_{{T(cdot)in B}}(X)=1_{{T(X)in B}}.
$$ This shows $P(T(X)in B|X=x) = 1_{{T(x)in B}}$ and $T(X)|_{X=x}$ has a degenerate distribution at $T(x)$.
answered Jan 10 at 10:42
SongSong
11.4k628
$endgroup$
$begingroup$
@ Song I really appreciate your answer. As you say, I'm not so familiar with measure theory but I think I manage to understand it. Thanks.
$endgroup$
– ykkxyz
Jan 11 at 15:30
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3067830%2fwhat-is-the-conditional-probability-density-function-of-a-statistic-given-its-sa%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
I'm not sure that OP is familiar with the measure-theoretic definition of conditional expectation and probability, but I'll present it anyway. (This is very roughly written assuming that you have little background on measure theory.) If $X,Y$ are random variables such that $E[|X|]<infty$, then the conditional expectation of $X$ given $Y$ is defined as the unique random variable $ E[X|Y] = Phi(Y)$ (almost surely) for some (Borel-measurable) function $Phi$ such that
$$E[Xf(Y)] = E[Phi(Y)f(Y)]tag{*}
$$ holds for all bounded (Borel-measurable) function $f$.
We often write $Phi(y)$ as $E[X|Y=y]$. If $Y$ is a discrete random variable, then the conditional expectation calculated by $$
E[X|Y=y] = frac{E[X1_{{Y=y}}]}{P(Y=y)},tag{**}
$$ coincides with the $Phi(y)$ in the measure theoretic definition $(*)$ (You may be able to check this.) If $X,Y$ are random variables for which their joint p.d.f. exists, it can also be calculated explicitly by
$$
E[X|Y=y]=frac{int xf_{XY}(x,y)dxdy}{f_Y(y)}.tag{***}
$$ One can also prove that the one obtained from $(***)$ satisfies the definition of $(*)$. In this sense, $(*)$ is a generalization of those elementary definitions. However, if it is not the either cases, it is hard to obtain explicit formula for $Phi$ and one should rely on the (abstract) definition $(*)$.
Now, the following seemingly obvious statement about conditional expectation can be shown directly by the definition: If $E|F(X)|<infty$, then (almost surely)
$$
E[F(X)|X] = F(X).
$$Proof: Obviously, $F(X)$ is a function of $X$. By uniqueness it is sufficient to check $(*)$ holds. This is true since
$$
E[F(X) f(X)] = E[E[F(X)|X] f(X)]=E[F(X)f(X)]
$$ for all bounded $f$. $blacksquare$
This is the formalization of "Uncertainty disappears". It remains to check if $$P(T(X)in B|X) = 1_{{T(X)in B}}=begin{cases} 1quadtext{if }; T(X)in B\0quadtext{otherwise}end{cases}.$$ (Here, $1_{A}(x) = 1_{xin A}$ denotes the indicator function.) It follows from
$$
P(T(X)in B|X)=E[1_{{T(X)in B}}|X]= E[1_{{T(cdot)in B}}(X)|X]=1_{{T(cdot)in B}}(X)=1_{{T(X)in B}}.
$$ This shows $P(T(X)in B|X=x) = 1_{{T(x)in B}}$ and $T(X)|_{X=x}$ has a degenerate distribution at $T(x)$.
answered Jan 10 at 10:42
SongSong
11.4k628
$endgroup$
$begingroup$
@ Song I really appreciate your answer. As you say, I'm not so familiar with measure theory but I think I manage to understand it. Thanks.
$endgroup$
– ykkxyz
Jan 11 at 15:30
add a comment |
$begingroup$
I'm not sure that OP is familiar with the measure-theoretic definition of conditional expectation and probability, but I'll present it anyway. (This is very roughly written assuming that you have little background on measure theory.) If $X,Y$ are random variables such that $E[|X|]<infty$, then the conditional expectation of $X$ given $Y$ is defined as the unique random variable $ E[X|Y] = Phi(Y)$ (almost surely) for some (Borel-measurable) function $Phi$ such that
$$E[Xf(Y)] = E[Phi(Y)f(Y)]tag{*}
$$ holds for all bounded (Borel-measurable) function $f$.
We often write $Phi(y)$ as $E[X|Y=y]$. If $Y$ is a discrete random variable, then the conditional expectation calculated by $$
E[X|Y=y] = frac{E[X1_{{Y=y}}]}{P(Y=y)},tag{**}
$$ coincides with the $Phi(y)$ in the measure theoretic definition $(*)$ (You may be able to check this.) If $X,Y$ are random variables for which their joint p.d.f. exists, it can also be calculated explicitly by
$$
E[X|Y=y]=frac{int xf_{XY}(x,y)dxdy}{f_Y(y)}.tag{***}
$$ One can also prove that the one obtained from $(***)$ satisfies the definition of $(*)$. In this sense, $(*)$ is a generalization of those elementary definitions. However, if it is not the either cases, it is hard to obtain explicit formula for $Phi$ and one should rely on the (abstract) definition $(*)$.
Now, the following seemingly obvious statement about conditional expectation can be shown directly by the definition: If $E|F(X)|<infty$, then (almost surely)
$$
E[F(X)|X] = F(X).
$$Proof: Obviously, $F(X)$ is a function of $X$. By uniqueness it is sufficient to check $(*)$ holds. This is true since
$$
E[F(X) f(X)] = E[E[F(X)|X] f(X)]=E[F(X)f(X)]
$$ for all bounded $f$. $blacksquare$
This is the formalization of "Uncertainty disappears". It remains to check if $$P(T(X)in B|X) = 1_{{T(X)in B}}=begin{cases} 1quadtext{if }; T(X)in B\0quadtext{otherwise}end{cases}.$$ (Here, $1_{A}(x) = 1_{xin A}$ denotes the indicator function.) It follows from
$$
P(T(X)in B|X)=E[1_{{T(X)in B}}|X]= E[1_{{T(cdot)in B}}(X)|X]=1_{{T(cdot)in B}}(X)=1_{{T(X)in B}}.
$$ This shows $P(T(X)in B|X=x) = 1_{{T(x)in B}}$ and $T(X)|_{X=x}$ has a degenerate distribution at $T(x)$.
answered Jan 10 at 10:42
SongSong
11.4k628
$endgroup$
$begingroup$
@ Song I really appreciate your answer. As you say, I'm not so familiar with measure theory but I think I manage to understand it. Thanks.
$endgroup$
– ykkxyz
Jan 11 at 15:30
add a comment |
$begingroup$
I'm not sure that OP is familiar with the measure-theoretic definition of conditional expectation and probability, but I'll present it anyway. (This is very roughly written assuming that you have little background on measure theory.) If $X,Y$ are random variables such that $E[|X|]<infty$, then the conditional expectation of $X$ given $Y$ is defined as the unique random variable $ E[X|Y] = Phi(Y)$ (almost surely) for some (Borel-measurable) function $Phi$ such that
$$E[Xf(Y)] = E[Phi(Y)f(Y)]tag{*}
$$ holds for all bounded (Borel-measurable) function $f$.
We often write $Phi(y)$ as $E[X|Y=y]$. If $Y$ is a discrete random variable, then the conditional expectation calculated by $$
E[X|Y=y] = frac{E[X1_{{Y=y}}]}{P(Y=y)},tag{**}
$$ coincides with the $Phi(y)$ in the measure theoretic definition $(*)$ (You may be able to check this.) If $X,Y$ are random variables for which their joint p.d.f. exists, it can also be calculated explicitly by
$$
E[X|Y=y]=frac{int xf_{XY}(x,y)dxdy}{f_Y(y)}.tag{***}
$$ One can also prove that the one obtained from $(***)$ satisfies the definition of $(*)$. In this sense, $(*)$ is a generalization of those elementary definitions. However, if it is not the either cases, it is hard to obtain explicit formula for $Phi$ and one should rely on the (abstract) definition $(*)$.
Now, the following seemingly obvious statement about conditional expectation can be shown directly by the definition: If $E|F(X)|<infty$, then (almost surely)
$$
E[F(X)|X] = F(X).
$$Proof: Obviously, $F(X)$ is a function of $X$. By uniqueness it is sufficient to check $(*)$ holds. This is true since
$$
E[F(X) f(X)] = E[E[F(X)|X] f(X)]=E[F(X)f(X)]
$$ for all bounded $f$. $blacksquare$
This is the formalization of "Uncertainty disappears". It remains to check if $$P(T(X)in B|X) = 1_{{T(X)in B}}=begin{cases} 1quadtext{if }; T(X)in B\0quadtext{otherwise}end{cases}.$$ (Here, $1_{A}(x) = 1_{xin A}$ denotes the indicator function.) It follows from
$$
P(T(X)in B|X)=E[1_{{T(X)in B}}|X]= E[1_{{T(cdot)in B}}(X)|X]=1_{{T(cdot)in B}}(X)=1_{{T(X)in B}}.
$$ This shows $P(T(X)in B|X=x) = 1_{{T(x)in B}}$ and $T(X)|_{X=x}$ has a degenerate distribution at $T(x)$.
answered Jan 10 at 10:42
SongSong
11.4k628
$endgroup$
I'm not sure that OP is familiar with the measure-theoretic definition of conditional expectation and probability, but I'll present it anyway. (This is very roughly written assuming that you have little background on measure theory.) If $X,Y$ are random variables such that $E[|X|]<infty$, then the conditional expectation of $X$ given $Y$ is defined as the unique random variable $ E[X|Y] = Phi(Y)$ (almost surely) for some (Borel-measurable) function $Phi$ such that
$$E[Xf(Y)] = E[Phi(Y)f(Y)]tag{*}
$$ holds for all bounded (Borel-measurable) function $f$.
We often write $Phi(y)$ as $E[X|Y=y]$. If $Y$ is a discrete random variable, then the conditional expectation calculated by $$
E[X|Y=y] = frac{E[X1_{{Y=y}}]}{P(Y=y)},tag{**}
$$ coincides with the $Phi(y)$ in the measure theoretic definition $(*)$ (You may be able to check this.) If $X,Y$ are random variables for which their joint p.d.f. exists, it can also be calculated explicitly by
$$
E[X|Y=y]=frac{int xf_{XY}(x,y)dxdy}{f_Y(y)}.tag{***}
$$ One can also prove that the one obtained from $(***)$ satisfies the definition of $(*)$. In this sense, $(*)$ is a generalization of those elementary definitions. However, if it is not the either cases, it is hard to obtain explicit formula for $Phi$ and one should rely on the (abstract) definition $(*)$.
Now, the following seemingly obvious statement about conditional expectation can be shown directly by the definition: If $E|F(X)|<infty$, then (almost surely)
$$
E[F(X)|X] = F(X).
$$Proof: Obviously, $F(X)$ is a function of $X$. By uniqueness it is sufficient to check $(*)$ holds. This is true since
$$
E[F(X) f(X)] = E[E[F(X)|X] f(X)]=E[F(X)f(X)]
$$ for all bounded $f$. $blacksquare$
This is the formalization of "Uncertainty disappears". It remains to check if $$P(T(X)in B|X) = 1_{{T(X)in B}}=begin{cases} 1quadtext{if }; T(X)in B\0quadtext{otherwise}end{cases}.$$ (Here, $1_{A}(x) = 1_{xin A}$ denotes the indicator function.) It follows from
$$
P(T(X)in B|X)=E[1_{{T(X)in B}}|X]= E[1_{{T(cdot)in B}}(X)|X]=1_{{T(cdot)in B}}(X)=1_{{T(X)in B}}.
$$ This shows $P(T(X)in B|X=x) = 1_{{T(x)in B}}$ and $T(X)|_{X=x}$ has a degenerate distribution at $T(x)$.
answered Jan 10 at 10:42
SongSong
11.4k628
edited Jan 10 at 18:27
answered Jan 10 at 10:42
SongSong
11.4k628
answered Jan 10 at 10:42
SongSong
11.4k628
answered Jan 10 at 10:42
SongSong
11.4k628
11.4k628
$begingroup$
@ Song I really appreciate your answer. As you say, I'm not so familiar with measure theory but I think I manage to understand it. Thanks.
$endgroup$
– ykkxyz
Jan 11 at 15:30
add a comment |
$begingroup$
@ Song I really appreciate your answer. As you say, I'm not so familiar with measure theory but I think I manage to understand it. Thanks.
$endgroup$
– ykkxyz
Jan 11 at 15:30
$begingroup$
@ Song I really appreciate your answer. As you say, I'm not so familiar with measure theory but I think I manage to understand it. Thanks.
$endgroup$
– ykkxyz
Jan 11 at 15:30
$begingroup$
@ Song I really appreciate your answer. As you say, I'm not so familiar with measure theory but I think I manage to understand it. Thanks.
$endgroup$
– ykkxyz
Jan 11 at 15:30
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3067830%2fwhat-is-the-conditional-probability-density-function-of-a-statistic-given-its-sa%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
Well, since $T=t(X_1,ldots,X_n)$ is a function of the data $X=(X_1,ldots,X_n)$, conditional distribution of $T$ given $X$ is just constant $T=t(X)$. Uncertainty disappears.
$endgroup$
– Song
Jan 9 at 20:49
$begingroup$
@Song "Uncertainty disappears." Intuitively, yes. Then, how should I formulate the intuition, like when samples are discrete random variables?
$endgroup$
– ykkxyz
Jan 10 at 3:58