Mixing
Estimating posterior distribution when realization is unknown
$begingroup$
I'm working on a problem that can be reduced down the following scenario:
Consider two people, Alice and Bob. Alice has a prior probability distribution on a variable (which is known by Bob as well) while Bob knows the true probability distribution.
Alice privately draws a sample from the true (unknown, to her) distribution and updates her prior to a posterior factoring in the sampled value.
Since the sampling was private, Bob does not know what the sample was; however, Bob still wants to estimate what Alice's posterior is. What is this estimate?
My guess: It seems like Bob should consider every possible sample with its corresponding likelihood (from the true distribution) and use those to update the prior. So, in expectation, Bob's estimate of Alice's prior would just be the prior "sharpened" around the mean of the true probability distribution?
Question: How do I formally write this? Does it make sense to write the "expected" posterior given a collection of events (samples)?
probability parameter-estimation
asked Jan 11 at 23:12
jonemjonem
378415
$endgroup$
add a comment |
$begingroup$
I'm working on a problem that can be reduced down the following scenario:
Consider two people, Alice and Bob. Alice has a prior probability distribution on a variable (which is known by Bob as well) while Bob knows the true probability distribution.
Alice privately draws a sample from the true (unknown, to her) distribution and updates her prior to a posterior factoring in the sampled value.
Since the sampling was private, Bob does not know what the sample was; however, Bob still wants to estimate what Alice's posterior is. What is this estimate?
My guess: It seems like Bob should consider every possible sample with its corresponding likelihood (from the true distribution) and use those to update the prior. So, in expectation, Bob's estimate of Alice's prior would just be the prior "sharpened" around the mean of the true probability distribution?
Question: How do I formally write this? Does it make sense to write the "expected" posterior given a collection of events (samples)?
probability parameter-estimation
asked Jan 11 at 23:12
jonemjonem
378415
$endgroup$
add a comment |
$begingroup$
I'm working on a problem that can be reduced down the following scenario:
Consider two people, Alice and Bob. Alice has a prior probability distribution on a variable (which is known by Bob as well) while Bob knows the true probability distribution.
Alice privately draws a sample from the true (unknown, to her) distribution and updates her prior to a posterior factoring in the sampled value.
Since the sampling was private, Bob does not know what the sample was; however, Bob still wants to estimate what Alice's posterior is. What is this estimate?
My guess: It seems like Bob should consider every possible sample with its corresponding likelihood (from the true distribution) and use those to update the prior. So, in expectation, Bob's estimate of Alice's prior would just be the prior "sharpened" around the mean of the true probability distribution?
Question: How do I formally write this? Does it make sense to write the "expected" posterior given a collection of events (samples)?
probability parameter-estimation
asked Jan 11 at 23:12
jonemjonem
378415
$endgroup$
I'm working on a problem that can be reduced down the following scenario:
Consider two people, Alice and Bob. Alice has a prior probability distribution on a variable (which is known by Bob as well) while Bob knows the true probability distribution.
Alice privately draws a sample from the true (unknown, to her) distribution and updates her prior to a posterior factoring in the sampled value.
Since the sampling was private, Bob does not know what the sample was; however, Bob still wants to estimate what Alice's posterior is. What is this estimate?
My guess: It seems like Bob should consider every possible sample with its corresponding likelihood (from the true distribution) and use those to update the prior. So, in expectation, Bob's estimate of Alice's prior would just be the prior "sharpened" around the mean of the true probability distribution?
Question: How do I formally write this? Does it make sense to write the "expected" posterior given a collection of events (samples)?
probability parameter-estimation
probability parameter-estimation
asked Jan 11 at 23:12
jonemjonem
378415
asked Jan 11 at 23:12
jonemjonem
378415
edited Jan 12 at 18:46
jonem
asked Jan 11 at 23:12
jonemjonem
378415
asked Jan 11 at 23:12
jonemjonem
378415
asked Jan 11 at 23:12
jonemjonem
378415
378415
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
I do not think that quite works in the way you describe it. This could get quite complicated as Alice's prior could be over various possible distributions and not necessarily over a single parameter
But for a simple example let's suppose we have a Bernoulli random variable $X$ with $mathbb P(X=1)=p$ and $mathbb P(X=1)=1-p$, and Bob knows that $p=0.7$ while Alice does not and instead Alice's prior for $p$ is uniform on $[0,1]$ updated after a single observation. This diagram shows what happens to the densities:
- Alice's prior density is the horizontal black line
- With an observation of $1$ she has a posterior density like the upward sloping red line and Bob knows this has probability $0.7$
- With an observation of $0$ she has a posterior density like the downward sloping pink line and Bob knows this has probability $0.3$
- So Bob could take a weighted average of Alice's two possible posterior densities to give the blue line
I do not think Bob's weighted average of Alice's posteriors can quite be described as just being the prior "sharpened" around the mean of the true probability distribution i.e. around $0.7$; the mode of the blue density is $1$, which is some way from $0.7$. What is true that the mean of the weighted average posterior of about $0.5667$ is closer to the true value of $0.7$ than the mean of $0.5$ of the prior was, but that is a rather weaker statement and would get even more convoluted if Alice's prior was over distributions rather over a single value
answered Jan 12 at 0:19
HenryHenry
100k480165
$endgroup$
$begingroup$
Thank you. I wonder if looking at Gaussian distributions would be somewhat tractable as well since it is self-conjugate?
$endgroup$
– jonem
Jan 12 at 18:47
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3070435%2festimating-posterior-distribution-when-realization-is-unknown%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
I do not think that quite works in the way you describe it. This could get quite complicated as Alice's prior could be over various possible distributions and not necessarily over a single parameter
But for a simple example let's suppose we have a Bernoulli random variable $X$ with $mathbb P(X=1)=p$ and $mathbb P(X=1)=1-p$, and Bob knows that $p=0.7$ while Alice does not and instead Alice's prior for $p$ is uniform on $[0,1]$ updated after a single observation. This diagram shows what happens to the densities:
- Alice's prior density is the horizontal black line
- With an observation of $1$ she has a posterior density like the upward sloping red line and Bob knows this has probability $0.7$
- With an observation of $0$ she has a posterior density like the downward sloping pink line and Bob knows this has probability $0.3$
- So Bob could take a weighted average of Alice's two possible posterior densities to give the blue line
I do not think Bob's weighted average of Alice's posteriors can quite be described as just being the prior "sharpened" around the mean of the true probability distribution i.e. around $0.7$; the mode of the blue density is $1$, which is some way from $0.7$. What is true that the mean of the weighted average posterior of about $0.5667$ is closer to the true value of $0.7$ than the mean of $0.5$ of the prior was, but that is a rather weaker statement and would get even more convoluted if Alice's prior was over distributions rather over a single value
answered Jan 12 at 0:19
HenryHenry
100k480165
$endgroup$
$begingroup$
Thank you. I wonder if looking at Gaussian distributions would be somewhat tractable as well since it is self-conjugate?
$endgroup$
– jonem
Jan 12 at 18:47
add a comment |
$begingroup$
I do not think that quite works in the way you describe it. This could get quite complicated as Alice's prior could be over various possible distributions and not necessarily over a single parameter
But for a simple example let's suppose we have a Bernoulli random variable $X$ with $mathbb P(X=1)=p$ and $mathbb P(X=1)=1-p$, and Bob knows that $p=0.7$ while Alice does not and instead Alice's prior for $p$ is uniform on $[0,1]$ updated after a single observation. This diagram shows what happens to the densities:
- Alice's prior density is the horizontal black line
- With an observation of $1$ she has a posterior density like the upward sloping red line and Bob knows this has probability $0.7$
- With an observation of $0$ she has a posterior density like the downward sloping pink line and Bob knows this has probability $0.3$
- So Bob could take a weighted average of Alice's two possible posterior densities to give the blue line
I do not think Bob's weighted average of Alice's posteriors can quite be described as just being the prior "sharpened" around the mean of the true probability distribution i.e. around $0.7$; the mode of the blue density is $1$, which is some way from $0.7$. What is true that the mean of the weighted average posterior of about $0.5667$ is closer to the true value of $0.7$ than the mean of $0.5$ of the prior was, but that is a rather weaker statement and would get even more convoluted if Alice's prior was over distributions rather over a single value
answered Jan 12 at 0:19
HenryHenry
100k480165
$endgroup$
$begingroup$
Thank you. I wonder if looking at Gaussian distributions would be somewhat tractable as well since it is self-conjugate?
$endgroup$
– jonem
Jan 12 at 18:47
add a comment |
$begingroup$
I do not think that quite works in the way you describe it. This could get quite complicated as Alice's prior could be over various possible distributions and not necessarily over a single parameter
But for a simple example let's suppose we have a Bernoulli random variable $X$ with $mathbb P(X=1)=p$ and $mathbb P(X=1)=1-p$, and Bob knows that $p=0.7$ while Alice does not and instead Alice's prior for $p$ is uniform on $[0,1]$ updated after a single observation. This diagram shows what happens to the densities:
- Alice's prior density is the horizontal black line
- With an observation of $1$ she has a posterior density like the upward sloping red line and Bob knows this has probability $0.7$
- With an observation of $0$ she has a posterior density like the downward sloping pink line and Bob knows this has probability $0.3$
- So Bob could take a weighted average of Alice's two possible posterior densities to give the blue line
I do not think Bob's weighted average of Alice's posteriors can quite be described as just being the prior "sharpened" around the mean of the true probability distribution i.e. around $0.7$; the mode of the blue density is $1$, which is some way from $0.7$. What is true that the mean of the weighted average posterior of about $0.5667$ is closer to the true value of $0.7$ than the mean of $0.5$ of the prior was, but that is a rather weaker statement and would get even more convoluted if Alice's prior was over distributions rather over a single value
answered Jan 12 at 0:19
HenryHenry
100k480165
$endgroup$
I do not think that quite works in the way you describe it. This could get quite complicated as Alice's prior could be over various possible distributions and not necessarily over a single parameter
But for a simple example let's suppose we have a Bernoulli random variable $X$ with $mathbb P(X=1)=p$ and $mathbb P(X=1)=1-p$, and Bob knows that $p=0.7$ while Alice does not and instead Alice's prior for $p$ is uniform on $[0,1]$ updated after a single observation. This diagram shows what happens to the densities:
- Alice's prior density is the horizontal black line
- With an observation of $1$ she has a posterior density like the upward sloping red line and Bob knows this has probability $0.7$
- With an observation of $0$ she has a posterior density like the downward sloping pink line and Bob knows this has probability $0.3$
- So Bob could take a weighted average of Alice's two possible posterior densities to give the blue line
I do not think Bob's weighted average of Alice's posteriors can quite be described as just being the prior "sharpened" around the mean of the true probability distribution i.e. around $0.7$; the mode of the blue density is $1$, which is some way from $0.7$. What is true that the mean of the weighted average posterior of about $0.5667$ is closer to the true value of $0.7$ than the mean of $0.5$ of the prior was, but that is a rather weaker statement and would get even more convoluted if Alice's prior was over distributions rather over a single value
answered Jan 12 at 0:19
HenryHenry
100k480165
answered Jan 12 at 0:19
HenryHenry
100k480165
answered Jan 12 at 0:19
HenryHenry
100k480165
answered Jan 12 at 0:19
HenryHenry
100k480165
100k480165
$begingroup$
Thank you. I wonder if looking at Gaussian distributions would be somewhat tractable as well since it is self-conjugate?
$endgroup$
– jonem
Jan 12 at 18:47
add a comment |
$begingroup$
Thank you. I wonder if looking at Gaussian distributions would be somewhat tractable as well since it is self-conjugate?
$endgroup$
– jonem
Jan 12 at 18:47
$begingroup$
Thank you. I wonder if looking at Gaussian distributions would be somewhat tractable as well since it is self-conjugate?
$endgroup$
– jonem
Jan 12 at 18:47
$begingroup$
Thank you. I wonder if looking at Gaussian distributions would be somewhat tractable as well since it is self-conjugate?
$endgroup$
– jonem
Jan 12 at 18:47
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3070435%2festimating-posterior-distribution-when-realization-is-unknown%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
