Mixing
Prove 0<=entropy<=1 [closed]
$begingroup$
The entropy of a Bernoulli random variable X with $P(X=1)=q$ is given by
B(q)=-qlog(q)-(1-q)log(1-q)
How do we prove 0<=B(q)<=1?
inequality entropy bernoulli-numbers
asked Jan 24 at 5:37
GqqnbigGqqnbig
275212
$endgroup$
closed as off-topic by Henning Makholm, NCh, Did, Shaun, saz Jan 27 at 18:05
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Henning Makholm, NCh, Did, Shaun, saz
If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
$begingroup$
The entropy of a Bernoulli random variable X with $P(X=1)=q$ is given by
B(q)=-qlog(q)-(1-q)log(1-q)
How do we prove 0<=B(q)<=1?
inequality entropy bernoulli-numbers
asked Jan 24 at 5:37
GqqnbigGqqnbig
275212
$endgroup$
closed as off-topic by Henning Makholm, NCh, Did, Shaun, saz Jan 27 at 18:05
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Henning Makholm, NCh, Did, Shaun, saz
If this question can be reworded to fit the rules in the help center, please edit the question.
$begingroup$
One of the inequalities should be pretty immediate ...
$endgroup$
– Henning Makholm
Jan 24 at 5:46
3
$begingroup$
Have you tried directly finding the extreme values, eg by differentiating B(q) with respect to q?
$endgroup$
– Michael Hartley
Jan 24 at 5:48
add a comment |
$begingroup$
The entropy of a Bernoulli random variable X with $P(X=1)=q$ is given by
B(q)=-qlog(q)-(1-q)log(1-q)
How do we prove 0<=B(q)<=1?
inequality entropy bernoulli-numbers
asked Jan 24 at 5:37
GqqnbigGqqnbig
275212
$endgroup$
The entropy of a Bernoulli random variable X with $P(X=1)=q$ is given by
B(q)=-qlog(q)-(1-q)log(1-q)
How do we prove 0<=B(q)<=1?
inequality entropy bernoulli-numbers
inequality entropy bernoulli-numbers
asked Jan 24 at 5:37
GqqnbigGqqnbig
275212
asked Jan 24 at 5:37
GqqnbigGqqnbig
275212
asked Jan 24 at 5:37
GqqnbigGqqnbig
275212
asked Jan 24 at 5:37
GqqnbigGqqnbig
275212
asked Jan 24 at 5:37
GqqnbigGqqnbig
275212
275212
closed as off-topic by Henning Makholm, NCh, Did, Shaun, saz Jan 27 at 18:05
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Henning Makholm, NCh, Did, Shaun, saz
If this question can be reworded to fit the rules in the help center, please edit the question.
closed as off-topic by Henning Makholm, NCh, Did, Shaun, saz Jan 27 at 18:05
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Henning Makholm, NCh, Did, Shaun, saz
If this question can be reworded to fit the rules in the help center, please edit the question.
$begingroup$
One of the inequalities should be pretty immediate ...
$endgroup$
– Henning Makholm
Jan 24 at 5:46
3
$begingroup$
Have you tried directly finding the extreme values, eg by differentiating B(q) with respect to q?
$endgroup$
– Michael Hartley
Jan 24 at 5:48
add a comment |
$begingroup$
One of the inequalities should be pretty immediate ...
$endgroup$
– Henning Makholm
Jan 24 at 5:46
3
$begingroup$
Have you tried directly finding the extreme values, eg by differentiating B(q) with respect to q?
$endgroup$
– Michael Hartley
Jan 24 at 5:48
$begingroup$
One of the inequalities should be pretty immediate ...
$endgroup$
– Henning Makholm
Jan 24 at 5:46
$begingroup$
One of the inequalities should be pretty immediate ...
$endgroup$
– Henning Makholm
Jan 24 at 5:46
3
3
$begingroup$
Have you tried directly finding the extreme values, eg by differentiating B(q) with respect to q?
$endgroup$
– Michael Hartley
Jan 24 at 5:48
$begingroup$
Have you tried directly finding the extreme values, eg by differentiating B(q) with respect to q?
$endgroup$
– Michael Hartley
Jan 24 at 5:48
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
To show $B(q) ge 0$ we need to show that $qlog(q)+(1-q)log(1-q) le 0$. This is true as both $q,1-q ge 0$ and $log(q),log(1-q) le 0$.
To show $B(q) le 1$, we need to show $-qlog(q)-(1-q)log(1-q) le 1$. To do this we find the minimum of $qlog(q)+(1-q)log(1-q)$. If we can show this is greater than or equal to $-1$ we are done. Taking the derivative with respect to $q$ gives $1+log(q)-1-log(1-q) = log(q)-log(1-q) = log(frac{q}{1-q})$. This equals $0$ when $frac{q}{1-q} = 1$ so $q=frac{1}{2}$. The second derivative test will show this is the minimizer, so the minimum is $log(frac{1}{2})=-log(2) > -1$.
answered Jan 24 at 7:00
Erik ParkinsonErik Parkinson
1,17519
$endgroup$
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
To show $B(q) ge 0$ we need to show that $qlog(q)+(1-q)log(1-q) le 0$. This is true as both $q,1-q ge 0$ and $log(q),log(1-q) le 0$.
To show $B(q) le 1$, we need to show $-qlog(q)-(1-q)log(1-q) le 1$. To do this we find the minimum of $qlog(q)+(1-q)log(1-q)$. If we can show this is greater than or equal to $-1$ we are done. Taking the derivative with respect to $q$ gives $1+log(q)-1-log(1-q) = log(q)-log(1-q) = log(frac{q}{1-q})$. This equals $0$ when $frac{q}{1-q} = 1$ so $q=frac{1}{2}$. The second derivative test will show this is the minimizer, so the minimum is $log(frac{1}{2})=-log(2) > -1$.
answered Jan 24 at 7:00
Erik ParkinsonErik Parkinson
1,17519
$endgroup$
add a comment |
$begingroup$
To show $B(q) ge 0$ we need to show that $qlog(q)+(1-q)log(1-q) le 0$. This is true as both $q,1-q ge 0$ and $log(q),log(1-q) le 0$.
To show $B(q) le 1$, we need to show $-qlog(q)-(1-q)log(1-q) le 1$. To do this we find the minimum of $qlog(q)+(1-q)log(1-q)$. If we can show this is greater than or equal to $-1$ we are done. Taking the derivative with respect to $q$ gives $1+log(q)-1-log(1-q) = log(q)-log(1-q) = log(frac{q}{1-q})$. This equals $0$ when $frac{q}{1-q} = 1$ so $q=frac{1}{2}$. The second derivative test will show this is the minimizer, so the minimum is $log(frac{1}{2})=-log(2) > -1$.
answered Jan 24 at 7:00
Erik ParkinsonErik Parkinson
1,17519
$endgroup$
add a comment |
$begingroup$
To show $B(q) ge 0$ we need to show that $qlog(q)+(1-q)log(1-q) le 0$. This is true as both $q,1-q ge 0$ and $log(q),log(1-q) le 0$.
To show $B(q) le 1$, we need to show $-qlog(q)-(1-q)log(1-q) le 1$. To do this we find the minimum of $qlog(q)+(1-q)log(1-q)$. If we can show this is greater than or equal to $-1$ we are done. Taking the derivative with respect to $q$ gives $1+log(q)-1-log(1-q) = log(q)-log(1-q) = log(frac{q}{1-q})$. This equals $0$ when $frac{q}{1-q} = 1$ so $q=frac{1}{2}$. The second derivative test will show this is the minimizer, so the minimum is $log(frac{1}{2})=-log(2) > -1$.
answered Jan 24 at 7:00
Erik ParkinsonErik Parkinson
1,17519
$endgroup$
To show $B(q) ge 0$ we need to show that $qlog(q)+(1-q)log(1-q) le 0$. This is true as both $q,1-q ge 0$ and $log(q),log(1-q) le 0$.
To show $B(q) le 1$, we need to show $-qlog(q)-(1-q)log(1-q) le 1$. To do this we find the minimum of $qlog(q)+(1-q)log(1-q)$. If we can show this is greater than or equal to $-1$ we are done. Taking the derivative with respect to $q$ gives $1+log(q)-1-log(1-q) = log(q)-log(1-q) = log(frac{q}{1-q})$. This equals $0$ when $frac{q}{1-q} = 1$ so $q=frac{1}{2}$. The second derivative test will show this is the minimizer, so the minimum is $log(frac{1}{2})=-log(2) > -1$.
answered Jan 24 at 7:00
Erik ParkinsonErik Parkinson
1,17519
answered Jan 24 at 7:00
Erik ParkinsonErik Parkinson
1,17519
answered Jan 24 at 7:00
Erik ParkinsonErik Parkinson
1,17519
answered Jan 24 at 7:00
Erik ParkinsonErik Parkinson
1,17519
1,17519
add a comment |
add a comment |
$begingroup$
One of the inequalities should be pretty immediate ...
$endgroup$
– Henning Makholm
Jan 24 at 5:46
3
$begingroup$
Have you tried directly finding the extreme values, eg by differentiating B(q) with respect to q?
$endgroup$
– Michael Hartley
Jan 24 at 5:48