Mixing
Hessian to show convexity - check my approach please
$begingroup$
I need to check the convexity of $f(x)$ for these two questions, using the Hessian matrix. I am aware the function can be said to be convex if over the domain of $f$ the hessian is defined and is positive semidefinite.
First question: $$f(x) = begin{cases}
x_1x_2 & xin mathbb{R}^n_+ \
+infty & otherwise
end{cases}
$$
then the hessian:
$$H = begin{pmatrix} 0 & 1 \ 1 & 0 end{pmatrix} $$
So this has eigenvalues $$lambda_1 = lambda_2 = 0 $$
Does this mean the matrix can be called positive-semidefinite? Can we conclude $f(x)$ is convex?
Second question:
$$f(x) = begin{cases}
frac{x_1^2}{x_2} & xin mathbb{R} times mathbb{R}_{++} \
+infty & otherwise
end{cases}$$
$$H = begin{pmatrix} frac{2}{x_2} & frac{-2x_1}{x_2^2} \ frac{-2x_1}{x_2^2} & frac{2x_1^2}{x_2^3} end{pmatrix} $$
since $lambda_1, lambda_2 >0$ can we conclude $f(x)$ again is convex? I have read my notes and textbooks carefully, but am unsure what specifically is meant by a positive definite matrix, and in regards to question 1, if the eigenvalues are both 0, does this satisfy being positive definite and result in convexity?
Have I approached these questions correctly?
matrices convex-analysis partial-derivative
asked Jan 18 '15 at 21:57
diabloescobardiabloescobar
704
$endgroup$
|
show 1 more comment
$begingroup$
I need to check the convexity of $f(x)$ for these two questions, using the Hessian matrix. I am aware the function can be said to be convex if over the domain of $f$ the hessian is defined and is positive semidefinite.
First question: $$f(x) = begin{cases}
x_1x_2 & xin mathbb{R}^n_+ \
+infty & otherwise
end{cases}
$$
then the hessian:
$$H = begin{pmatrix} 0 & 1 \ 1 & 0 end{pmatrix} $$
So this has eigenvalues $$lambda_1 = lambda_2 = 0 $$
Does this mean the matrix can be called positive-semidefinite? Can we conclude $f(x)$ is convex?
Second question:
$$f(x) = begin{cases}
frac{x_1^2}{x_2} & xin mathbb{R} times mathbb{R}_{++} \
+infty & otherwise
end{cases}$$
$$H = begin{pmatrix} frac{2}{x_2} & frac{-2x_1}{x_2^2} \ frac{-2x_1}{x_2^2} & frac{2x_1^2}{x_2^3} end{pmatrix} $$
since $lambda_1, lambda_2 >0$ can we conclude $f(x)$ again is convex? I have read my notes and textbooks carefully, but am unsure what specifically is meant by a positive definite matrix, and in regards to question 1, if the eigenvalues are both 0, does this satisfy being positive definite and result in convexity?
Have I approached these questions correctly?
matrices convex-analysis partial-derivative
asked Jan 18 '15 at 21:57
diabloescobardiabloescobar
704
$endgroup$
$begingroup$
yes - I have fixed it now, thanks
$endgroup$
– diabloescobar
Jan 18 '15 at 23:33
$begingroup$
You are incorrect about the eigenvalues of $H$ for the first function, which is not convex. Sanity check: the only symmetric matrix with all zero eigenvalues is the all-zeros matrix.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:16
$begingroup$
As for the second function, you have not established what the eigenvalues are. You claim they are positive, but given the mistake in the first instance, skepticism about your second claim is warranted. The truth is that $H$ is positive definite over the domain of the function, but this is best proven not with eigenvalues but by examining the matrices' minors, and proving those are positive.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:20
$begingroup$
So its always better to check if the minors are positive than if the eigenvalues are positive to check for positive semi-definiteness ?
$endgroup$
– diabloescobar
Jan 19 '15 at 12:44
$begingroup$
It's better to do what is easier. I'd say usually that will be to check the principal minors.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:31
|
show 1 more comment
$begingroup$
I need to check the convexity of $f(x)$ for these two questions, using the Hessian matrix. I am aware the function can be said to be convex if over the domain of $f$ the hessian is defined and is positive semidefinite.
First question: $$f(x) = begin{cases}
x_1x_2 & xin mathbb{R}^n_+ \
+infty & otherwise
end{cases}
$$
then the hessian:
$$H = begin{pmatrix} 0 & 1 \ 1 & 0 end{pmatrix} $$
So this has eigenvalues $$lambda_1 = lambda_2 = 0 $$
Does this mean the matrix can be called positive-semidefinite? Can we conclude $f(x)$ is convex?
Second question:
$$f(x) = begin{cases}
frac{x_1^2}{x_2} & xin mathbb{R} times mathbb{R}_{++} \
+infty & otherwise
end{cases}$$
$$H = begin{pmatrix} frac{2}{x_2} & frac{-2x_1}{x_2^2} \ frac{-2x_1}{x_2^2} & frac{2x_1^2}{x_2^3} end{pmatrix} $$
since $lambda_1, lambda_2 >0$ can we conclude $f(x)$ again is convex? I have read my notes and textbooks carefully, but am unsure what specifically is meant by a positive definite matrix, and in regards to question 1, if the eigenvalues are both 0, does this satisfy being positive definite and result in convexity?
Have I approached these questions correctly?
matrices convex-analysis partial-derivative
asked Jan 18 '15 at 21:57
diabloescobardiabloescobar
704
$endgroup$
I need to check the convexity of $f(x)$ for these two questions, using the Hessian matrix. I am aware the function can be said to be convex if over the domain of $f$ the hessian is defined and is positive semidefinite.
First question: $$f(x) = begin{cases}
x_1x_2 & xin mathbb{R}^n_+ \
+infty & otherwise
end{cases}
$$
then the hessian:
$$H = begin{pmatrix} 0 & 1 \ 1 & 0 end{pmatrix} $$
So this has eigenvalues $$lambda_1 = lambda_2 = 0 $$
Does this mean the matrix can be called positive-semidefinite? Can we conclude $f(x)$ is convex?
Second question:
$$f(x) = begin{cases}
frac{x_1^2}{x_2} & xin mathbb{R} times mathbb{R}_{++} \
+infty & otherwise
end{cases}$$
$$H = begin{pmatrix} frac{2}{x_2} & frac{-2x_1}{x_2^2} \ frac{-2x_1}{x_2^2} & frac{2x_1^2}{x_2^3} end{pmatrix} $$
since $lambda_1, lambda_2 >0$ can we conclude $f(x)$ again is convex? I have read my notes and textbooks carefully, but am unsure what specifically is meant by a positive definite matrix, and in regards to question 1, if the eigenvalues are both 0, does this satisfy being positive definite and result in convexity?
Have I approached these questions correctly?
matrices convex-analysis partial-derivative
matrices convex-analysis partial-derivative
asked Jan 18 '15 at 21:57
diabloescobardiabloescobar
704
asked Jan 18 '15 at 21:57
diabloescobardiabloescobar
704
edited Jan 18 '15 at 23:33
diabloescobar
asked Jan 18 '15 at 21:57
diabloescobardiabloescobar
704
asked Jan 18 '15 at 21:57
diabloescobardiabloescobar
704
asked Jan 18 '15 at 21:57
diabloescobardiabloescobar
704
704
$begingroup$
yes - I have fixed it now, thanks
$endgroup$
– diabloescobar
Jan 18 '15 at 23:33
$begingroup$
You are incorrect about the eigenvalues of $H$ for the first function, which is not convex. Sanity check: the only symmetric matrix with all zero eigenvalues is the all-zeros matrix.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:16
$begingroup$
As for the second function, you have not established what the eigenvalues are. You claim they are positive, but given the mistake in the first instance, skepticism about your second claim is warranted. The truth is that $H$ is positive definite over the domain of the function, but this is best proven not with eigenvalues but by examining the matrices' minors, and proving those are positive.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:20
$begingroup$
So its always better to check if the minors are positive than if the eigenvalues are positive to check for positive semi-definiteness ?
$endgroup$
– diabloescobar
Jan 19 '15 at 12:44
$begingroup$
It's better to do what is easier. I'd say usually that will be to check the principal minors.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:31
|
show 1 more comment
$begingroup$
yes - I have fixed it now, thanks
$endgroup$
– diabloescobar
Jan 18 '15 at 23:33
$begingroup$
You are incorrect about the eigenvalues of $H$ for the first function, which is not convex. Sanity check: the only symmetric matrix with all zero eigenvalues is the all-zeros matrix.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:16
$begingroup$
As for the second function, you have not established what the eigenvalues are. You claim they are positive, but given the mistake in the first instance, skepticism about your second claim is warranted. The truth is that $H$ is positive definite over the domain of the function, but this is best proven not with eigenvalues but by examining the matrices' minors, and proving those are positive.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:20
$begingroup$
So its always better to check if the minors are positive than if the eigenvalues are positive to check for positive semi-definiteness ?
$endgroup$
– diabloescobar
Jan 19 '15 at 12:44
$begingroup$
It's better to do what is easier. I'd say usually that will be to check the principal minors.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:31
$begingroup$
yes - I have fixed it now, thanks
$endgroup$
– diabloescobar
Jan 18 '15 at 23:33
$begingroup$
yes - I have fixed it now, thanks
$endgroup$
– diabloescobar
Jan 18 '15 at 23:33
$begingroup$
You are incorrect about the eigenvalues of $H$ for the first function, which is not convex. Sanity check: the only symmetric matrix with all zero eigenvalues is the all-zeros matrix.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:16
$begingroup$
You are incorrect about the eigenvalues of $H$ for the first function, which is not convex. Sanity check: the only symmetric matrix with all zero eigenvalues is the all-zeros matrix.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:16
$begingroup$
As for the second function, you have not established what the eigenvalues are. You claim they are positive, but given the mistake in the first instance, skepticism about your second claim is warranted. The truth is that $H$ is positive definite over the domain of the function, but this is best proven not with eigenvalues but by examining the matrices' minors, and proving those are positive.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:20
$begingroup$
As for the second function, you have not established what the eigenvalues are. You claim they are positive, but given the mistake in the first instance, skepticism about your second claim is warranted. The truth is that $H$ is positive definite over the domain of the function, but this is best proven not with eigenvalues but by examining the matrices' minors, and proving those are positive.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:20
$begingroup$
So its always better to check if the minors are positive than if the eigenvalues are positive to check for positive semi-definiteness ?
$endgroup$
– diabloescobar
Jan 19 '15 at 12:44
$begingroup$
So its always better to check if the minors are positive than if the eigenvalues are positive to check for positive semi-definiteness ?
$endgroup$
– diabloescobar
Jan 19 '15 at 12:44
$begingroup$
It's better to do what is easier. I'd say usually that will be to check the principal minors.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:31
$begingroup$
It's better to do what is easier. I'd say usually that will be to check the principal minors.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:31
|
show 1 more comment
1 Answer
1
active
oldest
votes
$begingroup$
This answer addresses the first question.
A real-valued, symmetric matrix is positive-semidefinite if and only if all its eigenvalues are nonnegative -- $0$ counts, no matter how many times it occurs.
Also, positive-semidefiniteness everywhere of the Hessian is equivalent to convexity; see this Math.SE question. Hessian matrices of real-valued $C^2$ functions are symmetric. So $0$ eigenvalues do count in this equivalence too; if your computation was correct, then the first function would be convex.
However, your first eigenvalue computation is incorrect. Indeed, we see that
$$H begin{pmatrix} 1 \ -1 end{pmatrix} = begin{pmatrix} -1 \ 1 end{pmatrix} = (-1) begin{pmatrix} 1 \ -1 end{pmatrix},$$
so there is an eigenvalue of $-1$. Also, there is an eigenvalue of $1$, since
$$H begin{pmatrix} 1 \ 1 end{pmatrix} = begin{pmatrix} 1 \ 1 end{pmatrix}.$$
Since the number of eigenvalues is less than or equal to the dimension, we have eigenvalues of $pm 1$.
[How did I magically create these eigenvectors? Well, I saw that $H^2 = I$, so I figured the constants were nice, and since the matrix looks like a reflection over the axis $x_1 = x_2$ (i.e., changing the roles of $x_1$ and $x_2$), I figured the lines $x_2 = x_1$ and $x_2 = -x_1$ would be relevant.]
Regarding the "usual" method of characteristic polynomial for finding the eigenvalues, I get the equation for $det (lambda I - H ) = 0$ of
$$lambda^2 - 1 = 0.$$
Did you make a computational error here?
In any event, with the correct computation, the first function is not convex. [Geometrically, if you are familiar with convexity of one-dimensional functions, if you let $x_2 = -x_1$, you get $g(x_1) overbrace{=}^{defined} f(x_1, -x_1) = - x_1^2$, which shows a ``concave down'' effect].
P.S. Since for diagonalizable matrices (including real symmetric matrices), the determinant of the matrix is the product of the eigenvalues, and the determinant of your second matrix is 0 (I think), you should have at least one zero eigenvalue. You should double-check that the other eigenvalue is positive.
edited Apr 13 '17 at 12:20
Community♦
1
answered Jan 19 '15 at 0:32
Charles BakerCharles Baker
749615
$endgroup$
$begingroup$
thank you for your response. I am confused as to why my hessian was incorrect, after all is it not just $$f_{x_1x_1}, f_{x_1x_2}$$ $$f_{x_2x_1}, f_{x_2x_2}$$ Where $f_{x_1x_1}$ would obviously yield $(x_2)' = 0$ ? What have I done wrong, is my understanding of the definition of hessian incorrect? Thanks for your response.
$endgroup$
– diabloescobar
Jan 19 '15 at 0:58
1
$begingroup$
@diabloescobar: your Hessian is fine, at least for the first function. Your eigenvalue computations are not.
$endgroup$
– Michael Grant
Jan 19 '15 at 1:43
$begingroup$
@MichaelGrant Thank you, I am not a smart man -_-
$endgroup$
– diabloescobar
Jan 19 '15 at 12:43
$begingroup$
either could I also enquire, what is meant by the $mathbb{R} times mathbb{R}_{++}$ My error previously was that I forgot you had to take the determinant to find the eigenvalues how silly.
$endgroup$
– diabloescobar
Jan 19 '15 at 13:25
$begingroup$
What that means is that the function is defined for any real value of the first argument, but only for positive values of the second argument.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:32
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1109682%2fhessian-to-show-convexity-check-my-approach-please%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
This answer addresses the first question.
A real-valued, symmetric matrix is positive-semidefinite if and only if all its eigenvalues are nonnegative -- $0$ counts, no matter how many times it occurs.
Also, positive-semidefiniteness everywhere of the Hessian is equivalent to convexity; see this Math.SE question. Hessian matrices of real-valued $C^2$ functions are symmetric. So $0$ eigenvalues do count in this equivalence too; if your computation was correct, then the first function would be convex.
However, your first eigenvalue computation is incorrect. Indeed, we see that
$$H begin{pmatrix} 1 \ -1 end{pmatrix} = begin{pmatrix} -1 \ 1 end{pmatrix} = (-1) begin{pmatrix} 1 \ -1 end{pmatrix},$$
so there is an eigenvalue of $-1$. Also, there is an eigenvalue of $1$, since
$$H begin{pmatrix} 1 \ 1 end{pmatrix} = begin{pmatrix} 1 \ 1 end{pmatrix}.$$
Since the number of eigenvalues is less than or equal to the dimension, we have eigenvalues of $pm 1$.
[How did I magically create these eigenvectors? Well, I saw that $H^2 = I$, so I figured the constants were nice, and since the matrix looks like a reflection over the axis $x_1 = x_2$ (i.e., changing the roles of $x_1$ and $x_2$), I figured the lines $x_2 = x_1$ and $x_2 = -x_1$ would be relevant.]
Regarding the "usual" method of characteristic polynomial for finding the eigenvalues, I get the equation for $det (lambda I - H ) = 0$ of
$$lambda^2 - 1 = 0.$$
Did you make a computational error here?
In any event, with the correct computation, the first function is not convex. [Geometrically, if you are familiar with convexity of one-dimensional functions, if you let $x_2 = -x_1$, you get $g(x_1) overbrace{=}^{defined} f(x_1, -x_1) = - x_1^2$, which shows a ``concave down'' effect].
P.S. Since for diagonalizable matrices (including real symmetric matrices), the determinant of the matrix is the product of the eigenvalues, and the determinant of your second matrix is 0 (I think), you should have at least one zero eigenvalue. You should double-check that the other eigenvalue is positive.
edited Apr 13 '17 at 12:20
Community♦
1
answered Jan 19 '15 at 0:32
Charles BakerCharles Baker
749615
$endgroup$
$begingroup$
thank you for your response. I am confused as to why my hessian was incorrect, after all is it not just $$f_{x_1x_1}, f_{x_1x_2}$$ $$f_{x_2x_1}, f_{x_2x_2}$$ Where $f_{x_1x_1}$ would obviously yield $(x_2)' = 0$ ? What have I done wrong, is my understanding of the definition of hessian incorrect? Thanks for your response.
$endgroup$
– diabloescobar
Jan 19 '15 at 0:58
1
$begingroup$
@diabloescobar: your Hessian is fine, at least for the first function. Your eigenvalue computations are not.
$endgroup$
– Michael Grant
Jan 19 '15 at 1:43
$begingroup$
@MichaelGrant Thank you, I am not a smart man -_-
$endgroup$
– diabloescobar
Jan 19 '15 at 12:43
$begingroup$
either could I also enquire, what is meant by the $mathbb{R} times mathbb{R}_{++}$ My error previously was that I forgot you had to take the determinant to find the eigenvalues how silly.
$endgroup$
– diabloescobar
Jan 19 '15 at 13:25
$begingroup$
What that means is that the function is defined for any real value of the first argument, but only for positive values of the second argument.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:32
add a comment |
$begingroup$
This answer addresses the first question.
A real-valued, symmetric matrix is positive-semidefinite if and only if all its eigenvalues are nonnegative -- $0$ counts, no matter how many times it occurs.
Also, positive-semidefiniteness everywhere of the Hessian is equivalent to convexity; see this Math.SE question. Hessian matrices of real-valued $C^2$ functions are symmetric. So $0$ eigenvalues do count in this equivalence too; if your computation was correct, then the first function would be convex.
However, your first eigenvalue computation is incorrect. Indeed, we see that
$$H begin{pmatrix} 1 \ -1 end{pmatrix} = begin{pmatrix} -1 \ 1 end{pmatrix} = (-1) begin{pmatrix} 1 \ -1 end{pmatrix},$$
so there is an eigenvalue of $-1$. Also, there is an eigenvalue of $1$, since
$$H begin{pmatrix} 1 \ 1 end{pmatrix} = begin{pmatrix} 1 \ 1 end{pmatrix}.$$
Since the number of eigenvalues is less than or equal to the dimension, we have eigenvalues of $pm 1$.
[How did I magically create these eigenvectors? Well, I saw that $H^2 = I$, so I figured the constants were nice, and since the matrix looks like a reflection over the axis $x_1 = x_2$ (i.e., changing the roles of $x_1$ and $x_2$), I figured the lines $x_2 = x_1$ and $x_2 = -x_1$ would be relevant.]
Regarding the "usual" method of characteristic polynomial for finding the eigenvalues, I get the equation for $det (lambda I - H ) = 0$ of
$$lambda^2 - 1 = 0.$$
Did you make a computational error here?
In any event, with the correct computation, the first function is not convex. [Geometrically, if you are familiar with convexity of one-dimensional functions, if you let $x_2 = -x_1$, you get $g(x_1) overbrace{=}^{defined} f(x_1, -x_1) = - x_1^2$, which shows a ``concave down'' effect].
P.S. Since for diagonalizable matrices (including real symmetric matrices), the determinant of the matrix is the product of the eigenvalues, and the determinant of your second matrix is 0 (I think), you should have at least one zero eigenvalue. You should double-check that the other eigenvalue is positive.
edited Apr 13 '17 at 12:20
Community♦
1
answered Jan 19 '15 at 0:32
Charles BakerCharles Baker
749615
$endgroup$
$begingroup$
thank you for your response. I am confused as to why my hessian was incorrect, after all is it not just $$f_{x_1x_1}, f_{x_1x_2}$$ $$f_{x_2x_1}, f_{x_2x_2}$$ Where $f_{x_1x_1}$ would obviously yield $(x_2)' = 0$ ? What have I done wrong, is my understanding of the definition of hessian incorrect? Thanks for your response.
$endgroup$
– diabloescobar
Jan 19 '15 at 0:58
1
$begingroup$
@diabloescobar: your Hessian is fine, at least for the first function. Your eigenvalue computations are not.
$endgroup$
– Michael Grant
Jan 19 '15 at 1:43
$begingroup$
@MichaelGrant Thank you, I am not a smart man -_-
$endgroup$
– diabloescobar
Jan 19 '15 at 12:43
$begingroup$
either could I also enquire, what is meant by the $mathbb{R} times mathbb{R}_{++}$ My error previously was that I forgot you had to take the determinant to find the eigenvalues how silly.
$endgroup$
– diabloescobar
Jan 19 '15 at 13:25
$begingroup$
What that means is that the function is defined for any real value of the first argument, but only for positive values of the second argument.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:32
add a comment |
$begingroup$
This answer addresses the first question.
A real-valued, symmetric matrix is positive-semidefinite if and only if all its eigenvalues are nonnegative -- $0$ counts, no matter how many times it occurs.
Also, positive-semidefiniteness everywhere of the Hessian is equivalent to convexity; see this Math.SE question. Hessian matrices of real-valued $C^2$ functions are symmetric. So $0$ eigenvalues do count in this equivalence too; if your computation was correct, then the first function would be convex.
However, your first eigenvalue computation is incorrect. Indeed, we see that
$$H begin{pmatrix} 1 \ -1 end{pmatrix} = begin{pmatrix} -1 \ 1 end{pmatrix} = (-1) begin{pmatrix} 1 \ -1 end{pmatrix},$$
so there is an eigenvalue of $-1$. Also, there is an eigenvalue of $1$, since
$$H begin{pmatrix} 1 \ 1 end{pmatrix} = begin{pmatrix} 1 \ 1 end{pmatrix}.$$
Since the number of eigenvalues is less than or equal to the dimension, we have eigenvalues of $pm 1$.
[How did I magically create these eigenvectors? Well, I saw that $H^2 = I$, so I figured the constants were nice, and since the matrix looks like a reflection over the axis $x_1 = x_2$ (i.e., changing the roles of $x_1$ and $x_2$), I figured the lines $x_2 = x_1$ and $x_2 = -x_1$ would be relevant.]
Regarding the "usual" method of characteristic polynomial for finding the eigenvalues, I get the equation for $det (lambda I - H ) = 0$ of
$$lambda^2 - 1 = 0.$$
Did you make a computational error here?
In any event, with the correct computation, the first function is not convex. [Geometrically, if you are familiar with convexity of one-dimensional functions, if you let $x_2 = -x_1$, you get $g(x_1) overbrace{=}^{defined} f(x_1, -x_1) = - x_1^2$, which shows a ``concave down'' effect].
P.S. Since for diagonalizable matrices (including real symmetric matrices), the determinant of the matrix is the product of the eigenvalues, and the determinant of your second matrix is 0 (I think), you should have at least one zero eigenvalue. You should double-check that the other eigenvalue is positive.
edited Apr 13 '17 at 12:20
Community♦
1
answered Jan 19 '15 at 0:32
Charles BakerCharles Baker
749615
$endgroup$
This answer addresses the first question.
A real-valued, symmetric matrix is positive-semidefinite if and only if all its eigenvalues are nonnegative -- $0$ counts, no matter how many times it occurs.
Also, positive-semidefiniteness everywhere of the Hessian is equivalent to convexity; see this Math.SE question. Hessian matrices of real-valued $C^2$ functions are symmetric. So $0$ eigenvalues do count in this equivalence too; if your computation was correct, then the first function would be convex.
However, your first eigenvalue computation is incorrect. Indeed, we see that
$$H begin{pmatrix} 1 \ -1 end{pmatrix} = begin{pmatrix} -1 \ 1 end{pmatrix} = (-1) begin{pmatrix} 1 \ -1 end{pmatrix},$$
so there is an eigenvalue of $-1$. Also, there is an eigenvalue of $1$, since
$$H begin{pmatrix} 1 \ 1 end{pmatrix} = begin{pmatrix} 1 \ 1 end{pmatrix}.$$
Since the number of eigenvalues is less than or equal to the dimension, we have eigenvalues of $pm 1$.
[How did I magically create these eigenvectors? Well, I saw that $H^2 = I$, so I figured the constants were nice, and since the matrix looks like a reflection over the axis $x_1 = x_2$ (i.e., changing the roles of $x_1$ and $x_2$), I figured the lines $x_2 = x_1$ and $x_2 = -x_1$ would be relevant.]
Regarding the "usual" method of characteristic polynomial for finding the eigenvalues, I get the equation for $det (lambda I - H ) = 0$ of
$$lambda^2 - 1 = 0.$$
Did you make a computational error here?
In any event, with the correct computation, the first function is not convex. [Geometrically, if you are familiar with convexity of one-dimensional functions, if you let $x_2 = -x_1$, you get $g(x_1) overbrace{=}^{defined} f(x_1, -x_1) = - x_1^2$, which shows a ``concave down'' effect].
P.S. Since for diagonalizable matrices (including real symmetric matrices), the determinant of the matrix is the product of the eigenvalues, and the determinant of your second matrix is 0 (I think), you should have at least one zero eigenvalue. You should double-check that the other eigenvalue is positive.
edited Apr 13 '17 at 12:20
Community♦
1
answered Jan 19 '15 at 0:32
Charles BakerCharles Baker
749615
edited Apr 13 '17 at 12:20
Community♦
1
edited Apr 13 '17 at 12:20
Community♦
1
edited Apr 13 '17 at 12:20
Community♦
1
1
answered Jan 19 '15 at 0:32
Charles BakerCharles Baker
749615
answered Jan 19 '15 at 0:32
Charles BakerCharles Baker
749615
answered Jan 19 '15 at 0:32
Charles BakerCharles Baker
749615
749615
$begingroup$
thank you for your response. I am confused as to why my hessian was incorrect, after all is it not just $$f_{x_1x_1}, f_{x_1x_2}$$ $$f_{x_2x_1}, f_{x_2x_2}$$ Where $f_{x_1x_1}$ would obviously yield $(x_2)' = 0$ ? What have I done wrong, is my understanding of the definition of hessian incorrect? Thanks for your response.
$endgroup$
– diabloescobar
Jan 19 '15 at 0:58
1
$begingroup$
@diabloescobar: your Hessian is fine, at least for the first function. Your eigenvalue computations are not.
$endgroup$
– Michael Grant
Jan 19 '15 at 1:43
$begingroup$
@MichaelGrant Thank you, I am not a smart man -_-
$endgroup$
– diabloescobar
Jan 19 '15 at 12:43
$begingroup$
either could I also enquire, what is meant by the $mathbb{R} times mathbb{R}_{++}$ My error previously was that I forgot you had to take the determinant to find the eigenvalues how silly.
$endgroup$
– diabloescobar
Jan 19 '15 at 13:25
$begingroup$
What that means is that the function is defined for any real value of the first argument, but only for positive values of the second argument.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:32
add a comment |
$begingroup$
thank you for your response. I am confused as to why my hessian was incorrect, after all is it not just $$f_{x_1x_1}, f_{x_1x_2}$$ $$f_{x_2x_1}, f_{x_2x_2}$$ Where $f_{x_1x_1}$ would obviously yield $(x_2)' = 0$ ? What have I done wrong, is my understanding of the definition of hessian incorrect? Thanks for your response.
$endgroup$
– diabloescobar
Jan 19 '15 at 0:58
1
$begingroup$
@diabloescobar: your Hessian is fine, at least for the first function. Your eigenvalue computations are not.
$endgroup$
– Michael Grant
Jan 19 '15 at 1:43
$begingroup$
@MichaelGrant Thank you, I am not a smart man -_-
$endgroup$
– diabloescobar
Jan 19 '15 at 12:43
$begingroup$
either could I also enquire, what is meant by the $mathbb{R} times mathbb{R}_{++}$ My error previously was that I forgot you had to take the determinant to find the eigenvalues how silly.
$endgroup$
– diabloescobar
Jan 19 '15 at 13:25
$begingroup$
What that means is that the function is defined for any real value of the first argument, but only for positive values of the second argument.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:32
$begingroup$
thank you for your response. I am confused as to why my hessian was incorrect, after all is it not just $$f_{x_1x_1}, f_{x_1x_2}$$ $$f_{x_2x_1}, f_{x_2x_2}$$ Where $f_{x_1x_1}$ would obviously yield $(x_2)' = 0$ ? What have I done wrong, is my understanding of the definition of hessian incorrect? Thanks for your response.
$endgroup$
– diabloescobar
Jan 19 '15 at 0:58
$begingroup$
thank you for your response. I am confused as to why my hessian was incorrect, after all is it not just $$f_{x_1x_1}, f_{x_1x_2}$$ $$f_{x_2x_1}, f_{x_2x_2}$$ Where $f_{x_1x_1}$ would obviously yield $(x_2)' = 0$ ? What have I done wrong, is my understanding of the definition of hessian incorrect? Thanks for your response.
$endgroup$
– diabloescobar
Jan 19 '15 at 0:58
1
1
$begingroup$
@diabloescobar: your Hessian is fine, at least for the first function. Your eigenvalue computations are not.
$endgroup$
– Michael Grant
Jan 19 '15 at 1:43
$begingroup$
@diabloescobar: your Hessian is fine, at least for the first function. Your eigenvalue computations are not.
$endgroup$
– Michael Grant
Jan 19 '15 at 1:43
$begingroup$
@MichaelGrant Thank you, I am not a smart man -_-
$endgroup$
– diabloescobar
Jan 19 '15 at 12:43
$begingroup$
@MichaelGrant Thank you, I am not a smart man -_-
$endgroup$
– diabloescobar
Jan 19 '15 at 12:43
$begingroup$
either could I also enquire, what is meant by the $mathbb{R} times mathbb{R}_{++}$ My error previously was that I forgot you had to take the determinant to find the eigenvalues how silly.
$endgroup$
– diabloescobar
Jan 19 '15 at 13:25
$begingroup$
either could I also enquire, what is meant by the $mathbb{R} times mathbb{R}_{++}$ My error previously was that I forgot you had to take the determinant to find the eigenvalues how silly.
$endgroup$
– diabloescobar
Jan 19 '15 at 13:25
$begingroup$
What that means is that the function is defined for any real value of the first argument, but only for positive values of the second argument.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:32
$begingroup$
What that means is that the function is defined for any real value of the first argument, but only for positive values of the second argument.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:32
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1109682%2fhessian-to-show-convexity-check-my-approach-please%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
yes - I have fixed it now, thanks
$endgroup$
– diabloescobar
Jan 18 '15 at 23:33
$begingroup$
You are incorrect about the eigenvalues of $H$ for the first function, which is not convex. Sanity check: the only symmetric matrix with all zero eigenvalues is the all-zeros matrix.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:16
$begingroup$
As for the second function, you have not established what the eigenvalues are. You claim they are positive, but given the mistake in the first instance, skepticism about your second claim is warranted. The truth is that $H$ is positive definite over the domain of the function, but this is best proven not with eigenvalues but by examining the matrices' minors, and proving those are positive.
$endgroup$
– Michael Grant
Jan 19 '15 at 0:20
$begingroup$
So its always better to check if the minors are positive than if the eigenvalues are positive to check for positive semi-definiteness ?
$endgroup$
– diabloescobar
Jan 19 '15 at 12:44
$begingroup$
It's better to do what is easier. I'd say usually that will be to check the principal minors.
$endgroup$
– Michael Grant
Jan 19 '15 at 15:31