Mixing
Why are the non-diagonals in a Jacobian zeros?
$begingroup$
From the Matrix Calculus for Deep Learning, in the "Derivatives of vector element-wise binary operators" section, it says
Any time the general function is a vector, we know that $f_i(w)$ reduces to $f_i(w_i) = w_i$.
Why is it that $f_i(w)$ reduces to $f_i(w_i) = w_i$ for $y=w+x$ ?
Can someone show an example of why when i != j, the partial derivatives are zero?
Is this answer satisfactory? Something like this?
$frac{d}{dw_i}f_i(w_j)$ when i != j, the scalar derivative is 0
matrix-calculus jacobian
asked Jan 15 at 13:37
alvasalvas
679
$endgroup$
migrated from stats.stackexchange.com Jan 15 at 14:13
This question came from our site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.
add a comment |
$begingroup$
From the Matrix Calculus for Deep Learning, in the "Derivatives of vector element-wise binary operators" section, it says
Any time the general function is a vector, we know that $f_i(w)$ reduces to $f_i(w_i) = w_i$.
Why is it that $f_i(w)$ reduces to $f_i(w_i) = w_i$ for $y=w+x$ ?
Can someone show an example of why when i != j, the partial derivatives are zero?
Is this answer satisfactory? Something like this?
$frac{d}{dw_i}f_i(w_j)$ when i != j, the scalar derivative is 0
matrix-calculus jacobian
asked Jan 15 at 13:37
alvasalvas
679
$endgroup$
migrated from stats.stackexchange.com Jan 15 at 14:13
This question came from our site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.
$begingroup$
In the quoted passage I don't see anything about $ineq j.$ It's $w_i$ and $x_i.$ Also, the passage assumes at the start that $f(mathbf w) = mathbf w,$ so that simplifies a lot of things right away.
$endgroup$
– David K
Jan 15 at 14:18
add a comment |
$begingroup$
From the Matrix Calculus for Deep Learning, in the "Derivatives of vector element-wise binary operators" section, it says
Any time the general function is a vector, we know that $f_i(w)$ reduces to $f_i(w_i) = w_i$.
Why is it that $f_i(w)$ reduces to $f_i(w_i) = w_i$ for $y=w+x$ ?
Can someone show an example of why when i != j, the partial derivatives are zero?
Is this answer satisfactory? Something like this?
$frac{d}{dw_i}f_i(w_j)$ when i != j, the scalar derivative is 0
matrix-calculus jacobian
asked Jan 15 at 13:37
alvasalvas
679
$endgroup$
From the Matrix Calculus for Deep Learning, in the "Derivatives of vector element-wise binary operators" section, it says
Any time the general function is a vector, we know that $f_i(w)$ reduces to $f_i(w_i) = w_i$.
Why is it that $f_i(w)$ reduces to $f_i(w_i) = w_i$ for $y=w+x$ ?
Can someone show an example of why when i != j, the partial derivatives are zero?
Is this answer satisfactory? Something like this?
$frac{d}{dw_i}f_i(w_j)$ when i != j, the scalar derivative is 0
matrix-calculus jacobian
matrix-calculus jacobian
asked Jan 15 at 13:37
alvasalvas
679
asked Jan 15 at 13:37
alvasalvas
679
asked Jan 15 at 13:37
alvasalvas
679
asked Jan 15 at 13:37
alvasalvas
679
asked Jan 15 at 13:37
alvasalvas
679
679
migrated from stats.stackexchange.com Jan 15 at 14:13
This question came from our site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.
migrated from stats.stackexchange.com Jan 15 at 14:13
This question came from our site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.
$begingroup$
In the quoted passage I don't see anything about $ineq j.$ It's $w_i$ and $x_i.$ Also, the passage assumes at the start that $f(mathbf w) = mathbf w,$ so that simplifies a lot of things right away.
$endgroup$
– David K
Jan 15 at 14:18
add a comment |
$begingroup$
In the quoted passage I don't see anything about $ineq j.$ It's $w_i$ and $x_i.$ Also, the passage assumes at the start that $f(mathbf w) = mathbf w,$ so that simplifies a lot of things right away.
$endgroup$
– David K
Jan 15 at 14:18
$begingroup$
In the quoted passage I don't see anything about $ineq j.$ It's $w_i$ and $x_i.$ Also, the passage assumes at the start that $f(mathbf w) = mathbf w,$ so that simplifies a lot of things right away.
$endgroup$
– David K
Jan 15 at 14:18
$begingroup$
In the quoted passage I don't see anything about $ineq j.$ It's $w_i$ and $x_i.$ Also, the passage assumes at the start that $f(mathbf w) = mathbf w,$ so that simplifies a lot of things right away.
$endgroup$
– David K
Jan 15 at 14:18
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
The reason that the off-diagonal elements are 0 is because the derivative of a constant is 0. The key bit is that $w_i$ looks like a constant if we take the derivative with respect to $w_j$ for $j ne i$.
Here is more of the matrix calculus article I co-authored:
answered Jan 15 at 17:56
Terence ParrTerence Parr
1093
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3074478%2fwhy-are-the-non-diagonals-in-a-jacobian-zeros%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
The reason that the off-diagonal elements are 0 is because the derivative of a constant is 0. The key bit is that $w_i$ looks like a constant if we take the derivative with respect to $w_j$ for $j ne i$.
Here is more of the matrix calculus article I co-authored:
answered Jan 15 at 17:56
Terence ParrTerence Parr
1093
$endgroup$
add a comment |
$begingroup$
The reason that the off-diagonal elements are 0 is because the derivative of a constant is 0. The key bit is that $w_i$ looks like a constant if we take the derivative with respect to $w_j$ for $j ne i$.
Here is more of the matrix calculus article I co-authored:
answered Jan 15 at 17:56
Terence ParrTerence Parr
1093
$endgroup$
add a comment |
$begingroup$
The reason that the off-diagonal elements are 0 is because the derivative of a constant is 0. The key bit is that $w_i$ looks like a constant if we take the derivative with respect to $w_j$ for $j ne i$.
Here is more of the matrix calculus article I co-authored:
answered Jan 15 at 17:56
Terence ParrTerence Parr
1093
$endgroup$
The reason that the off-diagonal elements are 0 is because the derivative of a constant is 0. The key bit is that $w_i$ looks like a constant if we take the derivative with respect to $w_j$ for $j ne i$.
Here is more of the matrix calculus article I co-authored:
answered Jan 15 at 17:56
Terence ParrTerence Parr
1093
answered Jan 15 at 17:56
Terence ParrTerence Parr
1093
answered Jan 15 at 17:56
Terence ParrTerence Parr
1093
answered Jan 15 at 17:56
Terence ParrTerence Parr
1093
1093
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3074478%2fwhy-are-the-non-diagonals-in-a-jacobian-zeros%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
In the quoted passage I don't see anything about $ineq j.$ It's $w_i$ and $x_i.$ Also, the passage assumes at the start that $f(mathbf w) = mathbf w,$ so that simplifies a lot of things right away.
$endgroup$
– David K
Jan 15 at 14:18