Mixing
Determine stationary distribution of Markov Chain
$begingroup$
I am struggling with finding the stationary distribution(s) for a discrete Markov chain with the following transition probability matrix
begin{bmatrix}1/3&2/3&0&0&0\
1/2&1/2&0&0&0\
0&1/2&0&1/2&0\
0&0&0&1/4&3/4\
0&0&0&1/3&2/3end{bmatrix}
Since this matrix is singular, it is not possible to determine $(I-P)^{-1}$. This I already tried, I also suspect that their might be more than one stationary distribution.
does anybody have an idea?
markov-chains
asked Jan 25 at 8:46
Jonathan KierschJonathan Kiersch
1139
$endgroup$
add a comment |
$begingroup$
I am struggling with finding the stationary distribution(s) for a discrete Markov chain with the following transition probability matrix
begin{bmatrix}1/3&2/3&0&0&0\
1/2&1/2&0&0&0\
0&1/2&0&1/2&0\
0&0&0&1/4&3/4\
0&0&0&1/3&2/3end{bmatrix}
Since this matrix is singular, it is not possible to determine $(I-P)^{-1}$. This I already tried, I also suspect that their might be more than one stationary distribution.
does anybody have an idea?
markov-chains
asked Jan 25 at 8:46
Jonathan KierschJonathan Kiersch
1139
$endgroup$
$begingroup$
The third position clearly is not part of any stationary distribution. Using the top left you can find a stationary distribution for the first two positions; using the bottom right you can find a stationary distribution for the last two positions. Any overall stationary distribution will be a convex combination of these two
$endgroup$
– Henry
Jan 25 at 8:54
add a comment |
$begingroup$
I am struggling with finding the stationary distribution(s) for a discrete Markov chain with the following transition probability matrix
begin{bmatrix}1/3&2/3&0&0&0\
1/2&1/2&0&0&0\
0&1/2&0&1/2&0\
0&0&0&1/4&3/4\
0&0&0&1/3&2/3end{bmatrix}
Since this matrix is singular, it is not possible to determine $(I-P)^{-1}$. This I already tried, I also suspect that their might be more than one stationary distribution.
does anybody have an idea?
markov-chains
asked Jan 25 at 8:46
Jonathan KierschJonathan Kiersch
1139
$endgroup$
I am struggling with finding the stationary distribution(s) for a discrete Markov chain with the following transition probability matrix
begin{bmatrix}1/3&2/3&0&0&0\
1/2&1/2&0&0&0\
0&1/2&0&1/2&0\
0&0&0&1/4&3/4\
0&0&0&1/3&2/3end{bmatrix}
Since this matrix is singular, it is not possible to determine $(I-P)^{-1}$. This I already tried, I also suspect that their might be more than one stationary distribution.
does anybody have an idea?
markov-chains
markov-chains
asked Jan 25 at 8:46
Jonathan KierschJonathan Kiersch
1139
asked Jan 25 at 8:46
Jonathan KierschJonathan Kiersch
1139
asked Jan 25 at 8:46
Jonathan KierschJonathan Kiersch
1139
asked Jan 25 at 8:46
Jonathan KierschJonathan Kiersch
1139
asked Jan 25 at 8:46
Jonathan KierschJonathan Kiersch
1139
1139
$begingroup$
The third position clearly is not part of any stationary distribution. Using the top left you can find a stationary distribution for the first two positions; using the bottom right you can find a stationary distribution for the last two positions. Any overall stationary distribution will be a convex combination of these two
$endgroup$
– Henry
Jan 25 at 8:54
add a comment |
$begingroup$
The third position clearly is not part of any stationary distribution. Using the top left you can find a stationary distribution for the first two positions; using the bottom right you can find a stationary distribution for the last two positions. Any overall stationary distribution will be a convex combination of these two
$endgroup$
– Henry
Jan 25 at 8:54
$begingroup$
The third position clearly is not part of any stationary distribution. Using the top left you can find a stationary distribution for the first two positions; using the bottom right you can find a stationary distribution for the last two positions. Any overall stationary distribution will be a convex combination of these two
$endgroup$
– Henry
Jan 25 at 8:54
$begingroup$
The third position clearly is not part of any stationary distribution. Using the top left you can find a stationary distribution for the first two positions; using the bottom right you can find a stationary distribution for the last two positions. Any overall stationary distribution will be a convex combination of these two
$endgroup$
– Henry
Jan 25 at 8:54
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
Let a row vector $X=(x_1,x_2,x_3,x_4,x_5)$ be a stationary distribution. Then X=XP holds. Solving it directly, we deduce that there exists a real number s and t such that $x_1=3s, x_2=4s, x_3=0, x_4=4t, x_5=9t$. Together with the condition that $x$ is stochastic, we have $7s+13t=1$. Note that this means if you have a five dimensional row vector vector v then $vP^n$ converges to a vector $(3s,4s,0,4t,9t)$ for some real number $s$ and $t$. The values of $s$ and $t$ are determined by the initial condition.
answered Jan 25 at 8:57
NotoriousJuanGNotoriousJuanG
843
$endgroup$
$begingroup$
How do you derive 7s+13t=1.... Isn't it supposed to be 1=7/3t+13/4s?
$endgroup$
– Jonathan Kiersch
Jan 25 at 9:07
$begingroup$
$x_1+x_2+x_3+x_4+x_5=3s+4s+0+4t+9t=7s+13t=1$
$endgroup$
– NotoriousJuanG
Jan 25 at 10:18
add a comment |
$begingroup$
You can find stationary distributions by solving the equations $xP=x$ for the row vector $x$ where $P$ is th given stochastic matrix. Indeed there are multiple solutions in this case. $x=(t,frac 4 3 t,0,s, frac 9 4 s)$ is a solution for any positive $t$ and $s$ and we have to normalize this by the condition $sum x_i=1$.
answered Jan 25 at 8:54
Kavi Rama MurthyKavi Rama Murthy
68.3k53069
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3086885%2fdetermine-stationary-distribution-of-markov-chain%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Let a row vector $X=(x_1,x_2,x_3,x_4,x_5)$ be a stationary distribution. Then X=XP holds. Solving it directly, we deduce that there exists a real number s and t such that $x_1=3s, x_2=4s, x_3=0, x_4=4t, x_5=9t$. Together with the condition that $x$ is stochastic, we have $7s+13t=1$. Note that this means if you have a five dimensional row vector vector v then $vP^n$ converges to a vector $(3s,4s,0,4t,9t)$ for some real number $s$ and $t$. The values of $s$ and $t$ are determined by the initial condition.
answered Jan 25 at 8:57
NotoriousJuanGNotoriousJuanG
843
$endgroup$
$begingroup$
How do you derive 7s+13t=1.... Isn't it supposed to be 1=7/3t+13/4s?
$endgroup$
– Jonathan Kiersch
Jan 25 at 9:07
$begingroup$
$x_1+x_2+x_3+x_4+x_5=3s+4s+0+4t+9t=7s+13t=1$
$endgroup$
– NotoriousJuanG
Jan 25 at 10:18
add a comment |
$begingroup$
Let a row vector $X=(x_1,x_2,x_3,x_4,x_5)$ be a stationary distribution. Then X=XP holds. Solving it directly, we deduce that there exists a real number s and t such that $x_1=3s, x_2=4s, x_3=0, x_4=4t, x_5=9t$. Together with the condition that $x$ is stochastic, we have $7s+13t=1$. Note that this means if you have a five dimensional row vector vector v then $vP^n$ converges to a vector $(3s,4s,0,4t,9t)$ for some real number $s$ and $t$. The values of $s$ and $t$ are determined by the initial condition.
answered Jan 25 at 8:57
NotoriousJuanGNotoriousJuanG
843
$endgroup$
$begingroup$
How do you derive 7s+13t=1.... Isn't it supposed to be 1=7/3t+13/4s?
$endgroup$
– Jonathan Kiersch
Jan 25 at 9:07
$begingroup$
$x_1+x_2+x_3+x_4+x_5=3s+4s+0+4t+9t=7s+13t=1$
$endgroup$
– NotoriousJuanG
Jan 25 at 10:18
add a comment |
$begingroup$
Let a row vector $X=(x_1,x_2,x_3,x_4,x_5)$ be a stationary distribution. Then X=XP holds. Solving it directly, we deduce that there exists a real number s and t such that $x_1=3s, x_2=4s, x_3=0, x_4=4t, x_5=9t$. Together with the condition that $x$ is stochastic, we have $7s+13t=1$. Note that this means if you have a five dimensional row vector vector v then $vP^n$ converges to a vector $(3s,4s,0,4t,9t)$ for some real number $s$ and $t$. The values of $s$ and $t$ are determined by the initial condition.
answered Jan 25 at 8:57
NotoriousJuanGNotoriousJuanG
843
$endgroup$
Let a row vector $X=(x_1,x_2,x_3,x_4,x_5)$ be a stationary distribution. Then X=XP holds. Solving it directly, we deduce that there exists a real number s and t such that $x_1=3s, x_2=4s, x_3=0, x_4=4t, x_5=9t$. Together with the condition that $x$ is stochastic, we have $7s+13t=1$. Note that this means if you have a five dimensional row vector vector v then $vP^n$ converges to a vector $(3s,4s,0,4t,9t)$ for some real number $s$ and $t$. The values of $s$ and $t$ are determined by the initial condition.
answered Jan 25 at 8:57
NotoriousJuanGNotoriousJuanG
843
answered Jan 25 at 8:57
NotoriousJuanGNotoriousJuanG
843
answered Jan 25 at 8:57
NotoriousJuanGNotoriousJuanG
843
answered Jan 25 at 8:57
NotoriousJuanGNotoriousJuanG
843
843
$begingroup$
How do you derive 7s+13t=1.... Isn't it supposed to be 1=7/3t+13/4s?
$endgroup$
– Jonathan Kiersch
Jan 25 at 9:07
$begingroup$
$x_1+x_2+x_3+x_4+x_5=3s+4s+0+4t+9t=7s+13t=1$
$endgroup$
– NotoriousJuanG
Jan 25 at 10:18
add a comment |
$begingroup$
How do you derive 7s+13t=1.... Isn't it supposed to be 1=7/3t+13/4s?
$endgroup$
– Jonathan Kiersch
Jan 25 at 9:07
$begingroup$
$x_1+x_2+x_3+x_4+x_5=3s+4s+0+4t+9t=7s+13t=1$
$endgroup$
– NotoriousJuanG
Jan 25 at 10:18
$begingroup$
How do you derive 7s+13t=1.... Isn't it supposed to be 1=7/3t+13/4s?
$endgroup$
– Jonathan Kiersch
Jan 25 at 9:07
$begingroup$
How do you derive 7s+13t=1.... Isn't it supposed to be 1=7/3t+13/4s?
$endgroup$
– Jonathan Kiersch
Jan 25 at 9:07
$begingroup$
$x_1+x_2+x_3+x_4+x_5=3s+4s+0+4t+9t=7s+13t=1$
$endgroup$
– NotoriousJuanG
Jan 25 at 10:18
$begingroup$
$x_1+x_2+x_3+x_4+x_5=3s+4s+0+4t+9t=7s+13t=1$
$endgroup$
– NotoriousJuanG
Jan 25 at 10:18
add a comment |
$begingroup$
You can find stationary distributions by solving the equations $xP=x$ for the row vector $x$ where $P$ is th given stochastic matrix. Indeed there are multiple solutions in this case. $x=(t,frac 4 3 t,0,s, frac 9 4 s)$ is a solution for any positive $t$ and $s$ and we have to normalize this by the condition $sum x_i=1$.
answered Jan 25 at 8:54
Kavi Rama MurthyKavi Rama Murthy
68.3k53069
$endgroup$
add a comment |
$begingroup$
You can find stationary distributions by solving the equations $xP=x$ for the row vector $x$ where $P$ is th given stochastic matrix. Indeed there are multiple solutions in this case. $x=(t,frac 4 3 t,0,s, frac 9 4 s)$ is a solution for any positive $t$ and $s$ and we have to normalize this by the condition $sum x_i=1$.
answered Jan 25 at 8:54
Kavi Rama MurthyKavi Rama Murthy
68.3k53069
$endgroup$
add a comment |
$begingroup$
You can find stationary distributions by solving the equations $xP=x$ for the row vector $x$ where $P$ is th given stochastic matrix. Indeed there are multiple solutions in this case. $x=(t,frac 4 3 t,0,s, frac 9 4 s)$ is a solution for any positive $t$ and $s$ and we have to normalize this by the condition $sum x_i=1$.
answered Jan 25 at 8:54
Kavi Rama MurthyKavi Rama Murthy
68.3k53069
$endgroup$
You can find stationary distributions by solving the equations $xP=x$ for the row vector $x$ where $P$ is th given stochastic matrix. Indeed there are multiple solutions in this case. $x=(t,frac 4 3 t,0,s, frac 9 4 s)$ is a solution for any positive $t$ and $s$ and we have to normalize this by the condition $sum x_i=1$.
answered Jan 25 at 8:54
Kavi Rama MurthyKavi Rama Murthy
68.3k53069
answered Jan 25 at 8:54
Kavi Rama MurthyKavi Rama Murthy
68.3k53069
answered Jan 25 at 8:54
Kavi Rama MurthyKavi Rama Murthy
68.3k53069
answered Jan 25 at 8:54
Kavi Rama MurthyKavi Rama Murthy
68.3k53069
68.3k53069
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3086885%2fdetermine-stationary-distribution-of-markov-chain%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
The third position clearly is not part of any stationary distribution. Using the top left you can find a stationary distribution for the first two positions; using the bottom right you can find a stationary distribution for the last two positions. Any overall stationary distribution will be a convex combination of these two
$endgroup$
– Henry
Jan 25 at 8:54