Mixing
Markov chain duration [closed]
$begingroup$
What is the formula to find average duration of state s in a Markov chain given a transition matrix?
I tried to recall the concept but could not find any references.
markov-chains markov-process transition-matrix
asked Jan 25 at 5:39
zmicerzmicer
1
$endgroup$
closed as off-topic by Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra Jan 25 at 15:02
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra
If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
$begingroup$
What is the formula to find average duration of state s in a Markov chain given a transition matrix?
I tried to recall the concept but could not find any references.
markov-chains markov-process transition-matrix
asked Jan 25 at 5:39
zmicerzmicer
1
$endgroup$
closed as off-topic by Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra Jan 25 at 15:02
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra
If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
$begingroup$
What is the formula to find average duration of state s in a Markov chain given a transition matrix?
I tried to recall the concept but could not find any references.
markov-chains markov-process transition-matrix
asked Jan 25 at 5:39
zmicerzmicer
1
$endgroup$
What is the formula to find average duration of state s in a Markov chain given a transition matrix?
I tried to recall the concept but could not find any references.
markov-chains markov-process transition-matrix
markov-chains markov-process transition-matrix
asked Jan 25 at 5:39
zmicerzmicer
1
asked Jan 25 at 5:39
zmicerzmicer
1
asked Jan 25 at 5:39
zmicerzmicer
1
asked Jan 25 at 5:39
zmicerzmicer
1
asked Jan 25 at 5:39
zmicerzmicer
1
1
closed as off-topic by Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra Jan 25 at 15:02
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra
If this question can be reworded to fit the rules in the help center, please edit the question.
closed as off-topic by Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra Jan 25 at 15:02
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Did, d80d2729a352b1366139fc119d3345, José Carlos Santos, mrtaurho, Riccardo.Alestra
If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
I understand that we are talking about a discrete time process. If the probability of staying in state $s$ from one period to another is
$$P(X_n=s|X_{n-1}=s)=q,quad forall nge1,$$
then the number $N$ of consecutive repetitions of the state $s$ is a random variable with geometric distribution, that is
$$Nsim mathcal G(1-q),$$
since every transition can be seen as a dicotomic experiment where the "fail" consists in staying at state $s$ (with probability $1-q$) and "success" is changing to any other state (with probability $q$).
In fact, there are several alternative definitions of a geometric r.v., but here I consider it as the number of "fail" results before the first "success". In this case, we have $E(N)=frac q{1-q}$.
Since the actual number of successive $s$ states would be $N+1$, if we consider the first time we reach that state, the expected number of $s$ states once the chain reaches it would be
$$E(N+1)=1+frac q{1-q}=frac1{1-q},$$
counting the current time, or
$$E(N)=frac q{1-q},$$
if we count from next time on.
answered Jan 25 at 6:19
Alejandro Nasif SalumAlejandro Nasif Salum
4,765118
$endgroup$
$begingroup$
I have an example of a transition matrix P = (7/8, 1/8; 1/8, 7/8) and it says that the expected time in each of the two states is 8 periods... I cannot figure out the calculations...
$endgroup$
– zmicer
Jan 25 at 6:48
1
$begingroup$
Take $q=7/8$ and you'll get the result.
$endgroup$
– Alejandro Nasif Salum
Jan 25 at 6:52
$begingroup$
@ math.stackexchange.com/users/481187/alejandro-nasif-salum :) Thanks!
$endgroup$
– zmicer
Jan 25 at 6:55
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
I understand that we are talking about a discrete time process. If the probability of staying in state $s$ from one period to another is
$$P(X_n=s|X_{n-1}=s)=q,quad forall nge1,$$
then the number $N$ of consecutive repetitions of the state $s$ is a random variable with geometric distribution, that is
$$Nsim mathcal G(1-q),$$
since every transition can be seen as a dicotomic experiment where the "fail" consists in staying at state $s$ (with probability $1-q$) and "success" is changing to any other state (with probability $q$).
In fact, there are several alternative definitions of a geometric r.v., but here I consider it as the number of "fail" results before the first "success". In this case, we have $E(N)=frac q{1-q}$.
Since the actual number of successive $s$ states would be $N+1$, if we consider the first time we reach that state, the expected number of $s$ states once the chain reaches it would be
$$E(N+1)=1+frac q{1-q}=frac1{1-q},$$
counting the current time, or
$$E(N)=frac q{1-q},$$
if we count from next time on.
answered Jan 25 at 6:19
Alejandro Nasif SalumAlejandro Nasif Salum
4,765118
$endgroup$
$begingroup$
I have an example of a transition matrix P = (7/8, 1/8; 1/8, 7/8) and it says that the expected time in each of the two states is 8 periods... I cannot figure out the calculations...
$endgroup$
– zmicer
Jan 25 at 6:48
1
$begingroup$
Take $q=7/8$ and you'll get the result.
$endgroup$
– Alejandro Nasif Salum
Jan 25 at 6:52
$begingroup$
@ math.stackexchange.com/users/481187/alejandro-nasif-salum :) Thanks!
$endgroup$
– zmicer
Jan 25 at 6:55
add a comment |
$begingroup$
I understand that we are talking about a discrete time process. If the probability of staying in state $s$ from one period to another is
$$P(X_n=s|X_{n-1}=s)=q,quad forall nge1,$$
then the number $N$ of consecutive repetitions of the state $s$ is a random variable with geometric distribution, that is
$$Nsim mathcal G(1-q),$$
since every transition can be seen as a dicotomic experiment where the "fail" consists in staying at state $s$ (with probability $1-q$) and "success" is changing to any other state (with probability $q$).
In fact, there are several alternative definitions of a geometric r.v., but here I consider it as the number of "fail" results before the first "success". In this case, we have $E(N)=frac q{1-q}$.
Since the actual number of successive $s$ states would be $N+1$, if we consider the first time we reach that state, the expected number of $s$ states once the chain reaches it would be
$$E(N+1)=1+frac q{1-q}=frac1{1-q},$$
counting the current time, or
$$E(N)=frac q{1-q},$$
if we count from next time on.
answered Jan 25 at 6:19
Alejandro Nasif SalumAlejandro Nasif Salum
4,765118
$endgroup$
$begingroup$
I have an example of a transition matrix P = (7/8, 1/8; 1/8, 7/8) and it says that the expected time in each of the two states is 8 periods... I cannot figure out the calculations...
$endgroup$
– zmicer
Jan 25 at 6:48
1
$begingroup$
Take $q=7/8$ and you'll get the result.
$endgroup$
– Alejandro Nasif Salum
Jan 25 at 6:52
$begingroup$
@ math.stackexchange.com/users/481187/alejandro-nasif-salum :) Thanks!
$endgroup$
– zmicer
Jan 25 at 6:55
add a comment |
$begingroup$
I understand that we are talking about a discrete time process. If the probability of staying in state $s$ from one period to another is
$$P(X_n=s|X_{n-1}=s)=q,quad forall nge1,$$
then the number $N$ of consecutive repetitions of the state $s$ is a random variable with geometric distribution, that is
$$Nsim mathcal G(1-q),$$
since every transition can be seen as a dicotomic experiment where the "fail" consists in staying at state $s$ (with probability $1-q$) and "success" is changing to any other state (with probability $q$).
In fact, there are several alternative definitions of a geometric r.v., but here I consider it as the number of "fail" results before the first "success". In this case, we have $E(N)=frac q{1-q}$.
Since the actual number of successive $s$ states would be $N+1$, if we consider the first time we reach that state, the expected number of $s$ states once the chain reaches it would be
$$E(N+1)=1+frac q{1-q}=frac1{1-q},$$
counting the current time, or
$$E(N)=frac q{1-q},$$
if we count from next time on.
answered Jan 25 at 6:19
Alejandro Nasif SalumAlejandro Nasif Salum
4,765118
$endgroup$
I understand that we are talking about a discrete time process. If the probability of staying in state $s$ from one period to another is
$$P(X_n=s|X_{n-1}=s)=q,quad forall nge1,$$
then the number $N$ of consecutive repetitions of the state $s$ is a random variable with geometric distribution, that is
$$Nsim mathcal G(1-q),$$
since every transition can be seen as a dicotomic experiment where the "fail" consists in staying at state $s$ (with probability $1-q$) and "success" is changing to any other state (with probability $q$).
In fact, there are several alternative definitions of a geometric r.v., but here I consider it as the number of "fail" results before the first "success". In this case, we have $E(N)=frac q{1-q}$.
Since the actual number of successive $s$ states would be $N+1$, if we consider the first time we reach that state, the expected number of $s$ states once the chain reaches it would be
$$E(N+1)=1+frac q{1-q}=frac1{1-q},$$
counting the current time, or
$$E(N)=frac q{1-q},$$
if we count from next time on.
answered Jan 25 at 6:19
Alejandro Nasif SalumAlejandro Nasif Salum
4,765118
edited Jan 25 at 6:52
answered Jan 25 at 6:19
Alejandro Nasif SalumAlejandro Nasif Salum
4,765118
answered Jan 25 at 6:19
Alejandro Nasif SalumAlejandro Nasif Salum
4,765118
answered Jan 25 at 6:19
Alejandro Nasif SalumAlejandro Nasif Salum
4,765118
4,765118
$begingroup$
I have an example of a transition matrix P = (7/8, 1/8; 1/8, 7/8) and it says that the expected time in each of the two states is 8 periods... I cannot figure out the calculations...
$endgroup$
– zmicer
Jan 25 at 6:48
1
$begingroup$
Take $q=7/8$ and you'll get the result.
$endgroup$
– Alejandro Nasif Salum
Jan 25 at 6:52
$begingroup$
@ math.stackexchange.com/users/481187/alejandro-nasif-salum :) Thanks!
$endgroup$
– zmicer
Jan 25 at 6:55
add a comment |
$begingroup$
I have an example of a transition matrix P = (7/8, 1/8; 1/8, 7/8) and it says that the expected time in each of the two states is 8 periods... I cannot figure out the calculations...
$endgroup$
– zmicer
Jan 25 at 6:48
1
$begingroup$
Take $q=7/8$ and you'll get the result.
$endgroup$
– Alejandro Nasif Salum
Jan 25 at 6:52
$begingroup$
@ math.stackexchange.com/users/481187/alejandro-nasif-salum :) Thanks!
$endgroup$
– zmicer
Jan 25 at 6:55
$begingroup$
I have an example of a transition matrix P = (7/8, 1/8; 1/8, 7/8) and it says that the expected time in each of the two states is 8 periods... I cannot figure out the calculations...
$endgroup$
– zmicer
Jan 25 at 6:48
$begingroup$
I have an example of a transition matrix P = (7/8, 1/8; 1/8, 7/8) and it says that the expected time in each of the two states is 8 periods... I cannot figure out the calculations...
$endgroup$
– zmicer
Jan 25 at 6:48
1
1
$begingroup$
Take $q=7/8$ and you'll get the result.
$endgroup$
– Alejandro Nasif Salum
Jan 25 at 6:52
$begingroup$
Take $q=7/8$ and you'll get the result.
$endgroup$
– Alejandro Nasif Salum
Jan 25 at 6:52
$begingroup$
@ math.stackexchange.com/users/481187/alejandro-nasif-salum :) Thanks!
$endgroup$
– zmicer
Jan 25 at 6:55
$begingroup$
@ math.stackexchange.com/users/481187/alejandro-nasif-salum :) Thanks!
$endgroup$
– zmicer
Jan 25 at 6:55
add a comment |
