Limit distribution of a reducible Markov chain
$begingroup$
I had a simple question yesterday when I was trying to solve an exercise on a reducible,aperiodic Markov Chain. The state spase S was $$S={1,...,7}$$ and we could partition it into two closed classes ${5,6,7} bigcup {3,4}$ but the class ${1,2}$ was open (thus transient). The exercise was giving the transition matrix and was asking to find the stationary distributions. At the end it was also asking to find the limit distribution, given that the INITIAL distribution was $$P[X_0=1]=P[X_0=2]=frac{1}{2}$$ I only know what to do in case the initial probability gives mass 1 to some state (I find the absorption probabilities in each closed class, etc..) but when I have initial distribution which gives mass to both of these transient states I do not know what to do. Any helpful ideas/theorems about this??
Thanks a lot !
$$ begin{pmatrix}
1/3 & 1/6 & 1/3 & 0 & 0 & 1/6 \
3/5 & 0 & 0 & 1/5 & 1/5 & 0 & 0 \
0 & 0 & 1/2 & 1/2 & 0 & 0 & 0 \
0 & 0 & 1/4 & 3/4 & 0 & 0 & 0 \
0 & 0 & 0 & 0 & 0 & 1 & 0 \
0 & 0 & 0 & 0 & 0 & 1/4 & 3/4 \
0 & 0 & 0 & 0 & 1/2 & 0 & 1/2 end{pmatrix} $$
markov-chains
asked Jan 18 at 17:41
Petros KarajanPetros Karajan
214
$endgroup$
|
show 1 more comment
$begingroup$
I had a simple question yesterday when I was trying to solve an exercise on a reducible,aperiodic Markov Chain. The state spase S was $$S={1,...,7}$$ and we could partition it into two closed classes ${5,6,7} bigcup {3,4}$ but the class ${1,2}$ was open (thus transient). The exercise was giving the transition matrix and was asking to find the stationary distributions. At the end it was also asking to find the limit distribution, given that the INITIAL distribution was $$P[X_0=1]=P[X_0=2]=frac{1}{2}$$ I only know what to do in case the initial probability gives mass 1 to some state (I find the absorption probabilities in each closed class, etc..) but when I have initial distribution which gives mass to both of these transient states I do not know what to do. Any helpful ideas/theorems about this??
Thanks a lot !
$$ begin{pmatrix}
1/3 & 1/6 & 1/3 & 0 & 0 & 1/6 \
3/5 & 0 & 0 & 1/5 & 1/5 & 0 & 0 \
0 & 0 & 1/2 & 1/2 & 0 & 0 & 0 \
0 & 0 & 1/4 & 3/4 & 0 & 0 & 0 \
0 & 0 & 0 & 0 & 0 & 1 & 0 \
0 & 0 & 0 & 0 & 0 & 1/4 & 3/4 \
0 & 0 & 0 & 0 & 1/2 & 0 & 1/2 end{pmatrix} $$
markov-chains
asked Jan 18 at 17:41
Petros KarajanPetros Karajan
214
$endgroup$
$begingroup$
Giving the transition matrix would make things easier (to answer) here. An answer of the kind "take 1/2 of the limit distribution for the case of giving full probability to the state 5 and also take 1/2 of the limit distribution for the case of giving full probability to the state 6 and add them" is not really beautiful.
$endgroup$
– dan_fulea
Jan 18 at 18:03
$begingroup$
OK,let's suppose we have the following example in page 1 : win.tue.nl/~resing/SOR/handout.pdf and you want to answer question 2) .
$endgroup$
– Petros Karajan
Jan 18 at 18:20
$begingroup$
It may be useful to use mathjax, and include the example in the original post, math.meta.stackexchange.com/questions/5020/…
$endgroup$
– dan_fulea
Jan 18 at 18:21
$begingroup$
I will write it afterwards. Do you have any idea about the solution?
$endgroup$
– Petros Karajan
Jan 18 at 18:26
$begingroup$
It should be a "linear play". More exactly, in some notation i am used to, the initial distribution is a row vector, shape $Ntimes 1$, where $N$ (six or seven in our case) is the number of states, and $A$, the transition matrix, is a an $Ntimes N$ matrix. So we need to compute something like $lim_n qA^n$ for $nto infty$. This expression is linear in $q$. So if we know the answer for the canonical base $e_1=[1,0,0,0,dots]$, $e_2=[0,1,0,0,dots]$, ... then we need to linearly combine them using the weights in $q$.
$endgroup$
– dan_fulea
Jan 18 at 18:33
|
show 1 more comment
$begingroup$
I had a simple question yesterday when I was trying to solve an exercise on a reducible,aperiodic Markov Chain. The state spase S was $$S={1,...,7}$$ and we could partition it into two closed classes ${5,6,7} bigcup {3,4}$ but the class ${1,2}$ was open (thus transient). The exercise was giving the transition matrix and was asking to find the stationary distributions. At the end it was also asking to find the limit distribution, given that the INITIAL distribution was $$P[X_0=1]=P[X_0=2]=frac{1}{2}$$ I only know what to do in case the initial probability gives mass 1 to some state (I find the absorption probabilities in each closed class, etc..) but when I have initial distribution which gives mass to both of these transient states I do not know what to do. Any helpful ideas/theorems about this??
Thanks a lot !
$$ begin{pmatrix}
1/3 & 1/6 & 1/3 & 0 & 0 & 1/6 \
3/5 & 0 & 0 & 1/5 & 1/5 & 0 & 0 \
0 & 0 & 1/2 & 1/2 & 0 & 0 & 0 \
0 & 0 & 1/4 & 3/4 & 0 & 0 & 0 \
0 & 0 & 0 & 0 & 0 & 1 & 0 \
0 & 0 & 0 & 0 & 0 & 1/4 & 3/4 \
0 & 0 & 0 & 0 & 1/2 & 0 & 1/2 end{pmatrix} $$
markov-chains
asked Jan 18 at 17:41
Petros KarajanPetros Karajan
214
$endgroup$
I had a simple question yesterday when I was trying to solve an exercise on a reducible,aperiodic Markov Chain. The state spase S was $$S={1,...,7}$$ and we could partition it into two closed classes ${5,6,7} bigcup {3,4}$ but the class ${1,2}$ was open (thus transient). The exercise was giving the transition matrix and was asking to find the stationary distributions. At the end it was also asking to find the limit distribution, given that the INITIAL distribution was $$P[X_0=1]=P[X_0=2]=frac{1}{2}$$ I only know what to do in case the initial probability gives mass 1 to some state (I find the absorption probabilities in each closed class, etc..) but when I have initial distribution which gives mass to both of these transient states I do not know what to do. Any helpful ideas/theorems about this??
Thanks a lot !
$$ begin{pmatrix}
1/3 & 1/6 & 1/3 & 0 & 0 & 1/6 \
3/5 & 0 & 0 & 1/5 & 1/5 & 0 & 0 \
0 & 0 & 1/2 & 1/2 & 0 & 0 & 0 \
0 & 0 & 1/4 & 3/4 & 0 & 0 & 0 \
0 & 0 & 0 & 0 & 0 & 1 & 0 \
0 & 0 & 0 & 0 & 0 & 1/4 & 3/4 \
0 & 0 & 0 & 0 & 1/2 & 0 & 1/2 end{pmatrix} $$
markov-chains
markov-chains
asked Jan 18 at 17:41
Petros KarajanPetros Karajan
214
asked Jan 18 at 17:41
Petros KarajanPetros Karajan
214
edited Jan 18 at 18:42
Petros Karajan
asked Jan 18 at 17:41
Petros KarajanPetros Karajan
214
asked Jan 18 at 17:41
Petros KarajanPetros Karajan
214
asked Jan 18 at 17:41
Petros KarajanPetros Karajan
214
214
$begingroup$
Giving the transition matrix would make things easier (to answer) here. An answer of the kind "take 1/2 of the limit distribution for the case of giving full probability to the state 5 and also take 1/2 of the limit distribution for the case of giving full probability to the state 6 and add them" is not really beautiful.
$endgroup$
– dan_fulea
Jan 18 at 18:03
$begingroup$
OK,let's suppose we have the following example in page 1 : win.tue.nl/~resing/SOR/handout.pdf and you want to answer question 2) .
$endgroup$
– Petros Karajan
Jan 18 at 18:20
$begingroup$
It may be useful to use mathjax, and include the example in the original post, math.meta.stackexchange.com/questions/5020/…
$endgroup$
– dan_fulea
Jan 18 at 18:21
$begingroup$
I will write it afterwards. Do you have any idea about the solution?
$endgroup$
– Petros Karajan
Jan 18 at 18:26
$begingroup$
It should be a "linear play". More exactly, in some notation i am used to, the initial distribution is a row vector, shape $Ntimes 1$, where $N$ (six or seven in our case) is the number of states, and $A$, the transition matrix, is a an $Ntimes N$ matrix. So we need to compute something like $lim_n qA^n$ for $nto infty$. This expression is linear in $q$. So if we know the answer for the canonical base $e_1=[1,0,0,0,dots]$, $e_2=[0,1,0,0,dots]$, ... then we need to linearly combine them using the weights in $q$.
$endgroup$
– dan_fulea
Jan 18 at 18:33
|
show 1 more comment
$begingroup$
Giving the transition matrix would make things easier (to answer) here. An answer of the kind "take 1/2 of the limit distribution for the case of giving full probability to the state 5 and also take 1/2 of the limit distribution for the case of giving full probability to the state 6 and add them" is not really beautiful.
$endgroup$
– dan_fulea
Jan 18 at 18:03
$begingroup$
OK,let's suppose we have the following example in page 1 : win.tue.nl/~resing/SOR/handout.pdf and you want to answer question 2) .
$endgroup$
– Petros Karajan
Jan 18 at 18:20
$begingroup$
It may be useful to use mathjax, and include the example in the original post, math.meta.stackexchange.com/questions/5020/…
$endgroup$
– dan_fulea
Jan 18 at 18:21
$begingroup$
I will write it afterwards. Do you have any idea about the solution?
$endgroup$
– Petros Karajan
Jan 18 at 18:26
$begingroup$
It should be a "linear play". More exactly, in some notation i am used to, the initial distribution is a row vector, shape $Ntimes 1$, where $N$ (six or seven in our case) is the number of states, and $A$, the transition matrix, is a an $Ntimes N$ matrix. So we need to compute something like $lim_n qA^n$ for $nto infty$. This expression is linear in $q$. So if we know the answer for the canonical base $e_1=[1,0,0,0,dots]$, $e_2=[0,1,0,0,dots]$, ... then we need to linearly combine them using the weights in $q$.
$endgroup$
– dan_fulea
Jan 18 at 18:33
$begingroup$
Giving the transition matrix would make things easier (to answer) here. An answer of the kind "take 1/2 of the limit distribution for the case of giving full probability to the state 5 and also take 1/2 of the limit distribution for the case of giving full probability to the state 6 and add them" is not really beautiful.
$endgroup$
– dan_fulea
Jan 18 at 18:03
$begingroup$
Giving the transition matrix would make things easier (to answer) here. An answer of the kind "take 1/2 of the limit distribution for the case of giving full probability to the state 5 and also take 1/2 of the limit distribution for the case of giving full probability to the state 6 and add them" is not really beautiful.
$endgroup$
– dan_fulea
Jan 18 at 18:03
$begingroup$
OK,let's suppose we have the following example in page 1 : win.tue.nl/~resing/SOR/handout.pdf and you want to answer question 2) .
$endgroup$
– Petros Karajan
Jan 18 at 18:20
$begingroup$
OK,let's suppose we have the following example in page 1 : win.tue.nl/~resing/SOR/handout.pdf and you want to answer question 2) .
$endgroup$
– Petros Karajan
Jan 18 at 18:20
$begingroup$
It may be useful to use mathjax, and include the example in the original post, math.meta.stackexchange.com/questions/5020/…
$endgroup$
– dan_fulea
Jan 18 at 18:21
$begingroup$
It may be useful to use mathjax, and include the example in the original post, math.meta.stackexchange.com/questions/5020/…
$endgroup$
– dan_fulea
Jan 18 at 18:21
$begingroup$
I will write it afterwards. Do you have any idea about the solution?
$endgroup$
– Petros Karajan
Jan 18 at 18:26
$begingroup$
I will write it afterwards. Do you have any idea about the solution?
$endgroup$
– Petros Karajan
Jan 18 at 18:26
$begingroup$
It should be a "linear play". More exactly, in some notation i am used to, the initial distribution is a row vector, shape $Ntimes 1$, where $N$ (six or seven in our case) is the number of states, and $A$, the transition matrix, is a an $Ntimes N$ matrix. So we need to compute something like $lim_n qA^n$ for $nto infty$. This expression is linear in $q$. So if we know the answer for the canonical base $e_1=[1,0,0,0,dots]$, $e_2=[0,1,0,0,dots]$, ... then we need to linearly combine them using the weights in $q$.
$endgroup$
– dan_fulea
Jan 18 at 18:33
$begingroup$
It should be a "linear play". More exactly, in some notation i am used to, the initial distribution is a row vector, shape $Ntimes 1$, where $N$ (six or seven in our case) is the number of states, and $A$, the transition matrix, is a an $Ntimes N$ matrix. So we need to compute something like $lim_n qA^n$ for $nto infty$. This expression is linear in $q$. So if we know the answer for the canonical base $e_1=[1,0,0,0,dots]$, $e_2=[0,1,0,0,dots]$, ... then we need to linearly combine them using the weights in $q$.
$endgroup$
– dan_fulea
Jan 18 at 18:33
|
show 1 more comment
1 Answer
1
active
oldest
votes
$begingroup$
As dan_fulea noticed above, it suffices to calculate the case when $ P[X_0=1]=1 $ and $ P[X_0=2]=1 $ . The first is already treated in the notes: it gives the limit vector $$ begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} $$
For the case $ P[X_0=2]=1 $ , since $q_{2,E_1}=frac{10}{17}$ and $ q_{2,E_2}=frac{7}{17}$, we get the limit vector : $$begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$ Hence, the final answer to our question is : $$ frac{1}{2}begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} + frac{1}{2} begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$
answered Jan 18 at 19:02
Petros KarajanPetros Karajan
214
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3078555%2flimit-distribution-of-a-reducible-markov-chain%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
As dan_fulea noticed above, it suffices to calculate the case when $ P[X_0=1]=1 $ and $ P[X_0=2]=1 $ . The first is already treated in the notes: it gives the limit vector $$ begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} $$
For the case $ P[X_0=2]=1 $ , since $q_{2,E_1}=frac{10}{17}$ and $ q_{2,E_2}=frac{7}{17}$, we get the limit vector : $$begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$ Hence, the final answer to our question is : $$ frac{1}{2}begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} + frac{1}{2} begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$
answered Jan 18 at 19:02
Petros KarajanPetros Karajan
214
$endgroup$
add a comment |
$begingroup$
As dan_fulea noticed above, it suffices to calculate the case when $ P[X_0=1]=1 $ and $ P[X_0=2]=1 $ . The first is already treated in the notes: it gives the limit vector $$ begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} $$
For the case $ P[X_0=2]=1 $ , since $q_{2,E_1}=frac{10}{17}$ and $ q_{2,E_2}=frac{7}{17}$, we get the limit vector : $$begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$ Hence, the final answer to our question is : $$ frac{1}{2}begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} + frac{1}{2} begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$
answered Jan 18 at 19:02
Petros KarajanPetros Karajan
214
$endgroup$
add a comment |
$begingroup$
As dan_fulea noticed above, it suffices to calculate the case when $ P[X_0=1]=1 $ and $ P[X_0=2]=1 $ . The first is already treated in the notes: it gives the limit vector $$ begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} $$
For the case $ P[X_0=2]=1 $ , since $q_{2,E_1}=frac{10}{17}$ and $ q_{2,E_2}=frac{7}{17}$, we get the limit vector : $$begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$ Hence, the final answer to our question is : $$ frac{1}{2}begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} + frac{1}{2} begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$
answered Jan 18 at 19:02
Petros KarajanPetros Karajan
214
$endgroup$
As dan_fulea noticed above, it suffices to calculate the case when $ P[X_0=1]=1 $ and $ P[X_0=2]=1 $ . The first is already treated in the notes: it gives the limit vector $$ begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} $$
For the case $ P[X_0=2]=1 $ , since $q_{2,E_1}=frac{10}{17}$ and $ q_{2,E_2}=frac{7}{17}$, we get the limit vector : $$begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$ Hence, the final answer to our question is : $$ frac{1}{2}begin{pmatrix} 0, & 0, & frac{11}{51}, & frac{22}{51}, & frac{18}{221}, & frac{24}{221}, & frac{36}{221} end{pmatrix} + frac{1}{2} begin{pmatrix} 0, & 0, & frac{10}{51}, & frac{20}{51}, & frac{21}{221}, & frac{28}{221}, & frac{42}{221} end{pmatrix} $$
answered Jan 18 at 19:02
Petros KarajanPetros Karajan
214
answered Jan 18 at 19:02
Petros KarajanPetros Karajan
214
answered Jan 18 at 19:02
Petros KarajanPetros Karajan
214
answered Jan 18 at 19:02
Petros KarajanPetros Karajan
214
214
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3078555%2flimit-distribution-of-a-reducible-markov-chain%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Giving the transition matrix would make things easier (to answer) here. An answer of the kind "take 1/2 of the limit distribution for the case of giving full probability to the state 5 and also take 1/2 of the limit distribution for the case of giving full probability to the state 6 and add them" is not really beautiful.
$endgroup$
– dan_fulea
Jan 18 at 18:03
$begingroup$
OK,let's suppose we have the following example in page 1 : win.tue.nl/~resing/SOR/handout.pdf and you want to answer question 2) .
$endgroup$
– Petros Karajan
Jan 18 at 18:20
$begingroup$
It may be useful to use mathjax, and include the example in the original post, math.meta.stackexchange.com/questions/5020/…
$endgroup$
– dan_fulea
Jan 18 at 18:21
$begingroup$
I will write it afterwards. Do you have any idea about the solution?
$endgroup$
– Petros Karajan
Jan 18 at 18:26
$begingroup$
It should be a "linear play". More exactly, in some notation i am used to, the initial distribution is a row vector, shape $Ntimes 1$, where $N$ (six or seven in our case) is the number of states, and $A$, the transition matrix, is a an $Ntimes N$ matrix. So we need to compute something like $lim_n qA^n$ for $nto infty$. This expression is linear in $q$. So if we know the answer for the canonical base $e_1=[1,0,0,0,dots]$, $e_2=[0,1,0,0,dots]$, ... then we need to linearly combine them using the weights in $q$.
$endgroup$
– dan_fulea
Jan 18 at 18:33