How do I show that the mean recurrence time for transient states is infinity?












0












$begingroup$


The random variable $T_i$, the "Hitting Time of $i$" is defined to be the first $n$ such that $X_n=i$ given that $X_0=i$.



By the mean recurrence time of $T_i$, I mean the expected value of this random variable.



I wish to show that if $i$ is transient, then the expectation does not converge to any finite real number. While this, intuitively makes sense, I do not know how to formally prove this and any help is appreciated.






probability markov-chains







share|cite|improve this question









$endgroup$












  • $begingroup$
    A given state $i$ is transient iff $P_i(T_i=+infty)ne0$. Then $E_i(T_i)geqslant E_i(T_imathbf 1_{T_i=+infty})=+inftycdot P_i(T_i=+infty)=+infty$, as desired.
    $endgroup$
    – Did
    Jan 9 at 0:07
















0












$begingroup$


The random variable $T_i$, the "Hitting Time of $i$" is defined to be the first $n$ such that $X_n=i$ given that $X_0=i$.



By the mean recurrence time of $T_i$, I mean the expected value of this random variable.



I wish to show that if $i$ is transient, then the expectation does not converge to any finite real number. While this, intuitively makes sense, I do not know how to formally prove this and any help is appreciated.






probability markov-chains







share|cite|improve this question









$endgroup$












  • $begingroup$
    A given state $i$ is transient iff $P_i(T_i=+infty)ne0$. Then $E_i(T_i)geqslant E_i(T_imathbf 1_{T_i=+infty})=+inftycdot P_i(T_i=+infty)=+infty$, as desired.
    $endgroup$
    – Did
    Jan 9 at 0:07














0












0








0





$begingroup$


The random variable $T_i$, the "Hitting Time of $i$" is defined to be the first $n$ such that $X_n=i$ given that $X_0=i$.



By the mean recurrence time of $T_i$, I mean the expected value of this random variable.



I wish to show that if $i$ is transient, then the expectation does not converge to any finite real number. While this, intuitively makes sense, I do not know how to formally prove this and any help is appreciated.






probability markov-chains







share|cite|improve this question









$endgroup$




The random variable $T_i$, the "Hitting Time of $i$" is defined to be the first $n$ such that $X_n=i$ given that $X_0=i$.



By the mean recurrence time of $T_i$, I mean the expected value of this random variable.



I wish to show that if $i$ is transient, then the expectation does not converge to any finite real number. While this, intuitively makes sense, I do not know how to formally prove this and any help is appreciated.






probability markov-chains




probability markov-chains






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked Nov 14 '18 at 18:24









Agnishom ChattopadhyayAgnishom Chattopadhyay

1,6121818




1,6121818












  • $begingroup$
    A given state $i$ is transient iff $P_i(T_i=+infty)ne0$. Then $E_i(T_i)geqslant E_i(T_imathbf 1_{T_i=+infty})=+inftycdot P_i(T_i=+infty)=+infty$, as desired.
    $endgroup$
    – Did
    Jan 9 at 0:07


















  • $begingroup$
    A given state $i$ is transient iff $P_i(T_i=+infty)ne0$. Then $E_i(T_i)geqslant E_i(T_imathbf 1_{T_i=+infty})=+inftycdot P_i(T_i=+infty)=+infty$, as desired.
    $endgroup$
    – Did
    Jan 9 at 0:07
















$begingroup$
A given state $i$ is transient iff $P_i(T_i=+infty)ne0$. Then $E_i(T_i)geqslant E_i(T_imathbf 1_{T_i=+infty})=+inftycdot P_i(T_i=+infty)=+infty$, as desired.
$endgroup$
– Did
Jan 9 at 0:07




$begingroup$
A given state $i$ is transient iff $P_i(T_i=+infty)ne0$. Then $E_i(T_i)geqslant E_i(T_imathbf 1_{T_i=+infty})=+inftycdot P_i(T_i=+infty)=+infty$, as desired.
$endgroup$
– Did
Jan 9 at 0:07










2 Answers
2






active

oldest

votes


















1












$begingroup$

Note that state $i$ is persistent iff,
$$ P(X_n = i text{ for some } n geq 1| X_0 = i) = 1$$



Each state is transient or persistent.

The hitting time of state $i$, $T_i$, is a random variable defined as the first time we visit state $i$:
$$ T_i = min {n | n geq 1, X_n = i} $$
where $T_i$ is defined as $infty$ if this visit never happens.



We now show that $ P(T_i = infty | X_0 = i) > 0 $ iff state $i$ is transient.
Then, the required result on the mean recurrence time follows, because the mean recurrence time $mu_i$ is defined as:
$$mu_i = E(T_i| X_0 = i) $$



Suppose state $i$ is transient. Then,
$$
begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
& = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
& > 1 - 1 = 0.
end{align}
$$



Suppose state $i$ is persistent. Then,
$$
begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
& = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
& = 1 - 1 = 0.
end{align}
$$



This shows both directions, and completes the proof.






share|cite|improve this answer









$endgroup$





















    0












    $begingroup$

    I don't think you have the definition quite right. The first passage time $T_i$ is the minimum $n$ such that $X_n = i$ given $X_0 = i$, and is defined to be $infty$ if no such $n$ exists.



    In that light, we just need to know that a state $i$ is transient if there's some other state $j$ such that $i rightarrow j$, but $j notrightarrow i$. That is, if a (finite-state, discrete-time) Markov chain initialized at state $i$ reaches state $j$ it almost surely never returns to $i$. Because $i rightarrow j$, this happens with positive probability and thus $T_i = infty$ with positive probability. (Which implies $mathbb{E}[T_i] = infty$).



    One reference you may find useful is these lecture notes.






    share|cite|improve this answer











    $endgroup$













    • $begingroup$
      Your definition of transience is wrong. Most transient Markov chains are such that $ito j$ and $jto i$ for every $i$ and $j$ in their state space.
      $endgroup$
      – Did
      Jan 9 at 0:06










    • $begingroup$
      But only in the case of an infinite state space, right? I was only talking about the finite-state case (and probably should have said that more clearly).
      $endgroup$
      – Will Brannon
      Jan 9 at 3:52










    • $begingroup$
      Then you are describing (non-)communicability between states rather than transience.
      $endgroup$
      – Did
      Jan 9 at 6:48











    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "69"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2998632%2fhow-do-i-show-that-the-mean-recurrence-time-for-transient-states-is-infinity%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    1












    $begingroup$

    Note that state $i$ is persistent iff,
    $$ P(X_n = i text{ for some } n geq 1| X_0 = i) = 1$$



    Each state is transient or persistent.

    The hitting time of state $i$, $T_i$, is a random variable defined as the first time we visit state $i$:
    $$ T_i = min {n | n geq 1, X_n = i} $$
    where $T_i$ is defined as $infty$ if this visit never happens.



    We now show that $ P(T_i = infty | X_0 = i) > 0 $ iff state $i$ is transient.
    Then, the required result on the mean recurrence time follows, because the mean recurrence time $mu_i$ is defined as:
    $$mu_i = E(T_i| X_0 = i) $$



    Suppose state $i$ is transient. Then,
    $$
    begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
    & = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
    & > 1 - 1 = 0.
    end{align}
    $$



    Suppose state $i$ is persistent. Then,
    $$
    begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
    & = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
    & = 1 - 1 = 0.
    end{align}
    $$



    This shows both directions, and completes the proof.






    share|cite|improve this answer









    $endgroup$


















      1












      $begingroup$

      Note that state $i$ is persistent iff,
      $$ P(X_n = i text{ for some } n geq 1| X_0 = i) = 1$$



      Each state is transient or persistent.

      The hitting time of state $i$, $T_i$, is a random variable defined as the first time we visit state $i$:
      $$ T_i = min {n | n geq 1, X_n = i} $$
      where $T_i$ is defined as $infty$ if this visit never happens.



      We now show that $ P(T_i = infty | X_0 = i) > 0 $ iff state $i$ is transient.
      Then, the required result on the mean recurrence time follows, because the mean recurrence time $mu_i$ is defined as:
      $$mu_i = E(T_i| X_0 = i) $$



      Suppose state $i$ is transient. Then,
      $$
      begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
      & = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
      & > 1 - 1 = 0.
      end{align}
      $$



      Suppose state $i$ is persistent. Then,
      $$
      begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
      & = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
      & = 1 - 1 = 0.
      end{align}
      $$



      This shows both directions, and completes the proof.






      share|cite|improve this answer









      $endgroup$
















        1












        1








        1





        $begingroup$

        Note that state $i$ is persistent iff,
        $$ P(X_n = i text{ for some } n geq 1| X_0 = i) = 1$$



        Each state is transient or persistent.

        The hitting time of state $i$, $T_i$, is a random variable defined as the first time we visit state $i$:
        $$ T_i = min {n | n geq 1, X_n = i} $$
        where $T_i$ is defined as $infty$ if this visit never happens.



        We now show that $ P(T_i = infty | X_0 = i) > 0 $ iff state $i$ is transient.
        Then, the required result on the mean recurrence time follows, because the mean recurrence time $mu_i$ is defined as:
        $$mu_i = E(T_i| X_0 = i) $$



        Suppose state $i$ is transient. Then,
        $$
        begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
        & = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
        & > 1 - 1 = 0.
        end{align}
        $$



        Suppose state $i$ is persistent. Then,
        $$
        begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
        & = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
        & = 1 - 1 = 0.
        end{align}
        $$



        This shows both directions, and completes the proof.






        share|cite|improve this answer









        $endgroup$



        Note that state $i$ is persistent iff,
        $$ P(X_n = i text{ for some } n geq 1| X_0 = i) = 1$$



        Each state is transient or persistent.

        The hitting time of state $i$, $T_i$, is a random variable defined as the first time we visit state $i$:
        $$ T_i = min {n | n geq 1, X_n = i} $$
        where $T_i$ is defined as $infty$ if this visit never happens.



        We now show that $ P(T_i = infty | X_0 = i) > 0 $ iff state $i$ is transient.
        Then, the required result on the mean recurrence time follows, because the mean recurrence time $mu_i$ is defined as:
        $$mu_i = E(T_i| X_0 = i) $$



        Suppose state $i$ is transient. Then,
        $$
        begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
        & = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
        & > 1 - 1 = 0.
        end{align}
        $$



        Suppose state $i$ is persistent. Then,
        $$
        begin{align} P(T_i = infty | X_0 = i) & = P(X_n neq i text{ for all } n geq 1 | X_0 = i) \
        & = 1 - P(X_n = i text{ for some } n geq 1 | X_0 = i) \
        & = 1 - 1 = 0.
        end{align}
        $$



        This shows both directions, and completes the proof.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered Feb 13 at 7:00









        AmeyaAmeya

        113




        113























            0












            $begingroup$

            I don't think you have the definition quite right. The first passage time $T_i$ is the minimum $n$ such that $X_n = i$ given $X_0 = i$, and is defined to be $infty$ if no such $n$ exists.



            In that light, we just need to know that a state $i$ is transient if there's some other state $j$ such that $i rightarrow j$, but $j notrightarrow i$. That is, if a (finite-state, discrete-time) Markov chain initialized at state $i$ reaches state $j$ it almost surely never returns to $i$. Because $i rightarrow j$, this happens with positive probability and thus $T_i = infty$ with positive probability. (Which implies $mathbb{E}[T_i] = infty$).



            One reference you may find useful is these lecture notes.






            share|cite|improve this answer











            $endgroup$













            • $begingroup$
              Your definition of transience is wrong. Most transient Markov chains are such that $ito j$ and $jto i$ for every $i$ and $j$ in their state space.
              $endgroup$
              – Did
              Jan 9 at 0:06










            • $begingroup$
              But only in the case of an infinite state space, right? I was only talking about the finite-state case (and probably should have said that more clearly).
              $endgroup$
              – Will Brannon
              Jan 9 at 3:52










            • $begingroup$
              Then you are describing (non-)communicability between states rather than transience.
              $endgroup$
              – Did
              Jan 9 at 6:48
















            0












            $begingroup$

            I don't think you have the definition quite right. The first passage time $T_i$ is the minimum $n$ such that $X_n = i$ given $X_0 = i$, and is defined to be $infty$ if no such $n$ exists.



            In that light, we just need to know that a state $i$ is transient if there's some other state $j$ such that $i rightarrow j$, but $j notrightarrow i$. That is, if a (finite-state, discrete-time) Markov chain initialized at state $i$ reaches state $j$ it almost surely never returns to $i$. Because $i rightarrow j$, this happens with positive probability and thus $T_i = infty$ with positive probability. (Which implies $mathbb{E}[T_i] = infty$).



            One reference you may find useful is these lecture notes.






            share|cite|improve this answer











            $endgroup$













            • $begingroup$
              Your definition of transience is wrong. Most transient Markov chains are such that $ito j$ and $jto i$ for every $i$ and $j$ in their state space.
              $endgroup$
              – Did
              Jan 9 at 0:06










            • $begingroup$
              But only in the case of an infinite state space, right? I was only talking about the finite-state case (and probably should have said that more clearly).
              $endgroup$
              – Will Brannon
              Jan 9 at 3:52










            • $begingroup$
              Then you are describing (non-)communicability between states rather than transience.
              $endgroup$
              – Did
              Jan 9 at 6:48














            0












            0








            0





            $begingroup$

            I don't think you have the definition quite right. The first passage time $T_i$ is the minimum $n$ such that $X_n = i$ given $X_0 = i$, and is defined to be $infty$ if no such $n$ exists.



            In that light, we just need to know that a state $i$ is transient if there's some other state $j$ such that $i rightarrow j$, but $j notrightarrow i$. That is, if a (finite-state, discrete-time) Markov chain initialized at state $i$ reaches state $j$ it almost surely never returns to $i$. Because $i rightarrow j$, this happens with positive probability and thus $T_i = infty$ with positive probability. (Which implies $mathbb{E}[T_i] = infty$).



            One reference you may find useful is these lecture notes.






            share|cite|improve this answer











            $endgroup$



            I don't think you have the definition quite right. The first passage time $T_i$ is the minimum $n$ such that $X_n = i$ given $X_0 = i$, and is defined to be $infty$ if no such $n$ exists.



            In that light, we just need to know that a state $i$ is transient if there's some other state $j$ such that $i rightarrow j$, but $j notrightarrow i$. That is, if a (finite-state, discrete-time) Markov chain initialized at state $i$ reaches state $j$ it almost surely never returns to $i$. Because $i rightarrow j$, this happens with positive probability and thus $T_i = infty$ with positive probability. (Which implies $mathbb{E}[T_i] = infty$).



            One reference you may find useful is these lecture notes.







            share|cite|improve this answer














            share|cite|improve this answer



            share|cite|improve this answer








            edited Jan 8 at 23:42

























            answered Jan 8 at 23:36









            Will BrannonWill Brannon

            111




            111












            • $begingroup$
              Your definition of transience is wrong. Most transient Markov chains are such that $ito j$ and $jto i$ for every $i$ and $j$ in their state space.
              $endgroup$
              – Did
              Jan 9 at 0:06










            • $begingroup$
              But only in the case of an infinite state space, right? I was only talking about the finite-state case (and probably should have said that more clearly).
              $endgroup$
              – Will Brannon
              Jan 9 at 3:52










            • $begingroup$
              Then you are describing (non-)communicability between states rather than transience.
              $endgroup$
              – Did
              Jan 9 at 6:48


















            • $begingroup$
              Your definition of transience is wrong. Most transient Markov chains are such that $ito j$ and $jto i$ for every $i$ and $j$ in their state space.
              $endgroup$
              – Did
              Jan 9 at 0:06










            • $begingroup$
              But only in the case of an infinite state space, right? I was only talking about the finite-state case (and probably should have said that more clearly).
              $endgroup$
              – Will Brannon
              Jan 9 at 3:52










            • $begingroup$
              Then you are describing (non-)communicability between states rather than transience.
              $endgroup$
              – Did
              Jan 9 at 6:48
















            $begingroup$
            Your definition of transience is wrong. Most transient Markov chains are such that $ito j$ and $jto i$ for every $i$ and $j$ in their state space.
            $endgroup$
            – Did
            Jan 9 at 0:06




            $begingroup$
            Your definition of transience is wrong. Most transient Markov chains are such that $ito j$ and $jto i$ for every $i$ and $j$ in their state space.
            $endgroup$
            – Did
            Jan 9 at 0:06












            $begingroup$
            But only in the case of an infinite state space, right? I was only talking about the finite-state case (and probably should have said that more clearly).
            $endgroup$
            – Will Brannon
            Jan 9 at 3:52




            $begingroup$
            But only in the case of an infinite state space, right? I was only talking about the finite-state case (and probably should have said that more clearly).
            $endgroup$
            – Will Brannon
            Jan 9 at 3:52












            $begingroup$
            Then you are describing (non-)communicability between states rather than transience.
            $endgroup$
            – Did
            Jan 9 at 6:48




            $begingroup$
            Then you are describing (non-)communicability between states rather than transience.
            $endgroup$
            – Did
            Jan 9 at 6:48


















            draft saved

            draft discarded




















































            Thanks for contributing an answer to Mathematics Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2998632%2fhow-do-i-show-that-the-mean-recurrence-time-for-transient-states-is-infinity%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Human spaceflight

            Image
            Human spaceflight From Wikipedia, the free encyclopedia Jump to navigation Jump to search "Space traveler" redirects here. For other uses, see Space traveler (disambiguation). @media all and (max-width:720px){.mw-parser-output .tmulti>.thumbinner{width:100%!important;max-width:none!important}.mw-parser-output .tmulti .tsingle{float:none!important;max-width:none!important;width:100%!important;text-align:center}} Apollo 11 crewmember Buzz Aldrin on the Moon, 1969 International Space Station crewmember Tracy Caldwell Dyson views the Earth, 2010 Space Shuttle Discovery heads into space with a crew aboard, STS-121 in 2006 Inside a space suit on the Canadarm, 1993 Human spaceflight (also referred to as crewed spaceflight or manned spaceflight ) is space travel with a crew or passengers aboard the spacecraft. Spacecraft carrying people may be operated directly, by human crew, or it may be e
            Read more

            Can not write log (Is /dev/pts mounted?) - openpty in Ubuntu-on-Windows?

            Image
            0 1 What exactly does the error message beginning with E: mean? I assume it has to do with the file structure on Ubuntu-on-Windows, but what exactly? The following package was automatically installed and is no longer required: os-prober Use 'apt-get autoremove' to remove it. The following extra packages will be installed: libxslt1.1 The following NEW packages will be installed: libxslt1.1 xmlstarlet 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded. Need to get 435 kB of archives. After this operation, 1023 kB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu/ trusty/main libxslt1.1 amd64 1.1.28-2build1 [145 kB] Get:2 http://archive.ubuntu.com/ubuntu/ trusty/universe xmlstarlet amd64 1.5.0-1 [290 kB]
            Read more

            File:DeusFollowingSea.jpg

            Image
            File:DeusFollowingSea.jpg From Wikipedia, the free encyclopedia Jump to navigation Jump to search File File history File usage No higher resolution available. DeusFollowingSea.jpg ‎ (238 × 211 pixels, file size: 12 KB, MIME type: image/jpeg ) Summary [ edit ] Media data and Non-free use rationale Description Cover art of the Deus CD Following Sea Author or copyright owner Deus Source (WP:NFCC#4) rocksucker.co.uk Date of publication 2012 Use in article (WP:NFCC#7) Following Sea Purpose of use in article (WP:NFCC#8) to serve as the primary means of visual identification at the top of the article dedicated to the work in question. Not replaceable with free media because (WP:NFCC#1) n.a. Minimal use (WP:NFCC#3) Low res image used in infobox of the article on the album Respect for commercial opportunities (WP:NFCC#2) n.a. Fair use Fair use of co
            Read more