How to show two different definitions of $alpha$-strongly convex are equivalent?












3












$begingroup$


In my optimization lecture notes I have the following definition:



Definition: Let $f: mathbb{R}^n rightarrow mathbb{R}$ be differentiable. The $f$ is strongly convex it $exists$ a positive constant $alpha > 0$ such that
$$
langle nabla f(y) - nabla f(x),y-x rangle geq alpha||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{1}
$$
However, in the Online Linear Optimization via Smoothing on page 2, it says the function is $beta$-strongly convex if



$$
f(y) -f(x) - langle nabla f(x),y-x rangle geq frac{beta}{2}||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{2}
$$



How can we show that (1) is equivalent to (2) with appropriate choice of $beta$?






linear-algebra optimization convex-analysis convex-optimization







share|cite|improve this question











$endgroup$

















    3












    $begingroup$


    In my optimization lecture notes I have the following definition:



    Definition: Let $f: mathbb{R}^n rightarrow mathbb{R}$ be differentiable. The $f$ is strongly convex it $exists$ a positive constant $alpha > 0$ such that
    $$
    langle nabla f(y) - nabla f(x),y-x rangle geq alpha||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{1}
    $$
    However, in the Online Linear Optimization via Smoothing on page 2, it says the function is $beta$-strongly convex if



    $$
    f(y) -f(x) - langle nabla f(x),y-x rangle geq frac{beta}{2}||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{2}
    $$



    How can we show that (1) is equivalent to (2) with appropriate choice of $beta$?






    linear-algebra optimization convex-analysis convex-optimization







    share|cite|improve this question











    $endgroup$















      3












      3








      3


      1



      $begingroup$


      In my optimization lecture notes I have the following definition:



      Definition: Let $f: mathbb{R}^n rightarrow mathbb{R}$ be differentiable. The $f$ is strongly convex it $exists$ a positive constant $alpha > 0$ such that
      $$
      langle nabla f(y) - nabla f(x),y-x rangle geq alpha||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{1}
      $$
      However, in the Online Linear Optimization via Smoothing on page 2, it says the function is $beta$-strongly convex if



      $$
      f(y) -f(x) - langle nabla f(x),y-x rangle geq frac{beta}{2}||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{2}
      $$



      How can we show that (1) is equivalent to (2) with appropriate choice of $beta$?






      linear-algebra optimization convex-analysis convex-optimization







      share|cite|improve this question











      $endgroup$




      In my optimization lecture notes I have the following definition:



      Definition: Let $f: mathbb{R}^n rightarrow mathbb{R}$ be differentiable. The $f$ is strongly convex it $exists$ a positive constant $alpha > 0$ such that
      $$
      langle nabla f(y) - nabla f(x),y-x rangle geq alpha||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{1}
      $$
      However, in the Online Linear Optimization via Smoothing on page 2, it says the function is $beta$-strongly convex if



      $$
      f(y) -f(x) - langle nabla f(x),y-x rangle geq frac{beta}{2}||y-x||^2 ,,,,,,,,,forall x,y in mathbb{R}^n tag{2}
      $$



      How can we show that (1) is equivalent to (2) with appropriate choice of $beta$?






      linear-algebra optimization convex-analysis convex-optimization




      linear-algebra optimization convex-analysis convex-optimization






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited Jan 11 at 8:21









      gerw

      19.6k11334




      19.6k11334










      asked Jan 10 at 22:56









      SaeedSaeed

      1,080310




      1,080310






















          2 Answers
          2






          active

          oldest

          votes


















          3












          $begingroup$

          Let’s take $(1)$ and write $g_1=f-frac{alpha}{2}|cdot|_2^2$. Then $(1)$ is equivalent to $langle nabla g_1(y)-nabla g_1(x),,y-x rangle geq 0$, ie $g_1$ convex.



          Do the same for $(2)$ with $g_2= f-frac{beta}{2}|cdot|_2^2$.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            In (1) there is no $f$, we have $nabla f$, could you write what you mean. I do not think it is as easy as that your saying.
            $endgroup$
            – Saeed
            Jan 11 at 13:42



















          0












          $begingroup$

          I will post the following derivation which is expressed in Mindlak's answer just to have better understanding:



          From (1) we have:



          $$
          langle nabla f(y) -nabla f(x),y-x rangle geq alpha||y-x||^2=
          alpha langle y - x,y-x rangle,,,,,,,,,forall x,y in mathbb{R}^n
          $$
          So,
          $$
          langle nabla f(y) - alpha y -(nabla f(x)-alpha x),y-x rangle geq 0,,,,,,,,,forall x,y in mathbb{R}^n tag{3}
          $$
          (3) means $g_1(x)=f(x)-frac{alpha}{2} |x|_2^2$ is convex, because it is the monotonicity property of convex function. From
          $$
          g_1(s) geq g_1(t) + langle nabla g_1(t),s-t rangle ,,,,,,,,,forall s,t in mathbb{R}^n
          $$
          we have



          $$
          f(y)-frac{alpha}{2} |y|_2^2 geq f(x)-frac{alpha}{2} |x|_2^2 + langle nabla f(x)-2alpha x,y-x rangle ,,,,,,,,,forall y,x in mathbb{R}^n
          $$
          Yields



          $$
          f(y)-f(x)+ langle nabla f(x),y-x rangle geq -frac{alpha}{2} |x|_2^2 +frac{alpha}{2} |y|_2^2 +langle -alpha x,y-x rangle =frac{alpha}{2} |y-x|_2^2
          \
          forall ,,,,,y,x in mathbb{R}^n
          $$
          and $alpha = beta$






          share|cite|improve this answer











          $endgroup$













            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "69"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            noCode: true, onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3069280%2fhow-to-show-two-different-definitions-of-alpha-strongly-convex-are-equivalent%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            3












            $begingroup$

            Let’s take $(1)$ and write $g_1=f-frac{alpha}{2}|cdot|_2^2$. Then $(1)$ is equivalent to $langle nabla g_1(y)-nabla g_1(x),,y-x rangle geq 0$, ie $g_1$ convex.



            Do the same for $(2)$ with $g_2= f-frac{beta}{2}|cdot|_2^2$.






            share|cite|improve this answer











            $endgroup$













            • $begingroup$
              In (1) there is no $f$, we have $nabla f$, could you write what you mean. I do not think it is as easy as that your saying.
              $endgroup$
              – Saeed
              Jan 11 at 13:42
















            3












            $begingroup$

            Let’s take $(1)$ and write $g_1=f-frac{alpha}{2}|cdot|_2^2$. Then $(1)$ is equivalent to $langle nabla g_1(y)-nabla g_1(x),,y-x rangle geq 0$, ie $g_1$ convex.



            Do the same for $(2)$ with $g_2= f-frac{beta}{2}|cdot|_2^2$.






            share|cite|improve this answer











            $endgroup$













            • $begingroup$
              In (1) there is no $f$, we have $nabla f$, could you write what you mean. I do not think it is as easy as that your saying.
              $endgroup$
              – Saeed
              Jan 11 at 13:42














            3












            3








            3





            $begingroup$

            Let’s take $(1)$ and write $g_1=f-frac{alpha}{2}|cdot|_2^2$. Then $(1)$ is equivalent to $langle nabla g_1(y)-nabla g_1(x),,y-x rangle geq 0$, ie $g_1$ convex.



            Do the same for $(2)$ with $g_2= f-frac{beta}{2}|cdot|_2^2$.






            share|cite|improve this answer











            $endgroup$



            Let’s take $(1)$ and write $g_1=f-frac{alpha}{2}|cdot|_2^2$. Then $(1)$ is equivalent to $langle nabla g_1(y)-nabla g_1(x),,y-x rangle geq 0$, ie $g_1$ convex.



            Do the same for $(2)$ with $g_2= f-frac{beta}{2}|cdot|_2^2$.







            share|cite|improve this answer














            share|cite|improve this answer



            share|cite|improve this answer








            edited Jan 11 at 13:53

























            answered Jan 11 at 0:53









            MindlackMindlack

            4,830210




            4,830210












            • $begingroup$
              In (1) there is no $f$, we have $nabla f$, could you write what you mean. I do not think it is as easy as that your saying.
              $endgroup$
              – Saeed
              Jan 11 at 13:42


















            • $begingroup$
              In (1) there is no $f$, we have $nabla f$, could you write what you mean. I do not think it is as easy as that your saying.
              $endgroup$
              – Saeed
              Jan 11 at 13:42
















            $begingroup$
            In (1) there is no $f$, we have $nabla f$, could you write what you mean. I do not think it is as easy as that your saying.
            $endgroup$
            – Saeed
            Jan 11 at 13:42




            $begingroup$
            In (1) there is no $f$, we have $nabla f$, could you write what you mean. I do not think it is as easy as that your saying.
            $endgroup$
            – Saeed
            Jan 11 at 13:42











            0












            $begingroup$

            I will post the following derivation which is expressed in Mindlak's answer just to have better understanding:



            From (1) we have:



            $$
            langle nabla f(y) -nabla f(x),y-x rangle geq alpha||y-x||^2=
            alpha langle y - x,y-x rangle,,,,,,,,,forall x,y in mathbb{R}^n
            $$
            So,
            $$
            langle nabla f(y) - alpha y -(nabla f(x)-alpha x),y-x rangle geq 0,,,,,,,,,forall x,y in mathbb{R}^n tag{3}
            $$
            (3) means $g_1(x)=f(x)-frac{alpha}{2} |x|_2^2$ is convex, because it is the monotonicity property of convex function. From
            $$
            g_1(s) geq g_1(t) + langle nabla g_1(t),s-t rangle ,,,,,,,,,forall s,t in mathbb{R}^n
            $$
            we have



            $$
            f(y)-frac{alpha}{2} |y|_2^2 geq f(x)-frac{alpha}{2} |x|_2^2 + langle nabla f(x)-2alpha x,y-x rangle ,,,,,,,,,forall y,x in mathbb{R}^n
            $$
            Yields



            $$
            f(y)-f(x)+ langle nabla f(x),y-x rangle geq -frac{alpha}{2} |x|_2^2 +frac{alpha}{2} |y|_2^2 +langle -alpha x,y-x rangle =frac{alpha}{2} |y-x|_2^2
            \
            forall ,,,,,y,x in mathbb{R}^n
            $$
            and $alpha = beta$






            share|cite|improve this answer











            $endgroup$


















              0












              $begingroup$

              I will post the following derivation which is expressed in Mindlak's answer just to have better understanding:



              From (1) we have:



              $$
              langle nabla f(y) -nabla f(x),y-x rangle geq alpha||y-x||^2=
              alpha langle y - x,y-x rangle,,,,,,,,,forall x,y in mathbb{R}^n
              $$
              So,
              $$
              langle nabla f(y) - alpha y -(nabla f(x)-alpha x),y-x rangle geq 0,,,,,,,,,forall x,y in mathbb{R}^n tag{3}
              $$
              (3) means $g_1(x)=f(x)-frac{alpha}{2} |x|_2^2$ is convex, because it is the monotonicity property of convex function. From
              $$
              g_1(s) geq g_1(t) + langle nabla g_1(t),s-t rangle ,,,,,,,,,forall s,t in mathbb{R}^n
              $$
              we have



              $$
              f(y)-frac{alpha}{2} |y|_2^2 geq f(x)-frac{alpha}{2} |x|_2^2 + langle nabla f(x)-2alpha x,y-x rangle ,,,,,,,,,forall y,x in mathbb{R}^n
              $$
              Yields



              $$
              f(y)-f(x)+ langle nabla f(x),y-x rangle geq -frac{alpha}{2} |x|_2^2 +frac{alpha}{2} |y|_2^2 +langle -alpha x,y-x rangle =frac{alpha}{2} |y-x|_2^2
              \
              forall ,,,,,y,x in mathbb{R}^n
              $$
              and $alpha = beta$






              share|cite|improve this answer











              $endgroup$
















                0












                0








                0





                $begingroup$

                I will post the following derivation which is expressed in Mindlak's answer just to have better understanding:



                From (1) we have:



                $$
                langle nabla f(y) -nabla f(x),y-x rangle geq alpha||y-x||^2=
                alpha langle y - x,y-x rangle,,,,,,,,,forall x,y in mathbb{R}^n
                $$
                So,
                $$
                langle nabla f(y) - alpha y -(nabla f(x)-alpha x),y-x rangle geq 0,,,,,,,,,forall x,y in mathbb{R}^n tag{3}
                $$
                (3) means $g_1(x)=f(x)-frac{alpha}{2} |x|_2^2$ is convex, because it is the monotonicity property of convex function. From
                $$
                g_1(s) geq g_1(t) + langle nabla g_1(t),s-t rangle ,,,,,,,,,forall s,t in mathbb{R}^n
                $$
                we have



                $$
                f(y)-frac{alpha}{2} |y|_2^2 geq f(x)-frac{alpha}{2} |x|_2^2 + langle nabla f(x)-2alpha x,y-x rangle ,,,,,,,,,forall y,x in mathbb{R}^n
                $$
                Yields



                $$
                f(y)-f(x)+ langle nabla f(x),y-x rangle geq -frac{alpha}{2} |x|_2^2 +frac{alpha}{2} |y|_2^2 +langle -alpha x,y-x rangle =frac{alpha}{2} |y-x|_2^2
                \
                forall ,,,,,y,x in mathbb{R}^n
                $$
                and $alpha = beta$






                share|cite|improve this answer











                $endgroup$



                I will post the following derivation which is expressed in Mindlak's answer just to have better understanding:



                From (1) we have:



                $$
                langle nabla f(y) -nabla f(x),y-x rangle geq alpha||y-x||^2=
                alpha langle y - x,y-x rangle,,,,,,,,,forall x,y in mathbb{R}^n
                $$
                So,
                $$
                langle nabla f(y) - alpha y -(nabla f(x)-alpha x),y-x rangle geq 0,,,,,,,,,forall x,y in mathbb{R}^n tag{3}
                $$
                (3) means $g_1(x)=f(x)-frac{alpha}{2} |x|_2^2$ is convex, because it is the monotonicity property of convex function. From
                $$
                g_1(s) geq g_1(t) + langle nabla g_1(t),s-t rangle ,,,,,,,,,forall s,t in mathbb{R}^n
                $$
                we have



                $$
                f(y)-frac{alpha}{2} |y|_2^2 geq f(x)-frac{alpha}{2} |x|_2^2 + langle nabla f(x)-2alpha x,y-x rangle ,,,,,,,,,forall y,x in mathbb{R}^n
                $$
                Yields



                $$
                f(y)-f(x)+ langle nabla f(x),y-x rangle geq -frac{alpha}{2} |x|_2^2 +frac{alpha}{2} |y|_2^2 +langle -alpha x,y-x rangle =frac{alpha}{2} |y-x|_2^2
                \
                forall ,,,,,y,x in mathbb{R}^n
                $$
                and $alpha = beta$







                share|cite|improve this answer














                share|cite|improve this answer



                share|cite|improve this answer








                edited Jan 17 at 17:32

























                answered Jan 11 at 15:20









                SaeedSaeed

                1,080310




                1,080310






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Mathematics Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3069280%2fhow-to-show-two-different-definitions-of-alpha-strongly-convex-are-equivalent%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    Questions related to Moebius Transform of Characteristic Function of the Primes

                    Image
                    2 $begingroup$ Consider the function defined in (1) below related to the fundamental prime counting function $pi(x)$ . Note that A143519(n) is not multiplicative. (1) $quad f(x)=sumlimits_{n=1}^{x}A143519(n)$ https://oeis.org/A143519 The following plot illustrates $f(x)$ defined in formula (1) above. Figure (1) : Illustration of $f(x)$ defined in formula (1) The integer zeros of $f(x)$ for $xle 10,000$ are listed in (2) below. (2) $quad$ {1,6,9,12,19,30,79,80,81,116,193,201,287,288,291,668,673,679,680,685,686,1109} The zero crossings of $f(x)$ for $xle 10,000$ where $f(x)$ doesn't settle at zero are listed in (3) below. (3) $quad$ {14,21,33,114,115,118,195,286,290,295,442,445,665,667,670,671,678,682} Question (1) : Does $f(x)$ have a finite number of integer zeros, and if so ...
                    Read more

                    List of scandals in India

                    Image
                    List of scandals in India From Wikipedia, the free encyclopedia Jump to navigation Jump to search This is a dynamic list and may never be able to satisfy particular standards for completeness. You can help by expanding it with reliably sourced entries. This article has multiple issues. Please help improve it or discuss these issues on the talk page . (Learn how and when to remove these template messages) This article has an unclear citation style . The references used may be made clearer with a different or consistent style of citation and footnoting. ( January 2014 ) (Learn how and when to remove this template message) This list is in reverse order. As a consequence it appears to be slanted towards recent events. Please try to keep recent events in historical perspective and add more content related to non-recent events. ( May 2013 ) (Learn how and when to remove this template message) ...
                    Read more

                    Can not write log (Is /dev/pts mounted?) - openpty in Ubuntu-on-Windows?

                    Image
                    0 1 What exactly does the error message beginning with E: mean? I assume it has to do with the file structure on Ubuntu-on-Windows, but what exactly? The following package was automatically installed and is no longer required: os-prober Use 'apt-get autoremove' to remove it. The following extra packages will be installed: libxslt1.1 The following NEW packages will be installed: libxslt1.1 xmlstarlet 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded. Need to get 435 kB of archives. After this operation, 1023 kB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu/ trusty/main libxslt1.1 amd64 1.1.28-2build1 [145 kB] Get:2 http://archive.ubuntu.com/ubuntu/ trusty/universe xmlstarlet amd64 1.5.0-1 [290 kB] ...
                    Read more