Python animation : Large amount of data, slow hist animation









up vote
0
down vote

favorite












The iteration update very slow, n+=3 for each time only but my data has 10000 elements.Like, It tries to update every single frame n=1,n=2,n=3.. but the hist function is really power consuming. I don't know if there are any way I could skip frames like from n=1 go straight to n=500 and to n=1000.



import matplotlib.animation as animation
import numpy as np
import matplotlib.pyplot as plt
n=10000
def update(curr):
if curr==n:
a.event_source.stop()
first_histogram.cla()
sec_histogram.cla()
thi_histogram.cla()
for_histogram.cla()
first_histogram.hist(x1[:curr], bins=np.arange(-6,2,0.5))
sec_histogram.hist(x2[:curr], bins=np.arange(-1,15,1))
thi_histogram.hist(x3[:curr], bins=np.arange(2,22,1))
for_histogram.hist(x4[:curr], bins=np.arange(13,21,1))
first_histogram.set_title('n='.format(curr))
fig=plt.figure()
gspec=gridspec.GridSpec(2,2)
first_histogram=plt.subplot(gspec[0,0])
sec_histogram=plt.subplot(gspec[0,1])
thi_histogram=plt.subplot(gspec[1,0])
for_histogram=plt.subplot(gspec[1,1])
a = animation.FuncAnimation(fig,update,blit=True,interval=1,repeat=False)


How can I make it faster ? Thank you!










share|improve this question



























    up vote
    0
    down vote

    favorite












    The iteration update very slow, n+=3 for each time only but my data has 10000 elements.Like, It tries to update every single frame n=1,n=2,n=3.. but the hist function is really power consuming. I don't know if there are any way I could skip frames like from n=1 go straight to n=500 and to n=1000.



    import matplotlib.animation as animation
    import numpy as np
    import matplotlib.pyplot as plt
    n=10000
    def update(curr):
    if curr==n:
    a.event_source.stop()
    first_histogram.cla()
    sec_histogram.cla()
    thi_histogram.cla()
    for_histogram.cla()
    first_histogram.hist(x1[:curr], bins=np.arange(-6,2,0.5))
    sec_histogram.hist(x2[:curr], bins=np.arange(-1,15,1))
    thi_histogram.hist(x3[:curr], bins=np.arange(2,22,1))
    for_histogram.hist(x4[:curr], bins=np.arange(13,21,1))
    first_histogram.set_title('n='.format(curr))
    fig=plt.figure()
    gspec=gridspec.GridSpec(2,2)
    first_histogram=plt.subplot(gspec[0,0])
    sec_histogram=plt.subplot(gspec[0,1])
    thi_histogram=plt.subplot(gspec[1,0])
    for_histogram=plt.subplot(gspec[1,1])
    a = animation.FuncAnimation(fig,update,blit=True,interval=1,repeat=False)


    How can I make it faster ? Thank you!










    share|improve this question

























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      The iteration update very slow, n+=3 for each time only but my data has 10000 elements.Like, It tries to update every single frame n=1,n=2,n=3.. but the hist function is really power consuming. I don't know if there are any way I could skip frames like from n=1 go straight to n=500 and to n=1000.



      import matplotlib.animation as animation
      import numpy as np
      import matplotlib.pyplot as plt
      n=10000
      def update(curr):
      if curr==n:
      a.event_source.stop()
      first_histogram.cla()
      sec_histogram.cla()
      thi_histogram.cla()
      for_histogram.cla()
      first_histogram.hist(x1[:curr], bins=np.arange(-6,2,0.5))
      sec_histogram.hist(x2[:curr], bins=np.arange(-1,15,1))
      thi_histogram.hist(x3[:curr], bins=np.arange(2,22,1))
      for_histogram.hist(x4[:curr], bins=np.arange(13,21,1))
      first_histogram.set_title('n='.format(curr))
      fig=plt.figure()
      gspec=gridspec.GridSpec(2,2)
      first_histogram=plt.subplot(gspec[0,0])
      sec_histogram=plt.subplot(gspec[0,1])
      thi_histogram=plt.subplot(gspec[1,0])
      for_histogram=plt.subplot(gspec[1,1])
      a = animation.FuncAnimation(fig,update,blit=True,interval=1,repeat=False)


      How can I make it faster ? Thank you!










      share|improve this question















      The iteration update very slow, n+=3 for each time only but my data has 10000 elements.Like, It tries to update every single frame n=1,n=2,n=3.. but the hist function is really power consuming. I don't know if there are any way I could skip frames like from n=1 go straight to n=500 and to n=1000.



      import matplotlib.animation as animation
      import numpy as np
      import matplotlib.pyplot as plt
      n=10000
      def update(curr):
      if curr==n:
      a.event_source.stop()
      first_histogram.cla()
      sec_histogram.cla()
      thi_histogram.cla()
      for_histogram.cla()
      first_histogram.hist(x1[:curr], bins=np.arange(-6,2,0.5))
      sec_histogram.hist(x2[:curr], bins=np.arange(-1,15,1))
      thi_histogram.hist(x3[:curr], bins=np.arange(2,22,1))
      for_histogram.hist(x4[:curr], bins=np.arange(13,21,1))
      first_histogram.set_title('n='.format(curr))
      fig=plt.figure()
      gspec=gridspec.GridSpec(2,2)
      first_histogram=plt.subplot(gspec[0,0])
      sec_histogram=plt.subplot(gspec[0,1])
      thi_histogram=plt.subplot(gspec[1,0])
      for_histogram=plt.subplot(gspec[1,1])
      a = animation.FuncAnimation(fig,update,blit=True,interval=1,repeat=False)


      How can I make it faster ? Thank you!







      python animation matplotlib






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 10 at 3:37









      eyllanesc

      68.7k93052




      68.7k93052










      asked Nov 10 at 3:36









      Patrick Nguyen

      31




      31






















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          0
          down vote



          accepted










          There are several things to note here.



          blit=True is not useful when clearing the axes in between. It would either not take effect, or you would get wrong tick labels on the axes.
          It would only be useful if the axes limits do not change from frame to frame. However in a normal histogram, where more and more data is animated, this would necessarily need to be the case, else your bars either grow out of the axes, or you do not see the low numbers at the start. As an alternative, you could plot a normalized histogram (i.e. a density plot).



          Also, interval=1 is not useful. You will not be able to animate 4 subplots with a 1 millisecond frame rate on any normal system. Matplotlib is too slow for that. However, consider that the human brain can usually not resolve framerates above some 25 fps, i.e. 40 ms, anyways. That's probably the frame rate to aim at (although matplotlib may not achieve that)



          So a way to set this up is simply via



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)

          def update(curr):
          ax.clear()
          ax.hist(x1[:curr], bins=np.arange(-6,2,0.5))
          ax.set_title('n='.format(curr))

          fig, ax = plt.subplots()
          a = animation.FuncAnimation(fig, update, frames=len(x1), interval=40, repeat=False, blit=False)

          plt.show()


          If you feel like you want to arrive more quickly at the final number of items in the list, use less frames. E.g. for a 25 times faster animation, show only every 25th state,



          a = animation.FuncAnimation(fig, update, frames=np.arange(0, len(x1)+1, 25),
          interval=40, repeat=False, blit=False)


          This code runs with a framerate of 11 fps (interval of ~85 ms), so it's slower than specified, which in turn means, we could directly set interval=85.



          In order to increase the frame rate one may use blitting.
          For that, you will need to not update the axes limits at all. To optimize further you may precompute all the histograms to show. Note however that the axes limits should then not change, so we set them at the beginning, which leads to a different plot.



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)
          bins = np.arange(-6,2,0.5)
          hist = np.empty((len(x1), len(bins)-1))
          for i in range(len(x1)):
          hist[i, :], _ = np.histogram(x1[:i], bins=bins)

          def update(i):
          for bar, y in zip(bars, hist[i,:]):
          bar.set_height(y)
          text.set_text('n='.format(i))
          return list(bars) + [text]

          fig, ax = plt.subplots()
          ax.set_ylim(0,hist.max()*1.05)
          bars = ax.bar(bins[:-1], hist[0,:], width=np.diff(bins), align="edge")
          text = ax.text(.99,.99, "", ha="right", va="top", transform=ax.transAxes)

          ani = animation.FuncAnimation(fig, update, frames=len(x1), interval=1, repeat=False, blit=True)

          plt.show()


          Running this code give me a framerate of 215 fps, (4.6 ms per frame), so we could set the interval to 4.6 ms.






          share|improve this answer




















          • Thank you, you really nailed it and gave me a lot of insights!
            – Patrick Nguyen
            Nov 11 at 17:57










          Your Answer






          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "1"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













           

          draft saved


          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53235798%2fpython-animation-large-amount-of-data-slow-hist-animation%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          0
          down vote



          accepted










          There are several things to note here.



          blit=True is not useful when clearing the axes in between. It would either not take effect, or you would get wrong tick labels on the axes.
          It would only be useful if the axes limits do not change from frame to frame. However in a normal histogram, where more and more data is animated, this would necessarily need to be the case, else your bars either grow out of the axes, or you do not see the low numbers at the start. As an alternative, you could plot a normalized histogram (i.e. a density plot).



          Also, interval=1 is not useful. You will not be able to animate 4 subplots with a 1 millisecond frame rate on any normal system. Matplotlib is too slow for that. However, consider that the human brain can usually not resolve framerates above some 25 fps, i.e. 40 ms, anyways. That's probably the frame rate to aim at (although matplotlib may not achieve that)



          So a way to set this up is simply via



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)

          def update(curr):
          ax.clear()
          ax.hist(x1[:curr], bins=np.arange(-6,2,0.5))
          ax.set_title('n='.format(curr))

          fig, ax = plt.subplots()
          a = animation.FuncAnimation(fig, update, frames=len(x1), interval=40, repeat=False, blit=False)

          plt.show()


          If you feel like you want to arrive more quickly at the final number of items in the list, use less frames. E.g. for a 25 times faster animation, show only every 25th state,



          a = animation.FuncAnimation(fig, update, frames=np.arange(0, len(x1)+1, 25),
          interval=40, repeat=False, blit=False)


          This code runs with a framerate of 11 fps (interval of ~85 ms), so it's slower than specified, which in turn means, we could directly set interval=85.



          In order to increase the frame rate one may use blitting.
          For that, you will need to not update the axes limits at all. To optimize further you may precompute all the histograms to show. Note however that the axes limits should then not change, so we set them at the beginning, which leads to a different plot.



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)
          bins = np.arange(-6,2,0.5)
          hist = np.empty((len(x1), len(bins)-1))
          for i in range(len(x1)):
          hist[i, :], _ = np.histogram(x1[:i], bins=bins)

          def update(i):
          for bar, y in zip(bars, hist[i,:]):
          bar.set_height(y)
          text.set_text('n='.format(i))
          return list(bars) + [text]

          fig, ax = plt.subplots()
          ax.set_ylim(0,hist.max()*1.05)
          bars = ax.bar(bins[:-1], hist[0,:], width=np.diff(bins), align="edge")
          text = ax.text(.99,.99, "", ha="right", va="top", transform=ax.transAxes)

          ani = animation.FuncAnimation(fig, update, frames=len(x1), interval=1, repeat=False, blit=True)

          plt.show()


          Running this code give me a framerate of 215 fps, (4.6 ms per frame), so we could set the interval to 4.6 ms.






          share|improve this answer




















          • Thank you, you really nailed it and gave me a lot of insights!
            – Patrick Nguyen
            Nov 11 at 17:57














          up vote
          0
          down vote



          accepted










          There are several things to note here.



          blit=True is not useful when clearing the axes in between. It would either not take effect, or you would get wrong tick labels on the axes.
          It would only be useful if the axes limits do not change from frame to frame. However in a normal histogram, where more and more data is animated, this would necessarily need to be the case, else your bars either grow out of the axes, or you do not see the low numbers at the start. As an alternative, you could plot a normalized histogram (i.e. a density plot).



          Also, interval=1 is not useful. You will not be able to animate 4 subplots with a 1 millisecond frame rate on any normal system. Matplotlib is too slow for that. However, consider that the human brain can usually not resolve framerates above some 25 fps, i.e. 40 ms, anyways. That's probably the frame rate to aim at (although matplotlib may not achieve that)



          So a way to set this up is simply via



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)

          def update(curr):
          ax.clear()
          ax.hist(x1[:curr], bins=np.arange(-6,2,0.5))
          ax.set_title('n='.format(curr))

          fig, ax = plt.subplots()
          a = animation.FuncAnimation(fig, update, frames=len(x1), interval=40, repeat=False, blit=False)

          plt.show()


          If you feel like you want to arrive more quickly at the final number of items in the list, use less frames. E.g. for a 25 times faster animation, show only every 25th state,



          a = animation.FuncAnimation(fig, update, frames=np.arange(0, len(x1)+1, 25),
          interval=40, repeat=False, blit=False)


          This code runs with a framerate of 11 fps (interval of ~85 ms), so it's slower than specified, which in turn means, we could directly set interval=85.



          In order to increase the frame rate one may use blitting.
          For that, you will need to not update the axes limits at all. To optimize further you may precompute all the histograms to show. Note however that the axes limits should then not change, so we set them at the beginning, which leads to a different plot.



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)
          bins = np.arange(-6,2,0.5)
          hist = np.empty((len(x1), len(bins)-1))
          for i in range(len(x1)):
          hist[i, :], _ = np.histogram(x1[:i], bins=bins)

          def update(i):
          for bar, y in zip(bars, hist[i,:]):
          bar.set_height(y)
          text.set_text('n='.format(i))
          return list(bars) + [text]

          fig, ax = plt.subplots()
          ax.set_ylim(0,hist.max()*1.05)
          bars = ax.bar(bins[:-1], hist[0,:], width=np.diff(bins), align="edge")
          text = ax.text(.99,.99, "", ha="right", va="top", transform=ax.transAxes)

          ani = animation.FuncAnimation(fig, update, frames=len(x1), interval=1, repeat=False, blit=True)

          plt.show()


          Running this code give me a framerate of 215 fps, (4.6 ms per frame), so we could set the interval to 4.6 ms.






          share|improve this answer




















          • Thank you, you really nailed it and gave me a lot of insights!
            – Patrick Nguyen
            Nov 11 at 17:57












          up vote
          0
          down vote



          accepted







          up vote
          0
          down vote



          accepted






          There are several things to note here.



          blit=True is not useful when clearing the axes in between. It would either not take effect, or you would get wrong tick labels on the axes.
          It would only be useful if the axes limits do not change from frame to frame. However in a normal histogram, where more and more data is animated, this would necessarily need to be the case, else your bars either grow out of the axes, or you do not see the low numbers at the start. As an alternative, you could plot a normalized histogram (i.e. a density plot).



          Also, interval=1 is not useful. You will not be able to animate 4 subplots with a 1 millisecond frame rate on any normal system. Matplotlib is too slow for that. However, consider that the human brain can usually not resolve framerates above some 25 fps, i.e. 40 ms, anyways. That's probably the frame rate to aim at (although matplotlib may not achieve that)



          So a way to set this up is simply via



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)

          def update(curr):
          ax.clear()
          ax.hist(x1[:curr], bins=np.arange(-6,2,0.5))
          ax.set_title('n='.format(curr))

          fig, ax = plt.subplots()
          a = animation.FuncAnimation(fig, update, frames=len(x1), interval=40, repeat=False, blit=False)

          plt.show()


          If you feel like you want to arrive more quickly at the final number of items in the list, use less frames. E.g. for a 25 times faster animation, show only every 25th state,



          a = animation.FuncAnimation(fig, update, frames=np.arange(0, len(x1)+1, 25),
          interval=40, repeat=False, blit=False)


          This code runs with a framerate of 11 fps (interval of ~85 ms), so it's slower than specified, which in turn means, we could directly set interval=85.



          In order to increase the frame rate one may use blitting.
          For that, you will need to not update the axes limits at all. To optimize further you may precompute all the histograms to show. Note however that the axes limits should then not change, so we set them at the beginning, which leads to a different plot.



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)
          bins = np.arange(-6,2,0.5)
          hist = np.empty((len(x1), len(bins)-1))
          for i in range(len(x1)):
          hist[i, :], _ = np.histogram(x1[:i], bins=bins)

          def update(i):
          for bar, y in zip(bars, hist[i,:]):
          bar.set_height(y)
          text.set_text('n='.format(i))
          return list(bars) + [text]

          fig, ax = plt.subplots()
          ax.set_ylim(0,hist.max()*1.05)
          bars = ax.bar(bins[:-1], hist[0,:], width=np.diff(bins), align="edge")
          text = ax.text(.99,.99, "", ha="right", va="top", transform=ax.transAxes)

          ani = animation.FuncAnimation(fig, update, frames=len(x1), interval=1, repeat=False, blit=True)

          plt.show()


          Running this code give me a framerate of 215 fps, (4.6 ms per frame), so we could set the interval to 4.6 ms.






          share|improve this answer












          There are several things to note here.



          blit=True is not useful when clearing the axes in between. It would either not take effect, or you would get wrong tick labels on the axes.
          It would only be useful if the axes limits do not change from frame to frame. However in a normal histogram, where more and more data is animated, this would necessarily need to be the case, else your bars either grow out of the axes, or you do not see the low numbers at the start. As an alternative, you could plot a normalized histogram (i.e. a density plot).



          Also, interval=1 is not useful. You will not be able to animate 4 subplots with a 1 millisecond frame rate on any normal system. Matplotlib is too slow for that. However, consider that the human brain can usually not resolve framerates above some 25 fps, i.e. 40 ms, anyways. That's probably the frame rate to aim at (although matplotlib may not achieve that)



          So a way to set this up is simply via



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)

          def update(curr):
          ax.clear()
          ax.hist(x1[:curr], bins=np.arange(-6,2,0.5))
          ax.set_title('n='.format(curr))

          fig, ax = plt.subplots()
          a = animation.FuncAnimation(fig, update, frames=len(x1), interval=40, repeat=False, blit=False)

          plt.show()


          If you feel like you want to arrive more quickly at the final number of items in the list, use less frames. E.g. for a 25 times faster animation, show only every 25th state,



          a = animation.FuncAnimation(fig, update, frames=np.arange(0, len(x1)+1, 25),
          interval=40, repeat=False, blit=False)


          This code runs with a framerate of 11 fps (interval of ~85 ms), so it's slower than specified, which in turn means, we could directly set interval=85.



          In order to increase the frame rate one may use blitting.
          For that, you will need to not update the axes limits at all. To optimize further you may precompute all the histograms to show. Note however that the axes limits should then not change, so we set them at the beginning, which leads to a different plot.



          import numpy as np
          import matplotlib.pyplot as plt
          import matplotlib.animation as animation

          x1 = np.random.normal(-2.5, 1, 10000)
          bins = np.arange(-6,2,0.5)
          hist = np.empty((len(x1), len(bins)-1))
          for i in range(len(x1)):
          hist[i, :], _ = np.histogram(x1[:i], bins=bins)

          def update(i):
          for bar, y in zip(bars, hist[i,:]):
          bar.set_height(y)
          text.set_text('n='.format(i))
          return list(bars) + [text]

          fig, ax = plt.subplots()
          ax.set_ylim(0,hist.max()*1.05)
          bars = ax.bar(bins[:-1], hist[0,:], width=np.diff(bins), align="edge")
          text = ax.text(.99,.99, "", ha="right", va="top", transform=ax.transAxes)

          ani = animation.FuncAnimation(fig, update, frames=len(x1), interval=1, repeat=False, blit=True)

          plt.show()


          Running this code give me a framerate of 215 fps, (4.6 ms per frame), so we could set the interval to 4.6 ms.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 11 at 0:41









          ImportanceOfBeingErnest

          119k10119192




          119k10119192











          • Thank you, you really nailed it and gave me a lot of insights!
            – Patrick Nguyen
            Nov 11 at 17:57
















          • Thank you, you really nailed it and gave me a lot of insights!
            – Patrick Nguyen
            Nov 11 at 17:57















          Thank you, you really nailed it and gave me a lot of insights!
          – Patrick Nguyen
          Nov 11 at 17:57




          Thank you, you really nailed it and gave me a lot of insights!
          – Patrick Nguyen
          Nov 11 at 17:57

















           

          draft saved


          draft discarded















































           


          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53235798%2fpython-animation-large-amount-of-data-slow-hist-animation%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          這個網誌中的熱門文章

          How to read a connectionString WITH PROVIDER in .NET Core?

          In R, how to develop a multiplot heatmap.2 figure showing key labels successfully

          Museum of Modern and Contemporary Art of Trento and Rovereto