python - Matplotlib can't render multiple contour plots on Django

Question

Whenever (at least) 2 people try to generate a contour plot in my application, at least one of them will receive a random error depending on how far the first person managed to draw.. ("unknown element o", "ContourSet must be in current Axes" are just two of the possibilities)

The following is a cut down test that can produce the error, if you try to load this page in 2 or more tabs at once, the first will render correctly whilst the second will produce an error. (Easiest way I found to do this was to click the refresh page button in chrome with the middle mouse button a couple times)

views.py

def home(request):
    return render(request, 'home.html', {'chart': _test_chart()})


def _test_chart():
    import base64
    import cStringIO
    import matplotlib
    matplotlib.use('agg')
    from matplotlib.mlab import bivariate_normal
    import matplotlib.pyplot as plt
    import numpy as np
    from numpy.core.multiarray import arange

    delta = 0.5

    x = arange(-3.0, 4.001, delta)
    y = arange(-4.0, 3.001, delta)
    X, Y = np.meshgrid(x, y)
    Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
    Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
    Z = (Z1 - Z2) * 10

    fig = plt.figure(figsize=(10, 5))
    plt.contour(X, Y, Z, 10, colors='k')

    jpg_image_buffer = cStringIO.StringIO()
    fig.savefig(jpg_image_buffer)

    array = base64.b64encode(jpg_image_buffer.getvalue())
    jpg_image_buffer.close()
    return array

home.html (just this one line is enough)

<img src="data:image/png;base64,{{ chart }}" />

I've tried using mpld3 instead to handle the generation of the image and this still produces different errors so I know its definitely not the saving of the figure but more its generation. I've also tried using a ThreadPool and Threading to no avail, from what I can tell it seems like creating a contour plot in matplotlib cannot support multiple instances which will never work for a website...

My only clear solution I can think of right now is to replace matplotlib with something else which I really don't want to do.

Is there a way to generate contour plots with matplotlib that will work for me?

Answer 1

First, let me start by saying that this is much more easy to reproduce by calling _test_chart in a couple threads

from threading import Thread
for i in xrange(2):
    Thread(target=_test_chart).start()

Doing the above, one will work as desired whilst the second one will crash.

---

The simple reason for this is that the pyplot module is not designed for multithreading and therefore the two charts are getting their data mixed up as they attempt to draw.

This can be better explained by mdboom

...pyplot is used for convenient plotting at the commandline and keeps around global state. For example, when you say plt.figure() it adds the figure to a global list and then sets the "current figure" pointer to the most recently created figure. Then subsequent plotting commands automatically write to that figure. Obviously, that's not threadsafe...

There are two ways to fix this issue,

1. Force these charts to be drawn in different processes.

for i in xrange(2):
    pool = Pool(processes=1)
    pool.apply(_test_chart)   

Whilst this will work you will find that there is a noticable drop in performance since it will often take just as long to create the process as it will to generate the chart (which I didn't think was acceptable!)

2. The real solution is to use the OO interface modules of Matplotlib which will then allow you to work with the correct objects - essentially this works down to working with subplots rather than plots. For the given example in the question, this would look like the following

def _test_chart2():

    delta = 0.5

    x = arange(-3.0, 4.001, delta)
    y = arange(-4.0, 3.001, delta)
    X, Y = np.meshgrid(x, y)
    Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
    Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
    Z = (Z1 - Z2) * 10

    fig = figure(figsize=(10, 5))

    ax1 = fig.add_subplot(111)
    extents = [x.min(), x.max(), y.min(), y.max()]
    im = ax1.imshow(Z,
                    interpolation='spline36',
                    extent=extents,
                    origin='lower',
                    aspect='auto',
                    cmap=cm.jet)
    ax1.contour(X, Y, Z, 10, colors='k')

    jpg_image_buffer = cStringIO.StringIO()
    fig.savefig(jpg_image_buffer)

    array = base64.b64encode(jpg_image_buffer.getvalue())
    jpg_image_buffer.close()

    return array