Area weighted regridder caching

[tv3141]

Area weighted averaging is quite expensive. According to the Iris documentation it should be possible to 'cache' the regridder : 

http://scitools.org.uk/iris/docs/latest/userguide/interpolation_and_regridding.html?highlight=regridding#caching-a-regridder

However, I could not observe any benefits in doing that. The cost of setting up the regridder is negligible.
An example showing this by regridding a single field is below. The only thing that seems to be cached is an uninitialised chunk of memory.

import numpy as np
import iris
print iris.__version__
from time import time

import matplotlib.pyplot as plt

import iris
import iris.quickplot as qplt
%matplotlib inline

resolution = {
    'N24':   (  24 * 3/4. * 2, 2 *   24),
    'N44':   (  44 * 3/4. * 2, 2 *   44),
    'N96':   (  96 * 3/4. * 2, 2 *   96),
    'N160':  ( 160 * 3/4. * 2, 2 *  160),
    'N216':  ( 216 * 3/4. * 2, 2 *  216),
    }

def create_cube(lat, long):
    """
    Create test cube.

    lat, long - cube dimensions
    """
    # lat coord
    spacing = 180. / lat
    lower_bound = -90 + spacing/2.
    upper_bound =  90 - spacing/2.
    latitude  = iris.coords.DimCoord(np.linspace(lower_bound, upper_bound, lat),
                                     standard_name='latitude',
                                     units='degrees')

    #long coord
    spacing = 360. / long
    lower_bound =   0 + spacing/2.
    upper_bound = 360 - spacing/2.
    longitude = iris.coords.DimCoord(np.linspace(lower_bound, upper_bound, long),
                                     standard_name='longitude',
                                     units='degrees')

    # create cube
    cube = iris.cube.Cube(np.random.rand(lat, long),
                          dim_coords_and_dims=[(latitude, 0), (longitude, 1)])
    # add bounds
    cube.coord('latitude').guess_bounds()
    cube.coord('longitude').guess_bounds()

    # add name
    cube.long_name = 'Test cube'

    return cube

print "# Test effect of 'caching'"

print '# Create regridder'
cube1 = create_cube(*resolution['N44'])
cube2 = create_cube(*resolution['N96'])
cube1.data
cube2.data
ts = time()
areaweighted_regridder = iris.analysis.AreaWeighted().regridder(cube1, cube2)
print areaweighted_regridder._target_grid_cube_cache
print round(time() - ts, 3), 's'

print '# Regrid with cached regridder'
for cube in [cube1] * 3:
    ts = time()
    result = areaweighted_regridder(cube)
    print round(time() - ts, 3), 's'
    
    plt.imshow(areaweighted_regridder._target_grid_cube_cache.data, interpolation='nearest', cmap='hot')
    plt.title('Output of np.empty is cached (uninitialised).')
    plt.colorbar()
    plt.show()
    plt.close()
    

print '# Regrid with new regridder'
scheme = iris.analysis.AreaWeighted()
for cube1 in [cube1] * 3:
    ts = time()
    result = cube1.regrid(cube2, scheme)
    print round(time() - ts, 3), 's'

Output:

# Test effect of 'caching'
# Create regridder
None
0.001 s
# Regrid with cached regridder
5.162 s
<plot>
5.087 s
<plot>
5.094 s
<plot>
# Regrid with new regridder
5.088 s
5.081 s
5.151 s

[bjlittle]

Ping @abooton 👍

[ehogan]

Hi @tv3141 :)

Running your example on my machine using 738093f returns:

# Test effect of 'caching'
# Create regridder
None
0.001 s
# Regrid with cached regridder
1.371 s
<plot>
1.423 s
<plot>
1.331 s
<plot>
# Regrid with new regridder
1.424 s
1.29 s
1.361 s

Running your example using the latest commit on the area_weighted_regridding feature branch (326b92d) returns:

# Test effect of 'caching'
# Create regridder
unknown / (unknown)                 (latitude: 144; longitude: 192)
     Dimension coordinates:
          latitude                           x               -
          longitude                          -               x
0.409 s
# Regrid with cached regridder
0.058 s
<plot>
0.061 s
<plot>
0.06 s
<plot>
# Regrid with new regridder
0.438 s
0.424 s
0.434 s

Creating the regridder is now slower (the weights are now cached when the regridder is created). However, regridding is much quicker, as are the overall timings :)

[pp-mo]

? Closed by #3617 ?
Needs checking

[ehogan]

? Closed by #3617 ?

#3623? :)

[pp-mo]

#3623?

By all means !
I couldn't work out what key item to reference.

The question remains : whether we are sure that this fixes the original issue.
? can you confirm ?
A actual standard test of performance would be ideal, but I don't think it fits into the existing test strategy.

[ehogan]

I believe this comment on #2472 answers your question :)

[abooton]

Was released in iris v2.4 (not v3)