Include stats in the response to map instantiation requests

[pabletecodes]

Required for CartoDB/cartodb#11783.

For now, we just need to know the number of vertexes of all the layers in the "layer group".

cc: @rochoa

[jorgesancha]

hey @CartoDB/engine this is one to pick up as soon as possible to progress vector rendering

[ethervoid]

Feature stats for Maps-API

We need to gather the number of vertex a layer has in order to send to the client where it's going to decide if renders using vector or raster

Node-windshaft

• We need to gather the number of vertex for every layer in the map instantiation. I think the best place to introduce this change is [here]https://github.com/CartoDB/Windshaft-cartodb/blob/master/lib/cartodb/controllers/map.js#L275). We could include an asynchronous method to calculate the number of features with a query like this SELECT sum(ST_NPoints(geom)) FROM {LAYER_NAME}

• We could include the vertex calculation in Camshaft for the analyses but how about the layers without analyses? Even if we don't have analyses we have the source node so we could gather the number of vertex.

Windshaft

We should include this new stats in every layer. This place looks a good place for it.

Performance

One of the most concerning issue is the performance. We don't want to penalize our current situation by including a new query to calculate the number of vertex of a layer so we're going to need:

• To measure the amount of time we spent doing the query
• To have a way to stop gathering stats if everything goes south
• To check for possible hot spots in the vertex calculation with choose method. For example if we use ST_NPoints() we should check for possible performance penalties with huge polygons, etc.
• To gather the stats from the Postgresql stats, if feasible, instead of calculate them to gain some performance because we don't need a precise number. We could use _postgis_stats function like here:

  cartodb_dev_user_23c39318-a446-4c62-ab2c-32e78757ba19_db=# select _postgis_stats('streets_paving'::regclass ,'the_geom');
                                                                                                           _postgis_stats
  --------------------------------------------------------------------------------------------------------------------------   ----------------------------------------------------------------------------------------------------------
   {"ndims":2,"size":[27,28],"extent":{"min":[-122.323,37.7237],"max":     
  [-122.117,37.8842]},"table_features":3888,"sample_features":3888,"not_null_features":3888,"histogram_features":3888,"histo  gram_cells":756,"cells_covered":3888}
  (1 row)
  

 We could check if the stats are available and use it like [here]
(https://github.com/CartoDB/cartodb/blob/af7858c55a2fd5088c9b7343c7c61277b782735f/lib/carto/bounding_box_service.rb#L28-L44). [Here]
(https://github.com/postgis/postgis/blob/bd3177743db5cb786deb05d7327f39a5cec2daf6/postgis/gserialized_estimate.c#L28-L58)
is the theory of operation of this function. One downside of this approach is that it only returns the number of features
but not the number of vertex. We should know if it's enough to use the features instead of the vertex.

[dgaubert]

Ready for CR:

• windshaft-stats
• Windshaft
• Windshaft-cartodb

[rochoa]

Is this blocked by CartoDB/cartodb-postgresql#301? Or do we have any other reason to not merge/release this?

[ethervoid]

Nop, the postgresql issue is not blocking the release. We're waiting for the smoke tests to pass in order to release it.

[pabletecodes]

Nobody reported anything related to this in the smoke tests, so... 🚀 (unless @donflopez has something to say)