Test: Invalid aggregation test with PROJ 5.1

[Algunenano]

I've got the aggregated ids are unique for default aggregation failing when running the tests under PROJ 5.1. Apparently it checks that there are no duplicated points between tiles, but there is a point appearing in both tested tiles:

      1) aggregated ids are unique for default aggregation
      2) "after each" hook for "aggregated ids are unique for default aggregation"


  0 passing (415ms)
  2 failing

  1) aggregation mvt (mapnik) aggregated ids are unique for default aggregation:

      Uncaught AssertionError [ERR_ASSERTION]: 1 == 0
      + expected - actual

      -1
      +0
      
      at testClient.getTile (test/acceptance/aggregation.js:2183:36)
      at Function.finish (test/support/test-client.js:838:20)
      at next (node_modules/step/lib/step.js:51:23)
      at test/support/test-client.js:124:16
      at Server.<anonymous> (test/support/assert.js:120:24)
      at emitCloseNT (net.js:1656:8)
      at process._tickCallback (internal/process/next_tick.js:178:19)

  2)  "after each" hook for "aggregated ids are unique for default aggregation":

      AssertionError [ERR_ASSERTION]: uncaughtException:

AssertionError [ERR_ASSERTION]: 1 == 0
    at testClient.getTile (test/acceptance/aggregation.js:2183:36)
    at Function.finish (test/support/test-client.js:838:20)
    at next (node_modules/step/lib/step.js:51:23)
    at test/support/test-client.js:124:16
    at Server.<anonymous> (test/support/assert.js:120:24)
    at emitCloseNT (net.js:1656:8)
    at process._tickCallback (internal/process/next_tick.js:178:19)
      + expected - actual

      -1
      +0
      
      at Context.<anonymous> (test/support/test_helper.js:87:12)

The feature that appears in both tiles is this one:

  {
    "type": "Feature",
    "id": 4,
    "geometry": {
      "type": "Point",
      "coordinates": [
        0,
        0
      ]
    },
    "properties": {
      "_cdb_feature_count": 1,
      "cartodb_id": 4,
      "value": 0
    }
  }

With the bug fixed in PROJ 5.1, the point (0,0) appears in all 4 tiles that contain that point. I don't know if this is a bug in the test or the implementation of the default aggregation.

cc/ @CartoDB/rt-managers

[rafatower]

cc/ @jgoizueta , who is more or less aware of this issue and will probably take over this in the next RT shift.

[jgoizueta]

@Algunenano can you check your tests are OK with #998 ?

[Algunenano]

The test I was complaining about is now fixed, but now there are other ones:

$ ./node_modules/.bin/_mocha -c -u tdd -t 5000 /home/raul/dev/public/Windshaft-cartodb/test/acceptance/aggregation.js --grep "post-aggregation"


  aggregation
    mvt (mapnik)
      1) on demand post-aggregation stats are available with default aggregation
      2) "after each" hook for "on demand post-aggregation stats are available with default aggregation"


  0 passing (265ms)
  2 failing

  1) aggregation mvt (mapnik) on demand post-aggregation stats are available with default aggregation:

      Uncaught AssertionError [ERR_ASSERTION]: 12 == 13
      + expected - actual

      -12
      +13
      
      at testClient.getLayergroup (test/acceptance/aggregation.js:2346:32)
      at test/support/test-client.js:907:20
      at Server.<anonymous> (test/support/assert.js:120:24)
      at emitCloseNT (net.js:1656:8)
      at process._tickCallback (internal/process/next_tick.js:178:19)

  2)  "after each" hook for "on demand post-aggregation stats are available with default aggregation":

      AssertionError [ERR_ASSERTION]: uncaughtException:

AssertionError [ERR_ASSERTION]: 12 == 13
    at testClient.getLayergroup (test/acceptance/aggregation.js:2346:32)
    at test/support/test-client.js:907:20
    at Server.<anonymous> (test/support/assert.js:120:24)
    at emitCloseNT (net.js:1656:8)
    at process._tickCallback (internal/process/next_tick.js:178:19)
      + expected - actual

      -1
      +0
      
      at Context.<anonymous> (test/support/test_helper.js:87:12)

      1) post-aggregation count adapts to zoom level with default aggregation
      2) "after each" hook for "post-aggregation count adapts to zoom level with default aggregation"


  1 passing (402ms)
  2 failing

  1) aggregation mvt (mapnik) post-aggregation count adapts to zoom level with default aggregation:

      Uncaught AssertionError [ERR_ASSERTION]: 8 == 9
      + expected - actual

      -8
      +9
      
      at testClient.getLayergroup (test/acceptance/aggregation.js:2387:32)
      at test/support/test-client.js:907:20
      at Server.<anonymous> (test/support/assert.js:120:24)
      at emitCloseNT (net.js:1656:8)
      at process._tickCallback (internal/process/next_tick.js:178:19)

  2)  "after each" hook for "post-aggregation count adapts to zoom level with default aggregation":

      AssertionError [ERR_ASSERTION]: uncaughtException:

AssertionError [ERR_ASSERTION]: 8 == 9
    at testClient.getLayergroup (test/acceptance/aggregation.js:2387:32)
    at test/support/test-client.js:907:20
    at Server.<anonymous> (test/support/assert.js:120:24)
    at emitCloseNT (net.js:1656:8)
    at process._tickCallback (internal/process/next_tick.js:178:19)
      + expected - actual

      -1
      +0
      
      at Context.<anonymous> (test/support/test_helper.js:87:12)

And finally:

2) torque tiles at 0,0 point tl:

      Uncaught AssertionError [ERR_ASSERTION]: [ { x__uint8: 1,
    y__uint8: 0,
    vals__uint8: [ 1 ],
    dates__uint16: [ 0 ] } ] deepEqual []
      + expected - actual

      -[
      -  {
      -    "dates__uint16": [
      -      0
      -    ]
      -    "vals__uint8": [
      -      1
      -    ]
      -    "x__uint8": 1
      -    "y__uint8": 0
      -  }
      -]
      +[]
      
      at test/acceptance/ported/torque_zero_zero.js:77:24
      at test/acceptance/ported/support/test_client.js:414:24
      at taskDone (test/support/test_helper.js:158:13)
      at Command.callback (test/support/test_helper.js:168:17)
      at normal_reply (node_modules/redis/index.js:726:21)
      at RedisClient.return_reply (node_modules/redis/index.js:824:9)
      at JavascriptRedisParser.returnReply (node_modules/redis/index.js:192:18)
      at JavascriptRedisParser.execute (node_modules/redis-parser/lib/parser.js:574:12)
      at Socket.<anonymous> (node_modules/redis/index.js:274:27)
      at addChunk (_stream_readable.js:274:12)
      at readableAddChunk (_stream_readable.js:261:11)
      at Socket.Readable.push (_stream_readable.js:218:10)
      at TCP.onread (net.js:581:20)

  3)  "after each" hook for "tl":

      AssertionError [ERR_ASSERTION]: uncaughtException:

AssertionError [ERR_ASSERTION]: [ { x__uint8: 1,
    y__uint8: 0,
    vals__uint8: [ 1 ],
    dates__uint16: [ 0 ] } ] deepEqual []
    at test/acceptance/ported/torque_zero_zero.js:77:24
    at test/acceptance/ported/support/test_client.js:414:24
    at taskDone (test/support/test_helper.js:158:13)
    at Command.callback (test/support/test_helper.js:168:17)
    at normal_reply (node_modules/redis/index.js:726:21)
    at RedisClient.return_reply (node_modules/redis/index.js:824:9)
    at JavascriptRedisParser.returnReply (node_modules/redis/index.js:192:18)
    at JavascriptRedisParser.execute (node_modules/redis-parser/lib/parser.js:574:12)
    at Socket.<anonymous> (node_modules/redis/index.js:274:27)
    at addChunk (_stream_readable.js:274:12)
    at readableAddChunk (_stream_readable.js:261:11)
    at Socket.Readable.push (_stream_readable.js:218:10)
    at TCP.onread (net.js:581:20)
      + expected - actual

      -1
      +0

The first 2 seem to be related to playing too close to (0,0) with transformations. And the last one can be fixed by just bringing the changes done to Windshaft in CartoDB/Windshaft@f447db1.

[jgoizueta]

For the aggregations I think the right thing to do is to avoid duplicates, so I reponed #889
I'll look at the non-aggregation cases now