incorrect TZ for Page, AZ

[pirxpilot]

Page, AZ - 36.9147, -111.4558 - should be in America/Phoenix (no DST) and not in America/Denver TZ

I added test in pirxpilot/tz-lookup repo

Not sure if this is a problem with incorrect shapefile DB or an issue in tz-lookup. I emailed Eric Muller - I'll let you know if I get a response.

[ghost]

This is a known bug on this end caused by the compression algorithm we use to pack the timezones into a small file. I am working on a fix (bugfix branch), but haven't had a chance to look at it recently!

[pirxpilot]

Thanks - I'll check the branch and see if I can help with anything.

[ghost]

I spent some hours yesterday doing a deep dive into this. Here's what I've found:

1. In order to make timezone lookups fast (and the data files small), we convert the vector timezone map into a raster image, and then compress that image with a custom algorithm that results in a small data file that still admits fast pixel lookups.
2. The problem here is that rasterization is inexact: if you are near a timezone border—as Page, AZ is—your pixel might be assigned to the timezone on the other side of the border instead of on your side. The only way to fix this is to increase the resolution of the rasterization.
3. Presently, we use an image resolution of 24576×12288. This is about the maximum resolution that my poorly-written, hand-rolled rasterizer can handle; and it's also about the maximum resolution that we can pack into a data file of a sane size. (Uncompressed, the data file is about half a gigabyte; compressed, it's about a megabyte.)
4. I just updated tz-lookup to v6.0.6: this replaces my hand-rolled rasterizing script with GDAL, which is a real library maintained by real people. It can handle just about any resolution we care to throw at it. This eliminates the first bottleneck.
5. The next step is to make the data files smaller (in order to make it easier to add more data into them). The idea I plan to try is this: due to the nature of the compression scheme, most of the space in the data file is taken up by areas near timezone borders. This is wasteful, though, since most timezone borders are far away from populated areas (and therefore small errors in the exact location of the border are unlikely to matter). So, update my packing script to compress to full resolution only near populated areas; if we're far from a populated area, halt at a lower resolution in order to save space. (The 10m "urban areas" vector map at Natural Earth Data seems to be suitable for this kind of use.) This would eliminate the second bottleneck.
6. After that, we should be free to quadruple the resolution of the source image, which I believe should fix this issue.

[ghost]

In the urban branch, I have code which now generates maps at double the resolution (49152× 24576) and also uses a bitmap of populated areas in order to (lossily) generate much smaller data files. (In fact, the new map compresses down to 86KB, or 0.004%.)

The downside to this is that there are errors caused by the lossy compression—a few of the test cases (including Page, AZ, of course :) ) fail. I assume that this is either due to a bug in the packing code or because the urban cover map isn't high-quality enough (e.g. omitting urban centers for a few cities). So there's still some more investigating to do.

[ghost]

Okay, dug into this yet more, and confirmed that Page, AZ is listed as being America/Denver in our source data. I've contacted the maintainer of the source about the issue.

Continuing to investigate workarounds on my end in the meantime.

[evansiroky]

Hello there, I'm the maintainer of timezone-boundary-builder. Timezone-boundary-builder creates a timezone shapefile of the world using data from OpenStreetMap.

I've created an issue evansiroky/timezone-boundary-builder#9 to come up with some better borders in Northeastern Arizona.

@pirxpilot if you have detailed local knowledge of where exactly the timezone boundaries should be in this area, I'd love to hear your input!

@ironwallaby I'd recommend switching your data source to use timezone-boundary-builder. timezone-boundary-builder has much more precise and accurate borders and includes territorial waters.

[pirxpilot]

Sorry. I don't really have any local data on Page, AZ - I learned about the discrepancy from Furkot users. Page is a tourist hub in the area and our users were complaining about incorrect times.

[ghost]

@evansiroky Thanks for the heads-up; I'll take a look at ingesting your data set to see how it compares. (It should be a straightforward change to the pipeline.)