QGIS 3.10.3 LTR - Problems with CRS reprojection-conversions to OSGB1936 (PROJ 5.x specific)

[nicknn]

Describe the bug
This is a repeat of this bug: QGIS 3.10 Projected CRS do not display properly #33129
#33129

The same issue affects QGIS 3.10.3 LTS on Linux (Xubuntu 18.04 LTS, 64-bit via Long Term Release Repository .../debian-ltr bionic main)

How to Reproduce
Opening an existing QGIS project created under 3.4.x LTR.
Project CRS is OSGB 1936. All shapefile and raster layers are native OSGB1936. QuickMapServices Google satellite is the problem layer. In 3.4.x the QuickMapServices layer displays correctly with no error messages.

On opening the project again in 3.10.3 I am presented with a previously unseen dialogue box Select Transformation For [filename].

Selecting any option but the first transform results in the QuickMapServices layer displaying no data, just white screen. Transform 3 would be my preferred conversion. No on-screen error messages.

Removing the layer and adding it again does nothing: the Select Transformation For [filename] does not open and nothing appears on screen. Most other options in QuickMapServices fail to show. No on-screen error messages.

Reinstalling the plugin does nothing.

Workaround
Problem can be worked around by:

1. starting the project again but setting the project CRS to OSGB1936 before adding any layers. QuickMapServices layers then display correctly.
2. starting the project again, importing an initial OSGB1936 shapefile and making whatever transform option is preferred in the Select Transformation For [filename] dialogue box. QuickMapServices layers then display correctly.
3. finding and downloading the proj-datumgrid-europe-latest.zip file, extracting OSTN15_NTv2_OSGBtoETRS.gsb and moving it to /usr/share/proj/

The 3rd workaround solves the problem and the Select Transformation For [filename] dialogue box never appears for any layer, either in an existing or new project. By not appearing, though, I have no idea what level of accuracy has been applied to the transform of CRS that the Select Transformation For [filename] dialogue box provides (when it works correctly).

All of the above are hoops to jump through that never previously existed and #3, although effective, is very much less than obvious. Is the issue as simple as not having included a data file with the release?

QGIS and OS versions

QGIS version
3.10.3-A Coruña
QGIS code revision
0e1f846
Compiled against Qt
5.9.5
Running against Qt
5.9.5
Compiled against GDAL/OGR
2.2.3
Running against GDAL/OGR
2.2.3
Compiled against GEOS
3.6.2-CAPI-1.10.2
Running against GEOS
3.6.2-CAPI-1.10.2 4d2925d6
Compiled against SQLite
3.22.0
Running against SQLite
3.22.0
PostgreSQL Client Version
10.12 (Ubuntu 10.12-0ubuntu0.18.04.1)
SpatiaLite Version
4.3.0a
QWT Version
6.1.3
QScintilla2 Version
2.10.2
PROJ.4 Version
493
OS Version
Ubuntu 18.04.4 LTS
Active python plugins
mmqgis;
LAStools;
chm_from_lidar-master;
GeoCoding;
quick_map_services;
db_manager;
processing

[nyalldawson]

Please test with 3.12 or a nightly of 3.10.4. there been a relevant fix applied to those releases (not in 3.10.3)

[nicknn]

Nightly build as below:

Irregular problems with exported images from the Layout Manager that did not occur in 3.4.x and are associated with the project as set out in the original post.

How to Reproduce

1. Open new project
   Opens in WGS 84 (default projection)

2. Add existing vector layer in OSGB1936
   Select Transformation For [filename] dialogue box appears

3. Select option 3
   Vector layer opens and project defaults to OSGB1936

4. Add QuickMapServices Google satellite layer - no Select Transformation For [filename] dialogue box

5. Open Layout Manager

6. Create new layout

7. Add a new map to the layout

8. Export as Image

9. Select JPEG as file format

10. PPI set to 300

11. Tick Include World File

12. Export messages consistently state that the export executed successfully

• but rendering of the combined image is not stable. Most times it displays correctly as the combined Google image and vector but sometimes (<10% of occasions) there is no Google image, only the vector image is exported

13. Import the new JPEG file to the project and it appears where it should in the project

Continue

14. Start a new project - opens in WGS 84 (default projection)
15. Add the JPEG created above
    Raster opens in WGS84 - no Select Transformation For [filename] dialogue box
16. Add QuickMapServices Google satellite layer - no Select Transformation For [filename] dialogue box
    No QuickMapServices Google satellite layer visible behind JPEG
17. Zoom to QuickMapServices Google satellite layer and it shows the entire layer
18. Zoom to the JPEG layer created above - no QuickMapServices Google satellite layer visible behind JPEG
19. Double click JPEG layer

• can produce this error (<10% occasions):

2020-02-29T21:56:20 CRITICAL Qt : QXcbConnection: XCB error: 3 (BadWindow), sequence: 23910, resource id: 10337627, major code: 40 (TranslateCoords), minor code: 0

and

2020-02-29T22:11:52 CRITICAL Qt : QXcbConnection: XCB error: 3 (BadWindow), sequence: 10614, resource id: 8624124, major code: 40 (TranslateCoords), minor code: 0

• If not this error, then (at least 50% of occasions):

20. Change CRS of the JPEG layer to WGS 84 has no effect (not placed over Google layer)
21. Change CRS to OSGB1936 produces a distorted in WGS 84 but if OTF projection changed to OSGB1936 places correctly
22. As 21 above, change CRS of Google layer opens Select Transformation For [filename] dialogue box, option 3 selected.
    Redraws correctly

In addition to the CRITICAL error above, there are multiple logs for

2020-02-29T22:21:11 WARNING Transform error caught: Could not transform bounding box to target CRS

and multiple

2020-02-29T21:47:44 WARNING Could not reproject layer extent: Could not transform bounding box to target CRS

Note
If at 3 above, option 1 is selected, there seem to be no issues (not exhaustively tested)

QGIS and OS versions
via Long Term Release Repository .../debian-nightly-ltr bionic main

QGIS version
3.10.3-A Coruña
QGIS code revision
7e531f1
Compiled against Qt
5.9.5
Running against Qt
5.9.5
Compiled against GDAL/OGR
2.2.3
Running against GDAL/OGR
2.2.3
Compiled against GEOS
3.6.2-CAPI-1.10.2
Running against GEOS
3.6.2-CAPI-1.10.2 4d2925d6
Compiled against SQLite
3.22.0
Running against SQLite
3.22.0
PostgreSQL Client Version
10.12 (Ubuntu 10.12-0ubuntu0.18.04.1)
SpatiaLite Version
4.3.0a
QWT Version
6.1.3
QScintilla2 Version
2.10.2
PROJ.4 Version
493
OS Version
Ubuntu 18.04.4 LTS
This copy of QGIS writes debugging output.
Active python plugins
mmqgis;
LAStools;
chm_from_lidar-master;
GeoCoding;
quick_map_services;
db_manager;
processing

[nicknn]

Did not mean to close this: there are issues.

[nyalldawson]

The above report was running 3.10.3, not .4.

[gioman]

PROJ.4 Version
493
Compiled against GDAL/OGR
2.2.3
Running against GDAL/OGR
2.2.3

@nyalldawson also for this type of issues, does it make any sense testing against such old dependencies?

[nyalldawson]

Ah I missed that it was proj < 6. In that case no, it'll never be fixed. I consider proj < 6 deprecated effectively unsupported now.

[gioman]

Closing?

[nyalldawson]

I think we leave it open until we put a hard requirement on proj 6. I'm not personally going to work on this, but there's always a chance someone else is interested.

[nicknn]

I take it that further investigation at my end will not be needed whatever the irregular result. As to 3.10.4, it is not making its way onto my system via the debian-nightly-ltr repositories, only updated versions of .3

[gioman]

s to 3.10.4, it is not making its way onto my system via the debian-nightly-ltr repositories, only updated versions of .3

@nicknn Also the newest, necessary, versions of GDAL and PROJ will only make into the next Ubuntu LTR (20.04). I'm also as many others an Ubuntu/derivate user and my suggestion is to embrace as soon as you can 20.04, probably the RC are already out.

[github-actions]

The QGIS project highly values your report and would love to see it addressed. However, this issue has been left in feedback mode for the last 14 days and is being automatically marked as "stale".
If you would like to continue with this issue, please provide any missing information or answer any open questions. If you could resolve the issue yourself meanwhile, please leave a note for future readers with the same problem and close the issue.
In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this issue.
If there is no further activity on this issue, it will be closed in a week.

[nicknn]

Update and close:
Since raising the issue I have upgrade the OS to Xubuntu 20.04.1 (fresh install, not an in-place upgrade) and QGIS to 3.10.13 LTR.
With this setup I have not encountered problems with CRS reprojection-conversions to OSGB1936 or any other CRS. QuickMapServices layers are also stable in this setup.
Thanks for the help and continued good work on QGIS.