Speed improvement in the render of geometries for qgis #17441
Comments
|
Author Name: Matthias Kuhn (@m-kuhn) Hi Alvaro, Thank you for looking into this issues and contributing. The best thing to do is to open a pull request on github, so it can be reviewed, commented and eventually merged. |
|
Author Name: Alvaro Huarte (@ahuarte47) Ok, thank you very much |
|
Author Name: Matthias Kuhn (@m-kuhn) I forgot to mention that it would be best to make one pull request for each improvement. So in case one of them is not accepted, it will still be possible to merge the good parts. |
|
Author Name: Alvaro Huarte (@ahuarte47) ok, I will divide this issue in four single childs tasks. |
|
Author Name: Giovanni Manghi (@gioman) Hi Alvaro, do you have a branch that can be tested? Cheers! |
|
Author Name: Alvaro Huarte (@ahuarte47) Hi Giovanni, I am testing my changes, if it successly in some days I will try a pull request. |
|
Author Name: Alvaro Huarte (@ahuarte47) I'm making progress, I think I get promising results. Attach a table of results for three tested layers.
|
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
wow, I can't wait to test your patches! :) |
|
Author Name: Alvaro Huarte (@ahuarte47)
|
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
Hi Alvaro, can you please make a Pull Request in the qgis repo? This way other will be able to see it and review and then commit. Thanks! |
|
Author Name: Alvaro Huarte (@ahuarte47) Done. |
|
Author Name: Alvaro Huarte (@ahuarte47) Changes goto branch: |
|
Author Name: Alvaro Huarte (@ahuarte47) I have implemented the simplification of geometries in the vector-providers. The general behavior simplifies the geometries post-fetch the feature from the provider, but OGR-provider simplifies the geometries pre-fetch the features having a extra seed improvement. Te final result gets ~3x painting speed faster that original code New branch: I think ready for test! :-)
|
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
compiling right now :) |
|
Author Name: Giovanni Manghi (@gioman)
got an error during make |
|
Author Name: Alvaro Huarte (@ahuarte47) Hi Giovanni, I use Visual Studio 2010 and I don't get these errors. I will try fix it! |
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
Hi Alvaro, I use Ubuntu Linux, gcc 4.6.3 |
|
Author Name: Matthias Kuhn (@m-kuhn) It's probably a simple #include <limits.h> |
|
Author Name: Alvaro Huarte (@ahuarte47) Done, thanks! |
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
Hi Alvaro! I made a few quick tests with shapefiles, measuring just the time to open them (with the "time" command). I have not had the time to try evaluate the improvement when zooming in/out. The shapefiles (polygons) that I tested are: http://faunalia.pt/downloads/GTBs_Shapefile.zip un-patched QGIS giovanni@sibirica ~/Desktop $ time qgis MAMMTERR.shp user 0m10.377s giovanni@sibirica ~/Desktop $ time qgis GTBs_2011.shp user 1m2.896s patched QGIS giovanni@sibirica ~/Desktop $ time qgis MAMMTERR.shp user 0m6.352s giovanni@sibirica ~/Desktop $ time qgis GTBs_2011.shp user 0m35.278s So there is really a substantial improvement! I tested also another shape (big, more that 1.2gb of small polygons) where things get worst(?) http://www.igeo.pt/scrif/crif/CRIF2011shp.zip unpatched giovanni@sibirica ~/Desktop $ time qgis crif2011.shp user 0m35.186s patched giovanni@sibirica ~/Desktop $ time qgis crif2011.shp user 0m45.571s Can I test also the vectors you are using to check the rendering times? Cheers! |
|
Author Name: Alvaro Huarte (@ahuarte47) Hi Giovanni, Some shapes... http://idena.navarra.es/descargas/CARTO1_Lin_6CNivelD.zip I am going to test your shapes! |
|
Author Name: Matthias Kuhn (@m-kuhn) Running a rendering performance test once may be strongly biased by other processes running. It's really simple to use. Here some more information: |
|
Author Name: Alvaro Huarte (@ahuarte47) Hi, I made tests with 'qgis_bench' to see the speed. Soft/hard: +WindowsXP+SP3, 32bits, 4gb RAM+. My shapefiles tested: |. Dataset |. Value |. Master |. Patched | | |
|
Author Name: Nathan Woodrow (@NathanW2) Nice! |
|
Author Name: Alvaro Huarte (@ahuarte47)
|
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
I agree |
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
very nice results indeed! as soon as I will understand how to use qgis_bench I will make tests too :) |
|
Author Name: Alvaro Huarte (@ahuarte47) Hi Giovanni, qgis_bench is easy :-) I execute in a Console Window (CMD)... {%BIN_PATH%}/qgis_bench -i 10 -p test_project.qgs >> test_project.log -i: number of desired iterations Regards |
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
I made also tests with qgis_bench |. Dataset |. Value |. Master |. Patched | | |
|
Author Name: Jürgen Fischer (@jef-n)
|
|
Author Name: Alvaro Huarte (@ahuarte47) Giovanni Manghi wrote:
What you're saying makes sense, I may be trying to simplify polygons with <=5 points (grids) which themselves are not generalizable, |
|
Author Name: Alvaro Huarte (@ahuarte47) Alvaro Huarte wrote:
I answer to myself :-), the patch try not to simplify polygons of 5 points, or linestrings of 2 points, but try to find out if it is possible to disable the 'AntiAliasing' for render the geometry faster. If the end is not possible to disable the AA (The geometry is not so far), an overload occurs in the process. This can be avoided disabling for this layer the drawing simplification. |
|
Author Name: Alvaro Huarte (@ahuarte47) regis Haubourg wrote:
Done! !DefaultSimplifyConfig.jpg! Regards
|
|
Author Name: Alvaro Huarte (@ahuarte47)
|
|
Author Name: Alvaro Huarte (@ahuarte47) Hi, I release a new windows installer which contains:
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v50 I made new tests with the latest version to see the speed. Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM. My shapefiles tested: |. Dataset |. Value |. Master |. Patched | | |
|
Author Name: Nyall Dawson (@nyalldawson) Here's some more string updates for this latest addition: "By default simplify geometries" -> "Simplify geometries by default" And for the text below the checkbox: "If checked, new vector layers added to the map will automatically use geometry simplification to speed up rendering. ... (then the rest of what you have)" Thanks! |
|
Author Name: Nyall Dawson (@nyalldawson) Alvaro - these improvements look fantastic! Your hard work is much appreciated! One related idea I was planning on investigating was adding an option to disable rendering of features smaller than a certain size (in mm). You'll see a similar option in the labelling properties for a vector layer under "Rendering" -> "Suppress labelling of features smaller than ". Having the ability to skip rendering of small features could potentially lead to some more performance gains (as well as having cartographic benefits). Obviously, it should be disabled by default for layers as it would only be useful in some circumstances. There was a bit of discussion on the QGIS dev list about this a month or so ago -- I had a quick initial attempt at implementing it but have got side tracked on composer work (and it's looking like this will tie me up for the whole of the 2.2 dev cycle). If you wanted to experiment with this, there's code for checking the size of a feature in mm in QgsPalLayerSettings::checkMinimumSizeMM . I had a quick play here nyalldawson@27a42e1 (no gui, hard coded to a minimum size of 2mm) which yielded some benefit. This was done in QgsVectorLayer::drawRendererV2, I think moving it further up the chain could improve the performance some more. |
|
Author Name: Alvaro Huarte (@ahuarte47) Nyall Dawson wrote:
Done, thanks Nyall! |
|
Author Name: Alvaro Huarte (@ahuarte47) Nyall Dawson wrote:
Hi Nyall, I did not know this discussion!, I have involved in QGIS later. Coincidences of life, this patch proposes an implementation of that issue. Now, I'm a little waiting for comments and suggestions to see if it is interesting to approval. On the subject of not paint "very small" elements seems like a good idea in case of layers as you mention, in other cases (very dense layers) would appear whole areas unpainted, but I would not mix this with this patch but on a new patch. If you want I can try to implement it. Regards |
|
Author Name: Nyall Dawson (@nyalldawson) Alvaro Huarte wrote:
I know - it's great timing! Thanks for all the work you've put in developing these much needed improvements for QGIS.
I don't want to add to your workload, but it'd be great if you could check it out. I've realized I'm not going to get a chance to tackle this for 2.2. Based on your past work you've got a much better understanding of optimising code like this then I do! |
|
Author Name: Alvaro Huarte (@ahuarte47) Nyall Dawson wrote:
Thanks Nyall, I also really appreciate the tests, advice and opinions of the community, they serve me to keep learning every day. And your english corrections ;-) |
|
Author Name: Jeremy Palmer (@palmerj) I've just seen this pull request. Dumb question, but why would you implement the user defined |
|
Author Name: Alvaro Huarte (@ahuarte47) Jeremy Palmer wrote:
Hi Jeremy, the software automatically calculates the simplification factor based on geometry and current map2pixel setting (Slider with minimum value). I added the possibility of increase this factor to get 'extra' simplification to accelerate even more the render of very large layers. As I said above the user, in most cases, do not worry about modifying this value, but in other cases it may be interesting. |
|
Author Name: Jeremy Palmer (@palmerj) Hi Alvaro, Thanks for the reply. In that case it might be worth stating on the slider label that it is a advanced experimental option. Cheers, |
|
Author Name: Alvaro Huarte (@ahuarte47) Jeremy Palmer wrote:
Hi Jeremy, I suppose we should add an entry in the user manual and explain this configuration. Alvaro |
|
Author Name: Alvaro Huarte (@ahuarte47) Hi all, There is a modification in behavior of this patch (#980). Now, it contains the advise of @nyalldawson (#1020 (comment)) to enable AA-disabling only when the simplification slider is greater than minimum value. Best regards |
|
Author Name: Alvaro Huarte (@ahuarte47)
|
|
Author Name: Alvaro Huarte (@ahuarte47) I am pleased to announce that this patch seems to have been merged! Thank you very much for your support and advices! |
|
Author Name: Giovanni Manghi (@gioman) Alvaro Huarte wrote:
Alvaro, many thanks to YOU :) |
|
Author Name: Giovanni Manghi (@gioman)
|
|
Author Name: Alvaro Huarte (@ahuarte47) What about add documentation to user manual? |
|
Author Name: Alvaro Huarte (@ahuarte47) Hi, I committed a new version of geometry simplification to speed up the vector drawing. It is inspired in this PR, but it contains several changes based on advice received ( @Matthias-Kuhn thanks! ) and new features:
I have not created a new pull request because of I would greatly appreciate if you could test it with your data and SO's. I have tested it in windows I get similar results as old version for shapefiles (OGR-provider). The new simplification on database side for postgres provider improves the old results too. I will create a table of tests to compare data and configurations. About postgres simplification, the ST_simplify function needs a tolerance parameter, I use map2pixel/5.0 as input value, it is experimental and I must define it better (All ideas are welcome). This simplification can be applicable to other database providers (MySQL, SQL Server, Oracle...) Thank you very much! |
qgib
added
Feature Request
Vectors
The autogenerated ids are quite long and this leads that they where very unconfortable to use in the configuration of relation reference in the attribute type widget of layer properties - attribute form. So now the names are used in the combo instead of the ids. This effects only the GUI. Fix: qgis#17441
Author Name: Alvaro Huarte (@ahuarte47)
Original Redmine Issue: 8725
Redmine category:vectors
Assignee: Alvaro Huarte
The render of geometries for vector layers in qgis can be optimized.
There are some tricks that sure make fast the rendering of big shapes:
I have implements this tricks and qgis has better performance of 300% in some map situations.
I want validate the code and I will try share it.
The text was updated successfully, but these errors were encountered: