WFS-NG Check #2492

jodygarnett · 2019-07-12T16:44:20Z

Looking at an axis order issue encountered by GeoNetwork when updating to GeoTools 21.x:

set up fixture template for GeoServer online tests
check WFSDataStore ability to reproject based on axis order change

Reference:

jodygarnett · 2019-07-12T17:13:54Z

Making some notes:

Many objects do not have toString implementations making debugging annoying
DataStore uses InProcessingLocking manager when it should delegate locking (if available) to the WFS Service
Uses ConcurrentHashMap for names, remoteFeatureTypes rather than define a WFSContentEntry
Has remoteStoredQueries,storedQueryDescriptionTypes,storedQueryDescriptionTypesLock rather than a data structure to manage stored queries

Testing against GeoServer 2.15.2:

bin/startup.sh does not set -XX:SoftRefLRUPolicyMSPerMB=36000
WFS 1.0.0 GetCapabilities lists EPSG::4326 with bounds in x/y order
WFS 1.1.0 GetCapabilities lists urn:ogc:def:crs:EPSG::4326 with bounds in x/y order
WFS 2.0.0 GetCapabilities lists urn:ogc:def:crs:EPSG::4326 with bounds in x/y order

Layer preview shows inconsistent results:

WFS 1.0.0 GML2 output uses srsName http://www.opengis.net/gml/srs/epsg.xml#4326 with data in x/y order
WFS 1.0.0 GML3.1 output uses srsName http://www.opengis.net/gml/srs/epsg.xml#4326 with data in x/y order
WFS 1.1.0 GML2 output uses srsName http://www.opengis.net/gml/srs/epsg.xml#4326 with data in y/x order
WFS 1.1.0 GML3.1 output uses srsName urn:x-ogc:def:crs:EPSG:4326 with data in y/x order
WFS 1.1.0 GML3.2 output uses srsName http://www.opengis.net/gml/srs/epsg.xml#4326 with data in x/y order
WFS 2.0.0 GML3.2 output uses srsName http://www.opengis.net/gml/srs/epsg.xml#4326 with data back to x/y order again
WFS 2.0.0 GML uses srsName urn:ogc:def:crs:EPSG::4326 with data y/x order

Examples:

Used Strict CITE compliance for the following tests.

Changing srsName as part of request provides some control:

WFS 1.1.0 GetFeature GML2, GML3, GML32

GML	srsName	data	request	url
GML2	http://www.opengis.net/gml/srs/epsg.xml#4326	y/x		http://localhost:8080/geoserver/wfs?service=wfs&request=GetFeature&version=1.1.0&typeName=topp:states&outputFormat=GML2
GML2	http://www.opengis.net/gml/srs/epsg.xml#4326	x/y	EPSG:4326	http://localhost:8080/geoserver/wfs?service=wfs&request=GetFeature&version=1.1.0&typeName=topp:states&outputFormat=GML2&srsName=EPSG:4326
GML3	urn:x-ogc:def:crs:EPSG:4326	y/x		http://localhost:8080/geoserver/wfs?service=wfs&request=GetFeature&version=1.1.0&typeName=topp:states&outputFormat=GML3
GML3	urn:x-ogc:def:crs:EPSG:4326	x/y	EPSG:4326	http://localhost:8080/geoserver/wfs?service=wfs&request=GetFeature&version=1.1.0&typeName=topp:states&outputFormat=GML3&srsName=EPSG:4326
GML3	urn:x-ogc:def:crs:EPSG:4326	x/y	urn:ogc:def:crs:EPSG::4326	http://localhost:8080/geoserver/wfs?service=wfs&request=GetFeature&version=1.1.0&typeName=topp:states&outputFormat=GML3&srsName=urn:ogc:def:crs:EPSG::4326
GML32	urn:ogc:def:crs:EPSG::4326	x/y		http://localhost:8080/geoserver/wfs?service=wfs&request=GetFeature&version=1.1.0&typeName=topp:states&outputFormat=GML32
GML32	http://www.opengis.net/gml/srs/epsg.xml#4326	y/x	EPSG:4326	http://localhost:8080/geoserver/wfs?service=wfs&request=GetFeature&version=1.1.0&typeName=topp:states&outputFormat=GML32&srsName=EPSG:4326
GML32	urn:x-ogc:def:crs:EPSG:4326	y/x	urn:ogc:def:crs:EPSG::4326	http://localhost:8080/geoserver/wfs?service=wfs&request=GetFeature&version=1.1.0&typeName=topp:states&outputFormat=GML32&srsName=urn:ogc:def:crs:EPSG::4326

WFS 2.0

WFS 2.0 specification indicates WGS84BoundingBox is expected in X/Y order at all times

jodygarnett · 2019-07-23T17:27:12Z

ogr output shows WGS84 EPSG:4326:

ogrinfo -so -al states.shp 
INFO: Open of `states.shp'
      using driver `ESRI Shapefile' successful.

Layer name: states
Metadata:
  DBF_DATE_LAST_UPDATE=1904-07-25
Geometry: Polygon
Feature Count: 49
Extent: (-124.731422, 24.955967) - (-66.969849, 49.371735)
Layer SRS WKT:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9122"]],
    AUTHORITY["EPSG","4326"]]
STATE_NAME: String (25.0)
STATE_FIPS: String (2.0)
SUB_REGION: String (7.0)
STATE_ABBR: String (2.0)
LAND_KM: Real (19.9)
WATER_KM: Real (19.9)
PERSONS: Real (19.9)
FAMILIES: Real (19.9)
HOUSHOLD: Real (19.9)
MALE: Real (19.9)
FEMALE: Real (19.9)
WORKERS: Real (19.9)
DRVALONE: Real (19.9)
CARPOOL: Real (19.9)
PUBTRANS: Real (19.9)
EMPLOYED: Real (19.9)
UNEMPLOY: Real (19.9)
SERVICE: Real (19.9)
MANUAL: Real (19.9)
P_MALE: Real (19.9)
P_FEMALE: Real (19.9)
SAMP_POP: Real (19.9)

And showing some content shows X/Y order:

OGRFeature(states):48
  STATE_NAME (String) = Washington
  STATE_FIPS (String) = 53
  SUB_REGION (String) = Pacific
  STATE_ABBR (String) = WA
  LAND_KM (Real) = 172447.205000000
  WATER_KM (Real) = 12226.630000000
  PERSONS (Real) = 4866692.000000000
  FAMILIES (Real) = 1264934.000000000
  HOUSHOLD (Real) = 1872431.000000000
  MALE (Real) = 2413747.000000000
  FEMALE (Real) = 2452945.000000000
  WORKERS (Real) = 1830031.000000000
  DRVALONE (Real) = 1700872.000000000
  CARPOOL (Real) = 282240.000000000
  PUBTRANS (Real) = 104403.000000000
  EMPLOYED (Real) = 2293961.000000000
  UNEMPLOY (Real) = 139216.000000000
  SERVICE (Real) = 637487.000000000
  MANUAL (Real) = 302635.000000000
  P_MALE (Real) = 0.496000000
  P_FEMALE (Real) = 0.504000000
  SAMP_POP (Real) = 736744.000000000
  MULTIPOLYGON (((-122.400749 48.225395,...

jodygarnett · 2019-07-25T05:18:11Z

Moving this discussion over to geoserver/geoserver#3674

aaime · 2019-09-21T13:14:02Z

So is there anything left to do here?

aaime · 2019-10-07T08:50:59Z

@jodygarnett if there is no more activity here I'd suggest to close it, it's been around for a while with no sign of resolution?

jodygarnett · 2019-10-15T04:52:31Z

Marking as ready for review, will look at travis-ci failure.

As many of the online tests were written for the old implementation they do not all pass; by merging this in it corrects one serious flaw; and allows some tests to be turned on using a geoserver.properties file.

Signed-off-by: Jody Garnett <jody.garnett@gmail.com>

WFS 2.0 returns a different set up expected features Signed-off-by: Jody Garnett <jody.garnett@gmail.com>

Signed-off-by: Jody Garnett <jody.garnett@gmail.com>

jodygarnett · 2019-10-15T05:01:11Z

@aaime thanks for the encouragement, this is ready for your review as an improvement. I can now meet my requirement, and have made it possible to run and pass a few more tests along the way.

aaime · 2019-10-21T10:08:22Z

.../unsupported/wfs-ng/src/test/java/org/geotools/data/wfs/online/v2_0/GeoServerOnlineTest.java

@@ -33,7 +33,7 @@
 public class GeoServerOnlineTest extends AbstractWfsDataStoreOnlineTest {

    public static final String SERVER_URL =
-            "http://localhost:9090/geoserver/wfs?service=WFS&request=GetCapabilities&version=2.0.0";


Is the test run on a normal maven build (hopefully not)? There are around a few annoying tests that just fail if there is a GeoServer running locally that does not have the layers they expect.

Checked, no worries, they are not run at all

jodygarnett · 2019-11-26T18:14:43Z

.../unsupported/wfs-ng/src/test/java/org/geotools/data/wfs/online/v2_0/GeoServerOnlineTest.java

@@ -33,7 +33,7 @@
 public class GeoServerOnlineTest extends AbstractWfsDataStoreOnlineTest {

    public static final String SERVER_URL =
-            "http://localhost:9090/geoserver/wfs?service=WFS&request=GetCapabilities&version=2.0.0";
+            "http://localhost:8080/geoserver/wfs?service=WFS&request=GetCapabilities&version=2.0.0";


We should move this to a test fixture then

jodygarnett · 2019-12-02T14:42:40Z

I would like to backport this, but I understand a geoserver test case needs to be modified at the same time

dromagnoli · 2019-12-02T16:13:53Z

I would like to backport this, but I understand a geoserver test case needs to be modified at the same time

If you need it, @jodygarnett, this is the commit I made for that GeoServer test fix
geoserver/geoserver@e03b7dc

jodygarnett mentioned this pull request Jul 12, 2019

Geotools v21.4 upgrade geonetwork/core-geonetwork#3886

Closed

jodygarnett mentioned this pull request Oct 1, 2019

[GEOT-6355] WFS POST Request contains Key-Value-Pairs in URL fails #2528

Merged

jodygarnett marked this pull request as ready for review October 15, 2019 04:51

jodygarnett added 4 commits October 14, 2019 21:54

Add a fixture template for GeoServerWFSOnlineTest

ecf920f

Signed-off-by: Jody Garnett <jody.garnett@gmail.com>

WFS 2.0 requires typesnames (not typename)

37691e0

WFS 2.0 returns a different set up expected features Signed-off-by: Jody Garnett <jody.garnett@gmail.com>

Improve wfs-ng test coverage

41d34e5

Signed-off-by: Jody Garnett <jody.garnett@gmail.com>

formatting fixes

e8b28aa

Signed-off-by: Jody Garnett <jody.garnett@gmail.com>

jodygarnett force-pushed the wfs-ng-check branch from 9c767f3 to e8b28aa Compare October 15, 2019 04:57

jodygarnett requested a review from aaime October 15, 2019 04:59

aaime reviewed Oct 21, 2019

View reviewed changes

jodygarnett commented Nov 26, 2019

View reviewed changes

tbarsballe merged commit 05b0287 into geotools:master Nov 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WFS-NG Check #2492

WFS-NG Check #2492

jodygarnett commented Jul 12, 2019 •

edited

Loading

jodygarnett commented Jul 12, 2019 •

edited

Loading

jodygarnett commented Jul 23, 2019 •

edited

Loading

jodygarnett commented Jul 25, 2019

aaime commented Sep 21, 2019

aaime commented Oct 7, 2019

jodygarnett commented Oct 15, 2019

jodygarnett commented Oct 15, 2019

aaime Oct 21, 2019

aaime Nov 26, 2019

jodygarnett Nov 26, 2019

jodygarnett commented Dec 2, 2019

dromagnoli commented Dec 2, 2019 •

edited

Loading

WFS-NG Check #2492

WFS-NG Check #2492

Conversation

jodygarnett commented Jul 12, 2019 • edited Loading

jodygarnett commented Jul 12, 2019 • edited Loading

WFS 1.1.0 GetFeature GML2, GML3, GML32

WFS 2.0

jodygarnett commented Jul 23, 2019 • edited Loading

jodygarnett commented Jul 25, 2019

aaime commented Sep 21, 2019

aaime commented Oct 7, 2019

jodygarnett commented Oct 15, 2019

jodygarnett commented Oct 15, 2019

aaime Oct 21, 2019

Choose a reason for hiding this comment

aaime Nov 26, 2019

Choose a reason for hiding this comment

jodygarnett Nov 26, 2019

Choose a reason for hiding this comment

jodygarnett commented Dec 2, 2019

dromagnoli commented Dec 2, 2019 • edited Loading

jodygarnett commented Jul 12, 2019 •

edited

Loading

jodygarnett commented Jul 12, 2019 •

edited

Loading

jodygarnett commented Jul 23, 2019 •

edited

Loading

dromagnoli commented Dec 2, 2019 •

edited

Loading