-
Notifications
You must be signed in to change notification settings - Fork 7
pcapi 1.4
- Introduction
- API Calls
This is a draft API specification for the Personal Cloud middleware distributed for internal feedback. This document is tightly coupled to the "PC API Requirements" document.
The following assumptions were made:
-
Its main purpose is to act as a middleware for the EDINA mobile Fieldtrip app.
-
It should provide all facilities that are not implementable at the client side due to portability, processing, storage and framework limitations of a mobile phone
-
main functions should be indexing and searching of arbitrary media files as well a providing a compatibility layer to other storage cloud providers (e.g. dropbox, flickr) etc.
The API is purely REST and tries to leverage most of HTTP (including PUT, DELETE etc.). All API calls are meant to be generic and compatible with both a command line HTTP client like curl as well as a browser-based XMLHttpRequest client.
The Application Programming Interface (API) described here relates to the web service “PC API” available at endpoint: https://localhost:8080/pcapi/
Before issuing any API calls the mobile app should first authenticate. This is a non-trivial concept and is best explained by example.
NOTE: A Secure transport protocol (SSLv3/TLS) is recommended for all communications
A simple GET request that returns a list of all supported storage providers together with their capabilities as json:
curl localhost:8080/1.4/pcapi/auth/providers
Will return something similar to:
{
"edina": ["search", "synchronize", "delete"],
"dropbox" : ["oauth", "search", "synchronize", "delete"],
"googledrive" : ["oauth", "search", "synchronize", "delete"]
}
###Status Codes
| code | description | comments |
|---|---|---|
| 200 (OK) | A (possibly empty) list of known providers was returned |
The URL pattern used for filesystem access is as follows:
fieldtripgb.edina.ac.uk/1.3/fs/<provider>/<userid>/<path>
| name | details | mandatory |
|---|---|---|
| provider | The storage provider to use | true |
| userid | A token that allows the user to authenticate. This is provider dependent and is generated from the provider’s equivalent tokens (e.g. oAuth access tokens). See authentication. | true |
| path | The relative filesystem path pointing to a file or folder which will be appended to the user’s sandbox. | true |
Create a file with HTTP PUT (overwrite) or POST (create)
curl -X PUT -T myfile localhost:8080/1.3/pcapi/fs/dropbox/userid/myDirectory/myfilename
All intermediate directories are created on the fly e.g. if myDirectory does not exist it will be created.
PUT requests should place all file contents inside the body of the request. If the target file already exists it will be overwritten.
When POSTing a file, the file contents must be sent as multipart/form-data inside a field named file. This is done for compatibility with cordova. If a file with the same name already exists, then the uploaded file will be saved under a different name. The new name is in the value of the path property.
Example with POST:
curl -X POST --form file=@somefile localhost:8080/1.3/pcapi/fs/dropbox/userid/myDirectory/myfilename
Sample response:
{
"error": 0, "msg": "File uploaded",
"path": "/mydir/file(2).txt"
}
Delete a file with HTTP DELETE
curl -X DELETE localhost:8080/1.3/pcapi/fs/dropbox/userid/myDirectory/myfilename
List files using GET
curl localhost:8080/1.3/pcapi/fs/dropbox/userid/myDirectory
NOTE:
Directories are now automatically probed (no need for a trailing slash).
A simple GET request pointing to the file will download it in its original format.
curl localhost:8080/1.3/pcapi/fs/dropbox/userid/myDirectory/myfilename?format=jpeg&dimensions=80,80
If the asset is an image file then the optional arguments format and dimensions will attempt to convert the asset to the specified format and dimensions before downloading:
e.g.
curl localhost:8080/1.3/pcapi/fs/dropbox/userid/myDirectory/myfilename?format=jpeg&dimensions=80,80
NOTE: format and dimensions are not yet implemented
The URL pattern used for accessing editors and records should start with /editors or /records respectively. E.g.
localhost:8080/1.3/pcapi/editors/<PROVIDER>/<USERID>/<PATH>
The PATH convention used is as follows:
Each record has its own folder named after the record "name".
All editors are in a single folder.
A simple GET request will return all editors with the default ".edtr" extension in the metadata property. Furthermore all editor files will be parsed for a survey title which will be added to the names property. If the parsing fails (no title, unsupported editor format etc.) then javascript's null will be used instead:
curl localhost:8080/1.3/pcapi/editors/local/userid/
Example Output:
{
"metadata": [
"/editors/ddddd.edtr",
"/editors/eeee.edtr",
"/editors/test.edtr",
"/editors/trees.edtr"
],
"names": [
"Survey 1",
"My Survey Title"
"Another title"
null
],
"error": 0
}
A simple GET request returns all record files as a JSON array:
curl localhost:8080/pcapi/records/dropbox/userid
The optional filter argument will filter the returned list of records based on the following values:
Where:
- editor: Filter based on editor id using the format
filter=editor&id=myeditor.edtr - envelope: Filter based on bounding box parameter bbox using the format
filter=envelope&bbox=<xmin>, <ymin>, <xmax>,<ymax> - date: Filter based on a UTC date range defined by parameters start_date and end_date. The end_date is optional and will default to the current time if omitted. Example:
filter=date&start_date=2014-04-25T11:27:30.138Z&end_date=2015-04-25T11:30:330.138Zformat: Return geodata in a specific format, either GeoJSON, KML or CSV.filter=format&frmt=geojson. Additional, the special parameter frmt=database will export all data to a preconfigured PostGIS database according to the values of ./etc/config.ini. - mimetype: Filter on mime type of media format
NOTE:
Currently implemented filters are envelope, editor, format and date.
Example Output:
{
"records": [
{
"duncan st": {
"fields": [{
"id": "fieldcontain-image-1",
"val": "1366031020249.jpg",
"label": "Take"
}],
"point": {
"lat": 55.93561960435346,
"alt": null,
"lon": -3.1763742274678237
},
"editor": "litter in edinburgh.edtr",
"timestamp": "2013-04-15T13:04:00.464Z",
"name": "duncan st"
}
},
{
"desk (5)": {
"fields": [{
"id": "fieldcontain-image-1",
"val": "1365757260427.jpg",
"label": "Image"
}],
"name": "desk (5)",
"editor": "image.edtr",
"timestamp": "2013-04-12T09:01:18.840Z",
"point": {
"lat": 55.93749020527392,
"alt": null,
"lon": -3.1780926655209907
}
}
}],
"error": 0
}
This is an example of having a GET request in order to get back a single record or editor:
curl localhost:8080/1.3/pcapi/editors/dropbox/userid/editor.edtr
Example output:
<form id="form532" data-ajax="false">
<div class="fieldcontain" id="fieldcontain-text-1">
<label for="form-text-1">Title</label>
<input name="form-text-1" id="form-text-1" type="text" required="" placeholder="record name" maxlength="15">
</div>
<div class="fieldcontain" id="fieldcontain-radio-1">
<fieldset data-role="controlgroup">
<legend>Choose</legend>
<label for="form-radio-1">Happy</label>
<input name="form-radio1" id="form-radio-1" value="Happy" type="radio" required="">
<label for="form-radio-2">Sad</label>
<input name="form-radio1" id="form-radio-2" value="Sad" type="radio" required="">
</fieldset>
</div>
<div class="fieldcontain" id="fieldcontain-image-1">
<div class="button-wrapper button-camera">
<input name="form-image-1" id="form-image-1" type="file" accept="image/png" capture="camera" class="camera">
<label for="form-image-1">Take</label>
</div>
</div>
<div id="another_test_form-buttons" class="fieldcontain ui-grid-a">
<div class="ui-block-a">
<input type="submit" name="record" id="532_record" value="Save">
</div>
<div class="ui-block-b">
<input type="button" name="cancel" id="532_cancel" value="Cancel">
</div>
</form>
curl localhost:8080/1.3/pcapi/records/dropbox/userid/record/record.json
Example output:
{
"editor": "another test form.edtr",
"fields": [
{
"id": "fieldcontain-radio-1",
"label": "Choose",
"val": "Happy"
}
],
"name": "feelings",
"point": {
"lon": -3.179603088585148,
"lat": 55.940356699067394,
"alt": null
},
"timestamp": "2013-04-16T10:28:08.204Z"
}
A get request that will return all Key/Value pairs associated with a record:
curl localhost:8080/1.3/pcapi/record/dropbox/userid/my_record_name/myrecord
NOTE:
Not implemented. Currently not part of a use case.
A POST request can point to either a file or a folder. The depth of the path tree is important. If only the record name is provided (depth is 1) then the POST data will be saved as record.json inside that record folder. If that folder already exists a new one will be created. This is best explained by
example:
Assuming the PATH is /myrecord (depth is 1):
If /myrecord does not exist then the file is saved at /myrecord/record.json
If it does exist then the file is saved at /myrecord (2)/record.json (or keeps trying new folder names until a free name is found)
If the PATH is /myrecord/somefile (depth is 2) then POST will create that file or a similarly named file if it already exists.
For cordova compatibility, the file contents may be sent using as multipart/form-data inside a field named file like in the following example:
curl --form file=@somerecord.json localhost:8080/1.3/pcapi/record/dropbox/userid/myrecord/myrecord.json
Example Output:
{
"msg": "File uploaded",
"path": "/records/myrecord/myrecord(2).json", "error": 0
}
A PUT request must always point to a file (normally a record file or an asset file). If that file exists it will be overwritten:
curl -X PUT -T myrecord localhost:8080/1.3/pcapi/record/dropbox/userid/myrecord/myrecord.json
The optional parameter autoexport=true will also export the record to the database table corresponding to the editor/form that generated that record.
Example Output:
{
"msg": "File uploaded", "path":
"/records/myrecord/myrecord.json", "error": 0
}
Same call as geteditors but issued as a PUT request:
curl -X PUT -T myeditor localhost:8080/1.3/pcapi/editor/dropbox/userid/myeditor
Example Output:
{
"msg": "File uploaded",
"path": "/editors/hggjg.edtr", "error":0
}
A POST request of a json object containing the updated K/V fields. Only the changed values should be submitted. To delete a tag, just use an empty string as a value.
Example with curl to create or update a key named Author with value John Doe and to delete a tag named Copyright in one operation:
curl --data 'metadata={"Author":"John Doe","Copyright":""}' https://localhost:8080/pcapi/record/dropbox/userid/myDirectory/myrecord
NOTE:
Not implemented in the API. Records can be edited by the authoring tool.
This is just a special case of update record. To geotag a record just set the GPSLatitude and GPSLongitude keys to a valid lat/lon value e.g.:
curl --data 'metadata={"GPSLongitude":-3.1901,"GPSLatitude":55.9545}' https://localhost:8080/1.3/pcapi/record/dropbox/userid/myDirectory/myrecord
NOTE:
Not implemented. Records are currently geotagged by the authoring tool.
This is an example of a DELETE request:
curl -X DELETE
localhost:8080/1.3/pcapi/records/dropbox/userid/record
The response we get is:
{
"msg": "/records/record deleted",
"error": 0
}
Publish record to PostGIS and (optionally) a Geoserver W*S endpoint
Publishes a record after it has been uploaded with all its assets. Publication depends on a preconfigured PostGIS database specified in pcapi.ini. Optionally a Geoserver endpoint can also be configured to provide access to those records from an OGC compliant WMS/WFS interface.
curl localhost:8080/1.3/pcapi/records/local/userid/record?ogc_sync=true
The nominal response is:
{
"msg": "INSERT 1 0",
"error": 0
}
Synchronization is currently implemented using a simple scheme — it answers the question "What has changed in the remote filesystem since last time I asked?". It is implemented as a single call with an optional CURSOR parameter:
https://localhost:8080/1.3/pcapi/sync/PROVIDER/USERID
or
https://localhost:8080/1.3/pcapi/sync/provider/userid/cursor
| name | details | mandatory |
|---|---|---|
| userid | The USERID returned from a valid authentication. | true |
| provider | The storage provier (e.g. Dropbox) which must return synchronize in its capabilities. | true |
| cursor | A string of characters that represent the state of the filesystem at some point in time. By omitting this argument a new cursor string is created | false |
Returns:
A JSON object with a list of deleted and updated files.
First get a cursor string which represents the current state of the filesystem
curl https://localhost:8080/1.3/pcapi/sync/dropbox/tchzufdt438y76q
This will return a JSON object similar to:
{
"cursor": "Au92d4xq943JlFRE9apZ_IN4VNL15fbjTY8UrtoZgSvAeJxjZcjYzcA4Y",
"error": 0
}
The cursor string should be stored and reused every time a sync is required. E.g. If 2 image files were added (or modified) and another one was deleted the following API call will provide that information:
curl https://localhost:8080/1.3/pcapi/sync/dropbox/tchzufdt438y76q/Au92d4xq943JlFRE9apZ_IN4VNL15fbjTY8UrtoZgSvAeJxjZcjYzcA4Y
Which will return something like:
{
"deleted": ["/records/image1.jpg","/records/image2.jpg"],
"updated": ["/records/record2/audio.wav"], "error": 0
}
The client app should then update its local cache by deleting all entries under deleted and by downloading/overwriting all entries under updated. After synching is complete the client would normally request a new cursor string to reset the tracking of the filesystem to its current state.
Authentication is implemented as a compatibility layer on top of of other cloud providers. The main principle behind the current authentication scheme is that the PC middleware should not act as the user itself but as a middleware with a "power of attorney" for sandboxed access to a fieldtrip-specific subfolder. This requires the user to be redirected to the provider’s website during the temporary token generation.
Specifically:
-
The PC middleware should never know the user’s actual user name and password
-
It should instead use a temporary token for sandboxed access to the user’s folders.
-
Time of validity for the temporary token is usually about a month (it depends on the provider).
Initiate a connection to the specified provider using a GET request in the format
https://localhost:8080/1.3/pcapi/auth/<provider>
or
https://localhost:8080/1.3/pcapi/auth/<provider>/<userid>
Both forms take the optional GET parameter async=true|false to specify whether polling is desired.
| name | details | mandatory |
|---|---|---|
| userid | This is the userid taken from a previous login. It can be passed here to resume that connection without having to relogin to the provider. | false |
| async | specify whether polling is desired. Polling is used when the app has no way of knowing when the user has finished logging in to the provider. | false |
| callback | a URL to redirect the user after login is complete. This parameter should only be used by browser-based clients like the authoring tool because it requires listening on a public internet IP. | false |
Returns:
A JSON object with a url property (hosted by the provider and not by edina) that the user can use to provide his real credentials, a user id key userid that is used by the PC middleware to keep track of the connection and a state variable which can be either 0 (token needs verification using url) or 1 (already connected, you can start issuing commands).
First issue a login request
curl https://localhost:8080/1.3/pcapi/auth/dropbox
Will return a JSON object similar to:
{
"url": "https://www.dropbox.com/1/oauth/authorize?oauth_token=tchzufdt438y76q",
"state": 0, "userid": "tchzufdt438y76q"
}
Now a browser needs to open up (embedded or standalone) pointing to the above url. This url is provider-specific (e.g. dropbox) and will let the user grant temporary access to the fieldtrip app. We assume that the app has some way of knowing when the user has finished logging in with the browser. Once this is done, the app should notify the PC middleware with the following call:
curl https://localhost:8080/1.3/pcapi/auth/dropbox/tchzufdt438y76q
which should return:
{
"url": "https://www.dropbox.com/1/oauth/authorize?oauth_token=tchzufdt438y76q",
"state": 1, "userid": "tchzufdt438y76q"
}
NOTE:
state = 1 means that the userid can be used for all other API calls.
All calls should have the async=true option on.
curl https://localhost:8080/1.3/pcapi/auth/dropbox?async=true
This will return a JSON object similar to:
{
"url": "https://www.dropbox.com/1/oauth/authorize?oauth_token=tchzufdt438y76q&callback=http://localhost:8080/pcapi/callback",
"state": 0, "userid": "tchzufdt438y76q"
}
Now the app needs to open up a browser (embedded or standalone) pointing to the above url. We assume that the app cannot know when the user has finished logging in with the browser. It has to start polling the middle-ware at regular intervals (e.g. 1 sec.) to check that the user has finished logging in. This happens when the return value for state is 1 instead of 0.
curl https://localhost:8080/1.3/pcapi/auth/dropbox/tchzufdt438y76q?async=true
which should eventually return:
{
"url": "https://www.dropbox.com/1/oauth/authorize?oauth_token=tchzufdt438y76q&callback=http://localhost:8080/pcapi/callback",
"state": 1, "userid": "tchzufdt438y76q"
}
The URL pattern used for accessing all the records should start with /records. E.g.
localhost:8080/1.3/pcapi/records/<PROVIDER>/<USERID>/<PATH>
The PATH convention used is as follows:
For getting all the assets you just go for assets.
For getting specific assets such as images, audio, gpx in two different formats
A simple GET request that will search and return for all the records that have images or audio files or tracks will be:
curl localhost:8080/1.3/pcapi/editors/records/userid/assets/
Example Output:
{
"records": [
{
"building": {
"fields": [
{
"id": "fieldcontain-image-1",
"val": "1365773558058.jpg",
"label": "Image"
}
],
"point": {
"lat": 55.93963413449301,
"alt": 110.9000244140625,
"lon": -3.1841814019307977
},
"editor": "image.edtr",
"timestamp": "2013-04-12T13:32:49.665Z",
"name": "building"
}
},
{
"desk (3)": {
"fields": [
{
"id": "fieldcontain-image-1",
"val": "1365582544988.jpg",
"label": "Image"
}
],
"name": "desk (3)",
"editor": "image.edtr",
"timestamp": "2013-04-10T08:29:25.551Z",
"point": {
"lat": 55.934124205790596,
"alt": null,
"lon": -3.1744371625638523
}
}
}
]
}
The same GET request as before that will return back all the assets but in url format in order the user afterwards to get a public url of the image is:
curl localhost:8080/1.3/pcapi/editors/records/userid/assets/?frmt=url
Example Output:
{
"records": [
"findhorn place/1363864408127.jpg",
"bottle/1365586638581.jpg",
"desk (2)/1365583269426.jpg",
],
"error": 0
}
A simple GET request that will search and return all the records that have only images would be:
curl localhost:8080/1.3/pcapi/editors/records/userid/assets/images/
Example Output:
{
"records": [
{
"duncan st": {
"fields": [
{
"id": "fieldcontain-image-1",
"val": "1366031020249.jpg",
"label": "Take"
}
],
"point": {
"lat": 55.93561960435346,
"alt": null,
"lon": -3.1763742274678237
},
"editor": "litter in edinburgh.edtr",
"timestamp": "2013-04-15T13:04:00.464Z",
"name": "duncan st"
}
},
{
"desk (5)": {
"fields": [
{
"id": "fieldcontain-image-1",
"val": "1365757260427.jpg",
"label": "Image"
}
],
"name": "desk (5)",
"editor": "image.edtr",
"timestamp": "2013-04-12T09:01:18.840Z",
"point": {
"lat": 55.93749020527392,
"alt": null,
"lon": -3.1780926655209907
}
}
}
]
}
A simple GET request that will search and return all the records that have only images in url format would be:
curl localhost:8080/1.3/pcapi/editors/records/userid/assets/images/?frmt=url
A simple GET request that will search and return all the records that have only audio would be:
curl localhost:8080/1.3/pcapi/editors/records/userid/assets/audio/
If the user wants all the audio files in url format then he needs to add this parameter:
curl localhost:8080/1.3/pcapi/editors/records/userid/assets/audio/?frmt=url
A simple GET request that will search and return all the records that have only gpx files would be:
curl localhost:8080/1.3/pcapi/editors/records/userid/assets/gpx/
If the user wants all the audio files in url format then he needs to add this parameter:
curl localhost:8080/1.3/pcapi/editors/records/userid/assets/gpx/?frmt=url