`easy-rdf-endpoint` - Use Docker to deploy and use a RDF Catalog

Overview

This project provides a simple Dockerized RDF endpoint that simplifies catalog deployment of .rdf files, provides an SPARQL Endpoint based on vemonet/rdflib-endpoint and incorporates interfaces for both semantic and syntactic validation of RDF files.

Tip

Quick deployment: Use our pre-built container images with docker-compose -f docker-compose.ghcr.yml up -d to deploy a complete RDF ecosystem in under 5 minutes, without having to build images locally!

About

This project provides a simple Dockerized RDF endpoint that simplifies catalog deployment, provides a SPARQL endpoint

SPARQL Features: Advanced RDF Query Capabilities
- Integrated YASGUI editor with multi-tab support
- Pre-configured example queries
- Machine-readable SPARQL endpoint
Validator: RDF Catalog Validation
- Semantic validation using SHACL
- Syntactic validation using RIOT
- Interactive validator interface
- Downloadable validation errors and warnings reports
- At-a-glance SHACL validation statistics
Public RDF catalog: RDF Catalog Serving:
- Serves Catalog .rdf files with proper Content-Type: application/rdf+xml
- Compatible with DCAT-AP federators and RDF parsers
Easy Deployment:
- Docker and Kubernetes ready
- Environment-based configuration
Security & Performance:
- NGINX reverse proxy
- SSL/TLS support
- Configurable CORS

Quick Start

Use Codespaces to test easy-rdf-endpoint in your browser

Getting started

Requirements:

Docker

Clone and configure

Before starting the deployment, you'll need to set up a .env file. This file is crucial as it contains environment variables that the application needs to run properly.

Clone project

cd /path/to/my/project
git clone https://github.com/mjanez/easy-rdf-endpoint.git & cd easy-rdf-endpoint

Place your RDF catalog file in the ./data folder:

# Copy your existing RDF catalog file
cp /path/to/your/my_custom_catalog.rdf ./data/

# Or create a sample RDF file if you don't have one
cat > ./data/my_custom_catalog.rdf << 'EOF'
<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF
  xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  xmlns:dcat="http://www.w3.org/ns/dcat#"
  xmlns:dct="http://purl.org/dc/terms/">
  
  <dcat:Catalog rdf:about="http://example.org/catalog">
  <dct:title xml:lang="en">Sample Data Catalog</dct:title>
  <dct:description xml:lang="en">A sample RDF catalog for testing</dct:description>
  <dct:publisher rdf:resource="http://example.org/publisher"/>
  <dct:issued>2023-01-01</dct:issued>
  </dcat:Catalog>
</rdf:RDF>
EOF

# Make sure the file has proper permissions
chmod 644 ./data/my_custom_catalog.rdf

Copy the .env.example template and modify the resulting .env to suit your needs.

cp .env.example .env

Adjust the vars as necessary, example the server name if not use localhost:

# Server Configuration
PROXY_SERVER_NAME=my-example-sparql-server.org

# RDF Catalog File Configuration
# RDF filename in ./data to be served at localhost/catalog.rdf and loaded to Virtuoso
CATALOG_FILE=my_custom_catalog.rdf

# Edit your custom SPARQL queries that load as tabs at startup
EXAMPLE_SPARQL_QUERIES='{
    "HVD info": {
        "query": "prefix dct: <http://purl.org/dc/terms/>\nprefix dcatap: <http://data.europa.eu/r5r/>\nprefix dcat: <http://www.w3.org/ns/dcat#>\n\nselect distinct ?title ?hvdCategory ?applicableLegislation ?accessService ?accessURL ?license\nwhere {\n  ?catalogo ?cp ?d.\n\n    # Dataset y su categoría HVD\n    ?d dcatap:applicableLegislation <http://data.europa.eu/eli/reg_impl/2023/138/oj>.\n    ?d a dcat:Dataset.\n    optional { ?d dcatap:hvdCategory ?hvdCategory. }\n    \n    # Distribution and its props\n    ?d dcat:distribution ?dist.\n    ?dist dcatap:applicableLegislation <http://data.europa.eu/eli/reg_impl/2023/138/oj>.\n    \n    optional { \n        ?dist dct:title ?title.\n        FILTER(langMatches(lang(?title), \"en\"))\n    } \n    optional { ?dist dcatap:applicableLegislation ?applicableLegislation. } \n    optional { ?dist dcat:accessURL ?accessURL. } \n    optional { ?dist dcat:accessService ?accessService. } \n    optional { ?dist dct:license ?license. } \n}\nORDER BY ?dist"
    },
    "Count stats": {
        "query": "PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\nPREFIX dcat: <http://www.w3.org/ns/dcat#>\nPREFIX dct: <http://purl.org/dc/terms/>\n\nSELECT \n  (COUNT(DISTINCT ?dataset) AS ?totalDatasets)\n  (COUNT(DISTINCT ?distribution) AS ?totalDistributions)\n  (COUNT(DISTINCT ?dataservice) AS ?totalDataservices)\n  (COUNT(DISTINCT ?publisher) AS ?totalPublishers)\nWHERE {\n  {\n    ?dataset a dcat:Dataset .\n    OPTIONAL { ?dataset dct:publisher ?publisher }\n    OPTIONAL { ?dataset dcat:distribution ?distribution }\n  }\n  UNION\n  {\n    ?dataservice a dcat:DataService ;\n                 dcat:servesDataset ?dataset .\n  }\n}"
    },
    "Query types": {
        "query": "PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\nPREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\nPREFIX dct: <http://purl.org/dc/terms/>\nSELECT DISTINCT ?type ?label (COUNT(?s) as ?count) ?comment WHERE {\n    ?s rdf:type ?type .\n    OPTIONAL { ?type rdfs:label ?label }\n    OPTIONAL { ?type rdfs:comment ?comment }\n} \nGROUP BY ?type ?label ?comment\nORDER BY DESC(?count)\nLIMIT 10"
    }
}'

Build & up the container.

docker compose up -d

Tip

Use pre-built images: To save time, you can use our pre-built images hosted on GitHub Container Registry:

docker compose -f docker-compose.ghcr.yml up -d

This approach is faster as it pulls ready-to-use images instead of building them locally!

Then, access your RDF file an endpoints.

Endpoints

The endpoints will be available at:

Landing pagehttp://localhost:5000
- Access your RDF file at: http://localhost:5000/catalog
- Default query editor (Yasgui): http://localhost:5000/sparql-editor
- SPARQL endpoint: http://localhost:5000/sparql
- RDF syntactic/semantic validator: http://localhost:5000/validator

Project Structure

/easy-rdf-endpoint
│── /data/catalog.rdf          # Your Catalog RDF file (replace with your own)
│── /doc/                      # References, as images or documents, used in repo
│── /easy-rdf-endpoint/        # RDF+SPARQL Endpoint setup
│── /nginx/                    # NGINX docker setup
│── /src/                      # Now using NGINX instead of the old RDF serving mode
│── .env.example               # Sample ENVVars file for use with Docker Compose
│── docker-compose.yml         # Docker Compose config
│── README.md                  # Documentation

Tip

Replace catalog.rdf with your actual RDF file.
Works with Codespaces, Docker, Kubernetes, and any containerized environment.

Translation

After updating the translation file (messages.po), don't forget to compile it to generate the .mo file, e.g English:

cd easy-rdf-endpoint/src/rdf-validator

# Extract i18n texts and update POT
xgettext -d messages --from-code=UTF-8 -o locales/messages.pot app.py

# Compile MO files (English)
msgfmt -o locales/en/LC_MESSAGES/messages.mo locales/en/LC_MESSAGES/messages.po

Update SSL Certificate

To update the local SSL certificate, follow these steps:

Generate a new certificate and private key:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout nginx/setup/easy-rdf-endpoint_local.key \
  -out nginx/setup/easy-rdf-endpoint_local.crt \
  -subj "/C=ES/ST=Madrid/L=Madrid/O=Development/CN=localhost"

Verify that the files have been created correctly:

ls -l nginx/setup/easy-rdf-endpoint_local.*

Restart the nginx container to apply the changes:

docker compose restart nginx

Caution

This certificate is for local development only. In production, use a valid certificate from a certificate authority.

Development

Using DevContainers

This project supports development using DevContainers, which provides a consistent development environment for all contributors.

Requirements:

Docker

VS Code or GitHub Codespaces
VS Code Remote - Containers extension

Getting Started with DevContainer

Clone the repository:

git clone https://github.com/mjanez/easy-rdf-endpoint.git
cd easy-rdf-endpoint

Open in VS Code:

code .

When prompted "Reopen in Container", click "Reopen in Container". Alternatively:
- Press F1
- Type "Remote-Containers: Reopen in Container"
- Press Enter

VS Code will build and start the development container. This may take a few minutes the first time.

What's Included

The development container includes:

Python 3.11 environment
Required Python packages pre-installed
Popular VS Code extensions for Python/Docker development
Git configuration
Port forwarding for the application

Working in the Container

Once inside the container you can:

Run the application: exec gunicorn --bind 0.0.0.0:5000 app:app
Modify the easy-rdf-endpoint/setup/rdflib-endpoint/sparql_override.py
Execute SPARQL queries: http://localhost:5000/sparql
Debug with VS Code's integrated debugger

Image Credits

Some icons used in this project are from:

SVG Repo under CC Attribution License
ckanext-schemingdcat repository

License

This work is a developed inspired by rdflib-endpoint by Vincent Emonet, licensed under MIT License.

This derivative work is licensed under Creative Commons Attribution 4.0 International License.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

`easy-rdf-endpoint` - Use Docker to deploy and use a RDF Catalog

Overview

About

Quick Start

Getting started

Clone and configure

Endpoints

Project Structure

Translation

Update SSL Certificate

Development

Using DevContainers

Getting Started with DevContainer

What's Included

Working in the Container

Image Credits

License

Files

README.md

Latest commit

History

README.md

File metadata and controls

easy-rdf-endpoint - Use Docker to deploy and use a RDF Catalog

Overview

About

Quick Start

Getting started

Clone and configure

Endpoints

Project Structure

Translation

Update SSL Certificate

Development

Using DevContainers

Getting Started with DevContainer

What's Included

Working in the Container

Image Credits

License

`easy-rdf-endpoint` - Use Docker to deploy and use a RDF Catalog