Replies: 9 comments 22 replies
-
|
Well there's Commons RDF but that's essentially a dead project because none of the contributing projects could agree on anything 🤣 Turning this around could we think about inverting the control flow here? Part of the overhead right now AFAIK is that we have to hand full control over to Titanium and then convert its final output data model back into Jena's data model. If instead Titanium had an output interface in the vein of Jena's StreamRDF then we could hand a Jena specific implementation of that interface to Titanium. Then Titanium call the relevant methods on its interface as it produces triples which Jena can then immediately convert to its own types in a streaming fashion. I don't know if this is feasible within the JSON-LD processing model but perhaps worth considering. WDYT? |
Beta Was this translation helpful? Give feedback.
-
|
Ok, not sure what aspect of Note that the lack of ability to stream process JSON-LD doesn't mean that that style of output interface isn't a possibility to consider, ultimately JSON-LD, like any RDF serialisation should just emit a list of triples when parsed. If you have some idea of what this hypothethical interface looks like in your head could you perhaps sketch it out as psuedo-code or equivalent so we aren't talking at cross purposes? |
Beta Was this translation helpful? Give feedback.
-
|
My goal for now, is to collect ideas, level of an interest, and requirements, then we can sketch the interface. Having an interface that allows you to pull out quads in a stream-like way is one option. I'm not saying it's a bad idea. The first question is, do we commit to it? Second, what are the requirements? And third, what should it look like? |
Beta Was this translation helpful? Give feedback.
-
|
Retro-fitting interface to existing code runs up against the fact that then there two ways to doing things. i.e. it is a significant impact on the existing system. Expanding on Rob's point - Having the parser stream out would be very good - no whole-parse temporary copy. Could the parser take functions (with generics?) to call to deliver the parts of URIs, literals, bnodes? Then Titanium user can create the RDF terms using native objects and stream as quads/triples? It's easier to work in "generic RDF" - a triple is 3 RDF terms, no restrictions on predicate must be URIs. This may be a part of why CommonsRDF does not get traction. At lower levels of systems, the asymmetric subject/predicate/object is a nuisence. |
Beta Was this translation helpful? Give feedback.
-
|
I've made a fork of Titanium where I use generics. I'll try to see if I can post some code to show how this helps with my use in RDF4J. |
Beta Was this translation helpful? Give feedback.
-
|
Here are some of the changes I've made to Titanium: When converting JSON-LD to RDF I've added generics, a consumer and a value factory. Here is how I use it from RDF4J: When writing data from RDF4J to JSON-LD I pass in my own dataset adapter: And I've made adapter classes for converting between the RDF4J representation and the Titanium representation of triples/quads/iris/etc. These could be optimized a lot when converting back and froth by keeping track of the internal RDF4J object. |
Beta Was this translation helpful? Give feedback.
-
|
Current Titanium API works as a supplier (PULL), and accepts only complete data (PULL). requirements
PUSH: Titanium -> Jena// Jena calls toRDF
void JsonLd.toRDF(jsonld).options(...).produce(consumer);
//Titanium writes RDF to the provided consumer inside produce method
loop [
// Jena provided code called inside produce method
consumer.dataset(...);
consumer.accept(...)
]
// Jena gets back full control and might close the consumer
consumer.end(); // can be dropped, as toRDF's done writingPUSH: Jena -> Titanium// Jena calls fromRDF
<RDFConsumer, Supplier<JsonArray>> consumer = JsonLd.fromRDF().options(...).consumeAndSupply();
// Jena writes RDF to the consumer
loop [
consumer.dataset(...).accept(...).accept(...).accept(...) // Flow?
.dataset(...).accept(...);
]
// Jena closes the consumer
consumer.end();
// Jena gets JSON-LD
JsonArray jsonld = consumer.get(); // could act as end()APIinterface RDFConsumer {
dataset(name);
accept(resource, predicate, value, datatype);
accept(resource, predicate, value);
// end can be dropped to keep it minimal
end();
}Just brainstorming, thank you for the comments. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah I think this looks like a good direction to explore. For the Does this mean there's multiple overloads for accept(String resource, URI predicate, URI value);
accept(URI resource, URI predicate, String value, URI datatype);
accept(URI resource, URI predicate, URI value);The main possible confusion I could see there is how you send Blank Nodes. For the subject position it's kinda obvious that I think the other possibility that was suggested was something like the following: interface RDFConsumer<T> {
// Converters from simple types to the consumers strong type
T uri(String uri);
T blankNode(String id);
T literal(String value, String lang, String datatype);
// Output
dataset(T name);
accept(T resource, T predicate, T value);
end();
}And then your loop becomes something like the following: loop [
// Jena provided code called inside produce method
T graph = ...;
T subject = consumer.uri(u);
// etc.
consumer.dataset(...);
consumer.accept(...)
]Makes the logic on Titanium's side slightly more complex but its allowing the consumer to control the target types directly. For going the other way it gets more complex, so maybe you need |
Beta Was this translation helpful? Give feedback.
-
|
Creating Java URI objects is very expensive, performance wise. I recommend avoiding that at all cost. Generics is preferable for RDF4J. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, @afs I’d love to hear your feedback before it goes out. Thanks! package com.apicatalog.rdf;
/**
* RDF dataset consumer interface designed for
* speed, streamlined processing, and efficiency.
*
* This interface minimizes unnecessary object instantiation
* and facilitates seamless integration with third-party libraries,
* enabling efficient RDF data processing.
*/
public interface RdfConsumer {
/**
* Sets the default graph as the active scope.
* Invoked when triples belong to the unnamed, default graph.
*/
void defaultGraph();
/**
* Sets a named graph as the active scope.
* Ensures that subsequent triples are associated with the specified graph.
*
* @param graph The name of the graph (IRI or blank node identifier).
* @param blankGraph {@code true} if the graph name is a blank node identifier.
*/
void namedGraph(String graph, boolean blankGraph);
/**
* Accepts a new RDF triple where the object is an IRI or a blank node.
* The triple is processed within the currently active graph scope.
*
* @param subject The subject of the triple (IRI or blank node identifier).
* @param blankSubject {@code true} if the subject is a blank node identifier.
* @param predicate The predicate of the triple (IRI or blank node identifier).
* @param blankPredicate {@code true} if the predicate is a blank node identifier.
* @param object The object of the triple (IRI or blank node identifier).
* @param blankObject {@code true} if the object is a blank node identifier.
*/
void accept(
String subject,
boolean blankSubject,
String predicate,
boolean blankPredicate,
String object,
boolean blankObject
);
/**
* Accepts a new RDF triple where the object is a literal value.
* The triple is processed within the currently active graph scope.
* Optimized for efficient handling of typed and language-tagged literals.
*
* @param subject The subject of the triple (IRI or blank node identifier).
* @param blankSubject {@code true} if the subject is a blank node identifier.
* @param predicate The predicate of the triple (IRI or blank node identifier).
* @param blankPredicate {@code true} if the predicate is a blank node identifier.
* @param literal The literal value of the object.
* @param datatype The datatype IRI of the literal, never null.
* @param language The language tag of the literal (optional, may be {@code null}).
*/
void accept(
String subject,
boolean blankSubject,
String predicate,
boolean blankPredicate,
String literal,
String datatype,
String language
);
} |
Beta Was this translation helpful? Give feedback.
-
|
This looks like it would make it a lot easier to reduce the overhead between Titanium and RDF4J too 😀! I get the tradeoffs that make it necessary to have booleans for denoting if the values represent blank nodes. Will the value for the blank node start with I assume that we will be able to provide an implementation at parse time, so that we can create a separate implementation based on which database backend the data should be stored in. Otherwise it probably wouldn't work to have the statefulness for the named graph. Keeping the design purely functional would be something that I would prefer. E.g. add another argument to the end of the two accept methods for a string that represents the named graph, or null for the default graph. |
Beta Was this translation helpful? Give feedback.
-
|
@hmottestad Thank you for the feedback!
Agreed, the javadoc will be updated. A blank node name starts with
Yes, the goal is to allow consumers to set their own implementation while also providing some generic implementations to facilitate adoption. e.g. RdfDatasetSupplier or RdfNQuadConsumer. See proposed use JSON-LD to RDF example.
The design reflects graph processing from Titanium's perspective, where graphs are separate. It sets the scope once and then emits all triples belonging to that graph. It's aimed to strike a balance between a purely functional design, where a single method would have ten parameters, and SAX like approach with "begin/end" methods and a method for each primitive and its variants. What about making the API fluent and perhaps change name from RdfConsumer defaultGraph();
RdfConsumer namedGraph(String graph, boolean blankGraph);
// usage
consumer.defaultGraph().triple(...); This would allow an overlay implementation, a helper on top, that emits complete N-Quads ( |
Beta Was this translation helpful? Give feedback.
-
|
For RDF 1.2 proofing, add a base direction argument to the accept-litteral form. JSON-LD already has base direction. void accept(
String subject,
boolean blankSubject,
String predicate,
boolean blankPredicate,
String literal,
String datatype,
String language,
String baseDirection
);There is also "triple term" coming but it's not JSON-LD yet. |
Beta Was this translation helpful? Give feedback.
-
|
That's correct, thank you. In the proposal, language includes a direction, serialized in the form Options:
Thank you all for your time! |
Beta Was this translation helpful? Give feedback.
-
Yes, but JSON-LD-star and Titanium already have some partial support built-in. I would love to reflect this, but given the various interested parties, I only wish to settle on a plain consumer for now. ;) |
Beta Was this translation helpful? Give feedback.
-
I tend towards this - all look to be workable. |
Beta Was this translation helpful? Give feedback.
-
|
I agree about making it quads (only) but if, for Titanium, the triples style fits better then fine. It's not a huge difference. re: name "accept" or "triple" or "quad" - no preference, just not both "quad" and "triple". Is the blank node identifier a system generated unique id (UUID-like) or a local label encountered by the JSON-LD parser? If a blank node name starts with Will Titanium be able to stream-write via the supplier interface? |
Beta Was this translation helpful? Give feedback.
-
It depends. For lists, it’s always generated. For nodes, if it's present, it’s reused; otherwise, it’s generated. It’s not a UUID, it's unique only within a dataset.
Yes, a quad is emitted when created, effectively blocking further processing until the callback finishes. |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Agreed, but given the number of libraries depending on Titanium, I have to keep the option, even marked as deprecated. I guess Jena blocks the use of the useGeneralizedRdf option to prevent blank predicates from being emitted. (I've removed the mention of blank predicates from the consumer Javadoc.) |
Beta Was this translation helpful? Give feedback.
-
|
RDF4J has also chosen not to attempt to implement generalised RDF. |
Beta Was this translation helpful? Give feedback.
-
|
@afs @hmottestad Thank you very much for the feedback! This was very productive, and I hope we've found a good compromise while keeping the objective. If it's okay with you, then it's close to being shipped, perhaps after some quick polishing and refactoring, within a week. package com.apicatalog.rdf;
/**
* An optimized RDF quad consumer interface designed for high-speed processing
* and seamless integration with third-party libraries.
*
* This interface minimizes unnecessary object instantiation, enhancing
* efficiency in RDF data processing.
*/
public interface RdfQuadConsumer {
/**
* Accepts an RDF quad where the object is an IRI or a blank node.
*
* @param subject The subject of the quad (IRI or blank node identifier
* prefixed with "_:"); never {@code null}.
* @param predicate The predicate of the quad (IRI); never {@code null}.
* @param object The object of the quad (IRI or blank node identifier
* prefixed with "_:"); never {@code null}.
* @param graph The graph (IRI or blank node identifier prefixed with "_:"),
* or {@code null} for the default graph.
*
* @return An instance enabling fluent programming; never {@code null}.
*/
RdfQuadConsumer quad(
String subject,
String predicate,
String object,
String graph);
/**
* Accepts an RDF quad where the object is a literal with a datatype. Optimized
* for efficient handling of typed literals.
*
* @param subject The subject of the quad (IRI or blank node identifier
* prefixed with "_:"); never {@code null}.
* @param predicate The predicate of the quad (IRI); never {@code null}.
* @param literal The literal value of the object; never {@code null}.
* @param datatype The datatype IRI of the literal; never {@code null}.
* @param graph The graph (IRI or blank node identifier prefixed with "_:"),
* or {@code null} for the default graph.
*
* @return An instance enabling fluent programming; never {@code null}.
*/
RdfQuadConsumer quad(
String subject,
String predicate,
String literal,
String datatype,
String graph);
/**
* Accepts an RDF quad where the object is a localized string value. Optimized
* for efficient handling of language-tagged literals.
*
* @param subject The subject of the quad (IRI or blank node identifier
* prefixed with "_:"); never {@code null}.
* @param predicate The predicate of the quad (IRI); never {@code null}.
* @param literal The literal value of the object; never {@code null}.
* @param language The language code of the literal; never {@code null}.
* @param direction The text direction of the literal (optional, may be
* {@code null}).
* @param graph The graph (IRI or blank node identifier prefixed with "_:"),
* or {@code null} for the default graph.
*
* @return An instance enabling fluent programming; never {@code null}.
*/
RdfQuadConsumer quad(
String subject,
String predicate,
String literal,
String language,
String direction,
String graph);
} |
Beta Was this translation helpful? Give feedback.
-
|
One thing I just realised is that JSON-LD often knows the namespace already. It would be a huge savings if Titanium didn't first have to concatenate the namespace and local name, before we have to split it up again. Not sure how to support that without an explosion of different methods and arguments. |
Beta Was this translation helpful? Give feedback.
-
|
@hmottestad The JSON-LD to RDF algorithm uses expansion as the first step, and unfortunately, the information about what has been expanded to what is missing at the RDF stage. BTW, I would love to have an alternative JSON-LD processing mode that preserves the mapping between the compacted and expanded forms, providing information on which context term was used to compact which term, but that's out of scope. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, package com.apicatalog.rdf.api;
/**
* RDF quad consumer interface designed for high-performance processing and
* seamless integration with third-party libraries.
* <p>
* This interface minimizes unnecessary object instantiation, improving
* efficiency when handling RDF data at scale.
* <p>
* Use the provided static helper methods to analyze and validate consumer
* parameters.
*/
@FunctionalInterface
public interface RdfQuadConsumer {
/**
* Consumes an RDF quad where the {@code object} may be an IRI, blank node,
* typed literal, or language-tagged literal.
* <p>
* This method provides fine-grained control over RDF quad data, allowing
* precise handling of datatypes, language tags, and text direction.
*
* @param subject the subject of the quad; must be an IRI or blank node
* identifier prefixed with "<code>_:</code>". Must not be
* {@code null}.
* @param predicate the predicate of the quad; must be an IRI. Must not be
* {@code null}.
* @param object the object of the quad; must be either:
* <ul>
* <li>an IRI</li>
* <li>a blank node identifier prefixed with
* "<code>_:</code>"</li>
* <li>a literal value, when {@code datatype} is not
* {@code null}</li>
* </ul>
* Must not be {@code null}.
* <p>
* Use {@link #isValidObject(String, String, String)},
* {@link #isLiteral(String, String, String)},
* {@link #isLangString(String, String, String)}, and
* {@link #isDirLangString(String, String, String)} to validate
* and classify the input.
* @param datatype the datatype IRI of the literal. Must be {@code null} if
* {@code object} is not a literal. Must not be {@code null}
* when {@code language} or {@code direction} is provided.
* @param language the language tag of the literal. May be {@code null}, but
* must not be {@code null} if {@code direction} is provided.
* @param direction the text direction of the literal. Optional; may be
* {@code null}.
* @param graph the graph name of the quad; must be an IRI or blank node
* identifier prefixed with "<code>_:</code>". May be
* {@code null} to indicate the default graph.
*
* @return a reference to this consumer, enabling fluent chaining; never
* {@code null}.
*
* @throws RdfConsumerException if an error occurs while processing the quad
* statement.
*/
RdfQuadConsumer quad(
String subject,
String predicate,
String object,
String datatype,
String language,
String direction,
String graph) throws RdfConsumerException;
/**
* Determines if the provided combination of {@code datatype}, {@code language},
* and {@code direction} qualifies the object as RDF literal.
*
* @param datatype the datatype IRI
* @param language the language tag
* @param direction the text direction
* @return {@code true} indicating a literal, otherwise {@code false}.
*/
static boolean isLiteral(String datatype, String language, String direction) {
return datatype != null;
}
/**
* Determines if the provided combination of {@code datatype}, {@code language},
* and {@code direction} qualifies the object as an RDF language-tagged string
* literal with no specified direction.
*
* @param datatype the datatype IRI
* @param language the language tag
* @param direction the text direction
* @return {@code true} if the provided object is RDF language-tagged literal,
* otherwise {@code false}.
*/
static boolean isLangString(String datatype, String language, String direction) {
return datatype != null && language != null && direction == null;
}
/**
* Determines if the provided combination of {@code datatype}, {@code language},
* and {@code direction} qualifies the object as an RDF directional
* language-tagged string literal with a specified direction.
*
* @param datatype the datatype IRI
* @param language the language tag
* @param direction the text direction
* @return {@code true} if the provided object is RDF directional
* language-tagged literal, otherwise {@code false}.
*/
static boolean isDirLangString(String datatype, String language, String direction) {
return datatype != null && language != null && direction != null;
}
/**
* Validates whether the object is a valid RDF object based on the presence of
* {@code datatype}, {@code language}, and {@code direction}.
*
* @param datatype the datatype IRI
* @param language the language tag
* @param direction the text direction
* @return {@code true} if the object is valid according to RDF term rules.
*/
static boolean isValidObject(String datatype, String language, String direction) {
return datatype != null
? (language != null || direction == null)
: (language == null && direction == null);
}
/**
* Checks whether the provided resource identifier represents a blank node. A
* blank node identifier must start with "<code>_:</code>".
*
* @param resource the resource identifier to check; may be {@code null}.
* @return {@code true} if the resource is a non-null blank node identifier;
* otherwise {@code false}.
*/
static boolean isBlank(String resource) {
return resource != null && resource.startsWith("_:");
}
}Supported by |
Beta Was this translation helpful? Give feedback.
-
|
Great news! |
Beta Was this translation helpful? Give feedback.
-
|
I've done a quick conversion and it works nicely. |
Beta Was this translation helpful? Give feedback.
-
|
Great to hear! It's the bare minimum - the API and the helper methods. I hope any missing patterns will emerge over time. Also, the |
Beta Was this translation helpful? Give feedback.
-
I'm not seeing a change to Writing is harder to abstract away completely because it is where application choices (context, compaction. framing, ...) come in. Jena provides some options but, when the app wants detailed control, it is better for the application to convert a Jena RDF Dataset to a Titanium RdfDataset and use the Titanium APIs. Jena code would be no more than translating from some Jena JSON-LD specific settings to Titanium calls in a 1-1 fashion. |
Beta Was this translation helpful? Give feedback.
-
You are right. It was meant as the next candidate to use the interface. e.g.: var consumer = fromRdf().consumer();
thirdParty.emit(consumer);
var json = consumer.get();The long-term goal is to strip Titanium JSON-LD of RDF types, except for the consumer interface, and let the end-user decide which implementation (if any) to use for materializing statements. FYI: The RDF Canon implementation benefits quite a bit from having an interface free of materialized objects. It uses its own performance-optimized internal quad representation, designed specifically for efficient RDF canonicalization and nothing else. |
Beta Was this translation helpful? Give feedback.
-
Let's collaborate on a new repository with RDF dataset interfaces that would allow seamless integration between Jena & Titanium and maybe other libs, like URDNA2015, N-QUADS Reader/Writer (bundled with Titanium now) etc.
The interface(s) should be minimal and single purpose, to exchange RDF dataset, i.e. reading. It is hard, and there are existing attempts, but something like micro-jars, containing only minimum necessary, single purpose, interfaces could be a way to deal with limitations/features coming from strongly typed language and dependencies linking enforcing strong consistency. In comparison to the JavaScript world where compatibility is declared.
Beta Was this translation helpful? Give feedback.
All reactions