Shared config objects

[kirkbro]

Is there any way to use an object multiple times in a spec and have it generate a shared config type?

E.g.

example {
  a: ${shared}
  b: [${shared}]
}

Where shared could be something like:

shared {
  c: string
  d: {
    e: int
    f: double
  }
}

I often run into this use case, where different parts of the config share a common object.
The above generates two differently named types for shared, forcing you to implement whatever logic consumes the shared object twice.

Is there a way around this, or do you have a recommended approach for such scenarios?

[carueda]

Agree with your observation. This is a weak point in the tool.

Here's a basic idea toward providing support for this:

• introduce some special syntax or annotation to indicate that a given object is to be referenced in other places, eg.:

"^commonObj" {
  c: string
  d: {
    e: int
  }
}

or:

#@shared 
commonObj {
  c: string
  d: {
    e: int
  }
}

• then, refer to the definition above like this:

example {
  a: "^commonObj"
  b: [ "^commonObj" ]
}

Something like the above should not be too difficult to implement, but I'm not sure when I'll have the time. In the mean time, any reactions to the above would be most welcome. Thanks!

[carueda]

The ${shared} mechanism to refer to a shared object is actually appealing (compared to a new mechanism like "^commonObj"), but I should have mentioned that the wrapper generator starts by "resolving", that is, "expanding" the given input. Unconditionally removing this .resolve() call would be a breaking change for any users out there already making use of the ${foo} reference mechanism. That's why my suggested alternatives.

However, we could consider an option, say --shared-objects (or even, --no-resolve), that, when given, would make the generator skip the .resolve() call and just use the given input directly. Then it can process any ${x} references for reuse.

The adjustments to your input example would be:

#@only-for-ref   (*)
shared {
  c: string
  d: {
    e: int
  }
}

example {
  a: ${shared}
  b: [ ${shared} ]
}

(*) An optional @only-for-ref annotation would indicate that the object is only for reference in other places, not for direct generation of a config entry right where introduced.

[kirkbro]

I actually just used ${shared} to show the intent of sharing objects, but I agree it is somewhat appealing. I guess an argument in its favour is that any time you use simple substitution in a spec, the generated classes will always be interchangeable, so you might as well share a single class.

However, it gets a bit tricky when combined with object merging (a conceivable use case would be to extend another spec object, e.g. ${master.object} { extension: int }). Or defining the type of something using substitution, e.g. specifying the type of all IDs in the spec.

But if we can disregard those edge cases, your input example, combined with a feature option to be backwards compatible, seems like a great solution.

[carueda]

Just did a quick test of having a: ${shared} and loading it with no .resolve() call. I thought that a simple string ${shared} would be associated with a so tscfg could proceed with the suggested processing above, but actually Config complains with:

com.typesafe.config.ConfigException$NotResolved: need to Config#resolve(),
 see the API docs for Config#resolve(); 
 substitution not resolved: ConfigReference(${shared})

But this is not too bad actually. The idea now is even simpler (for the user):

The example input config would look like this:

#@define
Shared {
  c: string
  d: {
    e: int
  }
}

example {
  a: Shared
  b: [ Shared ]
}

• the @define (or perhaps @shared) annotation indicates a definition to be used elsewhere in the config spec;
• and the reference to the definition is just with the simple name of the defined attribute;
• with the above:
  • generator continues to call .resolve() upon loading the input config -- no changes there;
  • no need for any additional command line options; the new feature is explicitly exercised with the use of @define.

Thoughts?

[kirkbro]

Interesting that it's not allowed to not resolve substitutions.

#@define
Shared {
  c: string
  d: {
    e: int
  }
}

example {
  a: Shared
  b: [ Shared ]
}

Wow, I really like this idea. It fits the style much better than the substitution "hack". On @define vs @shared, I think @define makes the most sense.

Where would it be allowed to define these objects? For my use limiting them to the spec root would suffice, but I guess for very large specs it could make sense to define them local to their use. I.e.

a {
  #@define
  Shared {
    b: string
    c {
      d: int
    }
  }

  e: Shared
  f: [ Shared ]
}

On the other hand it would raise questions about scoping etc., so it's probably not worth the hassle.

[carueda]

Yeah, scoping, recursion (eg., a Tree struct definition), and similar aspects arise from just the ability to name and refer to such definitions. So, yes (and at the moment mainly due to time constraints), this will have to start with some initial simple/experimental implementation.

[carueda]

0.9.94 just released, please give it a try when you get a chance. Thanks!

[kirkbro]

Sorry, haven't had a chance to try this out before now.

There seems to be an issue related to naming.
Here is a minimal example:

This works

#@define
Struct = {
  a = int
}

exampleD {
  test = Struct
}

public static class ExampleD {
  public final Struct test;
  ...
}

This does not work

#@define
Struct = {
  a = int
}

exampleE {
  test = Struct
}

public static class ExampleE {
  public final java.lang.String test;
  ...
}

All I'm running with is --dd, --java, and --spec.

Similar behavior can be achieved by modifying the shared type name. I can't seem to get get the project to run from source to test, but it feels like it's related to hashing?

[carueda]

Thanks for catching that! I just reproduced it.