V2 API extensibility thoughts (The case against metadata)

I think this post is probably going to be a bit more rambling than suitable for a specific bug report,
I’ve been using tectonic again recently with the v2 cli, and the nomencl package.

This requires running an external command makeindex , (documented here)

like makeindex 〈filename〉.nlo -s nomencl.ist -o 〈filename〉.nls

Where nomencl.ist presumably should come from the bundle, or be copied into the src/ dir
Currently I just manually copied the nomencl.ist into src, and only rerun the makeindex manually when necessary.

But it seemed worth thinking about how this running of an external command should integrate with Tectonic.toml, I presume in the spirit of cargo, the thing to do would be implement something like an external makeindexwatch command which is responsible for watching the .nlo file and producing the .nls one.

A bit of a pet-peeve of mine with cargo’s external commands is there is very limited support for external commands to add fields to Cargo.toml, there is the field package.metadata and the workspace.metadata
tables.

Which basically are untyped/ignored by cargo. (Any valid toml is valid in the metadata field).
While it seems to me that the whole thing could only be improved by using some form of Linked data - Wikipedia as the top-level file, and building a schema Schema.org - Wikipedia which only includes the things in the toml which we do know about.

Its worth noting that Techtonic.toml doesn’t currently appear to have a metadata field, this would be the obvious place where I ask for one, however i’m going to argue against the adding of one.
If we consider that we could currently write a tool which takes a Tectonic.toml write a schema for it, and translate the toml contents to some other format (e.g. to JSON-LD would be the obvious example).

Currently we have no problems doing so, e.g. for every valid Tectonic.toml file there is a 1:1 injective mapping to schema, but once we add fields like metadata this is no longer possible, We could schemaize the fields we know about we can only consider metadata to be e.g. an arbitrary blob of JSON.

That is in my opinion until fields like metadata are added there is an automatic infallible upgrade path to a schemaized translation of the toml data. If tectonic automatically converted them, and was built to consume the schemized version, any tools which need metdata would have an alternate path to adding it.

That would be: convert your toml to some less pretty format which contains the schema, build a schema for your metadata, and use a linked data format to combine them.

My question I guess, is if we are interested in experimenting with tools which convert toml to, and teaching tectonic to read a schemaized version of its contents.

Lastly in the meantime while have no such tools, whether we could enact a moratorium on fields like metadata despite the gap this seemingly creates for 3rd party tools, until such a time that it is either clear that a schemaized index won’t happen or work for some unforeseen reason, or we do have them in which case they would be entirely unnecessary and the moratorium becomes permanent.