I will be talking about a proposal for a generalised FHIR CodeSystem "query" syntax: VCL (ValueSet Compose Language) at FHIR DevDays. I'm pre-emptively creating this thread in order to gather feedback.

Interesting. Dynamically fetching a VS as a response?

@Michael Lawley I think I have a clash. Is it different to the past? Is it 1:1 semantically with ValueSet.compose? Do any extensions have specific syntax?

@Lin Zhang The main use case would be to have implicit ValueSets (URL patterns that can be used by $expand, $validate-code, bindings and anywhere else a ValueSet URL/canonical is used, without having to define and create a ValueSet resource on the server) defined using VCL. There are a bunch of implicit ValueSets already available for SNOMED CT, and a few for other CodeSystems like LOINC, UCUM and RxNorm, but VCL implicits would work for any CodeSystem.

@Grahame Grieve I don't think its 100% 1:1 with compose - I think there are some things that compose can't do. I'm not sure there are any extensions in the implementation at present, although there might (need to) be something to put in a TerminologyCapabilities to indicate that VCL is supported

I was thinking of extensions that might be found in compose. In particular, itemWeight and order are candidates, since they are found in Questionnaire value sets where this might be particularly useful

and also required supplement

I'll leave Michael to comment on his implementation, but from my personal point of view, I think it would be difficult to support weight and order in a way that doesn't make the syntax overly verbose. Required supplement maybe, but I think it may fall into the "if you need these, use a compose" category

Parity with compose was not a design goal. As Jim said, there are also things that go beyond ValueSet.compose, specifically property navigation, which otherwise would require a code system to have defined “reverse” properties (eg ‘contains’ AND ‘contained-by’). This is particularly useful for medication CodeSystems like dm+d and RxNorm.

Regarding supplements, you can Redfern to them in the $expand / $validate-code call, but if you want to specify them for a binding, then perhaps supplements can also support the implicit valueset pattern? That wouldn’t cover multiple supplements at once, but that would be a rare case.

Parity with compose was not a design goal I will be very much opposed to anything in VCL that can't be expressed in a valueset compose

perhaps supplements can also support the implicit valueset pattern how would that work?

My proposed implicit syntax is: [CS-URI]?vs=vcl/[exp] which would then give this: http://loinc.org?vs=vcl/SCALE_TYP=Doc So, for supplements, you'd use the supplement URI as the base, rather than the code system URI

I'm not understanding how that creates an implicit supplement

I would instead characterise it as defining an implicit ValueSet for the CodeSystem that the supplement is for, but also with the supplement extension so that the supplement is in-scope for evaluation of $expand/$validate-code

really? I'm not understanding that. Supplements won't be defined implicitly, so the implicit URL space won't be able to define the supplement

I'm not understanding also. This is not defining a Supplement, it is defining a ValueSet that references a Supplement.

how does it reference the supplement then?

So, the implicit valueset URI http://hl7.org/fhir/CodeSystem/example-supplement?vs=vcl/legacy=true would define an implicit ValueSet on the supplemented CodeSystem http://hl7.org/fhir/CodeSystem/example with a compose.include.filter of legacy = true and a http://hl7.org/fhir/StructureDefinition/valueset-supplement extension that references the supplement http://hl7.org/fhir/CodeSystem/example-supplement { "resourceType" : "ValueSet" , "extension" : [{ "url" : "http://hl7.org/fhir/StructureDefinition/valueset-supplement" , "valueCanonical" : "http://hl7.org/fhir/CodeSystem/example-supplement" }], "compose" : { "include" : [{ "system" : "http://hl7.org/fhir/CodeSystem/example" , "filter" : [{ "property" : "legacy" , "op" : "=" , "value" : "true" }] }] } }

Before we get too complicated help me understand something. Would this query syntax be essentially identifying the syntax with a mime-type and then the expression is sent as a parameter just like we could do with a valueSet.expression? This syntax can be published and adopted by anyone and then would not have to align with what can be in a compose. Is that the point? Otherwise what we send as an "Operation-based" expansion - PLEASE do not call such a thing an implicit value set - should be something that can be in a compose.

What is valueSet.expression?

it's an extension that @Robert McClure likes very much

the implicit valueset URI http://hl7.org/fhir/CodeSystem/example-supplement?vs=vcl/legacy=true would define an implicit ValueSet on the supplemented CodeSystem http://hl7.org/fhir/CodeSystem/example with a compose.include.filter of legacy = true and a http://hl7.org/fhir/StructureDefinition/valueset-supplement extension that references the supplement http://hl7.org/fhir/CodeSystem/example-supplement how did you derive that the code system is http://hl7.org/fhir/CodeSystem/example ?

Presumably via CodeSystem.supplements?

I don't like that as an implicit value set - it's structurally different than all the others because the others, it's a straight transform without any look ups

but how does it compare to the rest of the VCL... I'll have to see

VCL, syntax wise, is pretty much a cut down version of ECL, and then augmented to handle the filter.op operators. In terms of use in ValueSet.compose, I am suggesting a filter vcl that's defined for all CodeSystems, and takes the expression as its value.

I've posted the draft ANTLR grammar to the FHIR ticket https://jira.hl7.org/browse/FHIR-36651?focusedId=218959#comment-218959

In terms of use in ValueSet.compose, I am suggesting a filter vcl that's defined for all CodeSystems, and takes the expression as its value. So there's two ways of doing the identical thing? that doesn't make sense at all. ECL is a horrible syntax, and what we should have is iso-morphic to what we already have in compose

in other words, whatever VCL is, it should have a defined transform to a proper valueset.compose

Grahame Grieve said : ECL is a horrible syntax I think this is pretty subjective. I certainly don't agree

Also pretty orthogonal to whether the semantics should be isomorphic to compose

I agree that it's partly orthogonal. But on objective grounds, you would not get to ECL as a design based on DSL first principles unless your thinking started with the existing SCT expression syntax

And yet the comment on the ticket explicitly says: 1) define a URL-friendly syntax for ValueSet.compose (1a) use a subset/variant of ECL since that’s a “known” syntax

sigh. I wrote that since that's what I agreed, but I didn't like it then, and still don't like it. But we also agreed: define a URL-friendly syntax for ValueSet.compose Defining something that goes into a particular filter is not a URL-friendly syntax for ValueSet.compose.

The proposal also includes embedding the expressions in ValueSet URLs for use in expand, validate-code, bindings and other places where ValueSet URLs/canonicals are used

yes, that's in line with what we talked about

I guess its not necessary to support a VCL filter in expressions as well, although it seems a bit arbitrary to say you can use it one place (URLs) but not in others (inside a compose where you might want to flesh out metadata, attach extensions, etc)

Whether its capable of more expressive things than existing compose filters (e.g. navigating through chains of code-typed properties) does affect that, obviously

The proposal is vastly simpler than ECL and has a straightforward transformation to ValueSet.compose except for a couple of things: 1. You might need multiple ValueSets due to nesting, 2. Navigation - not supported in compose, but needed for the ticket, 3. Term filtering - same (but also not yet in grammar above) Variations from ECL also clean up a bunch of URI unfriendly issues. Finally, having readable attribute/property names makes a huge difference to readability over snomed

what's with navigation? Can you explain that?

Something like give me all the ingredients of these drugs.

or "the body sites of all lower limb fractures"

... I'm feeling as though this is not a complete explanation

Here's an example with RxNorm - what are the dose forms of products that contain acetaminophen (161): (consists_of.has_ingredient = 161).has_dose_form

You can try it here: https://r4.ontoserver.csiro.au/fhir/ValueSet/$expand?url=http://www.nlm.nih.gov/research/umls/rxnorm?vs=vcl/(consists_of.has_ingredient = 161).has_dose_form

I'm not sure what has_ingredient and has_dose_form are... booleans?

They are properties

yes but what type?

code (or coding)

so you could do that now with a nested value set, right?

I don't believe so, no

It's establishing relationships through properties. That's not the same as a hierarchical value set. If you want to select condition, you don't want body sites. But you might want to know what the body site possibilities are for a selected condition.

has_ingredient = 161 is just a standard VS filter but you can't then do consists_of in http://my.valueset defined as above since in takes a list of codes, not a ValueSet URI (except for SNOMED)

The .has_dose_form part you can't do at all

oh. I thought we allowed in {valueset}.

then we could do it. and we should support that, definitely. Directly, not just via vcl

Grahame Grieve said : then we could do it. and we should support that, definitely. Directly, not just via vcl Do what? Path expressions as filter properties, or valueset URL as in values?

I like both :)

But the latter is the more moderate change for implementers

Looking at these medication-related use cases, it seems that the overarching problem is that an information model has been rendered into a terminology with less than optimal results. Querying drug properties becomes somewhat easier, and more intuitive, in a FHIR Medication Resource - for example, there are common names for elements for which each Code System has a different property name.

Is https://jira.hl7.org/browse/FHIR-36651?focusedId=218959#comment-218959 the latest ANTLR grammar? A couple of notes: I think some definitions are missing (e.g., NOT_IN , DESCENDANT_OR_SELF_OF ) -- or something is missing in my understanding of ANTLR I think some definitions are dangling/unused (e.g., value ) How do you deal with concept ids that have . in them like ICD10 (eg., G44.89 ), without mistaking these for property navigations? Is there some way to quote or escape concept ids? When propPath is present, it always seems to end in a dot? This confuses me, because an expression like *: a.b = 'c' doesn't doesn't have a . after b FHIR tends to use , to mean "or", e.g., for search query params. It's a bit confusing to use , for "and" in VCL The use of : as a separator between conceptExpr and constraint (e.g., *: a='foo' ) and also between modifiers and equal sign (e.g. a.b:exists=true ) seems confusing to me -- doesn't it create an ambiguity if a property called exists is defined on the code system? Is it not possible to embed an expression in a constraint, like *: dose_form:vcl=(sty='SBD') ? This seems potentially important for concept-to-concept relationships. I think this is a built-in capability in a class-based system like ECL-over-SNOMED. Are conjunctions allowed in a concept expression, and are disjunctions allowed in constraints, enabling (<<'pill'),(<<'generic'): color='blue';tint='blue' (for generic pills that are colored or tinted blue)?

Yes, it's incomplete - the various lexical rules were omitted for brevity, and would address your first few points (quoting etc). I'll have to do a deep dive to check off everything here. "," for AND is the convention in ECL and I see that creates a tension wrt FHIR's SEARCH syntax. I think these sorts of issues will need to be shaken out Yes, the ':exists' could be an issue if you had a property called 'exists'. Technically ':exists' would always be the modifier, and ': exists' (with a space after the ':') would allow you to refer to the property.

Thanks! ': exists' (with a space after the ':') Per the goals you outlined in your DevDays talk, I was hoping the parser would not be sensitive to whitespace (i.e., that : between constraints wouldn't need a subsequent space)... perhaps could avoid this by not reusing colon for both of these cases.

I'll try to write out a peg grammar that I can compare.

PEG --> expression tree (paste into https://peggyjs.org/online.html ) Peggy Grammar {{ const __STAR = { "__ALL" : true }; }} expr = base : simple_expr _ clauses : ( @ sep _ @ simple_expr ) * { return clauses . reduce (( acc , [ sep , expr ]) => { if ( sep === "," ) return { type : "ExpressionConjunction" , and : [ acc , expr ]} if ( sep === ";" ) return { type : "ExpressionDisjunction" , or : [ acc , expr ]} }, base ) } simple_expr = v : ( t : constraints { return [ null , t ]} / c : concept_expr _ ":" _ t : constraints { return [ c , t ]} / c : concept_expr { return [ c , []]} ) { return { "type" : "SimpleExpr" , concept : v [ 0 ] ?? { "type" : "Concept" , root : __STAR }, "constraints" : v [ 1 ]}} concept_expr = m : relationship_modifier ? _ e : ( "*" { return __STAR } / identifier / "(" _ @ expr _ ")" ) s : property_path_suffix { return { "type" : "Concept" , root : e , hierarchy : m , suffixPath : s } } constraints = head : constraint _ suffix : ( @ sep _ @ constraint ) * { return suffix . reduce (( acc , [ sep , expr ]) => { if ( sep === "," ) return { type : "ConstraintConjunction" , and : [ acc , expr ]} if ( sep === ";" ) return { type : "ConstraintDisjunction" , or : [ acc , expr ]} }, head ) } constraint = p : property_path _ m : ( constraint_modifier ) ? _ "=" _ v : value { return { onPath : p , modifier : m , value : v } } relationship_modifier = "<<" / "<" / ">" / ">>" sep = "," / ";" value = value_constant / expr string = "'" _ [ ^ ']* _ "' " digits = [0-9]+ number = n:$(negative:" - "? " + "? digits (" . " digits)?) {return parseFloat(n)} boolean = " true " / " false " value_constant = string / number / boolean property_path = head:identifier _ tail:(" . " _ @identifier)* {return [head].concat(tail)} property_path_suffix = (" . " _ @identifier)* identifier = ($[a-zA-Z0-9_-]+ / " '" _ @$[^' ] + _ "'" ) constraint_modifier = "!regex" / "!memberof" _ "whitespace" = [ \ t \ n \ r ] * Changes along the way in making this work: Added optional quoting for identifiers Replaced ":" in the modifiers to avoid this ambiguity Added support for ands/ors at the expression level as well as the constraint level

Is support still online for your server @Michael Lawley ? I tried the rxnorm link above but this failed. I tried to make one of my own like https://r4.ontoserver.csiro.au/fhir/ValueSet/$expand?url=http://snomed.info/sct?vs=vcl/* but this failed too

Ah, sorry, that was an experimental branch we had deployed. I'll have to do some work to get the merge working

No worries, I was just curious about how you're handling the parsing. When you reach the comma in an expression like: a=1,b=2:c=3 ... you have to be ready to begin parsing a new constraint within the current expression, or alternatively a whole new simple expression. I'm not sure how to choose correctly without lookahead.

This is the latest lexer file: lexer grammar VCLLexer ; // URI is expected to be %-encoded and will always be %-decoded before interpretation URI : LT [ a-zA-Z ] + COLON [ a-zA-Z0-9?=:;&_%+-.@#$^!(){}/ ] + GT ; EQ : '=' ; OPEN_ROUND : '(' ; CLOSE_ROUND : ')' ; COLON : ':' ; COMMA : ',' ; SEMI : ';' ; MINUS : [ mM ][ iI ][ nN ][ uU ][ sS ]; REGEX : ':' [ rR ][ eE ][ gG ][ eE ][ xX ] ; EXISTS : ':' [ eE ][ xX ][ iI ][ sS ][ tT ][ sS ] ; IS_A : ':' [ iI ][ sS ] '-' [ aA ] ; IN : ':' [ iI ][ nN ] ; NOT_IN : ':' [ nN ][ oO ][ tT ] '-' [ iI ][ nN ] ; TRUE : [ tT ][ rR ][ uU ][ eE ] ; FALSE : [ fF ][ aA ][ lL ][ sS ][ eE ] ; DESCENDANT_OR_SELF_OF : '<<' ; ANCESTOR_OR_SELF_OF : '>>' ; STAR : '*' ; STRING_VALUE : '"' (~ [ "\\] | '\\' ["\\ ] )* '"' ; CARET : '^' ; LTE : '<=' ; GTE : '>=' ; LT : '<' ; GT : '>' ; NUMBER : [ -+ ] ?( [ 0-9 ] * '.' )? [ 0-9 ][ 0-9 ] * ; NAME : [ a-zA-Z ][ -_a-zA-Z0-9 ] * ; DOT : '.' ; fragment ID : [ -_a-zA-Z0-9 ] + ; // FIXME support UTF8 chars too fragment ESC : '\\\'' | '\\\\' ; CODE : ID | '\'' ID (( ESC | [ . ] )+ ID )* '\'' ; WS : [ \t\r\n ] + -> skip ; // skip spaces, tabs, newlines and the latest grammar file parser grammar VCLGrammar ; options { tokenVocab = VCLLexer ; } vQuery : expr EOF ; expr : disjunctionExpr | exclusionExpr | simpleExpr ; disjunctionExpr : simpleExpr ( or simpleExpr )+ ; exclusionExpr : simpleExpr minus simpleExpr ; simpleExpr : ( STAR | conceptExpr ) | ( STAR | conceptExpr ) COLON constraint | constraint ; propNav : ( dot name )+ ; propPath : ( name dot )+ ; constraint : valueConstraint ( and valueConstraint )* ; valueConstraint : propPath ? ( simpleAttr | existsAttr | regexAttr | inAttr ) ; simpleAttr : name EQ value ; existsAttr : name EXISTS EQ ( TRUE | FALSE ) ; regexAttr : name REGEX EQ STRING_VALUE ; inAttr : name ( IN | NOT_IN ) EQ ( codelist | uri ) ; value : bool | code | conceptExpr | number | str ; codelist : OPEN_ROUND code ( COMMA code )* CLOSE_ROUND ; uri : URI ; name : NAME | NUMBER | STAR ; bool : TRUE | FALSE ; code : CODE | NAME | NUMBER ; number : NUMBER ; str : STRING_VALUE ; or : SEMI ; and : COMMA ; minus : MINUS ; dot : DOT ; conceptExpr : ( DESCENDANT_OR_SELF_OF | LT | GT | ANCESTOR_OR_SELF_OF )? ( code | OPEN_ROUND expr CLOSE_ROUND ) propNav ? ;

However, this doesn't look legal to me: a=1,b=2:c=3 There's no or or minus so it must be a simpleExpr , and must be a constraint since "a=1,b=2" is not a valid conceptExpr (anything other than a simple code needs to be quoted). As a constraint, it would need to be the valueConstraints: a=1 and b=2:c=3 , but "b=2" is not a valid code so "b=2:c=3" will fail to match the second alternative for simpleExpr , which is the only place a COLON can occur. At least, that's my human analysis - http://lab.antlr.org shows this: image.png

Thanks! It strikes me as slightly funny not to accept that, since it seems unambiguous. In my parser I'm exploring adding an optional ": _ constraints" at the end of expr.

What do they correspond to in a ValueSet compose? I'm lost as to what you're trying to express with the above example too?

It's equivalent to (a=1,b=2):c=3 or a=1,b=2,c=3 -- it's an AND filter on values of 3 properties, across all concepts

@Michael Lawley it would be good to bring VCL forward, but I can't support it unless it's got the same functionality as a value set compose. (though it may do that through extensions)

Why do you say that? Do you mean it needs to be identical functionality? Or a strict subset?

How do you want to address path navigation in ValueSet.compose?

Grahame Grieve said : Michael Lawley it would be good to bring VCL forward, but I can't support it unless it's got the same functionality as a value set compose. (though it may do that through extensions) One strategy for that is to introduce "vcl" as a well-known filter applicable for any code system.

But the other requires two additions: 1. Implicit reverse filter for every property, and 2. Fixing the "in" operator

I don't see value to making VCL able to do everything that ValueSet.compose can -- it will make the syntax horribly complicated. My guide has been simple things should be simple, and complex things should warrant an explicit ValueSet. But as the proposal stands now, ignoring the option of a "vcl" filter, it is a superset of ValueSet.compose.

I think a filter is a great idea; I'd really like this functionality for all kinds of ValueSets but I don't see the point of introducing new elements to shoehorn it in.

but then it would overlap with other functionality in ValueSet.compose and that would be problematic

and I agree that VCL doesn't need to do everything that ValueSet.compose does, but what would it not?

Regarding the "vcl" filter -- this would be analogous to SNOMED's "constraint" filter for ECL. ( @Josh Mandel , not a new element, just a well-known name that can be used for ValueSet.compose.include.filter.property .) And in terms of overlapping other functionality, a bunch (all?) of the new filter operators overlap with previously existing was to express the same thing. Things VCL would not do: multi-codesystem ValueSets; you don't want a syntax to have to cope with including CodeSystem URIs multi-version ValueSets; again, don't want to have to mess around with tracking versions/version scopes inside a VCL expression

how do you choose system right now in VCL? I don't remember off the top of my head?

The implicit ValueSet URI starts with the CodeSystem URI

And separately: for a VCL filter that's used within an explicit ValueSet, the ValueSet.compose.include.filter has a sibling ValueSet.compose.include.system that can specify the system alongside. e.g., { "system" : "http://www.nlm.nih.gov/research/umls/rxnorm" , "filter" : { "property" : "vcl" , "op" : "=" , "value" : "<<scdc:has_ingredient='loratadine'" } }

I've made some progress on an implementation in C#, backed by sqlite. Still finicky about whitespace, but I'm just getting a feel for what indexes make this work well. Demo at https://termsql.fly.dev/ValueSet/$vcl takes two URL params ( system and query -- I'm not proposing this as a good definition, it's just the current state of my demo). The demo server spins down with disuse, so might take a few sec to answer your first query. Should be very fast after that. It can answer questions like What RxNorms are branded drugs with "chewable" in the name and contain an active ingredient of cetirizine ? $ curl -G 'https://termsql.fly.dev/ValueSet/$vcl' -d "system=http://www.nlm.nih.gov/research/umls/rxnorm&query=*:tty=SBD,{{term='chewable'}},consists_of.has_ingredient=20610" { "system" : "http://www.nlm.nih.gov/research/umls/rxnorm" , "code" : "1020022" , "display" : "ZyrTEC 10 MG Chewable Tablet" } { "system" : "http://www.nlm.nih.gov/research/umls/rxnorm" , "code" : "1086794" , "display" : "Wal-Zyr 10 MG Chewable Tablet" } What LOINCs use the ultrasound method with a cardiology class and include the word "wall"? $ curl -G 'https://termsql.fly.dev/ValueSet/$vcl' -d "system=http://loinc.org&query=*:METHOD_TYP='LP6572-4',CLASS='CARD.US',{{term='wall'}}" { "system" : "http://loinc.org" , "code" : "18052-1" , "display" : "Right ventricular anterior wall Fractional thickening freewall by US" } { "system" : "http://loinc.org" , "code" : "18116-4" , "display" : "Left ventricular Segmental wall appearance by US" } { "system" : "http://loinc.org" , "code" : "18153-7" , "display" : "Right ventricular anterior wall Thickness during diastole by US" } { "system" : "http://loinc.org" , "code" : "18157-8" , "display" : "Right ventricular anterior wall Thickness during systole by US" } { "system" : "http://loinc.org" , "code" : "18178-4" , "display" : "Left ventricular Wall motion index by US" } { "system" : "http://loinc.org" , "code" : "18179-2" , "display" : "Left ventricular Wall segment [Identifier] by US" } { "system" : "http://loinc.org" , "code" : "18837-5" , "display" : "Left ventricle Segmental wall appearance by US Narrative" } { "system" : "http://loinc.org" , "code" : "18840-9" , "display" : "Left ventricular Wall motion index by US Narrative" } { "system" : "http://loinc.org" , "code" : "59097-6" , "display" : "Left ventricular Meridional wall stress [Pressure] by US" } Source at https://github.com/jmandel/TermSqlite

In the process I made some tweaks to VCL: Changed : in modifiers to ! to avoid the parsing ambiguity we talked through here Added support for {{term='something'}} constraints, similar to ECL's I think there's still some missing functionality in the language... Way to group constraints to say things like "and and b... or c" *:(a,b);c Ways to use expressions at the end of a property chain, like "find me branded drugs that have an ingredient with a name that includes "loratadine" -- like *: a.b = ({{term='something'}}) . This exists in ECL but not in the VCL proposal I think?

Changed : in modifiers to ! to avoid the parsing ambiguity we talked through here I'd been contemplating removing the standalone ':' from the syntax because it's really just a special-case of ',' (AND) Way to group constraints to say things like "and and b... or c" *:(a,b);c brackets are the way to group things, and are actually required if you have a mixture of AND and OR (ie there are no precedence rules that would allow something like "a,b;c,d"

I agree with your comment about special case of "AND" . I think eliminating this special case is a good simplification, despite the divergence from ecl. On round brackets, this makes sense but I didn't see it in your constraint grammar above. In other words I saw that expressions could be grouped but I'm not seeing constraints could be grouped

Hmm, maybe brackets didn't make it across when I was going for minimalism. I'm planning to do a refresh in the next couple of weeks to take all the feedback into account.

Removing the ':' would allow the modifiers to remain with a ':' prefix and thus retain similarity to _search syntax conventions which I think is desirable

Awesome. And it might be that grouping expressions is all you need (like, we can get rid of the and/or on constraints altogether, If you can parse a=1,b=2 as two single-constraint expressions AND'd together).

Re {{term='something'}} this would take us outside the ValueSet.compose scope

I mean... being able to query for terms by content, language, and use feels like a core use case to me. Seeing the patterns that people were developing to work around this limitation is what drove me to propose that jira issue in the first place.

We could always come up with some way to reduce this to "Filter-Based ValueSet queries" If that's important. But the filters start becoming compound things like "lang_en_and_use_friendly"

There's a real tension; labels/designations are not really a reliable way to identify concepts for a ValueSet, and this is one reason why this was not a feature of ECL for a very long time.

It's like using regex on the code -- works sometimes, for some code systems, but it's not necessarily the right approach

I think there are additional use cases for vcl though, including just an efficient way to do a search... And for better or worse we do see actual value sets that use this kind of logic

I know @Grahame Grieve wants a clear mapping into ValueSet.compose, But Grahame, don't you also write custom implementations for specific CodeSystems to actually make ValueSet.compose work, implementing the specific filters and properties that been defined in narrative documentation somewhere? (I'm coming at this from the other direction. I don't want to write custom code on the margin, but I'm willing to contemplate a bigger surface area for query functionality.)

Indeed. $expand?filter= gets you some way there, but we would need to extend the scope of ValueSet.compose

But Grahame, don't you also write custom implementations for specific CodeSystems to actually make ValueSet.compose work, implementing the specific filters and properties that been defined in narrative documentation somewhere? All servers must, but this is a feature of code systems, not value sets.

The other challenge here is what is the actual meaning of 'term=XYZ' - do we follow ECL's lead, or do we do something else

I agree there is a feature of code systems, but it is a feature that you implement in order to support value set expansion. I don't agree that all servers must, or at least I believe that I can write a generic server as long as I pre-populate the right side of properties in my code system resources. I mean, I've done enough work to convince myself that this feels right over loinc, rxorm, and snomed, but I could be off base. (I've also played with embedding a grammar or wasm module in a CodeSystem extension to validate codes from systems that use a compositional grammar like bcp47)

Michael Lawley said : The other challenge here is what is the actual meaning of 'term=XYZ' - do we follow ECL's lead, or do we do something else Key is being able to apply co-constraints on language (specific tag), use (specific coding), and the designation itself (regex, contains with wildcard, and maybe the order independent word prefix list that snomed defines)

I don't agree that all servers must, or at least I believe that I can write a generic server as long as I pre-populate the right side of properties in my code system resources if you only want to deal with CodeSystem sourced properties, maybe you don't. But if you want a production ready tx server, you'll run into the curly code systems for which you have to write code

Curly?

the ones with grammar or where people have defined very derived properties for filtering

eg RxNORM's specs that refer to queries against underlying tables, but also SNOMED's ECL Also I get the sense that there's gaps around things like the RxNorm-adjactent things like RxClass

Michael Lawley said : There's a real tension; labels/designations are not really a reliable way to identify concepts for a ValueSet, and this is one reason why this was not a feature of ECL for a very long time. @Josh Mandel Use of designations as an aspect of value set definitions is definitely IMO base functionality for FHIR compose. I'll say, given my armchair position it is not clear to me why that seems overly difficult or code system-specific. I'd really like to see this work demo'd someplace. Any plans? Of note, I'll NOT be at the May WGM (I'll be in Portugal...)

eg RxNORM's specs that refer to queries against underlying tables, Yes, but I think this is just a kind of shorthand; the actual functionality can be described in terms of straightforward property filters, no? also SNOMED's ECL Sure, that's fair -- to the extent that CodeSystems describe bespoke DSLs,that's a special case, but Having something like VCL standardized in FHIR will reduce the need for system-specific DSLs, at least in theory I'm not sure ECL support in SNOMED is that widespread -- like, it's awesome that Ontoserver support this, but am I right in thinking that tx.fhir.org hasn't implemented support (yet) for ECL? And we're still getting by? the ones with grammar I still think much of this can be described with a combination of: validate function or grammar (can be embedded in the CodeSystem, e.g. as webassembly to validate a string. This is "writing code", sure -- but it can be safe, reusable code that's morally shipped as part of the CodeSystem definition and doesn't have to be rewritten by every server canonicalize function (ditto)

I'd really like to see this work demo'd someplace. Any plans? Of note, I'll NOT be at the May WGM (I'll be in Portugal...) Sure -- I'd be happy to talk through what I've done so far. We could set up an hour to record and share a demo / conversation over zoom if you'd like @Robert McClure .

Yes, but I think this is just a kind of shorthand; the actual functionality can be described in terms of straightforward property filters, no? For RxNorm I do think some searches will require underlying table joins. For example finding all the SCDs for an IN. BUT you can use NLM API calls do do that for you. Any chance we can integrate authoritative API calls into this syntax so we do not have to require every server to re-do the work? Sure -- I'd be happy to talk through what I've done so far. We could set up an hour to record and share a demo / conversation over zoom if you'd like Yes. @Michael Lawley too? If so, late in the US day but early in the next AU day?

I'm not sure ECL support in SNOMED is that widespread -- like, it's awesome that Ontoserver support this, but am I right in thinking that tx.fhir.org hasn't implemented support (yet) for ECL? And we're still getting by? We're not really getting by. The single biggest reason people want additional terminology server support is ECL support. validate function or grammar (can be embedded in the CodeSystem, e.g. as webassembly to validate a string. This is "writing code", sure -- but it can be safe, reusable code really? I think that's extremely inappropriate from my pov. I would never let arbitrary code I didn't review run in the context of tx.fhir.org

Hmm, a couple of things to consider here: 1) with reproducible, signed build pipelines you can review the code if you want to, but still use the binary artifact, 2) webassembly is designed to run in a sandbox where it sees nothing but the data you pass in, and can't interact with the rest of your application, and can't consume more resources than you allocate. I'm not saying you don't review and use an allow-list. I'm just saying everyone shouldn't need to reinvent the wheel.

OK, I've updated my implementation to remove the *: c(and all other) concept specifier prefixes from the SimpleExpression grammar. This cleans up the syntax (and my code) nicely I think, though it leaves a bit more work for the query optimizer in common cases :-) Examples SNOMED CT : descendant of Surgical procedure on thorax and ancestor of Open tricuspid valvotomy <385487005,>174969001 RxNorm: Brand name drug with cetirizine as an ingredient consists_of.has_ingredient=20610,tty=SBD Loinc: All "Left pupil diameter" and all of its explicit ancestors >>'8640-5'

I'll restate my prior question, is there a way to include within VCL a way to include API calls to known services, like the RxNorm API? Or eventually a LOINC API? This way, every TS does not have to decide how to represent the code system, keep it up to date, etc.

I don't think that is a concern of VCL the expression language. Of course that is an important implementation concern. (I will reiterate that it would be good for us to spend some time in live discussion to develop/confirm a shared understanding of this domain.)

Sounds like a perfect May WGM session. Problem is I will not be there and I suspect neither will be @Michael Lawley . DevDays is a long time from now - and I'm not planning on attending. So virtual may be best. Announce on zulip to get other TS devs and sponsor via TI? @Carol Macumber @Rob Hausam @Reuben Daniels @Peter Jordan

OK, I fixed the performance issue I introduced in my demo when I simplified the VCL syntax. Queries like the ones above are now fast, as well as: <(>NDC=11523000708),(tty=SCD;tty=SBD) Roughly: "Given an NDC, what other strengths/forms are available for the same drug?"

The scope of VCL is ValueSet.compose. IMO calling arbitrary APIs from VCL is out of scope. Such a capability would likely need to contend with arbitrary JSON / XML / other result formats, authentication, API and endpoint stability & scalability, potential for exploiting terminology servers to DoS systems, etc

Added support for subqueries in the "=" position of a constraint, like part_of=(tty=MIN,has_part=28889) Ingredients that are part of combined drugs with loratadine Note, maybe the part_of=() should be part_of:in=() , or somesuch? I could be convinced about a better syntax! This is closely related to reverse property paths which I know you've got a proposal for @Michael Lawley . I'm planning to add that too, though I'd rather use a syntax like .^ to represent a reverse navigation, instead of prefixing properties with "from." (which can create an interpretation ambiguity).

For reverse navigation I'm not too fussed about the actual implicit filter name, just that it maps to something that makes sense in VCL and is unlikely to clash with an actual filter/property name.

Wrt part_of=() vs part_of:in=() , I favour the former since it's simpler and totally unambiguous even though the latter would be a "stricter" mapping of ValueSet.compose.include.filter

The custom validation code - that seems like a sensible place for a fhirpath style expression for validation? Such as this one for BCP47 grammar checking? https://www.rfc-editor.org/rfc/rfc4647.html#section-2.2 matches('^([a-zA-Z]{1,8}|\\*)(-([a-zA-Z0-9]{1,8}|\\*))*$') :test_tube: Test with FHIRPath-Lab

The functionality Grahame has in mind is more involved -- It requires being able to assign a whole set of different properties from the parse, describe a canonical parse, etc, evaluate logical checks, etc.

Michael Lawley said : The scope of VCL is ValueSet.compose. IMO calling arbitrary APIs from VCL is out of scope. Such a capability would likely need to contend with arbitrary JSON / XML / other result formats, authentication, API and endpoint stability & scalability, potential for exploiting terminology servers to DoS systems, etc I understand this introduces problematic risks. But use of an API to get members of an expansion is most definitely in the compose scope. But ok - I get that TS cannot manage risk.

Added support for suffix path navigation, including forward ( . ) and reverse ( .^ ) paths: (247867).consists_of.has_ingredient All ingredients in aspirin 325 MG / caffeine 4 MG Oral Tablet ( SCD 247867 ) (247867).consists_of.^ingredient_of Equivalent (using a reverse property path to navigate from SCDC to IN ) (tty=SCD),consists_of.has_ingredient=1886 All generic drugs with caffeine ( IN 1886 ) as an ingredient (tty=SCD),consists_of.^ingredient_of=1886 Equivalent

@Josh Mandel do you have an updated Peggy grammar for what you have implemented?

I do not, but my current parser in sprache is at https://github.com/jmandel/TermSqlite/blob/83ea445f178bfd986cd5c3096f6c9069421238f6/src/Vcl.cs#L291

Thanks - am working through refinements of the original proposed grammar. Am trying to reverse engineer things to understand what you've been exploring eg Constraint: https://github.com/jmandel/TermSqlite/blob/83ea445f178bfd986cd5c3096f6c9069421238f6/src/Vcl.cs#L360

Thanks! I would love to figure out how to take this forward.

From my end, it is mostly a lack of time / priority -- we don't have anyone banging on our door demanding this :)

Our public sandbox: https://r4.ontoserver.csiro.au/fhir now supports my current draft of VCL. This includes the syntax simplification removing the ':' stuff, and adding in nested exepression / sub-expression support.

The properties are typed and case sensitive, so this: (tty=SCD),consists_of.has_ingredient=1886 needs to be: (TTY="SCD"),consists_of.has_ingredient=1886 because TTY is declared as a string not a code.

Also, I don't understand the motivation for needing reverse property syntax: consists_of.^ingredient_of=1886 Can be expressed as: consists_of = 1886.ingredient_of

It can be harder to see the value of reverse properties in RxNorm because they have defined explicit forward and reverse versions of everything. But generally: if you need to traverse more than one property, and you only have unidirectional properties... Then the ability to reverse a property lets you create a straight chain, which you couldn't do otherwise. Reverse syntax is part of every graph reversal language I knowreverse syntax is part of every graph traversal language I know, including ECL.

(But I'm not telling you anything you don't know, so I'm probably missing something about your point.)

I think what's going in is a little subtle. My point was that extra syntax is not needed. If you only have the has_ingredient property, then you express the query as: consists_of.has_ingredient=1886 If you only have ingredient_of , then you can express the same thing as: consists_of = 1886.ingredient_of Similarly, if you only have ingredient_of , then: 247867.consists_of.has_ingredient All ingredients in aspirin 325 MG / caffeine 4 MG Oral Tablet (SCD 247867) Can be written as: ingredient_of = 247867.consists_of

Looking at ECL, the "dot" syntax is a shorthand for "=R", which is the reverse navigation (of =), but far far more readable :) Hence the ".^" syntax is the reverse of "=R", which is just "="

Sure, this might just be about readability. But if you have alternating property directions that forces you to keep nesting expression if you want to express it without the concept of reversing.

Perhaps it's just a question of " what's intuitive for me isn't what's intuitive for you". Are you saying that you find the Idea of explicit reverse navigation unintuitive, or simply that we could get by without it?

Perhaps, yes. I't s not so much unintuitive, but that I haven't felt the need for reverse navigation (probably because my thought processes here are grounded in ECL). Then that triggers my keep-it-minimal aesthetic. While this started as a variant on ECL, dropping the ":" from the syntax really changes the feel of it for me, which also helps with considering new things like this which may be more familiar/comfortable to people coming from other paradigms. So, I'm not against it, I just want to ensure that I understand the motivation behind it, and that it has a reasonable mapping back into ValueSet.compose

OK. From this discussion, I see now that it is neutral with respect to expressive power (thanks!), so I don't know that it has any implications for ValueSet.compose. (So my case for inclusion is: I find it more straightforward to read/weite a series of navigation steps, rather than nested expressions.)

Yes, I think that's a strong argument for inclusion.

Hello, I just watched an @Michael Lawley 's presentation "VCL, the ValueSet Compose Language, Brings CodeSystems to Life" from DevDays 2024 -really interesting video! I have a few follow-up questions about VCL implementation in Ontoserver: Is VCL fully implemented in Ontoserver? Can we use VCL with all CodeSystems in Ontoserver? Thank's

Michael and I grabbed some time while we're together at a connectathon in Philippines to discuss the VCL proposal. my contention is that VCL should be a syntactic variant of ValueSet compose. With that in mind, @Michael Lawley is making some minor adjustments to the VCL syntax to enable VCL to do what compose does, and I have proposed three tasks to improve the ValueSet.compose definition: FHIR-51117 , FHIR-51118 , FHIR-51119

note that those tasks stand on their own without VCL. They should be actioned (or not) for their own reasons

...minor adjustments to the VCL syntax... for some definition of "minor" :-)

Hi @Michael Lawley , In Germany in the MII, we are interested in VCL. We like to use implicit ValueSets in FHIR search and CQL for our cohort queries. Currently we expand the codes at query creation time. The main use will be a simple is-a filter. I tried VCL URL's like http://loinc.org?vs=vcl/SCALE_TYP='LP32888-7' from your slides on https://r4.ontoserver.csiro.au/fhir but it doesn't seam to work. ECL works. Do you have a current implementation that you can activate or that we can active on our Ontoserver? Or do I use the wrong ValueSet URL syntax? I can't find any in https://build.fhir.org/ig/FHIR/ig-guidance/vcl.html and I took [CodeSystem_URI]?vs=vcl/[expression] from your slides.

I believe that the VCL validator at http://fhir.org/vcl is current, and I don't think your syntax is correct

http://fhir.org/VCL?v1=[expression] The expression itself now contains the code system URI

It looks like we're still missing doc on this syntax from the IG page - it was the last bit that we sorted out

contributions welcome

https://github.com/FHIR/ig-guidance/pull/59 @Grahame Grieve I think your resolver is not up to date with this version of the syntax - it seems to still be using the old path-based syntax: https://www.fhir.org/VCL/[expression] Also, the SNOMED examples on that page with versions are wrong; they use datestamps instead of proper SNOMED version URIs with the relevant moduleId

I thought you updated it?

Ah, it seems I missed that bit (I now see there were two places to update). I'll get on it.

Thanks. I have still problem getting this to work. The URL http://fhir.org/VCL?v1=(http://loinc.org)(parent^{LP46821-2,LP259418-4}) from the PR doesn't work with https://r4.ontoserver.csiro.au/fhir . curl -sL "https://r4.ontoserver.csiro.au/fhir/ValueSet/\$expand?url=http%3A%2F%2Ffhir.org%2FVCL%3Fv1%3D(http%3A%2F%2Floinc.org)" | jq -r '.issue[].diagnostics' [ 001b5791-f2bf-4a95-ba00-459782d3c0c2 ] : Could not find value set http://fhir.org/VCL?v1 =( http://loinc.org ) and version null. If this is an implicit value set please make sure the url is correct. Implicit values sets for different code systems are specified in https://www.hl7.org/fhir/terminologies-systems.html.

How is this represented in FHIR VS.compose? consists_of^{has_ingredient^{has_tradename=2201670}} -- it means "codes that consist of ingredients that have a trademark of 2201670" correct?

I put this VCL prototype together, would love any feedback: https://rh-ffq-demo-98d562d0b0db.herokuapp.com/

@Brian Kaney I tried (http://loinc.org)(parent^{LP46821-2,LP259418-4}) and while the parsing and explain looks good, the Valueset translation uses the default system URI instead of LOINC.

Alexander Kiel said : (http://loinc.org)(parent^{LP46821-2,LP259418-4}) Fixed, thanks! I also improved the explainer quite a bit, and there is a link to the source as well now.

This is awesome, thanks Brian!

Alexander Kiel said : Thanks. I have still problem getting this to work. The URL http://fhir.org/VCL?v1=(http://loinc.org)(parent^{LP46821-2,LP259418-4}) from the PR doesn't work with https://r4.ontoserver.csiro.au/fhir . curl -sL "https://r4.ontoserver.csiro.au/fhir/ValueSet/\$expand?url=http%3A%2F%2Ffhir.org%2FVCL%3Fv1%3D(http%3A%2F%2Floinc.org)" | jq -r '.issue[].diagnostics' [ 001b5791-f2bf-4a95-ba00-459782d3c0c2 ] : Could not find value set http://fhir.org/VCL?v1 =( http://loinc.org ) and version null. If this is an implicit value set please make sure the url is correct. Implicit values sets for different code systems are specified in https://www.hl7.org/fhir/terminologies-systems.html. @Michael Lawley Can you please look at this? Thanks.

We don't yet have the VCL support merged in to the release version of Ontoserver. We're looking to get it merged in very soon.

can anyone help me with an example of using VCL to define a valueset for patient administrative gender as "male or female"? I can't figure it out

The VCL expression would be: (http://hl7.org/fhir/administrative-gender)(male;female)

The implicit ValueSet URI would be http://fhir.org/VCL?v1=(http://hl7.org/fhir/administrative-gender)(male;female)

Is there already a Java VCL Parser?

Yep. org.hl7.fhir.r5.terminologies.utilities.VCLParser in the org.hl7.fhir.core:org.hl7.fhir.r5 package

Works for me. Thanks.

Does VCL no longer have support for dotted path traversal? I see examples mention (consists_of.has_ingredient) but it's not clear what this means (this is in a list examples without commentary) or if it's still a valid production. I used to support things like (tty=SCD),consists_of.has_ingredient=1886 in my implementation and possibly this now requires curly braces? (If so, how come?)

Also some comments on grammar inconsistency with examples: https://chatgpt.com/s/t_6984033064e88191966cb3506728fae9

@Michael Lawley

dotted path traversal is still there: filter : ... | (code | codeList | STAR | URI | filterList ) DOT property // operator "of" and maps to a proposed new operator "of". But, I think the equivalent expression is now: (tty=SCD),1886^{consists_of.has_ingredient}

Thanks for the proof-read. I wish I'd had (was proficient with using) AI coding tools when this work was done. The prose is wrong. The grammar is right. Good catch. I think they should be excluded from quoted values so this is a grammar bug. I hate non-ASCII quotes and "smart" copy-paste / editors Yes, this is a limitation we couldn't avoid unless we had required quoting of codes including hyphens. Exclusion was felt to be the rarer use-case and thus bracketing the LHS expression is the appropriate solution if you want to avoid whitespace. This is deliberate. We need to be able to disambiguate {filter} lexically, and this is not possible if we allowed singleton code lists.

filter : ... | (code | codeList | STAR | URI | filterList ) DOT property // operator "of" I was struggling to see how how you could chain dots like a.b.c

Also: why require "concept" to appear in <<123 --- like concept << 123 seems to add noise especially when it's repeated in lots of places within an expression. I think this new too?

Yes, there have been a lot of revisions to the syntax. Basically we started from scratch and built it up from the structure of ValueSet.compose but guided by the syntax choices of previous versions (in turn guided by SNOMED's ECL). So concept << 123 is a triple that directly maps onto ValueSet.compose.include.filter 's property op value structure.

I see. I haven't followed closely (obviously) but I have been having a really hard time wrapping my head around these choices/syntax. The documentation is not helping, so that might be the most important place to focus. It seems like there might not be much interest in "improving" the syntax further though at this point (given that ~improvements are actually trade-offs)? This is outside of the context of an hl7 workgroup project, right?

And re: (tty=SCD),1886^{consists_of.has_ingredient} ... I don't think that's quite right because that (gramatically) treates 1866 as a property rather than a concept. So the following seems to be the new way to express this: (http://www.nlm.nih.gov/research/umls/rxnorm)TTY=SCD,consists_of^{has_ingredient=1886}

I will confess to much preferring the previous syntax. Although it had limitations I think we could have worked around those. Grahame argued IIRC, that the syntax should more closely match the VS compose structure and that would make it easier for people to use.

I have a couple of tweaks I will suggest that respect the "compatible with compose" line.

I have no opinion about the syntax, but I do believe it should be conceptually equivalent to ValueSet. But we did create several tasks to improve ValueSet to match