Questionnaires are excellent tools for data capture. They allow tight control over what data is gathered and ensure information is gathered consistently across multiple users. However, data gathered using different questionnaires - or even different versions of the same questionnaire - is often not comparable. It is also not very searchable or easily integrated with discrete data sources. Because of this, the general recommendation in FHIR is to use questionnaires for raw data capture but then to convert the resulting QuestionnaireResponse instances into other FHIR resources - Observations, MedicationStatements, FamilyMemberHistories, etc. This allows the data gathered to then be easily combined with other data into FHIR documents and messages and exposed over FHIR REST interfaces.

Such conversion can be done with custom code written on a Questionnaire by Questionnaire basis. However, it makes the process much easier if it's possible to write generic software that can convert any arbitrary QuestionnaireResponse into appropriate FHIR resources leveraging metadata embedded in the Questionnaire. This portion of the SDC guide defines mechanisms for doing so.

Note to balloters: The definition based $extract functionality has been refined significantly along with a new mechanism for extraction Template based extraction that uses contained resource(s) as template(s) for extracting data.
Feedback is encouraged for this new functionality.

Caveats, considerations and rules with form data extraction

• Extraction is a step that only makes sense to occur once a QuestionnaireResponse is completed. Prior to that, the information to create valid resource instances may not be available and conversion logic is likely to fail. The types of data produced will vary by Questionnaire.
• Some questionnaires might result in a single resource. Others will produce a Bundle of resources. In some cases, the result might be a transaction intended to create some resources and update others.
• If the Questionnaire is being designed with conversion to a resource in mind, conversion will be a straight-forward process because the questions will align well with how FHIR stores data - and the questions can even be tuned to align with particular profiles around the use of terminology, which elements are mandatory, etc. However, if converting data from questionnaires designed without FHIR in mind, mapping may be more challenging. Required elements may need to be inferred, codes may need to be transformed and other transformations may be necessary to ensure that converted data meets expectations for use, such as aligning with country-specific implementation guides. In some cases, alignment may not be possible.
• Once a QuestionnaireResponse has been converted, it might not be necessary to retain the QuestionnaireResponse any longer, as all data access and subsequent maintenance will occur through the 'traditional' resources. However, in many environments, the QuestionnaireResponse will be retained anyhow to keep a record of the original source-of-truth and for traceability reasons.
• As with population, 'extraction' can happen in one of two ways: The SDC Form Filler can invoke the $extract operation on a SDC Form Manager to generate the resource or Bundle containing the information expressed in the QuestionnaireResponse. Alternatively, the SDC Form Filler can perform the extraction process locally without relying on an external process. Some Form Fillers might rely on the operation approach for certain extraction techniques and perform others internally.
• When resources have been generated from a QuestionnaireResponse, the Provenance instance associated with the creation of the resource instance(s) can and SHOULD include an entity reference of type 'source' that points back to the original QuestionnaireResponse. If Observations are generated, they can also have an explicit derivedFrom link pointing back to the QuestionnaireResponse.
• In theory, it is possible to use Questionnaire as a user-facing interface to allow maintenance of one or more resources. The source data can be used to populate the QuestionnaireResponse and once the response is 'submitted', the data can then be extracted and used to update the existing resources. For this to work, the id of the resource must be retained through the round-trip process to allow for the update. This can be supported by storing the id as a hidden question not shown to the user. This question item would have a type of 'string' and, if using the definition-based extraction approach, a definition that corresponded to the 'id' element of the relevant resource type (e.g. http://hl7.org/fhir/StructureDefinition/AllergyIntolerance#AllergyIntolerance.id). Note that the inclusion of the id as a hidden question is only relevant for the definition-based and StructureMap-based approaches. It is not needed for the Observation-based approach.
• Sometimes the author interested in making a Questionnaire "extractable" does not have authority to make changes to the "official" Questionnaire. In other cases, there might be one official Questionnaire, but a need to create extracted resources that comply with different sets of profiles - and thus a need for different metadata in the Questionnaire to support the extraction process. In this case, rather than basing the extraction on the original Questionnaire, it can be based on a derived Questionnaire - one that has a Questionnaire.derivedFrom relationship to the same canonical URL the QuestionnaireResponse refers to. The derived Questionnaire would contain the same content as the base Questionnaire, but would have additional extensions inserted to support data extraction.
• When capturing quantities, it's common for questionnaires to prompt for the numeric value and to note the 'fixed' unit as part of the question. For example, "Please specify the patient's weight in kilograms". When extracting the value for representation in a resource, a unit will be added. The questionnaire-unit extension SHOULD be included on the question to support the extraction process.
• When performing extraction, the considerations around errors and lack of support for expressions may come into play. In addition, similar considerations can also be relevant in terms of what 'extraction' approach the Questionnaire supports, as opposed to what level of extraction capability the Form Filler has. The desired behavior of alerting the user that extraction won't be possible is the same whether the issue is "handles FHIRPath vs. CQL" or "uses Observation-based vs. definition-based".
• When generating or updating resources based on information in a QuestionnaireResponse, there may be a desire to capture where that information was sourced from, what software was used to perform conversion, etc. The way to represent this information in FHIR is using the Provenance resource. The QuestionnaireResponse would be referenced using Provenance.entity.what where the Provenance.entity.role would be 'source'. Software information would be captured in Provenance.agent. If the produced records are Observations, the resulting Observation(s) can also directly point to the QuestionnaireResponse using Observation.derivedFrom.
• A single Questionnaire can sometimes include data from multiple subjects. The extraction process needs to be aware of what the subject for a given section of the Questionnaire is and take that into account when creating resources based on the QuestionnaireResponse. (E.g. don't create a heart-rate observation on the mother if that part of the Questionnaire is capturing information about their child. In SDC, the fact that a particular 'group' in a Questionnaire has a distinct subject is communicated using the isSubject extension. An equivalent extension flags the element within the QuestionnaireResponse.
• When selecting data for extraction, systems must always check for the presence of modifier elements or other elements (e.g. Quantity.comparator) that may shift the meaning of the data elements selected and ensure that records whose meaning might be atypical are either not used for extraction or are appropriately flagged to the user for review and possible adjustment.

Extraction service

Like Questionnaire population, extracting data from a QuestionnaireResponse is a complex process involving querying existing FHIR data and using more advanced technologies such as FHIRPath and StructureMap. It's therefore a function that systems may also wish to offload to a separate system. The QuestionnaireResponse extract has been created for this purpose. It takes in a completed QuestionnaireResponse and returns either an individual FHIR resource or a Bundle of resources, depending on the type of Questionnaire. The operation does not post the created resources to a server. It's up to the client system to determine what action(s) to take with the created content.

NOTE: It's the responsibility of the client system to ensure that any generated resources are valid against necessary profiles, etc. before using content produced by this operation.

Designing Questionnaires to support data extraction

This specification defines four different mechanisms to embed information in Questionnaires to support subsequent resource extraction:

• Observation-based
• Definition-based
• Template-based
• StructureMap-based

Systems are free to experiment with other extraction mechanisms but cannot expect support for those from other SDC-conformant systems.

Each mechanism has its own profile that includes the additional resource elements or extensions relevant for supporting a particular mechanism: SDC Questionnaire Extract - Observation, SDC Questionnaire Extract - Definition, SDC Questionnaire Extract - Template, and SDC Questionnaire Extract - Structure Map profiles.
Each profile identifies specific 'must support' elements and extensions that systems that claim to support a specific SDC extraction mechanism SHALL be capable of extracting data, as befits the CapabilityStatement(s) they claim conformance to. Each system should choose which approach(es) it wishes to use and support based on the elements specified in that profile.

Some of these mechanisms make use of FHIR-based queries, FHIRPath and/or CQL as well as extensions that include expressions in one of these languages. Implementers should read the Using Expressions page for background and guidance on these technologies and extensions.

Observation-based extraction

This is the simplest of the extraction mechanisms. It leverages the same data elements as are used for the Observation-based population mechanism. It takes advantage of the fact that most questions in the healthcare space typically correspond to the value element of an Observation. It also takes advantage of the Questionnaire.item.code element that identifies what a concept each question or group corresponds to.
The SDC Questionnaire Extract - Observation profile has been created to support this mechanism. An example for this profile can be found here.

To use this method:

1. Include the item.code element on each question to be extracted. Typically, this will be a LOINC code, but in some jurisdictions/environments, SNOMED CT or other codes may be relevant.
2. Groups can also have an item.code present - this might represent the code of the panel or the Observation.code of an Observation with no value but with multiple Observation.component elements. Child question items can then assert the item.code of the "member-of" Observations or the Observation.component.code values.
3. To signal that the item.code is intended for use in extraction (as opposed to just providing metadata about the Questionnaire item), the questionnaire-observationExtract extension must also be included (and set to true). This extension can be specified either at the root Questionnaire or on an individual question or group item (not a display item) that indicates that the observation-based approach should be used to extract either that particular item (based on the code present) or all items in the questionnaire (if they have a code present).
4. If the extension appears on specific item.code elements rather than the item as a whole, then only the specified codes should be propagated rather than all. For example, an item might list codes for "Body weight", "Body weight (clothed)" and "Body weight (unclothed)". In that case, only the "Body weight" code would be appropriate to include in the extracted content, even though all three might be appropriate for population. If any item.codes are tagged with 'true', then only the tagged Codings will propagate. I.e. any sibling item.codes with no extension are considered to have an extension of 'false'. If no extension appears on any of the item.codes but an extension appears on the item, an ancestor item or the Questionnaire as a whole, then all codes are considered to be roughly equivalent translations and to be appropriate to list as sibling Codings within Observation.code.

   If an item has the extension flag set to 'true', some descendant items may have the extension with a value of 'false'. If this occurs, then the item tagged with 'false' and its descendants will be excluded from the extraction process.

5. Following is the conceptual algorithm for observation-based extraction mechanism:
   • Start at the root of the Questionnaire and progress down through the items.
   • For a particular item, if the item itself or any of its codings have observationExtract set to 'true', then that item is to be extracted (if possible) and the presumption is that descendant items will also be extracted.
   • The presumption of "will extract" will change to false only if an item explicitly sets observationExtract = false at the 'item' level. It is still possible that descendant items might turn extraction back on again. I.e. the full depth must be traversed to catch all extraction candidates.
   • If an item is presumed to be extracted, it will extract if there are item.codes for that item that aren't explicitly marked with observationExtract=false.
   • The observation-extract-category extension only comes into play if a determination has been made that extraction will occur. The observation-extract-category on an item will override that declared on the base questionnaire or ancestor items. If there is a desire to add multiple categories, all must be declared on an item.

For example:

    
	  <item>
		<extension url="http://hl7.org/fhir/uv/sdc/StructureDefinition/sdc-questionnaire-observationExtract">
		  <valueBoolean value="true" />
		</extension>
		<linkId value="code-pop-demo"/>
		<code>
		  <system value="http://loinc.org"/>
		  <code value="29463-7"/>
		  <display value="Body weight"/>
		</code>
		<code>
		  <system value="http://loinc.org"/>
		  <code value="3141-9"/>
		  <display value="Body weight Measured"/>
		</code>
		<code>
		  <system value="http://loinc.org"/>
		  <code value="8341-0"/>
		  <display value="Dry body weight Measured"/>
		</code>
		<text value="What is your current weight?"/>
		<type value="quantity"/>
		<answerOption>
		  <valueCoding>
			<system value="http://unitsofmeasure.org"/>
			<code value="kg"/>
		  </valueCoding>
		</answerOption>
		<answerOption>
		  <valueCoding>
			<system value="http://unitsofmeasure.org"/>
			<code value="[lb_av]"/>
		  </valueCoding>
		</answerOption>
	  </item>
	
  

When performing the extraction process, the system will create a batch that will contain creates or updates of Observation instances. It will go through the QuestionnaireResponse and identify all answers marked for extraction (if the corresponding Questionnaire item or the root Questionnaire has a questionnaire-observationExtract extension). For each of those it will then determine whether to create a new observation, update an existing observation or do nothing. Guidelines for making this decision are as follows:

Take no action if:	the answer was populated from an existing Observation; the system rendering the QuestionnaireResponse can retain context and knows the 'id' of Observation to update; the author of the original Observation is the same as the current author of the QuestionnaireResponse; the context of the questionnaire and the use of it is one where updates are appropriate - as opposed to asserting a new Observation with a new performer and date; and the answer has not changed from the populated value;
Update if:	all of the conditions above apply with the exception that the value has changed.
Create a new Observation if:	the conditions in the preceding two rows are not met

If updating, the original Observation SHALL be adjusted to have the new value or component.value and the status changed to "amended", then PUT to the source system. If creating, data elements SHOULD be populated as follows:

• Observation.basedOn and Observation.partOf - copy from QuestionnaireResponse elements of the same name
• Observation.status - set to 'final'
• Observation.category - if this can be inferred from any of the Questionnaire.item.code values or from known context of the Questionnaire itself, then fill it in. It can also be asserted using the observation-extract-category extension.
• Observation.code - add all the Questionnaire.item.code values as Observation.code.coding instances
• Observation.subject - set to QuestionnaireResponse.subject
• Observation.encounter - set to QuestionnaireResponse.encounter (if an Encounter)
• Observation.effectiveDateTime - set to QuestionnaireResponse.authored.
  
  Note, this is an inference. It is important that the question text implies that the value is 'current' not 'historical' for this to be safe - otherwise do not include the questionnaire-observationExtract extension that marks the question as appropriate for extraction.
• Observation.issued - set to QuestionnaireResponse.authored
• Observation.performer - set to QuestionnaireResponse.author
• Observation.value[x] - set to QuestionnaireResponse.item.answer.value[x]
• Observation.derivedFrom - set to a reference to the QuestionnaireResponse
• Observation.interpretation and Observation.referenceRange - if these can be inferred from the QuestionnaireResponse.item.code (and for interpretation the answer value too), they can be populated, otherwise omit

If the Questionnaire.item that is linked to an Observation contains child items that are also linked to Observations, then things get more complex as a determination will need to be made on whether to link the parent to child as Observation.component or as Observation.hasMember. In the ideal situation, the system will recognize the Observation.item.code and know which approach is correct for that type of Observation. If not, then the system could query for other records of the child type and see if they appear as components anywhere. If unsure, systems should use "hasMember".

Considerations and rules when using this approach:

• If a questionnaire item has the questionnaire-unit extension, the Observation.value SHOULD be a valueQuantity rather than integer or decimal and the units should be taken from the extension value.
• If a question is skipped (no answer) or cleared, no Observation should be created. Existing Observations SHALL NOT be deleted.
• If a question has multiple answers, each answer SHALL be a separate Observation instance.
• There is no mechanism to support items that are mapped to Observation codes which then have nested items without codes - e.g. to capture the text description for an "other - please specify" code - one of the other extraction mechanisms will need to be used.
• Implementers are free to try combining this mechanism with the Definition-based approach. If they do, they should take care that a given item (and its children) are only handled by one approach or the other - not both.
• This approach does not allow for observations where Observation.focus is relevant or for capturing Observation.dataAbsentReason.
• Where an Observation is known to directly correlate to another resource element value (e.g. LOINC 21112-8 corresponds to Patient.birthDate), systems MAY take advantage of this knowledge to update the value of resources other than Observations, however such use is discouraged - using one of the other extraction techniques is likely better and safer.
• Obviously, this mechanism only works for questionnaire items that correspond to Observation values.

Definition-based extraction

This approach to extraction is more generic than the observation-based extraction. It supports extracting data into any type of FHIR resource rather than being limited to only Observation. It also supports extracting additional Observation properties not defined for the observation-based extract, e.g. explicit effective time ranges, interpretations, comments, etc.
The technique is called "definition-based" because it uses StructureDefinitions to identify resource types/profiles, and their properties which are then associated with items in the Questionnaire using the Questionnaire.item.definition element.
The SDC Questionnaire Extract - Definition profile has been created to support this mechanism.

The basic approach to the definition-based extraction mechanism is to walk the QuestionnaireResponse and for each item that has the sdc-questionnaire-definitionExtract extension, create a new stub resource of that type/profile, then scan the item and all its children and populate values in the resource based on items that have a matching definition property set, or have sdc-questionnaire-definitionExtractValue extensions on them that have the same definition canonical URL value as in the definitionExtract extension. Once all the children have been processed, the system will populate the bundle.entry.resource property with the extracted resource, and the bundle.entry.request.method and other properties with values based on the definitionExtract extension. If the definition was to a profile, any slicing information or fixed/pattern values should also be extracted from the profile and applied to the resource where appropriate. For example an identifier might have a fixed system property defined in a profile slice.

The item's definition property is composed of:

• The full canonical URL of the resource or profile.
  e.g. http://hl7.org/fhir/StructureDefinition/Patient or http://example.org/fhir/StructureDefinition/LocalizedPatientProfile
• Followed by a # symbol.
• Then the snapshot.element.id of the element in the StructureDefinition that corresponds to the Questionnaire item.
  e.g. Patient.name.given

Putting this all together, a definition property looks like this: http://hl7.org/fhir/StructureDefinition/Patient#Patient.name.given.

Note: The complete element id does not actually need to appear in a profile snapshot, it is also possible to 'extend' an id to walk further into the data types of the specified element.
For example, the example above http://hl7.org/fhir/StructureDefinition/Patient#Patient.name.given is valid, even though the referenced profile's snapshot only contains Patient.name, not Patient.name.given.
However, for more complex references (e.g. referring to a particular extension or a particular repetition), it will be necessary to define a distinct profile where slicing ensures that a single element id corresponds to the desired element. I.e. you would need to use http://example.org/fhir/StructureDefinition/LocalizedPatientProfile#Patient.name.given:foo where 'foo' was a slice that referred to the first middle name in order to tie to a definition that specific.

To use this method:

1. Traverse the QuestionnaireResponse from its root through each item iteratively, based on the structure in the Questionnaire.
2. Allocate fullUrl uuid variables (if needed): If an extracted resource needs to reference a resource that is created within the transaction bundle it requires a known entry.fulUrl value (as a new uuid value for each resource instance). The sdc-questionnaire-extractAllocateId extension can be used to allocate a named new uuid for this purpose. It should be defined on the item (or root) such that its children cover the resource that is to be created, and also the resources that are to reference it. This variable name can then be used in the definitionExtract extensions's fullUrl fhirpath expression, and also in any reference properties - typically using the definitionExtractValue extension (further described below).
   • This string value is not a fhirpath expression to be evaluated, it is the name for the variable that will be available to fhirpath expressions.
   • The allocateId extension will be made available to all fhirpath expressions on that node, and any child items.
   • Use the variable name in the allocateId to set the entry.fullUrl of the resource you want to reference in the definitionExtract extension's fullUrl property.
   • Use the variable name to set the reference property using the definitionExtractValue extension's expression property.
   • If this allocateId extension is defined at the root of the questionnaire, a single value will be allocated for the entire extracted transaction bundle.
   • If this allocateId extension is defined on a repeating group, a new value will be allocated for each repetition of the item.
   • The allocateId extension will be processed before any other extract extensions (such as definitionExtract or definitionExtractValue)
   • It is recommended to use meaningful names in the allocateId extension value, e.g. %newPatientUuid or %newEncounterUuid
   If the extracted resource being referenced repeats (multiple instances could be extracted), then the extract rules for the resources that reference that repeating item must also be within the same repeating group item in order for them to use the same allocated id.
3. Initiate Resource extraction: Include the sdc-questionnaire-definitionExtract extension on either the Questionnaire root or on other items within the Questionnaire to mark the point at which a new resource will be extracted (and what resource type/profile if applicable).
   This is a complex extension that includes the following properties: definition (resource/profile canonical URL), and optionally fullUrl, ifNoneMatch, ifModifiedSince, ifMatch and ifNoneExist (fhirpath expression expressions used to populate the transaction bundle entry containing the extracted resource).
   • If the extension is at the root of the Questionnaire, the resource will always be extracted, even if there are no answer items within the fully completed QuestionnaireResponse.
   • If the extension is on a specific item, the resource will only be extracted if there are answers to that item in the QuestionnaireResponse. (For items that repeat, a new instance of the resource will be created for each repetition).
   • If the resource is created by a profile definition rather than a core resource type profile, the profile name will be added to the meta of the extracted resource.
   • Any sdc-questionnaire-extractAllocateId extensions on the same item will be processed before the definitionExtract extension.
   • Once the resource is completely extracted, the system SHALL populate the bundle.entry.resource property with the extracted resource, then populate the other bundle.entry properties:
     • fullUrl - the result of the fhirpath expression for the fullUrl property, if none is provided, a new uuid will be allocated for the resource. (this will often be an expression that uses the named variable from the allocateId extension e.g. %newPatientUuid)
     • request.method - if the resource has no id property set the value to POST (requesting a create), otherwise set the value to PUT (requesting an update).
       The resource.id could be set using a hidden item (pre-populated during data-entry), or using the definitionExtractValue extension.
     • request.url - if the resource is being updated the url will be the resource type and the id property of the resource. e.g. Patient/123, otherwise it will be the resource type. e.g. Patient.
     • request.ifNoneMatch - the result of the fhirpath expression for the ifNoneMatch property
     • request.ifModifiedSince - the result of the fhirpath expression for the ifModifiedSince property (evaluates to a date time value - or nothing)
     • request.ifMatch - the result of the fhirpath expression for the ifMatch property
     • request.ifNoneExist - the result of the fhirpath expression for the ifNoneExist property
   • There can be multiple of these extensions interleaved on separate items, and also on the same item. However the definition property in the extension SHALL be unique in the scope that they are introduced in the questionnaire. i.e. You can't define two http://hl7.org/fhir/StructureDefinition/Condition extractions on the same item (or any of its children), but you can have multiple on sibling items do this.
     You could however have different profiles of the same resource overlapping. This is due to the definition property being the "scoper" to identify the resource/property to extract the item answer into.

   Note: In previous versions of the SDC specification the questionnaire-itemExtractionContext extension was used to indicate the resource type, however this has been deprecated and we encourage moving to the newer sdc-questionnaire-definitionExtract extension.

4. Indicate properties to populate in extracted resource: For the item and any descendant items of an item (or root) with the definitionExtractValue extension:
   • Set the Questionnaire.item.definition to associate an answer to the item with a property in the extracted resource.
     Note that the canonical URL to the profile in the definition MUST match the canonical URL in the definitionExtract extension (or the canonical URL of the resource type in the deprecated itemExtractionContext).
     (when interleaving profiles, the definitionExtract extension is the "scoper" for the property to extract the answer into).
   • In cases where you need to make a group item match up with a collection, or backbone element in the StructureDefinition, just use that element to connect them.
     e.g. Use the definition: http://hl7.org/fhir/StructureDefinition/Patient#Patient.name on a group item to collect the child items to the same Patient.name element. During extraction, the system will create a new patient name element for each repetition of the group item, and then populate each name with the child items.
     With this example, the nested child items would have definitions like http://hl7.org/fhir/StructureDefinition/Patient#Patient.name.given and http://hl7.org/fhir/StructureDefinition/Patient#Patient.name.family.
   • Items are not required in the questionnaire for every element level in the StructureDefinition. The system will create the necessary elements as needed.
     e.g. You could have a single item that captures Patient.contact.name.text and the system will create the necessary Patient.contact and Patient.contact.name elements to then set the text values in. In this case if the item was repeating, the last node in the iteration is the one that will be repeated (name in this case).
     If you intend to have a specific level of the element repeated, you should have a group item that repeats, and then have the item that corresponds to the element within that group.
     This technique is particularly useful when populating properties in CodeableConcepts, where you can drill directly into the Coding, or text property, and don't require another level in your questionnaire just to add that in.
   • Add definitionExtractValue extension(s) to the root/item to specify calculated or fixed values to populate into the resource when an answer for this item is entered (or always when at the root of questionnaire).
     This is particularly useful for populating fixed values, or calculated values that are not directly entered by the user.
     e.g. To assign the value from the QuestionnaireResponse.subject property into a property in the extracted resource.
     If definitionExtractValue is used on an item that also has its item.definition property set, that would be assumed to be extracted resource context. Hence you could set the system and type values in an identifier when associated with that item.
   • If a profile is used in the definition, any fixed values or patterns in the profile will be included in the extracted resource. These MAY be processed when stepping through sliced properties in the profile.
     e.g. To set other properties in an identifier property that is associated with a specific slice, such as a fixed system and type value.
   • Although the answer for a QuestionnaireResponse.item can repeat (indicated by the repeat property), the evaluation of the item will only occur once, and the multiple answers are processed for the one item at the same time. For fhirpath expressions, the answer will be a list of values (for the one item) that the expression must handle correctly to populate the item.
     e.g. If the item was for Patient.name.given (which can repeat), that "set" of answers will directly map into the resource property (which is also a collection). However if you tried to map that into the text property, which is singular, this would be an error. A fhirpath expression would be required to concatenate the values together such as answer.join(', ').
     As "group" items iterate differently to "question" items, the evaluation will process each iteration of the group separately.
   • Some type casting will be required as not all data-types are available in questionnaire items, however these can all be mapped.
5. Matching parent/child properties in the extracted resource instance: As described above, the system will create the necessary elements in the resource as needed, but this is not always obvious what to do when it's not a direct 1..1 mapping from the questionnaire to the resource.
   So this table is here to help guide the process with examples of how to populate the resource properties based various parent/child questionnaire item configurations:

   Example Parent	Parent Type	Example Child Definition	Possible Child Type	Notes
   Patient	definitionExtract	Patient.name	group item	Create a new HumanName object and add it to the name collection in patient
   Patient.name	group item	Patient.name.text	value item	Set the text property in the name object
   Patient	definitionExtract	Patient.name.text	value item/extractValue	Create a new HumanName object, add it to the name collection in patient, and set the text property
   Patient	definitionExtract	Patient.identifier.value	value item/extractValue	Create a new Identifier object, add it to the identifier collection in patient, and set the value property
   Patient	definitionExtract	Patient.extension	group item	Create a new Extension object and add it to the extension collection in patient
   Patient.extension	group item	Patient.extension.value	value item	Set the value property in the extension object
   Patient.extension	group item	Patient.extension.url	value item/extractValue	Set the URL property in the extension object
   Patient.extension.value	value item	Patient.extension.url	value item/extractValue	Set the URL property for the same extension instance that has the value in it (walks up the tree to the common ancestor)
   Patient.identifier.value	value item	Patient.identifier.type.text	value item/extractValue	Create a new CodeableConcept object, add it to the type collection of the identifier object that has the value in it, and set the text property in that type object

   Note: Anywhere in a resource that you need to set multiple item's values into properties of a backbone element, you will need to use a group item to collect the values together, and set the definition property on this group item to the backbone element. The system will then create the backbone element and populate it with the child items.

6. Handling sliced properties: If extracting a property via a sliced property:
   • If there are any associated fixed or pattern values in the slice, also populate those values into the resource instance
   • If the slice limits the types supported, ensure that the created type is supported (and cast the value if required)
   • After the property is extracted, if there are any required properties that have fixed/pattern values, also extract those
     e.g. In the body height profile, the category slice VSCat is required, and has a required coding property with a fixed value of "vital-signs". This would also need to be extracted.
   • A type slice can be invoked without an actual slice for choice typed properties e.g. value[x]
     The extraction process is able to determine the correct type to use based on the answer in the QuestionnaireResponse, or explicitly define it in the property name in the "definition" value e.g. http://hl7.org/fhir/StructureDefinition/Observation#Observation.valueQuantity
     This type slicing is really only required where the type is a choice and the value being set is not directly coming from the QuestionnaireResponse item and there is no ambiguity on the type to use. e.g. assigning an item to Patient.extension.value.code could be a Quantity or a Coding, thus needs to be disambiguated. Patient.extension.valueCoding.code or Patient.extension.valueQuantity.value would be the appropriate definitions to use. However if I was using the definition Patient.extension.value and the item was of type boolean or Coding, there is no need to type slice, as the value is of the correct type to just assign (it doesn't need to create the intermediate valueQuantity or valueCoding before trying to set the actual value from the item).
   (These could also be considered at the root level when processing a profiled resource)
7. Supporting the extraction process: If necessary, define questionnaire-hidden items that have Questionnaire.item.initial.value[x] or that use the questionnaire-initialExpression extension to define their content to use to populate resource elements that the user will not be filling in. (The initialExpressions might in turn depend on variable and questionnaire-launchContext extensions, used as described in the Expression-based population section, however that evaluation is all outside the scope of the $extract operation, and those variables are not available to $extract processing.
   This is particularly useful for retaining the resource ID of a resource that is extracted to use to update the resource (method = PUT).

Note: The extraction processes as defined does not support "merging" the answers into an existing instance of a resource as an update. Instead the "update" mode of this extraction process is a "replace" style process. So if you need to update content that was not created by the extraction routine, you will need to inject ALL data from the resource into the questionnaire (and hide properties as needed). This way all the data will be available to fill out into the resource so that it can be included as an update in the outgoing transaction bundle. This design was used as the target server for the bundle isn't necessarily going to be the same server that was pre-populated from.

Examples of questionnaires using this approach can be found here and in Questionnaire/extract-complex-defn3.

Other considerations and rules when using this approach:

• Fhirpath expressions used in the $extract approach do not have access to the launchContext, initialExpression, or variable extensions. They are only able to access the QuestionnaireResponse data and the Questionnaire data and specific extract only extensions (extractAllocateId). Refer to the expressions page for more details.
• If the result of evaluating the FHIRPath expressions is an invalid query, that is an error. Systems SHOULD log it and continue with extraction as if the query had returned no data. The error can be returned in the operation outcome alongside the transaction bundle.
• While validating a Questionnaire resource that uses the definition-based extraction mechanism, the system SHALL validate that the definition property matches up with the StructureDefinition that is being referenced. It can also warn if there is no definitionExtract that would initiate the extraction process for the resource/profile.
• The items in the Questionnaire might not have the same order as those in the resource. When serializing into XML, the official order must still be respected.

Template-based extraction

The template based approach provides an alternative to the definition based approach and where the full power of StructureMaps isn't required. It supports the same level of capability, however is unable to leverage any of the information inside a profile where the definition based approach can.
The SDC Questionnaire Extract - Template profile has been created to support this mechanism.

This technique is called "template-based" because it uses a template resource(s) to provide all the "boiler-plate" content for the resource that is to be extracted. These templated resources are contained within the Questionnaire resource and referred to by either the sdc-questionnaire-templateExtract or sdc-questionnaire-templateExtractBundle extensions.

The template is annotated with expressions to indicate which parts of the template should be populated with data from the QuestionnaireResponse, and which parts should be removed if no data is present. The expressions are defined using the FHIRPath language, and the QuestionnaireResponse context of the expressions is based on the location of the template reference in the questionnaire, and/or any inline context expressions defined in the template.

The difference between the templateExtract (resource) and templateExtractBundle extensions is that the template resource extension will extract each resource individually as a separate bundle entry and includes expressions to populate the other bundle entry properties, whereas the template bundle extension extract the transaction bundle as a single resource. The processing of the template is otherwise identical between the two extensions.

To use this method:

1. Include the sdc-questionnaire-templateExtract extension at the root or the questionnaire, and/or any items in the questionnaire that indicate a resource should be created for each answer in a questionnaire response for that item.
   This extension includes a relative reference to the resource template which SHALL be contained within the same Questionnaire resource, and will be extracted into a new entry in the output transaction bundle (with other values processed from the complex extension).
   Alternately use the sdc-questionnaire-templateExtractBundle extension at the root to use a template bundle. This is the same as the templateExtract extension but leverages a single bundle that has all templated resources in it.
2. Include any required template resources as contained resources in the questionnaire and refer to them using either the sdc-questionnaire-templateExtract or sdc-questionnaire-templateExtractBundle extension as described above.
3. If you need to dynamically reference between resources created in the output transaction bundle (using the entry.fullUrl property), include the allocateId extension at a point in the questionnaire that defines the scope that a new uuid is required for each iteration of the item. If at the root, a new value will be allocated only once, if on an item, it will be allocated for each related item that has the extension (a new value per item in a collection). The value is only available to child items of the item that has the extension (or itself).
   This value can be used in the templateExtract extension to set the fullUrl property of the extracted resource, and in the templateExtractValue extension to set the value for the referencing property.
   The allocateId extension is not a fhirpath expression to be evaluated, it is the name for the variable that will be available to fhirpath expressions, and it's value will be a new uuid for each iteration of the item that has the extension, before any expressions in the templateExtract, templateExtractBundle or templateExtractValue are evaluated.
4. During extraction a new resource will be created for each answer in the QuestionnaireResponse for the item that contains the templateExtract extension.
   The resource.id of the new resource will be removed (as it currently has the resource id to refer to in the template from the questionnaire definition). To set the resource.id property use the resourceId property in the templateExtract extension (this is not possible using the templateExtractBundle based extraction).
5. The fhirpath context of the extracted resource will be based on the location of the templateExtract extension in the questionnaire.
   If templateExtract is on the root of the questionnaire, the fhirpath context will be the QuestionnaireResponse resource.
   If templateExtract is on an item, the fhirpath context will be the item in the QuestionnaireResponse associated with that item's linkId.
   
6. The extraction engine SHALL scan the templated resource for any sdc-questionnaire-templateExtractContext or sdc-questionnaire-templateExtractValue extensions (in that order) and then:
   • Evaluate the fhirpath expression defined in the extension (only FHIRPath is supported).
   • If the expression returns no results, this templated property (context or value) will be removed from the extracted resource, and no further processing of that property will occur.
     If the template property is a in a collection, simply remove the templated value from the collection, don't clear the entire collection. There MAY be other templated properties to process in the collection (or static template data).
   • For every result returned by the expression, a new copy of the templated property will be cloned and scanning the child properties will continue.
   • If the context expression is defined as an Expression datatype, and it has a name property, then that variable will be made available to any child property extraction expressions.
   • Any context results will change the context to use for processing any child properties (and templateExtractValues on the same node).
   • Any extractValue results will be used to replace the property value in the extracted resource.
     Note that for primitive properties in json the _value and value represent the same FHIR property and must be considered the same property. So ensure that the extract extension is removed from _value, the value is set (and any other extensions can remain).
     This is important as some properties such as reference properties are required to have a valid value and not just an extension so they will have a value that is expected to be either removed (when no data) or replaced (when data is present).
7. When using the templateExtract extension (not the templateExtractBundle extension), the extraction engine SHALL create a transaction Bundle containing all the resources that were extracted from the QuestionnaireResponse. Each resource that is extracted will be added into the transaction bundle that is returned by the $extract operation.
   The fullUrl, resourceID and entry.request properties SHALL be updated with values calculated from the templateExtract complex extension.
   If a resource has it's ID populated, then it will be treated as an update to the existing resource (entry.request.method='PUT'), otherwise it will be treated as a create (entry.request.method='POST').

If fhirpath expressions produce errors/exceptions or invalid/incompatible value types, the extraction engine SHALL log the error and continue processing the next property. If there were any fatal or error level issues, the extraction engine SHALL return an OperationOutcome with the issues.
If there are only warning/information issues, they can be returned along with the transaction bundle.

Template based extraction using the templateExtractBundle extension is incompatible with modular forms.
If using the templateExtract extension with modular forms, the assembly will merge all the contained resources from the module into the output Questionnaire, and may need to rename the contained reference ids if collisions are to occur.

Note: Many templating engines have constructs like conditionals and loops. Although not immediately obvious, these can be implemented using this template approach. Conditional properties are implemented with the fhirpath expression in the templateExtractContext or templateExtractValue extension returning no results, and thus excluding the property, backbone element or entry in a collection from the extracted resource.
Loops are similar in nature where the looping is provided based on the fhirpath expression returning multiple results, and thus creating multiple entries in an array, or even multiple resources in the transaction bundle driven by either multiple items in the questionnaire, or contexts in the bundle template.
There are several examples of both of these in the template example, such as how it creates multiple names in the patient resource from multiple items in the questionnaire name item group, or where there is no IHI identifier added if the relevant item hasn't been answered in the questionnaire.

StructureMap-based extraction

The StructureMap approach is the most sophisticated approach of the three - and the most powerful. It allows significant transformation of data, including code translations when generating output resources. It also allows the conversion process between data and Questionnaire to be maintained independently and to draw on shared sources across Questionnaires. This can be an advantage in certain environments where the content of the questionnaire may need tight control, but the data environment can be more dynamic. This comes at the cost of requiring expertise in the FHIR mapping language, which is not (yet?) a common skill.
The SDC Questionnaire Extract - StructureMap profile has been created to support this mechanism. An example for this profile can be found here.

To use this method:

1. Include the questionnaire-targetStructureMap extension. This SHALL define a transform between the QuestionnaireResponse and either a single resource or a transaction Bundle containing the set of resources extracted from the QuestionnaireResponse.

To extract data from the completed QuestionnaireResponse, simply invoke the StructureMap on it. A sample Questionnaire and associated StructureMap that shows this approach can be found here and here.

Considerations when using this approach:

• This mode has the drawback that if the StructureMap execution fails, there will generally not be any data extracted from the Questionnaire. With the other approaches, if one Observation or context fails, the others might still work. As a result, the StructureMap must be designed to be very robust in the face of missing or potentially 'bad' data.
• The ability of StructureMaps to reference other StructureMaps allows for the possibility of re-use if certain sections of multiple questionnaires are consistent.

Output expectations

Following are the output expectations of $extract:

• Observation-based will always produce a 'transaction' Bundle, even if it's only creating or updating a single Observation.
• Definition-based will always produce a 'transaction' Bundle that indicates what resources are to be created or updated. The resources within the transaction can be anything (including Bundle resources such as FHIR documents)
• StructureMap-based may produce either a 'transaction' Bundle or a single resource instance (though the instance might itself be a different type of Bundle, such as a document or collection). It is possible for StructureMaps to map to non-FHIR data structures, however there's no "built-in" mechanism to serialize such models, so additional information would need to be passed into the $extract process to guide such serialization. Support for non-FHIR extracts is outside the scope of this specification.