FHIR CI-Build

This is the Continuous Integration Build of FHIR (will be incorrect/inconsistent at times).
See the Directory of published versions icon

FHIR Infrastructure icon Work GroupMaturity Level: 2Standards Status: Draft

The FHIR specification describes a set of resources, and several different frameworks for exchanging resources between different systems. Because of its general nature and wide applicability, the rules made in this specification are fairly loose. As a consequence, and in order to insure interoperability between applications claiming conformance to this specification, a testing framework has been established within the FHIR specification. To this end, the TestScript resource provides an implementation-agnostic description of tests that allows test execution engines to evaluate if a FHIR implementation conforms with the FHIR specification. Providing a clear and concise test methodology for the FHIR specification through the TestScript resource helps to enable interoperability among various FHIR server and client implementations.

Furthermore, the TestScript resource provides clear examples of the appropriate use of the FHIR specification through test-based documentation. The TestScript resource stands as a form of executable documentation allowing developers to examine the operations defined by the tests in order to understand how various RESTful API interactions and resources should be used in coordination. The tests can also be automatically executed against systems under development to determine how well the systems adhere to the specification.

The TestScript resource contains:

  • Name and description detailing the purpose of the test suite
  • Links describing how the test suite relates to the FHIR specification
  • A list of server interactions required to execute the test suite
  • A list of server interactions that the test suite validates the correctness of
  • The fixtures (required data or resources) the tests use during execution
  • A set of operations to set up the test suite environment
  • A list of tests each containing
    • Name and description of the test
    • Links describing how the test relates to the FHIR specification
    • A list of server interactions required to execute the test
    • A list of server interactions that the test validates the correctness of
    • A list of operations that provide the execution logic of the test
    • A list of assertions that provide the verification logic of the test
  • A set of operations to tear down the test environment

Pre-Processing

The TestScript execution workflow begins by determining if the test suite is appropriate for the system under test. This is determined by evaluating if the interactions defined in the TestScript setup and test action.operation sections are supported by the CapabilityStatement of the system under test. The test engine SHOULD provide a comparison of the system under test capabilities against the TestScript interactions.

If the server supports the requirements of the TestScript instance, any specified fixtures are loaded or retrieved. If the fixtures are marked as 'autocreate' then they are automatically created on the system(s) under test using 'create' operations. If any of the autocreate operations fail, then the tests in the TestScript are skipped.

Setup Execution

After the fixtures are loaded and autocreates are executed, the setup section is executed to establish the testing environment. The purpose of the setup section is typically to pre-load data (if it was not autocreated) or delete data required for the execution of the tests against the FHIR system(s) under test. The setup operations are executed once before all the tests are run (see Operation Execution). All operations in a setup section (including assertions) must complete successfully for the subsequent tests to be executed. If an assertion operation in the setup section fails, then execution and evaluation of the tests in the TestScript should be skipped. Technically, any operation (see the operations table for a complete listing) can be included in the setup section, but typical operations will be create, update, read, and vread.

Due to the possibility that the setup actions are not required on the server under test, the TestScript execution workflow MAY provide the capability of skipping or ignoring the setup section of the TestScript.

Test Execution

Once setup is complete, each test is executed. Tests contain a set of operations, and executing a test involves the evaluation of each operation listed in the test in the order defined by the test (see Operation Execution and the list of operations).

Teardown Execution

After all the tests have completed execution, the teardown section is executed. The purpose of the teardown section is to revert the FHIR server under test to a pre-test clean state. This requires removing any resources or artifacts generated as part of test suite setup or test execution. Technically, any operation (see the operations table for a complete listing) can be included in the teardown section, but the most often used operation will be delete. Assertions are not supported in the teardown section.

Due to the possibility that the teardown actions are not required on the server under test, the TestScript execution workflow MAY provide the capability of skipping or ignoring the teardown section of the TestScript.

Post-Processing

After the teardown section is executed, any fixtures that were marked 'autodelete' are removed from the server(s) under test. After this final stage, the execution of the TestScript is complete. The test engine SHALL provide the results of the TestScript execution with a generated TestReport resource. The test engine MAY also provide the results via additional generated output.

The fixtures section of the TestScript defines a set of resource instances that will be used as part of the setup, test, and teardown sections during TestScript execution. All defined fixtures are expected to be required in order for the test script to execute. Each fixture defines a resource instance by URI, and must be identified by an ID. The URI can be local or remote (i.e. another server than the one the TestScript resource resides), absolute or relative. The ID on the fixture is considered the "source" identifier of the fixture -- it is not the same thing as the resource ID on the server where it was hosted. The "source" identifier is used to define the fixture instance within the context of the TestScript. Operations reference the ID of a fixture to uniquely identify the fixture instance the operation is using ("sourceId") or acting against ("targetId"). Once a fixture has been instantiated on a server (typically by the use of a create operation), the fixture ID is mapped to the ID of the corresponding resource instance on the server. TestScript execution engines must maintain this relationship between fixture IDs and server resource IDs. The TestScript execution engine is responsible for translating the fixture IDs (whether provided to the operation as "source" or "target") to the ID of the resource on the server during execution.

Using the optional "autocreate" and "autodelete" elements (missing values default to false), fixtures can be configured to automatically be created during TestScript setup and automatically deleted during TestScript teardown. This means that additional "create" and "delete" operations in the TestScript.setup and TestScript.teardown sections are unnecessary.

The profiles section of the TestScript defines a set of FHIR profiles (see StructureDefinition resource) that will be used as part of the setup or test sections during TestScript execution. Each profile defines a StructureDefinition instance by URI, and must be identified by an ID. The URI SHALL reference a known FHIR profile corresponding to the FHIR specification version being tested. As with Fixtures, the ID on the profile is considered the "source" identifier of the profile. Operations reference the ID of a profile to uniquely identify the profile instance the operation is using ("validateProfileId").

See Use Profiles for more information.

The variables section of the TestScript defines a set of expressions whose evaluations will be used in substitutions. These substitutions are performed in operation request headers and URL paths as well as assertion values.

Without variables, search parameters and request headers (such as If-Modified-Since) would be specified in outgoing requests as literal values. Variables allow the values to be managed externally in fixtures or dynamically in server response fixtures. They would be defined to hold path expressions against the fixtures. The path expressions would not change from one server to another, but the fixture data could.

Using variables allows for the same test scripts to be executed against the same servers by different clients at the same time. Each client would change the fixture data (external to the test script) to make the data unique to that client. This ensures that the same delete/create/read operations executed concurrently by one client does not interfere with those of another client. That can be important within the context of a testing event such as a Connectathon. It can be very useful in year-round testing against public servers as well.

See Use Variables for more information.

The setup, test, and teardown sections of a TestScript can contain operation elements. Operations are intended to be executed in sequence and they represent the logic defined by the TestScript. Operations define a type, sourceId, targetId, destination, responseId, contentType, and other parameters. The type of the operation aligns with a corresponding RESTful API interaction. The sourceId and targetId of an operation define the context of the fixture data the operation is acting against (see Fixtures). The destination defines the server the operation is executed on and is only required for tests that include multiple servers. The responseId specifies a fixture ID to use to map to the server response. The contentType defines the format (XML or JSON) and the corresponding mime-type (application/fhir+xml and application/fhir+json) to use in the RESTful operation (defaulting to XML). The parameters of an operation allow providing additional data required for execution.

TestScript execution engines must load the operation details and execute the operation against the server(s) under test. Operations that are expected to result in an error or exception shall immediately be followed by one or more "assertion" operations (that test for those error conditions), otherwise the test fails. This allows for "negative" testing (for example, the test script may perform operations that should return a 4XX or 5XX HTTP response code). Test execution is halted as soon as an operation or assertion fails. If an operation or assertion fails, then the test ends in failure and the test script execution proceeds to the next test. Once all tests have completed execution, the teardown section is executed. Once teardown completes, the suite execution is complete. If any setup or test operation or assertion failed, the test script is marked in failure. Failures in teardown are ignored.

The "assertion" evaluates the results of previous operations to determine if the server under test behaves appropriately. In order to evaluate an assertion, the request, response, and results of the most recently executed operation must always be maintained by the TestScript execution engine.

The TestScript execution engine must implement the behavior of each assertion to determine if the response of the last operation meets the conditions of the assertion.

If the conditions of the assertion are met execution of the test continues. If the conditions of the assertion are not met, the test being executed fails and execution of the test is halted. Test execution will then continue with the next test.

Conditional Asserts

The ability to conditionally evaluate an assert is not defined as part of the native behavior within the TestScript resource or execution engine. This means that all subsequent test asserts are not evaluated when a prior assert within the same test evaluates to a failure. The current workaround is to set warningOnly=true for those asserts that require conditional evaluation.

TestScript authors may use the assert.stopTestOnFail boolean element to further control whether a failed assert will halt the current test's execution.

Another optional workaround is to provide the TestScript execution engine with access to an external Rules engine. The TestScript execution engine author is encouraged to build and publish this functionality within an appropriate FHIR ImplementationGuide.

To mark a resource as a test resource, use the testing tag:

<Patient xmlns="http://hl7.org/fhir">
  <meta>
    <security>
      <system value="http://terminology.hl7.org/CodeSystem/v3-ActReason"/>
      <code value="HTEST"/>
      <display value="test health data"/>
    </security>
  </meta>
</Patient>

Not all testing resources need to explicitly marked as 'HTEST' but, it may be necessary when performing testing on a production system. Note: This is a security label because different access control rules may apply to test resources.

To test if the create operation is properly supported on a server, run the operation as part of TestScript.test.

First, define the fixture as a reference at the top of the test script. The fixture will hold the body of the POST request:

<fixture id="example-patient"> <resource> <reference value="Patient/patient-example.xml"></reference> </resource> </fixture>

Note that it is illegal for the fixture to contain a resource id in a create operation.

Point the sourceId element of the create operation to the fixture id just defined:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <sourceId value="example-patient"></sourceId> </operation> </action>

There are two ways to verify that the create operation returned the right status code:

  1. Use assert.response:

    <action> <assert> <response value="created"></response> </assert> </action>
    See response codes for complete list.
  2. Use assert.responseCode explicitly:

    <action> <assert> <responseCode value="201"></responseCode> </assert> </action>

To test if search operation is properly supported on a server, run the operation as part of TestScript.test.

Use the resource element to specify the resource type and the params element to specify the search parameters:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="search"></code> </type> <resource value="Patient"></resource> <contentType value="json"></contentType> <params value="?given=John&amp;family=Doe"></params> <responseId value="R1"></responseId> </operation> </action>

The contentType element is optional and will default to "xml" which will translate to HTTP request header "Content-Type" being set to "application/fhir+xml" by test engines. In this case, though, it was used to set it to "application/fhir+json".

The responseId element was used to store the response in a reference called "R1". This reference will hold both the response headers and the response body.

Verify that the search operation returned the right status code:

<action> <assert> <response value="okay"></response> </assert> </action>

See response codes for complete list.

Verify that the search operation returned the right resource type:

<action> <assert> <resource value="Patient"></resource> </assert> </action>

There are many ways to verify that the search operation returned the right Patient:

  1. Explicitly compare the elements to known value:

    <action> <assert> <path value="fhir:Patient/fhir:birthDate/@value"></path> <sourceId value="R1"></sourceId> <value value="1974-12-31"></value> </assert> </action>

    The sourceId element is pointed to the responseId value of the search operation. If no sourceId is specified, then test engines will use the response of the last operation in the test script even if responseId was not specified in the operation.

    The path element holds a FHIRPath expression against the response body contents.

  2. Compare the elements in response to elements in another fixture that is either dynamically set by responseId or defined statically by the fixture element at the top of the script:

    <action> <assert> <compareToSourceId value="F1"></compareToSourceId> <compareToSourcePath value="fhir:Patient/fhir:birthDate/@value"></compareToSourcePath> <path value="fhir:Patient/fhir:birthDate/@value"></path> </assert> </action>

    This time the birthDate value in the response is compared to the birthDate value in a fixture called 'F1'.

  3. Verify that the response contains all the element/content in another fixture pointed to by minimumId.

    <action> <assert> <minimumId value="F1"></minimumId> <sourceId value="R1"></sourceId> </assert> </action>

    Test engines will parse the 'body' of the F1 fixture and verify that each element and its value matches the corresponding element in the R1 response body. In other words, R1 is verified to be a 'superset' of F1. The resource id element in the body will be ignored during comparison. The headers will also be ignored.

    F1 can be statically defined or it can be the responseId for another operation. If sourceId is not specified, then test engines will use the response of the last operation. So the previous assertion could have been defined as:

    <action> <assert> <minimumId value="F1"></minimumId> </assert> </action>

    See section Use minimumId for a more complete description of the test engine's comparison logic.


Test scripts should clean up resources created as part of execution. The TestScript.teardown operations will get executed once before the test script execution completes.

Here are a couple of ways to run delete operation in TestScript.teardown:

  1. Use conditional delete operation in TestScript.teardown:
    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="delete"></code> </type> <resource value="Patient"></resource> <params value="?given=John&amp;family=Doe"></params> </operation> </action>
  2. Use delete operation with targetId fixture.

    To do that, the resource must have been created during TestScript.setup or TestScript.test:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <responseId value="create-response"></responseId> <sourceId value="example-patient"></sourceId> </operation> </action>

    As part of TestScript.teardown, run the delete operation with targetId value pointed to sourceId value of the create operation:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="delete"></code> </type> <targetId value="create-response"></targetId> </operation> </action>

    Test engines will keep track of response headers and body of all operations.

    The delete operation's targetId value is expected to correspond to the responseId of a GET operation (such as search or read) or the sourceId of a POST/PUT operation (such as create).

    For targetId value corresponding to responseId of GET operations (such as search or read), test engines will use the resource type and id returned in the GET response body's resource to set the [type] and [id] in delete operation's URL, respectively.

    For targetId value corresponding to responseId of POST/PUT operations (such as create), test engines will use the resource type and id returned in the POST/PUT response "Location" header to set the [type] and [id] in delete operation's URL, respectively. This is the case in the example above.

    The targetId value cannot point to a statically defined fixture as the id in the fixture cannot be relied upon.


Deletion of resources created during test script execution should be done using delete operation in TestScript.teardown. See Perform delete operation in teardown for details.

There might be left-over resource instances though on the server from prior executions of the script that terminated prematurely through an error. Resources can be deleted in TestScript.setup as well to ensure reliable test execution.

To delete a resource in setup, the server is required to support Conditional Delete operation.

Use the params element to specify the search criteria for the delete operation:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="delete"></code> </type> <resource value="Patient"></resource> <params value="?family=Doe&amp;given=Joe"></params> </operation> </action>

Test engines will append the contents of the params element to url after [type]: "[base]/[type]?[search parameters]". The resource element value ("Patient") will be used to replace [type] in the url.


To test if a server supports conditional delete operation, run a create operation prior to the delete using a sourceId that points to a fixture defined at the top of the script:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <sourceId value="example-patient"></sourceId> </operation> </action>

Then use the params element to specify the search criteria for the delete operation:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="delete"></code> </type> <resource value="Patient"></resource> <params value="?family=Doe&amp;given=Joe"></params> </operation> </action>

Test engines will append the contents of the params element to url after [type]: "[base]/[type]?[search parameters]". The resource element value ("Patient") will be used to replace [type] in the url.

To verify that the delete operation returned the right status code:

<action> <assert> <operator value="in"></operator> <responseCode value="200,204"></responseCode> </assert> </action>

To verify that the resource was indeed deleted on the server, run a search using the same parameters and verify that the status code is 404 (not found):

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="search"></code> </type> <resource value="Patient"></resource> <params value="?given=John&amp;family=Doe"></params> </operation> </action>
<action> <assert> <response value="notFound"></response> </assert> </action>

To test if a server supports conditional create operation, use the 'If-None-Exist' request header:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <requestHeader> <field value="If-None-Exist"></field> <value value="Patient?given=John&amp;Doe&amp;birthdate=1974-12-31"></value> </requestHeader> <sourceId value="F1"></sourceId> </operation> </action>

The response code of 200 verifies that the resource already exists and did not get created:

<action> <assert> <responseCode value="200"></responseCode> </assert> </action>

Update operations require a resource id. The id must be present in the fixture (PUT body contents) as well as the URL. The values must match.

Because resource ids cannot be predicted on the server, it is best to retrieve the id on a resource freshly created as part of the script

There are many ways to do that. Below is a couple:

  1. Use update operation with targetId fixture pointing to create operation's responseId:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <responseId value="create-response"></responseId> <sourceId value="example-patient"></sourceId> </operation> </action>
    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="update"></code> </type> <resource value="Patient"></resource> <responseId value="R3"></responseId> <sourceId value="john-doe-update"></sourceId> <targetId value="create-response"></targetId> </operation> </action>

    Test engines will keep track of response headers and body of all operations.

    The update operation's targetId value is expected to correspond to the responseId of a GET operation (such as search or read) or the sourceId of a POST/PUT operation (such as create).

    For targetId value corresponding to responseId of GET operations (such as search or read), test engines will use the resource type and id returned in the GET response body's resource to set the [type] and [id] in update operation's URL, respectively. This is the case in the next example below.

    For targetId value corresponding to responseId of POST/PUT operations (such as create and update), test engines will use the resource type and id returned in the POST/PUT response "Location" header to set the [type] and [id] in update operation's URL, respectively. This is the case in the example above.

    The targetId value cannot point to a statically defined fixture as the id in the fixture cannot be relied upon.

  2. Use update operation with targetId fixture pointing to search operation's responseId:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <responseId value="R1"></responseId> <sourceId value="john-doe-patient"></sourceId> <!-- Fixture must be defined at the top of the script --> </operation> </action>
    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="search"></code> </type> <resource value="Patient"></resource> <params value="?family=Doe&amp;given=Joe"></params> <responseId value="R2"></responseId> </operation> </action>
    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="update"></code> </type> <resource value="Patient"></resource> <responseId value="R3"></responseId> <sourceId value="john-doe-update"></sourceId> <targetId value="R2"></targetId> </operation> </action>

After the update operation, test scripts would perform at least one more read/search operation to retrieve the contents of the updated resource and then perform assertions to verify that the data did indeed get updated on the server:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="search"></code> </type> <resource value="Patient"></resource> <params value="?family=Doe&amp;given=Joe"></params> <responseId value="R4"></responseId> </operation> </action>

Verify that the birthdate got updated and is being returned properly:

<action> <assert> <path value="fhir:Patient/fhir:birthDate/@value"></path> <sourceId value="R4"></sourceId> <value value="1974-12-31"></value> </assert> </action>

Unlike a regular update operation, a conditional update operation does not require a resource id in the URL (or the body of the PUT).

To test conditional update, use params element in the operation instead of targetId. The resource element will be required in this case.

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="update"></code> </type> <resource value="Patient"></resource> <params value="?family=Doe&amp;given=Joe"></params> <responseId value="R3"></responseId> <sourceId value="john-doe-update"></sourceId> <!-- john-doe-update fixture will have the 'correct' birthDate --> </operation> </action>

Test engines will append the contents of the params element to url after [type]: "PUT [base]/[type]?[search parameters]". The resource element value ("Patient") will be used to replace [type] in the URL.

Verify that the birthdate got updated and is being returned properly:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="search"></code> </type> <resource value="Patient"></resource> <params value="?family=Doe&amp;given=Joe"></params> <responseId value="R4"></responseId> </operation> </action>
<action> <assert> <path value="fhir:Patient/fhir:birthDate/@value"></path> <sourceId value="R4"></sourceId> <value value="1974-12-31"></value> </assert> </action>

The read operation requires the resource id in the URL. Since resource ids are unpredictable on servers, it's best to create the resource within the test script prior to executing the read operation:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <responseId value="create-response"></responseId> <sourceId value="example-patient"></sourceId> </operation> </action>

One way to execute the read operation is to run the read operation with targetId value pointed to responseId value of the create operation:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="read"></code> </type> <targetId value="create-response"></targetId> </operation> </action>

Test engines will keep track of response headers and body of all operations.

The read operation's targetId value is expected to correspond to the responseId of a GET operation (such as search or read) or the sourceId of a POST/PUT operation (such as create).

For targetId value corresponding to responseId of GET operations (such as search or read), test engines will use the resource type and id returned in the GET response body's resource to set the [type] and [id] in read operation's URL, respectively.

For targetId value corresponding to responseId of POST/PUT operations (such as create), test engines will use the resource type and id returned in the POST/PUT response "Location" header to set the [type] and [id] in read operation's URL, respectively. This is the case in the example above.

The targetId value cannot point to a statically defined fixture as the id in the fixture cannot be relied upon.


The vread operation requires the resource id as well as the resource version id in the URL. Since resource ids and version ids are unpredictable on servers, it's best to create the resource within the test script prior to executing the vread operation:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <responseId value="create-response"></responseId> <sourceId value="example-patient"></sourceId> </operation> </action>

One way to execute the vread operation is to run the vread operation with targetId value pointed to responseId value of the create operation:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="vread"></code> </type> <targetId value="create-response"></targetId> </operation> </action>

Test engines will keep track of response headers and body of all operations.

The vread operation's targetId value is expected to correspond to the responseId of a GET operation (such as search or read) or the sourceId of a POST/PUT operation (such as create).

For targetId value corresponding to responseId of GET operations (such as search or read), test engines will use the resource type, id and version id returned in the GET response body's resource to set the [type], [id] and [vid] in vread operation's URL, respectively.

For targetId value corresponding to responseId of POST/PUT operations (such as create), test engines will use the resource type, id and version id returned in the POST/PUT response "Location" header to set the [type], [id] and [vid] in vread operation's URL, respectively. This is the case in the example above.

The targetId value cannot point to a statically defined fixture as the id in the fixture cannot be relied upon.


The history operation can be executed in the following ways:

  1. GET [base]/[type]/[id]/_history{?[parameters]&_format=[mime-type]}

    Here the resource id is required in the URL. This is similar to read operation if targetId elemet is used. See Test read operation for details.

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="history"></code> </type> <targetId value="create-response"></targetId> <!-- or search-response or update-response --> </operation> </action>
    <action> <assert> <resource value="Bundle"></resource> </assert> </action>
  2. GET [base]/[type]/_history{?[parameters]&_format=[mime-type]}

    Here the resource id is not required in the URL.

    Instead of targetId element, the params element can be used to specify the search criteria for the history operation.

    In the following example, all history entries for John Doe patient would be returned by server:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="history"></code> </type> <resource value="Patient"></resource> <params value="?family=Doe&amp;given=Joe"></params> </operation> </action>
  3. GET [base]/_history{?[parameters]&_format=[mime-type]}

    Here neither the resource type nor the resource id is required in the URL. In the following example, no more than 50 history entries would be returned by server:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="history"></code> </type> <params value="?_count=50"></params> </operation> </action>

The default "Accept" header that will be set on all GET operations (such as read, vread, search, history, etc.) will be "application/fhir+xml".

There are two ways to change the default "Accept" header:

  1. Use the accept element:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="read"></code> </type> <accept value="json"></accept> <targetId value="F1"></targetId> </operation> </action>

    Test engines will set the Accept header to "application/fhir+json" if "json" is specified and will use "application/fhir+xml" if "xml" is specified.

  2. Use the requestHeader element to set "Accept" explicitly:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="read"></code> </type> <requestHeader> <field value="Accept"></field> <value value="application/fhir+json"></value> </requestHeader> <targetId value="F1"></targetId> </operation> </action>

    Test engines will take values specified for requestHeader "as-is" and not transform them. This might be useful for negative testing, e.g. the value can be set explicitly to "application/xml" or an invalid value and verify server response.


The default "Content-Type" header that will be set on all POST/PUT operations (such as create, update, etc.) will be "application/fhir+xml".

There are two ways to change the default "Content-Type" header:

  1. Use the contentType element:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <contentType value="json"></contentType> <targetId value="F1"></targetId> </operation> </action>

    Test engines will set the Content-Type header to "application/fhir+json" if "json" is specified and will use "application/fhir+xml" if "xml" is specified.

  2. Use the requestHeader element to set Content-Type explicitly:

    <action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <requestHeader> <field value="Content-Type"></field> <value value="application/fhir+json"></value> </requestHeader> <targetId value="F1"></targetId> </operation> </action>

    Test engines will take values specified for requestHeader "as-is" and not transform them. This might be useful for negative testing e.g. the value can be set explicitly to "application/xml" or an invalid value and verify server response.


There are two ways to verify the "Content-Type" header in response:

  1. Use the contentType element:

    <action> <assert> <contentType value="json"></contentType> </assert> </action>

    Test engines will verify that "application/fhir+json" is present in Content-Type header if "json" is specified and will verify that "application/fhir+xml" is present if "xml" is specified.

  2. Use the requestHeader element to verify Content-Type explicitly:

    <action> <assert> <headerField value="Content-Type"></headerField> <value value="application/fhir+json"></value> </assert> </action>

    Test engines will take values specified for headerField "as-is" and not interpret them.

    Note that test engines will not verify contentType in response if assertions for contentType are missing.


Profiles are defined against known FHIR profiles (see StructureDefinition resource) corresponding to the FHIR specification version being tested. They are used in the "assert.validateProfileId" and, when present in an assert, will invoke validation of the FHIR resource payload in either the request or response message.

Profiles would be defined at the top of the script.

Below is a profile that is defined against the FHIR Patient resource type and referenced by "P1":

<profile id="P1"> <reference value="http://hl7.org/fhir/StructureDefinition/Patient"></reference> </profile>

Test engines will not evaluate this at this point. They will store the profile reference in "P1" and will look for "P1" in an "assert.validateProfileId" element values during operation calls.

Here is a read operation for a Patient with a resource id of "example" followed by an assert that performs a FHIR validation using the base FHIR profile for the Patient resource type referenced by the "P1" profile. The validation will be performed against the response payload by default if the "assert.direction" is not present. If the validation returns any "fatal" or "error" messages, then this assert will be failed. Otherwise, if the validation returns any "warning" messages, then this assert will pass but with warnings. Finally, if the validation returns only "information" messages or no messages, then this assert will pass.

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="read"></code> </type> <resource value="Patient"></resource> <accept value="xml"></accept> <params value="/example"></params> </operation> </action> <action> <assert> <validateProfileId value="P1"></validateProfileId> </assert> </action>

Variables can be defined against static fixtures and dynamic operation responses. They can be used in "operation.params", "operation.requestHeader.value", "operation.url", and "assert.value" element values. As such they allow for the data used in operations and assertions to be externally defined. The data could be unique to each client involved in interactions with a server or could be unique to a given server database instance. This allows for multiple clients to execute the same test scripts concurrently against the same server.

Variables would be defined at the top of the script.

Below is a variable that is defined as the Location header to the response referenced by "R1":

<variable> <name value="V1"></name> <headerField value="Location"></headerField> <sourceId value="R1"></sourceId> </variable>

Test engines will not evaluate this at this point. They will store the expression in "V1" and will look for "${}" in "operation.params", "operation.requestHeader.value", and "operation.url" element values during operation calls.

Here is a read operation that will use the V1 variable. The variable expression was "Location against R1 response" (defined above). If a prior operation has not set R1 to be the responseId of the operation, then test engine will error out. Otherwise, V1 will be set to the Location header value of R1 response and that value will be substituted for ${V1} below. In other words, the read will be performed against the Location header value of R1 response.

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="read"></code> </type> <accept value="json"></accept> <responseId value="R2"></responseId> <url value="${V1}"></url> </operation> </action>

Below are three variables defined as path expressions against the static fixture referenced by "F1". The expressions are against the given name, family name, and birthDate of a patient resource. The resource data will be managed external to the test script.

<variable> <name value="PatientGivenName"></name> <path value="fhir:Patient/fhir:name/fhir:given/@value"></path> <sourceId value="F1"></sourceId> </variable> <variable> <name value="PatientFamilyName"></name> <path value="fhir:Patient/fhir:name/fhir:family/@value"></path> <sourceId value="F1"></sourceId> </variable> <variable> <name value="PatientDOB"></name> <path value="hir:Patient/fhir:birthDate/@value"></path> <sourceId value="F1"></sourceId> </variable>

Again, test engines will not evaluate the path expression at this point. They will look for anything wrapped in '${}' in "operation.params", "operation.requestHeader.value", "operation.url", and "assert.value" element values and substitute the placeholders with the evaluated expressions.

Here is a conditional create operation that will set the requestHeader using the PatientGivenName, PatientFamilyName, and PatientDOB variables defined above. The variable expressions were path expressions against the statically defined F1 fixture. They will be evaluated against the fixture body (containing resource) and the corresponding values will be extracted from the fixtures and used to substitute the variables in the requestHeader value below.

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="create"></code> </type> <requestHeader> <field value="If-None-Exist"></field> <value value="Patient?given=${PatientGivenName}&amp;${PatientFamilyName}&amp;birthdate=${PatientDOB}"></value> </requestHeader> <sourceId value="F1"></sourceId> </operation> </action>

Here is a search operation that will perform a search using the PatientGivenName, PatientFamilyName, and PatientDOB variables defined above. The variable expressions were path expressions against the statically defined F1 fixture. They will be evaluated against the fixture body (containing resource) and the corresponding values will be extracted from the fixtures and used to substitute the variables in the params value below.

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="search"></code> </type> <resource value="Patient"></resource> <accept value="json"></accept> <params value="?given=${PatientGivenName}&amp;family=${PatientFamilyName}&amp;birthdate=${PatientDOB}"></params> <responseId value="R3"></responseId> </operation> </action>

Here are the assertions that verify that the search was successful:

<action> <assert> <path value="Patient/name/given"></path> <value value="${PatientGivenName}"></value> </assert> </action> <action> <assert> <path value="Patient/name/family"></path> <value value="${PatientFamilyName}"></value> </assert> </action> <action> <assert> <path value="Patient/birthdate"></path> <value value="${PatientDOB}"></value> </assert> </action>

All of the previous assert examples illustrate the use of the path element for the evaluation of XPath icon or JsonPath icon formatted expressions. The use of FHIRPath icon expressions is also available and supported via the TestScript.variable.expression, TestScript.setup.action.assert.compareToSourceExpression and TestScript.setup.action.assert.expression elements.

FHIRPath icon expressions evaluate the FHIR resources and data types using the Logical views (or the UML diagrams) rather than the JSON, XML or TTL formats. When used within the supported TestScript elements, the FHIRPath expressions can be evaluated as either a path to a specific value or as a boolean expression against the given FHIR resource.

Using the previous search operation example, here are the 3 asserts re-written using FHIRPath expressions to extract the comparison element values. Note the use of the '[0]' index to extract the first instance of any repeating elements.

<action> <assert> <expression value="Bundle.entry[0].resource.name[0].given[0]"></expression> <value value="${PatientGivenName}"></value> </assert> </action> <action> <assert> <expression value="Bundle.entry[0].resource.name[0].family"></expression> <value value="${PatientFamilyName}"></value> </assert> </action> <action> <assert> <expression value="Bundle.entry[0].resource.birthdate"></expression> <value value="${PatientDOB}"></value> </assert> </action>

Also using the previous search operation example, here is an assert using a FHIRPath expression that evaluates to a boolean testing for a specific number (count) of Bundle.entry instances equal to 1. In this case the assert.value element is not used.

<action> <assert> <expression value="Bundle.entry.count() = 1"></expression> </assert> </action>

As previously stated, test engines will parse the 'body' of the minimumId fixture and verify that each element and its value matches the corresponding element and value in the 'body' of a source fixture. In other words, the source fixture is verified to be a 'superset' of the minimumId fixture. The resource id element in the body will be ignored during comparison. The headers will also be ignored.

The minimumId fixture can be statically defined or it can be either the requestId or responseId from another operation. If sourceId is not specified, then test engine SHALL use the response of the last operation. For example:

<action> <assert> <minimumId value="F1"></minimumId> </assert> </action>

This is a high-level functional description that does not provide sufficient information regarding the test engine requirements of how the detailed comparison is to be performed. In order to provide more clarity a test engine SHALL provide the following structural comparison behaviors of the minimumId fixture and the fixture being evaluated.

  • Attribute or element ordering - If the comparison encounters an order mismatch of an attribute or element between the minimumId fixture and the fixture being evaluated, this does not result in an error as long as the path to the attribute or element matches.
    • This is particularly important when comparing JSON formatted fixtures where attribute order is not significant.
    • For example, the following two fixtures are considered to be matching:
      { name: "hello", title: "goodbye" } { title: "goodbye", name: "hello" }
  • Array value ordering - The order of an array of the same attribute or element is allowed to differ in the comparison; also, additional attribute or element instances are allowed in between the minimumId fixture attribute or element instances such that the index can differ.
    • For example, the following two fixtures are considered equal even though the order of array elements differs:
      min fixture: { names: ["hello", "goodbye"] } compare fixture: { names: ["goodbye", "hello"] }
    • For example, these fixtures would match even though there are extra values:
      min fixture: { names: ["hello", "goodbye"] } compare1 fixture: { names: ["hello", "greeting", "goodbye"] } compare2 fixture: { names: ["greeting", "hello", "goodbye"] } compare3 fixture: { names: ["hello", "goodbye", "greeting"] }
  • Duplicate attributes or elements - When the minimumId fixture has duplicate attributes or elements (in a JSON attribute array or XML repeated element) the fixture being evaluated must have the same number of duplicated attributes or elements.
    • For example, the following two fixtures would not match as the compare fixture does not have the same number of duplicate repeating attributes as the min fixture:
      min fixture: { names: ["hello", "hello"] } compare1 fixture: { names: ["hello"] }
  • Comparison results - The test engine comparison results SHALL include all inconsistencies between the minimumId fixture and the fixture being evaluated.
  • Attribute or element present regardless of value - The ability to allow for the existence of an attribute or element regardless of its value is NOT supported with the use of minimumID due to the FHIR data requirement that an attribute or element value must be present.
    • For example, this minimumId fixture XML element without a value is invalid:
      <novalueelement></novalueelement>
    • Likewise, this concept cannot be expressed in JSON syntax as JSON attributes must always have a value or child attributes.
    • Support for this concept can be defined in separate assert(s) that test for the existence or absence of specific attribute(s) or element(s).


Servers are required to support "_format" in the request url to determine the response mime-type. See Content Type and Encodings

Use the params element to specify the _format:

<action> <operation> <type> <system value="http://hl7.org/fhir/testscript-operation-codes"></system> <code value="search"></code> </type> <resource value="Patient"></resource> <params value="?family=Doe&amp;given=Joe&amp;_format=application/fhir+json"></params> <responseId value="R1"></responseId> </operation> </action>

Use the requestHeader element to verify Content-Type explicitly:

<action> <assert> <headerField value="Content-Type"></headerField> <value value="application/fhir+json"></value> </assert> </action>

This table presents a summary of the constraints applicable to TestScript.setup.action.operation, TestScript.test.action.operation, and TestScript.teardown.action.operation elements. The operation elements should be configured consistently with the FHIR RESTful API summary.

read vread search history create update transaction conformance delete
resource The resource element is required to specify the resource ([type]) in the request URL when params element is used. Will be ignored if targetId or url are specified. In the case of targetId, the resource type will be extracted from the fixture. If targetId is specified, then [type] for request URL will be determined from targetId's fixture and resource element will be ignored. Otherwise, resource type will be extracted from sourceId's fixture if specified. For conditional updates, resource is required. N/A N/A If targetId is specified, then [type] for request URL will be determined from targetId and resource element will be ignored. For conditional deletes, resource is required.
accept The accept element can be used to specify the "Accept" header in the outgoing HTTP request. If "json" is specified, then "Accept" value of "application/fhir+json" will be set in the request header. If "xml" is specified, then "application/fhir+xml" will be used. N/A
contentType The contentType element can be used to specify the "Content-Type" header in the outgoing HTTP request. If "json" is specified, then "Content-Type" value of "application/fhir+json" will be set in the request header. If "xml" is specified, then "application/fhir+xml" will be used. N/A
destination If the TestScript is testing more than one FHIR server simultaneously, the destination identifies which test system the operation will receive this request message using zero-based indexing. The destination value MUST be equal to a defined TestScript.destination".
encodeRequestUrl The encodeRequestUrl element, if "true", can be used to specify that the destination test system will receive this request message URL query parameters in HTML (Percent) encoded UTF-8 format.
origin If the TestScript is testing more than one FHIR server simultaneously, the origin identifies which test system the operation will send this request message using zero-based indexing. The destination value MUST be equal to a defined TestScript.origin".
params The params element can be used to specify the [id] using variable substitutions and the rest of the highlighted portion in the request URL:

[base]/[type]/[id] {?_format=[mime-type]}

If used, then resource is required and targetId and url must not be specified.
The params element can be used to specify the [id] and [vid] using variable substitutions and the rest of the highlighted portion in the request URL:

[base]/[type]/[id]/_history/[vid] {?_format=[mime-type]}

If used, then resource is required and targetId and url must not be specified.
The params element can be used to specify the highlighted portion in the request URL:

[base]/[type]{?[parameters]{&_format=[mime-type]}}

If used, then resource is optional and targetId and url must not be specified.
The params element can be used to specify the [id] using variable substitutions and the rest of the highlighted portion in the following request URLs:
[base]/[type]/[id]/_history{?[parameters]&_format=[mime-type]}
[base]/[type]/_history{?[parameters]&_format=[mime-type]}
[base]/_history{?[parameters]&_format=[mime-type]}

If used, then resource is optional and targetId and url must not be specified.
N/A The params element can be used in conditional update operation to specify the highlighted portion of the request URL:

[base]/[type]?[search parameters]

If used, then resource is required and targetId and url must not be specified.
N/A The params element can be used to specify the highlighted portion in the request URL:

[base]/metadata {?_format=[mime-type]}

If used, then resource is ignored and targetId and url must not be specified.
The params element can be used to specify the [id] using variable substitutions in the request URL:

[base]/[type]/[id]

If used, then resource is required and targetId and url must not be specified.
requestHeader The requestHeader element allows for request headers to be specified explicitly. Test engines will take values specified for requestHeader "as-is" and not transform them. This allows for testing using:
requestId N/A N/A N/A N/A The requestId element can be used to reference the operation request containing response body and headers. If specified, the value can later be used in assertion sourceId to evaluate path (XPath/JSONPath) and headerFields against the request sent for an operation. N/A N/A
responseId The responseId element can be used to reference the operation response containing response body and headers. If specified, the value can later be used in assertion sourceId to evaluate path (XPath/JSONPath) and headerFields against the response received for an operation. N/A
sourceId N/A N/A N/A N/A The sourceId element points to a fixture to be used for the created resource. The fixture cannot contain the id element. The sourceId element points to a fixture to be used for the updated resource. Has to correspond to the responseId of an operation executed upstream in the test script. The response body must contain a resource with a resource id. The sourceId fixture cannot be statically defined because the id cannot be relied upon. Fixture to be used for the transaction. Has to be a Bundle. N/A N/A
targetId The targetId element can be used to specify the [type] and [id] in the request URL.

If used, then params and url must not be specified.

The targetId value has to correspond to the responseId of an operation executed upstream in the test script. The response body must contain a resource with a resource id. The targetId fixture cannot be statically defined because the id cannot be relied upon.
The targetId element can be used to specify the [type], [id], and [vid] in the request URL.

If used, then params and url must not be specified.

The targetId value has to correspond to the responseId of an operation executed upstream in the test script. The response body must contain a resource with a resource id. The targetId fixture cannot be statically defined because the id and vid cannot be relied upon.
The targetId element cannot be used as it's not allowed with params element The targetId element can be used to specify the [type], [id], and [vid] in the request URL.

If used, then params and url must not be specified.

The targetId value has to correspond to the responseId of an operation executed upstream in the test script. The response body must contain a resource with a resource id. The targetId fixture cannot be statically defined because the id and vid cannot be relied upon.
N/A. The [type] for the request URL will be extracted from sourceId. N/A. The [type] for the request URL will be extracted from sourceId. N/A. N/A. The targetId element can be used to specify the [type] and [id] in the request URL.

If used, then params and url must not be specified.

The targetId value has to correspond to the responseId of an operation executed upstream in the test script. The response body must contain a resource with a resource id. The targetId fixture cannot be statically defined because the id cannot be relied upon.
url The url element will contain the full HTTP URL for the operation. This should rarely be used in test scripts. One possible application would be to test if the Location header returned in a response is pointing to an expected resource. See testscript-example-search example.
Assertion Valid operator values Description
compareToSourceExpression equals | notEquals Asserts that the fhirpath compareToSourceExpression against the response body of compareToSourceId fixture evaluates to a value that is equal or notEqual to the evaluated value of the fhirpath expression which must be present also.
The default operator is "equals".
compareToSourcePath equals | notEquals Asserts that compareToSourcePath against the response body of compareToSourceId fixture evaluates to a value that is equal or notEqual to the evaluated value of path which must be present also.
The default operator is "equals".
contentType equals | notEquals | contains | notContains Asserts that the "Content-Type" in response header is or is not the specified value for contentType element depending on the operator used.
The default operator is "contains".
defaultManualCompletion manualEval Manual completion is used to pause the test engine execution and evaluation allowing an external review of the defined assert condition. The defaultManualCompletion defines the default manual completion outcome applied if one of the enumerated values in valueset-assert-manual-completion-codes is not applied.
The default operator is "manualEval".
expression equals | notEquals | in | notIn | greaterThan | lessThan | empty | notEmpty | contains | notContains | eval Asserts that the fhirpath expression against the response body evaluates to a value that contains, not contains, is equal, not equal, in, not in, greater than, or less than the value specified for the value element if present.
If the operator is "empty" or "notEmpty" then value will be ignored.
If the operator is "eval" or not defined then the fhirpath expression will be evaluated as a boolean to determine the pass/fail status.
If sourceId is not specified, then fhirpath expression will be evaluated against the last operation's response body.
The default operator is "eval".
headerField equals | notEquals | in | notIn | greaterThan | lessThan | empty | notEmpty | contains | notContains Asserts that the header specified for headerField element in the response contains, not contains, is equal, not equal, in, not in, greater than, or less than the value specified for value element if present.
If the operator is "empty" or "notEmpty" then value will be ignored.
If sourceId is not specified, then headerField will be evaluated against the last operation's response headers.
The default operator is "equals".
minimumId N/A Asserts that the response contains all the element/content in another fixture pointed to by minimumId element. This can be a statically defined fixture or one that is dynamically set via responseId.
The default operation is not defined and if present will be ignored.
navigationLinks N/A Asserts that the response Bundle contains or does NOT contain first, last, and next links depending on whether or not navigationLinks element is set to true or false.
The default operation is not defined and if present will be ignored.
path equals | notEquals | in | notIn | greaterThan | lessThan | empty | notEmpty | contains | notContains Asserts that the path against the response body evaluates to a value that contains, not contains, is equal, not equal, in, not in, greater than, or less than the value specified for value element if present.
If the operator is "empty" or "notEmpty" then value will be ignored.
If sourceId is not specified, then path will be evaluated against the last operation's response body.
The default operator is "equals".
requestMethod equals | notEquals Asserts that the operation HTTP request method is or is not one of the enumerated values in http-operations.
The default operator is "equals".
requestUrl equals | notEquals | contains | notContains Asserts that the operation HTTP request url path contains, does not contains, is equal, is not equal to the requestUrl value depending on the operator used.
The default operator is "equals".
resource equals | notEquals Asserts that the resource returned in the response body is or is not of the specified value for resource element.
The default operator is "equals".
response equals | notEquals Asserts that status code in the response is or is not one of the enumerated values in response abbreviations.
The default operator is "equals".
responseCode equals | notEquals | in | notIn | greaterThan | lessThan Asserts that status code in the response is equal, notEqual, in, not in, greater than, or less than the specified value(s) for responseCode element.
The default operator is "equals".
validateProfileId N/A Asserts that the response is valid according to the profile specified by validateProfileId element.
The default operation is not defined and if present will be ignored.