HL7 Implementation Guide: Patient Information Quality Improvement (PIQI) Framework, Edition 2, published by HL7 Cross-Group Projects Work Group. This guide is not an authorized publication; it is the continuous build for version 2.0.0-ballot built by the FHIR (HL7® FHIR® Standard) CI Build. This version is based on the current content of https://github.com/HL7/piqi/ and changes regularly. See the Directory of published versions

Conformance

This is a new page developed for the 2.0-ballot of the PIQI Implementation Guide as a standard for trial use (STU). As a new section, this content may change and considered a working draft until community review and feedback have been integrated.

PIQI Framework Scoring Endpoint — Conformance Guide

Conformance verbs in this guide follow RFC 2119 usage (MUST, SHOULD, MAY), consistent with HL7 conformance conventions.

1. Purpose and Scope

This guide defines the conformance requirements for a PIQI Framework Scoring Endpoint — a service capable of:

1. Providing access to PIQI Models (the structured definitions of data classes, entities, and scoring logic)
2. Providing access to Rubrics (the scoring criteria, weighting, and SAM-based evaluation logic applied against a PIQI Model)
3. Scoring messages — evaluating a submitted message against a rubric and returning a structured PIQI score
4. Auditing messages — optionally returning a full attribute-level evidence trail alongside the score

This guide assumes a RESTful, JSON-based API as the reference implementation. Implementers using other transports MUST preserve the semantics defined here even if syntax differs.

Out of scope: the internal algorithms used within individual SAMs, and the specific clinical terminology bindings (SNOMED CT, LOINC, RxNorm) used within a given PIQI Model — these are model-author concerns, not endpoint-conformance concerns.

2. Core Resource Model

A conformant endpoint works with three core resources:

2.1 Relationships

PIQIModel (1) ──< Rubric (many)
Rubric (1) ──< Message evaluations (many) ──> Scoring Result

A Rubric MUST declare exactly one bound PIQIModel. A scoring request MUST declare both a rubric and the model the message conforms to. The model in the request MUST match the model bound to the rubric.

3. Conformance Requirements Summary

A conformant PIQI endpoint MUST accept a message and a rubric reference, evaluate the message against the rubric's SAM-based criteria, and return a scoring result in the defined PIQI format — containing at minimum a numerator, denominator, and computed score at the message level and per data class, in both weighted and unweighted form.

All other capabilities described in this guide represent what a good or production-grade implementation looks like.

4. Endpoint Definitions

4.1 Import a PIQI Model

POST /piqi/models
Content-Type: application/json

Request body — JSON reference data for a PIQIModel

Responses

4.2 Import a Rubric

POST /piqi/rubrics
Content-Type: application/json

Request body — JSON reference data for a Rubric

Responses

4.3 Score a Message

A conformant endpoint MUST provide a scoring endpoint and SHOULD provide a separate scoring-with-audit endpoint.

4.3.1 Score Message

POST /PIQI/ScoreMessage
Content-Type: application/json

Evaluates a message against a rubric and returns a summary scoring result.

Request body:

{
  "PIQIModelMnemonic": "PATIENT_V1",
  "EvaluationRubricMnemonic": "MY_RUBRIC",
  "MessageData": "{\"patient\":{\"demographics\":{\"birthSex\":{\"codings\":[{\"system\":\"http://hl7.org/fhir/administrative-gender\",\"code\":\"male\",\"display\":\"male\"}],\"text\":\"male\"},\"birthDate\":\"2009-01-01\",\"ethnicity\":{\"codings\":[{\"system\":\"urn:oid:2.16.840.1.113883.6.238\",\"code\":\"2135-2\",\"display\":\"Hispanic or Latino\"}],\"text\":\"Hispanic or Latino\"}}}}",
  "ContributorID": "contributor-001",
  "DataSourceID": "datasource-001",
  "MessageID": "msg-00042"
}

MessageData is a JSON-serialized string in PIQI message format. The structure of the message is defined by the PIQI model bound to the rubric. See the reference implementation's sample files for complete message examples.

Response body:

{
  "succeeded": true,
  "elapsedTimeInMS": 142,
  "scoringData": {
    "contributorID": "contributor-001",
    "dataSourceID": "datasource-001",
    "messageID": "msg-00042",
    "evaluationRubric": "USCDI 3.1 Aligned Rubric",
    "processDate": "2026-07-23T10:00:00",
    "messageResults": {
      "numerator": 38,
      "denominator": 42,
      "piqiScore": 90,
      "weightedNumerator": 310,
      "weightedDenominator": 350,
      "weightedPIQIScore": 88,
      "criticalFailureCount": 0
    },
    "dataClassResults": [
      {
        "dataClassName": "Medications",
        "instanceCount": 5,
        "numerator": 12,
        "denominator": 14,
        "piqiScore": 85,
        "weightedNumerator": 95,
        "weightedDenominator": 110,
        "weightedPIQIScore": 86,
        "criticalFailureCount": 0
      }
    ],
    "informationalResults": [],
    "plausibilityResults": []
  }
}

Responses:

4.3.2 Score and Audit Message

POST /PIQI/ScoreAuditMessage
Content-Type: application/json

Evaluates a message and returns both a summary scoring result and a full attribute-level audit trace. The request body is identical to ScoreMessage. The response includes the same scoringData plus an additional auditedMessage object.

Additional response field — auditedMessage:

{
  "auditedMessage": {
    "entityModelMnemonic": "PAT_CLINICAL_V1",
    "contributorID": "contributor-001",
    "dataSourceID": "datasource-001",
    "messageID": "msg-00042",
    "audit": {
      "messageNumerator": 38,
      "messageDenominator": 42,
      "messageScore": 90,
      "messageNumeratorWeighted": 310,
      "messageDenominatorWeighted": 350,
      "messageScoreWeighted": 88,
      "messageCriticalFailureCount": 0
    },
    "root": {
      "rootName": "patient",
      "classes": [
        {
          "className": "demographics",
          "elements": [
            {
              "attributes": [
                {
                  "attributeName": "birthDate",
                  "data": { "$type": "tx", "text": "2009-01-01" },
                  "attributeAudit": {
                    "scoringData": { "attributeScore": 100, "attributeScoreWeighted": 100, "attributeCriticalFailureCount": 0, "attributeNumerator": 1, "attributeDenominator": 1 },
                    "assessmentItems": [
                      { "attributeMnemonic": "DEM_DOB", "attributeName": "birthDate", "assessment": "Date of birth is valid past date", "effect": "Scoring", "status": "Pass", "reason": "" }
                    ]
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  }
}

The audit response traces every attribute of every element in the message, showing exactly which assessments passed or failed and why. This provides a complete evidence trail traceable to the field level.

5. Versioning

• PIQIModel and Rubric resources SHOULD carry a MAJOR.MINOR version identifier.
• A scoring request MUST specify the model mnemonic the message conforms to. The engine MUST validate that this matches the model bound to the rubric before evaluation proceeds.
• Breaking changes to a rubric's criteria or SAM logic SHOULD increment at least the MINOR version to preserve traceability of results over time.

6. Error Handling

All error responses MUST conform to a consistent structure:

{
  "error": {
    "code": "MODEL_MISMATCH",
    "message": "PIQIModelMnemonic 'PATIENT_V2' does not match the model bound to rubric 'MY_RUBRIC'"
  }
}

When the error can be traced to a specific field in the request body, path SHOULD be included:

{
  "error": {
    "code": "SAM_NOT_FOUND",
    "message": "No SAM registered for mnemonic 'RXNORM_CODED'",
    "path": "criteria[2].samMnemonic"
  }
}

Servers SHOULD use a stable, documented set of code values so client integrations can branch on error type rather than parsing message text.

7. Security and Access Control

Deployments of a PIQI scoring endpoint SHOULD ensure that access is controlled in a manner appropriate to the sensitivity of the data being evaluated and the regulatory context of the deployment. The specific mechanism — whether authentication, authorization, network controls, or organizational policy — is outside the scope of this conformance guide.

Deployments handling regulated data SHOULD ensure that access to audit-grade outputs is restricted consistent with applicable organizational and regulatory policies.

8. Capability Discovery

GET /piqi/metadata

A conformant endpoint SHOULD expose a capability discovery endpoint that allows consumers to inspect the service before attempting an integration. This mirrors the role a FHIR CapabilityStatement plays in FHIR-based systems.

The response SHOULD include:

9. SAM Architecture

9.1 What is a SAM?

A SAM (Scoring and Assessment Method) is the fundamental unit of scoring logic in PIQI. Each SAM is a discrete, named function that evaluates one aspect of a message element and returns a result of pass, fail, skip, or informational. SAMs are the mechanism by which rubric criteria are executed — every criterion in a rubric references exactly one SAM by its mnemonic.

Each SAM MUST operate as an autonomous unit: given the same inputs it MUST produce the same output, and it MUST NOT depend on the results of other SAMs. The internal implementation of a SAM is outside the scope of this standard.

9.2 SAM Input and Output

Each SAM MUST accept the message element being evaluated along with any parameters defined on the criterion. Parameters are name/value pairs that allow a single SAM to be configured differently per criterion (e.g., a minimum threshold value, a target code system).

Each SAM MUST return one of four outcomes:

9.3 The SAM Registry and Extensibility

A conformant endpoint MUST maintain a registry of available SAMs, keyed by mnemonic. The SAM mnemonic declared on a rubric criterion MUST resolve to a registered SAM at evaluation time. If no matching SAM is found, the criterion MUST be treated as an error rather than silently skipped.

Implementations SHOULD support extending the SAM registry without modifying the core engine, allowing custom scoring logic to be added for domain-specific use cases.

9.4 Scoring Effect Types

Each criterion declares a scoring effect that controls how its result is counted and reported. There are four valid values:

Rubric authors MUST assign the correct scoring effect to each criterion. A criterion intended as advisory that is accidentally set to Scoring will penalize messages and skew results.

9.5 Criticality

Each criterion MAY be flagged as critical. When a scoring criterion marked critical fails, the failure is counted separately as a critical failure in addition to the normal fail count. Critical failures MUST be surfaced at every level of the scoring output so they are visible regardless of which summary level a consumer inspects.

Critical failures are intended to flag failures that are categorically unacceptable regardless of overall score (e.g., a missing patient identifier, an invalid code system reference).

9.6 Conditional Evaluation

Each criterion MAY declare a condition — a secondary SAM that is evaluated first. If the condition is not met, the primary SAM is skipped entirely and the criterion is recorded as skipped rather than failed. This is the mechanism for expressing "only evaluate X if Y is true."

Skipped criteria MUST be excluded from the denominator and MUST NOT affect the score.

10. Reference Data Requirements

10.1 Model and Rubric Availability

A conformant endpoint MUST have all referenced models and rubrics available and validated before accepting evaluation requests. The mechanism for storing and managing models and rubrics — whether files, a database, an API, or another store — is left to the implementer.

10.2 Reference Data Lookups

A conformant endpoint MUST have access to all reference data required by its SAMs at evaluation time. This includes code system definitions, value lists, and any terminology or range data needed to validate message content. The backing store for this data is left to the implementer.

10.3 Message Format

The message submitted for evaluation MUST be interpretable by the PIQI model bound to the rubric. The supported message formats are determined by the model definition and MUST be documented by the implementer so that submitters know what formats are accepted.

10.4 External Service Dependencies

SAMs MAY depend on external services (e.g., a terminology server, a FHIR endpoint) to perform their evaluation. Any such dependencies MUST be available and responsive at evaluation time, and SHOULD be documented as part of the rubric or SAM definition so that operators know what infrastructure is required.

11. Summary Checklist for Implementers

• Models and rubrics are configured and validated before the service accepts requests
• Each rubric criterion references a valid, registered SAM mnemonic
• Scoring, Informational, Plausibility, and Detection effects are correctly assigned to criteria
• Criticality indicators are applied to criteria whose failure is categorically unacceptable
• Conditional SAM logic is used where criteria should only evaluate under specific conditions
• Scoring and Audit response views are both available and consistently derived from the same evaluation
• Scores are reported with raw numerator/denominator counts included
• Weighted scores are reported alongside unweighted scores at message and data class levels
• Plausibility results are reported separately from scoring results (if supported)
• Versioning is maintained across Model → Rubric lineage
• Errors are structured and coded, not just free-text
• Access to the service is controlled at a level appropriate to the deployment environment
• The SAM registry is populated and all SAM mnemonics referenced by loaded rubrics resolve at startup
• A capability discovery endpoint is available describing supported models, rubrics, and SAMs