HL7 Informative Document: Patient Information Quality Improvement (PIQI) Framework, Edition 1, published by HL7 Cross-Group Projects Work Group. This guide is not an authorized publication; it is the continuous build for version 1.0.0 built by the FHIR (HL7® FHIR® Standard) CI Build. This version is based on the current content of https://github.com/HL7/piqi/ and changes regularly. See the Directory of published versions

PIQI Framework

High Level Component Diagram

Please refer to the appropriate PIQI Message Format Guide for specific PIQI model format implementation details.

The high-level component diagram can be found here

PIQI Message Philosophy

It is important to remember that the goal of the Patient Information Quality Improvement (PIQI) Framework is to assess the contents of a single structured data payload and provide a score based solely on the contents of that payload. Each rubric assessing a payload is designed in the context of a particular use case. This allows the following:

1) The assessment of the quality of that single payload. 2) The collection of statistics with aggregate values (Averages, Medians) and other ‘per payload’ statistics. 3) Allows payload level plausibility assessments.

Implementers of the PIQI Framework will likely aggregate scoring results for the purpose of qualitative root-cause analysis, but this specification is focused on the implementation of the framework itself. Future efforts may consider optimal patterns for analyzing data quality patterns across collections of messages.

PIQI Attribute Types

PIQI Elements are comprised of four attribute types: Simple, Codeable Concept, Observation Value, and Ranged Value. All attributes have a cardinality of 0..1, recognizing that a Codeable Concept collection attribute may contain multiple coding values.

Simple Attribute Type

A Simple Attribute is a string value in the JavaScript Object Notation (JSON) attribute structure. Simple Attributes can contain any string and can be used to represent any single basic data type like text, simple numbers, structured numbers, dates, and date ranges.

Example: In this example the effectiveDateTime attribute is a Simple Attribute.

    "Allergies": [
        {"effectiveDate": "20230111"}
    ]

Codeable Concept Attribute Type

A Codeable Concept attribute is a structure that can be represented as a simple attribute or a coded entity. Many attributes in patient data can be represented by a coded entity or a simple code value and as such is represented by a text sub-attribute and a codings collection.

Many of the important attributes in patient information are Codeable Concepts. Codeable Concepts are attributes that can be represented by a simple text string or by one or many codings, hence the codeable concept. If the attribute does contain one or more codings, each coding is comprised of three components: a code, display, and code system identifier.

All of these Coding sub attributes should be populated in order for the concept being exchanged to be validated.

The code is needed to provide a unique stable identifier for the concept in a given code system.

The code system identifier is needed to provide context for the meaning of a code, and to confirm that the code is valid in that code system and, if necessary, determine the status of that code.

The display is needed so that a human can verify that the code provided does, in fact, semantically match the concept in the identified code system. It is also necessary if the concept provided is being mapped/normalized to a standard or local terminology.

Codeable Concept sub-attributes

Coding Attributes

Codings exist as a substructure to the CodeableConcept type. The substructure is adopted from Fast Healthcare Interoperability Resources (FHIR) to express one or more codings associated with a given CodeableConcept.

Coding sub-attributes

Example:

    "performedTest": {
        "text": "MONOCYTES:NCNC:PT:BLD:QN:AUTOMATED COUNT",
        "codings": [
            {
                "code": "742-7",
                "display": "MONOCYTES:NCNC:PT:BLD:QN:AUTOMATED COUNT",
                "system": "LOINC"
            }
        ]
    }

Observation Value Attribute Type

An Observation Value attribute is a structure that can be represented as a simple attribute value or a structure observation value. Observation Value is a specialized attribute as many elements in patient information can be represented by a quantitative, text, or coding value.

Observation Value sub-attributes

Example of ResultValue Text Only:

    "resultValue": {
        "text": "NEG",
        "type": {
            "text": "ST"
        }
    }

Example of ResultValue Coding Sub-Attribute:

    "resultValue": {
        "text": "Negative",
        "type": {
            "text": "CE",
            "codings": [
                {
                    "system": "HL7-0125",
                    "code": "CE",
                    "display": "Coded element"
                }
            ]
        },
        "codings": [
            {
                "system": "SNOMEDCT",
                "code": "260385009",
                "display": "Negative"
            }
        ]
    }

Example of ResultValue Number Value Sub-Attribute:

    "resultValue": {
        "text": "100",
        "type": {
            "text": "NM",
            "codings": [
                {
                    "system": "HL7-0125",
                    "code": "NM",
                    "display": "Numeric"
                }
            ]
        },
        "number": "100"
    }

Example of ResultValue Numeric Range Sub-Attribute:

    "resultValue": {
        "text": "3.5-4.5",
        "type": {
            "text": "NR",
            "codings": [
                {
                    "system": "HL7-0125",
                    "code": "NR",
                    "display": "Numeric range"
                }
            ]
        },
        "number": "3.5",
        "number2": "4.5"
    }

Range Value Attribute Type

A Range Value attribute is a structure that can be represented as a simple attribute value or a structured range value. The Range Value is a specialized attribute as many elements in patient information can be represented by a structured range. The Range Value Attribute Type can only be used with numeric ranges and cannot be used for a date range.

Range Value sub-attributes

Example:

    "referenceRange": {
        "text": "14-88",
        "lowValue": "14",
        "highValue": "88"
    }

PIQI Data Models

The PIQI Framework depends on a patient-oriented, flat-data class collection information model. Depending on the use case, the structure of the model—including its data classes and attribute details—may vary. For example, the PIQI Clinical Data Model is designed to evaluate clinical information exchanged about a patient, so its data classes and attributes reflect common clinical data elements. In contrast, the PIQI Medical Claim Data Model includes data classes and attributes typically associated with claims information. The PIQI Framework functions consistently across different models, provided the model remains flat, includes a patient demographics element, and uses attribute types that align with the framework’s design.

PIQI Clinical Data Model

The PIQI Clinical Data Model is a simplified information model based on US Core profiles and is intended to focus on data classes and attributes that are highly relevant to patient clinical data quality and to simplify the qualitative assessment process. While it could be extended to include administrative and other data relative to the patient's care process, initially the model is purposefully limited to patient demographics, social, and clinical information.

PIQI Clinical Data Model Classes

PIQI Data Class Attributes

Code System Identifiers in PIQI

One of the issues with validating codeable concepts is that often different string values are used to identify the code system for a given code. For example, when sending a Logical Observation Identifiers Names and Codes (LOINC) code it is possible to see the following code system identifiers:

LOINC, LNC, LN Code System Simple string identifiers

CodeSystem Object Identifier (OID): 2.16.840.1.113883.6.1

CodeSystem Uniform Resource Locator (URL): http://loinc.org

To support code system identifier variants in PIQI there is a mechanism that allows for a code system mnemonic to be used in the configuration of an Evaluation Rubric to represent any valid code system identifier and identify the code system identifier that is required to validate that code in the implementation's terminology server.

For example, for LOINC this mechanism would establish the ‘LOINC’ mnemonic in the following manner:

This is another example of how PIQI is designed specifically for patient information. This function will allow, for example, an Evaluation Rubric that is checking for conformance to just pass ‘LOINC’ to the SAM that validated conformity and have it pass if any of the Code System Identifier aliases are used. It also allows a validation SAM to automatically validate a concept that is received using one of the code systems aliases. If at some point in the future another valid code system identifier is defined for a given code system, it can easily be added to this mechanism and automatically reflected in all configured Evaluation Rubrics.

Code System JSON structure

    "**codeSystems**": [
    {
      "name": "Logical Observation Identifiers Names and Codes",
      "mnemonic": "LOINC",
      "codeSystemIdentifiers": [
        "LN",
        "LNC",
        "urn:oid:2.16.840.1.113883.6.1",
        "2.16.840.1.113883.6.1",
        "http://loinc.org"
      ]
    },
    {
      "name": "SNOMED CT US Edition",
      "mnemonic": "SNO_INT_USEXT",
      "codeSystemIdentifiers": [
        "http://snomed.info/sct/731000124108",
        "SNOMED",
        "SCT",
        "SNOMEDCT",
        "urn:oid:2.16.840.1.113883.6.96",
        "http://snomed.info/sct",
        "2.16.840.1.113883.6.96",
        "SNO"
        ]
    },
    {
      "name": "ICD-10-CM",
      "mnemonic": "ICD10CM",
      "codeSystemIdentifiers": [
        "ICD10",
        "ICD10-CM",
        "ICD10CM",
        "2.16.840.1.113883.6.90",
        "urn:oid:2.16.840.1.113883.6.90",
        "http://hl7.org/fhir/sid/icd-10-cm",
        "https://www.cms.gov/medicare/icd-10/2022-icd-10-cm"
      ]
    },
    {
      "name": "ICD-9-CM Diagnosis",
      "mnemonic": "ICD9CMD",
      "codeSystemIdentifiers": [
        "I9",
        "ICD9",
        "ICD9CM",
        "ICD9-CM",
        "2.16.840.1.113883.6.103",
        "urn:oid:2.16.840.1.113883.6.103",
        "http://hl7.org/fhir/sid/icd-9-cm",
      ]
    }
    ]

PIQI Patient Assessment Application Hierarchy

When considering a PIQI information model it is worth noting that patient data in such models are organized into a hierarchical structure with four primary entity levels that can each be evaluated for quality. These entities are as follows:

• Data Model: A holistic entity that is comprised of all of the data class instances and attributes - the entire structured data submission.
• Data Class: A collection of instances of a data class.
• Element: A single instance of a data class collection of attributes.
• Attribute: A single data attribute that relates to a data class.

Reminder: It is important to restate that PIQI is intended to assess the quality of data payloads in real time, determining their suitability for use by the intended recipient, and not intended to evaluate data within records at rest.

PIQI Healthcare Data Quality Taxonomy (HDQT)

A standard classification taxonomy for patient information quality issues provides structure and clarity to the process of data quality management. This systematic approach allows for the identification and categorization of various types of errors or inconsistencies, which can range from minor inaccuracies to critical misinformation. By classifying these issues, healthcare data stewards can prioritize them based on their potential impact. This methodical categorization facilitates easier tracking and analysis of recurring problems, enabling healthcare organizations to implement targeted improvements. Moreover, standardized taxonomy aids communication among healthcare professionals by providing a common language for discussing quality issues. Ultimately, this leads to enhanced quality control, improved patient outcomes, and a reduction in the likelihood of medical errors, thereby fostering a safer and more reliable healthcare environment.

The PIQI Taxonomy consists of high-level categories, each containing more specific dimensions. It's essential to note that the assessment of patient information quality occurs at both the element and attribute levels, and not all dimensions apply uniformly to both.

Given that the dimensions are designed to categorize qualitative issues, they are presented from a negative perspective. Consequently, the percentages associated with these dimensions indicate the proportion of failures or issues identified within each dimension.

It is important to remember that these dimensions are available to be used by the SAMs which provide code assessments of the input to the SAM, be that an attribute, data class element, data class collection or the entire message. This means a HDQT is an abstract concept and its assignment to the SAM is based on a best-fit interpretation of the assessments intent.

The categories are as follows:

Availability Category

This category contains dimensions that relate to the presence of usable information.

Dimensions:

Accuracy Category

This category contains dimensions that relate to the innate validity of the data.

Dimensions:

Conformity Category

This category contains dimensions that relate to the integrity of coded information, for example verifying that a specific code is a member of a CodeSystem or Valueset.

Plausibility Category

This category contains dimensions that relate to whether the data makes sense based on some context.

PIQI Assessment Approach

The PIQI Framework assesses patient data that has been put into the PIQI Data Model using an Evaluation Rubric which is a collection of Evaluations, each evaluation consisting of a patient data entity to be evaluated, a configured Simple Assessment Module (SAM) to use in that evaluation, specific parameter values to use with that SAM, and scoring implications of the evaluation result.

PIQI Simple Assessment Modules

A PIQI SAM is a logical test that measures a specific data quality dimension for a model element or characteristic, following a defined pattern. The inputs depend on the pattern, but the output is always a simple result: pass, fail, or skip. SAMs can be grouped into hierarchical collections, allowing multiple checks to be combined in order to provide a broader picture of data quality for a given use case. A SAM that has been configured for use in an Evaluation Rubric is referred to simply as an Evaluation.

Each SAM is comprised of the following elements:

Example: High Level

Example: Attribute is Populated

Example: Coded Entity is Compatible

A SAM can have a prerequisite SAM. This is a SAM that must be run and pass prior to this SAM being run. Prerequisites can stack resulting in a series of SAMs that must be run on an entity for the assigned Evaluation SAM to be run and pass.

Prerequisite SAMs cannot have parameters that are not provided to the assigned Evaluation SAM.

One might be tempted to avoid prerequisite SAMs by encapsulating the prerequisite logic in the assigned Evaluation SAM. The reason PIQI does not do this is the information on the failure of the entity being evaluated is primary to the definition of the SAM and the SAM itself just returns a pass or fail. In the above example when the assigned assessment ‘Concept is Compatible’ is run the first prerequisite SAM, ‘Attribute is Populated’ is run on the entity. The idea that the entity fails to pass the assigned SAM is what establishes the entity's score. Recording the specific SAM that failed provides the statistics that are core to the PIQI approach. This conveys the nature of the failure, as opposed to ‘Concept is Compatible’ failing and having to return a reason for the failure. Additionally, when a pre-requisite SAM fails, that failure should be categorized against the dimension for the pre-requisite SAM, not the dimension for the assigned Evaluation SAM.

For more information on SAMs, please see Simple Assessment Modules SAMS

PIQI Evaluation Rubric

An Evaluation Rubric is always based on a given PIQI model and version. An Evaluation Rubric based on a given model should be compatible with any future version of the same model as PIQI models are backward compatible.

An Evaluation Rubric is the alignment of patient data entities to SAMs as a list of assessable items that contribute to provide a standard scoring rubric for that specific collection of patient data. Evaluation Rubrics can be established as a standard or custom scoring rubric.

A given Evaluation Rubric has a name, version and purpose, a PIQI model and model version along with a sequenced collection of evaluations comprised of SAMs configured and assigned to the model, data classes, elements and attributes with the necessary parameters, conditions, scoring effect, weight, and criticality.

Example: Graphical Depiction of an Evaluation Rubric for a data class

Evaluation Rubric Assessments

The core of an Evaluation Rubric is the sequenced collection of entity assessments. Each step has the following components:

Scoring Effect

An assigned Evaluation SAM is typically used to score the quality of the data in a message but can also be used to collect data without impacting the quality score. The Scoring Effect is essentially a switch that tells the PIQI engine to collect statistics for a given SAM without affecting the numerator or denominator of the score for the message.

Weight

In some cases, a PIQI user may want to weigh the score in a rubric allowing some SAMs to have more impact than others. By default, the weight for any assigned Evaluation SAM is one. If a particular Evaluation Rubric does use a weighted score for one or more SAMs the PIQI engine will produce a PIQI Score and a PIQI Weighted Score for the message being assessed. Both scores will be a percentage of a numerator over a denominator with a maximum score of 100% of all possible points. This practically means that a PIQI score for a message is simply the number of scored evaluations that passed, divided by the total number of scored evaluations performed, and a PIQI Weighted Score for a message is the sum of weighted values for all scored evaluations that passed, divided by the sum of weighted value for all scored evaluations, including both those that passed and failed.

Criticality

The criticality indicator on an assigned Evaluation SAM allows the user to flag that assessment as critically important to the rubric. As an alternative to a weighted score the criticality indicator provides the ability to indicate that message has critical qualitative issues that may make the data unusable without affecting the nature of the unweighted numerator/denominator percentage.

Conditional Assessments

In some cases, an assigned Evaluation SAM should only be applied if another condition is true. While this may seem similar to a prerequisite SAM it affects the scoring in a very different way.

A prerequisite SAM is part of a SAM definition. If a prerequisite SAM for an assigned Evaluation SAM fails, this is considered an entity assessment failure which increments the denominator by 1 but not the numerator for that assessment. In other words, the assessment is a failure. For example, the Attr_IsPopulated simple SAM is commonly a prerequisite for downstream SAMS such Attr_IsDate.

A conditional SAM is part of an Evaluation Rubric. If a conditional SAM for an assigned Evaluation SAM fails, this means the conditions for the assigned Evaluation SAM were not met and the assigned Evaluation SAM should not be run. Therefore it is neither a pass nor a fail. Accordingly, neither the numerator nor the denominator is incremented. For example, the Observation Value _IsQualitative SAM is a conditional SAM that must pass in order for a downstream SAMs that require a qualitative result to be run.

Evaluation Rubric Execution

The sequence of execution of the Evaluation Rubric Assessments controls the order of assessment within a given data class. The data class execution sequence is defined as the sequence of patient data loaded into the PIQI model. The sequence of assessment will repeat within a given data class until it has completed all the elements of that data class. Upon completion of the assessment of all the elements in a given data class, the data class assessments themselves (if defined) are executed. Upon completion of all elements and data class assessments, model scope assessments (if defined) are executed.

PIQI Scoring in a Nutshell

The objective for PIQI data quality scoring is to provide a useful score card that provides the data consumer with insight into the quality of a patient data payload based on a given Evaluation Rubric.

The most basic result returned from a PIQI scorecard evaluation would be the following:

Verbose Scoring

When the PIQI Framework is used in verbose scoring mode, the score information above is returned along with the original message with each element including the scoring details.

PIQI Technical Components

The PIQI Framework is actualized with the following components:

Structured Data Architecture Components

• HDQT Taxonomy – structured data
• PIQI Model Structure – structured data
• Standard Code System Mappings – structured data
• Simple Assessment Module Reference Library – structured data
• Evaluation Rubric Reference Library – structured data

Structured Data Asset Components

• Evaluation Rubric Definition(s) – structured data
• Evaluations - configured SAMs
• Simple Assessment Module – Evaluation Definition(s) – structured data
• SAM Specific evaluation content – structured data

Software Components

• PIQI Framework Scoring Engine RESTful Service
• FHIR (or equivalent) Terminology Server

PIQI Alliance as a Community of Practice

The PIQI Alliance serves as the community of practice to support the implementation and collaborative evolution of the PIQI Framework. The Alliance aspires to bring together stakeholders—such as healthcare organizations, vendors, and standards bodies—who share a common interest in improving patient data quality. Through encapsulated, portable, data-driven components and the ability to share knowledge and assets, the PIQI Alliance has the potential to accelerate the adoption and ongoing refinement of the framework.

PIQI Model Extensions

The PIQI Framework starts with a simple model intended to support the assessment of core clinical patient data. The simplicity of this model allows for extension through the addition of attributes and data classes. Ideally these extensions would be adopted into the core PIQI model through a standard process.

Shared Simple Assessment Modules

While many of the foundational, attribute level SAMs are algorithmic, SAMs that apply to codeable concepts, elements, data classes, and model level data often require structured content in the form of value sets, tuples, and other data patterns. To share and utilize rubrics that use SAMs that depend on such structures, PIQI evaluation engines must also have access to the supporting content. For example, to utilize a rubric including a SAM configured to verify that a codeable concept is an active LOINC term, that PIQI evaluation engine must be able to reference the current LOINC release. These data packages can be openly shared or licensed in a community portal. More sophisticated algorithmic SAMs, perhaps based on generative artificial intelligence (AI), could also be hosted as RESTful services through wrapper SAM interfaces.

Shared Evaluation Rubrics

While the Evaluation Rubrics in PIQI are designed to be configured to support the needs of the implementer, the ability to establish sanctioned standard Evaluation Rubrics is one of the most beneficial features of the PIQI Framework. The ability to publish and share an Evaluation Rubric, along with its dependent SAMs and PIQI Model components can minimize duplication of effort and provide a powerful way to create a common basis for understanding data quality across the community.

PIQI Endpoint Registration

The PIQI Alliance envisions that implementations of the PIQI Framework could be registered and certified within the community, with the goal of enabling participants to seamlessly share content—such as SAMs, Evaluation Rubrics, and PIQI Models—with other community members.

PIQI Data Source Unique Identifier

The PIQI Framework was created to assess and improve the quality of patient data from a given source. However, this raises a crucial question: How do we uniquely and unambiguously identify the source of that data?

In healthcare, we have various identifier types for institutions, facilities, and providers, but no unique identifier for the data entity responsible for generating and sharing patient data. This is analogous to an IP address in a computer network—an essential component that enables modern data-sharing networks to recognize and authenticate each participant.

This is not merely a "nice to have" feature. Establishing a unique data source identifier is critical for trust, traceability, and provenance in our national healthcare infrastructure.

While the PIQI Framework does not require a global unique data source identifier, nor does it include a methodology for assigning one, incorporating one would significantly enhance the ability to assess the quality and reliability of data exchange participants—regardless of their role or the nature of the coordinating entity.