HL7 Cross-Paradigm Implementation Guide: Patient Information Quality Improvement (PIQI) Framework, Edition 1, published by HL7 Cross-Group Projects Work Group. This guide is not an authorized publication; it is the continuous build for version 1.0.0-ballot built by the FHIR (HL7® FHIR® Standard) CI Build. This version is based on the current content of https://github.com/HL7/piqiimplementation/ and changes regularly. See the Directory of published versions

PIQI Framework

High Level Component Diagram

Please refer to the appropriate PIQI Message Format Guide for specific PIQI model format implementation details.

PIQI Message Philosophy

It is important to remember that the goal of the PIQI Framework is to assess the contents of a single structured data payload and provide a score based solely on the contents of that payload. Each rubric assessing a payload is designed in the context of a particular use case. This allows the following:

1) The assessment of the quality of that single payload. 2) The collection of statistics with aggregate values (Averages, Medians) and other ‘per payload’ statistics. 3) Allows payload level plausibility assessments.

PIQI Attribute Types

PIQI Elements are comprised of four attribute types: Simple, Codeable Concept, Observation Value, and Ranged Value.

Simple Attribute Type

A Simple Attribute is a string value in the JSON attribute structure. Simple Attributes can contain any string and can be used to represent any single basic data type like text, simple numbers, structured numbers, dates, and date ranges.

Example: In this example the effectiveDateTime attribute is a Simple Attribute.

    "labElements": [
        {"effectiveDateTime": "20230111135518"}
    ]

Codeable Concept Attribute Type

A Codeable Concept attribute is a structure that can be represented as a simple attribute or a coded entity. Many attributes in patient data can be represented by a coded entity or a simple code value and as such is represented by a text sub-attribute and a codings collection.

Many of the important attributes in patient information are Codeable Concepts. Codeable Concepts are attributes that can be represented by a simple text string or by one or many codings, hence the codeable concept. If the attribute does contain one or more coded concepts, each coded concept is comprised of three components: a code, display, and code system identifier.

All of these Coding sub attributes should be populated in order for the concept being exchanged to be validated.

The code is needed to provide a unique stable identifier for the concept in a given code system.

The code system identifier is needed to confirm that the code is valid in that code system and, if necessary, determine the status of that code.

The display is needed so that a human can verify that the code provided does, in fact, semantically match the concept in the identified code system. It is also necessary if the concept provided is being mapped/normalized to a standard or local terminology.

Codeable Concept sub-attributes

Coding sub-attributes

Example:

    "test": {
        "text": "MONOCYTES:NCNC:PT:BLD:QN:AUTOMATED COUNT",
        "codings": [
            {
                "code": "742-7",
                "display": "MONOCYTES:NCNC:PT:BLD:QN:AUTOMATED COUNT",
                "system": "LOINC"
            }
        ]
    }

Observation Value Attribute Type

An Observation Value attribute is a structure that can be represented as a simple attribute value or a structure observation value. Observation Value is a specialized attribute as many elements in patient information can be represented by a quantitative, text, or coded concepts value.

Observation Value sub-attributes

Example:

    "resultValue": {
        "text": "Abnormal",
        "type": {
            "text": "CE",
            "codings": [
                {
                    "system": "HL7-0125",
                    "code": "CE",
                    "display": "Coded Entry"
                },
                {
                    "system": "LOINC",
                    "code": "11111",
                    "display": "Abnormal"
                }
            ]
        }
    }

Range Value Attribute Type

A Range Value attribute is a structure that can be represented as a simple attribute value or a structure ranged value. The Range Value is a specialized attribute as many elements in patient information can be represented by a structured range. The Range Value Attribute Type can only be used with numeric ranges and cannot be used for a date range.

Range Value sub-attributes

Example:

    "referenceRange": {
        "text": "14-88",
        "lowValue": "14",
        "highValue": "88"
    }

PIQI Data Models

The PIQI Framework depends on a patient-oriented, flat-element collection information model. Depending on the use case, the structure of the model—including its data classes and element details—may vary. For example, the PIQI Clinical Data Model is designed to evaluate clinical information exchanged about a patient, so its data classes reflect common clinical data elements. In contrast, the PIQI Medical Claim Data Model includes data classes and attributes typically associated with claims information. The PIQI Framework functions consistently across different models, provided the model remains flat, includes a patient demographics element, and uses attribute types that align with the framework’s design.

All attributes have a cardinality of 0..1, recognizing that a CodeableConcept collection attribute may contain multiple CodeableConcepts values.

PIQI Clinical Data Model

The PIQI Clinical Data Model is a simplified information model based on US Core profiles and is intended to focus on elements and characteristics that are highly relevant to patient clinical data quality and to simplify the qualitative assessment process. While it could be extended to include administrative and other data relative to the patient's care process, initially the model is purposefully limited to patient demographics, social, and clinical information.

PIQI Clinical Data Model Classes

PIQI Data Class Attributes

Code System Identifiers in PIQI

One of the issues with validating codeable concepts is that often different string values are used to identify the code system for a given code. For example, when sending a LOINC code it is possible to see the following code system identifiers:

LOINC, LNC, LN Code System Simple string identifiers

CodeSystem OID: 2.16.840.1.113883.6.1

CodeSystem URL http://loinc.org

To support code system identifier variants in PIQI there is a mechanism that allows for a code system mnemonic to be used in the configuration of an evaluation profile to represent any valid code system identifier and identify the code system identifier that is required to validate that code in the implementation's terminology server.

For example, for LOINC this mechanism would establish the ‘LOINC’ mnemonic in the following manner:

This is another example of how PIQI is designed specifically for patient information. This function will allow, for example, an evaluation profile that is checking for conformance to just pass ‘LOINC’ to the SAM that validated conformity and have it pass if any of the Code System Identifier aliases are used. It also allows a validation SAM to automatically validate a concept that is received using one of the code systems aliases. If at some point in the future another valid code system identifier is defined for a given code system, it can easily be added to this mechanism and automatically reflected in all configured evaluation profiles.

Code System JSON structure

    "**codeSytems**": [
        {
            "mnemonic": "LOINC",
            "name": "Logical Observation Identifiers Names and Codes",
            "identifier": "<http://loinc.org>",
            "aliases": [
                "LOINC",
                "LOINC Code",
                "<http://loinc.org>",
                "urn:oid:2.16.840.1.113883.6.1"
            ],
            "terminologyServerStatus": "active"
        },
        {
            "mnemonic": "SNOMEDCT",
            "name": "SNOMED Clinical Terms",
            "identifier": "<http://snomed.info/sct>",
            "aliases": [
                "SNOMED CT",
                "SNOMEDCT",
                "sct",
                "urn:oid:2.16.840.1.113883.6.96"
            ],
            "terminologyServerStatus": "active"
        },
        {
            "mnemonic": "ICD10CM",
            "name": "International Classification of Diseases, Tenth Revision, Clinical Modification",
            "identifier": "<http://hl7.org/fhir/sid/icd-10-cm>",
            "aliases": [
                "ICD-10-CM",
                "ICD10CM",
                "urn:oid:2.16.840.1.113883.6.90"
            ],
            "terminologyServerStatus": "active"
        },
        {
            "mnemonic": "ICD9CM",
            "name": "International Classification of Diseases, Ninth Revision, Clinical Modification",
            "identifier": "<http://hl7.org/fhir/sid/icd-9-cm>",
            "aliases": [
                "ICD-9-CM",
                "ICD9CM",
                "urn:oid:2.16.840.1.113883.6.42"
            ],
            "terminologyServerStatus": "deprecated"
        }
    ]

PIQI Patient Assessment Application Hierarchy

When considering a PIQI information model it is worth nothing that patient data in this model organizes itself into a hierarchical structure with four primary entity levels that can each be evaluated for quality. These entities are as follows:

• Patient: A holistic entity that is comprised of all of the characteristics and elements.
• Data Class: A collection of data elements with the same attributes representing domain specific items.
• Element: A class specific entity that is comprised of a collection of data attributes.
• Element Attribute: A single data characteristic that relates to a domain specific element.

Reminder: It is important to restate that PIQI is intended to assess the quality of data payloads in real time, determining their suitability for use by the intended recipient, and not intended to evaluate data within records at rest.

PIQI Healthcare Data Quality Taxonomy Version 1.0

A standard classification taxonomy for patient information quality issues provides structure and clarity to the process of quality management. This systematic approach allows for the identification and categorization of various types of errors or inconsistencies, which can range from minor inaccuracies to critical misinformation. By classifying these issues, healthcare data stewards can prioritize them based on their potential impact. This methodical categorization facilitates easier tracking and analysis of recurring problems, enabling healthcare organizations to implement targeted improvements. Moreover, standardized taxonomy aids communication among healthcare professionals by providing a common language for discussing quality issues. Ultimately, this leads to enhanced quality control, improved patient outcomes, and a reduction in the likelihood of medical errors, thereby fostering a safer and more reliable healthcare environment.

The PIQI Taxonomy consists of high-level categories, each containing more specific dimensions. It's essential to note that the assessment of patient information quality occurs at both the element and attribute levels, and not all dimensions apply uniformly to both.

Given that the dimensions are designed to categorize qualitative issues, they are presented from a negative perspective. Consequently, the percentages associated with these dimensions indicate the proportion of failures or issues identified within each dimension.

The categories are as follows:

Availability Category

This category contains dimensions that relate to the presence of usable information.

Dimensions:

Accuracy Category

This category contains dimensions that relate to the innate validity of the data.

Dimensions:

Conformity Category

This category contains dimensions that relate to the integrity of coded information, for example verifying that specific code is a member of a CodeSystem or Valueset.

Plausibility Category

This category contains dimensions that relate to whether the data makes sense based on some context.

PIQI Assessment Approach

The PIQI Framework assess patient data that has been put into the PIQI Data Model using an Evaluation Rubric which is a collection of Evaluations, each evaluation has a configured Simple Assessment Module (SAM) with specific parameters and their individual scoring effects of the evaluation result for each patient data entity.

PIQI Simple Assessment Modules

PIQI SAMs are composable service endpoints that follow a consistent pattern to evaluate a patient message, data class, element or attribute, and return a simple ‘pass’, ‘fail’ or ‘conditions not met’ result. A SAM that has been configured for use in a Evaluation Rubric is referred to simply as an Evaluation.

Each SAM is comprised of the following elements:

Example: High Level

Example: Attribute is Present

Example: Coded Entity is Interoperable

A SAM can have a prerequisite SAM. This is a SAM that must be run and pass prior to this SAM being run. Prerequisites can stack resulting in a series of SAMs that must be run on an entity before in order for the assigned SAM to be run and pass.

Prerequisite SAMs cannot have parameters that are not provided to the assigned SAM.

One might be tempted to avoid prerequisite SAMs by encapsulating the prerequisite logic in the assigned SAM. The reason PIQI does not do this is the information on the failure of the entity being evaluated in primary to the definition of the SAM and the SAM itself just returns a pass or fail. In the above example when the assigned assessment ‘Concept is Compatible’ is run the first prerequisite SAM, ‘Attribute is Populated’ is run on the entity. If it fails, the ‘Attribute is Populated’ fails and not the ‘Concept is Compatible’. This conveys the nature of the failure, as opposed to ‘Concept is Compatible’ failing and having to return a reason for the failure. Additionally, when a pre-requisite SAM fails, that failure should be categorized against the dimension for the pre-requisite SAM, not the dimension for the assigned SAM.

The idea that the entity fails to pass the assigned SAM is what establishes the entity’s score, but the SAM or prerequisite SAM failure provides the statistics is a core principle of the PIQI approach.

PIQI Evaluation Rubric

An evaluation rubric is always based on a given PIQI model and version. A evaluation rubric based on a given model should be compatible with any future version of the same model as PIQI models are backwards compatible.

An evaluation rubric is the alignment of patient data entities to SAMs as a list of assessable items that contribute to provide a standard scoring rubric for that profile. Evaluation rubrics can be established as a standard or custom scoring rubric.

A given evaluation rubric has a name, version and purpose, a PIQI model and model version along with a sequenced collection of evaluations comprised of SAMs configured and assigned to the patient, data classes, elements and attributes with the necessary parameters, conditions, scoring effect, weight, and criticality.

Example: Graphical Depiction of an Evaluation profile for a data class

Evaluation Rubric Assessments

The core of an Evaluation Criteria is the sequenced collection of entity assessments. Each step has the following components:

Scoring Effect

An assigned SAM is typically used to score the quality of the data in a message but can also be used to collect data without impacting the quality score. The Scoring Effect is essentially a switch that tells the PIQI engine to collect statistics for a given SAM without effecting the numerator or denominator of the score for the message.

Weight

In some cases, a PIQI user may want to weigh the score in a rubric allowing some SAMs to have more impact than others. By default, the weight for any assigned SAM is one. If a particular evaluation profile does use a weighted score for one or more SAMs the PIQI engine will produce a PIQI Score and a PIQI Weighted Score for the message being assessed. Both scores will be a percentage of a numerator over a denominator with a maximum score of 100% of all possible points, weighted or unweighted.

Criticality

The criticality indicator on an assigned SAM allows the user to flag that assessment as critically important to the rubric. As an alternative to a weighted score the criticality indicator provides the ability to indicate that message has critical qualitative issues that may make the data unusable without effecting nature of the unweighted numerator/denominator percentage.

Conditional Assessments

In some cases, an assigned SAM should only be applied if another condition is true. While this may seem similar to a prerequisite SAM it affects the scoring in a very different way.

If a prerequisite SAM for an assigned SAM fails, this is considered an entity assessment failure which increments the denominator but not the numerator for that assessment. In other words, the assessment is a failure.

If a conditional SAM for an assigned SAM fails, this means the conditions for the assigned SAM were not met and the assigned SAM should not be run. Neither the numerator nor the denominator is incremented.

Evaluation Criteria Execution

The sequence of execution of the Evaluation Criteria Assessments controls the order of assessment within a given data class. The data class execution sequence is defined as the sequence of patient data loaded into the PIQI model. The sequence of assessment will repeat within a given data class until it has completed all the elements of that data class. Upon completion of the assessment of all the elements in a given data class, the data class assessments themselves (if defined) are executed. Upon completion of all elements and data class assessments, patient scope assessments (if defined) are executed.

PIQI Scoring in a Nutshell

The objective for PIQI data quality scoring is to provide a useful score card that provides the data consumer with insight into the quality of a patient data payload based on a given evaluation profile.

The most basic result returned from a PIQI scorecard evaluation would be the following:

Verbose Scoring

When the PIQI Framework is used in verbose scoring mode, the score information above is returned along with the original message with each element including the scoring details.

PIQI Technical Components

The PIQI Framework is actualized with the following components:

Structured Data Architecture Components

• HDQT Taxonomy – structured data
• PIQI Model Structure – structured data
• Standard Code System Mappings – structured data
• Simple Assessment Module Reference Library – structured data
• Evaluation Rubric Reference Library – structured data

Structured Data Asset Components

• Evaluation Rubric Definition(s) – structured data
• Evaluations - configured SAMs
• Simple Assessment Module – Evaluation Definition(s) – structured data
• SAM Specific evaluation content – structured data

Software Components

• PIQI Framework Scoring Engine RESTful Service
• FHIR (or equivalent) Terminology Server

PIQI Alliance as a Community of Practice

The PIQI Framework is designed to be implemented within a community of practice. With encapsulated, portable, content driven components, the ability to share knowledge, and evolve establishes great potential for rapid evolution and adoption.

PIQI Model Extensions

The PIQI Framework starts with a simple model intended to support the assessment of core clinical patient data. The simplicity of this model allows for extension through the addition of attributes and data classes. Ideally these extensions would be adopted into the core PIQI model through a standard process.

Shared Simple Assessment Modules

While many of the foundational, attribute level SAMs are algorithmic, SAMs that apply to codeable concepts, elements, data classes, and patient level data often require structured content in the form of value sets, tuples, and other data patterns. These data packages can be openly shared or licensed in a community portal. More sophisticated algorithmic SAMs, perhaps based on generative AI, could also be hosted as RESTful services through wrapper SAM interfaces.

Shared Evaluation Rubrics

While the Evaluation Rubrics in PIQI are designed to be configured to support the needs of the implementer, the ability to establish sanctioned standard evaluation profiles is one of the most beneficial features of the PIQI Framework. The ability to publish and share an evaluation profile, along with its dependent SAMs and PIQI Model components can minimize duplication of effort and provide a powerful way to create a common basis for understanding data quality across the community.

PIQI Endpoint Registration

Within the community of practice implementations of the PIQI Framework could be registered and certified to facilitate the ability to share content seamlessly with other community members.

PIQI Data Source Unique Identifier

The PIQI Framework was created to assess and improve the quality of patient data from a given source. However, this raises a crucial question: How do we uniquely and ubiquitously identify the source of that data?

In healthcare, we have various identifier types for institutions, facilities, and providers, but no unique identifier for the data entity responsible for generating and sharing patient data. This is analogous to an IP address in a computer network—an essential component that enables modern data-sharing networks to recognize and authenticate each participant.

This is not merely a "nice to have" feature. Establishing a unique data source identifier is critical for trust, traceability, and provenance in our national healthcare infrastructure.

While the PIQI Framework does not mandate a global unique data source identifier, incorporating one would significantly enhance our ability to assess the quality and reliability of data exchange participants—regardless of their role or the nature of the coordinating entity.