Consumer Mobile Health Application Functional Framework, Release 2
2.0.1 - CI build International flag

Consumer Mobile Health Application Functional Framework, Release 2, published by HL7 International / Mobile Health. This guide is not an authorized publication; it is the continuous build for version 2.0.1 built by the FHIR (HL7® FHIR® Standard) CI Build. This version is based on the current content of https://github.com/HL7/cmhaff-ig/ and changes regularly. See the Directory of published versions

: AIG.6 Generative AI and AI Agent Safety (Function)

Page standards status: Informative

Raw xml | Download


<Requirements xmlns="http://hl7.org/fhir">
  <id value="CMHAFFR2-AIG.6"/>
  <meta>
    <profile
             value="http://hl7.org/ehrs/uv/cmhaffr2/StructureDefinition/FMFunction"/>
  </meta>
  <text>
    <status value="extensions"/>
    <div xmlns="http://www.w3.org/1999/xhtml">
    
    
    
    
    <div id="requirements"><b>Criteria <a href="https://hl7.org/fhir/versions.html#std-process" title="Normative Content" class="normative-flag">N</a>:</b></div>
    <table id="statements" class="grid dict">
        
        <tr>
            <td style="padding-left: 4px;">
            AIG.6#01
            </td>
            <td style="padding-left: 4px;">
            SHALL
            
            
            <span style="border-radius: 999px;margin: 2px 2px;font-size: 10px;display: inline-block;padding: 3px 5px;letter-spacing: 0.5px;line-height: 1;white-space: nowrap;background-color: #f3e5ff;color: #7b1fa2;border: 1px solid #d1a3ff;">conditional</span>
            
            
            </td>
            <td style="padding-left: 4px;" class="requirement">
                <p>IF the system uses generative AI THEN the system SHALL implement controls proportionate to the intended use to reduce generation of unsafe, misleading, fabricated, discriminatory, or privacy-violating content and SHALL define the actions taken when such content is detected.</p>

                
            </td>
        </tr>
        
        <tr>
            <td style="padding-left: 4px;">
            AIG.6#02
            </td>
            <td style="padding-left: 4px;">
            SHALL
            
            
            <span style="border-radius: 999px;margin: 2px 2px;font-size: 10px;display: inline-block;padding: 3px 5px;letter-spacing: 0.5px;line-height: 1;white-space: nowrap;background-color: #f3e5ff;color: #7b1fa2;border: 1px solid #d1a3ff;">conditional</span>
            
            
            </td>
            <td style="padding-left: 4px;" class="requirement">
                <p>IF generative AI is used to provide health-related factual content or recommendations THEN the system SHALL ground the output in curated, version-controlled, and appropriately sourced content or another documented evidence process suitable to the intended use, and SHALL identify when the system cannot verify the requested information.</p>

                
            </td>
        </tr>
        
        <tr>
            <td style="padding-left: 4px;">
            AIG.6#03
            </td>
            <td style="padding-left: 4px;">
            SHALL
            
            
            <span style="border-radius: 999px;margin: 2px 2px;font-size: 10px;display: inline-block;padding: 3px 5px;letter-spacing: 0.5px;line-height: 1;white-space: nowrap;background-color: #f3e5ff;color: #7b1fa2;border: 1px solid #d1a3ff;">conditional</span>
            
            
            </td>
            <td style="padding-left: 4px;" class="requirement">
                <p>IF an AI agent can execute external actions, write to a record, transmit data, purchase a service, or otherwise act on behalf of a user THEN the system SHALL limit the agent's authorized actions, present the action and material consequences to the user before execution when appropriate, and maintain an auditable record of the action request, authorization, execution, and result.</p>

                
            </td>
        </tr>
        
        <tr>
            <td style="padding-left: 4px;">
            AIG.6#04
            </td>
            <td style="padding-left: 4px;">
            SHOULD
            
            
            <span style="border-radius: 999px;margin: 2px 2px;font-size: 10px;display: inline-block;padding: 3px 5px;letter-spacing: 0.5px;line-height: 1;white-space: nowrap;background-color: #f3e5ff;color: #7b1fa2;border: 1px solid #d1a3ff;">conditional</span>
            
            
            </td>
            <td style="padding-left: 4px;" class="requirement">
                <p>IF the system uses generative AI or an AI agent THEN the system SHOULD conduct scenario-based testing for prompt injection, jailbreaks, unsafe advice, disallowed actions, privacy leakage, and escalation to safe human or non-AI resources.</p>

                
            </td>
        </tr>
        
        <tr>
            <td style="padding-left: 4px;">
            AIG.6#05
            </td>
            <td style="padding-left: 4px;">
            MAY
            
            
            <span style="border-radius: 999px;margin: 2px 2px;font-size: 10px;display: inline-block;padding: 3px 5px;letter-spacing: 0.5px;line-height: 1;white-space: nowrap;background-color: #f3e5ff;color: #7b1fa2;border: 1px solid #d1a3ff;">conditional</span>
            
            
            </td>
            <td style="padding-left: 4px;" class="requirement">
                <p>IF the system uses generative AI THEN the system MAY use retrieval-augmented generation or other grounding methods when they improve evidence traceability, provided the system validates source quality, freshness, and access controls.</p>

                
            </td>
        </tr>
        
    </table>
    
</div>
  </text>
  <extension
             url="http://hl7.org/fhir/StructureDefinition/structuredefinition-wg">
    <valueCode value="mobile"/>
  </extension>
  <url value="http://hl7.org/ehrs/uv/cmhaffr2/Requirements/CMHAFFR2-AIG.6"/>
  <version value="2.0.1"/>
  <name value="AIG_6_Generative_AI_and_AI_Agent_Safety"/>
  <title value="AIG.6 Generative AI and AI Agent Safety (Function)"/>
  <status value="active"/>
  <date value="2026-11-18T11:24:54+00:00"/>
  <publisher value="HL7 International / Mobile Health"/>
  <contact>
    <telecom>
      <system value="url"/>
      <value value="http://www.hl7.org/Special/committees/mobile"/>
    </telecom>
  </contact>
  <description
               value="This category applies only where the application generates content or acts on a user's behalf. Generative and agentic functions introduce content, prompt-injection, and action risks that deterministic application logic does not."/>
  <jurisdiction>
    <coding>
      <system value="http://unstats.un.org/unsd/methods/m49/m49.htm"/>
      <code value="001"/>
      <display value="World"/>
    </coding>
  </jurisdiction>
  <statement>
    <extension
               url="http://hl7.org/ehrs/uv/cmhaffr2/StructureDefinition/requirements-dependent">
      <valueBoolean value="false"/>
    </extension>
    <key value="CMHAFFR2-AIG.6-01"/>
    <label value="AIG.6#01"/>
    <conformance value="SHALL"/>
    <conditionality value="true"/>
    <requirement
                 value="IF the system uses generative AI THEN the system SHALL implement controls proportionate to the intended use to reduce generation of unsafe, misleading, fabricated, discriminatory, or privacy-violating content and SHALL define the actions taken when such content is detected."/>
  </statement>
  <statement>
    <extension
               url="http://hl7.org/ehrs/uv/cmhaffr2/StructureDefinition/requirements-dependent">
      <valueBoolean value="false"/>
    </extension>
    <key value="CMHAFFR2-AIG.6-02"/>
    <label value="AIG.6#02"/>
    <conformance value="SHALL"/>
    <conditionality value="true"/>
    <requirement
                 value="IF generative AI is used to provide health-related factual content or recommendations THEN the system SHALL ground the output in curated, version-controlled, and appropriately sourced content or another documented evidence process suitable to the intended use, and SHALL identify when the system cannot verify the requested information."/>
  </statement>
  <statement>
    <extension
               url="http://hl7.org/ehrs/uv/cmhaffr2/StructureDefinition/requirements-dependent">
      <valueBoolean value="false"/>
    </extension>
    <key value="CMHAFFR2-AIG.6-03"/>
    <label value="AIG.6#03"/>
    <conformance value="SHALL"/>
    <conditionality value="true"/>
    <requirement
                 value="IF an AI agent can execute external actions, write to a record, transmit data, purchase a service, or otherwise act on behalf of a user THEN the system SHALL limit the agent's authorized actions, present the action and material consequences to the user before execution when appropriate, and maintain an auditable record of the action request, authorization, execution, and result."/>
  </statement>
  <statement>
    <extension
               url="http://hl7.org/ehrs/uv/cmhaffr2/StructureDefinition/requirements-dependent">
      <valueBoolean value="false"/>
    </extension>
    <key value="CMHAFFR2-AIG.6-04"/>
    <label value="AIG.6#04"/>
    <conformance value="SHOULD"/>
    <conditionality value="true"/>
    <requirement
                 value="IF the system uses generative AI or an AI agent THEN the system SHOULD conduct scenario-based testing for prompt injection, jailbreaks, unsafe advice, disallowed actions, privacy leakage, and escalation to safe human or non-AI resources."/>
  </statement>
  <statement>
    <extension
               url="http://hl7.org/ehrs/uv/cmhaffr2/StructureDefinition/requirements-dependent">
      <valueBoolean value="false"/>
    </extension>
    <key value="CMHAFFR2-AIG.6-05"/>
    <label value="AIG.6#05"/>
    <conformance value="MAY"/>
    <conditionality value="true"/>
    <requirement
                 value="IF the system uses generative AI THEN the system MAY use retrieval-augmented generation or other grounding methods when they improve evidence traceability, provided the system validates source quality, freshness, and access controls."/>
  </statement>
</Requirements>