<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2024 Coalition for Health AI, Inc.

The license for this source code can be found at https://mc.chai.org/LICENSE.
-->
<AppliedModelCard
	xmlns="https://mc.chai.org/v0.1/schema.xsd"
	xmlns:bibtex="http://bibtexml.sf.net/">
	<BasicInfo>
		<ModelName>BriefCase-Triage for Intracranial Hemorrhage (ICH)</ModelName>
		<ModelDeveloper>Aidoc</ModelDeveloper>
		<DeveloperContact>www.aidoc.com/contact-support/ or email:info@aidoc.com</DeveloperContact>
	</BasicInfo>
	<ReleaseInfo>
		<ReleaseStage>FDA Cleared, Commercial (K221240)</ReleaseStage>
		<ReleaseDate>May 2022</ReleaseDate>
		<ReleaseVersion>ICH-02-RT</ReleaseVersion>
		<GlobalAvailability>US, EU (CE Marked), UK (UKCA Marked), Canada (MDEL), Australia (ARTG), New Zealand, Israel (AMAR), UAE and South Africa.</GlobalAvailability>
		<RegulatoryApproval>Please find attached the ICH-02-RT [510(k) summary](https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K221240).</RegulatoryApproval>
	</ReleaseInfo>
	<ModelSummary>
		<Summary>The ICH solution is a radiological computer-aided triage and notification software indicated for use in the analysis of non-enhanced head CT images in adults aged 18 and older. The device is intended to assist hospital networks and appropriately trained medical specialists in workflow triage by flagging and communicating suspected positive findings of Intracranial hemorrhage (ICH) pathologies. The ICH solution uses an artificial intelligence algorithm to analyze images and highlight cases with detected findings on a standalone desktop application in parallel to the ongoing standard of care image interpretation. When suspected findings are detected, users receive notifications that include compressed preview images. These previews are for informational purposes only and are not intended for diagnostic use beyond the initial notification. The results of the ICH solution are intended to be used in conjunction with other patient information and based on their professional judgment, to assist with triage/prioritization of medical images. Ultimately, notified clinicians are responsible for viewing full images per the standard of care.</Summary>
		<Keywords />
	</ModelSummary>
	<UsesAndDirections>
		<IntendedUseAndWorkflow>The device is intended to assist hospital networks and appropriately trained medical specialists in workflow triage by flagging and communicating suspected positive findings of Intracranial hemorrhage (ICH) pathologies.</IntendedUseAndWorkflow>
		<PrimaryIntendedUsers>The BriefCase-Triage is intended to be used by appropriately trained medical specialists. Appropriately trained medical specialists for ICH are: Emergency physicians, radiologists, neuro-surgeons, neuro-interventionalists, interventional radiologists, and neuro-intensive care physicians.</PrimaryIntendedUsers>
		<HowToUse>The user is presented with notification for cases with suspected findings. Notifications include compressed preview images that are meant for informational purposes only and not intended for diagnostic use beyond notification. The results of the ICH solution are intended to be used in conjunction with other patient information and based on their professional judgment, to assist with triage/prioritization of medical images.</HowToUse>
		<TargetedPatientPopulation>patients Adults or transitional adolescents aged 18 and older.</TargetedPatientPopulation>
		<CautionedOutOfScopeSettings>Only for use in workflow triage, and only for use by appropriately trained medical specialists. Please refer to BriefCase-Triage User Guide ([UG0009](https://community.aidoc.com/private/login)) for further information on warnings and precautions.</CautionedOutOfScopeSettings>
	</UsesAndDirections>
	<Warnings>
		<KnownRisksAndLimitations>Notified clinicians are responsible for viewing full images per the standard of care. Please refer to BriefCase-Triage User Guide ([UG0009](https://community.aidoc.com/private/login)), label, and additional tutorial materials provided by the company for further information on risks and limitations.</KnownRisksAndLimitations>
		<KnownBiasesOrEthicalConsiderations>None</KnownBiasesOrEthicalConsiderations>
		<ClinicalRiskLevel>Low, Class II (FDA) / IIa (EU)</ClinicalRiskLevel>
	</Warnings>
	<TrustIngredients>
		<AISystemFacts>
			<OutcomesAndOutputs>Binary notification of suspected ICH cases. The desktop application feed displays incoming suspect cases. Hovering over the feed pops up a compressed, unannotated image that is captioned "not for diagnostic use." The radiologist prioritizes cases based on this additional info and reads the case as per standard of care.</OutcomesAndOutputs>
			<ModelType>Custom-built 3D deep convolutional neural network.</ModelType>
			<InputDataSource>The BriefCase-Triage imaging AI solutions are trained, tuned, and validated using diverse data sets. Specifically, Aidoc works with DICOM images, which capture anatomical and physiological data from medical imaging technologies like CT scans and X-rays. These data sets are sourced from a wide range of institutions, including community hospitals, academic centers, teleradiology providers, private clinics, and imaging centers across various global regions (e.g., east and west US, central US, EU, and other parts of the world).</InputDataSource>
			<OutputAndInputDataTypes>Input: Real World Digital imaging and communications in medicine (DICOM) images.

Inclusion criteria: Non-enhanced head or neck CT protocol with a 64-slice scanner or higher. Scans performed on adults/transitional adults ≥ 18 years of age. Slice thickness 0.625 mm - 5.0 mm.

Exclusion Criteria: All scans that are technically inadequate, including motion artifacts, severe metal artifacts, suboptimal bolus timing or an inadequate field of view.

Output: Binary notification of suspected ICH cases.</OutputAndInputDataTypes>
			<DevelopmentDataCharacterization>The ICH model has been trained, tuned, and validated on over 10,000 scans. The training and validation populations consist of a wide range of institutions and patient settings, including academic centers, community hospitals, teleradiology providers, private imaging centers, and clinics worldwide (specifically East, West, and Center of US, EU, Middle East, Oceania, Latin America and far east).

Analyzed data included key demographic elements such as age, gender, comorbidities, geographical location, patient settings, reason for exam, and technological characteristics (e.g., equipment manufacturer, slice thickness, and modality model distribution).

The time frame for the data used for the initial and retrained model construction and validation is 2015-2022.</DevelopmentDataCharacterization>
			<BiasMitigationApproaches>Aidoc's AI solutions undergo rigorous training, validation, and monitoring to ensure robust and equitable performance across diverse patient populations. Bias mitigation and advancing care equity are highly prioritized throughout the design and development of Aidoc solutions, while working within the known constraints of limited accessibility to certain data elements.

Recognizing these constraints, Aidoc creates proxies to effectively mitigate risks associated with the absence of certain demographic details, when applicable.

Specifically, race, ethnicity, language, sexual orientation, gender identity, social determinants of health data, and health status assessment data are not accessible as part of the DICOM image.

To address potential biases, Aidoc employs diverse data sets from a wide range of institutions and patient settings, including academic centers, community hospitals, teleradiology providers, private imaging centers, and clinics worldwide (specifically East, West, and Center of US, EU, Middle East, Oceania, Latin America and far east).

Aidoc validates and measures the AI model across subpopulations to ensure a normal and predictable performance on them. Additionally, key demographic elements are analyzed such as age, gender, comorbidities, geographical location, patient settings, reason for exam, and technological characteristics (e.g., equipment manufacturer, slice thickness, and modality model distribution);

Furthermore, Aidoc conducts multiple retraining throughout the lifecycle of each AI module to ensure increased generalizability across all the aforementioned parameters. While the exact size of Aidoc's training data sets is proprietary, over the model lifecycle, they may include up to millions of DICOM studies.</BiasMitigationApproaches>
			<OngoingMaintenance>Aidoc's quality management system is ISO 13485 MDSAP and EU MDR certified, and 21 CFR Part 820 compliant, requiring structured and validated installation, servicing, and Post-Market Surveillance (PMS) activities.

Specifically, Aidoc's PMS plan outlines the systematic collection and analysis of data regarding the device's safety and performance throughout its lifecycle.

Data analysis methods, both qualitative and quantitative, are applied to the collected data. The results are documented in a PMS report and in a Periodic Safety Update Report (PSUR). In addition, Aidoc implements site-specific controls, ensuring high standards are consistently maintained across all operational locations. The current ICH model represents the second FDA clearance for ICH at Aidoc. Aidoc periodically retrains the algorithm on additional data and subjects the retrained versions to the required regulatory review and controls.

An expanded dataset enhances the model's generalization capabilities and potentially elevates the product's performance (improved or non-inferior to the previously cleared device's Time-to-Notification and/or AUC). The inclusion of data from a broader and more diverse array of sources enhances the model's ability to perform effectively across a wider range of clinical and technical settings, particularly if any real-world data changes or if any data was underrepresented in the original dataset.

**Continuous monitoring**: As detailed in the previous answer, Aidoc has robust monitoring systems in place to track the AI performance and identify any emerging issues, such as AI drift. In addition, regular audits and assessments ensure the model continues to deliver fair and accurate prioritizations.

Furthermore, Aidoc remains committed to ongoing research and development to refine the AI and address any safety concerns that may arise. The Aidoc AI Monitoring team monitors algorithm performance 24/7 for the purpose of mitigating AI drift. The team monitors and considers aspects such as alert correctness, the timeliness of data available, data completeness, slice thickness, series correctness and relevancy, contrast phase, algorithm specificity, and algorithmic positive ratio.</OngoingMaintenance>
			<Security>Aidoc's products are structured based on international quality, privacy, and security standards and frameworks, including ISO 13485, ISO 27001, ISO 27017, ISO 27018, ISO 27799, SOC 2 Type 2, Cyber Essentials, and C5. Aidoc follows a systematic approach, integrating these frameworks into Aidoc's Information Security Management System (ISMS) to ensure a holistic and proactive risk management strategy.

Aidoc's risk management program involves regular risk assessments, leveraging common methodologies, to identify, evaluate, and prioritize risks. Continuous monitoring, periodic audits, and assessments contribute to the dynamic nature of Aidoc's risk management program, allowing us to adapt to evolving threats and vulnerabilities. Aidoc regularly monitors its server's logs for unusual activity and implements intrusion detection and prevention systems, paying attention to unusual or suspicious activities such as unexpected errors, system slowdowns, or strange user access patterns. Furthermore, Aidoc conducts multiple retraining throughout the lifecycle of each AI module.

Additionally, Aidoc's Governance, Risk, and Compliance (GRC) Program is responsible for enterprise oversight and direction for all security governance activities; governance for risk-related activities including Risk Identification, Management, Mitigation, and Remediation, and Risk Assessments; and responsibility for the planning, execution, and adherence with Aidoc Security Policies and Procedures, legal, regulatory, and contractual requirements.</Security>
			<Transparency>The company ensures Transparency, Intelligibility, and Accountability in Aidoc's medical devices via its product design and its Quality Management System (QMS). Some examples include (non-exhaustive list):
- Explainability and transparency features by design;
- Reporting and feedback mechanism directly via the device's UI and additional communication in different platforms;
- User guide, labeling, and additional tutorial materials are available electronically at all times via the user community;
- Robust user training.</Transparency>
		</AISystemFacts>
		<TransparencyInformation>
			<FundingSource>Aidoc's model creation is self-funded.</FundingSource>
			<StakeholdersConsulted>Physicians, health system administrators, and patient groups were engaged in the development of the model</StakeholdersConsulted>
		</TransparencyInformation>
	</TrustIngredients>
	<KeyMetrics>
		<UsefulnessUsabilityEfficacy>
			<MetricGoal>Evaluate the software's performance in identifying non-contrast head CT images containing intracranial hemorrhage (ICH) in 220 cases from 5 US-based clinical sites.</MetricGoal>
			<Result>Sensitivity was 96.15% (95% CI: 90.44%, 98.94%) and specificity was 94.83% (95% CI: 89.08%, 98.08%).</Result>
			<Interpretation>Primary endpoints of sensitivity and specificity with an 80% performance goal were met. Secondary endpoints were BriefCase time-to-notification compared to the predicate device, Positive Predictive Value (PPV), Negative Predictive Value (NPV), Positive Likelihood Ratio (PLR), and Negative Likelihood Ratio (NLR).</Interpretation>
			<TestType>Retrospective, blinded, multicenter study.</TestType>
			<TestingDataDescription>The mean age of patients whose scans were reviewed in the study was 65.1 years, with standard deviation of 18.8 years. Gender distribution was 49.8% male, and 50.2% female. Race distribution within the study data patient population was unavailable. None of the potential covariates demonstrated statistical significance, thus, device performance did not meaningfully interact with location, gender or age.</TestingDataDescription>
			<ValidationProcessAndJustification>Methods used are described in the [510(k) summary](https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K221240), pages 7-9.</ValidationProcessAndJustification>
		</UsefulnessUsabilityEfficacy>
		<FairnessEquity>
			<MetricGoal>Evaluate for differences in performance (sensitivity &amp; specificity) based on available socio-demographic variables of age, location and gender.</MetricGoal>
			<Result>No statistically significant interaction between performance and age, gender, or location.</Result>
			<Interpretation>Device performance did not meaningfully interact with location, gender or age (Race distribution for sample was unavailable).</Interpretation>
			<TestType>Retrospective, blinded, multicenter study</TestType>
			<TestingDataDescription>The mean age of patients whose scans were reviewed in the study was 65.1 years, with standard deviation of 18.8 years. Gender distribution was 49.8% male, and 50.2% female. Race distribution within the study data patient population was unavailable.</TestingDataDescription>
			<ValidationProcessAndJustification>[Link to Methods Description](https://www.accessdata.fda.gov/cdrh_docs/pdf22/K221240.pdf)</ValidationProcessAndJustification>
		</FairnessEquity>
		<SafetyReliability>
			<MetricGoal>The time-to-notification metric observed for the BriefCase software in the five medical centers was compared to the equivalent metric of prior predicate devices.</MetricGoal>
			<Result>Mean Time to Notification (sec) Predicate K203505 = 267.6, 95% CI = 246.0-289.5. Mean Time to Notification (sec) Briefcase = 33.5, 95% CI = 30.9-36.1</Result>
			<Interpretation>The time-to-notification results obtained for the subject BriefCase device showed improvement with regard to time savings to the standard of care review.</Interpretation>
			<TestType>Retrospective, blinded, multicenter study</TestType>
			<TestingDataDescription>The BriefCase time-to-notification includes the time to get the DICOM exam, de-identify it, upload it to the cloud, analyze and send a notification on a positive suspect case back to the desktop application. The BriefCase time-to-notification was measured for all True Positive cases (i.e., identified as positive both by the reviewers as well as the BriefCase device).</TestingDataDescription>
			<ValidationProcessAndJustification>[Link to Methods Description](https://www.accessdata.fda.gov/cdrh_docs/pdf22/K221240.pdf)

AiDoc develops software within a Design Control process that is aligned with FDA 21 CFR Part 820 and ISO 62304 Medical device software - Software life cycle processes.</ValidationProcessAndJustification>
		</SafetyReliability>
	</KeyMetrics>
	<Resources>
		<EvaluationReferences>Please reference our clinical [compendium](https://www.aidoc.com/learn/ebooks-whitepapers/aidoc-clinical-compendium/).</EvaluationReferences>
		<PeerReviewedPublications>Aidoc's clinical compendium with 100+ Peer-reviewed publications or abstract/conference presentations are available at the following [link](https://www.aidoc.com/learn/ebooks-whitepapers/aidoc-clinical-compendium/).</PeerReviewedPublications>
		<PatientConsentOrDisclosure>N/A</PatientConsentOrDisclosure>
	</Resources>
	<Bibliography />
</AppliedModelCard>