Anatomy of a Public Health Open Source Project: HLN's Immunization Calculation Engine (ICE)

Michael J. SuralikAn immunization information system (IIS) aggregates immunization information for children (and some adults) living or receiving immunization services in a jurisdiction. One of the core components of an IIS is its immunization evaluation and forecasting system: the computerized algorithm that is used to determine if vaccine doses that were administered to the patient are clinically valid (evaluation) and to project what doses are due now and in the future (forecasting). These algorithms are used to support clinical decision support (CDS) at the point of care and also to help public health agencies understand and manage the immunization status of whole populations. When IIS first came on the scene, electronic health records (EHR) were in their infancy. CDS for immunizations was not typically available at the point of care, and that situation persists for many clinicians today even with the advent of more modern EHRs.

Noam H. Arzt, Ph.D.HLN Consulting, LLC has been involved with the development of CDS systems like these for more than two decades. Way back in 1994, in the early days of IIS, we designed, developed and implemented an immunization evaluation and forecasting system (see Figure 1) from scratch as part of an IIS project for the State of New Jersey. In 2000, we began providing application development and support to the New York City Citywide Immunization Registry (CIR), along with their proprietary immunization evaluation and forecasting system, called the Immunization Calculation Engine (ICE), which had been developed by another vendor some years before. Yet again, in 2004, HLN incorporated an immunization and forecasting system used by several projects in the State of California into the integrated child health system that we were supporting for the State of Rhode Island, called KIDSNET.

These systems, like all other immunization evaluation and recommendation systems, are inherently difficult to develop, maintain, and support. The clinical knowledge that underpins the algorithms is developed by the Advisory Committee on Immunization Practices (ACIP), a federal advisory committee supported by the Centers for Disease Control and Prevention (CDC). But the ACIP recommendations are written with healthcare providers in mind, not programmers; it is not easy to convert their recommendations into a complete set of computable rules, and there are many exceptions which have to be taken into account. Invariably, these systems become unsupportable spaghetti code with increasing reliance on the one or maybe two programmers who made a significant investment of their time in learning the background, complex structure, and functionality of the system. And testing these systems has always been challenging due to their complex nature and the interconnectedness of some of the rules.

A New Approach

So, when the opportunity came up in 2012 to develop a new immunization evaluation and forecasting system for New York City (NYC), we knew we had to find a better way to mitigate some of the challenges experienced to-date. Therefore, before we started, we met with our colleagues in NYC (and from the State of Alabama who decided to join the project) and developed a clear set of project goals. This time around, we wanted to create a system that:

• Supports routinely administered vaccine groups from birth through adulthood
• Promotes clinical best practices
• Readily adapts to changing requirements
• Easily integrates with IIS and other systems
• Is freely available

This was a tall order, and in fact, NYC's funding for this innovative project was exhausted long before the project was completed. HLN had a decision to make: do we press on ahead to create a new product regardless of all the unfunded work that lay ahead of us, or do we walk away? As a company with its origins in the academic world and open source (many of us had previously been involved in the management of technology at the University of Pennsylvania), we decided that HLN would make the investment in continuing this work to develop a new version of ICE. Since we had started the project with NYC funding, we agreed that the new product would be released to the open source community (under the GNU LGPL v3 license). This was true to our original goal of making the software generally available and of benefit not only to the public health community but to the healthcare community at large.

We were also determined not to reinvent the wheel. As noted earlier, clinical decision support is critical at the point of care. For this reason, we did our research and quickly identified OpenCDS, an existing open source software platform, and toolkit that was specifically designed for developers to create their own CDS services that are standards-based and accessible via a web services interface. OpenCDS had been developed by Dr. Kensaku Kawamoto at the University of Utah, and we knew of Ken and his many years of deep experience in developing CDS systems and tools. OpenCDS provided us with a solid foundation upon which to build our own CDS service: it used industry standard software including Drools, a business rules management system, to store the rules and logic, the Object Management Group (OMG)/Health Level 7 (HL7) Decision Support Service (DSS) for its web services interface, and HL7's Virtual Medical Record (vMR) to represent the data being sent to and from the service.

But that was not enough - OpenCDS alone would not enable us to achieve all of our goals. Our next generation immunization evaluation and forecasting system also needed a robust administrative interface to support comprehensive regression testing as rules change, manage code sets and terminologies used by the rules, and, eventually, enable non-developers to author and publish rules without knowing Drools. To support these administrative functions we developed the CDS Administration Tool (CAT) as a companion web application to the ICE service. With its origins in a similar administrative tool used in NYC for CIR, CAT was written with JavaServer Faces for its graphical user interface and Enterprise Javabeans for its business logic. It relies on an off-the-shelf JDBC-compliant database, and a Java Platform Enterprise Edition-compliant application server (typically Glassfish, but we have also run it under JBoss).

So How Does It Work?

So how exactly does the ICE service work (see Figure 2)? A system that wants an evaluation and forecast from ICE starts by creating a message in HL7 vMR format (which is expressed in XML) with a specific set of inputs:

• Patient date of birth
• Patient gender
• Immunization history, expressed as vaccine codes (CVX) with associated dates of administration
• Disease immunity for a select set of conditions expressed as SNOMED-CT or ICD-10-CM codes
• An identifier for the immunization schedule to be invoked (ICE can support multiple simultaneous schedules, though most installations use only the single default schedule that comes with ICE)
• Date of evaluation, typically the date the service call is made (there are specific reasons why you might want ICE to "pretend" it is doing its evaluation on a different date, past or future)

Then the calling system wraps the vMR in a web services envelope and submits it to the ICE web service. Note that no additional patient-identifying information is sent. ICE does not care who the patient is. Only his/her age, gender, and immunization history is needed. And none of the input data is persisted in the ICE service beyond the time that it takes to process a request. Therefore, though a patient's entire immunization history must be sent every time that the system calls ICE, little to no protected health information (PHI) is at risk during a transaction.

The ICE service receives the message, unpacks the vMR payload, and processes the inputs using the rules of the immunization schedule. Then the ICE service generates its outputs and returns them as another vMR message in a web services envelope with the following information:

• For each vaccine dose that was submitted, ICE returns:
  • An evaluation as to whether the dose was valid, invalid, or accepted (a rare but necessary third possibility).
  • In addition, for each invalid dose, ICE also returns one or more reasons explaining why the dose was invalid.
• For each of the predefined vaccine groups that ICE is configured to support (there are currently fifteen), ICE returns:
  • A recommendation for the vaccine group: Recommended now, recommended in the future, conditionally recommended, or not recommended (usually when the series is completed)
  • Several dates, including the earliest date at which the next dose would be considered valid, the earliest recommended date for the next dose, and the date at which the next dose would be considered overdue.
  • A reason for the recommendation
• If a vaccine is not supported by ICE (like most travel vaccines), ICE returns a message that a recommendation is not available.

The ICE service has been tested under a variety of load conditions in both Test and Production environments. It can supports 24 hours a day, 7 days a week operations without interruption. The only time a restart is required is during a software upgrade which does not happen often (as opposed to a rules update) or, potentially, during updates to the underlying operating system. The response time for a single patient request as well as its overall throughput will vary according to how ICE is configured and the capacity of the underlying server.

To give a sense of performance for those interested in using ICE, HLN recently tested ICE on two Amazon "Compute Optimized" cloud servers with 8 virtual cores and 16GB RAM. The first server was running Amazon Linux, configured with Oracle Java 8, Apache Tomcat 8 (with a maximum heap space of 2GB). The second server was configured with Microsoft Windows Server 2016 and the same Java, Tomcat, and ICE software and settings. Each server demonstrated a sustained throughput of 75 requests per second with an average response time of 0.2 seconds per request, including the roundtrip time for the requests/responses to travel over the Internet to a remote client.

Open Source Distribution

ICE binaries and source code are stored on bitbucket and are freely distributed through our ICE project wiki. We provide everything that an organization needs to install and operate the ICE service:

• Binary distribution and source code
• Release notes for each version
• Directions for installing ICE
• An Implementation Guide specifying how to integrate client-side applications with the ICE service web-based test application (see Figure 3) that runs against an HLN-hosted instance of the ICE service. This application allows anyone with internet access to quickly and easily test the ICE rules by entering test cases interactively through a simple graphical user interface. It also allows users to test ICE by uploading vMR (or JSON) formatted input files and receive the results from our HLN-hosted instance of ICE.

We feel strongly that an organization or vendor should be able to use ICE "out of the box." To that end, the distribution itself contains not only the ICE service but also the default set of ICE rules that are compliant with the clinical guidelines established by the Advisory Committee on Immunization Practices (ACIP)-compliant. The rules for the ICE default schedule are comprehensively documented on our public wiki in a manner that is clear, precise, and unambiguous. We have had organizations tell us that even if they can't use ICE, they appreciate (and use!) this clinical knowledge written in an easy to understand and consistent manner.

Conclusion

On June 6, 2017, HLN was awarded the 2017 Upshot Award for Excellence in Vaccine Supply, Access, and Use by the National Vaccine Program Office (NVPO) for its ICE Open Source Immunization Forecaster. In the letter of award, Dr. Jewel Mullen, Principal Deputy Assistant Secretary for Health commented that,

Michael J. Suralik receives 2017 Upshot Award"HLN Consulting's efforts on the Immunization Calculation Engine (ICE) are impressive. This powerful tool including its open-source nature and seamless integration into a clinical workflow-holds great promise for improving clinical decision-support and ultimately vaccination rates. Thank you for daring to innovate, collaborate, and lead in an area that is not only complex but constantly evolving."

We consider ICE a true open source success story. Numerous organizations that had no involvement in the development of ICE have successfully taken this open source software, integrated it with their systems and utilized it to improve the health of their patients and populations. eClinicalWorks began rolling out ICE in their ambulatory EHR in 2013 with no active involvement by anyone at HLN. Several GE Centricity sites also use ICE with the assistance of middleware from a third party, Health 1 Technologies.

Various IIS now use ICE (at least in part) as their evaluation and forecasting system, including New Jersey, Michigan, and soon Rhode Island and New York City (which still uses an earlier, proprietary version). A major national Personal Health Record, CareDox, also incorporates ICE within its system. ICE was even selected for integration with the Veteran Health Administration's (VHA) VistA EHR; however, the VA's decision to migrate from VistA to Cerner ended that project.

Most recently, the Indian Health Service (IHS) began a project to migrate all of the implementations of their Resource and Patient Management System (RPMS) to ICE across all of their sites. As we look to the future, in conjunction with a new partnership with RedHat, HLN is now considering offering a cloud-based forecasting service which would provide yet another way for organizations to implement ICE quickly and easily.

Additional reading:

Michael J. Suralik, et al, The Immunization Calculation Engine, Open Source Clinical Decision Support for Immunizations, Journal of Healthcare Information Management, 27(3), Summer 2013.

Clinical Decision Support for Immunizations (CDSi): A Comprehensive, Collaborative Strategy, Biomedical Informatics Insights, Suppl. 2, October 2016.