VA Poised To Kick-Off Contract For New Scheduling System
The Department of Veterans Affairs has completed a series of one-on-one meetings with companies interested in taking on what is perhaps the biggest, most complex and important government IT challenge since the rollout of healthcare.gov — replacing VA’s antiquated patient scheduling system with commercial technologies that will enable veterans to see doctors and receive treatment when and where they need it.
The department is scheduled this month to release a request for proposals for its Medical Appointment Scheduling System, which will replace the patient scheduling module within the Veterans Integrated System Technology Architecture, VA’s main electronic health record system known as VistA. The scheduling system has been at the heart of the scandal involving veterans who died waiting for care after being placed on so-called secret wait lists. But VA officials recently outlined an aggressive plan to completely overhaul VistA scheduling along with the cultural and business process challenges that have contributed to VA’s problems...
VA plans to acquire its new commercial off-the-shelf (COTS) scheduling module in three phases. The first phase, known as version 1a, will provide immediate, short-term improvements to local scheduling...Gary Monger, a product engineer in the Architecture Strategy and Design Group within VA’s Office of Information and Technology, said there are a lot of integration points between VistA scheduling and other legacy VistA applications that will be handled through federated VistA scheduling adaptors that the vendor who wins the contract must develop. “There’s significant risk, cost and time associated with migrating those integration points to a new COTS scheduling application,” Monger said. “So the initial approach is to push that appointment data into the legacy VistA scheduling application.”...
This article raises grave concerns that the VA is giving up on their open source strategy. This is perhaps the most thorough and extensive article that claims that the VA's VistA Evolution strategy is based on purchasing commercial off-the-shelf technologies (COTS). This is not the only one. More than a dozen articles have appeared in the press in the past few weeks making the same point. It has been repeated so many times that in has become a drumbeat and de-facto assumption by reporters.
The COTS or bust approach would be consistent with rumors, as detailed by Bob Brewin in Nexgov that the new VA Secretary Robert McDonald is going to appoint P&G's CIO, Filippo Passerini, as VA CIO. As Brewin points out in his post, during an inteview last year, Passerini told Forbes that "he learned at P&G to treat IT as a commodity enabler of business functions," which Brewin says is "an indication of the approach he could take at VA." And Bob, one of our favorite writers, is usually right.
The danger is that the COTS approach would be a complete disaster for VistA, the VA, and its ability to care for veterans. The COTS approach has already proven to be a high risk approach with over $5 billion dollars wasted in attemted COTS enhancements to VistA in the past decade. The replacement of the Scheduling module, was in fact, one of those spectacular COTS failures.
Back to our friend Bob Brewin, who states in this other article "In 2009, the Veterans Affairs Department canceled its patient scheduling system -- dubbed the Replacement Scheduling Application Development Program -- after spending $167 million over eight years and failing to deliver a usable product.” And in an article previous to that one, Brewin states "Veterans Affairs Department Secretary Eric Shinseki launched a review of the department's information technology programs on March 23, focusing on the $167 million patient scheduling program, which has failed to deliver a usable system after eight years in development, NextGov has learned.”
So to review the story. The COTS approach to improving and enhancing VistA has already been a high-risk, spectacular failure over the past decade with more than $5 billion going down the drain. The fedscoop article acknowledges that following a COTS approach to “modernizing” the scheduling app is a “high risk” proposition. The article states:
Gary Monger, a product engineer in the Architecture Strategy and Design Group within VA’s Office of Information and Technology, said there are a lot of integration points between VistA scheduling and other legacy VistA applications that will be handled through federated VistA scheduling adaptors that the vendor who wins the contract must develop. “There’s significant risk, cost and time associated with migrating those integration points to a new COTS scheduling application,” Monger said. “So the initial approach is to push that appointment data into the legacy VistA scheduling application.”
So the COTS approach is a “high-risk” approach, yet the fedscoop article fails to mention that there is a completely opposite approach, one low-risk, low-cost approach based on an open source, collaborative development methodology. While the COTS approach is going to take years to deliver a solution, cost a huge fortune, and is almost certain to fail, one of the open source alternatives is already running in a hospital system, Oroville Hospital in Northern California
Those of you who read Open Health News should know by now that Oroville Hospital is an extraordinary success story when it comes to EHR implementations in the private sector. In short, Oroville Hospital self-implemented Vista over the past six years, and today it has a fully functional EHR. They have implemented VistA in the 153-bed hospital and in 23 surrounding clinics. In addition, the hospital system is part of a five- hospital Health Information Exchange (HIE) in Northern California. This HIE is seamlessly exchanging patient records among hospitals that have all different EHRs. Oroville has achieved 100% CPOE and has qualified for meaningful use. All of this for less than $8 million.
In comparison, other regional hospitals in California similar in size to Oroville have spent between $25 million to $30 million each for their proprietary EHR implementations. Those figures are for a basic EHR with no CPOE. If one adds all the features that Oroville has implemented in their VistA EHR, it would cost other hospitals between $50 million to $100 million to match those found at Oroville. And that’s just for the implementation costs. Those other regional hospitals in California are paying $5 million to $10 million a year in software licensing fees to their proprietary EHR vendors. Oroville’s annual EHR software licensing fees is…zero. And, as mentioned in this press release, since implementing VistA, Oroville Hospital has been able to document the quality of care they provide their patients. As a result, Oroville Hospital has been receiving one “America’s Best 100 Hospitals” award after another.
And why talk about Oroville Hospital? Well, Oroville Hospital won second place in the VA’s Scheduling App Competition. That competition was launched by former VA CIO Roger Baker in search of a better way for the VA to develop software to enhance and modernize VistA. In the face of billions of dollars lost in failed attempts to modernize VistA using the COTS approach, Baker decided to shift the software acquisition methodology to an open source methodology. He did this with full support from former VA Secretary Eric Shinseki who was very concerned about how the COTS approach was endangering the health and well being of the veterans being treated at VA facilities by failing to deliver needed improvements to VistA. This effort led to the creation of the Open Source Health Record Alliance (OSEHRA) a non-profit organization that could serve as a VistA code repository and exchange mechanism so that the VA could collaborate with the private sector in the continual enhancement and development of VistA.
Baker went so far as to have the VA formally participate as a case study in the 2012 Open Source Think Tank conference. This is one of the most interesting and extraordinary open source meetings out there. The conference is by invitation only and the organizers bring together some of the best minds in open source to examine case studies and propose solutions to the presenters. The other case study at that conference was that of the Genivi Alliance. This article goes into great detail about the conference and the background leaving to the VA participating at the meeting.
Back to the VA Scheduling App competition and the Oroville Hospital Scheduling Application. Oroville Hospital submitted software that they were actually developing for use in the hospital and it is that alpha version of the software that won second place. Details of their scheduling app are found in this article by Rob Tweed. Oroville kept improving the software and over the past few months the Oroville staff have been beta testing their scheduler app in several clinics and several hospital departments (my understanding is that imaging/radiology is one of them). I got a chance to visit Oroville Hospital a couple of months ago and was able to sit down with their chief developer, Zack Gonzales. Zach spent three hours going through the scheduler with me, as well as several other applications that Oroville Hospital is developing to enhance VistA.
What he showed me was simply astounding. Although still in beta test at the time, Oroville’s Scheduling App clearly meets and exceeds the VA’s requirements for a scheduling application. The cost of developing this Scheduling App, as well as dozens of other enhancements to Vista, are included as part of the “less than 8 million dollars” in total costs of implementing VistA calculation. Now let’s compare this a COTS scheduling solution.
The VA estimate is that a COTS scheduler is going to cost around $300 million. It will take several years to develop. It is a high risk proposition with multiple potential points of failure. It will be a locked-in proprietary solution. It will make some beltway bandits very rich. And in exchange they will make large contributions to the political campaigns of elected officials in the House and Senate Veterans Affairs Committees. So Congress wants a COTS solution.
The Oroville Scheduling solution is a low-risk open source solution. It is already in daily use at multiple clinics as well as several department of Oroville Hospital. It is built on top of the existing VA scheduling solution so it’s in effect a painless upgrade. It is built with one the most modern web technologies, EWD.js, which is an adaptation of Node.js that works with the MUMPS database. The great disadvantage to this approach is that with such a low-cost solution, there wont be much, if any, money left for the solution providers to give large polical campaign contributions. So Congress is against an open source solution.
While a COTS scheduler is going to take years to develop. Given that the Oroville Scheduler is an enhancement to the existing scheduler, it should not take more than a few weeks to upgrade, say the Phoenix VA Hospital, the epicenter of the VA scandal, to the new software. Why should the VA hand over $300 million to some random beltway bandit company to build something that already exists, and its working? Only someone with the satiric skills of Jonathan Swift, or Francois Rabelais, could properly describe this situation. Perhaps Jon Stewart needs to look into this. Any of our readers know him? Perhaps they can forward Stewart a link to this post.
The question yet remains, has the VA given up their open source strategy? Will the VA go back to its “pour billions down the toilet” previous COTS strategy? I talked to several of my friends at the VA and what they told me is that the VA is still fully committed to an open source strategy. The VA is in fact going to be laying out details of its open source strategy at the upcoming OSEHRA Open Source Summit in Bethesda, MD, September 3-5. The agenda shows an impressive number of senior VA folks talking open source. In fact, Stephen W. Warren is one of the keynote speakers. Jim Whitehurst, President and CEO of Red Hat is one of the other keynote speakers.
At the same time these VA contacts did acknowledge that the VA is a large organization and that there are many people in the VA fiercely opposed to an open source approach. Apparently the opposition is not necessarily to the use of open source software per se, but the cultural change that comes from an open source, collaborative software development methodology. The cultural change involves collaboration, transparency, innovation and accountability. Promotions and bonuses will be based on merit. Procuring COTS software maintains the existing culture of top down, central office control, no collaboration with the VA field offices, no transparency, and no accountability. If the project fails, well…"it was the fault of the government contractor…” Rob Tweed wrote another article that discusses the role of open source innovation vs. the COTS approach.
As for the upcoming scheduling RFP. What I was told is that there is no fix for COTS solutions. The VA wants a level playing field where open source proposals can compete with proprietary COTS solutions. But, as noted earlier, these statements have to be taken with a grain of salt as Robert MacDonald may have a different agenda and all other publications out there are saying that the VA's strategy will be a COTS strategy.
The author of the fedscoop article, Dan Verton, is an excellent reporter. I have read many of his articles over the years and found them to be well researched, thorough and accurate. And fedscoop is a great publication that does not shy from tough issues, exposing government procurement scandals, and stirring up controversy. So Verton may be absolutely correct and the VA may no longer be considering an open source solution and is simply planning to purchase a COTS product. After all, the slides in his article did come from the VA. It seems that we need Bob Brewin to look in to this story to find out what's really going on.
Two more important points to be made. VA’s scheduling software was not at fault in the Phoenix VA Scandal, despite what the press says. The staff gamed the system by not entering the appointment details into the computer. They wrote them down in a piece of paper so the details were never in VistA. No amount of money on this planet can fix a cheat like that. Edmund Billings provides details in this article.
Second point has to do with the other open source approaches to improving and enhancing the VA Scheduling App, and the rest of VistA for that matter. There are in fact, five open source approaches that have been proposed. From what we can tell, any of them can lead to a successful VistA evolution. We have focused on the Oroville Scheduler in here for two reasons.
First, I have spent a considerable amount of time writing about Oroville Hospital’s VistA implementation, so I am very familiar with their solutions. Second, Rob Tweed, which has been the champion of using Node.js to modernize VistA, is a very articulate writer and he has discovered that if he writes good articles, we will publish them in Open Health News. I personally read and edit all articles that are submitted to Open Health News, so the Oroville/EWD.js approach is what I know best.
I have, in fact, approached all the other teams that won in the VA Scheduling contest to write articles. Rob Tweed and the MedRed Team, which came in first place in the Scheduling Contest, took me up on it. Since then I approached several of the other teams with different open source solutions and technologies to write articles. I am still waiting for those articles. As soon as I finish this comment, I plan on reminding everyone to send me their articles so we can publish them and let the world know about the range of low-risk, low-cost open source technologies and approaches that exist to modernize VistA.
In addition, we would be more than willing to also publish articles by the COTS solutions teams and give them the opportunity to present their case. That’s the open source way.
Roger A. Maduro, Publisher and Editor-in-Chief, Open Health News.
- Tags:
- Access Clinic Administration Program
- Architecture Strategy and Design Group
- commercial off the shelf technologies (COTS)
- commercial off-the-shelf (COTS)
- commercial scheduling system
- COTS scheduling application
- COTS-based replacement of VistA scheduling capabilities
- federated VistA scheduling adaptors
- FedScoop
- Gary Monger
- Gerry Lowe
- graphical user interface (GUI)
- healthcare.gov
- Medical Appointment Scheduling System (MASS)
- Mike Davies
- Office of Information and Technology (OIT)
- resource management dashboard
- US Department of Veterans Affairs (VA)
- VA scheduling system
- Veteran Facing Applications
- veterans appointments
- Veterans Health Administration (VHA)
- VistA EHR
- VistA scheduling system
- Login to post comments
Comments
VA Evolution of Trust
Veteran's Affairs, as a representative microcosm of the healthcare industry, has been unable to muster the consensus (will) to guide their healthcare enterprise to higher quality, lower cost solutions because they are unwilling to purposefully define the complex lattice of fundamental trust abstractions to facilitate governance of their integrated healthcare enterprise (roughly Patients, Providers and Contractors). The assertions upon which the early enterprise was built relied on lofty voluntary duty while evolution and proliferation over the last few decades has managed to judiciously avoid substantive metrics or enforcement to ensure plenary cooperation (with few exceptions). Until the fundamental trust relationships are tightly articulated to define obligations and metrics (accountable behavior), this privileged trust model can only heap continued “unintended consequences” on the industry, mission Agency, populace and treasury.
Piecemeal reforms have wrought indignation over the obvious creation of opportune imbalances to attract privileged contributors (and their associated cost) to obligated participation, resulting in not only inefficient returns on dollars channeled (meaning prescribed bribery rates would be substantially cheaper) but also increased granularity/confusion from the layering of rent-seeking implementations over opaque accounting, misdirected codification, and loss of proliferation control.
The fundamental rules of engagement (trust model) must be comprehensively re-considered, rather than gerrymandered, to ensure control over the apparently never ending stream of contrivances (now boldly operating in the open) which violate the spirit of duty/accountability (for which the current rules are written) and are needed to encourage compliance and management of enforcement priorities...effectively a strong fences argument. Instead VA leadership insists on improper control of the options (on their way to consistently and deliberately answering the wrong questions) while exhibiting a disturbing sense of fortitude against engaging meaningful systemic remediation to end undue reliance on interpretive (subjective) gaps between legislation/policy, policy/procedure and exclusion by arbitrary condition...traditionally preserved as a discretionary trap door liberally deployed in their pursuit of homage. However these methods should not be confused with an inability to regulate or govern.
Such patterns of sanctioned misdirection are historically well and deeply engrained in a surprising percentage of public endeavors with access to the treasury. Unfortunately, our national proclivity to excess now threatens the status quo from all sides while remediation efforts are quickly redirected into ideological posturing, blame, threats and outright subterfuge (loss of vision/mission)…undeniable manifestations of persistent cultural problems and a deeply divided constituency.
There are few innocents in this conundrum...repeated unintended consequences, evidenced by our recurring inability to lead and encourage lean practices to match quality improvements have led to: 1) large scale loss of confidence in leadership, and worse, in the motives of leadership; and 2) the staging of intra-agency passive resistance to transparently motivated “reform” initiatives (and the perennial emergency re-solution conveniently restated to accomplish the same unspoken goals). As such, abuse of trust relationships has encouraged unofficial forking of established lines of authority/reporting to support off-mission avenues (often based on personal ideology) and virtually assuring loss of mission imperative and a global inability to improve/progress for the entire organizational. While maps of organization-wide descent into cultural chaos may vary...a very common risk of disingenuous leadership is such behavior threatens the personal value system of the rank and file who must accomplish it. Well meaning, dedicated employees are compelled to act in defense of their personal interpretation of mission priorities which creates competing layers of passive dissent, widespread “civil disobedience” and confusion toward accomplishing any enterprise goal.
When, as now, it is simply too much to expect repeated buy-in to indentured homage initiatives (often begun with feigned incompetence)...when it becomes impossible for a loyal employee to perform his/her job without injury to themselves...when employees, distributed across the system, spontaneously defeat a mission in misguided and desperate attempts to fulfill their perceived (unwritten) job description...it should be clear that breach of trust has created cultural problems that can only be resolved by restoration of prudent leadership and trust.
The extant example being VA's decision to support open source development of enterprise IT systems. Beginning with the distortion of the terminology, the adoption of experimental “open” models and pejorative (and possibly illegal) front-loading of contract preferences in the requirements and bid stage of the selections process...it seems only reasonable to request a hard and fast definition of the so-called “COTS” approach. As it will be informative in understanding, with specificity, how the COTS approach is differentiated from prior failed attempts and the OS native development approach...to which they are currently committed...but have failed to develop clear lines of procurement or "level the playing field", as promised. Clearly there is a decided poverty of leadership continuity, but rushing toward re-entering a failed model without the clarity of simple hindsight assessment or development of actionable policy/procedure to avoid repetitive mistakes is, again, questionable at best. One might ask why such a specification as COTS is not clearly written into the requirements documentation? The resounding answer might be that existing policy does not support such a position and selections wants to avoid a direct comparative evaluation.
Also conspicuously absent from this new phase of mega contracts (to the usual suspects) is any pledge of transparency in selections and accountability in contracting...even after numerous (even repetitive) challenges to their processes and the maturity of their approach...see nearly 25 years of GAO, OIG, OMB and congressional investigations and authoritative reporting/literature addressing precisely the same issues. The VA's new secretary is being set-up for a trial by fire following the flawed advice of the OI&T faction and depending on the follow-thru of the recently demonized rank and file civil servant (who actually believes their first job is to support the veteran). The treasury is being set-up for another massive bleeding and the injured vet is being set-up for another unnecessary delay...all because VA leadership has become more inured to the gatekeeper position (of what is turning out to be one of the largest government slush funds in existence today) than the mission of the Mission Agency.
Ask yourself why the recent VA Reform Bill (the actual title is offensive) that flew, effectively unopposed, through both houses of congress contained an entire section for the removal of VA senior executives...and what is being done with this provision since it was used to garner near unanimous support? Where are the meaningful investigations to call those actually responsible to answer for their perfectly legal breaches of the public trust (other than an hourly phone clerk). Better yet, when will conscientious civil servants avail themselves of their only opportunity to deliver the heads of the senior management thru passive disobedience? How much of the billions wasted will be recovered from opportunistic senior manager insuring his golden parachute subverting enterprise architecture and investment; the incompetent technology planning partners who doomed the efforts from the beginning; the manipulative competitive bid administrators who played the gap to justify the selection; the unprincipled attorney who wrote the specious contract provisions; the contributor contractors who milked the contract but did not deliver; and the incompetent project managers who turned a blind eye to their approvals for progress payments? (to name a few) Each of these players rationalizes his abuse of the trust model as immaterial to the whole...but when taken in concert the effect is crippling.
I also believe when we demand that our elected officials marshal the courage to ensure restitution of an accurate and representative trust model and the removal of barriers preventing responsible innovation and honest competition...IT issues such as these will quickly resolve.
Why doesn't COTS work?
The Commercial Over The Counter solutions don't work in hospitals because One Size does not fit all. There are as many ways of providing healthcare as there are hospitals. The developers of these system have never actually spent a night in a ward with the people that have to do their job every day and night. As it is, these solutions are usually so generalized as to be nearly impossible for most staffs to utilize properly. The VISTA model has grown up in the hospitals with the staff carefully directing the programmers as to what works and what doesn't. The programmers make the corrections so that the code is brought more in-line with the needs of the staff. In the last decade in the VA the programmers are being removed from the hospitals and centralized. This is medical software by Central Office Fiat and continues to miss the mark that the staff at the point of care is more and more disenfranchised. This is a disaster for such a software jewel as the VISTA model. This is why we formed WorldVistA.ORG to begin to build a community that can help to sponsor the further evolution of VISTA outside the VA, the Politics of the DoD, and the resource limitations of the IHS (Indian Health Service, the poorest of all those mentioned).
COTS has been tried a lot at great expense
The issues of Open Source are very important and need to be addressed by the user community. VISTA was an Open Source effort before there was the concept of Open Source. It has been running and adapting to the needs of the Hospitals being run by the VA, Indian Health Service, and the Department of Defense, as well as community hospitals. The VA Management has been trying to get COTS products into the VA for over 25 years with very little success. The VA, DoD, and IHS hospitals are still running VISTA Technology. If management would not hinder VISTA development the way they have for those 25 years, all of the commercial hospitals would be running VISTA technology and we would not have these terribly expensive forays into expensive COTS products. These have cost us way too much already. The problem of the VA is not the scheduling system, but one of capacity and communications between the hospitals which was discouraged by the VA management. The Management that has kept VISTA from evolving with the needs of the VA as well as lying to General Shinseki. He took the blame for his inherited staff lying to him. It is all designed to create disaster and hinder a solution that keeps coming from the VISTA system. The scheduling system could be done by abstracting the VISTA model to branch between VISTA hospitals and do the scheduling. It could also be expanded to include unused beds in community hospitals for Veterans' overflow. The VA should stop fighting their success and go with it as it evolves.