Table Of Contents
Cisco Medical Data Exchange Solution
Integrating the Healthcare Enterprise
SpiritEHR Software Architecture
Cisco Unified Computing Platform
Cisco Medical Data Exchange Planning and Design
Enterprise User Authentication (EUA)
Audit Trail and Node Authentication (ATNA)
Cross Enterprise User Assertion (XUA)
Non IHE Compliant—Supported Standard Compliant
Cisco Medical Data Exchange Implementation
Recommended Supporting Applications
High Availability Configuration
SpiritEHR Software Configuration
Tool Scripts and Useful Shortcuts
Configuring Organization, Roles, and Users
Distributing System Information
Cisco Medical Data Exchange Solution Application Deployment GuideLast Updated: November 3, 2011Contents
Cisco Medical Data Exchange Solution
The Cisco Medical Data Exchange Solution (MDES) is a Cisco Validated Design (CVD) that facilitates the exchange of information within and between healthcare organizations. The solution provides the primary components required to facilitate the exchange of information between disparate health information systems within a healthcare facility or organization, or to implement a local, regional, national, or international health information exchange (HIE).
The Cisco MDES provides network-based services that recognize information coming from known data sources, then formats and translates data into a common and usable format in real time. Additionally, the MDES provides the patient indexing and record locator services required to locate the correct patient information within the network, delivering the right information to the right person at the right time.
The ability to share information between disparate systems and across organizational boundaries provides the following benefits:
•
Better patient care and treatment, improved patient safety and satisfaction, and lower cost by providing faster access to records reducing patient wait time
•
•
Automated, streamlined workflows improve access to information with fewer errors and give healthcare providers fewer interruptions. Because the solution acts as middleware that allows the caregiver access to external information through their current healthcare application, there is no new system for the caregiver to learn.
The Cisco MDES is based on the internationally recognized Integrating the Healthcare Enterprise (IHE) framework, which uses security, networking, clinical data, and medication standards to remove the boundaries of interoperability among key stakeholders within the healthcare ecosystem. The Cisco MDES authenticates and authorizes the user, verifies and validates the identity of patients for whom data is being exchanged, and logs all transactions. Furthermore, if a remote practitioner does not have an electronic health record (EHR), the MDES can present the information through a web-based portal, providing services for viewing, accessing, or transacting services to align with patient needs.
The solution uses Cisco compute platforms ranging from the Cisco Unified Computing System (UCS) Express on the low end to Cisco Unified Computing Systems blade and rack servers for mid-range and high end. The UCS hosts an implementation of the IHE stack provided by Tiani Spirit (http://www.tiani-spirit.com), which allows the solution to scale from a small facility that needs to share information between a few systems to large HIEs.
Executive Overview
Today, most patient information is stored in disparate systems across the healthcare community (physician offices, clinics, and hospitals), and many of these systems do not interoperate. Practitioners in private practice may have difficulty obtaining complete information about a patient currently being hospitalized. In addition, practitioners may repeat tests and procedures because they do not have prior information about the patient.
The Cisco MDES can integrate intelligence into the Cisco Medical-Grade Network (MGN) to deliver needed interoperability to address the cost and quality challenges faced in the current healthcare industry. Government support is on the rise, including funding for medical data exchange to improve access to medical records. The EU epSOS project has 24M Euro funding for exchange of medical smart records and medication records between EU member states, and the MDES is the key component of this project. Meanwhile, the United States has allocated $36 billion in stimulus funds to install electronic health records. A key component of this funding is for interoperability solutions to meet the demands for access to care.
Manual paperwork processes are expensive to both patients and practitioners. For example, patients may have to hand-deliver a paper copy of their record to another physician, or the provider may have to fax it to the physician or copy the patient record to a CD. These tasks can cost $15 to $20 each and are not a reimbursable expense. The alternative is to proceed with the current patient encounter without prior visit history.
Creating interoperability among healthcare systems to seamlessly and easily exchange information in near real time is critical to making meaningful improvements in healthcare delivery. Interoperability is complex and cannot be solved by focusing only on individual standards and quasi-solutions. A new approach removes the complexities associated with interoperability and paves the way for next-generation healthcare.
The effective exchange of information enables practitioners to make better decisions at the point of care. Governments have realized this and are beginning to provide funding the creation and operations of electronic health record (EHR) exchange systems.
The Cisco MDES provides the ability to deliver the needed interoperability services to enable HIE, new models of telehealth, more efficient care, and other services. Cisco provides the foundation (transport, compute, and interoperability) from which to deliver applications and services to business and clinical services, as shown in Figure 1.
Figure 1 MDES—High Level Logical View
Imagine a healthcare community where access to a patient's medical history is available—from allergies, current medications, to medical and surgical history—according to the patient's care requirements. For example, a clinician may need to view only data regarding the patient's prior medication to confirm appropriate treatment, or a clinician may need to compare a prior patient encounter with a current one in real time, with multiple colleagues. In another example, a physician may want to order medication from a local pharmacy over a secure, interoperable network.
Across all levels of interoperability, a record of care is created based on a patient-centric, not an application-centric, view. In this way, healthcare information can be "pushed" to virtually any device, anytime, or anywhere over a secure and flexible network—much in the same way information is pushed to consumers online. Each of these levels requires secure connectivity and standards-based data access. This type of operating model enables practitioners to access all relevant information to make the best decisions related to their patients' care.
This model has many variations and can be delivered across many environments (see Figure 2). The market opportunity for the MDES ranges from being a component in a larger clinical system (for example, Electronic Health Network or Cisco HealthPresence) to a regional HIE to a state HIE or supporting a national health record system.
Figure 2 MDES Environments
Solution Description
The MDES solution integrates IHE standards-based applications from Tiani Spirit with Cisco compute platforms to form the basis for the efficient exchange of clinical information between disparate systems delivered as a network service. IHE defines the standards framework and business rules that enable the exchange. For more details on IHE, see http://www.ihe.net.
The MDES implements the complete stack of services, registries, and repositories required to provide seamless interoperability between disparate health information systems (see Figure 3). Depending on the scale of the implementation and the location within the network, this stack runs on Cisco Services-Ready Engines (SREs) residing in a Cisco Integrated Services Router (ISR) or as a virtualized service on a Cisco Unified Computing System (UCS).
Figure 3 Cisco MDES Building Blocks
Although disparate systems may comply with the various healthcare standards, they still may not be able to directly communicate. For example, different EHR systems may implement different versions of Health Level Seven International (HL7) such as 2.5 and 3.0. The Cisco MDES solution is able to bridge the syntactic gaps that exist through its advanced transformation services.
The MDES incorporates a master patient index and translation function directly into the network infrastructure. Patient records remain where they were created, but are accessible from anywhere within the healthcare network. The patient index creates a registry of patients shared across the data exchange environment. Entities within this environment may be either consumers or publishers of information. Figure 4 shows the basic functions provided to a publisher and consumer. A publisher needs an onsite MDES instance. A consumer that does not publish information may not need an MDES instance, depending on the capabilities of the local EHR or practice management system. There are several ways to integrate with systems to make participation in the exchange easier.
Figure 4 Solution Overview
MDES Architecture
Overall Architecture
There are several approaches to providing ubiquitous access to medical records. One is to create a central clinical data store. With this approach, all the medical information is uploaded to a central location. As the data is transferred to the central site, everything is converted to a vendor-independent format. The result is rapid, easy access to standardized records with redundancy and simplified backup.
However, this approach requires a very large central store to contain all the medical information. Although this approach may work well at the local and possibly regional level, it does not scale for larger deployments.
The second approach is to leave the medical data distributed at the local level and provide a means of indexing the patients and matching them with their records. This federated approach is extremely scalable because the only data that needs to be distributed through the network is the indexing data, which is very small. Because access to large files is usually confined to the local system, bandwidth typically is not an issue. Although the IHE framework is flexible enough to support either deployment model (and because the MDES is an IHE-based solution, the MDES supports either deployment as well), the recommended deployment model for the MDES is the federated model.
The system is deployed in a federated structure, as shown in Figure 5.
Figure 5 Federated Deployment Model—Logical View
The MDES solution communicates with local record repositories such as the clinical information system (CIS) and other IHE-compliant healthcare systems; and extracts patient indexing information and information on the documents associated with the patients. The information collected is then stored in a database residing on the compute module (at this level, this is typically a Cisco SRE unless a Cisco UCS is already deployed in the facility). The patient and document indices that were created are replicated one level higher in the hierarchical model to provide for high availability and redundancy. The Patient Index information is replicated to the highest level patient index (SuperPIX) in the system, or to a third-party Master Patient Index (if one is deployed). As long as patients stay within their local system, the document indexing data is never replicated beyond the two lowest level systems. When patients visit a remote facility outside of their local healthcare system, the remote (requesting) system securely locates the patients' local healthcare system and provides the remote physician with a comprehensive list of patient information. An authorized care provider can access the records that their access rights allow, keeping the entire transaction in compliance with local privacy laws. When remote records are created and associated with a local patient, links are created between the local and remote records.
Any data translations that need to occur are performed by the network infrastructure and do not require additional software or customization to the clinical information systems involved in the information-sharing transaction.
New medical information created during the treatment of the patient at the remote site resides at the remote site. This new information is made visible to the patient's local physician as their EHR systems are made aware of the new medical information. This meets the overall goal of achieving a seamless, patient-centric view of a patient's records and access to those records no matter where they are located.
Although there is a reference client implemented in the Tiani Spirit software, the typical end systems in the solution are third-party clinical applications. If no other client is available, the reference client in the solution may be used to access data in the MDES network by browsing into the MDES node. Depending on the size of the facility, specific application and hospital policy, the end systems are either department-based or deployed in a centralized data center.
MDES Software Overview
Tiani Spirit software is the key component of MDES. SpiritEHR is a standards-based implementation of the IHE software stack.
Integrating the Healthcare Enterprise
IHE is a global non-profit organization that has developed a framework of standards that allows disparate systems to exchange medical data. The IHE defines integration profiles within various domains. Profiles consist of actors that perform certain roles, transactions that may be performed between these actors, and the formats and protocols used to execute transactions. There are currently eleven defined domains including radiology, cardiology, pathology, and IT infrastructure. For example, there are twelve profiles within the IT infrastructure domain, including Cross-Enterprise Document Sharing (XDS) and Patient Identifier Cross Referencing (PIX). Figure 6 shows the XDS profile. The actors are the entities within the boxes. The transactions are labeled on the lines between the actors. As an example, the transactions in which the Document Consumer actor is involved are "Query Registry" and "Retrieve Document". For a list of all the actors implemented by the SpiritEHR software, download the latest IHE Integration Statement from http://www.tiani-spirit.com.
Figure 6 XDS Actors and Transactions
Profiles use existing information standards such as HL7, Clinical Context Object Workgroup (CCOW), Electronic Business using eXtensible Markup Language (ebXML), Clinical Document Architecture (CDA), and PDF for document formats and to execute these transactions. In the previous examples, the Query Registry transaction uses the SQL query language. The e-business registry information model (ebRIM) defines the information model and defines the subset of SQL that applies. The query can return metadata including the document URI. The Retrieve Document transaction retrieves the document using an HTTP GET. The document repository returns the exact byte stream that was delivered to the repository in the Provide & Register Document Set transaction between the Document Source and Document Repository actors. For specific information on IHE and the profiles MDES supports, see the following URL: http://www.ihe.net.
Every year connectathons are held by IHE in the United States, Europe, and Asia/Pacific in which vendors supporting IHE profiles test their compliance and interoperability. The Cisco MDES complies with 79 IHE stars (actor/profile tests) based on the most recent North American IHE Connectathon 2011 testing. IHE Connectathon results are available at http://sumo.irisa.fr/con_result/.
The IHE framework uses the concept of affinity domains. An affinity domain is a well-defined set of document registries/repositories and document sources/consumers that have agreed to share clinical documents. An affinity domain can be as small as a single provider organization or hospital, or can be much larger. In the logical data structure in Figure 5, an affinity domain is a Regional (orange) node, and the local (yellow) nodes associated with that Regional node. This allows for a very small scale implementation (one or two hospitals or groups) to implement the solution. As multiple instances of affinity domains are implemented over time, their simplified interconnection allows the healthcare providers to easily share their clinical information. This allows the system to grow from a few organizations to a larger regional, national, and international scale, avoiding the huge up-front investment of implementing from the top down with no guarantee of success.
SpiritEHR Software Architecture
SpiritEHR implements all essential server-side and client-side components for an electronic health record, including components for the following:
•
•
•
•
Figure 7 shows the functional stack implemented in SpiritEHR.
Figure 7 MDES Functional Stack
All SpiritEHR modules are implemented as J2EE applications and are deployed as a single J2EE archive (.ear) on a JBoss Enterprise Application Server.
Although the default database server deployed with MDES is MySQL, Tiani Spirit also supports Oracle, MS-SQL, and Hypersonic.
A graphical user interface based on Flash is provided for configuration and monitoring.
Figure 8 shows an overview of the SpiritEHR architecture.
Figure 8 SpiritEHR Software Architecture
Because SpiritEHR is based on a number of IHE profiles, the actors implemented may change over time as additional functionality and new profiles or domains are added to the IHE framework. To determine the actors currently implemented, see the download section of the latest IHE Integration Statement at the following URL: http://www.tiani-spirit.com.
The solution is capable of communicating with many non-IHE-compliant systems through the use of protocol adapters. These adapters are provided to allow the connection of non-IHE legacy systems such as hospital information systems and other ancillary systems throughout the healthcare enterprise. The adapters provide functionality for protocol conversion and implement various IHE actors and the business logic required to facilitate a completely connected ecosystem.
Although a SpiritEHR installation always contains all the components, only the needed modules are active at runtime, depending on the use case. The following core components are always required:
•
•
•
•
OpenLDAP is the default identity service deployed with MDES, but other identity services are supported.
Use Cases
The use cases are snapshots of parts related to specific levels within the federated deployment model previously described.
These levels are as follows:
•
The enterprise level relates to a single patient identity domain (one patient identity source actor). SpiritEHR provides several adapters that implement different IHE actors and allow non-IHE compliant systems that comply with a standard protocol (such as HL7) supported by SpiritEHR to communicate with the MDES system.
•
This level represents one XDS affinity domain and connects one or more patient identity domains, depending on the use case. This level includes an XDS registry one or more XDS repositories, usually a PIX manager and a PDQ supplier and the security-related parts (audit record repository, authentication, and authorization modules).
•
This level allows communication between different XDS affinity domains. A PIX manager and a PDQ supplier are provided for cross-community patient management.
Use Case—Small Community with a Central EHR
The participants (clinics, medical practices, and small hospitals) use a common EHR to share information, as shown in Figure 9.
Figure 9 Small Community Overview—Hierarchical View
Legacy clients (for example, hospital information systems) use a locally installed SpiritEHR to communicate with the centrally shared EHR system. The local installation provides adapters for validation, protocol conversion, and queuing messages.
IHE compliant clients communicate directly with the centrally shared EHR system.
The centrally installed web application is used to provide communication for participants without a medical information system. This is a limited functionality reference client; a full-function EHR is recommended whenever possible.
The centrally shared EHR components can be scaled depending on the participants' requirements (number of concurrent users, performance) and the total amount of shared data (number of patients/documents).
Use Case—Single EHRs with Cross-Community Communication
In this use case, each participant (typically hospitals or large clinics) has its own MDES instance for IHE-compliant patient and document management.
Laterally aligned communities can be collected hierarchically in any order, depending on the required cross-community communication.
Cross-community communication can either be done "on demand" following a user request, using the Cross Community Access (XCA) and/or Cross Community Patient Discovery (XCPD) profile, or "event driven" by using the system's adapters to forward information automatically. Changes to cross-community referenced patients are distributed to all involved participants.
IHE-compliant clients within the community (inside or outside the hospital) can communicate directly with the hospital's EHR system.
The web application may be used from within hospital's departments to access medical information if another solution is not available.
Figure 10 shows a cross-community overview.
Figure 10 Cross-Community Overview
To provide system interoperability and the exchange of medical data, MDES must provide a number of services, including the following:
•
•
•
•
Document, patient, and provider directories are implemented to provide the indexing required for these services. To comply with regulatory requirements, the ability to provide an audit trail of access to electronic protected health information (ePHI) must be provided. MDES provides a robust audit repository capability to fulfill this key requirement. (See Figure 11.)
Figure 11 MDES Software Stack
Data Flow Within Node
A message arrives on a TCP connection, which is associated with a configured IP port. The TCP listener thread passes the data to the adapter(s) associated with the connection. The adapter may do a variety of things, including one or more of the following:
•
•
•
•
•
–
–
Figure 12 shows an example of the connectivity between a hospital and a clinic.
Figure 12 Adaptor Work Flow
Cisco Solution Components
The Cisco MDES solution assumes that there is an existing network compliant with Cisco MGN 2.0 architectures. There are currently architectures for campus, wireless, and security, with other network areas in development. The documents defining these architectures can be found by visiting www.cisco.com/go/designzone and selecting Healthcare, then Cisco Medical-Grade Network 2.0.
MDES provides a network service, with the primary component being the SpiritEHR software. Because of the nature of the provided service, the application is typically distributed throughout the network. A variety of Cisco compute platforms are used, depending on the scale of the deployment, and where in the network it is deployed. For example, within a hospital with 100-200K patient visits a year, a single router with a pair of SREs provides sufficient scalability. By adding a second router with a pair of SREs, the scale expands to support up to 400K patient visits a year. For larger hospitals, a B or C Series Unified Computing System should be used. For aggregation nodes, a B or C Series Cisco Unified Computing System is more appropriate.
Services Ready Engine
The Cisco Services Ready Engine (SRE) modules are router blades for the second generation of Cisco ISR G2s that provide the capability to host Cisco, third-party, and custom applications. SRE modules come in ISM and SM form factors. MDES is not supported on the internal (ISM) model. Table 1 lists the features of the supported SRE modules.
SRE modules provide the compute platform onsite in hospitals, clinics, and large physician practices. The router platform hosting the SRE also provides routing/switching, security, and unified communications services.
The platform supported for the onsite systems is the Cisco UCS Express, which comprises the following:
•
•
•
•
Figure 13 shows the Cisco ISR Generation 2 and SRE.
Figure 13 Cisco Integrated Services Routers Generation 2 and Service-Ready Engines
This is typically deployed with two SRE modules; one running the application and web server, and the second hosting LDAP and the database application. In light use environments, the solution may be deployed with a single SRE.
For more information on SREs and UCS Express, see http://www.cisco.com/go/ucse.
Cisco Unified Computing Platform
The Cisco UCS integrates a low-latency unified network fabric with enterprise-class, x86- architecture servers along with virtualization resources (see Figure 14). The system is an integrated, scalable, modular platform in which all resources participate in a unified management domain. A single system scales to up to 40 blade chassis, 320 compute nodes, and thousands of virtual machines.
Figure 14 Cisco B Series Unified Computing Systems
Cisco UCS C-Series Rack-Mount Servers extend the Cisco UCS to a rack-mount form factor, including a standards-based unified network fabric, Cisco VN-Link virtualization support, and Cisco Extended Memory Technology. (See Figure 15.)
Figure 15 Cisco C Series Unified Computing Systems
Cisco UCS B- and C-Series servers provide the MDES compute platform when the SRE-based solutions do not provide sufficient scale, and in locations where the UCS platforms are already deployed. In the UCS environment, the MDES components are deployed as virtual machine instances. This testing uses VMware's vSphere Enterprise 4.1, which allows easy migration of virtual machines between the compute platforms.
For more information on the Cisco Unified Computing System, see http://www.cisco.com/go/ucs.
Network Architecture
Although not a requirement for the solution, a number of Cisco products can greatly enhance the performance and security of the solution. MDES handles electronic protected health information (ePHI), is deployed across a WAN, and requires high availability and the ability to balance users across multiple servers. The next subsections describe products that provide WAN optimization, offload SSL processing, and balance users across multiple servers.
Wide Area Application Services (WAAS)
Cisco Wide Area Application Services (WAAS) is a powerful application acceleration and WAN optimization solution that optimizes the performance of any TCP-based application operating in a WAN environment. WAN optimization is achieved by providing:
•
•
•
MDES data transported outside the firewalls of the organization should be encrypted. WAAS includes SSL optimization features that integrate transparently into the existing PKI trust model in customer deployments, and can be easily deployed without compromising the existing data center key management security.
WAAS runs on a variety of hardware platforms, including modules that fit in a Cisco ISR as well as appliances for both central site and remote locations (see Figure 16). For more information on WAAS, see http://www.cisco.com/go/waas.
Figure 16 WAAS Hardware Platform
Application Control Engine (ACE)
The Cisco Application Control Engine represents the next generation of application switches, delivering tightly integrated, essential application service functions in a single powerful system. It provides load balancing and content switching functions with granular traffic control based on customizable Layer 4 through 7 rules. The ACE provides the following services:
•
•
•
•
•
•
•
For large MDES or hosted deployments, ACE can provide high availability and distribute user load with server load balancing. It can also enhance server performance by taking over SSL processing and TCP processing using SSL offload and server TCP offload, freeing the servers to do additional application processing.
The ACE is available as a module for the Catalyst 6500 or as an appliance. For more information on ACE, see http://www.cisco.com/go/ace.
Figure 17 shows the MDES network architecture.
Figure 17 MDES Network Architecture
Solution Component Versions
Table 2 lists the specific components with software versions that are associated with the solution.
Cisco Medical Data Exchange Planning and Design
End Node Considerations
End nodes are typically deployed on Cisco UCS Express systems in Cisco ISRs onsite. Depending on the volume of traffic, the UCS Express may contain a single SRE or two SREs. Again, depending on volume, these SREs may be the 700 Series or 900 Series. UCS Express runs VMware ESXi as its virtualization engine. Applications (including MDES) run in a virtual machine on top of ESXi. Cisco UCS Express Version 1 (the version used in validation testing) uses VMware ESXi 4.1.
Very small installations (physician groups or small clinics) may share a single end node instance hosted at a central site. This instance may run on a UCS Express system or as a virtual machine running on a UCS depending on the scale of the deployment. Because the majority of traffic is HL7, and HL7 typically is not encrypted, all traffic leaving a facility must be secured over a private network or an encrypted VPN.
Onsite versus Hosted
Because MDES runs on IP, there is no absolute requirement that the MDES node be collocated with the clinical systems. As long as there is sufficient bandwidth and reliability, low enough latency, and sufficient security, the MDES solution may be hosted in a central location. The data involved is protected health information, so any data leaving the healthcare organization premises must be transported over either a private network or the data must be encrypted.
Data Center Considerations
The data center infrastructure should follow the best practices as defined in the data center CVDs at http://www.cisco.com/go/designzone. The validation lab used a pair of Cisco Catalyst 6509s as a core and a pair of Cisco Nexus 5010s as access. Because the data center consisted of a pair of Cisco UCS B Series blade servers and three Cisco UCS C Series rack servers, the distribution and services layers were skipped. A production environment is typically much larger that this, so those layers should be implemented. In fact, there is probably an existing data center infrastructure into which the solution will be integrated. The existing infrastructure should be evaluated and if it is not compliant with Cisco data center best practices, it should be upgraded to be compliant.
In the recommended deployment model, the MDES image in the edge node is replicated in the data center. When a transaction is performed on the edge node, it is replicated on the copy in the data center. This provides a real-time backup and allows for high availability. If documents are stored in the MDES repository, remote requests can be served from the data center, lowering the bandwidth requirements between the end node and the data center as well as the load on the onsite clinical system. There are nodes also required for aggregation and potentially a superPIX (although these functions may be consolidated in a single node depending on scale). In any but the smallest deployments, running these additional nodes on routers does not make sense. They may run on UCS C Series rack servers or, if the scale demands, on UCS B Series blade servers.
Security Considerations
In healthcare, patient privacy and security is always a primary concern. The Canadian Personal Information Protection and Electronic Documents Act (PIPEDA), the United States' Health Insurance Portability and Accountability (HIPAA) Act, Europe's EC 95/46 Directive, and Japan's HPB 517 regulate the privacy of identifiable patient information. It is critical that this information be protected both at rest and as it transits the network. Access by unauthorized individuals must be prevented, authorized individuals should have access only to the information they are entitled to access, and any access should be logged so the access can be audited. Compliance with the regulations referenced above requires a number of systems and policies, and while MDES cannot ensure compliance, with proper configuration and deployment the SpiritEHR application and Cisco infrastructure provide the mechanisms required to accomplish this level of security for the application and network.
Network/Data Center Security
The equipment is assumed to be located in a physically protected and actively monitored area. This is typically the case within the data center and for networking equipment. The threat of equipment modification should be protected against by means of physical security mechanisms. Remote locations, especially home offices, are not physically protected. Whenever access is extended outside of the enterprise, data should be encrypted complying with Cisco best practices. In addition to physical security, there must be protection against network access by unsupervised systems. This should be provided by deploying Cisco network security best practices as defined by the Cisco SAFE architecture. Design guides for the various network locations can be found by browsing to http://www.cisco.com/go/designzone and selecting "Security".
Several IHE integration profiles apply specifically to security, and all integration profiles have security components. The profiles that are specific to security include the following:
•
•
•
•
Enterprise User Authentication (EUA)
EUA defines a means to establish one name per user that can then be used by all of the devices and software that participate in the integration profile facilitating center.
Audit Trail and Node Authentication (ATNA)
The ATNA IHE integration profile establishes security measures that with the security policy and procedures provide patient information confidentiality, data integrity, and user accountability. ATNA establishes the characteristics of a basic secure node. The profile defines the following:
•
•
•
•
•
Cross Enterprise User Assertion (XUA)
Each enterprise provides its own mechanism for the authentication of systems, users, and applications that may or may not be compatible with the authentication mechanism of another enterprise. If information is to be exchanged between enterprises, a means must be provided to assure the providing entity that the requesting entity is authenticated and authorized to access the requested information. The XUA leverages web-services security, Security Assurance Markup Language (SAML) 2.0 Token Profile, and various profiles from W3C and OASIS to provide this mechanism. This is also required to create the proper audit trail.
For detailed information on these IHE integration profiles, see the IT Infrastructure Technical Framework at the following URL: http://www.ihe.net/Technical_Framework/index.cfm#IT.
The SpiritEHR software implements all of these integration profiles.
WAN Considerations
Traffic between MDES nodes can and should be encrypted, but traffic between clinical systems and the MDES node may or may not be encrypted. This is protected health information, so provisions must be made to secure the data any time it leaves the healthcare facility.
Firewall Traversal
Because MDES is distributed across the network, depending on the specifics of the network there is a chance that parent/child pairs will be separated by a firewall. Depending on the specifics of the network, a security policy may need to be configured to allow traffic between nodes.
Communications between parent/child nodes occur over HTTP, HTTPS, HL 7, and possibly LDAP (if LDAP replication is used). Table 3 lists the default ports.
If the high availability scenario using the onsite-clone in the data center is used, additional ports may be used. All ports are completely configurable through the SpiritEHR web interface, so the team integrating and configuring the SpiritEHR application should be consulted for specific port numbers.
High Availability
Onsite
Duplicating the primary UCS Express system and router can provide high availability onsite. Load can be shared between the two systems. The nodes can be configured so that any transaction directed to one system is forwarded to the other as well. This provides a real-time active backup of all data transactions. Policy routing can be configured so that if one node fails, the traffic is forwarded to the active node. This redundancy may be in addition to or instead of the recommended clone at the data center. If both onsite nodes fail, policy routing can be configured to forward all traffic to the clone in the data center (if implemented). Web services must be enabled on the clone in the data center for this level of redundancy to work.
Data Center
The first aspect of high availability in the data center is the end node clones hosted in the data center. They rely on the ability of the MDES node to forward all transaction requests to another node, and rely on no data center redundancy functions.
A second instance of each VM running in a backup data center provides optimal redundancy. If a second data center is not available, a second instance in the same data center is the next best option. Either of these is possible with additional configuration of each node and the node's neighbors. If resources are not available to run everything redundantly, VMware High Availability may be used. VMware High Availability does not run duplicate instances of each VM, but monitors virtual machines to detect operating system and hardware failures. When server or operating system failures are detected, the VMs are restarted on other physical servers within the resource pool without manual intervention. Although there is a delay while the new instance is started, when an MDES node is unable to forward a message, its queuing mechanism retries until it is successful, so no messages are lost.
Network/Database Services
MDES relies on a variety of network support services. Following are an explanation of each and how they are used.
LDAP
In addition to user authentication, the Lightweight Directory Access Protocol (LDAP) is used to store organizational data and system configuration information. MDES requires OpenLDAP version 2.3 or higher. If OpenLDAP is available in the organization, it may be used. If OpenLDAP is not available, it must be installed for the MDES application. Although OpenLDAP is an implementation of LDAP V3, and theoretically any implementation of LDAP V3 can be used, the only implementation tested with the solution is OpenLDAP.
Database
MDES provides document, patient, and provider registries and audit and document repositories, all requiring the ability to store data. This data along with additional configuration information, variables, and error logs are stored in the database. The default database for MDES is MySQL. Version 5.0.77 was used for validation testing. In addition to MySQL, Tiani Spirit has tested and implemented with Oracle. See the latest Tiani Spirit documentation for the supported versions of these databases.
Network Time Protocol
For regulatory reasons, MDES must log all transactions that access ePHI. The solution is typically deployed across multiple sites, potentially across a large geographic area. Troubleshooting problems without accurate time stamps presents a challenge. Accurate time stamps are a requirement for both of these reasons. The most accurate way to synchronize time across a network is Network Time Protocol (NTP). If NTP has not been previously deployed, it must be deployed in a redundant and trusted fashion for the MDES solution.
Domain Name Services
Each MDES node maintains a table of all of the other nodes in the network, and may need to communicate with any of the other nodes in the network. This mapping is by host name, so a mechanism must be in place to map host names to IP addresses. Domain Name Services (DNS) is the network service responsible for this mapping. Each MDES host should be registered with DNS and should be able to access DNS as a client.
MDES Software Requirements
Information in this section is primarily from the SpiritEHR Technical Manual, version 1.8.1 from Tiani Spirit. See this manual for more detailed installation and configuration information.
Hardware
The SpiritEHR software is extremely flexible and may be configured in a variety of ways (including centralized data storage), making hard and fast requirements difficult. However, the same general requirements apply for utilization up to a certain capacity, except for the available storage space for image and document data (RAID), which varies according to the requirements.
•
•
•
•
•
•
•
•
Table 4 lists the estimates of the starting requirements for storage. This may grow, depending on usage and configuration.
Operating Systems
The preferred operating systems are Red Hat Enterprise Linux 4 or higher, Red Hat Fedora Core 5 or higher, or CentOS 5. Usually older versions can also be used without problems, but beginning with RH-EL4, an appropriate LDAP-Server is supplied with the distribution, instead of having to be manually compiled and installed separately.
Other Linux distributions are in principle possible; consult with Tiani Spirit for confirmed compatibility.
The Cisco CVD testing was done using CentOS 5.5 and RHEL 5.5.
Virtualization
Cisco UCS Express runs VMware ESXi as the virtualization layer on the SREs. The version current with CVD testing is ESXi 4.1. ESXi 4.1 was used for virtualization on the UCS system in the data center as well.
vCenter 4.1.0 was used to manage the virtual machines running on the UCS. UCS Express currently does not support vCenter, so it must be managed using a vSphere client instance to connect to each SRE individually.
Solution Logical Design
The logical design of the solution needs to be defined. As described above, this is normally a federated data design with a hierarchical patient index terminating in a SuperPIX at the apex. There is a lot of flexibility in the software, so other topologies may be implemented if required by the customer situation. As described in the Data Center section above, it is recommended that the onsite node be cloned at the data center, allowing for a real-time backup. If the document store capability of MDES is used, documents are automatically backed up at the data center at the time of publication. Figure 18 shows the logical design used in the Cisco lab for CVD testing.
Figure 18 MDES CVD Lab Logical Design
In the data center are two aggregation nodes. This was to emulate two organizations. For an installation within an organization, a single combination aggregation/SuperPIX node should be sufficient. The hosted nodes are housed in an ISR 3945 in the data center housing four SREs. These could just as easily have been hosted on virtual machines in the UCS.
The logical design needs to be mapped into the schema. The schema defines the data structure used by the system. In a production environment, the application partner implementing the software lays out the schema. The high-level object identifier (OID) and affinity domains were modified in a sample schema provided by Tiani Spirit for the CVD lab implementation. For the lab, a descriptive string (ISEHC-Industry Solutions Engineering HealthCare) was used as the base OID because access was limited to a lab. In a production system, a valid registered OID should be used. This is something such as 2.16.17.710.777, so the OID for XDS affinity domain in Figure 19 would be 2.16.17.710.777.2.1.1 instead of ISEHC.2.1.1. Figure 19 shows the schema used in the lab.
Figure 19 Sample Schema
Figure 20 and Figure 21 show sample site mappings from the lab. The complete schema and mappings are located in Appendix A—ISEHC OID Schema.
Figure 20 OID Mapping (1)
Figure 21 OID Mapping (2)
Clinical Systems Integration
The purpose of this solution is to interconnect disparate clinical systems, so the integration with those systems is a key component of the deployment. Application partners who are familiar with the protocols involved and have been trained by Tiani Spirit or one of their training partners perform this function. Although integrating clinical applications is beyond the scope of this guide, an overview of the integration methods are described below.
There are a wide variety of clinical applications developed over a long period of time complying with a number of standards and various versions of those standards, as well as proprietary implementations. To achieve the patient-centric record, these applications need the ability to interface with MDES. There are several ways to interface with MDES, depending on how compliant the clinical application is with IHE. Table 5 lists the various scenarios and appropriate integration methods.
IHE Compliant
Configuration of MDES is through the Transaction Broker web-based interface. Figure 22 shows a sample screen.
Figure 22 Transaction Broker HL7 Configuration Interface
In this scenario, the actors on each end (clinical application and MDES) need to be configured and associated with their complimentary actors on the other end.
Non IHE Compliant—Supported Standard Compliant
In this scenario, an adapter must be used. An adapter is the IHE interface for systems that are not IHE compliant; for example, the medical document management MDM Adapter converts from HL7 to ebXML. An adapter validates the incoming message, checking syntactically for correctness and semantically for any existing project-specific restrictions. Configuration of the adapter is through the same Transaction Broker web-based interface.
Web Services
The final recommended integration method is with web services. Actually, Tiani Spirit has implemented a software layer that simplifies the implementation of IHE-standard transactions and includes business logic needed to combine various client-side IHE actors and their transactions for communications with the MDES backend system. This range of functions reduces the development effort needed to connect to MDES. This software label is called EhrBase. A developer can connect to EhrBase in two ways. This first is via web services and the second is to use EhrBase as a Java library. The second option is available only with close development cooperation with Tiani Spirit.
Figure 23 shows an overview of web services.
Figure 23 Web Services Overview
Tiani Spirit has made it very simple to interface with their software. For the developer who insists on doing everything themselves, they can use the SpiritEHRBase-WebServices wsdl/xsd files and generate the client-side code. Tiani Spirit has also developed a WebServices client called SpiritEhrWSClient. This does not include all available services provided by the SpiritEhrBase-WebServices, but does provide core functionality. The source code for SpiritEhrWsServer is available, and there are several ways of using it to simplify the client-side development. Finally, the easiest approach to accessing EhrBase functionality via web services is the EhrWsInterface Java interface.
All of this is thoroughly documented with appropriate source code, sample code fragments, and all of the necessary .jar, .war, and wsdl files on a development wiki. Access to the wiki is restricted. Contact Tiani Spirit for access details.
Cisco Medical Data Exchange Implementation
Recommended Supporting Applications
There may occasionally be a need to install various applications to troubleshoot or support Cisco MDES. The items described in the following subsections have been installed and used during the installation, configuration, and troubleshooting process. If used properly, they have not exhibited any issues with regard to stability and overall operation of MDES.
WebEx (Version 8.5)
Cisco WebEx is an Internet-based service offering that provides the ability to securely collaborate in real time with members who have been invited to an online conference. The service is often used by enterprises to provide collaboration with their employees. The collaboration allows members of the conference to IM and seamlessly share an application or entire desktop. In addition to online collaboration, WebEx also allows the recording of both the online session as well as the audio conference that is being shared by the attendees.
WebEx was used extensively to allow collaboration between Tiani Spirit engineers in Vienna, Austria, and Cisco engineers in Atlanta and San Jose to install, configure, and troubleshoot a lab in San Jose.
To access WebEx or to set up and account, visit http://www.webex.com.
MySQL Query Browser Tool (Version 1.2.12)
MDES extensively uses its database. Viewing and modifying the database is frequently required in troubleshooting the MDES software. Although there is a SQL browser built into the GUI management tool, sometimes access is required when the application is not running or the GUI is not available. The MySQL Query Browser tool provides a GUI to view and if necessary manipulate the data contained in the various tables within the MDES database. Changing the wrong value can affect the application in unpredictable ways, so you should work with Tiani Spirit or one of their application integration partners to ensure that you understand changes before making them.
jExplorer (Version 3.2.1)
In addition to the database, MDES uses LDAP extensively to store configuration, user, and variable data. In some cases (for example, when the admin user account times out), it is useful to explore and manually manipulate the directory. jExplorer provides a GUI to view and if necessary manipulate the data in the OpenLDAP directory (see Figure 24). Changing the wrong value can affect the application in unpredictable ways, so work with Tiani Spirit or one or their application integration partners to ensure that you understand changes before making them. During Cisco Validated Design (CVD) installation, configuration and testing phases, version 3.2.1 was used without adversely affecting the application.
Figure 24 JXplorer
WireShark
WireShark is a freely available packet sniffer application. This application is often used when troubleshooting network communications. Although the application logs were sufficient to troubleshoot application problems, WireShark was helpful in troubleshooting network problems and documenting traffic flow.
The version of WireShark used was version 1.0.15 and is shown in Figure 25.
Figure 25 WireShark
To reduce the memory overhead, Cisco recommends that WireShark be run only for short testing cycles of time, and that it not be used in a manner that ultimately consumes available memory. For traces that need to span multiple hours or days, Cisco recommends configuring WireShark to write these traces to a packet capture trace file when they reach 10-15 MB in size. It is also recommended that a capture filter be applied to the end systems so that the product does not capture all traffic that is seen, such as remote desktop-sharing protocols and other similar applications.
VNC (Version 4.1.3)
VNC is a freely available application that permits remote control of a Windows-based workstation or server (see Figure 26). The utility has been used to effectively allow remote access to MDES without negatively affecting availability.
However, the use of VNC may violate some security policies of healthcare delivery organizations. When VNC is used, Cisco recommends using encryption and strong passwords. This includes the remote viewer workstation that may have the ability to cache the userid/password and may thus bypass the required authentication policies in place.
There are multiple implementations of VNC, many of which are free. The package used for testing was RealVNC for Windows and Linux. It is offered in both free and paid versions. The official website for obtaining RealVNC is http://www.realvnc.com.
Note
Figure 26 Remote Access Client—VNC
VMware vSphere Client
The vSphere client is an application that allows for the management of an individual virtual server by pointing it directly to the IP address of the server; or it acts as the control interface into vCenter by pointing it to the IP address of the vCenter application, allowing for the management of multiple virtual servers (see Figure 27). It may be downloaded by browsing into the IP address of a server running vCenter. UCS Express currently does not support vCenter, so each SRE must be managed individually by pointing the client to the hypervisor address on each SRE. UCS supports vCenter, so the vSphere client provides the interface into vCenter to manage the UCS virtual environment.
Figure 27 vSphere
VMware vCenter
VMware vCenter Server provides unified management of all hosts and virtual machines in a data center from a single console. Management servers provide central management points for hosts and virtual machines, with inventory and performance information stored in a database. A VMware vCenter agent provides connectivity between the host and management server. Administrators can access VMware vCenter Server from the VMware vSphere Client from any Windows PC, or use the vCenter Web Access portal for remote access from any web browser. Version 4.1 of both vCenter and the vSphere Client was used for the CVD testing.
ISR Configuration
MDES is typically deployed in the edge node on a UCS Express system consisting of a pair of Cisco Service-Ready Engines (SREs) in a Cisco ISR router running VMware ESXi virtualization software. Usually, one SRE hosts the MDES applications and web services, and the second hosts LDAP and MySQL. The UCS Express may be ordered as a bundle, including the router, a single SRE module, and the SRE-V license. Some bundles also include switching. Additional SRE modules should be added and configured for UCS Express.
Router Configuration
For validation testing, the router interfaces associated with the service modules were configured according to the Installation and Configuration Guide for Cisco Services Ready Engine Virtualization at the following URL: http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/sre_v/1.0/user/guide/sre_v.pdf.
Both Layer 2 switching and Cisco IOS Layer 3 routing were tested. To conserve public IP address space, unnumbered interfaces were used for the service modules. Figure 28 shows the configuration of the Cisco IOS routed configuration interfaces.
Figure 28 Cisco IOS Routed Router Configuration
The second tested configuration is MGF Layer 2 switched mode, as shown in Figure 29. This is the recommended configuration.
Figure 29 Layer 2 Switched Router Configuration
For step-by-step configuration details, see the Installation and Configuration Guide for Cisco Services Ready Engine Virtualization at the following URL: http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/sre_v/1.0/user/guide/sre_v.pdf.
High Availability Configuration
As typically configured, access to the MDES network is through an application running in an SRE on a local node, with each transaction being forwarded to a clone in a data center and/or a local backup system. If the local node fails, it is critical that the edge traffic be redirected to the clone or backup system. Several levels of redundancy are available for the MDES solution. In the recommended deployment, when a transaction is sent to the onsite node, the request is processed locally as well as forwarded to a clone located in the data center, effectively providing a real-time backup. The next redundancy configuration option is to provide a second node onsite with or without a clone in the data center. The data center clone is recommended to provide an offsite copy of the data. Queries from remote nodes may then be serviced in the data center, eliminating the requirement that each remote request be forwarded to the onsite system.
High availability may be provided in any of the scenarios. In the simplest form of high availability, the status of the onsite system is monitored, and when it fails, traffic is forwarded to the clone in the data center. Onsite redundancy and load balancing for several nodes may be provided by deploying the nodes and using round-robin DNS to distribute users between the two instances. High availability is provided by Policy-Based Routing (PBR), as described below. For larger deployments (or if the hardware is already deployed), server load balancing (SLB) may be used to provide high availability.
Clone-Provided High Availability
In this model, the onsite system is monitored, and traffic is forwarded to the clone in the data center if the onsite node fails. This is accomplished using PBR with tracking, combined with Network Address Translation (NAT). Failover is triggered in the event of an SRE hardware failure, a virtual machine failure, or a software failure.
PBR allows the router to track the availability of the application using an ICMP echo or HTTP get, and set the next hop based on the result. If the onsite application is available, the next hop is the onsite application. If the onsite application does not respond, the packet is forwarded to the next router in the direction of the clone. At that router, the address of the clone is substituted for the address of the onsite as the destination of the packets. As packets return, the address of the onsite is substituted for the address of the clone as the source of the packet. Packets between the onsite and clone should be excluded from translation (if that traffic exists, the onsite is alive). Figure 30 shows a sample configuration.
Figure 30 Hosted Hospital High Availability Configuration
HCWAN-3945 Configuration
In this configuration, the keep-alive used is an HTTP get to the management interface on the SpiritEHR software. This should ensure the application is alive (as well as the OS and hardware). Cisco IOS Software 15.1 code (required to support UCS Express) prior to 15.1(4)M had a bug, so the HTTP get probe always fails. The router should be upgraded to this revision of the code or later, or use an ICMP probe. The ICMP probe ensures only that the SRE and OS are alive.
HCCORE1-6509 and HCCORE2-6509 (with different IP addresses)
Onsite redundancy may either be deployed with or without a load balancer.
High Availability with Onsite Redundancy Without Load Balancer
In this configuration, a pair of onsite MDES nodes share the load (see Figure 31). When a transaction enters one node, it is forwarded to both the other onsite node and the clone. If one node fails, the other node takes over. If both nodes fail, traffic is forwarded to the clone in the data center. This is accomplished using a similar mechanism to the previous configuration, but each router monitors both the local MDES instance and the MDES instance running in the other onsite router. If the local instance is down, PBR is used to forward it to the other onsite node. Because the destination address in the packets is incorrect, a secondary loopback interface must be created on each of the onsite MDES instances with the address of the other onsite instance. If both onsite instances are down, PBR is used to forward the packet to the HCWAN router in the data center where the address translation takes place. A secondary loopback interface can be created in the clone, and this same method can be used in the clone-provided high availability scenario. This eliminates the requirement for NAT, but PBR must be configured on each intermediate hop between the onsite and clone. Round robin DNS should be used to distribute the users between the routers, and HSRP or VRRP should be configured in the network for high availability if a router fails. Figure 31 shows the network diagram.
Figure 31 Hospital with Dual MDES Nodes High Availability Configuration
Following are sample configurations.
HCH1 Configuration
HCH2 Configuration
HCWAN Configuration
Configuring Loopback VMs
Finally, a secondary loopback interface must be configured on each of the MDES instances. To do this, make a copy of the primary loopback interface file and modify the device, IP address, netmask, and network. The steps are as follows:
1.
[root@hostname]# cd /etc/sysconfig/network-scripts[root@hostname]# cp ifcfg-lo ifcfg-lo:12.
DEVICE=loIPADDR=127.0.0.1NETMASK=255.0.0.0NETWORK=127.0.0.0# If you're having problems with gated making 127.0.0.0/8 a martian,# you can change this to something else (255.255.255.255, for example)BROADCAST=127.255.255.255ONBOOT=yesNAME=loopbackFollowing is ifcfg-lo:1 modified for HCH1:
DEVICE=lo:1IPADDR=172.28.207.204NETMASK=255.255.255.255NETWORK=172.28.207.204# If you're having problems with gated making 127.0.0.0/8 a martian,# you can change this to something else (255.255.255.255, for example)BROADCAST=127.255.255.255ONBOOT=yesNAME=loopbackThe second instance must also be modified; 172.28.207.212 is substituted for 172.28.207.204.
High Availability with Onsite Redundancy Using Server Load Balancing
The final high availability option uses server load balancing to load balance between two (or more) local MDES nodes. This may be achieved using Cisco IOS server load balancing or an ACE module. If using Cisco IOS server load balancing, check with Cisco Feature Navigator to ensure your platform is supported. With server load balancing, a virtual IP is assigned on the switch, and this is translated to one of the MDES nodes based on a load-balancing algorithm.
SRE Configuration
No special configuration is required on the SRE for the application. If the bundles are ordered and additional modules are configured for UCS Express, the base software comes preloaded on the modules. If the bundles are not ordered, or UCS Express is not configured on the additional SREs, SRE-V licenses must be ordered and software installed on the modules. For step-by-step software installation instructions see the Installation and Configuration Guide for Cisco Services Ready Engine Virtualization at the following URL: http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/sre_v/1.0/user/guide/sre_v.pdf.
UCS Configuration
This design guide is focused on the unique components of the MDES solution and not the underlying infrastructure. The solution assumes that Cisco best practices are followed in deploying the data center and networking infrastructure. The Cisco UCS configuration is well-documented in the Smart Business Architecture and Data Center sections in the Cisco Design Zone at the following URL: http:/www.cisco.com/go/designzone.
MDES Software Installation
This section describes the setup of the software of the MDES solution, starting with the virtualization software through the basic setup and configuration of the SpiritEHR software. Integration with clinical applications is beyond the scope of this guide.
High Level Setup
The setup includes the following:
•
•
•
•
•
•
VMware Infrastructure
Best practices for creating the VMware infrastructure are documented in VMware Infrastructure 3 in a Cisco Network Environment at the following URL: http://www.cisco.com/en/US/solutions/ns340/ns414/ns742/ns743/ns993/landing_dcVirt-VMware.html
Although this document was developed for VMware 3.x and MDES uses VMware 4.1, the concepts and principles articulated have not changed. This document should be followed when deploying MDES.
Creating a Virtual Machine
Virtual machine requirements are the same for all of the compute platforms. A VM may be created on one compute platform and migrated to another. If the compute platform is to be a UCS Express, one VM should be created for the directory/database functions (LDAP and SQL), and a second for the backend and client (if the client is to be used). These should execute on two different SRE modules. A single VM may be used if the compute platform is to be a UCS B or C Series server.
vCenter may be used with UCS B Series blade servers and UCS C Series rack servers. When this document was created, vCenter support was not available for UCS Express, but was expected shortly. The same client is used for individual VMs on the UCS Express platforms or vCenter for the other platforms. Downloaded this client by completing the following steps.
Procedure
Step 1
Figure 32 Browse to ESXi
Step 2
Figure 33 vSphere Login
Step 3
Figure 34 Virtual Machine Creation
Step 4
Figure 35 Selecting Custom
Step 5
The available data stores are displayed, as shown in Figure 36. On an SRE, these correspond to the available drive(s). On B or C Series UCS, these data stores have been created when setting up the VMware infrastructure.
Figure 36 VM Datastore
Step 6
Figure 37 OS Selection
Step 7
Figure 38 Memory Selection
Disk usage varies by the specifics of the implementation. See Table 4 to estimate starting requirements. In a production environment, Raid-5, a SAN or comparable is required. For testing, each VM was allocated 50 GB on a NetApp SAN (or the local drives on the SRE for those instances running on SRE modules).
Step 8
Figure 39 Disk Creation
Install the Operating System
The following two sections are largely based on the SpiritEHR Technical Manual, Version 1.8, from Tiani Spirit, with their permission. For much more detail on the installation and configuration of the software, refer to that document.
To install the operating system, complete the following steps.
Procedure
Step 1
a.
b.
c.
d.
Figure 40 Boot to Setup
Step 2
Step 3
The machine boots into the BIOS setup screen.
Step 4
Step 5
Figure 41 Map ISO
Step 6
The VM boots from the mapped drive and begins the installation process.
The following information is based on Red Hat Enterprise Linux version 5.6 and CentOS 5.5. Both Red Hat and CentOS are available on either CD or DVD. Installing Red Hat in graphical mode (just click Enter) causes the prompts to match the sequence below.
Step 7
a.
b.
c.
If you choose static, be sure to enter the hostname, default gateway, and a DNS server(s).
d.
e.
f.
g.
Step 8
Step 9
Step 10
Database
This guide assumes the database to be deployed is MySQL. Oracle, MS-SQL, and Hypersonic are also supported by Tiani Spirit. See the latest Tiani Spirit documentation for information on support of these databases.
If the application is running on Cisco UCS, the application and the database run on the same virtual machine. If the application is running on UCS Express, the MDES application runs on one SRE and the database runs on a second SRE. For UCS Express installation, MySQL should be installed on the second virtual machine, but is not required on the virtual machine running the application.
If MySQL was not installed in the previous step, the following commands can be used to install the application.
The database created is timed with username/password of tmed/tmed.
Following is an example of the my.cnf file required in /etc.
To start MySQL, use the following command:
Other Necessary Software
Other necessary software includes the following:
•
[root@hostname]# yum search openldap[root@hostname]# yum install openldap-servers openldap-clients•
The Java packages are bundled with the Spirit software.
Installing SpiritEHR Software
The MDES application software is currently distributed as a tar file. This file includes the following.
•
•
•
•
Three LDAP templates are provided to simplify initial configuration, as listed in Table 7.
Table 8 lists the basic directories used by the Spirit software.
The current installation source software can be downloaded from http://www.tiani-spirit.com/spirit_install, and should be downloaded to /home/spirit/install. If other required software (for example, Java) is downloaded independently, that source should be located in the same directory.
Change to the install directory and extract the tar file.
This creates a new spirit-install directory with the files and directories listed in Table 9
Installing Spirit
Change to this directory and run the install-spirit.sh script. This does the following:
•
•
•
•
•
Initializing LDAP
Usually the configurations and the schemata of the LDAP server can be found under /etc/openldap.
The following Spirit schemata must be saved in the directory /etc/openldap/schema/spirit and referenced in the configuration file. You can find the Spirit schemata in the /spirit-install/config/ [template]/ldap directory or in a build directory from a new version. They depend on a Spirit server version.
The following entries must be added or changed in the slapd.conf file.
Next, a basic configuration must be entered in the LDAP server. This is contained in the two files spirit-root.ldif and spirit-default-user.ldif. The configuration is set up by means of the commands listed below. The initial state can also be later reestablished in LDAP. There is a script for this purpose: /opt/spirit/sbin/renew-ldap.sh.
The ldif files referenced above vary depending on the template to be used. Table 10 lists the available templates.
The Gauteng template includes multiple templates for the various node types. Currently, there are templates for the following:
•
•
•
•
•
To initialize the LDAP server, you have to start init-ldap.sh with the ldif from the template you choose.
This script does the following:
•
•
•
•
•
Caution
Note
Initializing MySQL
In the next step, you must initialize the MySQL database. To set this up, use the script init-mysql-tmed.sh.
This script does the following:
•
•
Additionally, you have to create the tables for the Codewerk frontend with the command below. The file DumpForMySql.sql is in the spirit_install/config/[template]/sql directory.
By default, the server and client must be run on the same machine. If the database and applications are running on different machines, the following commands should be executed:
Setting Paths
To execute the Spirit scripts easily, set the following paths in file /etc/profile.
Updating Configuration Files
Copy all files from /spirit-install/config/[template]/bin to /opt/spirit/jboss-4.0.5.CR1/bin.
Checking Base Configuration
At this point, the software has been installed. Now you need to ensure that all of the required applications are going to be launched on system boot. Use chkconfig to ensure that NTP, MySQL server, OpenLDAP server, and the Spirit backend (and the Spirit frontend for those nodes that run the frontend) are configured to start at boot. Although all of the Spirit files should have the correct owner, ensure that is the case by changing the ownership to spirit.
If the deployment is on UCS Express, all of the services except the MySQL server and OpenLDAP server should be enabled to start at boot on the SRE running the core software. On the machine running support services, all of the services except jboss and jboss_web should be enabled to start at boot.
If a service needs to be changed, it can be accomplished with the command below.
Deploying on UCS Express
If running on UCS, the LDAP server, the SQL server, and the core software run on the same virtual machine. If running on UCS Express, the core software runs on a virtual machine on one SRE module and the LDAP server along with the SQL server run on a virtual machine on another SRE module.
The easiest way to create the two virtual machines is to create the primary machine and clone it using vCenter. Move the cloned machine to the second SRE and then turn on the appropriate services on each machine (see the previous section). The alternative is to do a complete install on one machine without OpenLDAP and MySQL:
1.
2.
3.
Either way, the following changes must be made.
Procedure
Step 1
host=localhostroot_dn=cn\=root,o\=spirit,c\=atroot_password={spirit}|Yl$y3z?%['yu|zh$1truststore=
Replace localhost with the name of the virtual machine running the LDAP (and SQL) server.
Step 2
This allows a client on any machine (with the appropriate credentials) to access the database.
Step 3
The localhost in red should be changed to the name (or IP address) of the VM running the MySQL database. The correct database should be specified in Transaction Broker.
After the system is installed and started, do the following.
Step 4
The screen shown in Figure 42 is displayed.
Figure 42 TBroker Login
Step 5
Step 6
The screen shown in Figure 43 is displayed.
Figure 43 Domain Selection
Step 7
The screen shown in Figure 44 is displayed.
Figure 44 TBroker Main Screen
Step 8
The screen shown in Figure 45 is displayed.
Figure 45 TBroker—Select Database
Step 9
SpiritEHR Software Configuration
This section describes the configuration of a basic system such as the one installed in the lab for testing. It does not include the integration of third-party clinical systems. In a production environment, integration with third-party clinical systems is a requirement. Configuration of the MDES software in a production environment should be performed by trained individuals with a thorough understanding of the MDES application, the protocols used by clinical systems, and the clinical systems to be integrated. This documentation does not provide sufficient information to complete that task. For more detailed information on the configuration of the MDES software, see the SpiritEHR technical manual, version 1.8, from Tiani Spirit. Before attempting to configure this solution, you should attend the integration courses offered by Tiani Spirit and their partners.
Using Config Mapper
ConfigMapper_done.xml is located at /opt/spirit/jboss/bin/CfgMapper. The Config Mapper file is used to easily set variables to be used by the Transaction Broker and other locations in the system. Because different parameters are used for the various node types, there are different Config Mapper files for each node type. The Gauteng profile contains Config Mapper files for the specific node types. Following is an example from an onsite node.
The variables in the Config Mapper file above fall into several categories. The first category is host mapping variables (red font). This set includes the following variables:
•
•
•
The second set of variables consists of descriptions (blue font). These are names and descriptions, with the exception of $SYSTEM_TYPE. This will be ON_SITE, ON_SITE_CLONE, PIX_AGGREGATION, or SUPER_PIX, depending on the node function.
The final set that must be modified is the OID mapping variables (green font). These correspond to the fields in the schema described in the Solution Logical Design (see the chart shown in Figure 46).
Figure 46 ISEHC OID Schema
The remaining entries are used by the SpiritEHR Web Client. Those that begin with $CDA are used as defaults for some of the healthcare provider fields. If the Web Client is to be used, "$HealthCareFacilityTypeCode" must be defined in the /opt/spirit/jboss/bin/XdsCodeMappings/AY_xdsCodes.xml file as a code under "healthcareFacilityTypeCode". Also, "$PracticeSettingCode" must be defined in the same file under "practiceSettingCode".
When the file has been modified for the local node, it should be renamed ConfigMapper.xml. When the spirit server is started, it checks in /opt/spirit/jboss/CfgMapper and if ConfigMapper.xml is there, it reads the contents and renames the file to ConfigMapper_done.xml.
Creating a Clone System
For a clone system, rather than starting with the default ConfigMapper_done.xml, start with the ConfigMapper_done.xml from the onsite node.
To copy the ConfigMapper and XdsCodeMappings from the onsite node, complete the following steps.
Procedure
Step 1
Step 2
Step 3
Step 4
Step 5
Starting the Software
At this point, the software is installed and may be started. The backend is the core of the system and is always required. It provides all of the IHE services and repositories, as well as the adapters to interface with external systems and manage the data. The second component is the reference client. This is a web client that may be used to provide access to the information in the backend. This is an optional component that may or may not be used. In the primary use cases, MDES is middleware that connects clinical systems, and these systems would be used to access clinical data through MDES, so the client would not be required.
During the installation of spirit-jboss, the corresponding runlevel script is also created under /etc/init.d/jboss. Thus JBoss can be started and stopped simply by entering the following:
If the client is required, it can be started and stopped by entering the following:
Testing Functionality
The first troubleshooting aid is the log file at /opt/spirit/jboss/server/spirit/log/server.log. As the application starts, all relevant information is logged here. If no errors are logged, the first hurdle is passed. Some minor errors may occur that do not impact the functionality, so the indication of some errors do not necessarily indicate the services have not been properly started. The log file /opt/spirit/jboss/server/spirit_web/log/server.log is the equivalent log for the web client.
The preconfigured services that should automatically start include the Web Administration Interface (WAI), Transaction Broker (TB), SpiritEHR (if the web client is to be used), and HL7. The WAI and TB are used to configure and manage the application, and SpiritEHR may be used to access information in MDES. The next step is the test the functionality of these services. Table 11 indicates how to test the pre-configured services.
Table 11 Testing Pre-Configured Services
8081
WAI and TB
http://hostname:8081/Spirit/web
8181
SpiritEHR
http://hostname:8181/SpiritEhrPortal
2400, 2398, 2399
HL7
Various functionality tests can then also be carried out in detail via the Web Administration interface and Transaction Broker. The URLs listed in Table 12 must be reachable and show a login page.
Table 12 Reachable URLs
WAI and TB
http://hostname:8081/Spirit/web
SpiritEHR
http://hostname:8181/SpiritEhrPortal
Default login for all three pages is admin/a.
If unable to browse into these URLs, examine the log files referenced above. There should be some indication of why the services did not start.
Tool Scripts and Useful Shortcuts
A variety of tools and useful scripts are located at /opt/spirit/sbin, and listed in Table 13. This directory should have been added to the path in a previous step.
The renew scripts should be used only in the test phase and never on a running production system. In a production system, the backup scripts and complog script should be added to /etc/crontab so that they are executed on a regular basis.
Table 14 lists shortcuts for frequently used commands.
For example, if the following command is issued after starting JBoss, any errors generated are displayed and an indication is generated when the application is started (which can take several minutes).
MDES Configuration
A complete configuration of the MDES system is beyond the scope of this document. This document is intended to provide directions to get the base system (as tested in the lab for this CVD) installed and functional. A working knowledge of Linux is assumed. A much deeper understanding of HL7 and the SpiritEHR application are required before attempting a complete MDES configuration and integration with third-party clinical applications. Tiani Spirit and their partners provide training on the configuration of this software. That training should be taken prior to attempting to configure the MDES solution in a production environment. Detailed information on the configuration can be found in the SpiritEHR technical manual, version 1.8 from Tiani Spirit.
Transaction Broker
The Transaction Broker is used to configure the individual components of the SpiritEHR (PIX, XDS, XCA, PWP, system reports, adapters, and so on). To use Transaction Broker, complete the following steps.
Procedure
Step 1
The screen shown in Figure 47 is displayed.
Figure 47 TBroker Login
Step 2
Step 3
The screen shown in Figure 48 is displayed.
Figure 48 Domain Selection
Step 4
The screen shown in Figure 49 is displayed.
Figure 49 TBroker Logging
Step 5
Figure 50 Tiani Spirit Logging
Any errors that occurred during the startup process, along with any other errors, are displayed in this log.
Configuring Organization, Roles, and Users
To configure organization, roles, and users, complete the following steps.
Procedure
Step 1
The screen shown in Figure 51 is displayed.
Figure 51 Select Organization Configuration
Step 2
The screen shown in Figure 52 is displayed.
Figure 52 Organization Configuration
Step 3
Step 4
Step 5
Figure 53 User Configuration
Step 6
Step 7
Figure 54 Admin User Organization Mapping
Step 8
Step 9
Step 10
Figure 55 Domain Selection
Step 11
Figure 56 Creating a Role
Step 12
Step 13
The screen shown in Figure 57 is displayed.
Figure 57 Set Role Permissions
Above is a sample display. You must select the permissions for each of the items on the left associated with the role you are creating. The default for all is unselected. For example, if the PDQ is not selected and checked, an individual with this role is not able to query patient demographic data.
An administrator may be able to:
•
•
A primary care physician may be able to:
•
•
•
A staff physician may be able to:
•
•
The healthcare organization can choose to create and limit whatever roles are appropriate. This is also where the view of patient records may be limited. You can limit the scope of the organizations to which the role has visibility as well as access. Access can get extremely granular to the document types, confidentiality codes, as well as medical fields. Figure 58 shows an example.
Figure 58 Set Role Visibility
Step 14
Step 15
Figure 59 Create User
Step 16
Step 17
Lifetime is how frequently the password must be changed.
Step 18
Figure 60 Map User to Organization
Step 19
Step 20
Figure 61 Map User to Role
Step 21
Step 22
Step 23
Step 24
Step 25
The created user should be able to browse to the web client at http://hostname:8181/SpiritEhrPortal, log in with the username/password and perform any function his assigned roles have permission to perform.
Distributing System Information
After the nodes are created, the information about each node must be shared with the other nodes in the network. This is achieved by executing the syscreate.sh script on each node.
Be sure to execute this process in the following order:
1.
2.
3.
4.
If the script is executed without an argument, it is executed on the local system. If a host is specified as an argument, that system is created. This causes several things to occur.
Domains are populated in the /opt/spirit/jboss/bin/XdsCodeMappings/domains.xml file. This file name may be renamed to something more meaningful; for example, the clinic domain file might be renamed clinic_domains.xml. Following is a sample from the clinic node (hcc1-2951) in the lab system after syscreate.sh has been executed on all of the nodes. The local host is represented as variables defined in the Config Mapper file. All the remote domains are explicitly listed.
<?xml version="1.0" encoding="ISO-8859-1"?><Codes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><CodeType name="domainCodes"><Code classification="IDDomainCodes" code="$LOCAL_XDS_DOM" display="$LOCAL_XDS_DESC"/><Code classification="IDDomainCodes" code="$LOCAL_PI_DOM" display="$LOCAL_PI_DESC"/><Code classification="IDDomainCodes" code="1.2.40.0.10.1.4.1.1" display="SSN"/><Code classification="IDDomainCodes" code="ISEHCLab.1000.903.1.1.3.1" display="HCPG3 PI Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1001.901.1.1.3.1" display="HCHH1 PI Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1001.990.1" display="DCHC HH XDS Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1000.904.1.1.3.1" display="HCPG4 PI Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1000.901.1.1.3.1" display="HCPG1 PI Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1004.901.1.1.3.1" display="HCSH1 PI Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1004.990.1" display="HCSH1 XDS Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1000.902.1.1.3.1" display="HCPG2 PI Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1000.990.1" display="HCDC PG XDS Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1002.901.1.1.3.1" display="HCH1 PI Domain"/><Code classification="IDDomainCodes" code="ISEHCLab.1002.990.1" display="HCH1 XDS Domain"/></CodeType><CodeType name="IDDomainCodes"><Code classCode="ISO" classification="$LOCAL_XDS_NS" code="$LOCAL_XDS_DOM" codingScheme="XDS_ID" display="$LOCAL_XDS_DESC"/><Code classCode="ISO" classification="$LOCAL_NS" code="$LOCAL_PI_DOM" codingScheme="PRIM_ID" display="$LOCAL_PI_DESC"/><Code classCode="ISO" classification="SSN" code="1.2.40.0.10.1.4.1.1" codingScheme="HC_ID" display="SSN"/><Code classCode="ISO" classification="1000.903.9.1.3.1" code="ISEHCLab.1000.903.1.1.3.1" codingScheme="PRIM_ID" display="HCPG3 PI Domain"/><Code classCode="ISO" classification="1001.901.9.1.3.1" code="ISEHCLab.1001.901.1.1.3.1" codingScheme="PRIM_ID" display="HCHH1 PI Domain"/><Code classCode="ISO" classification="$LOCAL_XDS_NS" code="ISEHCLab.1001.990.1" codingScheme="XDS_ID" display="DCHC HH XDS Domain"/><Code classCode="ISO" classification="1000.904.9.1.3.1" code="ISEHCLab.1000.904.1.1.3.1" codingScheme="PRIM_ID" display="HCPG4 PI Domain"/><Code classCode="ISO" classification="1000.901.9.1.3.1" code="ISEHCLab.1000.901.1.1.3.1" codingScheme="PRIM_ID" display="HCPG1 PI Domain"/><Code classCode="ISO" classification="1004.901.9.1.3.1" code="ISEHCLab.1004.901.1.1.3.1" codingScheme="PRIM_ID" display="HCSH1 PI Domain"/><Code classCode="ISO" classification="$LOCAL_XDS_NS" code="ISEHCLab.1004.990.1" codingScheme="XDS_ID" display="HCSH1 XDS Domain"/><Code classCode="ISO" classification="1000.902.9.1.3.1" code="ISEHCLab.1000.902.1.1.3.1" codingScheme="PRIM_ID" display="HCPG2 PI Domain"/><Code classCode="ISO" classification="$LOCAL_XDS_NS" code="ISEHCLab.1000.990.1" codingScheme="XDS_ID" display="HCDC PG XDS Domain"/><Code classCode="ISO" classification="1002.901.9.1.3.1" code="ISEHCLab.1002.901.1.1.3.1" codingScheme="PRIM_ID" display="HCH1 PI Domain"/><Code classCode="ISO" classification="$LOCAL_XDS_NS" code="ISEHCLab.1002.990.1" codingScheme="XDS_ID" display="HCH1 XDS Domain"/></CodeType><CodeType name="XdsDomains"><Code classification="XdsDomain" code="$LOCAL_XDS_DOM" display="$LOCAL_XDS_DESC"/><Code classification="XdsDomain" code="ISEHCLab.1001.990.1" display="DCHC HH XDS Domain"/><Code classification="XdsDomain" code="ISEHCLab.1004.990.1" display="HCSH1 XDS Domain"/><Code classification="XdsDomain" code="ISEHCLab.1000.990.1" display="HCDC PG XDS Domain"/><Code classification="XdsDomain" code="ISEHCLab.1002.990.1" display="HCH1 XDS Domain"/></CodeType><CodeType name="IDDomainInfo"><Code classCode="ISO" classification="RRI" code="$LOCAL_XDS_DOM" codingScheme="DCP" display="$LOCAL_XDS_DESC"/><Code classCode="ISO" classification="PI" code="$LOCAL_PI_DOM" codingScheme="$LOCAL_ID_PREFIX" display="HCHH PI Domain"/><Code classCode="ISO" classification="SS" code="1.2.40.0.10.1.4.1.1" codingScheme="SSN" display="SSN"/><Code classCode="ISO" classification="PI" code="ISEHCLab.1000.903.1.1.3.1" codingScheme="PG3" display="HCPG3 PI Domain"/><Code classCode="ISO" classification="PI" code="ISEHCLab.1001.901.1.1.3.1" codingScheme="HH1" display="HCHH PI Domain"/><Code classCode="ISO" classification="RRI" code="ISEHCLab.1001.990.1" codingScheme="DCP" display="DCHC HH XDS Domain"/><Code classCode="ISO" classification="PI" code="ISEHCLab.1000.904.1.1.3.1" codingScheme="PG4" display="HCPG4 PI Domain"/><Code classCode="ISO" classification="PI" code="ISEHCLab.1000.901.1.1.3.1" codingScheme="PG1" display="HCPG1 PI Domain"/><Code classCode="ISO" classification="PI" code="ISEHCLab.1004.901.1.1.3.1" codingScheme="HCSH" display="HCHH PI Domain"/><Code classCode="ISO" classification="RRI" code="ISEHCLab.1004.990.1" codingScheme="DCP" display="HCSH1 XDS Domain"/><Code classCode="ISO" classification="PI" code="ISEHCLab.1000.902.1.1.3.1" codingScheme="PG2" display="HCPG2 PI Domain"/><Code classCode="ISO" classification="RRI" code="ISEHCLab.1000.990.1" codingScheme="DCP" display="HCDC PG XDS Domain"/><Code classCode="ISO" classification="PI" code="ISEHCLab.1002.901.1.1.3.1" codingScheme="HCH1" display="HCHH PI Domain"/><Code classCode="ISO" classification="RRI" code="ISEHCLab.1002.990.1" codingScheme="DCP" display="HCH1 XDS Domain"/></CodeType></Codes>Syscreate also populates the System Tree table (sys) in the TMED database. This table maintains information for each of the other nodes in the network including system type, system ID, host name, patient identifier domain, XDS affinity domain identifier, XCA responding gateway (home community ID), parent, URL, and system name. Many of these are from the OID Schema referenced earlier. (See Figure 62.)
Figure 62 Sys Table
The following commands display the table.
Definitions
The following acronyms are used in this document:
•
•
•
•
•
•
•
•
•
•
–
–
–
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Appendix A—ISEHC OID Schema
