T1.3: Post-Installation Configuration

Overview

Database mirroring provides high availability and disaster recovery for UCR deployments. While mirroring can be configured at any point, InterSystems recommends configuring it before or immediately after UCR installation, as setting it up later requires additional steps to synchronize databases that already contain data.

Mirror Architecture

A mirror consists of:

• Failover members: A primary (active) and backup (passive) member that automatically switch roles if the primary fails. Both members maintain identical copies of mirrored databases.
• Async members: Read-only members that receive journal updates asynchronously. Used for disaster recovery or reporting workloads.
• Arbiter: An optional lightweight process that helps resolve failover decisions when network partitions occur.

When to Configure Mirroring

• Before UCR install (recommended): Configure the mirror on the IRIS for Health instances before running the UCR Installer. The Installer will then create UCR databases as mirrored databases automatically.
• After UCR install: If mirroring is configured after UCR databases already exist, the databases must be added to the mirror and the backup member must be initialized from a backup of the primary.

Journal Transfer and Synchronization

• Mirror members stay synchronized through continuous journal transfer from the primary to the backup
• The backup applies journal entries in real time, maintaining a near-identical database state
• If the backup falls behind (e.g., after a network interruption), it catches up by replaying accumulated journal files
• Async members receive journals with a configurable delay, providing a recovery point if data corruption occurs on the primary

Mirroring Considerations for UCR

• Mirror each component that requires high availability: Registry, Edge Gateways, Access Gateways
• The Registry is the highest-priority component for mirroring, as all other components depend on it
• Edge Gateways holding clinical data should be mirrored to prevent data loss
• Access Gateways can be mirrored or simply redeployed from configuration, depending on requirements

---

Overview

Each UCR component runs an Interoperability production containing business hosts that handle message flow. Post-installation configuration involves reviewing and adjusting these business hosts to match the deployment's requirements. The UCR Installer creates default productions with pre-configured business hosts, but many settings require site-specific adjustment.

Business Host Types

• Business Services: Entry points for incoming data. Examples include HL7 TCP listeners on Edge Gateways, SOAP web service endpoints on the Registry, and file-based import services.
• Business Processes: Orchestrate message routing and transformation. Examples include the SDA-to-ECR storage process on Edge Gateways and the PIX/PDQ coordination process on the Registry.
• Business Operations: Send data to external targets. Examples include the Registry notification operation, the Audit Edge forwarding operation, and SOAP client operations for inter-component queries.

Key Production Settings

For each business host, administrators can configure:

• Enabled/Disabled: Whether the business host is active
• Pool Size: Number of concurrent processing instances (covered in detail in Section 4)
• Queue timeout: How long messages wait in queue before timing out
• Retry interval and count: How failed operations are retried
• Alert settings: When to generate alerts for errors or queue buildup
• Custom settings: Component-specific configuration parameters

Namespace-Specific Productions

Each UCR namespace has its own production:

• HSREGISTRY: Registry production with MPI, consent, metadata registry, and notification services
• HSEDGE (per gateway): Edge Gateway production with data intake, ECR storage, and query response services
• HSACCESS: Access Gateway production with patient search, document retrieval, and Clinical Viewer support
• HSBUS: Bus production with external-facing services and internal routing
• HSODS: ODS production with FHIR server, data synchronization, and index management

---

Overview

UCR components form a federated architecture that communicates over the network using SOAP web services secured by SSL/TLS. Each component must be configured with the endpoints of the other components it needs to communicate with, and network infrastructure must permit traffic between them.

SSL/TLS Configuration

Each component requires SSL/TLS configurations created in the Management Portal under System Administration > Security > SSL/TLS Configurations:

• Server configurations: For accepting incoming connections (each component that hosts web services)
• Client configurations: For making outgoing connections to other components
• Certificate management: Each configuration references certificate files (server cert, private key, CA cert)
• Protocol versions: TLS 1.2 and TLS 1.3 are recommended; older protocols should be disabled

Registering Edge Gateways with the Registry

Each Edge Gateway must be registered in the Registry's system registry:

• Add the Edge Gateway as a system entry with its name, OID, and endpoint URLs
• Associate the gateway with its facility or facilities
• Configure the trust relationship via SSL/TLS certificates
• The Registry uses this registration to route queries and receive patient identity feeds

Firewall and Network Rules

Required network paths include:

• Edge Gateways to Registry (PIX, XDS.b, notifications)
• Access Gateways to Registry (PDQ, PIX consumer)
• Access Gateways to Edge Gateways (document retrieval)
• All components to Audit Edge (audit event forwarding)
• Default HTTPS port (443) or custom ports as configured
• Web Gateway ports for Management Portal access

Testing External Communication

After configuring endpoints:

• Use the Management Portal's "Test" button on SOAP operations to verify connectivity
• Check SSL/TLS handshake completion in connection logs
• Send test messages through Edge Gateway to Registry and verify processing
• Verify Access Gateway can query Edge Gateways through the Registry's metadata

---

Overview

Pool sizes control how many instances of a business host run concurrently within a production. A pool size of 1 means messages are processed serially; a pool size of N allows up to N messages to be processed in parallel. Proper pool sizing is critical for throughput and resource utilization.

Understanding Pool Size Behavior

• Pool size = 0: The business host is disabled and will not process messages
• Pool size = 1: Serial processing; messages queue and are processed one at a time
• Pool size > 1: Parallel processing; multiple messages processed simultaneously
• Pool size does not guarantee ordering; if message ordering is required, use pool size = 1
• Each pool instance consumes memory and potentially database connections

Registry Pool Size Recommendations

• PIX Manager service: 2-4, scaled by number of Edge Gateways submitting patient registrations
• PDQ service: 2-6, scaled by number of concurrent patient searches from Access Gateways
• XDS.b Registry service: 2-4, scaled by document registration volume
• Notification Delivery operation: 2-4, scaled by number of notification subscribers
• MPI matching process: 1 (serial processing ensures consistency of match decisions)

Edge Gateway Pool Size Recommendations

• Data intake service (HL7/SOAP): 2-4, scaled by volume and frequency of source system feeds
• SDA-to-ECR storage process: 2-4, matching intake throughput
• Query response service: 2-4, scaled by expected query load from Access Gateways
• Audit forwarding operation: 1-2

Access Gateway Pool Size Recommendations

• Patient search service: 2-6, scaled by concurrent Clinical Viewer users
• Document retrieval operation: 4-8, scaled by number of Edge Gateways queried in parallel
• CDA transform process: 2-4, scaled by document rendering workload

Impact of Incorrect Pool Sizing

• Too low: Message queues grow, response times increase, users experience delays, source system connections may time out
• Too high: Excessive memory consumption, database connection pool exhaustion, CPU contention, diminishing returns
• Monitoring: Watch queue depth, average processing time, and system resource utilization to guide adjustments

---

Overview

UCR components have dependencies on each other that dictate a required startup and shutdown sequence. Starting components out of order results in connection failures, error messages, and potential data processing issues. Administrators must follow the correct sequence during planned maintenance, upgrades, and disaster recovery.

Startup Sequence

The correct startup order is: 1. Registry (Hub): Start first. The Registry provides MPI, consent management, patient identity cross-referencing, and system registry services. No other component can function without it. 2. Audit Edge Gateway: Start early so other components can forward audit events immediately upon startup. 3. Edge Gateways: Start after the Registry is confirmed running. Edge Gateways register patient identities with the Registry and need it available for PIX and XDS.b operations. 4. Access Gateways: Start after Edge Gateways are available. Access Gateways query Edge Gateways for clinical data and the Registry for patient identity resolution. 5. Bus: Start after Registry and Edge Gateways, as it routes external requests to both. 6. ODS: Start last, as it synchronizes data from Edge Gateways.

Shutdown Sequence

Shutdown follows the reverse order: 1. ODS: Stop data synchronization first 2. Bus: Stop external-facing services 3. Access Gateways: Stop clinician access (provide advance notification to users) 4. Edge Gateways: Stop data intake (coordinate with source system administrators) 5. Audit Edge: Stop last among gateways to capture shutdown audit events 6. Registry: Stop last after all dependent components have stopped

Auto-Start Configuration

Productions can be configured to start automatically when the IRIS for Health instance starts:

• Set the "AutoStart" setting in the production configuration
• Ensure the startup order is respected if multiple components run on the same server
• Use startup dependencies or delays to sequence auto-starts correctly
• Monitor auto-start behavior after server reboots to verify correct sequencing

Common Startup Issues

• Components failing to connect to the Registry (verify Registry is running and endpoints are correct)
• SSL/TLS handshake failures after certificate expiration or renewal
• Business hosts failing to start due to resource constraints
• Stale connections from previous sessions requiring business host restart

---

Overview

The Master Patient Index is the core identity management component of UCR. It runs on the Registry and receives patient demographic feeds from all Edge Gateways. Proper MPI connectivity ensures that patient identities are correctly matched across the federation, enabling the consolidated patient record view.

Edge Gateway to MPI Communication

Each Edge Gateway must be configured to send patient demographics to the Registry MPI:

• The Edge Gateway's PIX source operation sends ADT (Admit/Discharge/Transfer) messages containing patient demographics to the Registry
• The Registry's PIX Manager service receives these messages and processes them through the MPI matching engine
• The SSL/TLS configuration for this connection must be properly set on both sides
• The Edge Gateway must be registered in the Registry's system registry with its facility association

MPI Matching Configuration

The MPI matching engine on the Registry is configured with:

• Matching algorithm: Probabilistic, deterministic, or hybrid approach for comparing patient records
• Search fields: Which demographic fields (name, date of birth, gender, SSN, address, phone) are used in matching
• Field weights: Relative importance of each field in the matching score calculation
• Scoring thresholds: Auto-link threshold (above which matches are confirmed automatically), review threshold (range requiring manual review), and no-match threshold (below which records are considered distinct)

EMPI Integration

If the deployment uses HealthShare EMPI (Enterprise Master Patient Index) alongside UCR:

• Configure the connection between the UCR Registry MPI and the EMPI system
• Define how patient identities flow between UCR and EMPI
• Configure the EMPI as the authoritative source for enterprise-wide patient identity
• Set up notification channels for identity link/unlink events from EMPI to UCR

Patient Link Notifications

When the MPI links or unlinks patient identities:

• The Registry sends notifications to affected Edge Gateways
• Edge Gateways update their local patient identity cross-references
• Access Gateways receive updated identity information for query routing
• Clinical Viewer sessions reflect the updated patient identity linkage

Testing MPI Connectivity

• Submit test patient demographics from an Edge Gateway
• Verify the Registry MPI receives and processes the demographics
• Check that matching results are correct (test with known duplicate and distinct patients)
• Verify link/unlink notifications are delivered to Edge Gateways
• Review the MPI management console for match quality metrics

---

Overview

After completing all post-installation configuration steps, a systematic validation process ensures the deployment is functioning correctly before going live. This validation covers individual component health, inter-component communication, end-to-end data flow, and security.

Production Status Verification

For each UCR component, verify:

• The production is running (green status in the Management Portal Production page)
• All business services, processes, and operations show active status
• No error conditions or suspended messages exist
• Queue depths are at zero or normal levels
• Event logs show no critical errors

Inter-Component Communication Tests

Test each communication path:

• Edge Gateway to Registry: Submit a patient identity (PIX) and verify the Registry processes it
• Edge Gateway to Registry: Submit document metadata (XDS.b) and verify it registers
• Access Gateway to Registry: Perform a patient search (PDQ) and verify results return
• Access Gateway to Edge Gateway: Request a clinical document and verify retrieval
• All components to Audit Edge: Verify audit events arrive at the Audit Edge repository
• Registry to Edge Gateways: Verify notification delivery for patient link events

End-to-End Data Flow Test

Perform a complete data flow test: 1. Submit a test HL7 message or SDA document to an Edge Gateway 2. Verify the Edge Gateway converts and stores the data as an ECR (Electronic Clinical Record) 3. Verify the Edge Gateway registers the patient identity with the Registry MPI 4. Verify the Edge Gateway registers document metadata with the Registry 5. Log into the Clinical Viewer via an Access Gateway 6. Search for the test patient and verify the patient appears 7. Open the patient chart and verify the submitted clinical data displays correctly 8. Verify the audit trail records all access events

Security Validation

• Confirm SSL/TLS is active on all inter-component connections
• Verify certificate validity dates and trust chains
• Test consent policy enforcement (if consent is configured)
• Verify user authentication and role-based access controls
• Review audit logs for completeness

Deployment Sign-Off Checklist

Before declaring the deployment ready for production use:

• All components running with no errors
• All inter-component communication paths verified
• End-to-end data flow test passed
• MPI matching producing correct results
• Audit trail functioning
• SSL/TLS active on all connections
• Backup procedures tested
• Monitoring and alerting configured
• Documentation updated with site-specific configuration details

---