T1.4: Upgrades Existing Installation

Overview

Reviewing release notes is the essential first step in any UCR upgrade. The release notes document all changes between the current version and the target version, including breaking changes, new features, bug fixes, and deprecated functionality. Failing to review release notes can lead to unexpected failures during or after the upgrade.

Where to Find Release Notes

• InterSystems documentation website under the HealthShare Unified Care Record product section
• The release notes are version-specific and cover the transition from one version to the next
• For multi-version upgrades (skipping versions), review the release notes for every intermediate version

Breaking Changes Section

The Breaking Changes section is the highest priority for review:

• Lists changes that may cause existing functionality to stop working
• Includes API changes to system-supplied classes that may break custom code
• Documents configuration format changes that require manual migration
• Identifies removed features or settings that must be replaced with alternatives
• May include changes to database schema, global structures, or class hierarchies

Compatibility Matrix

Before proceeding with an upgrade, verify:

• The target UCR version is supported on your current IRIS for Health version (or identify the required IRIS upgrade)
• Your operating system version is supported by the target UCR version
• Your web server and web gateway versions are compatible
• Any third-party integrations are compatible with the target version
• Mirror configurations are supported in the target version

Impact Assessment

After reviewing release notes, create an impact assessment:

• List all breaking changes that affect your deployment
• Identify custom code that references changed or deprecated APIs
• Document configuration changes that will need manual intervention
• Prioritize items that could block the upgrade

---

Overview

Edge Gateways require careful preparation before upgrade to ensure no clinical data is lost and the upgrade proceeds without complications. This preparation involves a controlled shutdown of data flows, queue drainage, production stoppage, and comprehensive backup.

Stopping Incoming Data Feeds

The first step is to stop new data from entering the Edge Gateway:

• Disable business services that receive HL7 messages (TCP listeners, file services)
• Coordinate with source system administrators to pause or queue messages on their side
• If using guaranteed delivery, source systems should hold messages until the Edge Gateway is back online
• Verify in the production monitor that no new messages are arriving
• Document which feeds were active so they can be re-enabled after upgrade

Draining Message Queues

After stopping incoming feeds:

• Monitor the queue depth for all business hosts in the Management Portal
• Allow currently queued messages to complete processing through to their final operations
• Check for messages in error or suspended status; resolve or defer them before proceeding
• Verify that all queues reach zero or an acceptable baseline level
• Pay special attention to the ECR storage process queue and audit forwarding queue

Stopping the Production

Once queues are drained:

• Stop the production gracefully using the Management Portal (Production > Stop)
• Wait for the production to reach a fully stopped state (all business hosts show "stopped")
• Document the final state of all business hosts and any remaining messages

Creating Backups

Before proceeding with the upgrade:

• Back up all databases in the Edge Gateway's namespace (IRIS backup or external backup tool)
• Back up the iris.cpf configuration file and SSL/TLS certificate files
• Export custom classes, DTL transforms, BPL processes, lookup tables, and custom XSLT files
• Store backups in a location separate from the instance being upgraded
• Verify backup integrity by checking file sizes and checksums

---

Overview

A UCR upgrade replaces system-supplied classes with new versions. Any direct modifications made to these system classes will be overwritten and lost. Understanding which classes are system-supplied versus custom, and identifying any unsupported modifications, is critical for a successful upgrade.

System-Supplied vs Custom Classes

• System-supplied classes: Classes in the HS.*, HSMOD.*, SchemaMap.*, and similar InterSystems-owned packages. These are replaced during upgrade.
• Custom classes: Classes in customer-defined packages (e.g., Custom.*, site-specific package names). These are preserved during upgrade but may need recompilation if they depend on changed system APIs.
• Generated classes: Some classes are auto-generated by the UCR framework. These are regenerated during upgrade.

Unsupported Modifications

InterSystems explicitly does not support direct modifications to system-supplied classes:

• Editing source code of HS.* classes directly
• Adding methods or properties to system classes
• Modifying system-supplied DTL transforms in the HS.* package
• Changing system-supplied BPL processes
• Altering system XData blocks (e.g., production definitions in system classes)

All such modifications are overwritten during upgrade without warning.

Supported Customization Approaches

The supported approach for customizing UCR behavior:

• Subclassing: Create custom subclasses that extend system classes and override specific methods
• Callback methods: Use documented callback hooks provided by the UCR framework
• Custom DTL/BPL: Create custom transforms and processes in custom packages, referencing them in production configurations
• Configuration settings: Use production settings and system configuration to adjust behavior without code changes
• Lookup tables: Use editable lookup tables for data mapping customizations

Pre-Upgrade Impact Analysis

Before upgrading, systematically identify unsupported modifications:

• Search for classes in the HS.* namespace that have been modified (compare with the original distribution)
• Review source control history for changes to system classes
• Check for custom methods added to system classes
• Identify any system DTL or BPL that has been modified
• Plan migration of unsupported modifications to supported approaches (subclasses, callbacks)

---

Overview

While custom code in customer-owned packages is generally preserved during upgrade, best practice is to export all customizations before upgrading as a safety measure. Certain customizations, particularly Clinical Viewer layouts, require explicit preservation steps.

Exporting Custom Classes

Before upgrade, export all custom classes:

• Use the Management Portal (System Explorer > Classes) to export custom packages to XML files
• Alternatively, use $system.OBJ.Export() from the terminal to export classes programmatically
• Include all custom packages: business services, processes, operations, DTL transforms, BPL processes, and utility classes
• Verify the export files contain all expected classes
• Store export files outside the instance directory structure

Clinical Viewer Layout Preservation

Custom Clinical Viewer layouts require special attention:

• Clinical Viewer layouts define the arrangement of charts, panels, and clinical data displays
• Custom layouts may include site-specific modifications to the patient summary, chart views, or clinical data panels
• Export custom layouts using the Clinical Viewer administration tools
• Document which layouts are custom vs standard so they can be restored after upgrade
• The upgrade may introduce new standard layout features that need to be reconciled with custom layouts

Custom DTL and BPL Backup

Data Transformation Language (DTL) and Business Process Language (BPL) customizations:

• Export all custom DTL classes that transform incoming data (HL7 to SDA, custom SDA mappings)
• Export all custom BPL classes that implement custom business logic
• Document which production components reference these custom transforms and processes
• After upgrade, verify that custom DTL/BPL classes compile successfully against the new system classes

Lookup Table Export

Custom lookup tables contain site-specific data mappings:

• Export lookup tables from the Management Portal (Interoperability > Lookup Tables)
• Include terminology mapping tables, code translation tables, and facility configuration tables
• Lookup tables are typically preserved during upgrade but should be backed up as a precaution

Configuration Documentation

Capture current configuration:

• Export production configurations (the production class definition)
• Document system-wide settings (credentials, SSL/TLS configurations, scheduled tasks)
• Record pool sizes, timeout values, and retry settings for all business hosts
• Save web application definitions and CSP application settings

---

Overview

The UCR upgrade is a two-phase process on each instance: first the underlying IRIS for Health platform is upgraded, then the UCR-specific components are upgraded using HS.Util.Installer. The upgrade must be executed in the correct order across the federation, starting with the Registry.

Upgrade Order Across the Federation

1. Registry (Hub): Upgrade first. The Registry is the central coordination point, and other components must communicate with a Registry at a compatible version. 2. Edge Gateways: Upgrade after the Registry. Each Edge Gateway can be upgraded independently, allowing rolling upgrades in large federations. 3. Access Gateways: Upgrade after the Edge Gateways they query are upgraded. 4. Bus: Upgrade after Registry and Edge Gateways. 5. ODS: Upgrade last, as it depends on data structures from Edge Gateways.

Phase 1: IRIS for Health Platform Upgrade

On each instance:

• Stop the IRIS for Health instance
• Run the new version IRIS for Health installer
• Select the upgrade option (the installer detects the existing instance)
• Apply the new license key if required
• Allow the installer to upgrade system databases (IRISSYS, IRISLIB)
• Start the upgraded instance and verify it runs the new version

Phase 2: UCR Namespace Upgrade with HS.Util.Installer

After the platform upgrade:

• Open a terminal session in the %SYS namespace
• Run the HS.Util.Installer upgrade method, specifying the upgrade manifest
• The Installer processes each UCR namespace on the instance (HSREGISTRY, HSEDGE, HSACCESS, etc.)
• For each namespace, the Installer updates system classes, regenerates production definitions, and applies schema changes
• Monitor the Installer output for errors or warnings

Installer Command-Line Options

Common HS.Util.Installer options:

• Upgrade method invocation from the %SYS namespace terminal
• Manifest file or parameter specifying the target component configuration
• Options to control which namespaces are upgraded (useful for partial upgrades)
• Logging options to capture detailed upgrade output
• The Installer writes to the HS.Util.Installer log and the IRIS messages log

Handling Upgrade Failures

If the Installer reports errors:

• Review the error messages in the Installer log
• Consult the release notes for known issues and workarounds
• Check if the error is on the safe-to-ignore list for this version transition
• If the error is critical, restore from backup and contact InterSystems support
• Do not proceed with upgrading other components if the Registry upgrade fails

---

Overview

After running both the IRIS platform upgrade and the HS.Util.Installer, administrators must review log files to confirm the upgrade completed successfully. Multiple log files capture different aspects of the upgrade process, and administrators must understand how to interpret their contents.

HS.Util.Installer Log

The primary log for UCR upgrade verification:

• Located in the instance's manager directory or accessible through the Management Portal
• Records each step of the UCR namespace upgrade process
• Shows which classes were updated, which schemas were migrated, and which configurations were modified
• Reports errors encountered during namespace processing
• A successful upgrade ends with a completion message indicating all namespaces were processed

IRIS Install Log

The platform upgrade creates its own log:

• Records the IRIS for Health installation/upgrade process
• Documents system database upgrades (IRISSYS, IRISLIB)
• Reports any issues with binary replacement or system configuration updates
• Located in the instance installation directory (typically install.log or install_log.txt)

messages.log Analysis

The IRIS messages log captures runtime events after upgrade:

• Post-upgrade startup messages showing the new version
• Database mount and integrity check results
• Web gateway configuration updates
• Any errors encountered during the first startup after upgrade
• Located in the instance's manager directory (mgr/messages.log)

Distinguishing Errors from Warnings

When reviewing logs:

• Critical errors: Indicate upgrade failure; examples include database corruption, class compilation failures for core UCR classes, missing required components
• Warnings: Indicate non-blocking issues; examples include deprecated syntax notices, optional feature configuration changes, informational migration messages
• Safe-to-ignore errors: Documented in the release notes; these are known issues that do not affect functionality
• When in doubt, check the release notes safe-to-ignore list or contact InterSystems support

Expected Log Messages

A successful upgrade typically shows:

• "Upgrade completed successfully" or similar completion message in the HS.Util.Installer log
• No critical errors in the IRIS install log
• Clean startup messages in messages.log showing the new version
• Successful compilation of all system classes
• Successful schema migration for all UCR databases

---

Overview

After the upgrade is confirmed successful on all instances, productions must be reactivated across the federation. This follows the standard startup sequence but includes additional verification steps and may require post-upgrade cleanup tasks such as ODS index rebuilding.

Pre-Restart Verification

Before starting any production:

• Confirm the upgrade log shows success on every instance in the federation
• Verify that all instances are running the same target version
• Check that SSL/TLS certificates are still valid and accessible
• Verify that license keys are active on all instances
• Confirm that custom code has been recompiled successfully (compile all custom packages)

Production Restart Sequence

Follow the standard UCR startup order: 1. Registry production: Start first. Verify MPI services, consent engine, and system registry are operational. Check that the Management Portal shows the production running with no errors. 2. Audit Edge production: Start to enable audit event collection from other components. 3. Edge Gateway productions: Start each Edge Gateway. Verify it can communicate with the Registry (test PIX and XDS.b operations). Do not re-enable source system data feeds yet. 4. Access Gateway productions: Start after Edge Gateways are confirmed running. Verify patient search and document retrieval work through the Clinical Viewer. 5. Bus production: Start if applicable. Verify external system connectivity. 6. ODS production: Start last. Verify FHIR server endpoints and data synchronization.

Re-Enabling Data Feeds

After all productions are running and verified:

• Re-enable source system data feeds to Edge Gateways one at a time
• Start with lower-volume feeds first to monitor processing behavior
• Verify that incoming messages process correctly through to ECR storage
• Check that patient identities register correctly with the Registry MPI
• Monitor queue depths and processing times during the initial data flow period

ODS Rebuild Requirements

After certain upgrades, the ODS may require maintenance:

• Index rebuild: New indexes introduced in the target version must be populated via the ODS index rebuild utility
• FHIR resource regeneration: If the FHIR schema changed, resources may need regeneration
• These rebuilds can be time-consuming for large datasets; plan accordingly and run during low-usage periods

Post-Upgrade Monitoring

During the first days after upgrade:

• Monitor production event logs for unexpected errors and watch queue depths for bottlenecks
• Verify Clinical Viewer performance, data display accuracy, and audit event collection
• Compare key performance metrics (response times, throughput) with pre-upgrade baselines
• Have rollback procedures and InterSystems support contact information readily available

---