T40.7: Secondary Environment Usage

Secondary environments, also known as peer environments, are a core CCR feature that enables organizations to maintain multiple namespaces in sync with a single primary environment. When changes are deployed to a primary environment, the same changes can be simultaneously deployed to all of its configured secondary environments.

The key characteristic of secondary environments is that they operate as parallel targets for the same Perforce branch. This means when an ItemSet is deployed to the primary BASE environment, it can also be deployed to secondary BASE environments like BUILD or DEV. Similarly, when an ItemSet is deployed to TEST, secondary TEST environments such as TRAIN or QD receive the same deployment. This parallelism extends to any workflow stage: BASE, TEST, UAT, or LIVE.

A common example is the HealthShare Unified Care Record architecture. HealthShare UCR deployments present two CCR system architecture options:

Option 1: Separate CCR Systems -- Create separate CCR Systems for the Edge Gateway, Access Gateway, and Registry. Each system has its own workflow. Secondary environments are used for multiple gateways within each system type. For example, if there are two Edge Gateways, one would be the primary and the other would be a secondary environment.

Option 2: Instance-wide CCR -- Use a single CCR System with a shared code database for all namespaces. This approach uses one CCR System and secondary environments to handle all the gateway namespaces. For example, if there are two Edge Gateways (E1 and E2) plus a Repository, each environment level (BASE, TEST, LIVE) would deploy to the primary environment plus the secondary gateways simultaneously.

In either architecture, the diagram shows how ItemSets flow from the Perforce repository to the primary environment and then fan out to secondary environments. At each stage (BASE, TEST, LIVE), the Repo (primary) receives the ItemSet first, and then E1 and E2 (secondaries) receive the same ItemSet in parallel.

Secondary environments for BASE have a special consideration: they require deployment of ItemSets during the In_BASE state. When the Require ItemSet setting is enabled (set to true), ItemSets are automatically created when changes are submitted, ensuring that the secondary environments stay synchronized with the primary.

A practical illustration of secondary environment topology is shown in the CCR training materials. At the BASE level, a primary environment might have a BUILD secondary. At the TEST level, a primary environment might have TRAIN and QD (Quality/Development) secondaries. At the LIVE level, a primary might have LIVE2 and TRAIN2 secondaries. Each of these secondary environments receives the same Perforce source as the primary when ItemSets are deployed.

Another important use case is the TRAIN environment. Projects often require a dedicated training environment for end-user training, separate from TEST to avoid collisions between testers and trainers. One approach is to periodically refresh TRAIN from TEST (or LIVE), which gives precise control over TRAIN contents but is time-intensive and requires a technical resource. The preferred alternative is to configure TRAIN as a secondary environment for TEST. This keeps TRAIN always up to date -- ItemSets are pushed to TRAIN and TEST simultaneously -- and does not require a separate technical resource for refresh. The tradeoff is that manual (non-code) changes must be made in both TEST and TRAIN, and configuration that is not yet ready for training use will appear in TRAIN.

---

Deploying ItemSets to secondary environments follows a specific procedure. First, the user must ensure that browser popups are allowed, since the deployment process opens new tabs for each target environment. Second, the user must be logged into the Management Portal of all environments (primary and secondaries). Third, the user clicks the "Deploy to All Rqd" link on the ItemSet, which triggers simultaneous deployment to all required environments.

The deployment mechanism opens a new browser tab for each environment. There are four possible outcomes for each tab:

1. Successful deployment: The tab automatically closes, indicating the ItemSet was deployed without errors. 2. Authentication failure: The tab remains open and displays the login page. The user must log in and retry the deployment. 3. Deployment error: The tab remains open and turns red, indicating an error occurred during deployment. The user must investigate and resolve the error. 4. Popup blocked: If browser popups are blocked, the tab may not open at all. The user must allow popups and retry.

For environments with multiple secondaries, all tabs open simultaneously. The user should monitor all tabs to ensure every environment deploys successfully before proceeding with the CCR workflow.

The Require ItemSet flag is a critical system configuration setting that governs whether secondary environments must receive every ItemSet deployed to the primary. When this flag is enabled, the system enforces that all secondary environments stay synchronized. When it is disabled, there is significant risk of environments drifting out of sync.

Two dangerous scenarios illustrate why Require ItemSet should generally remain enabled:

Scenario 1 (Code Drift): A Tier 1 CCR changes one line of code. The user does not deploy the class to the secondary environment because Require ItemSets is false. Later, a different Tier 1 CCR changes a different line of code in the same class (unrelated change). When this second change is deployed to the secondary environment, it unknowingly deploys both changed lines -- including the first change that was never reviewed or tested in that environment.

Scenario 2 (Dependency Failure): A Tier 1 CCR adds class definition B and edits preexisting class A to be dependent on B. The user does not deploy these classes to the secondary environment because Require ItemSets is false. Later, a different Tier 1 CCR edits class A for an unrelated change. The deployment to the secondary environment fails because class A still depends on class B, which was never deployed there. This creates a situation where the secondary environment cannot compile the code at all.

Both scenarios demonstrate that keeping Require ItemSet enabled is a best practice to prevent synchronization issues between primary and secondary environments.

When Require ItemSet is enabled and the secondary environment is properly configured, the ItemSet deployment process ensures that both the primary and all secondary environments receive exactly the same set of changes. The ItemSet contains the same classes and items for every target, creating a consistent, predictable deployment. This is especially important in HealthShare environments where Edge Gateways must maintain identical code to properly communicate with the Access Gateway and Registry components.

It is also worth noting that for secondary BASE environments specifically, ItemSets are automatically created when changes are submitted if the Require ItemSet setting is enabled. This automatic creation ensures that the deployment step cannot be forgotten or skipped, which is the root cause of both failure scenarios described above.

---

Configuring a secondary environment is done through the System Details page in the CCR Angular UI. Each primary environment (BASE, TEST, UAT, LIVE) has the ability to have secondary environments added to it. The configuration process involves three steps: expand the appropriate primary environment, click the + button located next to the Save button, and complete the secondary environment form.

The secondary environment form collects the following information:

• Nickname: A descriptive name for the secondary environment (e.g., "BUILD", "TRAIN", "QD"). This nickname is displayed in the ItemSet interface as "deploy to [nickname]", making it clear which environment is being targeted.
• Namespace: The InterSystems namespace for the secondary environment.
• Home Page: The URL of the environment's home page.
• Management Portal: The URL of the System Management Portal.
• Super Server: The super server port number (typically 1972).
• Terminal Port: The terminal port number (typically 23).
• ItemSet Controls (Optional): Additional controls for customizing ItemSet behavior in the secondary environment.

Secondary BASE environments have important configuration constraints. Because secondary BASE environments cannot identify whether items are checked out on the primary BASE environment, they must be restricted to prevent conflicts. There are two recommended configurations:

Read-only: The secondary BASE environment is completely read-only. Users can view code but cannot check out or modify items. This is the most restrictive and safest option.

Upload Disabled: The secondary BASE environment allows users to check out items (for testing possible changes) but does not allow uploading changes back to Perforce. This workflow follows a specific sequence: (1) Checkout an item in the secondary BASE to test a possible change. (2) Use the Undo Checkout action in the secondary BASE to release the item. This step is essential because ItemSets containing items that are checked out in the target environment cannot be deployed. (3) Make and upload the actual change in the primary BASE environment. (4) Deploy the ItemSet to all BASE environments. (5) Test the deployed change in the secondary BASE environments. (6) Progress the CCR normally through the workflow.

The Upload Disabled approach is particularly useful for HealthShare deployments where developers may need to verify how changes behave across multiple Edge Gateway namespaces before finalizing the change in the primary BASE environment. See ICC430 for detailed instructions on how to configure read-only and upload-disabled settings.

When choosing between read-only and upload-disabled for secondary BASE environments, consider the team's workflow:

• Read-only is simpler and eliminates any risk of accidental changes or checkout conflicts. It is the recommended default for most secondary BASE environments.
• Upload-disabled provides more flexibility for developers who need to experiment in the secondary environment before committing changes through the primary. It requires more discipline from the team to follow the Undo Checkout step.

In either case, the critical constraint remains: all code changes must be made and uploaded through the primary BASE environment, and ItemSet deployment is the mechanism that propagates those changes to secondaries.

---