T39.1: CCR Transport Basics

CCR Client Tools are the bridge between a local InterSystems environment and the CCR server. They are a set of classes and routines that ship with every InterSystems product and reside in the IRISLIB database, making them available to all namespaces within an installation.

The tools consist of the %Studio.SourceControl package, which provides the source control hooks used by Studio, Atelier, and the Management Portal, and the ^%buildccr routine, which is the primary configuration utility. Together they enable namespaces to act as clients to the CCR server and the Perforce Helix Server.

A critical point for practitioners is that CCR Client Tools can be updated without waiting for a full instance upgrade. Updates are delivered through an auto-generated Tier 1 CCR that progresses through the normal workflow. Because the update path follows the same CCR lifecycle as any other change, it is low risk and well documented. After an in-place upgrade of the InterSystems product, the IRISLIB/CACHELIB database is replaced, which can introduce outdated client tools. In that scenario, run Do ReloadTools^%buildccr. to reload the latest version. You can check the current version at any time with Do Version^%buildccr.

It is important to understand that the CCR ecosystem has two separate code bases. Changes to the centralized CCR application (anything in the CCR web UI) are instantly available to all users. However, changes to the CCR Client Tool code running inside each environment require an explicit update delivered via Tier 1 CCR. Slides discussing new client tool features reference the date the feature was added -- an environment has that feature only if its client tools have been updated since that date.

---

CCR Transport supports two modes of operation based on whether the environment has direct access to the InterSystems Perforce Helix Server.

A disconnected environment has no direct access to Perforce. It uses the CCR server as a proxy to submit changes to Perforce. All items travel via ItemSets -- XML files that package one or more Items along with CCR metadata. Disconnected mode is the standard configuration for all customer environments and for all non-BASE environments (TEST, UAT, LIVE), even when those environments sit within the InterSystems network. There is no compelling reason to configure TEST, UAT, or LIVE as connected because deploying ItemSets is straightforward and avoids unnecessary Perforce complexity.

A connected environment communicates directly with the Perforce Helix Server using the p4 command-line client. This mode is available only for BASE environments located inside the InterSystems firewall. Connected mode provides enhanced capabilities such as better diffing tools and the possibility for each developer to have their own BASE workspace. Communication occurs via the p4 client rather than through ItemSets.

When sending items from a disconnected BASE, the CCR Client Tools package the changed items into an ItemSet, which is then uploaded to the CCR server. The CCR server submits those changes to Perforce on behalf of the client. When sending items from a connected BASE, the CCR Client Tools communicate directly with Perforce to check in the source files, bypassing the need for ItemSet upload.

---

The upload phase is where changes made in the BASE development environment are sent to the central Perforce repository. The process differs depending on whether the BASE environment is connected or disconnected.

Disconnected BASE upload workflow: 1. The developer makes changes in the BASE environment (classes, routines, configuration data). 2. CCR Client Tools export the changed items to the source workspace -- a directory on the BASE machine's file system whose folder structure mirrors the Perforce branch structure. 3. An ItemSet is generated. An ItemSet is an XML file that packages one or more Items together with metadata related to the CCR. Think of it as an envelope for sending mail -- it contains the payload plus addressing information. 4. The ItemSet is uploaded to the CCR server via HTTPS (outbound on port 443 to ccr.intersystems.com and login.intersystems.com). 5. The CCR server submits the contents of the ItemSet to the appropriate Perforce BASE branch.

Connected BASE upload workflow: 1. The developer makes changes in the BASE environment. 2. CCR Client Tools export items to the source workspace. 3. The tools communicate directly with the Perforce Helix Server via the p4 client to check in the changes. 4. No ItemSet is required for this step.

In both cases, the upload occurs while the CCR is in the In_BASE state. Best practice dictates uploading changes from BASE as soon as possible, because stalled CCRs in BASE can block other developers whose changes touch the same items.

---

Once changes are committed to the BASE branch in Perforce, the CCR Transport mechanism handles moving them through TEST and LIVE via Perforce branch integration and ItemSet deployment.

Moving to TEST: 1. From BASE_Complete, the authorizeAndStartMoveToTEST transition is performed. This triggers the CCR server to integrate the changes from the BASE Perforce branch to the TEST Perforce branch. 2. A TEST ItemSet is automatically created and made available for download. 3. The CCR moves to the Pending_Manual_Move_To_TEST state. 4. The implementer navigates to the CCR Transport page in the TEST environment, downloads the ItemSet, and clicks "deploy to TEST." 5. Deployment writes the items into the TEST source workspace and loads them into the TEST database. 6. After deployment, the implementer performs the markMoveToTESTComplete transition, moving the CCR to In_TEST for testing.

Moving to LIVE: 1. From TEST_Ready_For_Customer, the authorizeAndStartMoveToLIVE transition triggers integration from the TEST Perforce branch to the LIVE Perforce branch. 2. A LIVE ItemSet is created. 3. The implementer downloads and deploys the ItemSet to the LIVE environment. 4. After deployment, markMoveToLIVEComplete moves the CCR to In_LIVE.

An important point: the startMoveToTEST transition only causes changes to be integrated to the TEST branch and an ItemSet to be created. Downloading and deploying the ItemSet is a separate, manual user action. Similarly, locked LIVE environments do not need to be unlocked to deploy ItemSets -- the deployment mechanism bypasses the lock.

---

One of the most critical best practices in CCR Transport is to always deploy every ItemSet. The Perforce repository is the single source of truth for all controlled items. Environments must stay synchronized with their corresponding Perforce branches.

When an ItemSet is not deployed, the environment drifts out of sync with Perforce. Consider this scenario: 1. A Tier 1 CCR updates item A and progresses through to LIVE in Perforce, but the LIVE ItemSet is not deployed to the LIVE environment. 2. A second CCR later modifies item A and is integrated to the LIVE branch. 3. When the second CCR's ItemSet is deployed, it delivers the latest version of item A from Perforce -- which now includes both changes. 4. The LIVE environment now contains both changes, but the first change was never independently validated in LIVE.

This creates significant risk because untested changes silently enter the production environment. CCR Transport always uses complete copies of items from Perforce, not differential patches, so any skipped deployment means the next deployment will include all accumulated changes.

The quiz answer from the course material confirms this: it is never appropriate to skip deploying an ItemSet. Deploy happens before starting testing. Deploying ItemSets does not require interaction with Perforce (assuming the ItemSet has already been generated). Not being a Perforce user is not a valid reason to skip deployment. Having a prerequisite CCR is not a reason to skip either.

---

Preview integrations are a proactive mechanism in CCR Transport that checks for potential merge conflicts before changes are actually integrated between Perforce branches. This check is triggered automatically at specific points in the CCR workflow.

When a CCR enters the BASE_Complete state (via passPeerReview from BASE_Pending_Peer_Review), the CCR server performs a preview integration between the BASE branch and the TEST branch. Similarly, when entering TEST_Ready_For_Customer (via passPeerReview from TEST_Pending_Peer_Review), a preview integration runs between the TEST branch and the LIVE branch. An additional preview occurs at UAT_Ready_For_Customer if the system includes a UAT phase.

The preview integration is a read-only operation -- it does not commit any changes to the target branch. Its purpose is to detect whether any diff chunks in the source item lack a matching base in the target branch, which would cause a merge conflict error during actual integration. By detecting these issues early, teams can resolve them before attempting the actual move.

A prerequisite for the preview integration to run is that the user who performed the passPeerReview transition must be logged into Perforce. If the user is not a Perforce user or is not logged in, the preview may not execute automatically.

Merge conflicts arise when the base for a diff chunk in the source item does not exist in the target environment. This typically happens when items have been modified in the target branch outside of the normal CCR workflow, or when multiple CCRs modify the same item and are progressed out of order.

---

Understanding which CCR states and transitions involve Perforce activity or ItemSet activity is essential for troubleshooting and operational planning.

States and transitions with Perforce activity:

• In_BASE (markPREPComplete destination): In disconnected mode, the CCR server submits the uploaded ItemSet to Perforce. In connected mode, the p4 client submits directly. A Perforce changelist is created and attached to the CCR record.
• BASE_Complete (passPeerReview destination): Preview integration runs between BASE and TEST branches if the passPeerReview user is logged into Perforce.
• authorizeAndStartMoveToTEST transition: The CCR server performs a real integration from the BASE branch to the TEST branch in Perforce.
• TEST_Ready_For_Customer (passPeerReview destination): Preview integration runs between TEST and LIVE branches.
• authorizeAndStartMoveToLIVE transition: Real integration from TEST to LIVE branch in Perforce.

States and transitions with ItemSet activity:

• In_BASE (disconnected only): ItemSet upload from BASE environment to CCR server.
• authorizeAndStartMoveToTEST: TEST ItemSet created on the CCR server.
• Pending_Manual_Move_To_TEST: ItemSet downloaded and deployed to TEST.
• authorizeAndStartMoveToLIVE: LIVE ItemSet created on the CCR server.
• Pending_Manual_Move_To_LIVE: ItemSet downloaded and deployed to LIVE.

Note that in connected BASE environments, there is no ItemSet activity during In_BASE because changes go directly to Perforce. The workflow diagram in ICC400 uses an envelope icon for ItemSet activity and a server icon for Perforce activity, providing a visual guide to where transport operations occur.

---

CCR Transport best practices focus on maintaining the integrity of the source control system and keeping environments synchronized.

General Transport Best Practices:

• Get help when unsure: The CCR transport process involves irreversible Perforce operations. If you encounter an error or are uncertain about a step, contact InterSystems support rather than attempting to fix it yourself.
• Proper baselining: When CCR is first enabled for a system, all existing items must be exported from the environment and submitted to Perforce. This baselining process is completed with help from InterSystems and establishes the foundation for all future transport operations.
• Limit CCR scope: Keep individual CCRs focused on a single logical change. Large CCRs with many items increase the risk of merge conflicts and make peer review more difficult.
• Progress quickly: Stalled CCRs often block the work of others, especially when they touch common items. Move CCRs through the workflow as quickly as can be done safely.
• Upload promptly: Upload changes from BASE to Perforce as soon as they are ready. Delays create bottlenecks.
• Maintain order: Progress CCRs in the order configuration changes were originally made. Out-of-order progression increases the risk of merge conflicts.
• Never modify controlled items in non-BASE environments: All changes must originate in BASE and flow through the CCR Transport workflow. Perforce must remain the source of truth. Proper configuration locks TEST/UAT/LIVE environments to prevent changes outside the CCR workflow.

Proper CCR Cancellation: Cancelling a CCR is a serious action that must be handled carefully:

• Only cancel if the change is no longer needed -- for example, if configuration was started for a new user who does not end up joining. Cancel is never the correct solution for errors in change progression such as merge conflicts.
• Always backout cancelled CCRs: When a CCR is cancelled, its changes must be backed out of every Perforce branch they have been integrated into. Then the backout ItemSets must be deployed to all affected environments. Failure to do this leaves orphaned changes in Perforce and environments, causing inconsistencies that can trigger merge conflicts for future CCRs.
• Improperly cancelled CCRs cause crises: If a CCR is cancelled without proper backout and deployment, the changes remain in Perforce branches and environments. Future CCRs that touch the same items will encounter unexpected states, leading to merge conflicts and confusion.

---