T39.2: CCR Transport Tools and Terminology

A changelist is the fundamental unit of change tracking in Perforce. It represents an atomic set of file modifications that are committed to the Perforce server in a single transaction. Every changelist receives a unique numerical identifier that serves as a permanent reference for that set of changes.

In the context of CCR, changelists serve as the bridge between the CCR workflow and the Perforce source control system. When a user creates an ItemSet and it is committed to Perforce, CCR packages all items from that ItemSet into a single changelist. This changelist is tagged with the CCR ID, establishing a clear link between the Perforce history and the CCR record.

It is important to understand that CCR does not submit the ItemSet file itself to Perforce. The ItemSet is a CCR-specific packaging mechanism. What gets submitted to Perforce are the individual items (class files, routines, web files, etc.) contained within that ItemSet. The changelist groups these items together as an atomic unit.

When a CCR is progressed from BASE to TEST using the authorizeAndStartMoveToTEST transition, Perforce integrates all changelists tagged with that CCR ID from the BASE branch to the TEST branch. The integration itself is submitted as a new changelist, also tagged with the CCR ID, creating a clear audit trail of how changes move between environments.

Source control, of which Perforce is one example used as the InterSystems standard, serves as a database for flat file items. It provides central storage for code, full versioning capabilities, prevents permanent deletion, and maintains all history. Source control answers the fundamental questions about every change: Who made it, What was changed, When it was changed, Where it was changed, Why it was changed, and How. It is also known as SCM (Source Control/Code Management). Perforce Helix was chosen by InterSystems for its speed, powerful visual client (P4V), excellent branching and merge capability, and low administrative effort. Perforce uses a client-server model with a single server hosted at Cambridge HQ (perforce.iscinternal.com:1666) for all CCR deployments, with a replica server in Sydney for local work.

---

The Perforce Details section is a dedicated area within every CCR record that provides comprehensive information about the CCR's interaction with Perforce. This section serves as the primary interface for viewing transport-related information and performing Perforce actions.

The Perforce Details section displays several key pieces of information at the top level: the Perforce Branch path (showing which depot path the CCR targets), the Perforce Job identifier, the Access Token for API access, and links to View or Download the Transport Log.

Below these fields, a tabbed interface provides access to: Itemset Details, Submitted Changes, Create Itemset, Perforce Integration, and Perforce Backout.

The Submitted Changes tab is particularly important for auditing and troubleshooting. When clicked, it displays all changelists that have been committed to Perforce and associated with this CCR. Each changelist entry shows the changelist number (which links to Swarm for InterSystems employees), the user who performed the check-in, the timestamp of the submission, and a description that typically references the ItemSet name and creation details.

Expanding a changelist reveals the individual files affected. Each file line shows three action icons (View, History, Diff), the action taken on the file (such as add, edit, delete, branch, integrate, or undo), and the full depot path with revision number. The View icon displays the entire contents of the item at that specific revision and provides links to diff against any other revision. The History icon shows a color-coded list of all revisions to that item. The Diff icon shows the specific diff chunks that changed at that revision.

Notably, the Submitted Changes tab shows no information about ItemSets because ItemSets are a CCR transport mechanism that is effectively disposable once the items have been successfully committed to Perforce.

Errors related to Perforce or ItemSets also appear in the Perforce Details section. Orange or red messages in Perforce Details should never be ignored. While CCR may allow progression of a change despite error messages, this does not mean the CCR should be progressed. This flexibility exists for rare situations, but ignoring errors without fully understanding and resolving them has caused crises in production environments.

---

Authentication with Perforce through the CCR interface is essential for performing transport-related actions. The system is designed to streamline this process by leveraging the existing CCR credentials wherever possible.

When a user logs into CCR, the system automatically attempts to authenticate with Perforce using the same username and the same password. This means that if a user's Perforce credentials match their CCR credentials, Perforce login happens transparently without any additional action required.

However, if the Perforce password differs from the CCR password, the user must explicitly provide their Perforce password to CCR. This is done through the Perforce Login dialog, which can be accessed by clicking the "log into Perforce" link displayed in the Perforce Details section of any CCR record.

The Perforce Details section always displays the current Perforce authentication status in the upper-right corner. When authenticated, it shows "logged into Perforce as [username]". When not authenticated, it displays a clickable "log into Perforce" link alongside the username. Clicking this link opens a dialog where the user enters their Perforce password to complete authentication.

New CCR users may encounter the "user is not a Perforce user" error message. This occurs because CCR maintains a cached list of Perforce users and periodically queries the Perforce server to refresh this list. If a user's Perforce account was recently created, it may not yet be reflected in CCR's cache. InterSystems employees can force an immediate resynchronization by navigating to Menu > Refresh Perforce Users.

It is important to note that not all CCR actions require Perforce authentication. Creating and deploying ItemSets to environments does not require a Perforce account. However, certain actions do require being logged into Perforce: clicking the startMoveToXXXX transition on Tier 1 or Tier 2 CCRs, manually triggering integration or backout through the Perforce Details section, and performing integration previews. Users without a Perforce account are blocked from these actions entirely, while users with an account who are not yet logged in will see the "Log into Perforce" field flash and turn yellow as a prompt.

---

Diff chunks are the mechanism by which Perforce tracks granular changes within files. When a file is edited and submitted, Perforce compares the new version against the previous revision and identifies the specific sections that differ. Each such section is called a diff chunk and represents either an insertion of new content, an update to existing content, or a deletion of content.

This granular tracking is fundamental to how CCR transport works. When changes need to move between Perforce branches (for example, from BASE to TEST), the integration process does not copy entire files. Instead, it moves only the diff chunks associated with the specific CCR record being progressed. This is a critical design feature because it allows multiple CCRs to make changes to the same file and progress independently, as long as their changes affect different sections of the file.

For integration to succeed, the base content for each diff chunk in the source branch must exist in the corresponding location in the target branch. The "base" is the original content that the diff chunk modifies. If Perforce can find this base content in the target item, it can apply the diff chunk cleanly.

A merge conflict occurs when Perforce cannot find the expected base content in the target environment. This typically happens when another CCR has already modified the same section of the file in the target branch, changing the base content that the current integration expects to find. When a merge conflict is detected, the integration fails with a merge conflict error and must be resolved before the CCR can progress.

An important nuance illustrated in the course material: if CCR #1 makes a change to one section of a file and CCR #2 makes a change to a completely different section of the same file, CCR #2 can successfully progress to TEST even if CCR #1 has not yet been progressed. This is because the diff chunks do not overlap, so the base content for CCR #2's changes still exists in the target branch regardless of whether CCR #1 has been integrated.

---

ItemSets are the packaging mechanism CCR uses to transport items between the Perforce server and client environments. They are found within the Perforce Details section of each CCR record, under the Itemset Details tab.

The CCR record distinguishes between two types of ItemSets. ItemSets for server are those submitted to CCR from the BASE environment -- they contain items that need to be committed to Perforce. Once successfully committed, these ItemSets are hidden by default. ItemSets for client are generated by CCR for deployment to target environments (such as TEST or LIVE). Once successfully deployed to all required environments, these are also hidden by default. A Show All toggle allows viewing of hidden ItemSets for audit purposes.

A common misconception is that ItemSets contain only diff chunks. In fact, ItemSets contain metadata plus complete copies of each item. The diff chunks associated with a CCR are integrated to a target branch and merged to create new revisions of items. The ItemSets then contain all of the new revisions (complete files) of the items for that CCR. This is important because it means ItemSets are self-contained deployment packages.

ItemSets for client have four possible statuses: Undeployed (not yet attempted), Deployed (successfully deployed, no further action required), Error (an issue must be fixed before progressing the CCR), and Downloaded (the ItemSet has been sent to the environment's filesystem but has not yet been loaded into the namespace).

ItemSets have a 24-hour expiration window to help prevent deploying outdated versions of items. If an ItemSet has expired, the Refresh Itemset button becomes available, allowing the user to generate a new ItemSet based on the latest version of items in Perforce.

When a user without a Perforce account uploads an ItemSet from a client environment to CCR, the ItemSet is queued on the CCR server. The entire queue for an organization is committed to Perforce when any user who is logged into Perforce uploads an ItemSet, or when any logged-in user clicks the "commit queue" link next to an uncommitted ItemSet. When the queue is committed, all resulting changelists reference the user who caused the queue to clear, not necessarily the user who originally created the ItemSet.

---

The source workspace is a fundamental concept in CCR Transport. It is a file-based replicate of the Perforce branch for a given environment, residing on the local file system of that environment's server. The source workspace mirrors the structure and content of the corresponding Perforce branch, containing the head revisions of all items.

The Perforce branch path follows a hierarchical naming convention: //custom_ccrs/<Cc>/<Site>/<Sys>/<Env>/, where custom_ccrs is the top-level depot for CCR applications, <Cc> is the country code, <Site> is the site code, <Sys> is the system code, and <Env> is the environment (BASE, TEST, UAT, or LIVE). For example, the CCR application's own BASE branch is //custom_ccrs/us/ISCX/CCR/BASE/. The corresponding source workspace on the environment follows the same structure: \source\custom_ccrs\us\ISCX\CCR\BASE\.

Within each branch, items are automatically organized into subfolders based on their type. Standard subfolders include cls/ for class files, cspapp/ for web files (JavaScript, CSP, CSS), inc/ for include files, prj/ for project files, rtn/ for routines, rul/ for Interoperability Message Rules, lut/ for Interoperability Lookup Tables, schema/hl7/ for HLv2 Custom Schemas, and ds/ for Business Intelligence dashboard definitions. Additional custom subfolders (such as /java) can be created as needed, with custom ImplementCCR routines or CCR Event Handlers providing automation support.

The read/write status of files in the source workspace is a critical part of Perforce's concurrency control framework. Read-only files indicate they are not currently checked out, while read/write files indicate active checkout. Users must never manually change file permissions through operating system file controls, as this bypasses Perforce's tracking and can lead to synchronization problems.

Two special subdirectories have specific integration rules: the /internal subdirectory is never integrated to LIVE and is intended for unit test files and sample data, while the /backup subdirectory is never integrated to any target branch and is used for items like System Default Settings.

For TEST and LIVE environments, the local source workspace should always exactly match the corresponding Perforce branch. For the BASE environment, the workspace should match the BASE branch except for items that are actively being edited (checked out). This consistency between the source workspace and Perforce branches is what enables reliable ItemSet deployment and ensures that the code running in each environment matches what Perforce records as the current state.

---

Baselining is the process of synchronizing the content of Perforce branches with the actual state of the environments they represent. It establishes a known, consistent starting point from which all future changes can be tracked, compared, and integrated.

The importance of baselining cannot be overstated. Without a proper baseline, Perforce has no reference point for any items that existed in the environment before CCR was introduced. If a user edits an item that was never baselined (never added to Perforce), several critical problems arise. First, it becomes impossible to diff the change because Perforce does not have a previous revision to compare against. Second, it becomes impossible to perform a proper backout because backing out would effectively delete the entire item rather than reverting it to a known previous state.

Baselining also establishes a clean integration history across all branches. When environments are properly baselined, each branch starts with a consistent set of items, and all subsequent changes are tracked as diff chunks relative to these baselines. Without this clean starting point, diff chunks become inconsistent between environments, leading to merge conflicts that are difficult or impossible to resolve cleanly.

The baselining process typically requires the involvement of an InterSystems employee due to its technical complexity and the need for direct Perforce access. For Tier 1 environments (which are directly connected to Perforce), baselining sometimes requires direct Perforce access. For Tier 2 environments (which connect to Perforce through the CCR online server), baselining always requires direct Perforce access.

Proper baselining is a prerequisite for reliable CCR transport operations. Organizations that skip or incompletely perform baselining will encounter increasing difficulties as changes accumulate, because the inconsistencies between Perforce's understanding of the code and the actual state of the environments will compound over time.

---