T41.2: Configures Namespaces, Databases, Memory

Mapping Types and Use Cases

Global Mappings: Allow multiple namespaces to share data by mapping globals to databases other than the namespace's default. Can map entire globals or specific subscript ranges. Example: `^Orders(2020):(2023)` maps years 2020-2022 (upper bound exclusive) to an ARCHIVE database while current data stays in production.

Common uses: Shared configuration data, multi-tenant reference data, data partitioning by date/region.

Routine Mappings: Share individual ObjectScript routines (MAC/INT/INC) across namespaces without duplication. Must map dependencies: if a routine uses $$$macros, include files must also be mapped.

Common uses: Shared utility routines, centralized business logic, system routine access.

Package Mappings (Recommended for Classes): Make entire ObjectScript packages visible across namespaces. Hierarchical - mapping `MyApp` provides access to all `MyApp.*` classes and sub-packages. System packages (%Library, %SYS) are automatically mapped to all namespaces.

Common uses: Shared class libraries, framework components, system package access.

Configuration and Maintenance

Creating Mappings:

• Via Management Portal: Navigate to namespace configuration, select appropriate mapping tab (Global/Routine/Package), add mapping with target database
• Programmatically: Use Config.MapGlobals, Config.MapRoutines, or Config.MapPackages classes in ObjectScript

Managing Mappings:

• Mappings take effect immediately (no restart required)
• Can view, modify, or delete through Management Portal or Config.* classes
• Stored in iris.cpf configuration file

Important Rules:

• Mapped content hides local content with the same name
• Most specific mapping takes precedence when overlaps occur
• Remote mappings (ECP) add network latency to access times
• Dependencies must be mapped together (routines with their INC files, etc.)

Database File Basics

InterSystems IRIS databases are stored in IRIS.DAT files that dynamically expand as needed, assuming free disk space is available. During database creation, administrators specify the initial size in megabytes. The default initial size is 1 MB, but administrators should size databases appropriately for expected data volumes. Databases can grow until they reach 32 terabytes when using the default 8KB block size.

Expansion Parameters

• Expansion parameter: Determines how much to add when growth is needed. Default of 0 indicates automatic calculation as 12% of current size or 10 MB, whichever is larger. When using the 12% calculation, expansion size is capped at 1GB.
• Maximum parameter: Specifies the maximum size the database can reach. Default of 0 indicates no maximum. Administrators should set appropriate maximums to prevent databases from consuming all available disk space.

Database Configuration Considerations

• Keep database sizes manageable based on available infrastructure for backup, restore, and integrity checks
• InterSystems IRIS supports multivolume databases to spread data across multiple IRIS.DAT files when needed
• Stream globals should be global-mapped to separate databases configured with 64KB block sizes for optimal performance
• Administrators can modify database sizes through the Management Portal without restarting IRIS
• The portal warns administrators when a specified size is within 90% of available disk space and requires confirmation

Database Limits

• Absolute limit: 15,998 databases per instance (practical limits are lower)
• Directory information cannot exceed 256KB: maximum_DBs = 258048/(avg_DB_path_length + 3)
• Mirrored databases count twice toward the limit, effectively limiting mirrored database count to 7,499
• Operating system restrictions on open files also limit simultaneously active databases

Overview

Memory allocation is one of the most critical configuration tasks after installing InterSystems IRIS. The system defaults to automatic allocation of 25% of total physical memory for the database cache, but this is explicitly documented as not appropriate for production use. Administrators must manually configure memory allocations before deploying to production or performing performance tests.

Memory configuration is managed through the Memory and Startup page at System Administration > Configuration > System Configuration > Memory and Startup.

Database Cache (Global Buffer Pool)

• System memory allocated for buffering data retrieved from physical databases
• Reduces costly I/O operations and provides much of IRIS's performance benefits
• For production systems, select "Specify Amount" instead of "Automatically" and enter an appropriate allocation in megabytes
• If multiple block sizes are enabled through the DBSizesAllowed setting, allocate memory for each enabled block size separately
• The number of global buffers created for each block size equals the allocation divided by the block size
• For ECP systems, allocate at least 50MB of 8KB buffers for ECP control structures

Routine Cache

• Stores compiled routines in memory
• Default "Automatic" setting allocates 10% of the database cache for the default 8KB buffer size
• Minimum: 80MB; Maximum: 1020MB
• For typical production instances with properly sized database caches, automatic routine cache allocation is sufficient

Shared Memory Heap (gmheap)

• Provides memory for IRIS operations other than routine and database caches
• Configured on the Advanced Memory page at System Administration > Configuration > Additional Settings > Advanced Memory
• Applications using many SQL queries or enabling parallel dejournaling require additional gmheap allocation
• Monitor usage through the Shared Memory Heap Usage page at System Operation > System Usage

Per-Process Memory (bbsiz)

• Specifies maximum memory allocable to a new IRIS process
• Allowed range: 256KB to 2,147,483,647KB
• InterSystems recommends setting to -1 (resolves to maximum value) for most circumstances
• Used for symbol tables, I/O device structures and buffers
• Allocated in increasing extents as required until maximum is reached
• If processes receive errors, increase this value and restart the affected process

Lock Table Purpose

The lock table in InterSystems IRIS manages locks that control concurrent access to resources. When multiple processes need coordinated access to data, IRIS uses the lock table to track which processes hold locks and which are waiting.

Sizing Considerations

• Lock table size determines how many concurrent locks the system can maintain
• If applications request more locks than the table can hold, operations may fail or block unexpectedly
• Size based on expected concurrent workload and the locking patterns of applications
• Applications using fine-grained locking (many small locks) require larger lock tables than applications using coarse-grained locking (fewer, larger locks)

Configuration Methods

Lock table configuration is typically performed through:

• CPF configuration file
• Management Portal settings for advanced system parameters

Monitoring

Monitoring lock table usage helps administrators determine if the current allocation is sufficient or if adjustment is needed.

Signs of Lock Table Exhaustion

• Unexpected lock timeouts
• Blocked processes
• Degraded application performance during concurrent operations

Journaling Overview

Journaling is a critical feature in InterSystems IRIS that records all changes made to databases in sequential journal files.

Database Journal Property

Each database has a journaling property that determines whether changes to that database are journaled. When creating or editing databases through the Management Portal, administrators can set the "Journal globals?" property to Yes or No.

Journaling State Impact

• Databases with journaling enabled: Provide the ability to restore to any point in time and replay transactions after failures, but incur the overhead of writing changes to journal files.
• Databases without journaling: Cannot be recovered to specific points in time if corruption or failures occur.

Best Practices

• Temporary data should use dedicated databases configured not to journal globals, rather than the default IRISTEMP database
• The journaling state is visible in the Management Portal's Local Databases page (Journal column)
• Ensure all databases containing critical application data or persistent state are journaled
• Databases storing only transient or easily reconstructible data may run without journaling for improved performance
• System databases like IRISLIB and IRISSYS have specific journaling configurations that support system operations

Purpose

Configuration merge is a powerful feature in InterSystems IRIS that allows administrators to customize the configuration parameter file (CPF) during deployment by specifying a configuration merge file. On UNIX and Linux systems, administrators can provide a merge file that IRIS uses to update the default CPF before the instance starts for the first time.

Use Cases

This capability is essential for:

• Automated deployments
• Infrastructure-as-code implementations
• Maintaining consistent configurations across multiple instances

Merge File Structure

The merge file contains configuration parameters organized in the same structure as the CPF file, with sections like [Startup], [config], [Namespaces], etc.

How It Works

During installation, IRIS reads the merge file and applies the specified settings to the default CPF, overriding default values with the merge file values. This allows administrators to pre-configure:

• Memory allocations
• Namespace definitions
• Database settings
• Security parameters
• Other system settings

All configuration happens without manual intervention through the Management Portal.

Benefits

• Supports version control of configuration settings
• Enables repeatable deployments
• Reduces manual configuration errors
• Ideal for container deployments, automated provisioning systems, and large-scale installations

After the initial merge and startup, the resulting CPF reflects all merged settings and can be further modified through the Management Portal or programmatically as needed.

Overview

Proper sizing of disk and memory resources is essential for InterSystems IRIS performance and reliability. Administrators must calculate requirements based on workload characteristics, growth projections, and performance targets. Undersized systems experience poor performance and operational issues, while significantly oversized systems waste resources and budget.

Disk Space Requirements

Database Storage: Each database is stored in IRIS.DAT files that grow dynamically. Key considerations:

• Maximum database size: 32 TB with 8KB blocks (default)
• Initial size: Set during database creation based on expected data volume
• Growth pattern: Databases expand by 12% of current size or 10 MB (whichever is larger), capped at 1 GB per expansion
• Planning rule: Allocate 2-3x the initial data size to accommodate growth without frequent expansions
• Example: For 100 GB of initial data, provision 200-300 GB of disk space

Journal Storage: Journal files record all database changes chronologically:

• Size planning: Allocate disk space for 2x daily transaction volume as a baseline
• Retention period: Factor in journal purge policies and disaster recovery requirements
• Example: If the system generates 50 GB of journal data daily, allocate at least 100 GB for journal storage
• Mirror configurations require additional journal space on both primary and backup

Backup Storage: Plan for backup staging areas if using local backups:

• Full backups: Space equal to total database size
• Incremental backups: Additional space based on daily change rate
• Retention: Multiple backup generations multiply space requirements

System and Temporary Storage:

• Installation directory: 2-5 GB depending on platform and components
• IRISTEMP database: Size based on application sorting, grouping, and temporary operations
• Typical range: 10-50 GB for production systems with moderate SQL usage

Total Disk Calculation Example: For a system with 500 GB production data:

• Databases: 500 GB × 2.5 (growth factor) = 1,250 GB
• Journals: 100 GB daily × 2 = 200 GB
• Backups: 500 GB (full) + 50 GB (incrementals) = 550 GB
• System/temp: 20 GB
• Total: ~2,020 GB (2 TB)

Memory Requirements

Database Cache (Global Buffer Pool): The most critical memory allocation for IRIS performance:

• Default: Automatic allocation at 25% of physical RAM is explicitly NOT production-ready
• Production recommendation: Allocate 50-80% of available RAM for database cache
• Buffer count formula: (Allocated MB × 1024) ÷ Block size in KB
• Example: 8,192 MB with 8KB blocks = (8192 × 1024) ÷ 8 = 1,048,576 buffers
• Multiple block sizes: Each block size requires separate allocation
• 8KB: Standard data and indices (primary allocation)
• 64KB: Stream globals and large objects (typically 5-10% of 8KB allocation)
• ECP consideration: Allocate at least 50 MB of 8KB buffers for ECP control structures

Routine Cache: Stores compiled routines in memory:

• Automatic setting: 10% of database cache (for 8KB buffers)
• Range: Minimum 80 MB, Maximum 1,020 MB
• For most production systems with properly sized database caches, automatic allocation is sufficient
• Exception: Systems with very large codebases may benefit from manual tuning

Shared Memory Heap (gmheap): Memory for system operations beyond caching:

• Default allocation is typically sufficient for basic operations
• Increase for: Heavy SQL query processing, parallel dejournaling, ECP configurations
• Monitor via: System Operation > System Usage > Shared Memory Heap Usage page
• Symptom of insufficiency: Errors related to shared memory allocation or SQL query failures

Per-Process Memory (bbsiz): Maximum memory per IRIS process:

• Recommended setting: -1 (resolves to maximum value: 2,147,483,647 KB)
• Used for: Symbol tables, I/O device structures and buffers, local variables
• Allocated dynamically as processes require additional memory
• Symptom of insufficiency: errors in processes

Total Memory Calculation Example: For a server with 64 GB physical RAM:

• Database cache: 64 GB × 65% = 41.6 GB (~42 GB)
• Routine cache: 42 GB × 10% = 4.2 GB (capped at 1020 MB = 1 GB)
• Gmheap: 256 MB (baseline)
• Operating system: 64 GB × 20% = 12.8 GB
• Application overhead: ~8 GB
• Total allocated: ~42 GB to IRIS + ~21 GB to OS/overhead

Calculation Formulas Summary

Buffer Count: ``` Global Buffers = (Allocated_Memory_MB × 1024) ÷ Block_Size_KB ``` Example: (16384 MB × 1024) ÷ 8 KB = 2,097,152 buffers

Database Expansion Size (automatic): ``` Expansion_Size = MAX(Current_Size × 0.12, 10 MB), capped at 1 GB ``` Example: 5 GB database → MAX(5120 × 0.12, 10) = MAX(614.4, 10) = 614.4 MB

Routine Cache Size (automatic): ``` Routine_Cache = Database_Cache_8KB × 0.10 Bounded by: MIN(MAX(80 MB, calculated_value), 1020 MB) ``` Example: 40 GB database cache → 4 GB × 10% = 409.6 MB (within bounds)

Planning Considerations

Workload Analysis:

• Transaction volume: Higher throughput requires more database cache
• Concurrent users: More users increase memory and disk I/O requirements
• Query complexity: SQL-heavy workloads need larger gmheap allocations
• Data characteristics: Large objects (streams, PDFs) benefit from 64KB buffers

Growth Projections:

• Historical growth rate: Analyze past 6-12 months of growth
• Business projections: Factor in expected business expansion
• Seasonal patterns: Account for peak periods in retail, healthcare, etc.
• Retention policies: Consider how data retention rules affect long-term storage

Performance Targets:

• Response time SLAs: May require larger cache to maintain sub-second queries
• Batch processing windows: Larger caches can significantly speed up bulk operations
• High availability requirements: Mirror configurations need sufficient capacity on both nodes

Best Practices:

• Start with documented recommendations and tune based on actual performance metrics
• Monitor cache hit ratios; target >95% for database cache efficiency
• Review disk space monthly; set alerts at 80% capacity
• Test resource allocations in pre-production before deploying to production
• Document sizing assumptions and calculations for future capacity planning

CPF Overview

The configuration parameter file (iris.cpf) is the central repository for system-wide configuration information in InterSystems IRIS. Located in the manager directory (install-dir/mgr/), this file contains most configurable settings for an IRIS instance.

CPF Organization

The CPF is organized into sections that group related parameters:

• [Startup]: Initialization settings
• [config]: General configuration
• [Namespaces]: Namespace definitions
• [Map.namespace]: Namespace mappings

A default CPF is deployed with every instance and serves as the baseline configuration.

Modification Methods

Administrators can modify the CPF through three primary methods:

1. Management Portal: User-friendly interface for most common configuration changes, automatically updating the CPF when administrators save changes through various configuration pages.

2. Terminal: Programmatic access to configuration settings through system classes and utilities, enabling scripted configuration changes.

3. Direct text editing: For advanced scenarios or bulk changes, administrators can directly edit the CPF file using a text editor while IRIS is stopped. This method requires careful attention to syntax and parameter validity.

Critical CPF Parameters

• Memory allocations (globals, routines, gmheap)
• Database directory paths
• Namespace definitions and mappings
• Security settings
• Network configuration (superserver port)
• Startup behavior

Restart Requirements

When making changes that require an IRIS restart, the Management Portal displays a warning message indicating that modifications won't take effect until restart. Some changes take effect immediately without restart, but if any change requiring restart is made, all changes are held until restart occurs.

Best Practices

• Always backup the CPF before making significant changes
• Document the purpose of customizations for future reference
• Include the CPF in backup procedures and version control systems to preserve configuration alongside application code

Overview

The InterSystems IRIS Management Portal displays numerous calculated values that help administrators understand system state and performance. These calculated values are derived from raw system metrics and provide meaningful insights that raw numbers alone cannot convey.

Memory-Related Calculated Values

• Buffer counts: Number of buffers created for each block size (calculated as allocated memory divided by block size)
• Routine cache size (automatic): 10% of database cache for 8KB buffers, minimum 80MB, maximum 1020MB
• Available gmheap memory: Total gmheap minus allocated memory for various subsystems

The Management Portal displays these calculations on the Memory and Startup page and the Shared Memory Heap Usage page.

Database-Related Calculated Values

• Size utilization: Current database size as a percentage of maximum (when maximum is set)
• Multivolume threshold: Size at which new volumes are created
• Expansion size (automatic): 12% of current size or 10MB, whichever is larger, capped at 1GB

The Local Databases page displays these calculated metrics alongside raw database information.

Database Hash Values

The Management Portal's GetNamespace method returns database hash values (crhash) for each database mapped to a namespace:

• Hashes are checksums calculated from database configuration and content state
• Clients can use these hashes to detect configuration changes without retrieving full database information
• When a database configuration or content changes, its hash value changes
• Useful for development tools maintaining local caches of database information

Performance-Related Calculated Values

• Global buffer cache hit ratios: Percentage of database requests served from memory without disk I/O
• Routine cache efficiency metrics
• Lock table utilization percentages

These calculated values appear on various monitoring pages under System Operation and help administrators identify performance bottlenecks and tune system parameters.