T1.2: Determines Data Structures in InterSystems IRIS

Overview

InterSystems IRIS provides three fundamental object class categories, each serving distinct architectural purposes.

Registered Objects (%RegisteredObject)

• Nature: Transient objects that exist only in memory during runtime
• Capabilities: Provide the core object interface including methods for creating instances, setting properties, and invoking instance methods
• Limitation: Cannot be saved to disk
• Use Cases: Ideal for temporary data structures, utility classes, and in-memory processing where persistence is not required

Persistent Classes (%Persistent)

• Inheritance: Extend %Persistent, which itself extends %RegisteredObject, inheriting all object interface capabilities
• Persistence Interface: Provides methods like %Save(), %OpenId(), %Delete(), and %Exists() for full CRUD operations
• Storage: Each persistent object instance is assigned a unique Object ID (OID) and stored in its own global node using a $List structure
• SQL Projection: Automatically projected to SQL tables, making them accessible through both object access and SQL queries
• Default Storage: Uses globals named with the class name appended with "D" for data and "I" for indexes

Serial Objects (%SerialObject)

• Nature: Special category designed for embedding within other objects
• No Independent Identity: Does not have an independent existence or unique OID
• Storage: Serialized and stored as part of parent object's data structure
• Example: An Address class defined as a serial object can be embedded in a Customer persistent class
• SQL Projection: Appear as multiple columns in the parent table, with column names formed by combining the property name, an underscore, and each embedded property name (e.g., Home_Street, Home_City)
• Use Cases: Ideal for complex data types that have no meaning independent of their parent entity

Overview

Index selection is critical for optimizing query performance in InterSystems IRIS.

Standard Indexes

• Purpose: Create an ordered mapping between one or more property values and the Object IDs of objects containing those properties
• Global Structure: Uses the class name with an "I" appended (e.g., ^MyApp.PersonI for MyApp.Person class)
• Subscript Layout:
• First subscript: Index name (allows multiple indexes in the same global)
• Second subscript: Collated data value (default SQLUPPER collation for case-insensitive sorting)
• Third subscript: Object ID
• Benefit: Enables rapid retrieval of objects matching specific criteria without scanning the entire extent

Bitmap Indexes

• Structure: Use compressed bitstrings where each bit position corresponds to an Object ID
• How They Work: For a given indexed value, the bitmap contains 1 for rows where that value is present and 0 where it is absent
• Best For: Low-cardinality columns (columns with few distinct values like State, Status, or Category) and complex queries combining multiple conditions
• Bitstring Functions: $Bit, $BitCount, $BitFind, $BitLogic for efficient logical operations (AND, OR) across multiple bitmap indexes
• Limitation: Only work with default storage structures using system-assigned numeric Object IDs

Index Selection Strategy

• When to Index: Fields frequently used in WHERE clauses, JOIN conditions, or ORDER BY clauses
• Without Indexes: The query optimizer must perform full table scans, examining every row to find matches
• High-Cardinality Columns: Use standard indexes for columns with many distinct values
• Low-Cardinality Columns: Consider bitmap indexes for columns with few distinct values
• Overhead Awareness: Indexes consume storage space and incur maintenance overhead during INSERT, UPDATE, and DELETE operations
• Special Index Types:
• IDKEY indexes: Define the unique identifier for objects
• Unique indexes: Enforce uniqueness constraints on specific fields or field combinations

Overview

InterSystems IRIS implements a unique unified data architecture where globals, objects, and SQL are three different views of the same underlying multidimensional storage.

Object Storage Model

• Storage Type: Every persistent class that uses %Storage.Persistent (the default) stores object instances in global nodes
• Storage Definition: Automatically managed by the class compiler, determines how properties map to global structures
• Default Naming: Data is stored in a global named with the complete class name plus a "D" suffix (^MyApp.PersonD)
• Node Structure: Each object instance occupies a single global node subscripted by its Object ID
• Property Storage: All non-transient properties are stored within that node using a $List structure

SQL Projection

• Automatic Table Creation: Each persistent class becomes a table with the same name
• Mapping: Properties become columns, each object instance becomes a row
• Object ID Access: Becomes the table's ID column (also accessible as %ID or ROWID)
• Bidirectional Definition: Define classes first and get SQL tables automatically, or define tables via DDL and get persistent classes automatically
• Identical Storage: The storage structures are identical regardless of which approach you use

Access Method Consistency

• Immediate Visibility: Data inserted via SQL INSERT is immediately accessible via object methods like %OpenId(), and objects saved via %Save() are immediately queryable via SQL SELECT
• Global Access: Provides the lowest-level view with direct manipulation using SET, KILL, MERGE, $ORDER, and $QUERY

Choosing Access Methods

• Object Access: For transactional business logic with strong typing and encapsulation
• SQL Access: For set-based operations and reporting
• Global Access: For low-level optimization or specialized data structures

All three access methods maintain consistency because they operate on the same underlying storage.

Overview

Stream data types solve the problem of storing and manipulating data larger than standard string length limits in InterSystems IRIS. Streams are essential for handling large objects such as documents, images, videos, XML files, or any content that exceeds the maximum string size.

Stream Class Hierarchy

InterSystems IRIS provides six main stream classes, all inheriting from %Stream.Object which defines the common stream interface.

Global Streams

• Classes: %Stream.GlobalCharacter and %Stream.GlobalBinary
• Storage: Data within the database in global nodes
• Chunking: System automatically splits stream data into chunks smaller than 32KB and writes into sequential global nodes
• Benefits: Full database integration, backup, restore, mirroring, and transaction processing participation
• Character Streams: Handle text data with character encoding awareness
• Binary Streams: Handle raw binary data like images or PDFs
• Use Cases: When you need full database integration, transactional consistency, and manageable data volume

File Streams

• Classes: %Stream.FileCharacter and %Stream.FileBinary
• Storage: External files on the file system; database stores only a reference to the file path
• Use Cases: Extremely large objects, or when files need to be shared with external applications
• Considerations:
• Require careful backup procedures (both database and referenced files)
• Don't participate in database mirroring
• Introduce dependencies on file system paths and permissions

Temporary Streams

• Classes: %Stream.TmpCharacter and %Stream.TmpBinary
• Purpose: Provide stream interfaces for transient data that doesn't need persistence
• Use Cases: Processing pipelines, data transformations, building responses for web services

Common Stream Methods

• Sequential Processing: Read(), Write(), Rewind(), AtEnd()
• Searching: FindAt() for searching within streams
• Error Handling: Always check status return values and examine %objlasterror when setting Filename property

Overview

InterSystems IRIS provides comprehensive native JSON support through multiple complementary approaches.

Dynamic Entities

• Classes: %Library.DynamicObject and %Library.DynamicArray
• Flexibility: Create and manipulate JSON structures at runtime without predefining schemas
• Literal Constructors: `set obj = {"name":"John", "age":30}` or `set arr = ["value1", "value2"]`
• Dynamic Expressions: Use parentheses for runtime evaluation, e.g., `{"date":($ZD($H,3))}`

Dynamic Entity Operations

Property Access:

• Dot syntax: `obj.name`
• Method syntax: `obj.%Get("name")`

Modification:

• `obj.age = 31` or `obj.%Set("age", 31)`
• `obj.%Remove("name")` to remove properties

Array Operations:

• `%Push()` and `%Pop()` for stack operations
• Methods can be chained for fluent interfaces

Serialization:

• `%ToJSON()`: Serialize dynamic entity to canonical JSON string
• `%FromJSON()` and `%FromJSONFile()`: Deserialize JSON strings or files back to dynamic entities

%JSON.Adaptor for Typed Classes

• Purpose: Enable any registered, serial, or persistent class to import and export JSON
• Usage: Add %JSON.Adaptor as a superclass
• Methods Gained: %JSONExport(), %JSONExportToStream(), %JSONExportToString(), %JSONImport()
• Customization: Property parameters for custom field names, inclusion/exclusion, and special formatting
• XData Mapping Blocks: Define multiple mappings for different JSON representations
• %JSON.Formatter: Additional control over JSON output formatting with indentation and line breaks

JSON and SQL Integration

• Store JSON in VARCHAR fields
• Use JSON functions in SQL queries
• Create indexes on JSON properties for efficient querying

This comprehensive JSON support makes InterSystems IRIS ideal for modern API-driven applications, microservices architectures, and scenarios requiring flexible schema-less data alongside traditional relational structures.