T1.2: Designs Advanced Schemas

Knowledge Review - InterSystems IRIS SQL Specialist

1. Differentiates date and time data types (%Date, %Time, %TimeStamp, %PosixTime)

Key Points

  • %Date: Stores date only, logical value is integer days since December 31, 1840
  • %Time: Stores time only, logical value is seconds past midnight (0-86399)
  • %TimeStamp: Combined date and time in YYYY-MM-DD HH:MM:SS.nnnnnnnnn format
  • %PosixTime: Encoded 64-bit timestamp, seconds elapsed from 1970-01-01 00:00:00
  • Each type has distinct SqlCategory values: DATE, TIME, TIMESTAMP, and POSIXTS

Detailed Notes

%Date Data Type

The %Date data type stores date-only values with a logical representation as an integer representing the number of days since December 31, 1840 (day 0).

  • Storage format: Compact integer representation
  • Date range: Supports dates from 0001-01-01 to 9999-12-31
  • Pre-1840 dates: The epoch (day 0) is 1840-12-31, but the integer is signed — dates before 1840-12-31 are stored as negative day counts. For example, 0001-01-01 is roughly day -671,861. This is why the supported range extends backwards well before 1840.
  • Display format: Locale-specific formats or ODBC format (YYYY-MM-DD)
  • SqlCategory: DATE

%Time Data Type

The %Time data type represents time-of-day values, storing the logical value as the number of seconds past midnight.

  • Value range: 0 to 86399 seconds
  • Display format: HH:MM:SS
  • SqlCategory: TIME
  • Use case: Efficient time arithmetic and comparisons

%TimeStamp Data Type

The %TimeStamp data type combines both date and time components into a single value.

  • Format: YYYY-MM-DD HH:MM:SS.nnnnnnnnn
  • Precision: Up to nine digits of fractional seconds
  • Conversion: Logical value obtained from $HOROLOG format using $ZDATETIME(h,3)
  • SqlCategory: TIMESTAMP
  • Use case: Recording exact moments in time (standard ODBC timestamp format)

%PosixTime Data Type

The %PosixTime data type offers an alternative timestamp representation using Unix epoch encoding.

  • Storage: Encoded 64-bit signed integer representing elapsed seconds with microsecond precision
  • Epoch: 1970-01-01 00:00:00 (Unix epoch, NOT the 1840 %Date epoch). The encoded form at the epoch is 1152921504606846976; the encoding packs the sign, seconds, and microseconds into 64 bits.
  • Date range: 0001-01-01 to 9999-12-31 (same span as other temporal types, achieved via the 64-bit signed integer range)
  • Precision: Exactly 6 digits of fractional seconds (microseconds)
  • SqlCategory: POSIXTS
  • Note: Dates before 1970 map to negative signed values in the encoding. %PosixTime and %Date use different epochs, so do not convert between them by simple subtraction — use TO_POSIXTIME() / TO_DATE() for correctness.

SqlCategory and Type Conversion

Each data type has a distinct SqlCategory classification that controls comparison operations.

  • Conversion functions: Use DATE(), TO_TIMESTAMP(), or TO_POSIXTIME() when mixing temporal types in comparisons or indexes
  • Schema design impact: Choice of temporal data type affects storage efficiency, query performance, and interoperability with external systems
  • Recommendations:
  • %Date and %Time: Most compact storage for date-only or time-only values
  • %TimeStamp: Human-readable precision timestamps
  • %PosixTime: Unix timestamp compatibility and high-precision temporal calculations

2. Collection Properties (list Of, array Of)

Key Points

  • Two built-in collection property forms: `Property X As list Of %Type` and `Property X As array Of %Type`.
  • `list Of`: stored inline in the parent row as a single `$LISTBUILD`-encoded column. No child table is created.
  • `array Of`: stored as a separate child table with a foreign key back to the parent — each array element becomes one row.
  • Both forms expose additional SQL projections: `()` table-valued function (for `list Of`) or a child table like `Parent_Collection` (for `array Of`) for SELECT queries.
  • Searching inside a `list Of` in SQL: use the `FOR SOME %ELEMENT(list) (value = 'X')` predicate. Don't use `LIKE '%X%'` against the $LISTBUILD blob — it'll match raw bytes and give wrong results.
  • Indexing inside collections: use a collection index (`Index i On Allergies(ELEMENTS)` for a list, or a bitmap on the child table for arrays).

Detailed Notes

list Of %String — inline storage

Class App.Patient Extends %Persistent {
  Property Name As %String;
  Property Allergies As list Of %String;
}

SQL view of the table:

IDNameAllergies
1Smith$LB("Penicillin","Peanut")

The Allergies column holds the $LISTBUILD-encoded blob directly; there is no child table. To search:

SELECT ID, Name FROM App.Patient
 WHERE FOR SOME %ELEMENT(Allergies) (%Value = 'Penicillin')

array Of %String — child table storage

Class App.Patient Extends %Persistent {
  Property Name As %String;
  Property Phones As array Of %String;
}

IRIS creates a child table App.Patient_Phones with columns (Patient, element_key, Phones). Each phone is one row. Standard SQL JOIN/WHERE works against the child table.

Comparison

Aspectlist Ofarray Of
SQL storageinline column, $LISTBUILD-encodedseparate child table
Keyed?positional (ordinal)keyed (arbitrary key per element)
Typical usesmall, fixed-form multi-value (allergies, tags)keyed attributes (phone numbers by type)
SearchingFOR SOME %ELEMENTJOIN on child table
Indexingcollection-element index on parentindex on child table

Documentation References

3. Serial Objects (%SerialObject)

Key Points

  • A `%SerialObject` class defines a reusable structure (e.g. `Address`) that is embedded into the parent row rather than stored as a separate persistent object.
  • Properties of a %SerialObject are serialised with `$LISTBUILD` and stored as a single column on the parent.
  • SQL projects the structure as flattened underscored columns: `HomeAddress_Street`, `HomeAddress_City`, etc.
  • No separate table; no foreign key. You cannot `SELECT * FROM App.Address` — a %SerialObject doesn't have its own extent.
  • Use when the embedded structure has no identity of its own and is always owned by its parent (addresses, money-with-currency, phone-with-extension).

Detailed Notes

Class App.Address Extends %SerialObject {
  Property Street As %String;
  Property City As %String;
}

Class App.Person Extends %Persistent {
  Property Name As %String;
  Property HomeAddress As App.Address;
}

SQL projection of App.Person:

IDNameHomeAddress_StreetHomeAddress_City
1Smith42 Main StBoston

To query a field inside the serial: SELECT Name, HomeAddress_City FROM App.Person. The underscore-flattened columns are SQL-addressable; the App.Address class itself is not a table.

Documentation References

4. Stream Properties (%Stream.GlobalCharacter, %Stream.GlobalBinary)

Key Points

  • Stream properties (`%Stream.GlobalCharacter`, `%Stream.GlobalBinary`) are designed for values larger than the 3.6 MB string limit — documents, images, serialized blobs.
  • Stored separately from the row's main data; the row holds only a locator.
  • SQL limitations: streams cannot be used directly in `WHERE`, `ORDER BY`, `GROUP BY`, or in most expressions. They are NOT indexable as a whole.
  • Supported SQL: `SELECT` (retrieves the stream), `INSERT`/`UPDATE` with a stream value, length/position functions where they apply.
  • To search INSIDE a stream, use `%CONTAINS` with a configured iFind/text index (`Index ci On (Content) As %iFind.Index.Basic`), not `LIKE`.

Detailed Notes

Class App.Document Extends %Persistent {
  Property Title As %String;
  Property Content As %Stream.GlobalCharacter;
  Index ci On (Content) As %iFind.Index.Basic;
}

Typical operations:

-- OK: retrieve stream (client assembles the stream from the cursor)
SELECT Title, Content FROM App.Document WHERE ID = 1

-- NOT supported: WHERE on a stream column directly
SELECT * FROM App.Document WHERE Content LIKE '%confidential%'   -- will error or return wrong results

-- Correct way: %CONTAINS with iFind index
SELECT ID, Title FROM App.Document WHERE %ID %FIND search_index(ci, 'confidential')

Documentation References

5. Computed and Calculated Properties

Key Points

  • `Calculated`: the value is computed every time it is read; no storage. Requires a `Get()` accessor (or an `SqlComputeCode` that IRIS uses to auto-generate one).
  • `SqlComputed`: IRIS generates the stored value from `SqlComputeCode`. When combined with `SqlComputeOnChange` (listing dependent properties), the value is recalculated automatically whenever those properties change — and then persisted to the column, so SQL queries against it are fast.
  • `Transient`: in-memory only; not stored, not projected to SQL.
  • Rule of thumb: use `Calculated` for simple always-fresh derivations that do NOT need to be queried; use `SqlComputed` + `SqlComputeOnChange` when the derived value needs to be indexable / WHERE-filterable against a large table.
  • Important: a `Calculated` property does NOT participate in SQL indexes (there's nothing to index). If `WHERE Age = 30` must hit an index on a 10M-row table, use `SqlComputed`, not `Calculated`.

Detailed Notes

// FullName — always fresh, recomputed on read, not indexable
Property FullName As %String [ Calculated,
  SqlComputeCode = {Set {*} = {FirstName}_" "_{LastName}},
  SqlComputed ];

// Age — stored, auto-updated when DateOfBirth changes, indexable
Property Age As %Integer [
  SqlComputed,
  SqlComputeCode = {Set {*} = $SYSTEM.SQL.Functions.DATEDIFF("yy",{DateOfBirth},$HOROLOG)},
  SqlComputeOnChange = DateOfBirth ];
Index AgeIdx On Age;

For a 10M-row table queried frequently on WHERE Age = 30, the second form is correct — the computed value is materialised into the row, and an index on Age makes the lookup O(log n) instead of full table scan.

6. Columnar Storage and OLTP vs Analytical Workloads

Key Points

  • Default IRIS storage is row-oriented — great for OLTP (fast inserts, lookups by primary key, retrieving whole rows).
  • For analytical workloads (aggregations over millions of rows on a few columns), columnar storage is dramatically more efficient: compression, vectorised scans, and CPU-friendly memory layout.
  • Enable columnar storage per-class (`Storage Columnar`) or per-column (`[ Columnar ]` on a property) depending on version/needs.
  • Mixed workloads** (fast OLTP inserts + periodic analytical reports) often benefit from **tiered storage: row-store for the hot path, columnar on specific columns for aggregation.
  • Columnar storage pays back on queries like `SELECT SUM(Amount) FROM Sales WHERE RegionId = ...` over a large table. It does NOT help much on `SELECT * FROM Sales WHERE ID = 1`.

Detailed Notes

Columnar storage is configurable directly in DDL — no class definition required.

-- Mixed: row-storage table with columnar storage on the columns that drive aggregation
CREATE TABLE App.Sale (
  OrderId  INTEGER PRIMARY KEY,
  RegionId INTEGER,
  Amount   NUMERIC(18,2) WITH STORAGETYPE = COLUMNAR,
  Quantity INTEGER       WITH STORAGETYPE = COLUMNAR
);

-- Full columnar table — every column columnar, best for analytic-only workloads
CREATE TABLE App.SalesFact (
  Id       BIGINT PRIMARY KEY,
  RegionId INTEGER,
  Amount   NUMERIC(18,2),
  Quantity INTEGER
) WITH STORAGETYPE = COLUMNAR;

-- Add a new columnar column to an existing table
ALTER TABLE App.Sale ADD COLUMN Discount NUMERIC(18,2) WITH STORAGETYPE = COLUMNAR;

The same effect can be expressed in an ObjectScript class definition (Property Amount As %Numeric(SCALE=2) [ Columnar ] for a column, or class-level [ StorageStrategy = Columnar ] for a full columnar table) — both routes produce the same physical layout.

7. Optimistic Concurrency Control

Key Points

  • Optimistic concurrency: readers don't lock — on update, check that the row hasn't changed since it was read.
  • Add a `%Integer`/`%TimeStamp` version column and declare it via the class parameter `VERSIONPROPERTY` (or the `VersionProperty` class keyword). IRIS increments it automatically on each successful save.
  • UPDATE logic then uses `WHERE ID = :id AND Version = :readVersion` — if zero rows update, another user beat you, and the application retries/merges/errors.
  • Alternative: compute an MD5/SHA hash of the row, recheck on update. Heavier; use VersionProperty when you can.
  • Pessimistic locking (`SELECT ... FOR UPDATE`) is the opposite strategy — simpler for short critical sections but blocks other readers.

Detailed Notes

Class App.Order Extends %Persistent [ VersionProperty = Version ] {
  Property Quantity As %Integer;
  Property Version As %Integer [ InitialExpression = 0 ];
}

Application pattern:

-- Read
SELECT Quantity, Version INTO :qty, :ver FROM App.Order WHERE ID = :id

-- ... user edits ...

-- Update, guarded by the version
UPDATE App.Order SET Quantity = :newQty WHERE ID = :id AND Version = :ver
IF %ROWCOUNT = 0 THEN
   -- lost-update: another session modified the row
END

Documentation References

Exam Preparation Summary

Critical Concepts to Master:

  1. Logical Storage Formats: Know that %Date stores days since Dec 31, 1840, %Time stores seconds past midnight, %TimeStamp uses ODBC format, %PosixTime uses encoded Unix timestamp
  2. SqlCategory Values: Memorize the four distinct categories (DATE, TIME, TIMESTAMP, POSIXTS) and their impact on comparisons
  3. Precision Differences: %TimeStamp supports up to 9 digits of fractional seconds, %PosixTime encodes exactly 6 digits, %Time has no fractional seconds
  4. Conversion Functions: Understand when to use DATE(), TO_TIMESTAMP(), TO_POSIXTIME() for type conversion
  5. Use Case Selection: Know when to choose each type - %Date for dates only, %Time for times only, %TimeStamp for general timestamps, %PosixTime for Unix compatibility

Common Exam Scenarios:

  • Selecting the appropriate temporal data type for a given application requirement
  • Converting between different temporal formats using SQL functions
  • Understanding the range and precision limitations of each type
  • Recognizing the encoded nature of %PosixTime values
  • Working with $HOROLOG and $ZTIMESTAMP special variables in SQL context

Hands-On Practice Recommendations:

  • Create tables with columns of each temporal type and insert sample data
  • Practice conversions between types using DATE(), TO_TIMESTAMP(), and TO_POSIXTIME()
  • Experiment with fractional seconds precision in %TimeStamp and %PosixTime
  • Query temporal data with different display modes (Logical, ODBC, Display)
  • Compare performance and storage characteristics of each type
  • Work with dates before 1970 to understand %PosixTime negative values

Report an Issue