T3.1: Uses Python with InterSystems IRIS

Knowledge Review - InterSystems IRIS Development Professional

1. Identifies Embedded Python capabilities in InterSystems IRIS

Key Points

  • In-Process Execution: Python runtime runs inside IRIS kernel for lowest latency
  • Language Keyword: Write Python methods in IRIS classes using `Language = python`
  • iris Module: Access IRIS features via `import iris` for globals, SQL, transactions, classes
  • Bidirectional Integration: Call Python from ObjectScript and ObjectScript from Python
  • Direct Global Access: Manipulate multidimensional storage structures with `iris.gref()`
  • SQL Stored Procedures: Create SQL functions, procedures, and triggers in Python

Detailed Notes

Overview

Embedded Python allows the Python runtime to execute within the same process as InterSystems IRIS, providing the tightest possible integration and lowest latency of all Python options. This architecture eliminates network overhead and serialization costs associated with client-server approaches.

Language Integration

  • Method Declarations: Write Python methods directly in IRIS class definitions by adding `Language = python` to ClassMethod or Method declarations
  • Bidirectional Calls: Python methods can be called from ObjectScript and vice versa, enabling seamless polyglot development
  • Service Requirement: The %Service_CallIn service must be enabled in the Management Portal for Embedded Python to function

The iris Module

The iris module serves as the gateway to IRIS functionality from Embedded Python. After importing iris, developers gain access to:

  • Transaction Processing: `iris.tstart()`, `iris.tcommit()`
  • Global Arrays: `iris.gref('^globalname')`
  • Class Instantiation: `iris.Package.ClassName._New()`
  • SQL Execution: `iris.sql.prepare()`

Data Type Handling

  • Automatic Conversions: The module automatically handles data type conversions between Python and ObjectScript
  • Supported Types: Lists, dictionaries, and custom objects
  • Property Access: Embedded Python can access persistent class properties, invoke class methods, and manipulate global nodes using familiar Python syntax

SQL Stored Procedures

Developers can create SQL stored procedures, functions, and triggers entirely in Python, executing with in-process performance.

Ideal Use Cases

  • Machine Learning: Importing libraries like numpy, pandas, scikit-learn for data analytics
  • Python Ecosystem: Leveraging Python's rich ecosystem for specific computations
  • Migration: Gradually migrating ObjectScript logic to Python while maintaining full integration with IRIS features

Documentation References

2. Evaluates Python strategy options for specific use cases

Key Points

  • DB-API/pyODBC: External SQL access for client applications and frameworks (pandas, SQLAlchemy)
  • Native API: Client-server access to globals and non-relational data structures
  • Python Gateway (PEX): IRIS calls external Python code for interoperability productions
  • Embedded Python: In-kernel execution for highest performance and tightest integration
  • Performance Trade-off: Client-server has network latency; Embedded Python has in-process speed
  • Use Case Mapping: Match architecture to application requirements and deployment constraints

Detailed Notes

Overview

InterSystems IRIS offers five distinct Python integration strategies, each optimized for different use cases. Understanding when to use each approach is critical for application architecture decisions.

DB-API Driver

  • Purpose: PEP 249-compliant relational database access from external Python applications
  • Ideal For: REST APIs, dashboards, web frameworks (Flask, Streamlit), and tools expecting standard database connectivity
  • Execution: Runs outside the IRIS process, connecting over TCP/IP or shared memory
  • Best For: External data processing and analytics workflows

pyODBC

  • Purpose: Legacy systems or environments requiring ODBC connectivity
  • Note: DB-API is now recommended for IRIS 2022.1 and later

Native API

  • Purpose: Client-server access to IRIS-specific features beyond SQL
  • Features: Direct global manipulation, class method invocation, and transaction control from external Python processes
  • Ideal For: Applications needing hierarchical or multidimensional data access patterns that don't map cleanly to SQL tables

Python Gateway (PEX Framework)

  • Purpose: Inverts control flow, allowing IRIS to initiate execution of external Python code
  • Architecture: Supports interoperability productions where IRIS orchestrates Python business services, processes, and adapters running as separate external services
  • Essential When: Integrating existing Python applications or when Embedded Python is unavailable

Embedded Python

  • Purpose: Highest-performance option, running Python within the IRIS kernel process
  • Benefits: Eliminates network latency, enables direct memory access to globals, allows seamless mixing of Python and ObjectScript within the same class
  • Excels At: Data-intensive operations close to the database, importing third-party Python packages, creating SQL stored procedures, augmenting ObjectScript classes with Python methods

Selection Criteria

  • Execution Location: Whether Python runs inside or outside IRIS
  • Performance Requirements: Latency tolerance for your use case
  • Direct Global Access: Need for direct global manipulation
  • Ecosystem Integration: Integration with external Python ecosystems
  • Control Flow: Whether IRIS initiates Python execution or Python initiates IRIS connections

Python Support Capabilities Matrix

  • SQL Stored Procedures: Only Embedded Python
  • Augmenting IRIS Classes: Only Embedded Python
  • External Client Applications: Only DB-API/Native API
  • Interoperability: Python Gateway is primary choice
  • Direct Global Manipulation: All except DB-API/pyODBC

Documentation References

Exam Preparation Summary

Critical Concepts to Master:

  1. Embedded Python Architecture: Understand that Python runs in-process with IRIS kernel (not client-server)
  2. Language Keyword: Know how to declare Python methods in IRIS classes with `Language = python`
  3. iris Module: Memorize key capabilities - transactions, globals, SQL, class access
  4. Five Python Options: DB-API, pyODBC, Native API, Python Gateway, Embedded Python
  5. Use Case Mapping: Match each option to appropriate scenarios (client apps, interoperability, in-kernel logic)
  6. Performance Characteristics: Embedded Python = lowest latency; client-server = network overhead
  7. Global Access: Know which options support direct global manipulation (not DB-API/pyODBC)
  8. Capabilities Matrix: Memorize which options support SQL stored procedures, augmenting classes, interoperability

Common Exam Scenarios:

  • Identifying when to use Embedded Python vs. DB-API for a given application requirement
  • Recognizing Python Gateway (PEX) as the solution for IRIS-initiated external Python execution
  • Understanding that SQL stored procedures in Python require Embedded Python
  • Knowing that external client applications (Flask, Streamlit) use DB-API, not Embedded Python
  • Selecting Native API when direct global access is needed from external Python
  • Understanding that Embedded Python provides lowest latency through in-process execution
  • Recognizing that augmenting existing IRIS classes with Python methods requires Embedded Python

Hands-On Practice Recommendations:

  • Write Python methods in IRIS classes using `Language = python` keyword
  • Import and use the iris module to access globals, execute SQL, and instantiate IRIS classes
  • Create a SQL stored procedure in Python and call it from SQL
  • Compare performance between DB-API (external) and Embedded Python (in-process) for same operation
  • Use `iris.gref()` to manipulate global nodes from Embedded Python
  • Build a simple external Python client using DB-API to query IRIS data
  • Evaluate a use case scenario and justify the selection of appropriate Python strategy
  • Enable %Service_CallIn in Management Portal to allow Embedded Python execution

Key Distinctions to Remember:

  • Embedded Python iris module (in-kernel) vs. DB-API iris module (client-server) - different APIs, not interchangeable
  • Python calls IRIS (DB-API, Native API) vs. IRIS calls Python (Python Gateway, Embedded Python)
  • SQL-only access (DB-API, pyODBC) vs. Full IRIS feature access (Native API, Embedded Python)
  • External process (DB-API, Native API, Python Gateway) vs. Same process (Embedded Python)
  • Shared memory option (DB-API when client on same machine) vs. TCP/IP required (remote clients)

Quick Reference - Python Strategy Decision Tree:

  1. Need SQL stored procedures/functions/triggers? → Embedded Python
  2. Need to augment existing IRIS classes with Python methods? → Embedded Python
  3. External Python client application (Flask, Streamlit, analytics)? → DB-API
  4. IRIS must call external Python code/services? → Python Gateway (PEX)
  5. Direct global access from external Python? → Native API
  6. Highest performance, tightest integration, in-process? → Embedded Python
  7. Interoperability production with Python components? → Python Gateway (PEX)
  8. Legacy IRIS versions (pre-2022.1) or existing ODBC infrastructure? → pyODBC

Important Implementation Details:

  • Python methods in classes must use underscore (_) instead of percent (%) in names: `iris._SYSTEM` not `iris.%SYSTEM`
  • Enable %Service_CallIn service or Embedded Python will fail with IRIS_ACCESSDENIED error
  • Use `iris.sql.prepare()` for parameterized SQL queries from Embedded Python
  • Status codes: 1 = success; use `iris.check_status()` to validate method execution
  • Global operations: `iris.gref('^globalname')` returns global reference; `.set()`, `.get()`, `.order()` methods available
  • Embedded Python supports optional shorter syntax for class references (IRIS 2024.2+)
  • DB-API uses `connect()` method; Embedded Python uses `import iris` (no connection needed - already in-process)

Report an Issue