# Olakai CLI The Olakai CLI is a command-line interface tool for managing your Olakai platform configuration directly from the terminal. Use it to configure agents, workflows, and platform settings - ideal for automation, scripting, and integration with coding agents. > **Current Version: 0.1.12** | Node.js 20+ | Package: `olakai-cli` | [GitHub](https://github.com/ailocalnode/olakai-cli) **Note:** The CLI is for configuration management. For reporting AI traffic (prompts, responses, metrics), use the [TypeScript SDK](/docs/olakai/api-sdk-typescript-client), [Python SDK](/docs/olakai/api-sdk-python-client), or [REST API](/docs/olakai/api-sdk-rest-api). The CLI uses secure OAuth 2.0 Device Flow authentication, allowing you to authenticate through your browser while keeping your terminal session secure. ## Installation Install the Olakai CLI globally using npm: ```bash npm install -g olakai-cli ``` Or using pnpm: ```bash pnpm add -g olakai-cli ``` ### Requirements - Node.js v20 or higher - An Olakai account with appropriate permissions ## Authentication The Olakai CLI uses OAuth 2.0 Device Flow for secure authentication. This flow allows you to authenticate in your browser while the CLI waits for authorization, eliminating the need to enter credentials directly in the terminal. ### Logging In To authenticate with Olakai: ```bash olakai login ``` This command will: 1. Generate a unique verification code 2. Display a URL and code for authentication 3. Automatically open your browser to the verification page 4. Wait for you to complete authentication in the browser **Example output:** ``` Logging in to Olakai (production)... To complete login, visit: https://app.olakai.ai/auth/device/verify?code=ABCD-1234 Or go to https://app.olakai.ai/auth/device/verify and enter code: ABCD-1234 Opening browser... Waiting for authorization... Logged in as user@example.com Account: your-account-id Role: ADMIN ``` ### Checking Your Identity To verify your current authentication status: ```bash olakai whoami ``` **Example output:** ``` Email: user@example.com Name: John Doe Role: ADMIN Account ID: clxxxxxxxxxxxxxxxxxxxxxxxxxx Environment: production ``` ### Logging Out To clear your stored credentials: ```bash olakai logout ``` ## Environment Options The CLI supports multiple environments for development and testing purposes: ```bash # Use production environment (default) olakai login # Use staging environment olakai --env staging login # Use local development environment olakai --env local login ``` Available environments: - `production` - Production Olakai instance (default) - `staging` - Staging environment for testing - `local` - Local development server ## Credential Storage The CLI securely stores authentication tokens in your system's configuration directory: - **Location**: `~/.config/olakai/credentials.json` - **Permissions**: Read/write for owner only (mode 0600) Tokens are: - Automatically refreshed when needed - Environment-specific (different tokens for production vs staging) - Validated before each command execution ## Command Reference ### Authentication Commands | Command | Description | |---------|-------------| | `olakai login` | Authenticate with Olakai using browser-based OAuth flow | | `olakai logout` | Clear stored credentials and log out | | `olakai whoami` | Display current user information and authentication status | ### Activity Commands | Command | Description | |---------|-------------| | `olakai activity list` | List prompt requests with filtering options | | `olakai activity get ` | Get detailed information about a prompt request | | `olakai activity kpis` | Get aggregated KPI values for agents/workflows | ### Agents Commands | Command | Description | |---------|-------------| | `olakai agents list` | List all agents | | `olakai agents get ` | Get agent details | | `olakai agents create` | Create a new agent | | `olakai agents update ` | Update an agent | | `olakai agents delete ` | Delete an agent | Manage your Olakai agents - the AI entities that process requests and report metrics. #### List Agents ```bash # List all agents in table format olakai agents list # Output as JSON olakai agents list --json # Include KPI definitions olakai agents list --include-kpis ``` **Example output:** ``` ID NAME ROLE WORKFLOW API KEY ------------ -------------------- -------- ------------ ------- clxxxxxxxxx... Customer Support Bot WORKER clxxxxxxxxx... Active clyyyyyyyy... Order Processor WORKER - None clzzzzzzzz... Main Coordinator COORDINATOR clxxxxxxxxx... Active ``` #### Get Agent Details ```bash # Get detailed information about an agent olakai agents get # Output as JSON olakai agents get --json ``` **Example output:** ``` ID: clxxxxxxxxxxxxxxxxxxxxxxxxxx Name: Customer Support Bot Description: Handles customer inquiries and support tickets Role: WORKER Source: CLI Category: support Workflow ID: clyyyyyyyyyyyyyyyyyyyyyyyyyy Workflow: Customer Service Workflow API Key: ID: clzzzzzzzzzzzzzzzzzzzzzzzzzz Key: olakai_sk_xxxx...xxxx Status: Active KPI Definitions: - Response Time (NUMERIC) - Customer Satisfaction (NUMERIC) ``` #### Create Agent ```bash # Create a basic agent olakai agents create --name "My Agent" # Create with full options olakai agents create \ --name "Customer Support Bot" \ --description "Handles customer inquiries" \ --role WORKER \ --workflow \ --with-api-key \ --category support # Output as JSON olakai agents create --name "My Agent" --json ``` **Options:** | Option | Description | |--------|-------------| | `--name ` | Agent name (required) | | `--description ` | Agent description | | `--role ` | `WORKER` or `COORDINATOR` (default: WORKER) | | `--workflow ` | Workflow ID to assign | | `--with-api-key` | Create an API key for this agent | | `--category ` | Agent category | | `--json` | Output as JSON | **Important:** When using `--with-api-key`, save the API key immediately - it will not be shown again. #### Update Agent ```bash # Update agent name olakai agents update --name "New Name" # Update multiple fields olakai agents update \ --name "Updated Agent" \ --description "Updated description" \ --role COORDINATOR \ --workflow # Clear workflow assignment olakai agents update --workflow "" ``` #### Delete Agent ```bash # Delete an agent (requires confirmation) olakai agents delete # Skip confirmation prompt olakai agents delete --force ``` ### Workflows Commands | Command | Description | |---------|-------------| | `olakai workflows list` | List all workflows | | `olakai workflows get ` | Get workflow details | | `olakai workflows create` | Create a new workflow | | `olakai workflows update ` | Update a workflow | | `olakai workflows delete ` | Delete a workflow | Manage workflows that organize and group your agents. #### List Workflows ```bash # List all active workflows olakai workflows list # Include agent details olakai workflows list --include-agents # Include inactive workflows olakai workflows list --include-inactive # Output as JSON olakai workflows list --json ``` **Example output:** ``` ID NAME AGENTS STATUS ------------ ---------------------- ------ ------ clxxxxxxxxx... Customer Service 3 Active clyyyyyyyy... Order Processing 2 Active clzzzzzzzz... Data Analysis Pipeline 5 Inactive ``` #### Get Workflow Details ```bash # Get detailed information about a workflow olakai workflows get # Output as JSON olakai workflows get --json ``` **Example output:** ``` ID: clxxxxxxxxxxxxxxxxxxxxxxxxxx Name: Customer Service Description: Handles all customer-facing AI interactions Status: Active Agents: 3 Created: 2024-01-15T10:30:00.000Z Updated: 2024-01-20T14:45:00.000Z Agents: - Customer Support Bot (WORKER) - API Key: Active - Escalation Handler (WORKER) - API Key: Active - Service Coordinator (COORDINATOR) - API Key: Active ``` #### Create Workflow ```bash # Create a basic workflow olakai workflows create --name "My Workflow" # Create with description olakai workflows create \ --name "Customer Service" \ --description "Handles all customer-facing AI interactions" # Output as JSON olakai workflows create --name "My Workflow" --json ``` **Options:** | Option | Description | |--------|-------------| | `--name ` | Workflow name (required) | | `--description ` | Workflow description | | `--json` | Output as JSON | #### Update Workflow ```bash # Update workflow name olakai workflows update --name "New Name" # Update description olakai workflows update --description "Updated description" # Activate a workflow olakai workflows update --active # Deactivate a workflow olakai workflows update --inactive ``` #### Delete Workflow ```bash # Delete a workflow (requires confirmation) olakai workflows delete # Skip confirmation prompt olakai workflows delete --force ``` ### KPI Commands | Command | Description | |---------|-------------| | `olakai kpis list` | List all KPI definitions | | `olakai kpis get ` | Get KPI definition details | | `olakai kpis create` | Create a new KPI definition | | `olakai kpis update ` | Update a KPI definition | | `olakai kpis delete ` | Delete a KPI definition | | `olakai kpis context-variables` | List available formula variables | | `olakai kpis validate` | Validate a formula expression | | `olakai kpis templates` | List available classifier templates | ### Custom Data Commands | Command | Description | |---------|-------------| | `olakai custom-data list` | List all custom data configurations | | `olakai custom-data get ` | Get custom data configuration details | | `olakai custom-data create` | Create a new custom data configuration | | `olakai custom-data update ` | Update a custom data configuration | | `olakai custom-data delete ` | Delete a custom data configuration | ### KPI Commands (Details) Manage KPI (Key Performance Indicator) definitions. KPIs allow you to define custom metrics computed from AI traffic data using formulas, classifiers, or LLM-based data capture. #### List KPIs ```bash # List all active KPI definitions olakai kpis list # Filter by agent olakai kpis list --agent-id # Include inactive KPIs olakai kpis list --include-inactive # Output as JSON olakai kpis list --json ``` **Example output:** ``` ID NAME CALCULATOR AGGREGATION STATUS ------------ -------------------- ---------- ----------- ------ clxxxxxxxxx... Response Quality formula AVERAGE Active clyyyyyyyy... Security Risk Score formula MAX Active clzzzzzzzz... Code Detection Rate classifier SUM Active ``` #### Get KPI Details ```bash # Get detailed information about a KPI olakai kpis get # Output as JSON olakai kpis get --json ``` **Example output:** ``` ID: clxxxxxxxxxxxxxxxxxxxxxxxxxx Name: Response Quality Score Description: Measures response quality based on length and format Type: USER_DEFINED Category: quality Agent ID: - Calculator: formula Aggregation: AVERAGE Unit: points Status: Active Calculator Parameters: { "formula": "IF(LEN(Response) > 100, 10, 5)" } Created: 2024-01-15T10:30:00.000Z Updated: 2024-01-20T14:45:00.000Z ``` #### Create KPI ```bash # Create a formula-based KPI olakai kpis create \ --name "PII Detection Rate" \ --calculator-id formula \ --formula "IF(PII detected, 1, 0)" \ --aggregation SUM \ --unit "incidents" # Create with full options olakai kpis create \ --name "Response Quality" \ --calculator-id formula \ --formula "IF(LEN(Response) > 100, 10, 5)" \ --description "Measures response quality" \ --category quality \ --agent-id \ --aggregation AVERAGE \ --unit "points" # Output as JSON olakai kpis create --name "My KPI" --calculator-id formula --formula "1" --json ``` **Options:** | Option | Description | |--------|-------------| | `--name ` | KPI name (required) | | `--calculator-id ` | Calculator type: `formula`, `classifier`, or `llm-data` (required) | | `--formula ` | Formula expression (required for formula calculator) | | `--description ` | KPI description | | `--type ` | `PREDEFINED` or `USER_DEFINED` (default) | | `--category ` | KPI category for grouping | | `--agent-id ` | Associate KPI with specific agent | | `--unit ` | Display unit (e.g., "ms", "%", "count") | | `--aggregation ` | SUM, AVERAGE, COUNT, MIN, MAX, or LATEST (default) | | `--json` | Output as JSON | **Calculator Types:** | Calculator | Purpose | Required Parameters | |-----------|---------|---------------------| | `formula` | Evaluate Excel-like expressions | `--formula` | | `classifier` | AI-based classification using context variables (`{{variable}}` syntax) | Configured in UI | | `llm-data` | LLM data capture detection | None | #### Update KPI ```bash # Update KPI name olakai kpis update --name "New Name" # Update formula olakai kpis update --formula "IF(CODE detected, 1, 0)" # Update multiple fields olakai kpis update \ --name "Updated KPI" \ --description "Updated description" \ --aggregation AVERAGE # Activate/deactivate KPI olakai kpis update --active olakai kpis update --inactive ``` #### Delete KPI ```bash # Delete a KPI (requires confirmation) olakai kpis delete # Skip confirmation prompt olakai kpis delete --force ``` #### Context Variables List available variables that can be used in KPI formulas: ```bash # List all context variables olakai kpis context-variables # Include agent-specific variables olakai kpis context-variables --agent-id # Output as JSON olakai kpis context-variables --json ``` **Example output:** ``` Available context variables for KPI formulas: NAME TYPE SOURCE DESCRIPTION -------------- ------ ------ ---------------------------------------- Prompt string shared The prompt text sent to the LLM Response string shared The LLM response text Documents count number shared Number of attached documents PII detected boolean shared Whether PII was detected PHI detected boolean shared Whether PHI was detected CODE detected boolean shared Whether code was detected SECRET detected boolean shared Whether secrets were detected Use these variable names in your formula expressions, e.g.: IF(PII detected, 1, 0) Documents count * 10 IF(CODE detected AND SECRET detected, "high-risk", "normal") ``` #### Validate Formula Test a formula expression before creating a KPI: ```bash # Validate a formula olakai kpis validate --formula "IF(PII detected, 1, 0)" # Include agent context olakai kpis validate --formula "Custom Variable * 2" --agent-id # Output as JSON olakai kpis validate --formula "1 + 1" --json ``` **Example output (valid):** ``` Formula is valid! Result type: number ``` **Example output (invalid):** ``` Formula is invalid: Unknown variable: "Invalid Var". Available variables: Prompt, Response, ... Position: character 4 ``` For comprehensive formula syntax, operators, and functions, see the [Formula Reference](/docs/olakai/formulas). #### Classifier Templates List available classifier templates that provide pre-configured AI-based KPI calculators: ```bash # List all available templates olakai kpis templates # Output as JSON olakai kpis templates --json ``` **Available Templates:** | Template ID | Description | Output Range | |-------------|-------------|--------------| | `sentiment_scorer` | Analyzes sentiment of AI interactions | 1-5 score | | `time_saved_estimator` | Estimates time saved by the AI interaction | 0-60 minutes | #### Creating Classifier KPIs with Templates Use `--template-id` with `--calculator-id classifier` to create a KPI from a pre-configured template: ```bash # Create a sentiment scoring KPI olakai kpis create --name "User Satisfaction" \ --calculator-id classifier --template-id sentiment_scorer \ --scope CHAT --agent-id $AGENT_ID # Create a time savings KPI olakai kpis create --name "Time Saved" \ --calculator-id classifier --template-id time_saved_estimator \ --scope CHAT --agent-id $AGENT_ID ``` **Template-Specific Options:** | Option | Description | |--------|-------------| | `--template-id ` | Classifier template to use (required with `classifier` calculator) | | `--scope ` | KPI scope: `PROMPT_REQUEST` or `CHAT` | | `--output-classes ` | Override the default output classes (JSON array of numbers) | | `--output-labels ` | Override the default output labels (JSON array of strings) | | `--sampling-factor ` | Sampling rate for classification (e.g., 1.0 = every event, 0.5 = 50%) | **Example with overrides:** ```bash # Create sentiment KPI with custom labels and 50% sampling olakai kpis create --name "User Satisfaction" \ --calculator-id classifier --template-id sentiment_scorer \ --scope CHAT --agent-id $AGENT_ID \ --output-classes '[1, 2, 3, 4, 5]' \ --output-labels '["Very Negative", "Negative", "Neutral", "Positive", "Very Positive"]' \ --sampling-factor 0.5 ``` ### Custom Data Commands | Command | Description | |---------|-------------| | `olakai custom-data list` | List all custom data configurations | | `olakai custom-data get ` | Get custom data configuration details | | `olakai custom-data create` | Create a new custom data configuration | | `olakai custom-data update ` | Update a custom data configuration | | `olakai custom-data delete ` | Delete a custom data configuration | Manage custom data configurations. Custom data fields allow you to send arbitrary metrics from your SDK integration that can be used as variables in KPI formulas. #### List Custom Data Configurations ```bash # List all custom data configurations olakai custom-data list # Output as JSON olakai custom-data list --json ``` **Example output:** ``` ID NAME TYPE DESCRIPTION ------------ ---------------------- ------ ---------------------------------------- clxxxxxxxxx... ToolsDiscovered NUMBER Total AI tools discovered clyyyyyyyy... ClassificationConfidence NUMBER AI classification confidence (0-1) clzzzzzzzz... model STRING Model used for the request ``` #### Get Custom Data Details ```bash # Get detailed information about a custom data config olakai custom-data get # Output as JSON olakai custom-data get --json ``` #### Create Custom Data Configuration ```bash # Create a NUMBER type custom data field olakai custom-data create \ --name "ToolsDiscovered" \ --type NUMBER \ --description "Total AI tools discovered by discovery agent" # Create a STRING type custom data field olakai custom-data create \ --name "model" \ --type STRING \ --description "Model used for the request" # Create a BOOLEAN type custom data field olakai custom-data create \ --name "IsAIPowered" \ --type BOOLEAN \ --description "Whether the tool is AI-powered" ``` **Options:** | Option | Description | |--------|-------------| | `--name ` | Configuration name (required) - used as variable name in formulas | | `--type ` | Data type: `STRING`, `NUMBER`, or `BOOLEAN` (required) | | `--description ` | Description of what this field represents | | `--json` | Output as JSON | **Important:** The `--name` must exactly match the key used in your SDK's `customData` object when reporting events. For example, if your KPI formula references `ToolsDiscovered`, you must: 1. Create a custom data config with `--name "ToolsDiscovered"` 2. Send `customData: { ToolsDiscovered: 15 }` in your SDK event #### Update Custom Data Configuration ```bash # Update name olakai custom-data update --name "NewName" # Update type olakai custom-data update --type NUMBER # Update description olakai custom-data update --description "Updated description" ``` #### Delete Custom Data Configuration ```bash # Delete a custom data config (requires confirmation) olakai custom-data delete # Skip confirmation prompt olakai custom-data delete --force ``` ### Activity Commands | Command | Description | |---------|-------------| | `olakai activity list` | List prompt requests with filters | | `olakai activity get ` | Get prompt request details | | `olakai activity kpis` | Get aggregated KPI values | Inspect AI activity and prompt requests. These commands allow you to validate your agent configurations by viewing the outcomes of AI interactions, including KPI values, analytics, and governance decorations. #### List Activity ```bash # List recent activity olakai activity list # Filter by agent olakai activity list --agent-id # Filter by workflow olakai activity list --workflow-id # Filter by date range olakai activity list --since 2025-01-01 --until 2025-01-31 # Pagination olakai activity list --limit 50 --offset 100 # Include prompt/response content olakai activity list --include-content # Include advanced analytics olakai activity list --include-analytics # Output as JSON (ideal for programmatic use) olakai activity list --json ``` **Example output:** ``` ID AGENT APP MODEL TOKENS TIME(ms) RISK STATUS ------------ ---------------- ------- ------------- ------ -------- ---- ---------- clxxxxxxxxx... Customer Support SDK gpt-4 1500 2340 No DECORATED clyyyyyyyy... Order Processor SDK claude-3 800 1200 Yes DECORATED clzzzzzzzz... Data Analyzer SDK gpt-4-turbo 2100 3100 No DECORATED Showing 3 of 150 results (offset: 0) Use --offset 20 to see more results ``` **Options:** | Option | Description | |--------|-------------| | `--agent-id ` | Filter by agent ID | | `--workflow-id ` | Filter by workflow ID | | `--since ` | Start date (ISO format) | | `--until ` | End date (ISO format) | | `--limit ` | Results per page (default: 20) | | `--offset ` | Skip first N results | | `--include-content` | Include prompt/response text | | `--include-analytics` | Include advanced analytics data | | `--json` | Output as JSON | #### Get Activity Details ```bash # Get details for a specific prompt request olakai activity get # Include prompt/response content olakai activity get --include-content # Output as JSON olakai activity get --json ``` **Example output:** ``` ID: clxxxxxxxxxxxxxxxxxxxxxxxxxx Created: 2025-01-13T10:30:00.000Z Agent: Customer Support Bot (clxxxxxx...) Workflow: Customer Service (clxxxxxx...) App: SDK Model: gpt-4 (CHAT) Tokens: 1500 Request Time: 2340ms High Risk: No Blocked: No Status: DECORATED Analytics: Task: Code Review Subtask: Security Analysis Time Saved: 15 minutes Risk Score: 0.2 KPIs: response_quality: 0.85 token_efficiency: 0.92 ``` **Options:** | Option | Description | |--------|-------------| | `--include-content` | Include full prompt/response text | | `--json` | Output as JSON | #### Get Activity KPIs Retrieve aggregated KPI values for an agent or workflow over a time period. ```bash # Get KPIs for an agent olakai activity kpis --agent-id # Get KPIs for a workflow olakai activity kpis --workflow-id # Specify date range olakai activity kpis --agent-id --since 2025-01-01 --until 2025-01-31 # Get period breakdown (hourly, daily, or weekly) olakai activity kpis --agent-id --period daily # Include per-prompt KPI atoms olakai activity kpis --agent-id --include-atoms # Output as JSON olakai activity kpis --agent-id --json ``` **Example output:** ``` Core KPIs: Complete Executions 150 Governance Compliance 94.5% ROI $825.00 Custom KPIs: NAME VALUE UNIT AGGREGATION DATA POINTS response_quality 0.85 - AVERAGE 150 token_efficiency 0.92 - AVERAGE 150 security_score 0.15 - MAX 150 Period Breakdown: 2025-01-13T00:00:00.000Z - 2025-01-14T00:00:00.000Z Complete Executions: 50 Governance Compliance: 96.0% ROI: $275.00 ``` **Options:** | Option | Description | |--------|-------------| | `--agent-id ` | Agent ID (required if no workflow-id) | | `--workflow-id ` | Workflow ID (required if no agent-id) | | `--since ` | Period start (ISO format) | | `--until ` | Period end (ISO format) | | `--period ` | Aggregation period: `hourly`, `daily`, or `weekly` | | `--include-atoms` | Include per-prompt KPI data | | `--json` | Output as JSON | **Core KPIs:** | KPI | Description | |-----|-------------| | Complete Executions | Total number of decorated prompt requests | | Governance Compliance | Percentage of requests that passed governance | | ROI | Calculated return on investment based on time saved | ### Global Options | Option | Description | |--------|-------------| | `-e, --env ` | Specify the environment (production, staging, local) | | `-V, --version` | Display CLI version | | `-h, --help` | Display help information | ## Troubleshooting ### "Not logged in" Error If you see this error, your token may have expired or you haven't authenticated yet: ```bash olakai login ``` ### Browser Doesn't Open Automatically If your browser doesn't open automatically, manually navigate to the URL displayed in the terminal and enter the verification code. ### Wrong Environment If you're connected to the wrong environment, log out and log back in with the correct environment flag: ```bash olakai logout olakai --env staging login ``` ### Authentication Timeout The verification code expires after a few minutes. If your login times out, simply run `olakai login` again to get a new code. ## Security Considerations - **Device Flow**: Uses RFC 8628 compliant device authorization flow - **No Password Entry**: Credentials are never entered directly in the terminal - **Token Expiry**: Access tokens expire after 30 days - **Secure Storage**: Credentials are stored with restrictive file permissions - **Environment Isolation**: Tokens are environment-specific to prevent accidental cross-environment operations