--- title: "SIP Traces & Diagnostics" description: "Documentation for SIP Traces & Diagnostics" --- ## Table of Contents 1. [Overview & Signaling Inspection Architecture](#1-overview--signaling-inspection-architecture) 2. [Business & Operational Significance](#2-business--operational-significance) 3. [🎯 User Roles & Key Capabilities](#3--user-roles--key-capabilities) 4. [Visual Interface & Call Sequence Drawer](#4-visual-interface--call-sequence-drawer) 5. [Field & Control Reference](#5-field--control-reference) 6. [Kamailio siptrace Module Engine Mechanics](#6-kamailio-siptrace-module-engine-mechanics) 7. [AI NOC Copilot Deep Diagnostics](#7-ai-noc-copilot-deep-diagnostics) 8. [Troubleshooting & Direct CLI Inspection](#8-troubleshooting--direct-cli-inspection) 9. [Model Context Protocol (MCP) AI Integration](#9-model-context-protocol-mcp-ai-integration) 10. [Glossary](#10-glossary) --- ## 1. Overview & Signaling Inspection Architecture In **Ring2All SBC**, the **SIP Traces & Diagnostics** module provides deep-packet inspection, interactive call ladder diagram visualization, and automated protocol diagnostics for all SIP signaling transiting the carrier perimeter. Powered by Kamailio's native `siptrace` module and integrated with PostgreSQL storage (`kamailio.sip_trace`), this module captures complete raw SIP messagesβ€”including full headers and SDP session descriptionsβ€”directly at the network boundary. ``` External Network Ring2All SBC Core (Kamailio) Diagnostic Subsystems β”‚ β”‚ β”‚ │──────── SIP Signaling (INVITE) ────────>β”‚ β”‚ β”‚ β”œβ”€β”€β”€ In-Memory Inspection ───────────────>β”‚ β”‚ β”‚ Match IP / Trunk Filters β”‚ β”‚ β”‚ Capture Raw Headers & SDP β”‚ β”‚ β”‚ INSERT INTO kamailio.sip_trace ─────>β”‚ β”‚<─────── 100 Trying / 180 Ringing ───────│ β”‚ β”‚<─────── 200 OK (Answered) ──────────────│ β”‚ │──────── ACK (Session Established) ─────>β”‚ β”‚ β”‚ β”‚ β–Ό β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ CallSequenceDrawer UI β”‚ β”‚ β”‚ β”‚ β€’ Sequence Ladder Flow β”‚ β”‚ β”‚ β”‚ β€’ Syntax Highlighting β”‚ β”‚ β”‚ β”‚ β€’ AI NOC Copilot (MCP) β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` To eliminate the performance degradation and disk exhaustion risks associated with indiscriminate packet tracing in high-CPS production environments, Ring2All SBC implements: 1. **On-Demand Capture with Safety Timers**: Tracing is inactive by default and automatically disables itself after a user-defined interval (1, 5, 15, or 30 minutes). 2. **Selective Pre-Filtering**: Signaling can be filtered at ingestion by source/destination IP or specific SIP Trunk accounts. 3. **Interactive Sequence Ladder Diagrams**: Chronological visual flow charts showing message progression, directionality, and elapsed inter-message intervals. 4. **AI NOC Copilot Integration**: Native connection to the Model Context Protocol (MCP) AI diagnostics engine for automated root-cause analysis of signaling anomalies. --- ## 2. Business & Operational Significance * **Accelerated Interconnect Turn-Up**: Reduces the time required to onboard new telecom carriers and enterprise SIP trunks from days to minutes by instantly visualizing protocol mismatches. * **Elimination of Server Access Overhead**: Enables NOC and support personnel to diagnose complex signaling failures directly from the web interface without requiring SSH access, `sngrep`, or packet captures (`tcpdump`). * **Root Cause Identification for Silent Failures**: Immediately isolates common VoIP anomalies such as 32-second call drops (missing `ACK`), one-way audio (SDP IP mismatch), or continuous `407 Proxy Authentication` loops. * **Automated AI Triage**: Translates complex RFC 3261 protocol discrepancies into plain-language operational summaries and actionable remediation steps. --- ## 3. 🎯 User Roles & Key Capabilities | Role | Primary Use Case | Key Capabilities | | :--- | :--- | :--- | | **Carrier Interconnect Engineer** | Trunk Interoperability & Turn-Up | Validate SDP offer/answer exchanges, verify custom SIP headers (P-Asserted-Identity, Diversion), and inspect codec negotiations. | | **NOC Telephony Analyst** | Real-Time Fault Resolution | Enable on-demand tracing for reported calling issues, inspect visual call ladders, and identify SIP error codes. | | **Security Operations Engineer** | Anti-Fraud & Attack Forensics | Inspect malformed SIP requests, track registration flood attempts, and verify IP spoofing patterns. | | **SBC Core Administrator** | Signaling Pipeline Optimization | Govern trace retention intervals, verify PostgreSQL database pruning, and tune Kamailio `siptrace` buffers. | | **AI Platform Copilot / NOC Diagnostic Agent** | Autonomous Signaling Inspection & Ladder Analysis | Initiate on-demand SIP captures with safety timers, extract chronological message ladders, decode SDP codecs, and detect protocol violations (missing ACK, 401 loops, 503 errors). | --- ## 4. Visual Interface & Call Sequence Drawer The interface comprises an on-demand capture management banner, a grouped sessions DataGrid, and an interactive slide-in Call Sequence Drawer. ### 4.1 On-Demand Capture & Traced Sessions Ledger The primary view displays the active tracing status, capture parameters, and a consolidated ledger of all traced call sessions with message counts and final response codes. ![SIP Traces and Diagnostics View](/screenshots/sbc/reports/cdr/sip-traces/sip-traces-list.png) ### 4.2 Interactive Call Sequence Ladder Drawer Clicking **View Sequence** or selecting a Call-ID opens the drawer, featuring the chronological ladder diagram, raw message syntax highlighter, and AI analysis tools. ![SIP Call Sequence Ladder and Raw Message Highlighter](/screenshots/sbc/reports/cdr/sip-traces/sip-traces-ladder.png) --- ## 5. Field & Control Reference ### Capture Control Banner | Control Element | Options / Constraints | Description | | :--- | :--- | :--- | | **Capture Status Indicator** | `Active` (Green) / `Passive` (Gray) | Real-time state of the Kamailio `siptrace` ingestion engine. | | **Capture Duration** | `1 Min` (Recommended), `5 Min`, `15 Min`, `30 Min` | Automated safety countdown timer after which tracing automatically deactivates. | | **IP Filter (From/To)** | Valid IPv4 / IPv6 (Optional) | Restricts packet ingestion to messages originating from or destined to the specified address. | | **SIP Account / Trunk Filter** | Text string (Optional) | Filters traffic by matching specific user accounts or trunk identifiers in the Request-URI. | | **AI Profile for Diagnostics** | Configured AI Profiles dropdown | Selects the language model and domain context profile utilized for automated troubleshooting. | | **Live Capture Toggle** | `Start Live Capture` / `Stop Capture` | Manually starts or stops signaling recording. | ### Traced Sessions DataGrid Columns | Column | Data Type | Description | | :--- | :--- | :--- | | **Result** | Status Badge | Final termination status of the session (`200 OK`, `487 Request Terminated`, `404 Not Found`, etc.). | | **Date / Time** | Timestamp / Duration | Starting timestamp and total elapsed conversational duration. | | **From (Caller)** | Transport / Socket | Originating transport socket (e.g., `udp:192.168.11.91:5060`). | | **To (Callee)** | Transport / Socket | Ingress SBC listening socket receiving the request. | | **Call ID** | Monospace String | Unique SIP Call-ID header identifying the end-to-end dialog. | | **Messages** | Integer Badge | Total count of SIP requests and responses captured within the session. | | **Actions** | Action Button | Opens the interactive sequence ladder drawer for signaling visualization. | --- ## 6. Kamailio siptrace Module Engine Mechanics Ring2All SBC leverages Kamailio's `siptrace` module configured with asynchronous database logging: ```text # Kamailio siptrace module configuration loadmodule "siptrace.so" modparam("siptrace", "db_url", "postgres://kamailio:SECRET@localhost/kamailio") modparam("siptrace", "table", "sip_trace") modparam("siptrace", "trace_on", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_to_database", 1) modparam("siptrace", "trace_mode", 0) modparam("siptrace", "duplicate_uri", "sip:127.0.0.1:9060") ``` ### Dynamic Activation Lifecycle 1. **Activation via RPC**: When an operator clicks *Start Live Capture*, the backend issues a JSON-RPC command to Kamailio: ```json {"jsonrpc": "2.0", "method": "siptrace.status", "params": [1], "id": 1} ``` 2. **Selective Packet Duplication**: Inbound and outbound SIP packets matching transaction flags are serialized along with network metadata (source IP, destination IP, timestamp, protocol). 3. **Automated Safety Deactivation**: A Node.js timer enforces the selected duration limit, issuing an RPC call to disable `siptrace.status` upon expiration, ensuring production memory and disk stability. --- ## 7. AI NOC Copilot Deep Diagnostics Integrated with the **Model Context Protocol (MCP)**, the **Diagnose with MCP** engine provides instant expert analysis of captured call flows. ### Common Diagnosed Anomalies ``` Symptom: Call drops after exactly 32 seconds SIP Ladder Pattern: Caller Ring2All SBC Carrier β”‚ β”‚ β”‚ │────── INVITE ───────>│────── INVITE ────────>β”‚ β”‚<───── 200 OK ────────│<───── 200 OK ─────────│ β”‚ β”‚ β”‚ β”‚ (ACK Missing) β”‚ (ACK Missing) β”‚ β”‚ β”‚ β”‚ β”‚ [Timer H Fires] β”‚ [Timer H Fires] β”‚ β”‚<───── BYE ───────────│<───── BYE ────────────│ AI Diagnosis: "The carrier terminated the call after 32 seconds because no SIP ACK was received following the 200 OK. Check NAT traversal on the caller network; the Contact header advertised an unreachable private IP (192.168.1.50) without NAT proxy rewriting." ``` Other automated detection capabilities include: * **SDP Codec Mismatches**: Highlights incompatible audio payloads (e.g., caller offers only `PCMU` while callee requires `G.729`). * **Authentication Failures**: Pinpoints wrong passwords, realm mismatches, or missing nonce responses in `401 Unauthorized` / `407 Proxy Authentication Required` exchanges. * **DNS SRV & Transport Failures**: Identifies transport timeouts when resolving upstream carrier hostnames over TCP or TLS. --- ## 8. Troubleshooting & Direct CLI Inspection ### Checking Runtime Tracing Status via CLI Query Kamailio to verify if tracing is actively capturing packets: ```bash # Check if siptrace module is currently active kamcmd siptrace.status ``` ### Inspecting Captured Messages in PostgreSQL Retrieve the raw signaling history for a specific `Call-ID`: ```bash psql -U kamailio -d kamailio -c " SELECT id, time_stamp, direction, fromip, toip, method, status FROM sip_trace WHERE callid = '0_715506604@192.168.11.91' ORDER BY id ASC;" ``` ### Cleaning Up Historic Trace Data To free database storage space following intensive troubleshooting sessions: ```bash # Prune traces older than 7 days psql -U kamailio -d kamailio -c " DELETE FROM sip_trace WHERE time_stamp < NOW() - INTERVAL '7 days';" ``` --- ## 9. Model Context Protocol (MCP) AI Integration The SIP Traces & Diagnostics subsystem is tightly coupled with the Ring2All SBC Model Context Protocol (MCP) server, empowering NOC AI Copilots to trigger on-demand trace captures, inspect chronological message ladders, and perform autonomous root-cause analysis on signaling failures. ### Available MCP Tools | Tool Name | Operation Type | Risk Level | Description | | :--- | :--- | :--- | :--- | | `capture_and_analyze_sip_trace` | Autonomous Workflow | `operational` | Activates on-demand capture for a specified duration, collects live SIP packets matching filters, extracts the call ladder, and returns an expert diagnostic breakdown. | | `start_sip_trace_capture` | State Mutation | `operational` | Starts live packet capture in Kamailio with an auto-off countdown timer and optional IP or extension filters. | | `stop_sip_trace_capture` | State Mutation | `operational` | Immediately halts live SIP trace recording and deactivates the `siptrace` module in Kamailio. | | `get_sip_trace_capture_status` | Status Query | `read` | Retrieves current capture operational status, remaining safety timer seconds, and active pre-filters. | | `get_sip_traces` | Forensic Query | `read` | Queries recent historical SIP call traces recorded in PostgreSQL, grouped by Call-ID. | | `explain_sip_call` | Deep Protocol Analysis | `read` | Extracts the full chronological ladder diagram, parsed headers, and SDP media codecs for a specific Call-ID string. | | `diagnose_sip_error_spikes` | Real-time Triage | `diagnostic` | Scans recent SIP signaling packets for surges of 4xx, 5xx, or 6xx errors, groups failure rates by carrier/gateway, and automatically isolates root causes (404, 486, 487, 500, 503). | --- ### Tool Schemas & Payloads #### 1. `capture_and_analyze_sip_trace` ##### Input Schema ```json { "type": "object", "properties": { "duration_seconds": { "type": "number", "description": "Duration in seconds to capture live SIP traffic (default 30, range 5-120)." }, "target_ip": { "type": "string", "description": "Target source or destination IP address to filter (e.g. '192.168.11.91')." }, "extension": { "type": "string", "description": "Target extension, phone number or user to filter (e.g. '+15551234567')." }, "method": { "type": "string", "description": "SIP Method filter (e.g. 'INVITE', 'REGISTER', 'OPTIONS', 'BYE')." }, "callid": { "type": "string", "description": "Filter for a specific Call-ID string if known." } } } ``` ##### Output Payload Example ```json { "success": true, "duration_seconds": 30, "captured_messages_count": 8, "calls_analyzed": 1, "analysis": { "call_id": "0_715506604@192.168.11.91", "caller": "sip:2001@192.168.11.91", "callee": "sip:+18005550199@sbc.ring2all.com", "final_status": "200 OK", "codecs_negotiated": ["Opus/48000/2", "PCMU/8000"], "anomalies_detected": [], "summary": "Successful call establishment without signaling violations. Normal teardown observed via BYE." } } ``` --- #### 2. `explain_sip_call` ##### Input Schema ```json { "type": "object", "properties": { "callid": { "type": "string", "description": "The exact SIP Call-ID string to inspect and reconstruct into a ladder diagram." } }, "required": ["callid"] } ``` ##### Output Payload Example ```json { "call_id": "0_715506604@192.168.11.91", "total_messages": 6, "ladder": [ { "step": 1, "time_delta_ms": 0, "direction": "inbound", "from": "192.168.11.91:5060", "to": "192.168.10.31:5060", "method": "INVITE", "cseq": "1 INVITE", "summary": "Initial INVITE offering Opus and PCMU" }, { "step": 2, "time_delta_ms": 4, "direction": "outbound", "from": "192.168.10.31:5060", "to": "192.168.11.91:5060", "status": 100, "cseq": "1 INVITE", "summary": "100 Trying" }, { "step": 3, "time_delta_ms": 142, "direction": "outbound", "from": "192.168.10.31:5060", "to": "192.168.11.91:5060", "status": 180, "cseq": "1 INVITE", "summary": "180 Ringing" }, { "step": 4, "time_delta_ms": 2180, "direction": "outbound", "from": "192.168.10.31:5060", "to": "192.168.11.91:5060", "status": 200, "cseq": "1 INVITE", "summary": "200 OK answering call" }, { "step": 5, "time_delta_ms": 2192, "direction": "inbound", "from": "192.168.11.91:5060", "to": "192.168.10.31:5060", "method": "ACK", "cseq": "1 ACK", "summary": "ACK received, media session confirmed" }, { "step": 6, "time_delta_ms": 45120, "direction": "inbound", "from": "192.168.11.91:5060", "to": "192.168.10.31:5060", "method": "BYE", "cseq": "2 BYE", "summary": "Caller hang-up via normal BYE" } ], "root_cause_diagnosis": "Normal session completion. RFC 3261 compliance verified." } ``` --- #### 3. `start_sip_trace_capture` ##### Input Schema ```json { "type": "object", "properties": { "duration_seconds": { "type": "number", "description": "Duration in seconds for the capture timer (default 60, min 10, max 300)." }, "target_ip": { "type": "string", "description": "Optional IP address filter for the on-demand trace capture." }, "extension": { "type": "string", "description": "Optional extension or account ID filter for the on-demand trace capture." } } } ``` ##### Output Payload Example ```json { "success": true, "status": "capturing", "duration_seconds": 60, "filters": { "target_ip": "192.168.11.91", "extension": null }, "expires_at": "2026-09-08T15:35:00.000Z" } ``` --- ### Natural Language AI Prompts #### English Examples * *"Analyze if there are recent SIP error spikes and triage the root causes."* * *"Start a 60-second SIP trace capture targeting IP 192.168.11.91 and analyze any INVITE messages."* * *"Explain the SIP signaling ladder for Call-ID 0_715506604@192.168.11.91 and check if the ACK was received."* * *"What is the current status of live SIP trace capture on the SBC?"* * *"Stop the ongoing SIP trace capture immediately."* #### Spanish Examples (EspaΓ±ol) * *"Analiza si hay picos de errores SIP recientes y triage de causas raΓ­z."* * *"Inicia una captura SIP de 60 segundos filtrando por la IP 192.168.11.91 y analiza cualquier mensaje INVITE."* * *"Explica el diagrama de escalera SIP para el Call-ID 0_715506604@192.168.11.91 y verifica si se recibiΓ³ el ACK."* * *"ΒΏCuΓ‘l es el estado actual de la captura de trazas SIP en el SBC?"* * *"DetΓ©n la captura de trazas SIP en curso de inmediato."* --- ### Enterprise Safeguards & Access Governance 1. **Mandatory Auto-Off Timer**: Every trace capture initiated via MCP enforces a hard countdown timer (maximum 300 seconds). Kamailio's `siptrace` module is automatically disabled upon expiration to protect database disk space and CPU queues. 2. **Selective Filter Enforcement**: Tracing supports strict IP or Extension pre-filters to prevent indiscriminate capture of unrelated subscriber traffic. 3. **Role-Based Execution Isolation**: Tools that modify capture state (`start_sip_trace_capture`, `stop_sip_trace_capture`, `capture_and_analyze_sip_trace`) require operational roles (`noc_network_engineer` or `super_admin`). Read-only users can only view previously recorded ladders. --- ## 10. Glossary * **ACK (Acknowledgment)**: SIP transaction confirming receipt of a final response to an `INVITE` request. * **Call Sequence Ladder**: Chronological diagram displaying vertical participant lifelines and horizontal message vectors. * **Model Context Protocol (MCP)**: Open standard enabling AI models to interact with local development tools, logs, and diagnostic databases. * **SDP (Session Description Protocol)**: Syntax format (RFC 4566) describing multimedia communication parameters, codecs, and transport ports. * **siptrace**: Kamailio core module responsible for copying SIP transactions into relational database tables.