--- title: "System & Resource Monitoring (SMR) Module Documentation" description: "Documentation for System & Resource Monitor" --- ## Table of Contents 1. [Module Overview (Technical)](#1-module-overview-technical) 2. [Module Overview (Commercial & Business Value)](#2-module-overview-commercial--business-value) 3. [Module Overview (End User & Administrator Experience)](#3-module-overview-end-user--administrator-experience) 4. [Real-Time Metrics & Telemetry Gauges](#4-real-time-metrics--telemetry-gauges) 5. [Kamcmd RPC Interactive Terminal](#5-kamcmd-rpc-interactive-terminal) 6. [Common Scenarios & Best Practices](#6-common-scenarios--best-practices) 7. [Troubleshooting & Verification](#7-troubleshooting--verification) 8. [Glossary](#8-glossary) --- ## 1. Module Overview (Technical) The **System Monitoring & Resources (SMR)** module in **Ring2All SBC** provides real-time visibility into the low-level health of Kamailio, RTPEngine, Linux kernel parameters, and hardware resources. It establishes high-frequency polling over the local binary RPC socket (`/var/run/kamailio/kamailio_ctl`) and RTPEngine control socket (`/var/run/rtpengine/rtpengine.sock`). ### Monitoring & Telemetry Pipeline ``` ┌────────────────────────────────────────────────────────────────────────┐ │ Ring2All SBC Core Daemons │ │ │ │ ┌───────────────────────┐ ┌─────────────────────────────┐ │ │ │ Kamailio │ │ RTPEngine │ │ │ │ • Shared Memory (SHM) │ │ • Kernel session table │ │ │ │ • Private Memory (PKG)│ │ • Media packet bitrate │ │ │ │ • Active Dialog count │ │ • Jitter & Packet loss │ │ │ └───────────┬───────────┘ └──────────────┬──────────────┘ │ └──────────────┼───────────────────────────────────────┼─────────────────┘ │ │ kamcmd binrpc socket NG control socket │ │ ┌──────────────▼───────────────────────────────────────▼─────────────────┐ │ Fastify Telemetry Collector (Server Settings / Monitoring Service) │ │ • Real-time WebSocket feed / 2-second polling │ │ • Threshold alert triggers (Memory > 85%, CPS Spike > 400) │ └───────────────────────────────────┬────────────────────────────────────┘ │ JSON Telemetry Feed ▼ Web UI: SMR Gauges & Administrative Terminal ``` --- ## 2. Module Overview (Commercial & Business Value) * **Guarantees Carrier-Grade High Availability:** Provides early warning of memory saturation, traffic spikes, or upstream carrier degradation before outages occur. * **Capacity Planning:** Accurate historical peak CPS and simultaneous call statistics enable informed hardware upgrades and bandwidth dimensioning. * **Instant Administrative Control:** Embedded secure web CLI executes Kamailio binary RPC commands safely without requiring direct SSH server access. --- ## 3. Module Overview (End User & Administrator Experience) * **Telemetry Dashboard:** Visual gauges for Calls Per Second (CPS), Active Concurrent Dialogs, CPU Load, and Shared Memory (SHM) consumption. * **Interactive Terminal (`kamcmd`):** Web-based console to run authorized Kamailio operational commands (e.g. `dispatcher.list`, `drouting.reload`, `htable.dump`, `core.ps`). * **Health Status Indicators:** Live status badge showing Kamailio daemon status, RTPEngine status, and PostgreSQL database latency. --- ## 4. Real-Time Metrics & Telemetry Gauges | Metric | Source | Warning Threshold | Critical Threshold | | :--- | :--- | :---: | :---: | | **Calls Per Second (CPS)** | `stats.get_statistics tm:` | > 300 CPS | > 500 CPS | | **Shared Memory (SHM)** | `core.shmmem` | > 80% used | > 90% used | | **Active Dialogs** | `dlg.list` / `dlg.profile_get_size` | > 80% license | > 95% capacity | | **RTPEngine Packet Drops** | `rtpengine.show` | > 1.5% loss | > 3.0% loss | --- ## 5. Kamcmd RPC Interactive Terminal The built-in web terminal allows executing verified diagnostic commands: ```bash # Check Kamailio process list and memory per worker kamcmd core.ps # View active dispatcher targets and latency kamcmd dispatcher.list # Invalidate LCR routing cache immediately kamcmd drouting.reload # Inspect rate-limited IPs in Pike table kamcmd htable.dump ipban ``` --- ## 6. Common Scenarios & Best Practices ### Scenario: Investigating a Traffic Spike 1. Open the **Monitoring (SMR)** view. 2. The **CPS Gauge** shows an abnormal jump from 15 to 280 CPS. 3. Check the **Pike Anti-Flood** table in the security section to verify whether an attacker IP is being quarantined. 4. Execute `kamcmd dlg.profile_get_size` in the terminal to inspect dialog allocation per trunk. --- ## 7. Troubleshooting & Verification | Symptom | Cause | Resolution | | :--- | :--- | :--- | | **Gauges showing "Unreachable"** | Kamailio ctl socket permissions or daemon not running. | Verify `/var/run/kamailio/kamailio_ctl` socket exists. Ensure backend user has rw access. | | **SHM Memory > 90%** | Insufficient shared memory allocated in `kamailio.cfg`. | Increase `-m ` parameter in Kamailio systemd unit file (e.g. from 64MB to 256MB). | --- ## 8. Glossary * **SHM (Shared Memory):** Fixed memory pool shared across all Kamailio worker processes for routing tables and dialog state. * **PKG (Private Memory):** Dedicated memory pool assigned to individual worker threads. * **Binrpc:** High-performance binary remote procedure call protocol used by `kamcmd`.