Rakshak Autonomous Security Assessment Benchmark

Executive Summary

Rakshak is an autonomous AI-powered cybersecurity system engineered by Rudraksh AGI, driven by the specialized Rakshak. Unlike conventional static vulnerability scanners or naive LLM wrappers that emit unverified text hallucinations, Rakshak orchestrates autonomous security assessments through dynamic plan generation, tool execution sandboxing, step-by-step ReAct reasoning, and automated adaptive replanning.

To evaluate the autonomous capabilities of Rakshak under real-world security testing conditions, an official benchmark was executed on September 09, 2026, targeting this containerized OWASP Juice Shop deployment.

Benchmark Overview

The primary objective of this benchmark was to rigorously evaluate how autonomously Rakshak operates when presented with a single high-level objective: "Execute a comprehensive security assessment of http://localhost:3000".

The benchmark evaluated the complete attack lifecycle:

• Reconnaissance & Asset Discovery: Web technology fingerprinting and port scanning.
• Application Mapping & Filtering: Header interrogation and Web Application Firewall (WAF) detection.
• Vulnerability Analysis & Fuzzing: Automated web vulnerability scanning, directory brute-forcing, and SQL injection testing.
• Dynamic Adaptation: Identifying failed execution states and autonomously replanning subsequent actions without human intervention.
• Multi-Agent Post-Processing: Activating specialized downstream agents (ReconAgent, ClassificationAgent, ResearchAgent, ReportingAgent).

Test Environment

The benchmark was conducted in a controlled, fully reproducible local environment. The target application and supporting infrastructure services were isolated as documented below:

Assessment Workflow: From Planning to Autonomous Adaptation

Upon receiving the assessment prompt, Rakshak's PlanGenerator constructed an initial 8-step deterministic assessment plan. As execution unfolded, the system dynamically reacted to environment feedback, triggering an autonomous replan.

Original Initial Plan (8 Steps):
  [Step 1] whatweb      -> Fingerprint web technologies & frameworks
  [Step 2] nmap         -> Port scan with default service version detection
  [Step 3] httpx        -> HTTP probe for headers, response codes, & CORS
  [Step 4] wafw00f      -> Web Application Firewall (WAF) detection
  [Step 5] nikto        -> Web server vulnerability scanner
  [Step 6] sqlmap       -> SQL injection detection & crawler
  [Step 7] gobuster     -> Directory and endpoint brute-forcing
  [Step 8] analyze      -> LLM-powered findings synthesis & final reporting

Dynamically Replanned Workflow (Expanded to 9 Steps after Step 5 failure):
  [Step 1] whatweb      -> SUCCESS (6,245ms) - OWASP Juice Shop identified
  [Step 2] nmap         -> SUCCESS (14,516ms) - Werkzeug on 8888, port 3000 open
  [Step 3] httpx        -> SUCCESS (2,396ms) - CORS wildcard header detected
  [Step 4] wafw00f      -> SUCCESS (1,478ms) - No WAF detected
  [Step 5] nikto        -> FAILED (2,913ms) - Target port 80 closed -> TRIGGERED REPLAN #1
  [Step 6] sqlmap       -> SUCCESS (4,883ms) - 3 SQL injection findings detected
  [Step 7] gobuster     -> SUCCESS (243ms) - Wildcard HTTP 200 response handled
  [Step 8] nmap (added) -> SUCCESS (1,791ms) - Targeted OS detection on port 3000
  [Step 9] analyze      -> SKIPPED - OpenCode connector timed out (600s)

Autonomous Reasoning & ReAct Performance

A core pillar of Rakshak is its step-by-step ReAct (Reason + Act) loop. After each tool finishes executing, the reasoning engine parses standard output, error codes, and operational context to evaluate whether the overarching plan remains valid or if dynamic remediation is required.

During the benchmark, Rakshak completed 7 evaluation cycles, achieving a flawless 7/7 (100%) ReAct reasoning accuracy:

Dynamic Decision-Making on Tool Failure: Autonomous Self-Healing & Adaptive Replanning

In conventional cybersecurity tooling, automated scanners follow rigid, static execution scripts. If a tool crashes, a targeted port is unreachable, or an application returns an unconventional status code, standard automation either terminates prematurely, declares the target dead, or dumps thousands of misleading false-positive reports.

Rakshak's Rakshak operates on a fundamentally different paradigm. It treats tool errors, network connection refusals, and anomalous HTTP responses as live environmental telemetry. Operating through a closed-loop ReAct (Reason + Act) supervisory state machine, the agent intercepts stdout and stderr streams in real-time, cross-references findings from earlier reconnaissance, and dynamically replans its execution path without requiring human intervention.

┌────────────────────────────────────────────────────────────────────────────────────────┐
│                   AUTONOMOUS ADAPTIVE REPLANNING & SELF-HEALING PIPELINE               │
└────────────────────────────────────────────────────────────────────────────────────────┘
                                      │
                         [Tool Dispatched: nikto on port 80]
                                      │
                         [Execution Failure: Connection Refused]
                                      │
                                      ▼
             ┌──────────────────────────────────────────────────┐
             │       ReAct Supervisory State Machine Intercept  │
             │       - Capture Stderr / Return Code (2,913ms)   │
             │       - Evaluate Environmental Feedback          │
             └──────────────────────────────────────────────────┘
                                      │
                                      ▼
             ┌──────────────────────────────────────────────────┐
             │       Context Cross-Correlation (Session Memory) │
             │       - Step 2 Nmap: Port 3000 OPEN (Juice Shop) │
             │       - Step 2 Nmap: Port 8888 OPEN (Flask Core) │
             │       - Step 5 Nikto: Port 80 CLOSED (Wrong Port)│
             └──────────────────────────────────────────────────┘
                                      │
                                      ▼
             ┌──────────────────────────────────────────────────┐
             │     Autonomous Tactical Decision (35,742ms LLM)  │
             │  "Target is not down. Port 80 is closed, but     │
             │   port 3000 is active. TRIGGER REPLAN #1."       │
             └──────────────────────────────────────────────────┘
                                      │
                                      ▼
             ┌──────────────────────────────────────────────────┐
             │       Dynamic Plan Expansion (PlanGenerator)     │
             │       - Original Plan: 8 Steps                   │
             │       - Replanned Plan: 9 Steps                  │
             │       - Injected Step 8: Targeted nmap on :3000  │
             └──────────────────────────────────────────────────┘
                                      │
                                      ▼
                         [Continuous Assessment Resumes]
                         [Zero Human Prompting Needed]

Architectural Paradigm: Legacy Scanners vs. Rakshak Rakshak

The contrast between conventional linear penetration testing scripts and Rakshak's autonomous reasoning engine is summarized below:

Deep Dive: The Four Dynamic Autonomous Decisions in the Benchmark

Post-Workflow Autonomous Multi-Agent Swarm

Following the execution of primary security tools, Rakshak automatically spawned specialized downstream agents to handle enrichment, classification, and CVE research:

LLM Call Performance & Latency Matrix

Rakshak executed a total of 11 LLM reasoning and orchestration calls during the benchmark session via the OpenCode MCP Bridge:

Tool Execution Performance

Rakshak executed a total of 10 discrete tool operations (7 initial plan tools, 1 replanned tool, and 2 auxiliary post-workflow checks). Total tool execution time was remarkably rapid at ~34.8 seconds, yielding a 90% success rate (9/10):

Verified Security Findings

The benchmark surfaced multiple confirmed security vulnerabilities and operational posture findings. The findings are classified below with strict adherence to verified evidence:

SQL Injection Evidence & Validation Deep Dive

SQL injection detection by autonomous agents frequently suffers from high false-positive rates when agents misinterpret generic HTTP 500 error pages. In Rakshak's run, sqlmap flagged 3 SQL injection vectors, which were subsequently validated against the live SQLite backend.

# Test 1: Single quote injection triggering SQLite syntax error
$ curl -s "http://localhost:3000/rest/products/search?q=test'"
Error: SQLITE_ERROR: near "'%'": syntax error

# Test 2: SQL comment injection triggering incomplete query state
$ curl -s "http://localhost:3000/rest/products/search?q=test'--"
Error: SQLITE_ERROR: incomplete input

# Technical Finding Summary:
# Target Endpoint : GET /rest/products/search?q=
# Vulnerable Param : q
# Database Engine : SQLite 3 (confirmed via verbose error responses)
# Severity        : High / Critical CVSS 8.6
# Security Impact : Complete database exfiltration, credential dumping, authentication bypass

Additional Evidence: Data Exposure & Credential Leakage

Beyond database injection, the benchmark captured critical API privacy and authorization flaws documented in the benchmark evidence artifacts:

Final Benchmark Score: 8.05 / 10

The performance of Rakshak was graded across seven core engineering and security evaluation categories, each weighted according to its importance in an autonomous offensive pipeline:

Coverage Gaps & Limitations

In keeping with our commitment to transparency, we openly document every execution bottleneck, skipped tool, and infrastructure failure encountered during this evaluation run:

Total Execution Time Breakdown

The complete benchmark session required 21 minutes and 29 seconds. The distribution across infrastructure, tooling, reasoning, and connection timeouts is summarized below:

Critical Engineering Insight: 45.5% of the total session runtime was spent waiting on an LLM connector timeout at step 8. The security testing and reasoning engines themselves operated with exceptional speed.

Engineering Lessons Learned

This live benchmark provided concrete insights into the realities of orchestrating autonomous agents against real-world security tooling:

1. LLM Connector Resilience Must Be Multi-Tiered: Relying on a single local MCP LLM bridge without aggressive sub-minute timeouts and instant fallback providers (e.g., Anthropic Claude or Google Gemini APIs) introduces single points of failure.
2. Context Token Budgeting in Post-Workflow Agents: The ClassificationAgent exceeded its token budget (10,543 / 8,192 tokens), forcing the LLM refine pass to be skipped. Structured summary chunking must be implemented before LLM ingestion.
3. Target URL & Port Parameter Propagation: The initial PlanGenerator passed localhost without specifying target port 3000 to Nikto, triggering an unnecessary failure. Tool command synthesizers must strictly inherit port metadata from reconnaissance findings.
4. ReAct Reasoning Outperforms Static Scripts: The 100% ReAct decision accuracy proved that dynamic agent self-evaluation is vastly superior to hardcoded sequential testing scripts when handling unexpected environmental states.

Post-Benchmark Improvements & Action Plan

Based on the empirical evidence gathered during this benchmark, Rudraksh AGI has established the following prioritization matrix for Rakshak engine enhancements:

Conclusion

The Rakshak benchmark represents a milestone in validating autonomous, evidence-grounded cybersecurity testing. With an overall composite score of 8.05 / 10, the Rakshak proved that autonomous agents can reliably execute complex security tools, accurately interpret findings without hallucinating, and intelligently adapt their strategies when tools fail.

Crucially, the benchmark demonstrated that the core security logic, reasoning loop, and tool execution layer are fundamentally sound—achieving 100% ReAct reasoning accuracy and 90% tool execution reliability. The identified limitations—principally LLM provider timeouts and token budget ceilings—are architectural engineering challenges that have already been integrated into our active development sprint.

Frequently Asked Questions (FAQ) & Answer Hub