OPNsense Intrusion Detection: Detecting and Simulating Real-World Attacks

· 13 min read
homelab-monitoring

Introduction

Expose a service to the internet and someone will probe it.

OPNsense ships with Suricata, an open-source IDS/IPS that can:

  • Detect suspicious traffic (port scans, brute force, exploits)
  • Alert on threats
  • Block malicious traffic automatically (IPS mode)

This guide covers IDS setup, real-world attack patterns, and safe simulation for testing.

Legal Disclaimer: Only perform security testing on systems you own or have explicit written permission to test. Unauthorized security testing is illegal in most jurisdictions.

Prerequisites

  • OPNsense firewall (version 23.x or newer)
  • Basic networking knowledge
  • A test environment (homelab recommended)
  • Admin access to OPNsense

Part 1: Understanding IDS vs IPS

IDS (Intrusion Detection System)

  • Passive — observes traffic, never blocks
  • Alerts only — logs suspicious activity
  • No risk of blocking legitimate traffic
  • Best for: learning attack patterns, initial deployment

IPS (Intrusion Prevention System)

  • Active — drops malicious packets in real-time
  • Can false-positive — may block legitimate traffic
  • Requires tuning before production use
  • Best for: production environments after IDS tuning
💡

Recommendation: Start with IDS mode to understand your traffic patterns, then gradually enable IPS after tuning rules to reduce false positives.

Part 2: Enabling Suricata IDS in OPNsense

Step 1: Install and Enable Suricata

  1. Navigate to ServicesIntrusion DetectionAdministration
  2. Check Enabled
  3. Select IPS mode (leave unchecked for IDS-only initially)
  4. Choose interface(s) to monitor:
    • WAN - Incoming internet traffic
    • LAN - Internal traffic (optional)
  5. Set Pattern matcher to Hyperscan (fastest, requires compatible CPU)
  6. Click Apply

Step 2: Download Rulesets

OPNsense supports multiple threat intelligence rulesets.

  1. Go to ServicesIntrusion DetectionAdministrationDownload
  2. Enable rulesets:
RulesetDescriptionRecommended
ET open/emerging-*Emerging Threats Open RulesYes (free)
abuse.ch/SSL BlacklistKnown malicious SSL certificatesYes (free)
abuse.ch/URLhausMalicious URL detectionYes (free)
Snort VRTSnort’s official rulesOptional (requires registration)
  1. Click Download & Update Rules
  2. Wait for download to complete (can take several minutes)

Step 3: Configure Rule Categories

  1. Go to ServicesIntrusion DetectionAdministrationRules
  2. Enable relevant categories:

High Priority (Enable These):

  • emerging-attack_response - Detect successful attacks
  • emerging-exploit - Known exploit attempts
  • emerging-malware - Malware communication
  • emerging-scan - Port scanning activity
  • emerging-web_server - Web application attacks
  • emerging-dos - Denial of service attempts

Medium Priority:

  • emerging-policy - Policy violations
  • emerging-trojan - Trojan activity
  • emerging-worm - Worm propagation
  1. Click Apply after enabling categories

Step 4: Start the Service

  1. Go to ServicesIntrusion DetectionAdministration
  2. Click Apply to save settings
  3. Verify the service is running in ServicesIntrusion DetectionService

Part 3: Real-World Attack Patterns

Know what attacks look like so you can interpret alerts and tune rules.

Attack Type 1: Port Scanning

Attackers probe for open ports and running services.

Real-world example:

# Attacker runs nmap to discover services
nmap -sS -sV -p 1-65535 target.example.com

What OPNsense sees:

  • Rapid connection attempts across many ports
  • SYN packets without completed handshakes
  • Single source IP, short timeframe

Suricata alerts:

[**] ET SCAN Potential SSH Scan [**]
[**] ET SCAN Suspicious inbound to Oracle SQL port 1521 [**]
[**] ET SCAN Potential VNC Scan 5900-5920 [**]

Attack Type 2: SSH Brute Force

Automated credential guessing against SSH.

Real-world example:

# Attacker uses hydra to brute force SSH
hydra -l root -P passwords.txt ssh://target.example.com

What OPNsense sees:

  • Many failed SSH auth attempts
  • Same source IP, rapid connection rate
  • Common usernames (root, admin, ubuntu)

Suricata alerts:

[**] ET SCAN Potential SSH Scan OUTBOUND [**]
[**] ET POLICY SSH session in progress on non-standard port [**]

Attack Type 3: Web Application Attacks

SQL injection, XSS, path traversal, and command injection.

Real-world examples:

SQL Injection:

GET /login?user=admin'%20OR%201=1-- HTTP/1.1

Path Traversal:

GET /api/files?path=../../../../etc/passwd HTTP/1.1

Command Injection:

POST /api/ping HTTP/1.1
host=; cat /etc/passwd

Suricata alerts:

[**] ET WEB_SERVER SQL Injection Attempt [**]
[**] ET WEB_SERVER Possible Path Traversal Attempt [**]
[**] ET WEB_SERVER cmd.exe In URI - Possible Command Execution [**]

Attack Type 4: Malware Communication (C2)

Compromised hosts phoning home to attacker-controlled servers.

What OPNsense sees:

  • Outbound connections to known-bad IPs/domains
  • Unusual DNS queries (DGA-generated domains)
  • Beaconing patterns (regular-interval connections)

Suricata alerts:

[**] ET MALWARE Known Malicious User-Agent [**]
[**] ET CNC Feodo Tracker Reported CnC Server [**]
[**] ET POLICY DNS Query to DynDNS Domain [**]

Attack Type 5: Denial of Service (DoS)

Flooding services with traffic to make them unavailable.

Types:

  • SYN Flood: mass TCP SYN packets, no handshake completion
  • UDP Flood: high-volume UDP packets
  • HTTP Flood: many HTTP requests to exhaust the web server

Suricata alerts:

[**] ET DOS Possible SYN Flood [**]
[**] ET DOS UDP Amplification Attack Inbound [**]

Part 4: Simulating Attacks Safely

Critical Warning: Only perform these tests on systems you own, in isolated environments. Never test against production systems or third-party infrastructure without explicit permission.

Setting Up a Safe Test Environment

Recommended Architecture:

┌─────────────────────────────────────────────────────┐
│                  Isolated Test Network              │
│                   (10.10.10.0/24)                   │
│                                                     │
│  ┌─────────────┐         ┌─────────────┐          │
│  │   Attacker  │         │   Target    │          │
│  │    VM       │────────▶│   Server    │          │
│  │ (Kali/Parrot)│        │  (Ubuntu)   │          │
│  │ 10.10.10.100│         │ 10.10.10.50 │          │
│  └─────────────┘         └─────────────┘          │
│         │                       │                  │
│         └───────────┬───────────┘                  │
│                     │                               │
│              ┌──────▼──────┐                       │
│              │  OPNsense   │                       │
│              │  (IDS/IPS)  │                       │
│              │ 10.10.10.1  │                       │
│              └─────────────┘                       │
└─────────────────────────────────────────────────────┘

Test Environment Setup

Option 1: Proxmox/VMware Lab

  1. Create an isolated virtual network (no internet access)
  2. Deploy OPNsense VM as router/firewall
  3. Deploy target VM (Ubuntu Server with services)
  4. Deploy attacker VM (Kali Linux)

Option 2: Docker-based Lab

# Create isolated network
docker network create --subnet=172.20.0.0/16 ids-test-net

# Run vulnerable target (intentionally vulnerable)
docker run -d --name target \
  --network ids-test-net \
  --ip 172.20.0.10 \
  vulnerables/web-dvwa

# Run attacker tools
docker run -it --name attacker \
  --network ids-test-net \
  --ip 172.20.0.100 \
  kalilinux/kali-rolling

Simulation 1: Port Scan Detection

On Attacker Machine (Kali):

# Install nmap if not present
apt update && apt install -y nmap

# Basic TCP SYN scan
nmap -sS 10.10.10.50

# Service version detection
nmap -sV -p 22,80,443 10.10.10.50

# Aggressive scan (more detectable)
nmap -A -T4 10.10.10.50

Expected OPNsense Alerts:

  1. Go to ServicesIntrusion DetectionAlerts
  2. Look for:
    • ET SCAN alerts
    • GPL SCAN nmap alerts
    • Multiple source IPs if scanning from different hosts

Verify Detection:

# On OPNsense (via SSH)
grep -i "scan" /var/log/suricata/eve.json | tail -20

Simulation 2: SSH Brute Force Detection

On Target Machine (ensure SSH is enabled):

# Install OpenSSH if not present
apt install openssh-server
systemctl start sshd

On Attacker Machine:

# Install hydra
apt install -y hydra

# Create small password list for testing
cat > /tmp/passwords.txt << 'EOF'
password
123456
admin
root
toor
letmein
EOF

# Run controlled brute force (limited attempts)
hydra -l root -P /tmp/passwords.txt \
  -t 4 -w 5 \
  ssh://10.10.10.50

Note: Use a small password list and limited threads (-t 4) to avoid overwhelming your test environment. Real attacks use millions of passwords.

Expected OPNsense Alerts:

  • ET SCAN Potential SSH Scan
  • ET POLICY Possible SSH brute force attempt

Simulation 3: Web Application Attack Detection

On Target Machine (run a simple web server):

# Quick Python web server
mkdir -p /var/www/test
echo "<?php echo 'Hello'; ?>" > /var/www/test/index.php
cd /var/www/test && python3 -m http.server 80

On Attacker Machine:

# SQL Injection attempt
curl "http://10.10.10.50/login?user=admin'%20OR%201=1--"

# Path traversal attempt
curl "http://10.10.10.50/files?path=../../../../etc/passwd"

# XSS attempt
curl "http://10.10.10.50/search?q=<script>alert('xss')</script>"

# Using Nikto web scanner
apt install -y nikto
nikto -h http://10.10.10.50

Expected OPNsense Alerts:

  • ET WEB_SERVER SQL Injection Attempt
  • ET WEB_SERVER Possible Path Traversal
  • ET WEB_SPECIFIC_APPS various alerts

Simulation 4: Malicious Traffic Detection

Test against known-bad indicators (safe test domains):

# Test DNS query to known test domain
# EICAR is the standard antivirus test file
nslookup eicar.com

# Attempt to download EICAR test file
# This is a safe, non-malicious test file recognized by security tools
curl -O https://secure.eicar.org/eicar.com.txt

Expected OPNsense Alerts:

  • Test malware signature alerts
  • Possible ET MALWARE or ET POLICY alerts

Simulation 5: DoS Pattern Detection

Warning: DoS testing can crash systems. Only perform on isolated test networks with limited intensity.

On Attacker Machine:

# Install hping3
apt install -y hping3

# SYN flood simulation (LOW RATE - for testing only)
# -S: SYN flag, -p: port, -i: interval (u100 = 100 microseconds)
hping3 -S -p 80 -i u1000 --count 100 10.10.10.50

# UDP flood simulation (LOW RATE)
hping3 --udp -p 53 -i u1000 --count 100 10.10.10.50

Expected OPNsense Alerts:

  • ET DOS alerts
  • GPL DOS alerts
  • High packet rate warnings

Part 5: Analyzing and Responding to Alerts

Viewing Alerts in OPNsense

  1. Real-time Alerts: ServicesIntrusion DetectionAlerts
  2. Alert Details: timestamp, source/dest IP, alert message, SID, severity

Understanding Alert Severity

SeverityMeaningAction
1 (High)Active attack or compromiseImmediate investigation
2 (Medium)Suspicious activityInvestigate within hours
3 (Low)Informational or policy violationReview during maintenance

Alert Triage Workflow

┌─────────────────────────────────────────────────────┐
│                  New Alert                           │
└─────────────────────────────────────────────────────┘


┌─────────────────────────────────────────────────────┐
│ 1. Is source IP internal or external?               │
│    Internal → Possible compromised host              │
│    External → Attack attempt                         │
└─────────────────────────────────────────────────────┘


┌─────────────────────────────────────────────────────┐
│ 2. What service is targeted?                        │
│    Critical service → High priority                  │
│    Non-existent service → Low priority (noise)       │
└─────────────────────────────────────────────────────┘


┌─────────────────────────────────────────────────────┐
│ 3. Is this a known false positive?                  │
│    Yes → Add to suppress list                        │
│    No → Investigate further                          │
└─────────────────────────────────────────────────────┘


┌─────────────────────────────────────────────────────┐
│ 4. Was the attack successful?                       │
│    Check target logs, verify service integrity       │
│    Successful → Incident response                    │
│    Blocked/Failed → Monitor for persistence          │
└─────────────────────────────────────────────────────┘

Creating Custom Rules

Add rules for threats specific to your environment.

  1. Go to ServicesIntrusion DetectionAdministrationUser defined
  2. Add custom rule:
# Block specific malicious IP
drop ip 192.168.1.100 any -> any any (msg:"Blocked malicious IP"; sid:9000001; rev:1;)

# Alert on access to sensitive internal path
alert http any any -> $HOME_NET any (msg:"Access to admin panel"; content:"/admin"; http_uri; sid:9000002; rev:1;)

# Alert on specific user-agent (known bad bot)
alert http any any -> $HOME_NET any (msg:"Known bad crawler"; content:"BadBot"; http_user_agent; sid:9000003; rev:1;)

Suppressing False Positives

When legitimate traffic triggers alerts:

  1. Go to ServicesIntrusion DetectionAdministrationUser defined
  2. Add suppression rule:
# Suppress by signature ID
suppress gen_id 1, sig_id 2100498

# Suppress by source IP
suppress gen_id 1, sig_id 2100498, track by_src, ip 192.168.1.100

# Suppress by destination IP
suppress gen_id 1, sig_id 2100498, track by_dst, ip 10.0.0.50

Part 6: Enabling IPS (Active Blocking)

After tuning IDS and learning your traffic patterns:

Step 1: Review Alert History

  1. Confirm no critical false positives over the past 1-2 weeks
  2. Document rules that need suppression
  3. Verify legitimate services are not flagged

Step 2: Enable IPS Mode

  1. Go to ServicesIntrusion DetectionAdministration
  2. Check IPS mode
  3. Set Default action to Alert initially
  4. Click Apply

Step 3: Gradually Enable Blocking

  1. Go to Rules tab
  2. Set high-confidence categories to Drop:
    • emerging-exploit
    • emerging-malware
    • emerging-attack_response
  3. Keep lower-confidence rules on Alert:
    • emerging-scan
    • emerging-policy

Step 4: Monitor for Issues

After enabling IPS:

  • Watch for blocked legitimate traffic
  • Check application logs for connection failures
  • Review IPS drop logs regularly

Part 7: Integration with Monitoring

Sending Alerts to Syslog

  1. Go to ServicesIntrusion DetectionAdministration
  2. Enable Send alerts to system log
  3. Configure syslog destination in SystemSettingsLogging / targets

Integration with Grafana/Prometheus

If you have Grafana monitoring (see Monitoring Proxmox with Grafana Stack):

Export Suricata stats to Prometheus:

  1. Install suricata-exporter on OPNsense or a separate host
  2. Configure scraping in Prometheus
  3. Build a Grafana dashboard for IDS metrics

Key metrics to track:

  • Total alerts per hour
  • Alerts by severity
  • Top source IPs
  • Most triggered rules

Email Notifications

  1. Go to SystemSettingsNotifications
  2. Configure SMTP settings
  3. Go to ServicesIntrusion DetectionAdministration
  4. Enable email alerts for high-severity events

Part 8: Best Practices

Regular Maintenance

TaskFrequency
Update rulesetsWeekly (automated)
Review alertsDaily
Tune false positivesAs needed
Review blocked traffic (IPS)Daily
Full security auditMonthly

Performance Considerations

  • CPU: Suricata is CPU-heavy; watch utilization
  • Memory: allocate enough RAM for rule processing
  • Interfaces: only monitor what you need
  • Rules: disable rules for services you don’t run

What NOT to Do

  1. Don’t enable IPS right away — learn your traffic first
  2. Don’t ignore alerts — unreviewed alerts are useless
  3. Don’t block everything — false positives break things
  4. Don’t skip updates — threats change constantly
  5. Don’t test on production — use isolated environments

Conclusion

OPNsense with Suricata gives you enterprise-grade intrusion detection for a homelab or small business.

  1. Start with IDS to learn traffic patterns
  2. Use quality rulesets (Emerging Threats recommended)
  3. Test safely in isolated environments
  4. Tune rules to cut false positives
  5. Move to IPS once you trust your rules
  6. Monitor continuously — detection without response is pointless

IDS/IPS is one layer. Combine it with firewall rules, system hardening, patching, and security awareness.

References