Study and Optimization of Maltrail Rule in Wazuh

1. Introduction

Maltrail is an open-source network threat detection system focused on monitoring suspicious network traffic, such as connections to long domains, unusual IPs, or heuristic patterns from logs. Wazuh integrated Maltrail rules and decoders via Pull Request #7031, allowing alerts from Maltrail to be received directly in Wazuh.

2. Original Maltrail Rule

Source: Contributed by mimugmail to the Wazuh repository, based on logs from Maltrail.

Purpose: Generate alerts based on Maltrail events such as connection_attempt, long domains, heuristic trails.

Example alert:

Alert 1608899641.11816: - Maltrail,connection_attempt,
Rule: 64522 (level 7) -> 'Medium critical Maltrail event triggered'
Src IP: 172.24.68.133
Dst IP: 8.8.8.8
Trail: (sdbsxsrbdbtxsdbsxdxdsffsxds).test.de
Category: long domain (suspicious)
Severity: 1

Advantages:

• Enables alerts from Maltrail directly in Wazuh.

• Easy installation with included decoder and three alert rules for different severity levels.

Disadvantages:

• Alerts are generic: only shows "Medium critical Maltrail event triggered" without detailed threat type, source, or MITRE mapping.

• Difficult to distinguish false positives from long domains or benign DNS connections.

• Lacks granular alert levels, leading to repeated alerts for the same event.

3. Optimizing the Maltrail Rule

Our team improved the Maltrail rule to:

• Provide detailed classification: Added fields such as category, trail, ref, application to distinguish threat types (suspicious DNS, heuristic trail, unusual IP).

• Improve alert levels: Adjusted severity according to threat type, avoiding generic "Medium critical" alerts.

• MITRE ATT&CK mapping: Added mapping (e.g., T1071.004 - Application Layer Protocol: DNS) for threat intelligence reporting.

• Reduce false positives: Whitelist test domains or internal IPs to ensure only truly suspicious events are alerted.

Example of improved alert:

Wazuh/Wazuh-Rule-Custom/Mailtrail

Result: Alerts are clear and categorized, helping SOC teams respond faster while reducing false positives from benign domains.

Detected medium severity threat: long domain (suspicious) from 10.xxx to 10.xxx

4. Conclusion

The original Maltrail rule in Wazuh is an important starting point for integrating Maltrail but is not fully optimized. Our modifications allow SOC teams to:

• Receive detailed alerts with clear threat classification.

• Reduce redundant alerts and improve incident response efficiency.

• Integrate MITRE ATT&CK mapping for threat intelligence reporting.

This optimization serves as a model for customizing other Maltrail rules or integrating custom logs into Wazuh while maintaining scalability and ease of maintenance.