Skip to content

aarondutton-grc/ioc-threat-feed-aggregator

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

IOC Threat Feed Aggregator

A Python command-line tool that checks domains, IPs, and URLs against multiple public threat intelligence feeds and aggregates the results into a unified severity verdict. Designed for SOC triage, incident response support, and third-party vendor risk assessment workflows.


Threat Intelligence Sources

Feed Provider What It Detects IOC Types
URLhaus Abuse.ch Malware distribution hosts and URLs (active payloads, droppers, loaders) Domain, IP, URL
Feodo Tracker Abuse.ch Active C2 botnet servers (Emotet, QakBot, Dridex, TrickBot, BazarLoader) IP only
AlienVault OTX AT&T Cybersecurity Community threat intelligence pulses — campaign attribution, adversary tracking Domain, IP

URLhaus and Feodo Tracker are both operated by Abuse.ch, a Swiss non-profit threat intelligence platform.


Features

  • Multi-feed aggregation — queries two providers and three feeds in a single command
  • IOC normalization — automatically parses IP:port, full URLs, plain IPs, and domains before querying feeds
  • Severity tiering — CRITICAL / HIGH / MEDIUM / LOW / CLEAN verdict per feed and overall
  • Bulk mode — accepts a text file of IOCs and produces a summary table
  • SIEM-ready JSON output--json writes pure JSON to stdout; all diagnostic messages go to stderr
  • Report generation--report html produces a self-contained browser report; --report csv produces a flat file for Excel or log ingestion
  • Graceful degradation — feeds without API keys are skipped cleanly without crashing
  • Feed integrity warning — flags if the Feodo feed loads fewer than 50 entries (indicates a partial/stale response)

Supported IOC Formats

The tool normalizes all of the following before querying feeds:

84.247.138.41              # plain IP
31.57.184.154:443          # IP:port  →  strips port, checks IP
evil.com                   # domain
https://evil.com/payload   # full URL  →  extracts host for OTX/Feodo, sends full URL to URLhaus

Setup

1. Clone and install dependencies

git clone https://github.com/aarondutton-grc/ioc-threat-feed-aggregator.git
cd ioc-threat-feed-aggregator
pip install requests python-dotenv tabulate colorama

2. Get API keys (both free)

Key Where to get it
OTX_API_KEY otx.alienvault.com → Settings → API
URLHAUS_API_KEY auth.abuse.ch → API Access

3. Create a .env file

Create a .env file in the same directory as the script:

OTX_API_KEY=your_otx_key_here
URLHAUS_API_KEY=your_urlhaus_key_here

Keys are loaded automatically at runtime via python-dotenv. They are never passed on the command line.

⚠️ Add .env to your .gitignore before committing.


Usage

# Single IOC
python3 IOCThreatFeedAggregator.py -i 185.220.101.34

# Single IOC — JSON output (stdout only, stderr stays in terminal)
python3 IOCThreatFeedAggregator.py -i evil.com --json

# Pipe JSON to file
python3 IOCThreatFeedAggregator.py -i evil.com --json > result.json

# Bulk IOC file
python3 IOCThreatFeedAggregator.py -f indicators.txt

# Bulk with HTML report
python3 IOCThreatFeedAggregator.py -f indicators.txt --report html

# Bulk with CSV report
python3 IOCThreatFeedAggregator.py -f indicators.txt --report csv

# Adjust delay between requests in bulk mode (default: 1.0s)
python3 IOCThreatFeedAggregator.py -f indicators.txt --delay 2.0

Bulk IOC file format

One indicator per line. Lines starting with # are treated as comments.

# Known bad IPs
185.220.101.34
91.92.254.13

# Domains
evil.com

# IP:port (port is stripped automatically)
31.57.184.154:443

# Full URLs
https://malicious-site.ru/payload.exe

Output

Terminal (non-JSON mode)

============================================================
  Checking IOC: 89.185.81.112
============================================================

  ▶  OVERALL VERDICT: CRITICAL
  IOC       : 89.185.81.112
  Checked   : 2026-05-11T14:05:47.652137+00:00

╭──────────────────────────┬──────────┬───────┬──────────────────────────────────────────────────╮
│ Feed                     │ Severity │ Hit   │ Key Details                                      │
├──────────────────────────┼──────────┼───────┼──────────────────────────────────────────────────┤
│ URLhaus (Abuse.ch)       │ CLEAN    │ — miss│ note: Not found in URLhaus database              │
│ Feodo Tracker (Abuse.ch) │ CLEAN    │ — miss│ note: Not found in Feodo C2 blocklist            │
│ AlienVault OTX           │ CRITICAL │ ✓ HIT │ pulse_count: 50 | malware_families: SSH Brute... │
╰──────────────────────────┴──────────┴───────┴──────────────────────────────────────────────────╯

JSON output

[
  {
    "ioc": "89.185.81.112",
    "ioc_type": "ip",
    "ioc_value": "89.185.81.112",
    "timestamp": "2026-05-11T14:05:47.652137+00:00",
    "overall": "CRITICAL",
    "feed_results": [
      {
        "feed": "URLhaus (Abuse.ch)",
        "hit": false,
        "severity": "CLEAN",
        "details": { "note": "Not found in URLhaus database" }
      },
      {
        "feed": "Feodo Tracker (Abuse.ch)",
        "hit": false,
        "severity": "CLEAN",
        "details": { "note": "Not found in Feodo C2 blocklist" }
      },
      {
        "feed": "AlienVault OTX",
        "hit": true,
        "severity": "CRITICAL",
        "details": {
          "pulse_count": 50,
          "malware_families": ["SSH Brute-Force"],
          "country": "RU",
          "asn": "AS207967 anton mamaev"
        }
      }
    ]
  }
]

Report files

Format Flag Contents
HTML --report html Self-contained browser report with severity badges, per-feed detail rows, summary stats
CSV --report csv Flat file — one row per IOC per feed, suitable for Excel or SIEM ingestion

Reports are saved as ioc_report_YYYYMMDD_HHMMSS.html / .csv in the working directory.


Severity Tiering

OTX (pulse count)

Pulse Count Severity
0 CLEAN
1 LOW
2–4 MEDIUM
5–9 HIGH
10+ CRITICAL

URLhaus (active URL count)

Condition Severity
No results CLEAN
Offline URLs only HIGH
Any active (online) URLs CRITICAL

Feodo Tracker

Any match = CRITICAL (all Feodo entries are confirmed active or recently active C2 servers).

Overall verdict

The highest severity across all feeds that returned a real verdict. Feeds that errored, were skipped, or returned N/A do not influence the overall score.


A Note on OTX Pulse Count Context

A high OTX pulse count indicates the IP or domain has been reported across many community threat intelligence campaigns, but does not distinguish between:

  • Active C2 infrastructure (high confidence, immediate action warranted)
  • Tor exit nodes / VPN infrastructure (frequently flagged, context-dependent)
  • Honeypot hits (IP was observed scanning — threat actor behavior, not hosting malware)

Always review pulse_names_sample and malware_families in the detail output before escalating. An IP appearing only in honeypot SSH brute-force feeds warrants a different response than one appearing in named malware campaigns.


GRC / SOC Relevance

Capability Maps To
Multi-feed IOC triage SOC Tier 1 alert enrichment workflow
Severity tiering IR escalation thresholds and runbook decision gates
JSON stdout / stderr separation SIEM pipeline ingestion (Splunk, Elastic, Chronicle)
CSV/HTML report output Audit evidence, tabletop exercise artifacts
Vendor IP screening Third-party risk assessment — TPRM due diligence
Bulk file mode Threat hunting across a watchlist or IP blocklist

Skills Demonstrated

  • Python API integration (REST, POST, auth headers, error handling)
  • Threat intelligence feed consumption (Abuse.ch, AlienVault OTX)
  • IOC normalization across multiple indicator formats
  • Multi-source data aggregation with independent scoring
  • Secure credential management via python-dotenv (no keys in CLI or source)
  • stdout/stderr separation for pipeline-safe output
  • Report generation (self-contained HTML, flat CSV)
  • GRC-aligned output design (timestamped, auditable, severity-tiered)

Dependencies

requests
python-dotenv
tabulate
colorama

Install: pip install requests python-dotenv tabulate colorama


Author

Aaron Dutton
License: MIT

About

Python CLI tool that checks domains, IPs, and URLs against Abuse.ch and AlienVault OTX threat intelligence feeds

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages