Splunk is the market-leading platform for turning machine data into answers — and for a SOC analyst, it is the single tool most likely to be in front of you every shift. It ingests logs from any source, indexes them for fast search, and lets you interrogate them with the Search Processing Language (SPL). The catch is usually cost, which is why so many people never get hands-on practice.
The good news: Splunk Enterprise ships a fully functional 60-day trial that converts to a perpetual Free licence letting you index up to 500 MB per day forever. That is more than enough to learn on, build detections against, and run a home SOC lab. This guide installs Splunk three ways (Linux bare metal, Docker, and a dedicated Ubuntu VM), wires up a Universal Forwarder on both Linux and Windows, configures indexes and inputs, and finishes with the first SPL searches every analyst should know.
◈ Table of Contents
01 — WHAT SPLUNK IS & WHY IT MATTERS
BEGINNERSplunk is a data platform that collects, indexes, and searches time-series machine data — logs, metrics, and events from servers, network gear, endpoints, and applications. For security work it functions as a SIEM: you point every log source at it, then hunt across all of them at once with SPL. Instead of grepping ten boxes over SSH, you ask one question in one place and get answers across your entire estate in seconds.
Everything flows through three logical stages, whether they run on one box (as they do in this lab) or across a cluster:
In a lab, all three roles live in a single Splunk Enterprise instance — a "single-instance deployment". You only split them across servers when data volume outgrows one machine. Learn the single box first; the concepts transfer directly to a distributed deployment.
Four building blocks cover almost everything you will touch in this guide. Understanding which is which saves hours of confusion later — especially the difference between the full Enterprise instance you install once and the lightweight Universal Forwarder you deploy on every machine you want to collect from.
| Component | Role | Where It Runs |
|---|---|---|
| Splunk Enterprise | Indexes data, runs Splunk Web, executes searches | Your SIEM server |
| Universal Forwarder (UF) | Tiny agent that ships logs to the indexer | Every monitored host |
| Index | A named store of events on disk (e.g. main, wineventlog) | On the indexer |
| Sourcetype | Label describing a data format so Splunk parses it correctly | Assigned at input |
Do not install full Splunk Enterprise on the machines you want to monitor — that wastes resources and a licence slot. Endpoints get the free, lightweight Universal Forwarder. Only your central SIEM box runs Enterprise.
There is no separate "Splunk Free" download — you install Splunk Enterprise, which starts as a 60-day full-feature trial and can be switched to the perpetual Free licence at any point. Free caps indexing at 500 MB/day and drops a handful of enterprise features, but it keeps unlimited search, so it is ideal for learning and small labs.
| Feature | Enterprise Trial (60d) | Free (perpetual) |
|---|---|---|
| Daily index volume | No practical cap | 500 MB/day |
| SPL search | Full | Full |
| Users & roles / login | Yes (auth) | No auth — single admin, no login |
| Scheduled alerts | Yes | Not available |
| Distributed search / clustering | Yes | Not available |
| Receive from forwarders | Yes | Yes |
Threat Hunting
Pivot across firewall, endpoint and auth logs in one SPL query to chase an indicator.
Detection Engineering
Prototype and tune detection logic against real event data before promoting it to production.
Certification Prep
Hands-on practice for Splunk Core Power User and any SOC-analyst role that lists Splunk.
Home SOC Lab
500 MB/day comfortably covers a handful of lab hosts, an AD domain, and a firewall feed.
02 — PREREQUISITES & REQUIREMENTS
BEGINNERSplunk is memory- and disk-hungry once data flows in, but a learning instance is modest. The reference hardware below runs the indexer, Splunk Web, and a couple of forwarders comfortably. An SSD is strongly recommended — index and search performance are dominated by disk I/O.
Splunk's reference server spec is 12+ CPU cores and 12 GB RAM for production indexing, but that is for heavy load. A lab that indexes well under the Free 500 MB/day limit runs fine on the figures below. Give it more RAM if you plan to run Splunk plus other lab VMs on the same host.
| Target | vCPU | RAM | Disk | OS |
|---|---|---|---|---|
| Lab minimum | 2 | 4 GB | 40 GB SSD | Ubuntu 22.04 / 24.04 LTS |
| Comfortable lab | 4 | 8 GB | 100 GB SSD | Ubuntu Server / RHEL 9 |
| Splunk reference | 12+ | 12+ GB | Fast RAID | Production sizing only |
| Universal Forwarder | 1 | 512 MB | ~500 MB | Any monitored host |
Splunk is officially supported only on 64-bit systems and dislikes running as root long-term. Create a dedicated splunk service account during install (covered in Method 1) rather than running the daemon as root — it is both safer and Splunk's recommended practice.
Splunk uses a small, fixed set of TCP ports. You will open these in your host firewall and reference them constantly. The three that matter most: 8000 (the web UI you log into), 8089 (management/REST, and how forwarders are administered), and 9997 (the receiving port forwarders send data to).
| Port | Purpose | Who Connects |
|---|---|---|
| 8000/tcp | Splunk Web UI (HTTP/HTTPS) | Your browser |
| 8089/tcp | Management / REST API (splunkd) | CLI, deployment, apps |
| 9997/tcp | Receiving / indexing from forwarders | Universal Forwarders |
| 8088/tcp | HTTP Event Collector (HEC) | Apps / scripts (optional) |
| 8065/8191 | App server / KV store internal | Splunk internal |
9997 is not open by default — a fresh Splunk install will not accept forwarder data until you explicitly enable receiving (Section 08). Forgetting this is the number-one reason "my forwarder installed but no data shows up".
Everything here is a free download from Splunk once you create a free splunk.com account. Grab the Splunk Enterprise package for your server OS and the matching Universal Forwarder for the machines you will monitor. Versions in this guide target the current 10.x release train.
| Item | Where | Notes |
|---|---|---|
| Splunk Enterprise (.tgz) | splunk.com/download | Linux tarball — used in Method 1 & 3 |
| Splunk Enterprise (.msi) | splunk.com/download | Windows installer if your SIEM is Windows |
| splunk/splunk image | Docker Hub | Official container — used in Method 2 |
| Universal Forwarder | splunk.com/download | Separate small package, per-OS |
| Free splunk.com account | splunk.com | Required to download; no cost |
Copy the download link from the Splunk site after choosing your package — it lets you wget the exact build straight onto a headless server, and Splunk provides the SHA-512 hash on the same page so you can verify the file before extracting it.
03 — METHOD 1: LINUX BARE METAL (RECOMMENDED)
BEGINNERThe tarball install on Linux is the cleanest, most transparent way to run Splunk and the one most tutorials and job environments assume. Everything lives under a single directory — $SPLUNK_HOME, which is /opt/splunk — so there is nothing scattered across the filesystem and uninstalling is just deleting the folder.
Best practice is to run Splunk as a non-root user. Create a splunk account, extract the tarball into /opt (which creates /opt/splunk), and hand ownership of that directory to the service account before the first start.
Do not extract into /opt/splunk/splunk by accident. The tarball already contains the top-level splunk/ folder, so extracting into /opt (not /opt/splunk) yields the correct /opt/splunk path. A doubled path breaks every command that follows.
The first start accepts the licence and forces you to set the admin username and password. You can do it interactively, or pre-seed credentials with --seed-passwd for an unattended build. Run these as the splunk user so file ownership stays correct.
- splunkd starts and prints the running PID
- "All preliminary checks passed" during first boot
- "The Splunk web interface is at http://<host>:8000"
Make Splunk survive reboots by registering it as a systemd service. Modern Splunk generates a proper unit file and runs the daemon under your service account. Run the enable command as root, telling it which user to run Splunk as.
Raise the open-file and process ulimits Splunk asks for at startup (it warns if they are too low). On systemd, set LimitNOFILE=64000 and LimitNPROC=16000 in the generated unit — low ulimits cause intermittent, hard-to-diagnose indexing stalls under load.
04 — METHOD 2: DOCKER & DOCKER COMPOSE
INTERMEDIATEThe official splunk/splunk image is the fastest way to a working instance — one command and you have Splunk Web on port 8000. It uses Splunk's Ansible-based entrypoint, configured entirely through environment variables. This is ideal for throwaway labs and for rebuilding a clean instance in seconds.
Two environment variables are mandatory: SPLUNK_START_ARGS=--accept-license and SPLUNK_PASSWORD (the admin password, minimum 8 characters). Publish port 8000 for the UI and the container boots a full single-instance Splunk Enterprise in a minute or two.
Without a volume, everything you index and configure is lost when the container is removed. Fine for a five-minute test — but for anything you want to keep, use the Compose file below so /opt/splunk/etc and /opt/splunk/var persist on named volumes.
A Compose file gives you a reproducible, persistent Splunk you can tear down and rebuild without losing data. It names the ports, sets the admin password, and mounts two named volumes for the Splunk config (etc) and index data (var). Save this as docker-compose.yml and run docker compose up -d.
To pin a version for reproducible labs, replace :latest with an explicit tag such as splunk/splunk:10.0. Pinning avoids a surprise major-version jump the next time you docker compose pull, which can change default configs under you.
05 — METHOD 3: DEDICATED UBUNTU SERVER VM
INTERMEDIATEA dedicated VM is the sweet spot for a persistent home SOC: isolated, snapshot-able, and always-on without cluttering your workstation. Any hypervisor works — VirtualBox, VMware, Proxmox, or Hyper-V. Build a minimal Ubuntu Server guest, then run the same tarball install as Method 1 inside it.
Create a VM with the "comfortable lab" spec (4 vCPU / 8 GB / 100 GB), install Ubuntu Server 24.04 LTS with OpenSSH selected, then patch it and set a static IP so your forwarders always know where to send data. A static address is not optional — forwarders are configured to point at a fixed indexer IP.
Inside the VM, perform the identical tarball install from Method 1, then open the Splunk ports in ufw so your browser and forwarders can reach the service. Do this only on an isolated lab network — never expose 8000/8089 to the internet.
Take a hypervisor snapshot named "clean-splunk" immediately after the first successful login. If you later mangle an inputs.conf or fill the disk with test data, you can roll back in seconds instead of rebuilding.
06 — FIRST LOGIN & SWITCHING TO THE FREE LICENCE
COREHowever you installed, the experience from here is identical. Log into Splunk Web, take a quick tour of the settings you will actually use, then convert the 60-day trial to the perpetual Free licence so your lab keeps working past day 60.
Browse to http://<your-server-ip>:8000 and sign in with admin and the password you set at first start. The landing page is the Search & Reporting app. The gear-and-cog Settings menu (top-right) is where all administration lives.
- 1Open
http://10.10.10.20:8000in a browser. - 2Log in: admin / the password from
--seed-passwd. - 3You land on Apps → Search & Reporting.
- 4Note the top nav:
Settings(admin),Activity, and the app bar.
Forgot the admin password? Stop Splunk, delete /opt/splunk/etc/passwd, create /opt/splunk/etc/system/local/user-seed.conf with a [user_info] stanza containing USERNAME = admin and PASSWORD = <new>, then start Splunk again to re-seed it.
By default you are on the 60-day Enterprise trial. To keep the instance alive indefinitely, switch to the Free licence group. This is a deliberate, reversible choice in the licensing UI — you can switch back to a trial or a paid licence later.
Portal: Settings → Licensing → Change license group → Free license
The Free licence removes login/authentication — anyone who can reach port 8000 gets admin. That is fine on an isolated lab network, but never place a Free instance anywhere reachable from an untrusted network.
Splunk indexes its own logs into the _internal index from the moment it starts, which is perfect for a first search before you have added any real data. Open the Search app and confirm events are flowing. This proves the search pipeline works end-to-end.
- Non-zero counts for sourcetypes like splunkd, splunkd_ui_access
- Events updating if you set the time picker to real-time
- No "no results found" — that would mean search is broken
07 — CORE CONFIG: INDEXES & DATA INPUTS
COREBefore data is useful you need somewhere to put it (an index) and a way to get it in (an input). Good index hygiene — separating, say, Windows logs from firewall logs — makes searches faster and retention easier to manage. You can do all of this in the UI or in .conf files; both are shown.
Rather than dumping everything into main, create purpose-built indexes so you can control retention and scope searches. Here we add a Linux and a Windows index. The UI path is quickest, but the CLI is scriptable and shows exactly what happens.
Portal: Settings → Indexes → New Index
The simplest input is a file or directory monitor. On the Splunk server itself you can index its own /var/log to get data flowing immediately, tagging it with a sourcetype and routing it to your Linux index. This is a great smoke test before you deploy forwarders.
Portal: Settings → Data inputs → Files & Directories → New
Always set an explicit sourcetype on every input. If you let Splunk guess, similar logs land under slightly different auto-generated sourcetypes and your searches silently miss events. Consistent sourcetypes are the foundation of reliable detections.
After adding an input, confirm events are actually indexed before moving on. Search the target index directly. If nothing appears, the input path, permissions, or index name is wrong — catch it now, not three steps later.
If index=linux_logs returns nothing but the file has data, check that the splunk user can actually read the file (auth.log is often root:adm). No read permission = no events, and Splunk logs the access error into _internal.
08 — UNIVERSAL FORWARDER DEPLOYMENT
INTEGRATIONReal SIEM value comes from pulling logs off other machines. The Universal Forwarder (UF) is a tiny agent that reads local logs and streams them to your indexer on port 9997. First enable receiving on the indexer, then deploy the UF to a Linux and a Windows host.
The indexer will not accept forwarder traffic until you turn on a receiving port. Enable 9997 once, on the Splunk Enterprise server. This is the step people forget — the forwarder connects, the port is closed, and no data arrives.
Portal: Settings → Forwarding and receiving → Configure receiving → New Receiving Port
On the machine you want to monitor, extract the separate Universal Forwarder package into /opt/splunkforwarder, start it, point it at your indexer, and tell it which logs to ship. The UF is deliberately minimal — no web UI, tiny footprint.
On Windows the UF ships as an MSI that can be driven fully from the command line — ideal for scripted rollout. The install flags set the receiving indexer and enable the Windows event-log inputs (Security, System, Application) in one shot. Run from an elevated prompt.
For real detection value on Windows, install Sysmon with a good config (e.g. the SwiftOnSecurity or Olaf Hartong baseline) and add a [WinEventLog://Microsoft-Windows-Sysmon/Operational] input. Sysmon process, network, and image-load events are the backbone of most Windows detections.
Back on the indexer, verify the forwarders are actually connecting and sending. The Forwarder Management page lists phoning-home instances, and a quick search against the internal metrics confirms data volume per host.
- Each endpoint hostname appears in the host list
- tcpin_connections shows the forwarder source IPs
- index=wineventlog and index=linux_logs both grow over time
09 — YOUR FIRST SPL SEARCHES
ADVANCEDSPL (Search Processing Language) is where Splunk earns its keep. Every search starts by selecting events, then pipes them through commands with |, exactly like a Unix shell pipeline. Below are the patterns a SOC analyst uses daily, building from a raw search up to a saved dashboard panel.
A search names an index and optional keywords, then filters. Start narrow: pick an index, a sourcetype, and a time range in the picker. Splunk auto-extracts fields you can filter on with field=value. Keep the most selective terms first — it is faster.
Set the time range with the picker, not by scrolling. Searching "All time" on a busy index is slow and needless — 99% of hunting happens in the last 15 minutes to 24 hours. Tight time ranges are the single biggest search-speed win.
Raw events are noise; aggregation is signal. The stats command turns thousands of events into a table — counts, sums, distinct values — and is the workhorse of every useful search. timechart does the same but bucketed over time for trend lines.
A search you re-run is a search worth saving. On the Free licence you cannot schedule alerts, but you can build dashboards that run these searches on demand — turning your hunting queries into a reusable SOC view. Save any good search straight to a dashboard from the results page.
- 1Run a search you like (e.g. the brute-force
statsabove). - 2Click Save As → Dashboard Panel.
- 3Create a new dashboard "SOC — Auth Monitoring" and add the panel.
- 4Repeat for Windows 4625, add a
timechartas a line panel. - 5Open the dashboard any time for an at-a-glance auth view.
Scheduled searches and email alerts are Enterprise-only — they are disabled on the Free licence. For alerting on a budget, either stay on the 60-day trial while learning, or graduate to Wazuh/Elastic which alert for free. Splunk Free is superb for interactive hunting and dashboards.
10 — MAINTENANCE & TROUBLESHOOTING
OPSTwo things break most lab Splunks: hitting the 500 MB/day licence limit, and forwarder data that never arrives. Both are easy to diagnose from Splunk's own internal logs. Keep an eye on licence usage and you will avoid the violation warnings that eventually disable search.
The Free licence tolerates occasional overages, but repeated violations within a rolling 30-day window will lock search. Track your daily volume so you can throttle noisy inputs before that happens. These commands and searches are your early-warning system.
The Monitoring Console (Settings → Monitoring Console) gives you pre-built health dashboards for indexing rate, licence usage, and resource use — enable it once and you get most of the operational visibility of a paid deployment for free.
Work this table top to bottom before assuming anything exotic — nearly every "no data" problem is a closed port, a permissions issue, or a wrong index name. Splunk's own _internal index almost always contains the error that explains the failure.
| Symptom | Likely Cause | Fix |
|---|---|---|
| Forwarder installed, no data in Splunk | Receiving port 9997 not enabled | splunk enable listen 9997 on the indexer |
| Cannot reach http://host:8000 | Firewall blocking / Splunk not started | Open 8000 in ufw; splunk status; splunk start |
| "Failed to bind to 0.0.0.0:8000" | Port already in use | Change web port: splunk set web-port 8001 |
| Search disabled / licence violation | Exceeded 500 MB/day too often | Reduce noisy inputs; check license_usage.log |
| File has data but index empty | splunk user cannot read the file | chmod/chgrp so the splunk user can read it |
| Events show wrong timestamp | Sourcetype time parsing off | Set TIME_FORMAT/TZ in props.conf for that sourcetype |
| Windows UF service not running | Bad MSI flags / creds | sc query SplunkForwarder; reinstall with correct flags |
Use btool to see the effective, merged configuration Splunk is actually running: /opt/splunk/bin/splunk btool inputs list --debug prints every input stanza and the exact file it came from — invaluable when a config "should" work but does not.
You now have a working Splunk indexing real data from Linux and Windows, with searchable indexes and a dashboard. Build on it — more data sources make every future detection tutorial land against a live environment.
- Add Sysmon on Windows and hunt process-creation events (EventCode 1)
- Forward pfSense / OPNsense firewall logs and map east-west traffic
- Install the free Splunk Add-on for Microsoft Windows for field parsing
- Learn KQL side-by-side by mirroring these searches in Microsoft Sentinel
- Point an Active Directory lab at Splunk and detect Kerberoasting (4769)
11 — SOURCES & REFERENCES
DOCS| Resource | What It Covers |
|---|---|
| Splunk Docs — Install on Linux | Tarball install, boot-start, systemd management |
| Splunk Docs — About Splunk Free | Free licence features and the 500 MB/day limit |
| Splunk Docs — Docker (splunk/splunk) | Container deploy, env vars, persistence |
| Splunk Docs — Universal Forwarder | Install and configure UF on *nix and Windows |
| Splunk Docs — Enable receiving | Configure the indexer to accept forwarder data |
| Splunk Search Reference (SPL) | stats, timechart, rex and command syntax |
| Project | Purpose |
|---|---|
| Splunk Add-on for Microsoft Windows | Field extraction for Windows event logs |
| Sysmon + config baseline | Rich endpoint telemetry for Windows detections |
| Splunk Monitoring Console | Built-in health and licence-usage dashboards |
| Boss of the SOC (BOTS) datasets | Free sample security data to practise SPL |
| MITRE ATT&CK — Enterprise | Technique context for the detections you build |
The Splunk Free licence disables authentication — keep every Free instance on an isolated lab network, never exposed to the internet. Only collect logs from systems you own or are authorized to monitor. CyberHawk Threat Intel accepts no responsibility for misuse.
◈ Stay Connected
Follow CyberHawk Threat Intel for threat intelligence, penetration testing tutorials, and hands-on SOC tooling content.
"They can't exploit you if you are the Exploit."