Elastic Security Pocket Book

Posted on by uplatzblog

Elastic Security Pocket Book — Uplatz

50 deep-dive flashcards • Wide layout • Fewer scrolls • 20+ Interview Q&A • Readable code examples

Section 1 — Fundamentals

1) What is Elastic Security?

Elastic Security is an open solution for SIEM (Security Information and Event Management) and endpoint security, built on the Elastic Stack (Elasticsearch, Logstash, Kibana, Beats). It enables threat detection, hunting, monitoring, and incident response.

# Elastic Security runs inside Kibana
# Requires Elasticsearch backend

2) Why Elastic Security?

Strengths: scalable search, flexible ingestion, powerful detection engine, free & open tier. Tradeoffs: operational overhead, tuning required for detection rules, scaling Elasticsearch clusters is non-trivial.

# Enable Elastic Security app from Kibana
stack_features:
  security_solution: true

3) Elastic Security Architecture

Data Sources → Ingestion (Beats, Logstash) → Elasticsearch → Detection Engine & Machine Learning → Kibana Security App (Dashboards, Alerts, Cases).

# Filebeat ships logs
filebeat modules enable system
filebeat -e

4) Core Components

1) Data Ingestion (Beats, Logstash, Elastic Agent). 2) Data Store (Elasticsearch). 3) Analytics & Detection (rules, ML jobs). 4) Visualisation (Kibana). 5) Response (cases, connectors).

# Example rule (KQL)
event.category: process and event.type: start

5) Elastic vs Traditional SIEM

Elastic: built on Elasticsearch, near real-time search, open APIs, integrated ML. Traditional SIEM: vendor-locked, slower ingestion, costly licenses. Elastic suits cloud-native and hybrid workloads.

# SIEM Query (KQL)
source.ip: "10.0.0.1"

6) Elastic Common Schema (ECS)

Defines a unified schema for log fields (e.g., event.category, host.ip). ECS standardises data to enable correlation across sources.

event.action: "login"
user.name: "alice"

7) Detection Engine

Runs rules continuously to detect suspicious behaviour. Rule types: KQL, EQL (event correlation), ML jobs, custom scripts. Alerts can trigger cases, webhooks, or SOAR workflows.

# Example EQL
process where process.name == "mimikatz.exe"

8) Timeline & Investigations

Timeline is the interactive investigation UI: query events, drag & drop observables, visualise sequences, and attach findings to cases.

# Timeline queries use KQL/EQL
event.category: network and destination.port: 443

9) Case Management

Analysts can open cases directly from alerts or investigations, assign them to teams, and push to external systems (Jira, ServiceNow, Slack).

# Create case in Kibana UI → Security → Cases

10) Q&A — “Elastic vs Splunk for Security?”

Answer: Elastic offers free/open, schema flexibility, and integrated ML. Splunk provides mature ecosystem and support but is costlier. Elastic is ideal for cloud-native, Splunk for enterprises with budget and legacy integration needs.

Section 2 — Data Ingestion & Normalisation

11) Beats

Lightweight shippers: Filebeat (logs), Winlogbeat (Windows events), Packetbeat (network), Auditbeat (audit logs), Metricbeat (metrics). Send directly to Elasticsearch or Logstash.

filebeat modules enable auditd
filebeat setup

12) Logstash

Pipeline for ingest → transform → ship. Parse, enrich, normalise logs to ECS before indexing. Use Grok filters, dissect, or JSON codec.

filter {
  grok { match => { "message" => "%{COMBINEDAPACHELOG}" } }
}

13) Elastic Agent & Fleet

Unified agent to collect logs, metrics, and endpoint security data. Managed via Fleet in Kibana. Simplifies deployment compared to multiple Beats.

sudo elastic-agent install --url=https://fleet-server:8220 --enrollment-token <token>

14) Integrations

Elastic has 100+ prebuilt integrations (AWS, Azure, GCP, Okta, Cisco, CrowdStrike). Normalise incoming logs into ECS automatically.

# Enable AWS CloudTrail integration from Kibana Integrations

15) Threat Intelligence Feeds

Ingest TI feeds (MISP, OpenCTI, AlienVault OTX) into Elasticsearch. Correlate indicators (IP, domain, hash) with logs to enrich detections.

indicator.type: "ip" and source.ip: 185.*

16) ECS Normalisation

Always map fields to ECS before detection. This ensures correlation across sources. Example: map src_ip to source.ip.

mutate { rename => { "src_ip" => "[source][ip]" } }

17) Parsing Unstructured Logs

Use Grok or regex patterns to extract fields. Map to ECS. Avoid leaving logs as raw strings.

%{IP:source.ip} %{WORD:http.method} %{URIPATHPARAM:http.url}

18) Enrichment

Enrich logs with GeoIP, ASN, threat intel, or user context. Store enrichments in ECS fields (e.g., geo.*, threat.*).

geoip { source => "source.ip" target => "source.geo" }

19) Pipelines

Ingest pipelines in Elasticsearch perform transforms before indexing: rename, drop, enrich. Useful for lightweight parsing without Logstash.

PUT _ingest/pipeline/geoip
{ "processors":[{ "geoip":{ "field":"ip" } }] }

20) Q&A — “When use Beats vs Elastic Agent?”

Answer: Beats are lightweight and modular but require separate installs. Elastic Agent unifies data + endpoint protection, easier at scale. Use Agent for new deployments, Beats for legacy/simple needs.

Section 3 — Detection & Response

21) Rule Types

KQL queries, EQL sequences, ML anomaly jobs, and threshold rules. Combine with actions (case, Slack, webhook).

event.category: authentication and event.outcome: failure

22) Prebuilt Rules

Elastic ships 700+ prebuilt rules (MITRE ATT&CK mapped). Enable, tune thresholds, add exceptions. Update via Detection Rule Repository.

# Kibana Security → Detections → Prebuilt Rules

23) Machine Learning Jobs

Unsupervised ML jobs detect anomalies (rare processes, unusual logins). Train on baseline and flag deviations. Requires Elastic Platinum license.

# Example ML detector
"detector_description": "rare process by user"

24) Alerting & Connectors

Alerts route to cases, Slack, Jira, ServiceNow, PagerDuty. Connectors define integration endpoints. Manage via Kibana → Stack Management → Connectors.

# Example webhook payload for alert
{ "alert":"High CPU", "host":"srv1" }

25) Endpoint Security

Elastic Agent provides EDR: malware prevention, behaviour monitoring, ransomware protection. Events flow into Security app for analysis.

elastic-agent install --endpoint-security

26) Threat Hunting

Use Timeline to hunt across large datasets. Pivot on IPs, hashes, users. Combine EQL sequences to spot attack chains.

sequence by user.name
  [authentication where event.outcome == "failure"]
  [process where process.name == "cmd.exe"]

27) SOAR Workflows

Automate response with cases + connectors. Example: auto-open Jira ticket on detection, auto-isolate endpoint with Elastic Agent.

action: isolate endpoint on detection rule trigger

28) MITRE ATT&CK Integration

Each rule maps to ATT&CK tactics/techniques. Provides common language for detection coverage. Use dashboards to assess gaps.

threat.tactic.name: "Credential Access"

29) Case Workflow

Each case tracks investigation, evidence, notes. Push/pull to Jira/ServiceNow. Analysts collaborate and resolve with audit trail.

# Kibana Security → Cases → New Case

30) Q&A — “Elastic Agent vs EDR vendors?”

Answer: Elastic Agent provides EDR integrated with SIEM, reducing vendor sprawl. Dedicated EDR vendors (CrowdStrike, SentinelOne) may offer richer telemetry but cost more.

Section 4 — Operations & Scaling

31) Cluster Sizing

Elastic Security relies on Elasticsearch cluster health. Size for daily ingest (GB/day), retention, query load. Scale horizontally; use hot-warm-cold tiers.

# Example sizing: 3 master, N data nodes, 1+ ingest nodes

32) Retention & ILM

Use Index Lifecycle Management (ILM) to roll indices, move to warm/cold, delete after N days. Tune retention vs cost vs compliance.

PUT _ilm/policy/security-logs
{ "phases": { "hot": {...}, "delete": {"min_age":"30d","actions":{"delete":{}}}}}

33) Scaling Ingestion

Use Kafka or Logstash pipelines for buffering. Partition large sources. Use Elastic Agent Fleet scaling for thousands of endpoints.

output.kafka:
  topic: "logs"

34) Multi-Tenancy

Use Spaces in Kibana to segment teams. Apply role-based access control (RBAC) with field- and index-level security.

role:
  indices: ["logs-*"]
  privileges: ["read"]

35) Elastic Security on Cloud

Elastic Cloud provides managed clusters. Benefits: auto-upgrades, scaling, snapshot backups. Faster to deploy vs self-managed.

# Deploy from cloud.elastic.co

36) Monitoring & Health

Use Stack Monitoring in Kibana. Watch heap usage, queue sizes, shard balance. Alert on unhealthy cluster states.

GET _cluster/health

37) High Availability

Deploy across AZs. Use snapshot/restore for DR. Replication ensures no data loss when nodes fail. Test failovers regularly.

index.number_of_replicas: 1

38) Securing the Stack

Enable TLS, role-based auth, audit logging. Rotate API keys. Restrict public exposure. Harden Fleet servers.

xpack.security.enabled: true
xpack.security.audit.enabled: true

39) Observability Tie-in

Integrate with Elastic Observability (APM, Metrics, Logs). Unified platform for performance + security telemetry. Useful for DevSecOps teams.

# Correlate traces with security logs by trace.id

40) Q&A — “How to reduce storage costs?”

Answer: Shorter retention, ILM cold tiers, searchable snapshots, store raw in S3 and keep hot indices only for active 30 days.

Section 5 — Security, Testing, Interview Q&A

41) Security Hardening

Apply TLS everywhere, RBAC, audit logs. Disable anonymous access. Use encrypted secrets. Regularly update Elastic stack.

elasticsearch.username: "elastic"
elasticsearch.password: ${ES_PASS}

42) Compliance

Elastic Security can support SOC2, ISO, GDPR logging requirements. Map ECS fields to compliance frameworks and retain data accordingly.

# Audit log example ECS
event.module: "auditd" event.type: "user"

43) Testing Detection Rules

Simulate attacks with Atomic Red Team or custom scripts. Validate that rules fire and alerts generate cases. Iterate on false positives.

Invoke-AtomicTest T1059 -ShowDetails -Confirm

44) Blue/Red Team Use

Blue teams monitor with Elastic Security. Red teams simulate adversary TTPs; results tune rules. Purple teaming aligns both.

# Rule triggered → create timeline → share with red team

45) Incident Response

Elastic Security integrates case management with endpoint isolation, TI enrichment, and ticketing. Automates triage and containment.

action: isolate_endpoint

46) Continuous Tuning

Regularly review rules, adjust thresholds, whitelist known benign events. Monitor detection coverage vs MITRE ATT&CK.

# Update detection rule exceptions list

47) Reducing False Positives

Contextualise alerts with threat intel, enrichments. Tune thresholds and exception lists. Run detection gap analysis.

exception_list: [ host.name: "dev-box" ]

48) Production Checklist

  • ECS mapping for all sources
  • Detection rules tuned & MITRE mapped
  • ILM retention policies applied
  • Cluster HA tested
  • RBAC & TLS enabled
  • Incident runbooks documented

49) Common Pitfalls

Skipping ECS mapping, leaving default passwords, no retention policies, ingest overload, ignoring cluster health, rule sprawl with no tuning.

50) Interview Q&A — 20 Questions

1) What is Elastic Security and its core components? (SIEM + EDR on Elastic Stack).

2) Difference between KQL and EQL? (KQL = search/filter, EQL = sequence correlation).

3) What is ECS? (Elastic Common Schema, unified field naming).

4) Beats vs Elastic Agent?

5) Role of Logstash?

6) How are detections automated? (rules engine + ML jobs).

7) How to enrich logs with threat intel?

8) How to scale Elastic cluster?

9) Explain ILM.

10) Endpoint protection capabilities?

11) What is Timeline?

12) Case management integration?

13) Threat hunting workflows?

14) Elastic vs Splunk vs QRadar?

15) MITRE ATT&CK mapping significance?

16) How to test detection rules?

17) Handling false positives?

18) Securing Elastic cluster?

19) Observability vs Security integration?

20) Future roadmap of Elastic Security?