spotlight

This page explains how to use the spotlight function in APL to compare a selected set of events against a baseline and surface the most significant differences.

Spotlight lets you set up an analysis inside a query. You define a comparison set of events and compare it to the implicit baseline (the rest of the events in scope). Spotlight evaluates every field you pass in, scores differences, and returns the most informative contrasts. You use it when you want fast root-cause analysis, anomaly investigation, or pattern discovery without hand-rolling many ad-hoc aggregations.

Spotlight is useful when you:

• Investigate spikes or dips in a time series and want to know what changed
• Explain why a subset of traces is slow or error-prone
• Find which attributes distinguish suspicious requests from normal traffic

This page explains the Spotlight APL function. For more information about how Spotlight works in the Axiom Console, see Spotlight.

For users of other query languages

If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.

Splunk SPL users

In Splunk, there is no one operator that compares a selected cohort to the baseline across many fields at once. You often create a flag with eval, run separate stats/eventstats for each field, and then appendpipe or join to compare rates. In APL, spotlight is an aggregation you call once inside summarize. You pass a Boolean predicate to define the cohort and a list of fields to inspect, and APL returns a scored table of differences.

Splunk example

index=web earliest=-15m
| eval is_error = if(status IN ("500","502","503"), 1, 0)
| stats count by method uri status geo_country geo_city
| eventstats sum(eval(is_error)) AS sel, sum(eval(1-is_error)) AS base
| ... (manual rate/lift calculations and sorting) ...

ANSI SQL users

Standard SQL does not include a built-in cohort-vs-baseline comparator. You typically CASE a selection flag, aggregate twice (selected vs baseline), compute proportions, deltas, and significance, then union and sort. In APL, you express the selection as a predicate and let spotlight compute proportions, lift, and scores for each field/value.

SQL example

WITH scoped AS (
  SELECT *,
         CASE WHEN status IN ('500','502','503') THEN 1 ELSE 0 END AS is_sel
  FROM sample_http_logs
  WHERE _time >= NOW() - INTERVAL '15' MINUTE
),
per_field AS (
  SELECT 'status' AS field, status AS value,
         AVG(is_sel)              AS sel_rate,
         AVG(1 - is_sel)          AS base_rate
  FROM scoped
  GROUP BY status
)
SELECT field, value, sel_rate, base_rate,
       sel_rate / NULLIF(base_rate,0) AS lift
FROM per_field
ORDER BY lift DESC

Usage

Syntax

summarize spotlight(SelectionPredicate, Field1, Field2, ..., FieldN)

You use spotlight inside summarize. The first argument defines the comparison set. The remaining arguments list the fields to analyze.

Parameters

Name	Type	Description
SelectionPredicate	Boolean expression	Defines the comparison set (selected cohort). Spotlight compares events where the predicate evaluates to true against the baseline (events where it evaluates to false) within the current query scope.
Field1 ... FieldN	field references	One or more fields to analyze. Include string or categorical fields (for proportions) and numeric or timespan fields (for distributional differences).

Returns

• Bar charts for categorical fields (strings, Booleans)
• Boxplots for numeric fields (integers, floats, timespans) with many distinct values

Use case examples

Log analysis

Find what distinguishes error responses from normal traffic in the last 15 minutes.

Query

['sample-http-logs']
| where _time >= now(-15m)
| summarize spotlight(status startswith "5", ['geo.country'], ['geo.city'], method, uri, req_duration_ms)

Run in Playground

This query keeps the last 15 minutes of traffic in scope and compares error responses to everything else. Spotlight ranks the strongest differences, pointing to endpoints, regions, and latency ranges associated with the errors.

Best practices

• Keep the where scope broad enough that the baseline remains meaningful. Over-filtering reduces contrast.
• Pass only fields that carry signal. Very high-cardinality identifiers can drown out more actionable attributes.
• Include numeric fields like req_duration_ms or duration to let Spotlight detect distribution shifts, not just categorical skews.

List of related functions

• where: Filters events before Spotlight runs. Use it to scope the time window or dataset; use spotlight to compare selected vs baseline inside that scope.
• summarize: Runs aggregations over events. spotlight is an aggregation you call within summarize.
• top: Returns the most frequent values. Use top for simple frequency counts; use spotlight to contrast a cohort against its baseline with lift and significance.
• lookup: Enriches events with reference attributes. Use lookup to add context before running spotlight across enriched fields.