Data onboarding
Paste raw log lines → detect the format → generate props.conf / transforms.conf / rex / CIM mapping → dry-run coverage on real data → apply to local/ with rollback. Since 1.7.0.
A new sourcetype means writing props.conf, extracting fields and mapping to CIM, the most time-consuming job a Splunk admin has. Data onboarding turns it into three steps whose outputs can each be read, edited and verified against real data. Licensed (same tier as detection rules); applying needs admin.
Step 1: detect
Paste 20–50 raw lines, or pick an existing index + sourcetype to sample ([onboard] sample_rows, default 50). Detection is rule-based, no model call:
| Output | Notes |
|---|---|
| Format | json / kv / csv / syslog / multiline |
| Timestamp | Candidate field and TIME_FORMAT (%b %d %H:%M:%S, %Y-%m-%dT%H:%M:%S, ISO without zone) |
| Line breaking | LINE_BREAKER / SHOULD_LINEMERGE |
| Field candidates | Direct for kv / json; regex candidates for syslog-style lines |
Step 2: generate
The model (agentic) writes the configuration from the detection and verifies with run_spl that | rex extracts the fields on the sample sourcetype:
| Output | Content |
|---|---|
props.conf | TIME_FORMAT / LINE_BREAKER / KV_MODE / EXTRACT-* / EVAL-* / FIELDALIAS-* |
transforms.conf | Only for csv or when a REPORT- is needed |
| rex list | Regex and target field for each EXTRACT- |
| CIM mapping | Recommended data model (Authentication / Network_Traffic / Web / Endpoint …) and field table; missing required fields are flagged |
The configuration blocks are editable. About 21–29 s for an sshd sample on the 1.9.0 test box.
Step 3: dry run
Runs the extraction over the last 1000 events (dryrun_rows, max 5000) of a given index / sourcetype and reports, per field, extraction rate, empty rate and a first sample; the CIM tab checks against the data model's field list (the real list from /datamodel/model when CIM is installed, a built-in minimum otherwise). Measured: 93.9% coverage over 15 fields for an sshd sample, 100% for pan:traffic.
Apply and roll back
"Apply" writes props.conf / transforms.conf into the target app's local/ (default search, [onboard] default_app) after backing up existing files as *.conf.bak-<timestamp>, then _reload; files created by this run carry a .created-by-copilot marker. "Roll back" touches only files with a backup or a marker and never deletes pre-existing customer configuration. "Download" hands you both files to place yourself.
Check the new fields in the Field dictionary afterwards; tstats sees index-time fields only, search-time extractions are verified with | stats count by <field>.
Baseline checks
Host-configuration baseline and OS end-of-life compliance patrol: a rule library, manual or daily runs, results and trends per host. Data from the CIM Endpoint / Change data models.
Security posture
The first look of a shift: four stat cards, an alert timeline, two Top 10s, the recent-alert queue. One aggregation search over the alert index.