Scheduled Job Creation via At Utility Inside a Container
Description
This rule detects execution of the "at" or "atd" job-scheduling utilities inside a Linux container. Containers are typically immutable, single-purpose, and do not run interactive job schedulers, so the use of "at" to queue a one-time command is anomalous. Adversaries abuse "at" to establish persistence or to execute a payload at a later time after gaining a foothold inside a container.
Query · esql
from logs-cloud_defend.process* metadata _id, _index, _version
| where host.os.type == "linux" and event.type == "start" and event.action == "exec" and
container.id like "*" and
(
process.name in ("at", "atd") or
kql("""process.args: "at" """)
)
| keep @timestamp, host.id, container.id, process.name, process.args, process.parent.name, process.parent.executable, user.id, _id, _index, _version
Known false positives
- Some container images bundle the "at"/"atd" utilities and a small number of workloads may legitimately schedule one-time jobs. Investigate the queued command and the scheduling principal; baseline and except known automation images where appropriate.
Analyst notes
This rule requires the Elastic Defend for Containers integration to be configured to collect container process events.
Triage and analysis
Investigating Scheduled Job Creation via At Utility Inside a Container
The "at" utility schedules a command to run once at a later time via the "atd" daemon. Inside a container this is rarely legitimate: container workloads are usually immutable and process-bound, with scheduling handled by the orchestrator (CronJobs), not an in-container scheduler. Adversaries use "at" to persist or to defer execution of a payload after compromising a container.
Possible investigation steps
- Identify the container (
container.id,container.name) and namespace (orchestrator.namespace) where "at" executed, and confirm whether in-container scheduling is expected for that workload. - Review the parent process (
process.parent.name,process.parent.executable) to determine whether the invocation came from an interactive shell or an automated process. - Inspect the command that was queued (review the container's
/var/spool/cron/atjobsoratq/at -coutput) to understand what will run and when. - Correlate with prior activity in the same container, such as tool installation, package manager use, or shell access.
False positive analysis
- A small number of images legitimately run "atd" and may schedule maintenance jobs. Baseline these images or namespaces and add exceptions.
Response and remediation
- If unexpected, isolate the container, remove the scheduled job (
at -r), capture the queued command for analysis, and redeploy the workload from a trusted image. - Rotate any credentials exposed to the container and review how the attacker gained execution inside it.