Kubernetes CronJobs: Scheduling, Retries and Troubleshooting

Kubernetes CronJobs create Jobs on a schedule. They are useful for repeatable work such as reconciliation, reports and periodic checks. Reliable scheduling also depends on the task itself: what happens if it runs late, retries after partial completion or overlaps another run?

This guide uses batch/v1 and an explicit time zone. The example prints a message in a development namespace; it does not implement a backup or business transaction. Check your cluster context and image policies before applying any manifest.

1. Understand CronJob, Job and Pod

The CronJob stores a schedule and a Job template. A created Job manages Pods that run the task. Investigate these layers separately: a missing Job suggests a scheduling problem; a pending Pod suggests placement or policy issues; a failed container requires application evidence.

2. Start with a small scheduled-task example

Save this as hourly-check.yaml. The sample runs at the top of each hour in UTC. Resource values are for this demonstration, not a sizing recommendation for application jobs.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: hourly-check
  namespace: scheduled-demo
spec:
  schedule: "0 * * * *"
  timeZone: "Etc/UTC"
  concurrencyPolicy: Forbid
  startingDeadlineSeconds: 300
  successfulJobsHistoryLimit: 2
  failedJobsHistoryLimit: 2
  jobTemplate:
    spec:
      backoffLimit: 1
      activeDeadlineSeconds: 120
      template:
        spec:
          automountServiceAccountToken: false
          restartPolicy: Never
          securityContext:
            runAsNonRoot: true
            runAsUser: 10001
            runAsGroup: 10001
            seccompProfile:
              type: RuntimeDefault
          containers:
            - name: check
              image: busybox:1.37.0
              command: ["/bin/sh", "-c", "date; echo Scheduled check"]
              securityContext:
                allowPrivilegeEscalation: false
                readOnlyRootFilesystem: true
                capabilities:
                  drop: ["ALL"]
              resources:
                requests:
                  cpu: 10m
                  memory: 16Mi
                limits:
                  cpu: 100m
                  memory: 64Mi

Use an approved image and pin its tested digest for reproducible production deployments. The application must work with the chosen user, filesystem permissions and security settings. A task that writes temporary files needs an appropriate writable volume.

kubectl config current-context
kubectl create namespace scheduled-demo
kubectl apply --dry-run=server -f hourly-check.yaml
kubectl apply -f hourly-check.yaml

Create the namespace only if it does not already exist. Server-side dry run checks API validation and admission in that cluster; it does not prove that a later Pod will run successfully.

3. Make the scheduling policy deliberate

timeZone is stable from Kubernetes 1.27. Without it, the controller’s local time zone applies. Use this field rather than putting TZ or CRON_TZ inside the schedule.

Forbid prevents overlapping Jobs created by this CronJob. Allow permits them; Replace replaces an active Job when the next execution is due. These policies do not coordinate different CronJobs or manually created Jobs that touch the same data.

The five-minute starting deadline limits how late a missed schedule can be started. A deadline is not a promise that every occurrence will run. Choose it against the useful lifetime of the work and monitor skipped or delayed processing.

4. Separate retries from execution deadlines

backoffLimit controls the Job’s retry allowance, while activeDeadlineSeconds bounds its active lifetime. A Pod restart policy is another control. Keep these settings consistent with what the application can safely repeat.

For a task that creates invoices, record a business idempotency key or another application-level safeguard. Kubernetes does not provide an exactly-once transaction guarantee. A retry should recognize completed work instead of sending the same invoice again.

5. Test without waiting for the schedule

Create a one-off Job from the template and inspect its results. The name must be unique for each test:

kubectl create job hourly-check-manual --from=cronjob/hourly-check -n scheduled-demo
kubectl get jobs,pods -n scheduled-demo
kubectl logs job/hourly-check-manual -n scheduled-demo

A manual test is separate from the CronJob controller’s scheduling and can overlap scheduled work. Use a harmless test workload and a controlled environment. Successful container execution should correspond to a real application outcome before using the pattern for business work.

6. Troubleshoot at the failing layer

kubectl get cronjob hourly-check -n scheduled-demo
kubectl describe cronjob hourly-check -n scheduled-demo
kubectl get jobs -n scheduled-demo --sort-by=.metadata.creationTimestamp
kubectl get events -n scheduled-demo --sort-by=.metadata.creationTimestamp

• No Job: check suspension, time zone, schedule, overlap policy and controller events.
• Pending Pod: inspect resource requests, quotas, selectors, admission decisions and image pulls.
• Failed Job: inspect Pod exit status, logs, application permissions and external dependencies.
• Successful Job but missing output: verify application result storage and business-level completion.

7. Treat updates and suspension carefully

Template edits apply to future Jobs, not Jobs that already exist. Setting spec.suspend pauses future scheduling and does not stop running work. Review catch-up behaviour before resuming a long pause; a starting deadline helps define which late work is still useful.

History limits retain only a selected number of completed Jobs. Keep required logs and task outcomes in your normal observability system before cleanup removes the underlying objects.

Frequently asked questions

Can a CronJob run twice?

Scheduling is approximate, so design for duplicate or missed execution. Overlap controls help but do not replace application-level idempotency.

Is a successful backup Job enough?

No. Verify that the backup exists, is accessible and can be restored through a tested recovery procedure.

References and related reading

• CronJob scheduling and limitations
• Job retries and deadlines
• Creating a Job from a CronJob
• Kubernetes runtime monitoring