Set Up Slack Alerts for Job Results

How to set up alerting to Slack for a CI job.

We can set up Slack alerts for job results by defining reporter_config in the job definition. This is possible for all job types: periodics, presubmits, and postsubmits. Prowgen is capable of adding this to the job definition.

1
2
3
4
5
6
reporter_config:
  slack:
    channel: '#forum'
    job_states_to_report:
    - failure
    report_template: Job {{.Spec.Job}} failed.

For example, by the above snippet, a Slack alert will be sent out to #forum channel when there is a failure of the job. The alert is formatted by report_template.

  • The channel has to be in coreos.slack.com.
  • The channel has to be public. If it is not then the @prow bot has to be added to it otherwise it won’t be able to properly post messages.
  • The state in job_states_to_report has to be a valid Prow job state. See upstream documentation.
  • The value of report_template is a Go template and it will be applied to the Prow job instance. The annotations such as {{.Spec.Job}} will be replaced by the job name when the alert is received in Slack. See upstream documentation for more fields of a Prow job. Note that no alerts will be sent out if the template is broken, e.g., cannot be parsed or applied successfully.

Setting up Slack notifications with Prowgen

Prowgen is the tool that is used to generate ProwJobs from ci-operator configuration. It is possible to instruct Prowgen to add a reporter_config to specific jobs. Doing so requires creating or updating the repo (or organization) in question’s .config.prowgen file. This YAML configuration file is stored in the repo’s ci-operator folder. For example, ci-tools configuration file is found at ci-operator/config/openshift/ci-tools/.config.prowgen. The following example illustrates how to utilize this config to add slack notifications for jobs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
slack_reporter:
- channel: "#slack-channel"
  job_states_to_report: #Accepts any ProwJob status
  - success
  - failure
  - error
  report_template: '{{if eq .Status.State "success"}} :rainbow: Job *{{.Spec.Job}}*
                           ended with *{{.Status.State}}*. <{{.Status.URL}}|View logs> :rainbow: {{else}}
                           :volcano: Job *{{.Spec.Job}}* ended with *{{.Status.State}}*. <{{.Status.URL}}|View
                           logs> :volcano: {{end}}'
  job_names: # Listing of job names (ci-operator's 'as' field) that this configuration applies to
  - unit
  - upload-results
  - lint
  excluded_variants: # It is possible to exclude notifications from specific variants, for example, when you don't require them from older releases
  - some-old-release

Once the configuration is added, simply use the make jobs target to generate the new job definitions containing the reporter_config. The SlackReporterConfig is provided for reference.