Job Primera job name is very important. Please make sure that the job names contain correct information. (see options below)
JobPrimer is the
ci-tool that is used to populate the
Jobs table. The
Jobs table dictates which periodic jobs are ingested during
disruption data gathering.
JobPrimer runs periodically in our CI. There are two sub commands described below that run on a cron schedule.
The configuration for
prime-job-tablewhich updates the table in big query can be found in the
DPCR Job Aggregation Configs(private repo)
High Level Diagram
How The Data Flows
We first look at the
origin/releaserepo to gather a list of the current release jobs that were created. The below command is ran to look through the current configuration and generate the job names.
./job-run-aggregator generate-job-names > pkg/jobrunaggregator/jobtableprimer/generated_job_names.txt
generated_jobs_names.txtis then committed to the repo.
You must then rebuild the binary so the newly generated list is correctly embedded.
We then create the jobs in the BigQuery table by running the
prime-job-tablecommand. This will use the embedded
generated_jobs_names.txtdata and generate the
Jobsrows based off of the naming convention (see below). After which the
Jobstable should be updated with the latest jobs.
Please make sure your job names follow the convention defined below. All job names must include adequate information to allow proper data aggregation.
- aws, gcp, azure, etc…
- Architecture: (default:
- arm64, ppc64le, s390x
- Upgrade: (default:
assumes NOT upgrade)
- Network: (default:
sdn && ipv4)
- sdn, ovn
- ipv6, ipv4
- Topology: (default:
- Serial: (default: