From YAML to Automation

From YAML to Automation#

Both GitHub and GitLab use YAML files as blueprints for automated workflows. However, it may be unclear how the content in these YAML blueprints is transformed into executable instructions and where these instructions actually run.

In the Runners section, we’ll focus on the execution environments. Here, we provide a high-level overview of how remote services convert YAML configurations into actionable steps.

  1. Syntax Parsing: The remote service first checks the YAML syntax for any format errors to ensure the file can be processed.

  2. Parsing and Validating Structure: Once syntax is confirmed, the remote service parses the YAML file, interpreting the structure based on service-specific expectations (like GitHub Workflow or GitLab Pipeline requirements). This involves identifying valid keys (such as jobs or script) and recognizing any errors in structure based on the remote services’ schema, not the YAML format itself.

  3. Generating Execution Instructions: After parsing, the remote service generates a list of jobs, interprets dependencies, and builds an ordered queue of sets of jobs according to the specified order, triggers, and conditions.

  4. Runner Polling for Jobs: Runners are configured to periodically connect to the remote service to check if they qualify for any jobs awaiting execution. When a runner polls, the remote service assigns available jobs based on the runner’s configuration and labels. The runner then downloads the necessary scripts and artifacts and starts executing the assigned job in its environment.

  5. Running on Runners: Each job typically contains information about the environment in which it should run and a set of instructions to perform. The runner first sets up the environment and then executes the commands or actions.

  6. Job Completion and Result Reporting: Upon completion of the job, the runner sends a status report back to the remote service, indicating whether the job succeeded, failed, or was canceled. This report updates the job’s status on the CI/CD dashboard.

  7. Artifact and Log Collection: The runner uploads any artifacts (e.g., compiled binaries, test results) and logs generated during execution to the remote service. These files are stored with the job record, making them accessible for review, debugging, or downstream jobs.

  8. Pipeline Status Update and Notifications: The remote service aggregates job statuses to update the overall pipeline status. If configured, notifications are sent to relevant team members via email, messaging platforms, or through the remote service’s UI.

  9. Retention and Cleanup: Artifacts and logs are retained according to the configured retention policy. After the retention period, the service automatically deletes these files to save storage space.