sourceconditiontarget

✔

✔

✔

Description

source

The Shell "source" retrieves the source value from the standard output of a shell command.

condition

The Shell "condition" validates the condition by checking the exit code of a shell command with a source as argument.

target

The Shell "targets" delegates the update of your file(s) to a shell command with a source as argument.

Parameters

The following parameters can be specified through the spec attributes map:

NameTypeDescriptionRequired
changedif

“changedif” defines how Updatecli interprets the result of the shell command.

In Updatecli, a success means nothing changed, a warning means something changed, and an error means something went wrong.

compatible:

  • source
  • condition
  • target

default: console/output

remark:

  • accepted kinds are “console/output”, “exitcode” and “file/checksum”.
  • “console/output” checks the command output. In a target, any output on stdout means something changed, otherwise nothing changed.
  • “exitcode” checks the command exit code.
  • “file/checksum” checks the checksum of files before and after the command.

example:

  targets:
    default:
      name: 'doc: synchronize release note'
      kind: 'shell'
      disablesourceinput: true
      spec:
        command: 'releasepost --dry-run="$DRY_RUN" --config {{ .config }} --clean'
        environments:
          - name: 'GITHUB_TOKEN'
          - name: 'PATH'
        changedif:
          kind: 'exitcode'
          spec:
            warning: 0
            success: 1
            failure: 2
  targets:
    default:
      disablesourceinput: true
      name: Example of a shell command with a checksum success criteria
      kind: shell
      spec:
        command: |
          yq -i '.a.b[0].c = "cool"' file.yaml
        changedif:
          kind: file/checksum
          spec:
            files:
              - file.yaml
commandstring

“command” defines the shell command to run.

compatible:

  • source
  • condition
  • target

remark:

  • “command” is required.
  • in a condition or a target, the source output is appended to the command as its last argument. The two following snippets are equivalent:
  targets:
    default:
      name: Example 2
      kind: shell
      sourceid: default
      spec:
        command: 'echo'
  targets:
    default:
      name: Example 2
      kind: shell
      disablesourceinput: true
      spec:
        command: 'echo {{ source "default"}}'
environmentsarray

“environments” defines the environment variables passed to the shell command.

compatible:

  • source
  • condition
  • target

default: depends on the operating system:

  • Windows: PATH, PSModulePath, PSModuleAnalysisCachePath, PATHEXT, TEMP, HOME, USERPROFILE, PROFILE
  • Darwin/Linux: PATH, HOME, USER, LOGNAME, SHELL, LANG, LC_ALL

remark:

  • for security reasons, Updatecli does not pass its whole environment to the shell command. It uses an allow list of environment variables instead.
  • “DRY_RUN” is reserved and set by Updatecli, so it cannot be defined.
  • “UPDATECLI_PIPELINE_STAGE” is set by Updatecli to the current stage.

example:

  • environments:
    • name: PATH
    • name: GITHUB_TOKEN
    namestring

“name” defines the environment variable name.

remark:

  • “name” is required.
    valuestring

“value” defines the environment variable value.

default: the value of the same variable in the Updatecli process environment.

remark:

  • when “value” is unset and the Updatecli process does not define the variable, an error is logged and the variable is passed with an empty value.
shellstring

“shell” defines the shell interpreter used to run the command.

compatible:

  • source
  • condition
  • target

default: depends on the operating system:

  • Windows: “powershell.exe -executionpolicy remotesigned -File”
  • Darwin/Linux: “/bin/sh”
workdirstring

“workdir” defines the working directory from where the command runs.

compatible:

  • source
  • condition
  • target

default: the scm checkout directory, or the manifest directory when no scm is set.

remark:

  • a relative path is joined to the default directory.
  • an absolute path is used as is.

Environments

NameRequiredDefaultDescription

name

✔

Environment variable name

value

Environments variable value. If the value is empty, then we inherit the value from the updatecli process.

The command does not inherit the whole environment of the Updatecli process - that would leak every secret in your CI environment into every script. Instead Updatecli works from an allow list.

Leaving environments unset does not mean an empty environment, though: a small default list is passed, which differs per operating system.

OSVariables passed when environments is unset

Linux / macOS

PATH, HOME, USER, LOGNAME, SHELL, LANG, LC_ALL

Windows

PATH, PATHEXT, TEMP, HOME, USERPROFILE, PROFILE, PSModulePath, PSModuleAnalysisCachePath

Setting environments replaces that default list rather than adding to it, so a script that needs both a token and PATH must ask for both:

environments:
  - name: PATH
  - name: GITHUB_TOKEN

Verified with a script that prints its own environment, MY_SECRET does not reach it:

$ MY_SECRET=leak updatecli diff --config shell.yaml
_ HOME LANG LC_ALL LOGNAME PATH PWD SHELL SHLVL UPDATECLI_PIPELINE_STAGE USER

To specify an environment variable, you can use one of the following examples:

  scenario1:
    name: PATH value is inherited from Updatecli process
    kind: shell
    spec:
      command: ./examples/scripts.d/env.sh
      environments:
        - name: PATH

  scenario2:
    name: MY_ENV value is set by the Updatecli manifest
    kind: shell
    spec:
      command: ./examples/scripts.d/env.sh
      environments:
        - name: MY_ENV
          value: setmyvalue

Variables injected by Updatecli

Two variables are added on top of the allow list, whatever environments says:

UPDATECLI_PIPELINE_STAGE

The stage running the command - source, condition or target. Useful for a single script shared between stages.

DRY_RUN

true under updatecli diff, false under updatecli apply.

Important

DRY_RUN is injected for targets only. It is absent from the environment of a source or a condition, so $DRY_RUN there expands to the empty string - not to false. A script guarding on if [ "$DRY_RUN" = "false" ] therefore never takes that branch when used as a source.

Shell Source

The command provided by your spec section is executed by updatecli to retrieve the source:

  • If the commands runs successfully (e.g. with an exit code of zero), the source takes the value of the inlined (e.g. removing the line returns) standard output.

  • An error is raised if the exit code of the command is different than zero. The error prints the content from both standard error (stderr) and output (stdout) of the command.

Source Examples

  • The following example should result in a source value of 1.2.3:

    updateCli.yaml:
    sources:
      example:
        kind: shell
        spec:
          command: echo 1.2.3
    $ updatecli apply
    # ...
    
    SOURCE:
    =======
    
    ✔ The shell 🐚 command "echo 1.2.3" ran successfully and retrieved the following source value: "1.2.3"
    
    # ...
  • The following example should result in a failing source:

    updateCli.yaml:
    sources:
      example:
        kind: shell
        spec:
          command: ls /foobar
    $ updatecli apply
    # ...
    
    SOURCE:
    =======
    
    ERROR: ✗ The shell 🐚 command "ls /foobar" failed with an exit code of 1 and the following messages:
    stderr=
    ls: /foobar: No such file or directory
    stdout=

Shell Condition

The command provided by your spec section is executed by updatecli to validate the condition:

  • If the commands runs successfully (e.g. with an exit code of zero), then the condition is validated.

  • Otherwise (an exit code different than zero) the condition is marked as not valid, and both the standard error and outputs are logged.

💡 The source associated to this condition (the default source or through the key sourceID) is appended as the last argument to the command line unless the attribute disablesourceinput is set to true.

Condition Examples

The following examples show how to test if the value from the source named newVersion equals to 1.2.3:

conditions:
  default:
    kind: shell
    sourceID: newVersion
    spec:
      # The value of the source "newVersion" is appended by updatecli
      command: test 1.2.3 ==
  • If the value of the source is 1.2.3, then you get:

    CONDITIONS:
    ===========
    
    ✔ The shell 🐚 command "test 1.2.3 == 1.2.3" successfully validated the condition.
  • If the value of the source is 2.0.0, than you get:

    CONDITIONS:
    ===========
    
    The shell 🐚 command "test 1.2.3 == 2.0.0" failed with an exit code of 1 and the following messages:
    stderr=
    
    stdout=
    
    ✗ condition not met, skipping pipeline

In this example the command is executed without the source appended:

updateCli.yaml:
conditions:
  checkIfMavenReleaseIsAvailable:
    kind: shell
    disablesourceinput: true
    spec:
      command: curl https://google.com
$ updatecli apply
# ...

CONDITIONS:
===========
✔ The shell 🐚 command "curl https://google.com" successfully validated the condition.
 # ...
 Run Summary
 ===========
 1 job run
 0 job succeed
 0 job failed
 1 job applied changes

Shell Target

The command provided by your spec section is executed by updatecli to change your files:

  • When the commands runs successfully (e.g. with an exit code of zero), the behavior depends on the content of the standard output:

    • If it is empty, then updatecli reports a success with no changes applied.

    • Otherwise updatecli reports a success with the content of the standard output as the resulting value of the change.

  • Otherwise (an exit code different than zero) the condition is marked as not valid, and both the standard error and outputs are logged.

Please note that:

  • 💡 The source associated to this target (the default source or through the key sourceID) is appended as the last argument to the command line.

  • 💡 The DRY_RUN environment variable is set to the value true when using updatecli diff to report that any change should only be reported and not applied.

Target Examples

Consider the following shell script target.sh:

#!/bin/bash
# Script "target.sh"
# The script check the content of the file "version.txt"
# - if different than $1 and DRY_RUN is set to:
#   - "false" then it updates it with the value of $1
#   - "true" then it only reports the value of $1
# - otherwise it exits without any value reported
version_file=version.txt

if test "$1" == "$(cat "${version_file}")"
then
  ## No change
  # early return with no output
  exit 0
else
  if test "$DRY_RUN" == "false"
  then
    ## Value changed to $1" - NO dry run
    # do something such as writing a file here
    echo "$1" > "${version_file}"
  fi
  # Report on stdout
  echo "$1"
  exit 0
fi

With the following manifest:

updateCli.yaml:
sources:
  default:
    kind: shell
    spec:
      command: echo 1.2.4
targets:
  default:
    name: setGrepVersion
    sourceID: default
    kind: shell
    spec:
      command: bash ./examples/updateCli.generic/shell/target.sh

You would have the following behaviors:

  • Running with dry run enabled:

    $ cat version.txt
    1.0.0
    
    $ updatecli diff
    #...
    
    TARGETS:
    ========
    
    **Dry Run enabled**
    
    ⚠ The shell 🐚 command "bash ./examples/updateCli.generic/shell/target.sh 1.2.4" ran successfully and reported the following change: "1.2.4".
    
    $ cat version.txt
    1.0.0 # No change
  • Applying the changes:

    $ updatecli apply
    #...
    
    TARGETS:
    ========
    
    ⚠ The shell 🐚 command "bash ./examples/updateCli.generic/shell/target.sh 1.2.4" ran successfully and reported the following change: "1.2.4".
    
    $ cat version.txt
    1.2.4 # Version changed

Reference

sources:
  newVersion:
    kind: shell
    name: Get new version
    spec:
      command: bash ./get-new-version.sh"
  failing:
    kind: shell
    name: Failing command
    spec:
      command: ls /foobar
conditions:
  checkIfVersionEquals123:
    kind: shell
    sourceId: newVersion
    spec:
      command: test 1.2.3 ==
targets:
  default:
    name: setGrepVersion
    sourceID: default
    kind: shell
    spec:
      command: bash apply.sh

FAQ

Why can’t I execute an updatecli manifest with a local script using the shell provider?

Updatecli behaves differently if it uses a SCM configuration or not. If no SCM configuration is provided, then it sets the working directory to where updatecli is executed. But if a SCM configuration is provided, then it clones the git repository in a temporary directory such as /tmp/updatecli and then set the working directory to that temporary such as /tmp/updatecli/<git repository>. While it allows updatecli to work from a "clean" directory, it makes the testing of the local updatecli manifest a bit more complicated. We are investigating the best solution to address this but for now the best way to test is to comment out scmid such as

targets:
  jsonschema:
    name: "Update updatecli jsonschema"
    kind: "shell"
    #scmid: "default"
    spec:
      command: "./updatecli/scripts/jsonschema.sh"