Azure/Azure Devops + az extension

From Ever changing code
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

What is Azure DevOps:

  • Azure Repos - SCM/VCS system hosted in Azure
  • Azure Pipelines - Build, test, release automations
  • Azure Boards - Kanban board; JIRA like to track work, code defects, issues using Kanban or Scrum
  • Azure Test Plans
  • Azure Artifacts - share Maven, npm, NuGet from private and public sources

dotnet commands reference

dotnet restore **/*.csproj # Restores the dependencies and tools of a project
dotnet build               # Builds a project and all of its dependencies
dotnet test **/*[Tt]ests/*.csproj # .NET test driver used to execute unit tests
dotnet publish # Publishes the application and its dependencies to a folder for deployment to a hosting system

az-cli with azure-devops extension

There is a helpful azure-devops extension to az utility to work with ADO. Install and usage, below:

<syntaxhighlightjs lang="bash"> az extension list az extension show --name azure-devops az extension add --name azure-devops [

 {
   "experimental": false,
   "extensionType": "whl",
   "name": "azure-devops",
   "path": "/home/piotr/.azure/cliextensions/azure-devops",
   "preview": false,
   "version": "0.18.0"
 }

]

  1. Azure login

az login # login to get subscription level authentication, >> not needed for DevOps login az account show

  1. Azure DevOps login

export AZURE_DEVOPS_EXT_PAT=*** # if not set you will be prompted az devops login # login to Azure DevOps Token: **** # <- generate a token in AzureDevops > UserSettings > Personal tokens

  1. Optional. Set default organization and project

az devops configure --defaults organization=https://myorg.visualstudio.com project=myproject

  1. Operations

GROUP_NAME=mygroup GROUP_ID=$(az pipelines variable-group list --group-name $GROUP_NAME --query '[].id' | jq -r .[]) az pipelines variable-group variable list --group-id $GROUP_ID -o yamlc # or --id

  1. >> Note secrets value won't show

az pipelines build list -o table az pipelines variable-group list --group-name $GROUP_NAME -o yamlc --query '[].variables' # color YAML

  1. | available outputs: json, jsonc, none, table, tsv, yaml, yamlc. Default: json.
  1. Long example
  2. Linux

az repos list --organization=https://myorg.visualstudio.com --project=myproject --query '[].{Name:name, Url:remoteUrl}' -o json | jq -r .[].Name

  1. PowerShell

(az repos list --query '[].{Name:name, Url:remoteUrl}' -o json | ConvertFrom-Json) | %{ git clone $_.Url } </syntaxhighlightjs>

Azure pipeline agents

Software included:

Environment:

# Default working directory, also where repos get cloned
$(Build.SourcesDirectory): /home/vsts/work/1/s

# Published artifacts
cd ../name-of-artifact/

# Layout of $(Agent.BuildDirectory) that is '/home/vsts/work/1', output of 'ls -la' is shown below:
$pwd; cd .. # level up from pwd
drwxr-xr-x 7 vsts docker 4096 Sep  1 22:11 .
drwxr-xr-x 7 vsts root   4096 Sep  1 22:11 ..
drwxr-xr-x 2 vsts docker 4096 Sep  1 22:11 TestResults
drwxr-xr-x 2 vsts docker 4096 Sep  1 22:11 a # pipeline artifact dir
drwxr-xr-x 2 vsts docker 4096 Sep  1 22:11 b # binaries directory
drwxr-xr-x 2 vsts docker 4096 Sep  1 22:11 s # source directory often $PWD, where the repo code gets checked out
drwxr-xr-x 6 vsts docker 4096 Sep  1 22:11 name-of-artifact

# Layout env variables, executed from a pipeline on the agent 
$ grep -e "DIRECTORY\|^AGENT" /tmp/1 | sort
AGENT_ACCEPTTEEEULA=True
AGENT_BUILDDIRECTORY=/home/vsts/work/1
AGENT_DISABLELOGPLUGIN_TESTFILEPUBLISHERPLUGIN=true
AGENT_DISABLELOGPLUGIN_TESTRESULTLOGPLUGIN=true
AGENT_HOMEDIRECTORY=/home/vsts/agents/2.174.1
AGENT_ID=9
AGENT_JOBNAME=terraform plan
AGENT_JOBSTATUS=Succeeded
AGENT_MACHINENAME=fv-az605
AGENT_NAME=Hosted Agent
AGENT_OSARCHITECTURE=X64
AGENT_OS=Linux
AGENT_READONLYVARIABLES=true
AGENT_RETAINDEFAULTENCODING=false
AGENT_ROOTDIRECTORY=/home/vsts/work
AGENT_TEMPDIRECTORY=/home/vsts/work/_temp
AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache
AGENT_VERSION=2.174.1
AGENT_WORKFOLDER=/home/vsts/work
BUILD_ARTIFACTSTAGINGDIRECTORY=/home/vsts/work/1/a
BUILD_BINARIESDIRECTORY=/home/vsts/work/1/b
BUILD_SOURCESDIRECTORY=/home/vsts/work/1/s
BUILD_STAGINGDIRECTORY=/home/vsts/work/1/a
COMMON_TESTRESULTSDIRECTORY=/home/vsts/work/1/TestResults
RUNNER_TOOLSDIRECTORY=/opt/hostedtoolcache
SYSTEM_ARTIFACTSDIRECTORY=/home/vsts/work/1/a
SYSTEM_DEFAULTWORKINGDIRECTORY=/home/vsts/work/1/s

$(Agent.BuildDirectory)   # /home/vsts/work/1
$(Build.SourcesDirectory) # /home/vsts/work/1/s
$(Build.Repository.LocalPath)


Troubleshoot snippets

- stage      : deploy
    displayName: deploy
    dependsOn  : plan
    jobs       :
      - deployment : deploy
        displayName: deploy
        environment: env-${{ parameters.environment }}
        strategy:
          runOnce:
            deploy:
              steps:
              - checkout: self  # this is needed for job 'deployment' type, common job type 'job' already contains git code
              - task       : AmazonWebServices.aws-vsts-tools.AWSShellScript.AWSShellScript@1
                displayName: deploy
                env        :
                  VARIBLE_1 : $(variable_2)
                inputs     :
                  awsCredentials      : $(service_connection_name)
                  regionName          : $(aws_region)
                  failOnStandardError : false
                  args                : ""
                  scriptType          : inline
                  #disableAutoCwd     : true
                  #workingDirectory   : $(Agent.BuildDirectory)/myrepo
                  inlineScript        : |
                    #!/bin/bash
                    sudo apt update  -qq
                    sudo apt install -qq tree
                    printf '\n[DEBUG] $SHELL       : %s' "$SHELL"
                    printf '\n[DEBUG] $BASH_VERSION: %s' "$BASH_VERSION"
                    printf '\n[DEBUG] $PDW         : %s' "$PWD"
                    printf '\n[DEBUG] $ENVIRONMENT=%s\n' "${ENVIRONMENT}"
                    printf '\n[DEBUG] $..(..Agent.BuildDirectory): %s' "$(Agent.BuildDirectory)"
                    printf '\n[DEBUG] pwd; ls -la\n'               ; pwd; ls -la
                    printf '\n[DEBUG] ls -la ../helm-charts\n'     ; ls -la ../helm-charts
                    printf '\n[DEBUG] ls -la ../s\n'               ; ls -la ../s
                    printf '\n[DEBUG] find ../.. -iname file.sh\n' ; find ../.. -iname file.sh
                    printf '\n[DEBUG] tree -aL 2 ..\n'             ; tree -aL 2 ..

                    time source  $(Agent.BuildDirectory)/myrepo/scripts/file.sh
                    #time source      $(Build.SourcesDirectory)/scripts/file.sh

Pipeline variables and secrets

Variables.md are available in expressions as well as scripts; see variables to learn more about how to use them. There are some predefined build and release variables you can also rely on. Syntax

regionName    : "${{ variables.aws_region }}"
regionName    : $(aws_region)

# set Dynamic Variable Secret
$token = curl ...
echo "##vso[task.setvariable variable=accesstoken;isSecret=true]$token"

# print secrets, the value vertically if you print them as chars:
$Password.ToCharArray()


Resources:

Pipeline tasks

Stage

<syntaxhighlightjs lang=yaml> stages:

 - stage      : deploy_helm
   displayName: deploy_helm
   jobs       :
     - job        : deploy_helm
       displayName: deploy_helm
       steps      :
         - task       : AmazonWebServices.aws-vsts-tools.AWSShellScript.AWSShellScript@1
           displayName: "deploy_helm"
           env        :
             DBPASSWORD : $(auth_service_client_secret)
             VALUES_PATH: $(values-path)
           inputs     :
           # https://github.com/aws/aws-toolkit-azure-devops/blob/master/Tasks/AWSShellScript/task.json
             awsCredentials: $(aws_creds)
             regionName          : $(aws_region)
             #failOnStandardError: true # default: false
             #disableAutoCwd     : true # default: false, The default behavior is to set the working directory to the script location
             #workingDirectory   : $(Build.SourcesDirectory) # | This enables you to optionally specify a different working directory.
             scriptType          : inline # default: filePath
             #filePath           : $(Build.SourcesDirectory)/scripts/connect.sh
             inlineScript        : |
               #!/bin/bash
               printf "\n[DEBUG] PWD: %s" "$PWD"
               time source $(Build.SourcesDirectory)/scripts/connect.sh || \
                 { printf "\n[ERROR] to connect"; exit 1; }
             args                : "" # Arguments passed to the shell script

</syntaxhighlightjs>

Conditions

parameters:
- name: enabled
  type: boolean
  default: false

stages:
  - stage: ${{ parameters.stage_name_prefix }}_build
    condition: and(succeeded(), eq('${{ parameters.enabled }}', 'true'))
    displayName: ${{parameters.stage_display_name}}
    jobs:

Logging and formatting in the pipeline

##[group]Beginning of a group
##[warning]Warning message
##[error]Error message
##[debug]Debug text
##[command]Command-line being run
##[endgroup]

Resources

Code Assessment Tools