Azure/Azure Devops + az extension

What is Azure DevOps:

• Azure Repos - SCM/VCS system hosted in Azure
• Azure Pipelines - Build, test, release automations
• Azure Boards - Kanban board; JIRA like to track work, code defects, issues using Kanban or Scrum
• Azure Test Plans
• Azure Artifacts - share Maven, npm, NuGet from private and public sources

dotnet commands reference

dotnet restore **/*.csproj # Restores the dependencies and tools of a project
dotnet build               # Builds a project and all of its dependencies
dotnet test **/*[Tt]ests/*.csproj # .NET test driver used to execute unit tests
dotnet publish # Publishes the application and its dependencies to a folder for deployment to a hosting system

az-cli with azure-devops extension

There is a helpful azure-devops extension to az utility to work with ADO. Install and usage, below:

• submodules: artifacts, boards, devops, pipelines, repos
• manage library/variable-groups with examples

<syntaxhighlightjs lang="bash"> az extension list az extension show --name azure-devops az extension add --name azure-devops [

 {
   "experimental": false,
   "extensionType": "whl",
   "name": "azure-devops",
   "path": "/home/piotr/.azure/cliextensions/azure-devops",
   "preview": false,
   "version": "0.18.0"
 }

]

1. Azure login

az login # login to get subscription level authentication, >> not needed for DevOps login az account show

1. Azure DevOps login

export AZURE_DEVOPS_EXT_PAT=*** # if not set you will be prompted az devops login # login to Azure DevOps Token: **** # <- generate a token in AzureDevops > UserSettings > Personal tokens

1. Optional. Set default organization and project

az devops configure --defaults organization=https://myorg.visualstudio.com project=myproject

1. Operations

GROUP_NAME=mygroup GROUP_ID=$(az pipelines variable-group list --group-name $GROUP_NAME --query '[].id' | jq -r .[]) az pipelines variable-group variable list --group-id $GROUP_ID -o yamlc # or --id

1. >> Note secrets value won't show

az pipelines build list -o table az pipelines variable-group list --group-name $GROUP_NAME -o yamlc --query '[].variables' # color YAML

1. | available outputs: json, jsonc, none, table, tsv, yaml, yamlc. Default: json.

1. Long example
2. Linux

az repos list --organization=https://myorg.visualstudio.com --project=myproject --query '[].{Name:name, Url:remoteUrl}' -o json | jq -r .[].Name

1. PowerShell

(az repos list --query '[].{Name:name, Url:remoteUrl}' -o json | ConvertFrom-Json) | %{ git clone $_.Url } </syntaxhighlightjs>

Azure pipeline agents

Software included:

• ubuntu 18.04
• Networking and weekly IP file

Environment:

# Default working directory, also where repos get cloned
$(Build.SourcesDirectory): /home/vsts/work/1/s

# Published artifacts
cd ../name-of-artifact/

# Layout of $(Agent.BuildDirectory) that is '/home/vsts/work/1', output of 'ls -la' is shown below:
$pwd; cd .. # level up from pwd
drwxr-xr-x 7 vsts docker 4096 Sep  1 22:11 .
drwxr-xr-x 7 vsts root   4096 Sep  1 22:11 ..
drwxr-xr-x 2 vsts docker 4096 Sep  1 22:11 TestResults
drwxr-xr-x 2 vsts docker 4096 Sep  1 22:11 a # pipeline artifact dir
drwxr-xr-x 2 vsts docker 4096 Sep  1 22:11 b # binaries directory
drwxr-xr-x 2 vsts docker 4096 Sep  1 22:11 s # source directory often $PWD, where the repo code gets checked out
drwxr-xr-x 6 vsts docker 4096 Sep  1 22:11 name-of-artifact

# Layout env variables, executed from a pipeline on the agent 
$ grep -e "DIRECTORY\|^AGENT" /tmp/1 | sort
AGENT_ACCEPTTEEEULA=True
AGENT_BUILDDIRECTORY=/home/vsts/work/1
AGENT_DISABLELOGPLUGIN_TESTFILEPUBLISHERPLUGIN=true
AGENT_DISABLELOGPLUGIN_TESTRESULTLOGPLUGIN=true
AGENT_HOMEDIRECTORY=/home/vsts/agents/2.174.1
AGENT_ID=9
AGENT_JOBNAME=terraform plan
AGENT_JOBSTATUS=Succeeded
AGENT_MACHINENAME=fv-az605
AGENT_NAME=Hosted Agent
AGENT_OSARCHITECTURE=X64
AGENT_OS=Linux
AGENT_READONLYVARIABLES=true
AGENT_RETAINDEFAULTENCODING=false
AGENT_ROOTDIRECTORY=/home/vsts/work
AGENT_TEMPDIRECTORY=/home/vsts/work/_temp
AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache
AGENT_VERSION=2.174.1
AGENT_WORKFOLDER=/home/vsts/work
BUILD_ARTIFACTSTAGINGDIRECTORY=/home/vsts/work/1/a
BUILD_BINARIESDIRECTORY=/home/vsts/work/1/b
BUILD_SOURCESDIRECTORY=/home/vsts/work/1/s
BUILD_STAGINGDIRECTORY=/home/vsts/work/1/a
COMMON_TESTRESULTSDIRECTORY=/home/vsts/work/1/TestResults
RUNNER_TOOLSDIRECTORY=/opt/hostedtoolcache
SYSTEM_ARTIFACTSDIRECTORY=/home/vsts/work/1/a
SYSTEM_DEFAULTWORKINGDIRECTORY=/home/vsts/work/1/s

$(Agent.BuildDirectory)   # /home/vsts/work/1
$(Build.SourcesDirectory) # /home/vsts/work/1/s
$(Build.Repository.LocalPath)

Troubleshoot snippets

- stage      : deploy
    displayName: deploy
    dependsOn  : plan
    jobs       :
      - deployment : deploy
        displayName: deploy
        environment: env-${{ parameters.environment }}
        strategy:
          runOnce:
            deploy:
              steps:
              - checkout: self  # this is needed for job 'deployment' type, common job type 'job' already contains git code
              - task       : AmazonWebServices.aws-vsts-tools.AWSShellScript.AWSShellScript@1
                displayName: deploy
                env        :
                  VARIBLE_1 : $(variable_2)
                inputs     :
                  awsCredentials      : $(service_connection_name)
                  regionName          : $(aws_region)
                  failOnStandardError : false
                  args                : ""
                  scriptType          : inline
                  #disableAutoCwd     : true
                  #workingDirectory   : $(Agent.BuildDirectory)/myrepo
                  inlineScript        : |
                    #!/bin/bash
                    sudo apt update  -qq
                    sudo apt install -qq tree
                    printf '\n[DEBUG] $SHELL       : %s' "$SHELL"
                    printf '\n[DEBUG] $BASH_VERSION: %s' "$BASH_VERSION"
                    printf '\n[DEBUG] $PDW         : %s' "$PWD"
                    printf '\n[DEBUG] $ENVIRONMENT=%s\n' "${ENVIRONMENT}"
                    printf '\n[DEBUG] $..(..Agent.BuildDirectory): %s' "$(Agent.BuildDirectory)"
                    printf '\n[DEBUG] pwd; ls -la\n'               ; pwd; ls -la
                    printf '\n[DEBUG] ls -la ../helm-charts\n'     ; ls -la ../helm-charts
                    printf '\n[DEBUG] ls -la ../s\n'               ; ls -la ../s
                    printf '\n[DEBUG] find ../.. -iname file.sh\n' ; find ../.. -iname file.sh
                    printf '\n[DEBUG] tree -aL 2 ..\n'             ; tree -aL 2 ..

                    time source  $(Agent.BuildDirectory)/myrepo/scripts/file.sh
                    #time source      $(Build.SourcesDirectory)/scripts/file.sh

Pipeline variables and secrets

Variables.md are available in expressions as well as scripts; see variables to learn more about how to use them. There are some predefined build and release variables you can also rely on. Syntax

regionName    : "${{ variables.aws_region }}"
regionName    : $(aws_region)

# set Dynamic Variable Secret
$token = curl ...
echo "##vso[task.setvariable variable=accesstoken;isSecret=true]$token"

# print secrets, the value vertically if you print them as chars:
$Password.ToCharArray()

Resources:

• variables-complete-guide blog

Pipeline tasks

• aws-toolkit-azure-devops set of tasks
• azure-pipelines-tasks by Microsoft

Stage

<syntaxhighlightjs lang=yaml> stages:

 - stage      : deploy_helm
   displayName: deploy_helm
   jobs       :
     - job        : deploy_helm
       displayName: deploy_helm
       steps      :
         - task       : AmazonWebServices.aws-vsts-tools.AWSShellScript.AWSShellScript@1
           displayName: "deploy_helm"
           env        :
             DBPASSWORD : $(auth_service_client_secret)
             VALUES_PATH: $(values-path)
           inputs     :
           # https://github.com/aws/aws-toolkit-azure-devops/blob/master/Tasks/AWSShellScript/task.json
             awsCredentials: $(aws_creds)
             regionName          : $(aws_region)
             #failOnStandardError: true # default: false
             #disableAutoCwd     : true # default: false, The default behavior is to set the working directory to the script location
             #workingDirectory   : $(Build.SourcesDirectory) # | This enables you to optionally specify a different working directory.
             scriptType          : inline # default: filePath
             #filePath           : $(Build.SourcesDirectory)/scripts/connect.sh
             inlineScript        : |
               #!/bin/bash
               printf "\n[DEBUG] PWD: %s" "$PWD"
               time source $(Build.SourcesDirectory)/scripts/connect.sh || \
                 { printf "\n[ERROR] to connect"; exit 1; }
             args                : "" # Arguments passed to the shell script

</syntaxhighlightjs>

Conditions

parameters:
- name: enabled
  type: boolean
  default: false

stages:
  - stage: ${{ parameters.stage_name_prefix }}_build
    condition: and(succeeded(), eq('${{ parameters.enabled }}', 'true'))
    displayName: ${{parameters.stage_display_name}}
    jobs:

Logging and formatting in the pipeline

##[group]Beginning of a group
##[warning]Warning message
##[error]Error message
##[debug]Debug text
##[command]Command-line being run
##[endgroup]

Resources

• azuredevopsdemogenerator
• #3 Azure DevOps w bazach danych BitPeak, youtube
• azure-devops-tips-each-loops-c082c692d025 medium 2019

Code Assessment Tools

• Security Tools
• SonarCloud
• SonarAnalyzer
• FxCop - check your code for security, performance, and design issues by Microsoft