Azure/Azure Devops + az extension
What is Azure DevOps:
- Azure Repos - SCM/VCS system hosted in Azure
- Azure Pipelines - Build, test, release automations
- Azure Boards - Kanban board; JIRA like to track work, code defects, issues using Kanban or Scrum
- Azure Test Plans
- Azure Artifacts - share Maven, npm, NuGet from private and public sources
dotnet commands reference
dotnet restore **/*.csproj # Restores the dependencies and tools of a project dotnet build # Builds a project and all of its dependencies dotnet test **/*[Tt]ests/*.csproj # .NET test driver used to execute unit tests dotnet publish # Publishes the application and its dependencies to a folder for deployment to a hosting system
az-cli with azure-devops extension
There is a helpful azure-devops
extension to az
utility to work with ADO. Install and usage, below:
- submodules: artifacts, boards, devops, pipelines, repos
- manage library/variable-groups with examples
<syntaxhighlightjs lang="bash"> az extension list az extension show --name azure-devops az extension add --name azure-devops [
{ "experimental": false, "extensionType": "whl", "name": "azure-devops", "path": "/home/piotr/.azure/cliextensions/azure-devops", "preview": false, "version": "0.18.0" }
]
- Azure login
az login # login to get subscription level authentication, >> not needed for DevOps login az account show
- Azure DevOps login
export AZURE_DEVOPS_EXT_PAT=*** # if not set you will be prompted az devops login # login to Azure DevOps Token: **** # <- generate a token in AzureDevops > UserSettings > Personal tokens
- Optional. Set default organization and project
az devops configure --defaults organization=https://myorg.visualstudio.com project=myproject
- Operations
GROUP_NAME=mygroup GROUP_ID=$(az pipelines variable-group list --group-name $GROUP_NAME --query '[].id' | jq -r .[]) az pipelines variable-group variable list --group-id $GROUP_ID -o yamlc # or --id
- >> Note secrets value won't show
az pipelines build list -o table az pipelines variable-group list --group-name $GROUP_NAME -o yamlc --query '[].variables' # color YAML
- | available outputs: json, jsonc, none, table, tsv, yaml, yamlc. Default: json.
- Long example
- Linux
az repos list --organization=https://myorg.visualstudio.com --project=myproject --query '[].{Name:name, Url:remoteUrl}' -o json | jq -r .[].Name
- PowerShell
(az repos list --query '[].{Name:name, Url:remoteUrl}' -o json | ConvertFrom-Json) | %{ git clone $_.Url } </syntaxhighlightjs>
Azure pipeline agents
Software included:
- ubuntu 18.04
- Networking and weekly IP file
Environment:
# Default working directory, also where repos get cloned $(Build.SourcesDirectory): /home/vsts/work/1/s # Published artifacts cd ../name-of-artifact/ # Layout of $(Agent.BuildDirectory) that is '/home/vsts/work/1', output of 'ls -la' is shown below: $pwd; cd .. # level up from pwd drwxr-xr-x 7 vsts docker 4096 Sep 1 22:11 . drwxr-xr-x 7 vsts root 4096 Sep 1 22:11 .. drwxr-xr-x 2 vsts docker 4096 Sep 1 22:11 TestResults drwxr-xr-x 2 vsts docker 4096 Sep 1 22:11 a # pipeline artifact dir drwxr-xr-x 2 vsts docker 4096 Sep 1 22:11 b # binaries directory drwxr-xr-x 2 vsts docker 4096 Sep 1 22:11 s # source directory often $PWD, where the repo code gets checked out drwxr-xr-x 6 vsts docker 4096 Sep 1 22:11 name-of-artifact # Layout env variables, executed from a pipeline on the agent $ grep -e "DIRECTORY\|^AGENT" /tmp/1 | sort AGENT_ACCEPTTEEEULA=True AGENT_BUILDDIRECTORY=/home/vsts/work/1 AGENT_DISABLELOGPLUGIN_TESTFILEPUBLISHERPLUGIN=true AGENT_DISABLELOGPLUGIN_TESTRESULTLOGPLUGIN=true AGENT_HOMEDIRECTORY=/home/vsts/agents/2.174.1 AGENT_ID=9 AGENT_JOBNAME=terraform plan AGENT_JOBSTATUS=Succeeded AGENT_MACHINENAME=fv-az605 AGENT_NAME=Hosted Agent AGENT_OSARCHITECTURE=X64 AGENT_OS=Linux AGENT_READONLYVARIABLES=true AGENT_RETAINDEFAULTENCODING=false AGENT_ROOTDIRECTORY=/home/vsts/work AGENT_TEMPDIRECTORY=/home/vsts/work/_temp AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache AGENT_VERSION=2.174.1 AGENT_WORKFOLDER=/home/vsts/work BUILD_ARTIFACTSTAGINGDIRECTORY=/home/vsts/work/1/a BUILD_BINARIESDIRECTORY=/home/vsts/work/1/b BUILD_SOURCESDIRECTORY=/home/vsts/work/1/s BUILD_STAGINGDIRECTORY=/home/vsts/work/1/a COMMON_TESTRESULTSDIRECTORY=/home/vsts/work/1/TestResults RUNNER_TOOLSDIRECTORY=/opt/hostedtoolcache SYSTEM_ARTIFACTSDIRECTORY=/home/vsts/work/1/a SYSTEM_DEFAULTWORKINGDIRECTORY=/home/vsts/work/1/s $(Agent.BuildDirectory) # /home/vsts/work/1 $(Build.SourcesDirectory) # /home/vsts/work/1/s $(Build.Repository.LocalPath)
Troubleshoot snippets
- stage : deploy displayName: deploy dependsOn : plan jobs : - deployment : deploy displayName: deploy environment: env-${{ parameters.environment }} strategy: runOnce: deploy: steps: - checkout: self # this is needed for job 'deployment' type, common job type 'job' already contains git code - task : AmazonWebServices.aws-vsts-tools.AWSShellScript.AWSShellScript@1 displayName: deploy env : VARIBLE_1 : $(variable_2) inputs : awsCredentials : $(service_connection_name) regionName : $(aws_region) failOnStandardError : false args : "" scriptType : inline #disableAutoCwd : true #workingDirectory : $(Agent.BuildDirectory)/myrepo inlineScript : | #!/bin/bash sudo apt update -qq sudo apt install -qq tree printf '\n[DEBUG] $SHELL : %s' "$SHELL" printf '\n[DEBUG] $BASH_VERSION: %s' "$BASH_VERSION" printf '\n[DEBUG] $PDW : %s' "$PWD" printf '\n[DEBUG] $ENVIRONMENT=%s\n' "${ENVIRONMENT}" printf '\n[DEBUG] $..(..Agent.BuildDirectory): %s' "$(Agent.BuildDirectory)" printf '\n[DEBUG] pwd; ls -la\n' ; pwd; ls -la printf '\n[DEBUG] ls -la ../helm-charts\n' ; ls -la ../helm-charts printf '\n[DEBUG] ls -la ../s\n' ; ls -la ../s printf '\n[DEBUG] find ../.. -iname file.sh\n' ; find ../.. -iname file.sh printf '\n[DEBUG] tree -aL 2 ..\n' ; tree -aL 2 .. time source $(Agent.BuildDirectory)/myrepo/scripts/file.sh #time source $(Build.SourcesDirectory)/scripts/file.sh
Pipeline variables and secrets
Variables.md are available in expressions as well as scripts; see variables to learn more about how to use them. There are some predefined build and release variables you can also rely on. Syntax
regionName : "${{ variables.aws_region }}" regionName : $(aws_region) # set Dynamic Variable Secret $token = curl ... echo "##vso[task.setvariable variable=accesstoken;isSecret=true]$token" # print secrets, the value vertically if you print them as chars: $Password.ToCharArray()
Resources:
Pipeline tasks
- aws-toolkit-azure-devops set of tasks
- azure-pipelines-tasks by Microsoft
Stage
<syntaxhighlightjs lang=yaml> stages:
- stage : deploy_helm displayName: deploy_helm jobs : - job : deploy_helm displayName: deploy_helm steps : - task : AmazonWebServices.aws-vsts-tools.AWSShellScript.AWSShellScript@1 displayName: "deploy_helm" env : DBPASSWORD : $(auth_service_client_secret) VALUES_PATH: $(values-path) inputs : # https://github.com/aws/aws-toolkit-azure-devops/blob/master/Tasks/AWSShellScript/task.json awsCredentials: $(aws_creds) regionName : $(aws_region) #failOnStandardError: true # default: false #disableAutoCwd : true # default: false, The default behavior is to set the working directory to the script location #workingDirectory : $(Build.SourcesDirectory) # | This enables you to optionally specify a different working directory. scriptType : inline # default: filePath #filePath : $(Build.SourcesDirectory)/scripts/connect.sh inlineScript : | #!/bin/bash printf "\n[DEBUG] PWD: %s" "$PWD" time source $(Build.SourcesDirectory)/scripts/connect.sh || \ { printf "\n[ERROR] to connect"; exit 1; } args : "" # Arguments passed to the shell script
</syntaxhighlightjs>
Conditions
parameters: - name: enabled type: boolean default: false stages: - stage: ${{ parameters.stage_name_prefix }}_build condition: and(succeeded(), eq('${{ parameters.enabled }}', 'true')) displayName: ${{parameters.stage_display_name}} jobs:
Logging and formatting in the pipeline
##[group]Beginning of a group ##[warning]Warning message ##[error]Error message ##[debug]Debug text ##[command]Command-line being run ##[endgroup]
Resources
- azuredevopsdemogenerator
- #3 Azure DevOps w bazach danych BitPeak, youtube
- azure-devops-tips-each-loops-c082c692d025 medium 2019
- Code Assessment Tools
- Security Tools
- SonarCloud
- SonarAnalyzer
- FxCop - check your code for security, performance, and design issues by Microsoft