Difference between revisions of "Jenkins/Job DSL, Pipelines, JaaC"
(23 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
= Jenkins DSL = | = Jenkins DSL = | ||
== Gradle == | == Build Jenkins DSL job from a command line == | ||
=== Gradle === | |||
Use to locally build your Jenkins DSL jobs before pushing to SCM. | Use to locally build your Jenkins DSL jobs before pushing to SCM. | ||
Line 30: | Line 31: | ||
</source> | </source> | ||
== Gradle wrapper == | === Gradle wrapper === | ||
The recommended way to execute any Gradle build is with the help of the Gradle Wrapper. The Wrapper is a script that invokes a declared version of Gradle, downloading it beforehand if necessary. It's one time operation. | The recommended way to execute any Gradle build is with the help of the Gradle Wrapper. The Wrapper is a script that invokes a declared version of Gradle, downloading it beforehand if necessary. It's one time operation. | ||
<source lang="bash"> | <source lang="bash"> | ||
Line 40: | Line 41: | ||
</source> | </source> | ||
=== Test Jenkins DSL jobs === | |||
<source> | |||
git clone git@github.com:sheehan/job-dsl-gradle-example.git | |||
. | |||
├── src | |||
│ ├── jobs # DSL script files | |||
│ ├── main | |||
│ │ ├── groovy # support classes | |||
│ │ └── resources | |||
│ │ └── idea.gdsl # IDE support for IDEA | |||
│ ├── scripts # scripts to use with "readFileFromWorkspace" | |||
│ └── test | |||
│ └── groovy # specs | |||
└── build.gradle # build file | |||
./gradlew test #uses Jenkins tests harness to test all .groove files in src/jobs/* | |||
</source> | |||
=== Create Jenkins DSL job from a command line === | |||
Rest APIs won't work with CSRF enabled, so to disable go to "Manage Jenkins" > "Configure Global Security" and select "Prevent Cross Site Request Forgery exploits.". | |||
<source lang="bash"> | |||
git clone git@github.com:sheehan/job-dsl-rest-example.git | |||
./gradlew rest -Dpattern=src/jobs/example1Jobs.groovy -DbaseUrl=http://our-jenkins/ -Dusername=foo -Dpassword=bar | |||
./gradlew rest -Dpattern=<pattern> -DbaseUrl=<baseUrl> [-Dusername=<username>] [-Dpassword=<password>] | |||
# pattern - ant-style path pattern of files to include. E.g. src/jobs/*.groovy | |||
# baseUrl - base URL of Jenkins server | |||
# username - Jenkins username, if secured | |||
# password - Jenkins password or token, if secured | |||
</source> | |||
=== References === | === References === | ||
*[https://marcesher.com/2016/07/01/jenkins-as-code-creating-jobs-from-the-command-line-during-development/ creating jobs from the command line] at [https://marcesher.com/category/work/ marcesher.com] | *[https://marcesher.com/2016/07/01/jenkins-as-code-creating-jobs-from-the-command-line-during-development/ creating jobs from the command line] at [https://marcesher.com/category/work/ marcesher.com] | ||
*[https://docs.gradle.org/4.7/userguide/gradle_wrapper.html The Gradle Wrapper] | *[https://docs.gradle.org/4.7/userguide/gradle_wrapper.html The Gradle Wrapper] | ||
*[https://github.com/sheehan/job-dsl-gradle-example Jenkins DSL Examples] Official sheehan gitrepo, includes tests | |||
*[https://github.com/jenkinsci/jenkins-test-harness jenkins-test-harness] Official jenkinsci gitrepo | |||
*[https://github.com/unguiculus/job-dsl-sample job-dsl-sample] Git repo Jenkins Job DSL setup with Gradle | |||
These examples have been put when Job DSL Plugin was at 1.68, 1.69 release version. | These examples have been put when Job DSL Plugin was at 1.68, 1.69 release version. | ||
Line 77: | Line 112: | ||
#we are using maven to compile the code as demonstration | #we are using maven to compile the code as demonstration | ||
|} | |} | ||
== Example - many jobs using a loop == | |||
Paste the snippet below into ''Build step > Process Job DSL > Use the provided DSL script'' | |||
<source lang="js"> | |||
100.times { | |||
job ('example' + it) {} | |||
} | |||
</source> | |||
Remember to set also: | |||
*Action for removed jobs: Delete | |||
*Action for removed views: Delete | |||
So, renamed jobs, deleted jobs will get removed from the jobs list. | |||
== Example - shell in line == | == Example - shell in line == | ||
Line 183: | Line 230: | ||
set -o pipefail | set -o pipefail | ||
# do stuff | # do stuff | ||
""".trim() | """.stripIndent().trim() | ||
// .trim() is actually enough but removing indents improves readability | |||
</source> | </source> | ||
Line 214: | Line 262: | ||
} | } | ||
</source> | </source> | ||
== Conditional step == | |||
<source lang="js"> | |||
steps { | |||
shell { command(shSupportScript } | |||
singleConditionalBuilder { | |||
condition { | |||
booleanCondition { | |||
token('${SAVE_TO_S3}') | |||
} | |||
runner { dontRun() } | |||
buildStep { | |||
shell { command(s3sync) } | |||
} | |||
} | |||
} | |||
} | |||
</source> | |||
== Variables in Groovy == | |||
<source lang="js"> | |||
// create a string variable | |||
def jobName = "Example_job_name" | |||
// reference the var in 2 ways | |||
job( "perf_" + jobName ) | |||
job("perf_${jobName}") { ... } | |||
</source> | |||
== Example: Create a custom view using configure block == | |||
<source lang="js"> | |||
def viewConfig = [ | |||
['viewName':'All', 'viewRegex': /(.*)/ ], | |||
['viewName':'SystemA', 'viewRegex': /(.*SystemA.*)/ ], | |||
['viewName':'PerfTest', 'viewRegex': /(perf_.*)/ ], | |||
['viewName':'SysAdmin', 'viewRegex': /(.*SysAdmin.*|AWS_.*|DSL_.*)/ ] | |||
] | |||
viewConfig.each { | |||
def viewName = it.viewName | |||
def viewRegex = it.viewRegex | |||
def columnJobDescription = { | |||
{ node -> | |||
node / "columns" << "jenkins.plugins.extracolumns.DescriptionColumn" { | |||
trim true | |||
columnWidth 20 | |||
displayLength 1 | |||
forceWidth false | |||
displayName false | |||
} | |||
} | |||
} | |||
listView(viewName) { | |||
jobs { | |||
regex(viewRegex) | |||
} | |||
columns { | |||
status() | |||
weather() | |||
lastBuildConsole() | |||
buildButton() | |||
jobNameColorColumn { | |||
colorblindHint('nohint') | |||
showColor(true) | |||
showDescription(true) | |||
showLastBuild(true) | |||
} | |||
progressBar() | |||
allStatusesColumn { | |||
colorblindHint('nohint') | |||
onlyShowLastStatus(false) | |||
timeAgoTypeString('DIFF') | |||
hideDays(1) | |||
} | |||
lastDuration() | |||
if (viewName == "PerfTest") { | |||
configure columnJobDescription() | |||
} | |||
} | |||
} | |||
} | |||
</source> | |||
Note of use of <tt>configure</tt> block, is to produce XML <code><jenkins.plugins.extracolumns.DescriptionColumn> ... </jenkins.plugins.extracolumns.DescriptionColumn></code> as native DSL does not support fully. The only thing cannot be controlled is the order of child element (here first, not always what we want). This is due to <code>node / "columns"</code> will find the <code>columns</code> node, creating it if it doesn't exist. If attributes are specified, it will find the first child which carries those attributes. It has a very low precedence in the order of operation, so you need to wrap parenthesis around some operations. | |||
Below you can see XML for '''PerfTest''' view | |||
<source lang="xml" line> | |||
<hudson.model.ListView> | |||
<link type="text/css" id="dark-mode" rel="stylesheet" /> | |||
<style type="text/css" id="dark-mode-custom-style" /> | |||
<name>PerfTest</name> | |||
<filterExecutors>false</filterExecutors> | |||
<filterQueue>false</filterQueue> | |||
<properties class="hudson.model.View$PropertyList" /> | |||
<jobNames> | |||
<comparator class="hudson.util.CaseInsensitiveComparator" /> | |||
</jobNames> | |||
<jobFilters /> | |||
<columns> | |||
<jenkins.plugins.extracolumns.DescriptionColumn plugin="extra-columns@1.18"> | |||
<displayName>false</displayName> | |||
<trim>true</trim> | |||
<displayLength>1</displayLength> | |||
<columnWidth>20</columnWidth> | |||
<forceWidth>false</forceWidth> | |||
</jenkins.plugins.extracolumns.DescriptionColumn> | |||
<hudson.views.StatusColumn /> | |||
<hudson.views.WeatherColumn /> | |||
<jenkins.plugins.extracolumns.LastBuildConsoleColumn plugin="extra-columns@1.18" /> | |||
<hudson.views.BuildButtonColumn /> | |||
<com.robestone.hudson.compactcolumns.JobNameColorColumn plugin="compact-columns@1.10"> | |||
<colorblindHint>nohint</colorblindHint> | |||
<showColor>true</showColor> | |||
<showDescription>true</showDescription> | |||
<showLastBuild>true</showLastBuild> | |||
</com.robestone.hudson.compactcolumns.JobNameColorColumn> | |||
<org.jenkins.ci.plugins.progress__bar.ProgressBarColumn plugin="progress-bar-column-plugin@1.0" /> | |||
<com.robestone.hudson.compactcolumns.AllStatusesColumn plugin="compact-columns@1.10"> | |||
<colorblindHint>nohint</colorblindHint> | |||
<timeAgoTypeString>DIFF</timeAgoTypeString> | |||
<onlyShowLastStatus>false</onlyShowLastStatus> | |||
<hideDays>1</hideDays> | |||
</com.robestone.hudson.compactcolumns.AllStatusesColumn> | |||
<hudson.views.LastDurationColumn /> | |||
</columns> | |||
<includeRegex>(perf_.*)</includeRegex> | |||
<recurse>false</recurse> | |||
</hudson.model.ListView> | |||
</source> | |||
=== References === | |||
*[https://github.com/jenkinsci/job-dsl-plugin/wiki/The-Configure-Block The-Configure-Block] github.com/jenkinsci/job-dsl-plugin | |||
*[http://nsbogan.com/mobile%20ci/2015/05/30/jenkins-job-dsl-configure-block Jenkins Job DSL - Configure Block] Blog 2015 | |||
== Example: Build Pipeline job == | == Example: Build Pipeline job == | ||
Jenkins DSL can build Jenkins Pipeline jobs as well. | |||
<source lang="js"> | <source lang="js"> | ||
pipelineJob("DSL_Pipeline_calls_other_pipeline") { | pipelineJob("DSL_Pipeline_calls_other_pipeline") { | ||
Line 312: | Line 493: | ||
= Jenkins Pipeline = | = Jenkins Pipeline = | ||
== Generic structure == | == Generic structure == | ||
<source lang=" | <source lang="java" line> | ||
pipeline { // | pipeline { //The main Pipeline directive | ||
agent any // | agent any //global agent (minion/slave) directive | ||
environment { //environment directive, sets environment | environment { //the environment directive, sets environment variables in global scope | ||
ENV_VAR = "value" | ENV_VAR = "value" | ||
} | } | ||
stages { // | parameters { //the Parameters directive, currently only allows for string and boolean parameters | ||
stage('Build') { //name of | string( name: 'PERSON', defaultValue: 'Mr Awesome', description: 'Who is the best?') | ||
booleanParam(name: 'IS_JENKINS_AWESOME', defaultValue: true, description: "Jenkins Awesome?" ) | |||
} | |||
triggers { //the Triggers directive | |||
cron('H * 0 0 1-5') | |||
} | |||
stages { //the Stages directive, this'd be analogous to a "Build Step" in the classic Project Configuration view | |||
stage('Build') { //the Stage directive, name of the stage | |||
steps { //it's many steps, often associated with plugins | steps { //it's many steps, often associated with plugins | ||
echo 'Building...' //prints a string | echo 'Building...' //prints a string | ||
} | } | ||
} | } | ||
stage('Test') { | stage('Test') { //the Stage directive | ||
steps { | steps { | ||
echo 'Testing...' | echo 'Testing...' | ||
Line 332: | Line 520: | ||
} | } | ||
} | } | ||
stage('Deploy') { | stage('Deploy') { //the Stage directive | ||
steps { | steps { | ||
echo 'Deploying...' | echo 'Deploying...' | ||
} | } | ||
} | |||
} | |||
post { //the Post directive, contains "Post-build" steps | |||
success { | |||
emailext( | |||
subject: "${env.JOB_NAME} [${env.BUILD_NUMBER}] Ran!", | |||
body: """ | |||
'${env.JOB_NAME} [${env.BUILD_NUMBER}]' Ran!": Check console output at ${env.JOB_NAME} [${env.BUILD_NUMBER}]/a> """, | |||
to: "your@email.com" ) | |||
} | } | ||
} | } | ||
Line 360: | Line 557: | ||
*[https://github.com/linuxacademy/content-jenkins-java-project/blob/master/Jenkinsfile Jenkins file] Linux Academy example | *[https://github.com/linuxacademy/content-jenkins-java-project/blob/master/Jenkinsfile Jenkins file] Linux Academy example | ||
= Proxies = | |||
== Gradle == | |||
<source> | |||
systemProp.http.proxyHost=172.31.101.100 | |||
systemProp.http.proxyPort=3128 | |||
systemProp.http.nonProxyHosts=*.nonproxyrepos.com|localhost|10.0.150.2|*.localdomain.local|127.0.0.0/8|10.0.0.0/8|172.31.0.0/16|*.local|172.16.0.0/12|192.168.0.0/16|10.6.0.15|*.aws.company.* | |||
systemProp.https.proxyHost=172.31.101.100 | |||
systemProp.https.proxyPort=3128 | |||
systemProp.https.nonProxyHosts=*.nonproxyrepos.com|localhost|10.0.150.2|*.localdomain.local|127.0.0.0/8|10.0.0.0/8|172.31.0.0/16|*.local|172.16.0.0/12|192.168.0.0/16|10.6.0.15|*.aws.company.* | |||
org.gradle.daemon=true | |||
</source> | |||
== Maven == | |||
<source> | |||
<proxy> | |||
<id>optional</id> | |||
<active>true</active> | |||
<protocol>http</protocol> | |||
<!-- <username>proxyuser</username> | |||
<password>proxypass</password> --> | |||
<host>localhost</host> | |||
<port>3128</port> | |||
<nonProxyHosts>local.net|some.host.com</nonProxyHosts> | |||
</proxy> | |||
</source> | |||
= References = | = References = | ||
* http://www.groovy-lang.org/closures.html | * http://www.groovy-lang.org/closures.html | ||
* https://github.com/jenkinsci/job-dsl-plugin/wiki/Job-DSL-Commands | * https://github.com/jenkinsci/job-dsl-plugin/wiki/Job-DSL-Commands | ||
* [http://codeventor.blogspot.co.uk/2015/06/jenkins-dsl-scripting-part-1-basics-tldr.html DSL Scripting with examples] Blogspot, conventor | * [http://codeventor.blogspot.co.uk/2015/06/jenkins-dsl-scripting-part-1-basics-tldr.html DSL Scripting with examples] Blogspot, conventor |
Revision as of 09:46, 19 July 2018
Jenkins DSL
Build Jenkins DSL job from a command line
Gradle
Use to locally build your Jenkins DSL jobs before pushing to SCM.
- Install in Windows
- Create C:\Gradle
- Download [1] archive and unzip to folder above
- Add C:\Gradle\gradle-4.7\bin to %Path% in system variables
setx path "%path%;C:\Gradle\gradle-4.7\bin"
- Verify gradle -v
$ gradle -v ------------------------------------------------------------ Gradle 4.7 ------------------------------------------------------------ Build time: 2018-04-18 09:09:12 UTC Revision: b9a962bf70638332300e7f810689cb2febbd4a6c Groovy: 2.4.12 Ant: Apache Ant(TM) version 1.9.9 compiled on February 2 2017 JVM: 1.8.0_161 (Oracle Corporation 25.161-b12) OS: Windows 10 10.0 x86
- Install in Linux
Note Ubuntu 16.04 by default would install via apt-get version 2.10-1.
wget https://services.gradle.org/distributions/gradle-4.7-bin.zip mkdir /opt/gradle && unzip -d /opt/gradle gradle-4.7-bin.zip export PATH=$PATH:/opt/gradle/gradle-4.7/bin
Gradle wrapper
The recommended way to execute any Gradle build is with the help of the Gradle Wrapper. The Wrapper is a script that invokes a declared version of Gradle, downloading it beforehand if necessary. It's one time operation.
piotr@ubuntu ~ $ gradle wrapper Starting a Gradle Daemon (subsequent builds will be faster) BUILD SUCCESSFUL in 12s 1 actionable task: 1 executed
Test Jenkins DSL jobs
git clone git@github.com:sheehan/job-dsl-gradle-example.git . ├── src │ ├── jobs # DSL script files │ ├── main │ │ ├── groovy # support classes │ │ └── resources │ │ └── idea.gdsl # IDE support for IDEA │ ├── scripts # scripts to use with "readFileFromWorkspace" │ └── test │ └── groovy # specs └── build.gradle # build file ./gradlew test #uses Jenkins tests harness to test all .groove files in src/jobs/*
Create Jenkins DSL job from a command line
Rest APIs won't work with CSRF enabled, so to disable go to "Manage Jenkins" > "Configure Global Security" and select "Prevent Cross Site Request Forgery exploits.".
git clone git@github.com:sheehan/job-dsl-rest-example.git ./gradlew rest -Dpattern=src/jobs/example1Jobs.groovy -DbaseUrl=http://our-jenkins/ -Dusername=foo -Dpassword=bar ./gradlew rest -Dpattern=<pattern> -DbaseUrl=<baseUrl> [-Dusername=<username>] [-Dpassword=<password>] # pattern - ant-style path pattern of files to include. E.g. src/jobs/*.groovy # baseUrl - base URL of Jenkins server # username - Jenkins username, if secured # password - Jenkins password or token, if secured
References
- creating jobs from the command line at marcesher.com
- The Gradle Wrapper
- Jenkins DSL Examples Official sheehan gitrepo, includes tests
- jenkins-test-harness Official jenkinsci gitrepo
- job-dsl-sample Git repo Jenkins Job DSL setup with Gradle
These examples have been put when Job DSL Plugin was at 1.68, 1.69 release version.
Example - explained line by line
job("Demo build job") { scm { git { remote { url 'https://github.com/lexandro/restapi-demo.git' } branch 'master' shallowClone true } } steps { maven('compile') } } |
|
Example - many jobs using a loop
Paste the snippet below into Build step > Process Job DSL > Use the provided DSL script
100.times { job ('example' + it) {} }
Remember to set also:
- Action for removed jobs: Delete
- Action for removed views: Delete
So, renamed jobs, deleted jobs will get removed from the jobs list.
Example - shell in line
hudson.FilePath workspace = hudson.model.Executor.currentExecutor().getCurrentWorkspace() String scriptSH1 = workspace.list("test-certs.sh")[0].read().getText() job("DSL_Inline_Certs_expiry_test_shell_inline") { description("Creates Certs_expiry_test job" authorization { blocksInheritance(true) permission('hudson.model.Item.Read:anonymous') permission('hudson.model.Item.Discover:anonymous') permissionAll('authenticated') } logRotator { daysToKeep(-1) numToKeep(10) artifactDaysToKeep(-1) artifactNumToKeep(-1) } wrappers { colorizeOutput() maskPasswords() preBuildCleanup() timestamps() buildNameSetter { template('#${BUILD_NUMBER} ${CHANNEL} ${ENV}') runAtStart(true) runAtEnd(false) } } publishers { wsCleanup { deleteDirectories(true) setFailBuild(false) cleanWhenSuccess(true) cleanWhenUnstable(false) cleanWhenFailure(false) cleanWhenNotBuilt(true) cleanWhenAborted(true) } } multiscm { git { remote { url("git@gitlab.com:pio2pio/dsl-jenkins.git") credentials("123abc12-1234-1234-1234-abc123abc123") branches('*/master') } extensions { relativeTargetDirectory("secrets-non-prod") } } git { remote { url("git@gitlab.com:pio2pio/dsl-jenkins.git") credentials("123abc12-1234-1234-1234-abc123abc123") branches('*/master') } extensions { relativeTargetDirectory("secrets-prod") } } } steps { shell { // command(scriptSH1) command('''#!/bin/bash red_bold="\e[1;31m" green="\e[32m" green_bold="\e[1;32m" yellow_bold="\e[1;93m" blue_bold="\e[1;34m" reset="\e[0m" datediff() { d1=$(date -d "$1" +%s) d2=$(date -d "$2" +%s) echo $(( (d1 - d2) / 86400 )) } set -f paths=('secrets-non-prod/ssl/*crt' 'secrets-prod/ssl/*crt') today=$(date +"%Y%m%d") today30=$(date -d "+30 days" +"%Y%m%d") for path in ${paths[@]}; do set +f #enable fileglobbing echo "" echo -e "${blue_bold} Certificates in ${path} ${reset}" for i in $(ls -1 $path); do enddate=$(date --date="$(openssl x509 -in $i -noout -enddate | cut -d= -f 2)" --iso-8601) enddate_d=$(date -d $enddate +"%Y%m%d") if [ $today -lt $enddate_d ] && [ $today30 -gt $enddate_d ]; then colour="${yellow_bold} WARN" elif [ $today30 -lt $enddate_d ]; then colour="${green_bold} PASS" else colour="${red_bold} ERRO" fi echo -e "${colour} ${enddate} $(basename $i) DaysToExpire: $(datediff $enddate_d $today) ${reset}" done | sort -k3r | column -t set -f done''') } } }
Example multiline shell script takes interpreter from first non-blank line. A workaround of putting .trim() at the end of the triple-double quote does work.
sh """ #!/bin/bash -xel set -o pipefail # do stuff """.stripIndent().trim() // .trim() is actually enough but removing indents improves readability
Example - groovy variable substitution and for.each
def owner = 'integrations' def project = 'jenkins-dsl' def branchApi = new URL("https://api.github.com/repos/${owner}/${project}/branches") def branches = new groovy.json.JsonSlurper().parse(branchApi.newReader()) branches.each { def branchName = it.name def jobName = "${owner}-${project}-${branchName}".replaceAll('/','-') job(jobName) { scm { git { remote { github("${owner}/${project}") } branch("${branchName}") createTag(false) } } triggers { scm('*/15 * * * *') } steps { shell('ls -l') } } }
Conditional step
steps { shell { command(shSupportScript } singleConditionalBuilder { condition { booleanCondition { token('${SAVE_TO_S3}') } runner { dontRun() } buildStep { shell { command(s3sync) } } } } }
Variables in Groovy
// create a string variable def jobName = "Example_job_name" // reference the var in 2 ways job( "perf_" + jobName ) job("perf_${jobName}") { ... }
Example: Create a custom view using configure block
def viewConfig = [ ['viewName':'All', 'viewRegex': /(.*)/ ], ['viewName':'SystemA', 'viewRegex': /(.*SystemA.*)/ ], ['viewName':'PerfTest', 'viewRegex': /(perf_.*)/ ], ['viewName':'SysAdmin', 'viewRegex': /(.*SysAdmin.*|AWS_.*|DSL_.*)/ ] ] viewConfig.each { def viewName = it.viewName def viewRegex = it.viewRegex def columnJobDescription = { { node -> node / "columns" << "jenkins.plugins.extracolumns.DescriptionColumn" { trim true columnWidth 20 displayLength 1 forceWidth false displayName false } } } listView(viewName) { jobs { regex(viewRegex) } columns { status() weather() lastBuildConsole() buildButton() jobNameColorColumn { colorblindHint('nohint') showColor(true) showDescription(true) showLastBuild(true) } progressBar() allStatusesColumn { colorblindHint('nohint') onlyShowLastStatus(false) timeAgoTypeString('DIFF') hideDays(1) } lastDuration() if (viewName == "PerfTest") { configure columnJobDescription() } } } }
Note of use of configure block, is to produce XML <jenkins.plugins.extracolumns.DescriptionColumn> ... </jenkins.plugins.extracolumns.DescriptionColumn>
as native DSL does not support fully. The only thing cannot be controlled is the order of child element (here first, not always what we want). This is due to node / "columns"
will find the columns
node, creating it if it doesn't exist. If attributes are specified, it will find the first child which carries those attributes. It has a very low precedence in the order of operation, so you need to wrap parenthesis around some operations.
Below you can see XML for PerfTest view
<hudson.model.ListView> <link type="text/css" id="dark-mode" rel="stylesheet" /> <style type="text/css" id="dark-mode-custom-style" /> <name>PerfTest</name> <filterExecutors>false</filterExecutors> <filterQueue>false</filterQueue> <properties class="hudson.model.View$PropertyList" /> <jobNames> <comparator class="hudson.util.CaseInsensitiveComparator" /> </jobNames> <jobFilters /> <columns> <jenkins.plugins.extracolumns.DescriptionColumn plugin="extra-columns@1.18"> <displayName>false</displayName> <trim>true</trim> <displayLength>1</displayLength> <columnWidth>20</columnWidth> <forceWidth>false</forceWidth> </jenkins.plugins.extracolumns.DescriptionColumn> <hudson.views.StatusColumn /> <hudson.views.WeatherColumn /> <jenkins.plugins.extracolumns.LastBuildConsoleColumn plugin="extra-columns@1.18" /> <hudson.views.BuildButtonColumn /> <com.robestone.hudson.compactcolumns.JobNameColorColumn plugin="compact-columns@1.10"> <colorblindHint>nohint</colorblindHint> <showColor>true</showColor> <showDescription>true</showDescription> <showLastBuild>true</showLastBuild> </com.robestone.hudson.compactcolumns.JobNameColorColumn> <org.jenkins.ci.plugins.progress__bar.ProgressBarColumn plugin="progress-bar-column-plugin@1.0" /> <com.robestone.hudson.compactcolumns.AllStatusesColumn plugin="compact-columns@1.10"> <colorblindHint>nohint</colorblindHint> <timeAgoTypeString>DIFF</timeAgoTypeString> <onlyShowLastStatus>false</onlyShowLastStatus> <hideDays>1</hideDays> </com.robestone.hudson.compactcolumns.AllStatusesColumn> <hudson.views.LastDurationColumn /> </columns> <includeRegex>(perf_.*)</includeRegex> <recurse>false</recurse> </hudson.model.ListView>
References
- The-Configure-Block github.com/jenkinsci/job-dsl-plugin
- Jenkins Job DSL - Configure Block Blog 2015
Example: Build Pipeline job
Jenkins DSL can build Jenkins Pipeline jobs as well.
pipelineJob("DSL_Pipeline_calls_other_pipeline") { def repo = 'https://github.com/user/yourApp.git' //set variables def repoSsh = 'git@git.company.com:user/yourApp.git' description("Your App Pipeline") properties { githubProjectUrl (repo) rebuild { autoRebuild(false) } } logRotator { numToKeep 30 } definition { cps { sandbox() script(""" node { stage 'Build' echo 'Compiling code...' stage "Test" build 'pipeline-or-free-style-being-called' //any Project can be called stage 'Deploy' echo "Deploying..." } """.stripIndent()) } } }
Job running JMeter performance test publishing/consuming JMS messages
... wip ...
This will build a job that runs JMeter test publishing and consuming messages directly to Wso2 Message Broker.
Scope:
- build parameterized Jenkins DSL job (optional: from Git repo)
- messages sent
- messages consumed
- pull mb docker container
- [done]pull JMeter
- run sample test against the docker container
- pull jmeter test
- prep jndi.properties connection factory
- [done]publish results to s3
Script to pull Jmeter
#!/bin/bash -e if [ ${JMETER_INIT} == 'true' ]; then if [ ! -f jmeter/bin/jmeter ]; then wget https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-4.0.zip unzip apache-jmeter-4.0.zip ln -s apache-jmeter-4.0 jmeter fi JMETER_LIBS_EXT="http://artifactory.com:8080/artifactory/performance/jmeter/client-libs/" wget -r -l1 -np -nH -R "index.html*" ${JMETER_LIBS_EXT} -A "*.jar" --cut-dirs=3 mv client-libs/*.jar jmeter/lib/ext/ fi if [ -f aggResults.jtl ]; then rm aggResults.jtl fi mkdir -p reports/${BUILD_ID} jmeter/bin/jmeter \ -Juser.jndipath=/tank/jenkins/workspace/perf_mb-direct/performance/jndi.properties \ -Juser.producerLoops=${Juser_producerLoops} \ -Juser.senderLoops=${Juser_producerLoops} \ -n -t performance/aggResults.jmx \ -l aggResults.jtl \ -e -o reports/${BUILD_ID} aws s3 sync reports/${BUILD_ID} s3://bucket-name-public/reports/${BUILD_ID} --quiet echo "Report for build ${BUILD_ID}: https://s3-eu-west-1.amazonaws.com/reports/${BUILD_ID}/index.html" echo "Aggregated results: http://bucket-name-public.s3-website-eu-west-1.amazonaws.com/"
References
- Apache JMeter internal link reference
- wso2-mb Github docker file
- wso2-mb Github docker file; simpler
- DSL Plugin Deprecations Github jenkinsci/job-dsl-plugin
- Jenkins + Groovy with the Job DSL Plugin Youtube official plugin video
Jenkins Pipeline
Generic structure
pipeline { //The main Pipeline directive agent any //global agent (minion/slave) directive environment { //the environment directive, sets environment variables in global scope ENV_VAR = "value" } parameters { //the Parameters directive, currently only allows for string and boolean parameters string( name: 'PERSON', defaultValue: 'Mr Awesome', description: 'Who is the best?') booleanParam(name: 'IS_JENKINS_AWESOME', defaultValue: true, description: "Jenkins Awesome?" ) } triggers { //the Triggers directive cron('H * 0 0 1-5') } stages { //the Stages directive, this'd be analogous to a "Build Step" in the classic Project Configuration view stage('Build') { //the Stage directive, name of the stage steps { //it's many steps, often associated with plugins echo 'Building...' //prints a string } } stage('Test') { //the Stage directive steps { echo 'Testing...' sh 'printenv' sh 'ant -f test.xml -v' junit "reports/${env.BUILD_NUMBER}_result.xml" } } stage('Deploy') { //the Stage directive steps { echo 'Deploying...' } } } post { //the Post directive, contains "Post-build" steps success { emailext( subject: "${env.JOB_NAME} [${env.BUILD_NUMBER}] Ran!", body: """ '${env.JOB_NAME} [${env.BUILD_NUMBER}]' Ran!": Check console output at ${env.JOB_NAME} [${env.BUILD_NUMBER}]/a> """, to: "your@email.com" ) } } }
Agent directive
pipeline { agent any //declared globally ... agent none ... agent { label 'minion-1' //can be declared within a single stage } ... agent { docker 'openjdk:8u121-jre' }
References
- Jenkins file Linux Academy example
Proxies
Gradle
systemProp.http.proxyHost=172.31.101.100 systemProp.http.proxyPort=3128 systemProp.http.nonProxyHosts=*.nonproxyrepos.com|localhost|10.0.150.2|*.localdomain.local|127.0.0.0/8|10.0.0.0/8|172.31.0.0/16|*.local|172.16.0.0/12|192.168.0.0/16|10.6.0.15|*.aws.company.* systemProp.https.proxyHost=172.31.101.100 systemProp.https.proxyPort=3128 systemProp.https.nonProxyHosts=*.nonproxyrepos.com|localhost|10.0.150.2|*.localdomain.local|127.0.0.0/8|10.0.0.0/8|172.31.0.0/16|*.local|172.16.0.0/12|192.168.0.0/16|10.6.0.15|*.aws.company.* org.gradle.daemon=true
Maven
<proxy> <id>optional</id> <active>true</active> <protocol>http</protocol> <!-- <username>proxyuser</username> <password>proxypass</password> --> <host>localhost</host> <port>3128</port> <nonProxyHosts>local.net|some.host.com</nonProxyHosts> </proxy>