Difference between revisions of "Jenkins/Job DSL, Pipelines, JaaC"

From Ever changing code
Jump to navigation Jump to search
m (Pio2pio moved page Jenkins DSL to Jenkins DSL and Pipeline without leaving a redirect)
(No difference)

Revision as of 14:37, 27 May 2018

Jenkins DSL

Example - explained line by line

job("Demo build job") {
 scm {
  git {
   remote {
   url 'https://github.com/lexandro/restapi-demo.git' 
  }
  branch 'master'
  shallowClone true
  }
 }
   steps {
  maven('compile')
 }
}
  1. In this line we are defining the closure as job and specifying the name. The name is mandatory!
  2. Here's the definition start for version controller configuration
  3. We are using git.
  4. Defining the remote endpoint for git.
  5. Specifying the URL of the repository
  6. Closing the block
  7. We are operation only on the master branch at the moment
  8. To speed up the checkout process we are using shallow cloning.
  9. closing block
  10. the definition of the checkout part is done
  11. Start the definition of build steps block
  12. we are using maven to compile the code as demonstration

Example - shell in line

hudson.FilePath workspace = hudson.model.Executor.currentExecutor().getCurrentWorkspace()
String scriptSH1 = workspace.list("test-certs.sh")[0].read().getText()

job("DSL_Inline_Certs_expiry_test_shell_inline") {
  description("Creates Certs_expiry_test job"
  authorization {
    blocksInheritance(true)
    permission('hudson.model.Item.Read:anonymous')
    permission('hudson.model.Item.Discover:anonymous')
    permissionAll('authenticated')
  }
  logRotator {
    daysToKeep(-1)
    numToKeep(10)
    artifactDaysToKeep(-1)
    artifactNumToKeep(-1)
  }
  wrappers {
    colorizeOutput()
    maskPasswords()
    preBuildCleanup()
    timestamps()
    buildNameSetter {
      template('#${BUILD_NUMBER} ${CHANNEL} ${ENV}')
      runAtStart(true)
      runAtEnd(false)
    }
  }  
  publishers {
    wsCleanup {
      deleteDirectories(true)
      setFailBuild(false)
      cleanWhenSuccess(true)
      cleanWhenUnstable(false)
      cleanWhenFailure(false)
      cleanWhenNotBuilt(true)
      cleanWhenAborted(true)
    }  
  }
  multiscm {
    git { remote {
            url("git@gitlab.com:pio2pio/dsl-jenkins.git")
            credentials("123abc12-1234-1234-1234-abc123abc123")
            branches('*/master') }
          extensions { relativeTargetDirectory("secrets-non-prod") }
    }     
    git { remote {
            url("git@gitlab.com:pio2pio/dsl-jenkins.git")
            credentials("123abc12-1234-1234-1234-abc123abc123")
            branches('*/master') } 
          extensions { relativeTargetDirectory("secrets-prod") }
    }
}
  steps {
    shell {
//    command(scriptSH1)
      command('''#!/bin/bash
red_bold="\e[1;31m"
green="\e[32m"
green_bold="\e[1;32m"
yellow_bold="\e[1;93m"
blue_bold="\e[1;34m"
reset="\e[0m"

datediff() {
    d1=$(date -d "$1" +%s)
    d2=$(date -d "$2" +%s)
    echo $(( (d1 - d2) / 86400 )) 
}

set -f 
paths=('secrets-non-prod/ssl/*crt' 'secrets-prod/ssl/*crt')
today=$(date +"%Y%m%d")
today30=$(date -d "+30 days" +"%Y%m%d")
for path in ${paths[@]}; do
  set +f  #enable fileglobbing
  echo ""
  echo -e "${blue_bold} Certificates in ${path} ${reset}"
  for i in $(ls -1 $path); do
    enddate=$(date --date="$(openssl x509 -in $i -noout -enddate | cut -d= -f 2)" --iso-8601)
    enddate_d=$(date -d $enddate +"%Y%m%d")
    if  [ $today -lt $enddate_d ] && [ $today30 -gt $enddate_d ]; then
      colour="${yellow_bold} WARN"
    elif [ $today30 -lt $enddate_d ]; then
      colour="${green_bold} PASS"
    else
      colour="${red_bold} ERRO"
    fi
    echo -e "${colour} ${enddate} $(basename $i) DaysToExpire: $(datediff $enddate_d $today) ${reset}" 
  done | sort -k3r | column -t 
  set -f 
done''')
    }
  }
}

Example multiline shell script takes interpreter from first non-blank line. A workaround of putting .trim() at the end of the triple-double quote does work. 

sh """
  #!/bin/bash -xel
  set -o pipefail
  # do stuff
  """.trim()

Example - groovy variable substitution and for.each

def owner = 'integrations'
def project = 'jenkins-dsl'
def branchApi = new URL("https://api.github.com/repos/${owner}/${project}/branches")
def branches = new groovy.json.JsonSlurper().parse(branchApi.newReader())
branches.each {
  def branchName = it.name
  def jobName = "${owner}-${project}-${branchName}".replaceAll('/','-')
  job(jobName) {
    scm {
        git {
            remote {
              github("${owner}/${project}")
            }
            branch("${branchName}")
            createTag(false)
        }
    }
    triggers {
        scm('*/15 * * * *')
    }
    steps {
        shell('ls -l')
    }
  }
}

Job running JMeter performance test publishing/consuming JMS messages

... wip ...

This will build a job that runs JMeter test publishing and consuming messages directly to Wso2 Message Broker.

Scope:

  • build parameterized Jenkins DSL job (optional: from Git repo)
    • messages sent
    • messages consumed
    • pull mb docker container
    • [done]pull JMeter
    • run sample test against the docker container
      • pull jmeter test
      • prep jndi.properties connection factory
    • [done]publish results to s3

Script to pull Jmeter 

#!/bin/bash -e

if [ ${JMETER_INIT} == 'true' ]; then

  if [ ! -f jmeter/bin/jmeter ]; then
    wget https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-4.0.zip
    unzip apache-jmeter-4.0.zip
    ln -s apache-jmeter-4.0 jmeter
  fi

  JMETER_LIBS_EXT="http://artifactory.com:8080/artifactory/performance/jmeter/client-libs/"
  wget -r -l1 -np -nH -R "index.html*" ${JMETER_LIBS_EXT} -A "*.jar" --cut-dirs=3
  mv client-libs/*.jar jmeter/lib/ext/

fi

if [ -f aggResults.jtl ]; then
  rm aggResults.jtl
fi

mkdir -p reports/${BUILD_ID}

jmeter/bin/jmeter \
  -Juser.jndipath=/tank/jenkins/workspace/perf_mb-direct/performance/jndi.properties \
  -Juser.producerLoops=${Juser_producerLoops} \
  -Juser.senderLoops=${Juser_producerLoops} \
  -n -t performance/aggResults.jmx \
  -l aggResults.jtl \
  -e -o reports/${BUILD_ID}
  
aws s3 sync reports/${BUILD_ID} s3://bucket-name-public/reports/${BUILD_ID} --quiet

echo "Report for build ${BUILD_ID}: https://s3-eu-west-1.amazonaws.com/reports/${BUILD_ID}/index.html"
echo "Aggregated results: http://bucket-name-public.s3-website-eu-west-1.amazonaws.com/"

References

Jenkins Pipeline

Generic structure

pipeline {              //has to be here
  agent any             // which agent to use/slave
  environment {         //environment directive, sets global environment variables
    ENV_VAR = "value"
  }
  stages {              //in freestyle project a stage it's a build          
    stage('Build') {    //name of a stage
      steps {           //it's many steps, often associated with plugins
        echo 'Building...'  //prints a string
      }
    }
    stage('Test') {
      steps {
        echo 'Testing...'
        sh 'printenv'
        sh 'ant -f test.xml -v'
        junit "reports/${env.BUILD_NUMBER}_result.xml"
      }
    }
    stage('Deploy') {
      steps { 
        echo 'Deploying...'
      }
    }
  }
}

Agent directive 

pipeline {
  agent any           //declared globally
...
  agent none
...
  agent {
    label 'minion-1'  //can be declared within a single stage
  }
...
  agent {
    docker 'openjdk:8u121-jre'
  }

References

References

Retrieved from ‘http://wiki.ciscolinux.co.uk/index.php?title=Jenkins/Job_DSL,_Pipelines,_JaaC&oldid=2740’