Install as standalone `.war` java deployment

Pre-requisites: Java 8

Install Jenkins on Ubuntu from repository

wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ /etc/apt/sources.list.d/jenkins.list' #add repo to sources.list
sudo apt-get update && sudo apt-get install jenkins

Install Jenkins on RedHat/CentOS/Fedora

wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
rpm --import http://pkg.jenkins.io/redhat-stable/jenkins.io.key
yum install -y jenkins-2.19.4-1.1
yum-config-manager --disable jenkins #disable source to prevent accidental update

Verify

cat /etc/passwd | grep jenkins #check if user has been created without shell
service jenkins status         #check if service is running

Run as a docker container

Install Docker

sudo docker pull jenkins:2.19.4
sudo mkdir /var/jenkins_home #a directory for the Jenkins container to share on the host
sudo docker run -d -u root -p 8080:8080 -p 50000:50000 \
  -v /var/jenkins_home:/var/jenkins_home jenkins:2.19.4
# -v maps the /var/jenkins_home directories between the two hosts

Generate SSH Keys for the Containerised Jenkins Master and get admin initial password

sudo docker exec -it <JENKINS_CONTAINTERID> bash
cd ~ && ssh-keygen -N '' && cat .ssh/id_rsa.pub
cat /var/lib/jenkins/secrets/initialAdminPassword

First run

Password preview sudo vi /var/lib/jenkins/secrets/initialAdminPassword
Url http://localhost:8080
Select plugins you wish to install using web based wizard
Once all downloaded, create your Admin user
Once completed you will see: "Jenkins is ready! Your Jenkins setup is complete.", press Start Jenkins

Proxy if needed

If you run on restricted port system you cannot control the box firewall you may want to proxy your connection to Jenkins port :8080. You can install nGinx to do this for you

sudo apt-get install nginx
sudo vi /etc/nginx/sites-enabled  #then enable proxy_pass
   location / {
      proxy_pass http://127.0.0.1:8080;
      }

Configure Master and Slave nodes

Master node

sudo hostnamectl set-hostname jenkins-master
sudo vi /etc/passwd                                             #make sure jenkins user have access to shell
sudo mkdir -p  /tank/jenkins; usermod -d /tank/jenkins jenkins  #update default jenkins HOME /var/lib/jenkins directory
sudo cp -r /etc/skel/. /tank/jenkins                            #copy .bashrc, .profile and other skeleton files
sudo passwd jenkins            #set a password
sudo su jenkins                #become ''jenkins'' user 
cd ~/.ssh && ssh-keygen        #generate ssh key pair
ssh-copy-id jenkins@localhost  #add public key to Jenkins (localhost) server, these might be minions workers
sudo visudo                    #allow jenkins sudo without password and disable requireTTY; makes easier to run builds
  jenkins ALL=(ALL)     NOPASSWD: ALL
  Defaults  !requiretty

The !requiretty will allow Jenkins to run bash scripts in a background otherwise you receive an error:

sudo: no tty present and no askpass program specified
Build step 'Execute shell' marked build as failure

Configure ssh keys and permissions

Required to ssh remote servers, execute bash shell scripts remotely, add slave nodes.

#generate a key pair for nologin jenkins user, while logged in as you
(master) $ sudo -u jenkins ssh-keygen -f /var/lib/jenkins/.ssh/id_rsa

Slave/minion/workers

There is no need to install anything apart allowing the master node to ssh to it

sudo hostnamectl set-hostname jenkins-minion
sudo mkdir -p  /tank/jenkins
sudo useradd jenkins -d /tank/jenkins #create jenkins user if does not exists, default HOME is /var/lib/jenkins
sudo usermod -d /tank/jenkins jenkins #or useradd -d /var/lib/jenkins jenkins #this is default JENKINS_HOME
sudo passwd jenkins  #set a password, may mach the master node one but not necessary
sudo visudo          #allow jenkins run sudo without password prompt; makes easier to run builds
  jenkins ALL=(ALL)     NOPASSWD: ALL

jenkins@master $ sudo -u jenkins ssh-copy-id -i /tank/jenkins/.ssh/id_rsa jenkins@slave #worked; default /var/lib/jenkins/.ssh/id_rsa 
jenkins@master $ ssh-copy-id jenkins@slave #allow password-less ssh to slave nodes, so master can send jobs to it
jenkins@slave  $ ssh-copy-id ubuntu@boxes  #allow password-less ssh to all other servers

If you have any plugins on Jenkins-master node that require external programs to run you should install these tools also on slaves. As all jobs are send to slaves these external programs should be there, most common are:

git
java - installed automatically but you need to have Oracle Java account credentials set up in the wide configuration settings. You can also install Java yourself.
maven
ansible
terraform

Jenkins Build-in CLI

Jenkins is listening on TCP port for JNLP agents. Download from http://jenkins-master:8080/cli, onto any server you wish to remotely control Jenkins

wget http://jenkins-master:8080/jnlpJars/jenkins-cli.jar

Running and connecting depends on a connection method you choose.

#username and password
java -jar jenkins-cli.jar -s http://jenkins-master:8080 list-jobs --username jenkins --password ***  
#ssh; the command is at the end, public key must be associated with Jenkins user at  http://yourserver.com/me/configure
java -jar jenkins-cli.jar -s http://jenkins-master:8080 -ssh -user piotr -i ~/.ssh/id_rsa install-plugin thinBackup
#api token method, example below displays console output
java -jar jenkins-cli.jar -s http://jenkins-master:8080 -auth user1:d4_apiToken_9c console "Job Name follow build number" 7

Run a parametrized job

Run a parametrized job, username position is required to be at the end; version verified here is 2.67

java -jar jenkins-cli.jar -s http://jenkins-master:8080 build 'Terraform' \
     -p ACTION=plan \
     -p LOCATION=dev11/midw/aat1 \
     -p BUILD_TAG_TERRAFORM=refs/heads/master \
     -p BUILD_TAG_SECRETS=refs/heads/master \
     --username user1 --password password1

For passing parameters use the -p key=value flag for each parameter you want to pass.

Workaround: Under "Authorization", you want to turn on "Matrix-based security" and then for the "anonymous" user enable "Read" mode for the "Job" section.

References: isignal.github.io

Remote access API

Jenkins provides machine-consumable remote access API to its functionalities. At the bottom right on each page you find REST API link describing all actions available on the resource page. The general rule is to add /api/ at the end of current url. There are additional information on the official page but worth to notice is to:

disable CSRF Protection or obtain the signature and send with each call
you can use HTTP BASIC authentication to authenticate remote API requests

Generic APIs documentation for a user is available at http://jenkins.com:8080/user/user1/api/

Example of a job APIs documentation: http://jenkins.com:8080/job/Freestyle_blabla/api/

Trigger a build:

curl -X POST http://jenkins.com:8080/job/Freestyle_blabla/build --user user1:d4***apiToken***cc

Retrieve a project config.xml file:

curl  http://jenkins.com:8080/job/Freestyle_blabla/config.xml --user user1:d4***apiToken***cc

Disable a project:

curl -X POST http://jenkins.com:8080/job/Freestyle_blabla/disable --user user1:d4***apiToken***cc

Enable a project:

curl -X POST http://jenkins.com:8080/job/Freestyle_blabla/enable --user user1:d4***apiToken***cc

If you have CSRF enabled, you should generate a crumb to authenticate with the Jenkins server and have $CRUMB available via Environment Variables, before attempting the other commands.

CRUMB=$(curl -s 'http://USER:PASS@<your-server>.com:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')

Delete a job

curl -H "$CRUMB" http://<your-server>.com:8080/job/Freestyles/job/My%20Freestyle%20Project/doDelete

Update description

curl -H "$CRUMB" http://<your-server>.com:8080/job/Freestyles/job/My%20Freestyle%20Project/description

Submit jobs using "Trigger builds remotely (e.g., from scripts)" option, here the custom token is build

curl -X POST -H "$CRUMB" http://<your-server>.com:8080/job/My%20Freestyle%20Project/build?token=build
curl -X POST -H "$CRUMB" http://<your-server>.com:8080/job/My%20Freestyle%20Project/build \
+-data token=build \
+-data-urlencode json='{"parameter": [{"name":"branch","value":"development"}]}'

Jenkins SSH

Jenkins acts as an SSH server, starting 1.446. This exposes a subset of Jenkins CLI commands (those that do not need any intelligence on the client side.) Plugins may also add additional capabilities through SSH server.

Jenkins SSH does not require any custom jar file on the client side, making it easier to access Jenkins from a variety of sources
Jenkins CLI client can be intelligent, performing file access and other interactive processing, whereas SSH client is dumb. Therefore, some commands can only run through Jenkins CLI and not via SSH.

REF: Jenkins SSH

Plugin manager - plugin specific version

You can download a specific version of plugin .hpi or .jpi file from Jenkins Plugins Archive page and by going to Settings > Plugins > Advanced > Upload Plugin > Browse...to .hpi/.jpi > Upload. That way you can install plugin version you want, it's opposite to upgrade/downgrade of single version only via GUI upgrade process.

Plugin: SSH Slaves plugin - adds slaves_nodes over ssh

This plugin allows you to manage slaves running on *nix machines over SSH. It adds a new type of slave launch method.

On the worker, create a workspace folder for jobs

$ mkdir ~/jenkins-minion_workspace

Go to Jenkins-master console and add this New Node

Go to http://jenkins-master:8080
Manage Jenkins > Manage Nodes > New Node with permanent agent and fill in with details

Add credentials, then select them on the main new node screen and save configuration.

If you have Java installed on the remote worker node, all you need to do is to Launch Agent. You should see that the worker is in service and it's time in sync

Plugin: Self-Organizing Swarm Plug-in Modules

This plugin enables slaves to auto-discover nearby Jenkins master and join it automatically, thereby forming an ad-hoc cluster. This plugin consists of two pieces:

self-contained CLI client that discovers a nearby Jenkins (via a UDP broadcast) and joins it
A plugin that needs to be installed on Jenkins master to accept swarm clients

Download a client

wget https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/swarm-client/3.3/swarm-client-3.3.jar

Run as anonymous user, in this case you need to enable Agent privileges on Jenkins-master node http://jenkins-master:8080/configureSecurity/ > Configure Global Security > Matrix-based security > all Agent permissions.

java -jar swarm-client-3.3.jar -name minion1 -master http://jenkins-master:8080/

Available options:

-name VAL :name of the slave
-username VAL :the Jenkins username for authentication
-password VAL :the Jenkins user password
-passwordEnvVariable VAL :environment variable that the password is stored in

When connected you should see on slave's

jenkins@jenkins-minion-2:~$ java -jar swarm-client-3.4a.jar -master http://jenkins-master:8080/ -name minion-2
Sep 16, 2017 6:25:03 PM hudson.plugins.swarm.Client main
INFO: Client.main invoked with: [-master http://jenkins-master:8080/ -name minion-2]
Sep 16, 2017 6:25:03 PM hudson.plugins.swarm.Client run
INFO: Discovering Jenkins master
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Sep 16, 2017 6:25:03 PM hudson.plugins.swarm.Client run
INFO: Attempting to connect to http://jenkins-master:8080/ 8d17dd57-bef9-4be9-9633-4f00dfe45ff2 with ID 16252dd0
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main createEngine
INFO: Setting up slave: minion-2-16252dd0
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://jenkins-master:8080/]
Sep 16, 2017 6:25:04 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, CLI2-connect, JNLP-connect, Ping, CLI-connect, JNLP2-connect]
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
  Agent address: jenkins-master
  Agent port:    40843
  Identity:      6c:9c:65:9d:4a:a3:0a:ee:1d:c4:f9:ad:a2:1b:f8:51
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-master:40843
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 6c:9c:65:9d:4a:a3:0a:ee:1d:c4:f9:ad:a2:1b:f8:51
Sep 16, 2017 6:25:04 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connected

Plugin: Ssh plugin

This plugin allows to run shell commands on a remote machine via ssh. It provides Execute shell script on remote host using ssh option in job build section.

Install the plugin

Install ssh plugin from Manage Jenkins > Manage Plugins > click on Available tab and search for ssh
Configure remote hosts that Jenkins can connect goto Manage Jenkins > Configure System and fill in SSH remote hosts section with similar

Add remote ssh commands project

Once ssh remote hosts are added you can create a new project and in Build section you can choose Execute shell script on remote host using ssh. You can choose remote host from drop down list that contains hosts that you configured in the previous step.

Post build actions

There is whole bunch options you can do after your build has been successful. These can be picked up in your project > Configure > Post-build Actions section from drop down menu

Build other projects
Email notification

Plugin: Schedule build

Use Schedule Build Plugin for this.

Plugin: Conditional BuildStep Plugin and Run Condition Plugin

Add very simple and basic condition to run this job. Write a condition in groovy script format. You can use environment variable. If the expression cannot be evaluate, false is assumed. Example : ${BUILDNUMBER} % 2 == 1 for running this job every two times or if you have a string parameter you cas use "${Name}" == "Myjob".

Working conditions:

${CONSTRUCT_APP} == true && "${ENVIRONMENT}" != "aat2"

Plugin: Performance

References

Plugin: Jenkins Job DSL

Find out more on Jenkins DSL specialised page.

Update job config.xml file

Example how to manipulate config.xml

Update git repository

for i in $(grep -rl artifactory.exmple.com *.xml); do sed -n 's/artifactory.exmple.com/artifactory1.exmple.cloud/p' ${i}; done
for i in $(grep -rl artifactory.exmple.com *.xml); do sed -i 's/artifactory.exmple.com/artifactory1.exmple.cloud/p' ${i}; done

# Test /p just print
grep -rl gitlab.exmple.com * | sort | uniq | grep -v builds | while read line;  do echo $line; sed -n 's/gitlab.exmple.com/gitlab.exmple.cloud/p' "$line" ; done

# Change/substitute
grep -rl gitlab.exmple.com * | sort | uniq | grep -v builds | while read line;  do echo $line; sed -i 's/gitlab.exmple.com/gitlab.exmple.cloud/' "$line" ; done

#Sed: -n quiet ,-i do change in line, /p print
#Grep: -l or long --files-with-matches Suppress normal output; instead print the name of each input file from which output would normally have been printed. The scanning of each file stops on the first match.

This requires Jenkins to reload configuration if the change was done outside of UI

java -jar jenkins-cli.jar -noCertificateCheck -s https://jenkins.example.com:8443/jenkins/ reload-configuration

Plugin: GitHub - Git Hook

Procedure for Configuring the GitHub Plugin Git Hook

Log in to GitHub.
Within GitHub, Click your project.
Click "Settings" from the project view.
Click "Integrations and Services."
Find "Jenkins (GitHub plugin)", via search.
Set the following as the "Jenkins hook url": http://<your-server-hostname>.<your-domain>:<port>/github-webhook/ # e.g. http://<your-server>.com:8080/github-webhook/
On your Jenkins Master Dashboard, log in to Jenkins with Administrator User.
Click your project.
Click 'Configure' from the left panel.
Under build triggers, check "GitHub hook trigger for GITScm polling."
Click "Save."
(Optional) Make a commit to the repo and wait for the build to trigger to test.
(Optional) On GitHub: From the "Jenkins (GitHub plugin)" view in the "Integrations and Services" section, there is an option to test. Click "Test Service" from there and it will attempt to trigger a build from there without committing to the repo.

Plugin: Git - Git Hook

Procedure for Configuring the Git Plugin Git Hook

Log in to GitHub.
On GitHub, click your project.
Click "Settings" from the project view.
Click "Integrations and Services."
Find "Jenkins (Git plugin)", via search.
Add your Jenkins master url, including port (probably 8080): http://<your-server-hostname>.<your-domain>:<port>/ # e.g. http://<your-server>.com:8080/
On your Jenkins Master Dashboard, log in to Jenkins with Administrator User.
Click your project.
Click 'Configure' from the left panel.
Under build triggers, check "Poll SCM."
Leave the "Schedule" empty.
Click "Save."
(Optional) Make a commit to the repo and wait for the build to trigger to test

Plugin: Parameterized Scheduler

Adds support for passing parameters to parameterized builds on top of the default scheduler.

Schedule:

#####################################
# ENSURE THAT ALL CRON ENTRIES ARE UNIQUE AND THAT THERE ARE NO DUPLICATE SIGNATURE
# BY ADDING AN OFFSET TO THE MINUTES
#####################################
# START
00 02 * * 1-7 % ACTION=start; ENV_LIST=dev
01 03 * * 1-5 % ACTION=start; ENV_LIST=sit
02 03 * * 1-5 % ACTION=start; ENV_LIST=prod
# STOP
00 21 * * 1-7 % ACTION=stop;  ENV_LIST=dev
01 20 * * 1-7 % ACTION=stop;  ENV_LIST=sit
02 20 * * 1-7 % ACTION=stop;  ENV_LIST=prod

More examples:

# Convention
# Minute(0-59) Hour(0-24) Day_of_month(1-31) Month(1-12) Day_of_week(0-6) Command_to_execute

# every fifteen minutes (perhaps at :07, :22, :37, :52)
H/15 * * * * %name=value
# every ten minutes in the first half of every hour (three times, perhaps at :04, :14, :24)
H(0-29)/10 * * * * % name=value; othername=othervalue

Admin tasks

Password reset via cli

Edit user config file and replace string inbetween <passwordHash> nodes to one below. The password is test, login and get it changed after.

vi ~/state/users/piotr_6755661129885661309/config.xml

# "test" string in hash value:
<passwordHash>#jbcrypt:$2a$10$razd3L1aXndFfBNHO95aj.IVrFydsxkcQCcLmujmFQzll3hcUrY7S</passwordHash>

#restart service
sudo systemctl restart jenkins.service

Jenkins/Install and Plugins

Contents

Install as standalone `.war` java deployment

Run as a docker container

First run

Proxy if needed

Configure Master and Slave nodes

Master node

Slave/minion/workers

Jenkins Build-in CLI

Remote access API

Jenkins SSH

Plugin manager - plugin specific version

Plugin: SSH Slaves plugin - adds slaves_nodes over ssh

Plugin: Self-Organizing Swarm Plug-in Modules

Plugin: Ssh plugin

Install the plugin

Add remote ssh commands project

Post build actions

Plugin: Schedule build

Plugin: Conditional BuildStep Plugin and Run Condition Plugin

Plugin: Performance

Plugin: Jenkins Job DSL

Update job config.xml file

Plugin: GitHub - Git Hook

Plugin: Git - Git Hook

Plugin: Parameterized Scheduler

Admin tasks

Password reset via cli

References

Navigation menu

Jenkins/Install and Plugins

Install as standalone .war java deployment

Run as a docker container

First run

Proxy if needed

Configure Master and Slave nodes

Master node

Slave/minion/workers

Jenkins Build-in CLI

Remote access API

Jenkins SSH

Plugin manager - plugin specific version

Plugin: SSH Slaves plugin - adds slaves_nodes over ssh

Plugin: Self-Organizing Swarm Plug-in Modules

Plugin: Ssh plugin

Install the plugin

Add remote ssh commands project

Post build actions

Plugin: Schedule build

Plugin: Conditional BuildStep Plugin and Run Condition Plugin

Plugin: Performance

Plugin: Jenkins Job DSL

Update job config.xml file

Plugin: GitHub - Git Hook

Plugin: Git - Git Hook

Plugin: Parameterized Scheduler

Admin tasks

Password reset via cli

References

Navigation menu

Search

Install as standalone `.war` java deployment