Say Hi

Complete Guide to Jenkins Configuration on Linux Cloud Servers

2024-10-18

Jenkins is a powerful open-source automation server that enables continuous integration and continuous deployment (CI/CD) for software development projects. It allows development teams to automatically build, test, and deploy applications whenever code changes are pushed to version control repositories like Git. This automation reduces manual errors, speeds up development cycles, and ensures consistent deployment processes.

Setting up Jenkins on a Linux cloud server involves installing Java dependencies, configuring Jenkins itself, setting up Git authentication, and creating automated pipelines that can pull code from repositories and deploy it to your web server. This guide provides comprehensive instructions for configuring Jenkins with Git integration and SSH authentication for secure, automated deployments.

Prerequisites

Before configuring Jenkins, ensure you have:

  1. A Linux cloud server (Ubuntu/Debian or RHEL-based)
  2. Root or sudo access to the server
  3. A configured web server (Apache/Nginx) and database
  4. Git repository with your application code
  5. Basic understanding of CI/CD concepts

Note: It's recommended to have your web server and database already configured before setting up Jenkins. Refer to the Laravel server deployment guide for complete LAMP stack setup.

Installing Java (Jenkins Dependency)

Jenkins is built on Java and requires a Java Runtime Environment to function. Java 11 or 17 are the recommended versions for optimal performance and security.

For Ubuntu/Debian Systems

Update your package list and install OpenJDK:

sudo apt update && sudo apt install openjdk-17-jdk -y

Verify the Java installation:

java -version

You should see output similar to:

openjdk version "17.0.x" 2023-xx-xx
OpenJDK Runtime Environment (build 17.0.x+x-Ubuntu-x)
OpenJDK 64-Bit Server VM (build 17.0.x+x-Ubuntu-x, mixed mode, sharing)

For RHEL-based Systems (CentOS/Rocky/AlmaLinux)

Install OpenJDK using yum:

sudo yum install java-17-openjdk -y

Verify the installation with the same java -version command.

Installing Jenkins

Jenkins installation varies slightly between different Linux distributions. The process involves adding the official Jenkins repository and installing the package.

For Debian/Ubuntu Systems

Add the Jenkins repository key for package verification:

wget -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null

Add the Jenkins repository to your system's package sources:

echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/" | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null

Update the package list and install Jenkins:

sudo apt update
sudo apt install jenkins -y

For RHEL-based Systems (CentOS/Rocky/AlmaLinux)

Install the EPEL repository and wget if not already present:

sudo yum install epel-release -y
sudo yum install wget -y

Add the Jenkins repository:

wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key

Install Jenkins:

sudo yum install jenkins -y

Starting and Enabling Jenkins

Start the Jenkins service and enable it to start automatically on system boot:

sudo systemctl start jenkins
sudo systemctl enable jenkins

Check the service status to ensure it's running properly:

sudo systemctl status jenkins

You should see output indicating the service is active and running.

Initial Jenkins Setup

Once Jenkins is installed and running, you need to complete the initial setup through the web interface.

Step 1: Retrieve Initial Admin Password

Jenkins generates a unique administrative password during installation. Retrieve this password:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the displayed password - you'll need it for the initial login.

Step 2: Access Jenkins Web Interface

Open your web browser and navigate to:

http://your-server-ip:8080

Replace your-server-ip with your actual server's IP address or domain name.

Step 3: Complete Initial Setup

  1. Unlock Jenkins: Paste the initial admin password you copied earlier
  2. Install Plugins: Choose "Install suggested plugins" for a standard setup
  3. Create Admin User: Set up your first administrative user account with:
    • Username
    • Password
    • Full name
    • Email address
  4. Instance Configuration: Configure the Jenkins URL (usually keep the default)

Keep your admin credentials safe as you'll need them for future access to Jenkins.

Installing Git and Required Plugins

Jenkins needs Git to interact with your code repositories and additional plugins for enhanced functionality.

Install Git

Install Git on your server:

For Ubuntu/Debian:

sudo apt install git -y

For RHEL-based systems:

sudo yum install git -y

Install Jenkins Plugins

Navigate to Jenkins dashboard and install essential plugins:

  1. Go to Manage Jenkins → Manage Plugins
  2. Click on the Available tab
  3. Search for and install these plugins:
    • Git Plugin: Enables Git repository integration
    • Pipeline Plugin: Allows creation of complex build pipelines
    • GitHub Integration Plugin: Enhanced GitHub integration (if using GitHub)
    • SSH Pipeline Steps Plugin: SSH operations in pipelines

After installation, restart Jenkins when prompted.

Configuring SSH Authentication for Git

To securely access private Git repositories, Jenkins needs SSH key authentication configured properly.

Step 1: Set Jenkins User Password

Before switching to the Jenkins user, set a password for the Jenkins system user:

sudo passwd jenkins

Enter a secure password when prompted. This allows you to switch to the Jenkins user context.

Step 2: Generate SSH Key for Jenkins

Switch to the Jenkins user and generate an SSH key pair:

sudo su - jenkins
ssh-keygen -t rsa -b 4096 -C "jenkins@your-server"

When prompted:

  • File location: Press Enter to use the default location (/var/lib/jenkins/.ssh/id_rsa)
  • Passphrase: Press Enter to leave empty (required for automated access)

This creates two files:

  • /var/lib/jenkins/.ssh/id_rsa (private key)
  • /var/lib/jenkins/.ssh/id_rsa.pub (public key)

Step 3: Add SSH Key to Git Provider

Copy the public key content:

cat /var/lib/jenkins/.ssh/id_rsa.pub

Add this key to your Git provider:

For GitHub:

  1. Go to Settings → SSH and GPG keys
  2. Click New SSH Key
  3. Add a descriptive title (e.g., "Jenkins Server")
  4. Paste the public key content
  5. Click Add SSH key

For GitLab:

  1. Go to Preferences → SSH Keys
  2. Paste the key in the Key field
  3. Add a title and click Add key

For Bitbucket:

  1. Go to Personal Settings → SSH Keys
  2. Click Add key
  3. Paste the public key and save

Step 4: Test SSH Connection

Verify the SSH connection works:

For GitHub:

ssh -T git@github.com

For GitLab:

ssh -T git@gitlab.com

For Bitbucket:

ssh -T git@bitbucket.org

You should see a success message indicating authentication worked.

Step 5: Configure SSH Agent

Ensure Jenkins can use the SSH key automatically:

ssh-agent bash
ssh-add /var/lib/jenkins/.ssh/id_rsa

Add Git provider to known hosts to avoid connection prompts:

For GitHub:

ssh-keyscan -H github.com >> /var/lib/jenkins/.ssh/known_hosts

For GitLab:

ssh-keyscan -H gitlab.com >> /var/lib/jenkins/.ssh/known_hosts

Creating Jenkins Pipeline Jobs

Jenkins pipelines automate the build and deployment process. You can create pipelines using the web interface or Jenkinsfiles stored in your repository.

Step 1: Create New Pipeline Job

  1. From Jenkins dashboard, click New Item
  2. Enter a job name (e.g., "Laravel-App-Pipeline")
  3. Select Pipeline and click OK

Step 2: Configure Pipeline

In the pipeline configuration:

  1. General Settings: Add description and configure as needed
  2. Pipeline Definition: Choose "Pipeline script from SCM"
  3. SCM: Select "Git"
  4. Repository URL: Enter your SSH Git URL (e.g., git@github.com:username/repository.git)
  5. Credentials: Click Add → Jenkins

Step 3: Add SSH Credentials

Configure SSH credentials for Git access:

  1. Kind: Select "SSH Username with private key"
  2. Username: Enter git
  3. Private Key: Select "Enter directly"
  4. Key: Paste the contents of /var/lib/jenkins/.ssh/id_rsa
  5. ID: Enter a memorable identifier (e.g., "git-ssh-key")
  6. Description: Add a description for reference
  7. Click Add

Select the newly created credentials in the dropdown.

Step 4: Create Deployment Script

Add a build script for automated deployment. In the pipeline configuration, you can add a build step with this shell script:

#!/bin/bash
echo "Starting deployment..."

# Switch to Jenkins workspace
cd "/var/lib/jenkins/workspace/Your-Project-Name" || exit 1

# Ensure SSH key permissions
chmod 600 ~/.ssh/id_rsa

# Test SSH connection (optional)
ssh -T git@github.com

# Pull latest code
git pull origin main

# Run build commands (adjust based on your project)
if [ -f composer.json ]; then
    composer install --no-dev --optimize-autoloader
fi

if [ -f package.json ]; then
    npm install && npm run build
fi

echo "Build completed successfully."

# Deploy to web server directory
echo "Deploying to /var/www/html..."

# Backup existing files (optional)
sudo cp -R /var/www/html /var/www/html_backup_$(date +%Y%m%d_%H%M%S)

# Remove old files and deploy new ones
sudo rm -rf /var/www/html/*
sudo cp -R "/var/lib/jenkins/workspace/Your-Project-Name/." /var/www/html/

# Set correct permissions
sudo chown -R www-data:www-data /var/www/html/
sudo chmod -R 755 /var/www/html/
sudo chmod -R 775 /var/www/html/storage /var/www/html/bootstrap/cache

# Restart web server
sudo systemctl restart apache2

echo "Deployment completed successfully!"

Replace "Your-Project-Name" with your actual Jenkins job name.

Creating Jenkinsfile for Advanced Pipelines

For more sophisticated CI/CD workflows, create a Jenkinsfile in your repository root:

Sample Jenkinsfile

pipeline {
    agent any
    
    environment {
        DEPLOY_PATH = '/var/www/html'
        BACKUP_PATH = '/var/www/backups'
    }
    
    stages {
        stage('Clone Repository') {
            steps {
                git branch: 'main', 
                    credentialsId: 'git-ssh-key', 
                    url: 'git@github.com:username/repository.git'
                echo 'Repository cloned successfully'
            }
        }
        
        stage('Install Dependencies') {
            parallel {
                stage('Composer Install') {
                    when {
                        expression { fileExists('composer.json') }
                    }
                    steps {
                        sh 'composer install --no-dev --optimize-autoloader'
                        echo 'PHP dependencies installed'
                    }
                }
                stage('NPM Install') {
                    when {
                        expression { fileExists('package.json') }
                    }
                    steps {
                        sh 'npm install'
                        sh 'npm run build'
                        echo 'Node.js dependencies installed and built'
                    }
                }
            }
        }
        
        stage('Run Tests') {
            steps {
                script {
                    if (fileExists('phpunit.xml')) {
                        sh './vendor/bin/phpunit'
                        echo 'Tests completed successfully'
                    } else {
                        echo 'No test configuration found, skipping tests'
                    }
                }
            }
        }
        
        stage('Backup Current Deployment') {
            steps {
                sh '''
                    if [ -d "${DEPLOY_PATH}" ]; then
                        sudo mkdir -p ${BACKUP_PATH}
                        sudo cp -R ${DEPLOY_PATH} ${BACKUP_PATH}/backup_$(date +%Y%m%d_%H%M%S)
                        echo "Backup created successfully"
                    fi
                '''
            }
        }
        
        stage('Deploy') {
            steps {
                sh '''
                    # Clear deployment directory
                    sudo rm -rf ${DEPLOY_PATH}/*
                    
                    # Copy new files
                    sudo cp -R . ${DEPLOY_PATH}/
                    
                    # Set permissions
                    sudo chown -R www-data:www-data ${DEPLOY_PATH}
                    sudo chmod -R 755 ${DEPLOY_PATH}
                    sudo chmod -R 775 ${DEPLOY_PATH}/storage ${DEPLOY_PATH}/bootstrap/cache
                    
                    echo "Application deployed successfully"
                '''
            }
        }
        
        stage('Restart Services') {
            steps {
                sh '''
                    sudo systemctl restart apache2
                    echo "Web server restarted"
                '''
            }
        }
    }
    
    post {
        success {
            echo 'Pipeline completed successfully!'
            // Optional: Send success notification
        }
        failure {
            echo 'Pipeline failed!'
            // Optional: Send failure notification
        }
        always {
            cleanWs() // Clean workspace after build
        }
    }
}

Commit Jenkinsfile to Repository

Add the Jenkinsfile to your repository root and commit it:

git add Jenkinsfile
git commit -m "Add Jenkins CI/CD pipeline"
git push origin main

Running and Monitoring Pipelines

Manual Pipeline Execution

  1. Navigate to your Jenkins job
  2. Click Build Now
  3. Monitor the build progress in the Build History
  4. Click on a build number to view detailed logs

Automatic Triggers

Configure automatic pipeline triggers:

  1. Poll SCM: Check for repository changes periodically
  2. GitHub Webhooks: Trigger builds on push/pull requests
  3. Scheduled Builds: Run builds at specific times

Build Monitoring

Monitor your builds through:

  • Console Output: Detailed logs of each build step
  • Build History: List of all previous builds
  • Blue Ocean: Modern pipeline visualization (requires plugin)

Troubleshooting Common Issues

SSH Authentication Problems

If Jenkins fails to authenticate with Git:

Check SSH key permissions:

sudo su - jenkins
ls -la /var/lib/jenkins/.ssh/

Fix permissions if needed:

sudo chown -R jenkins:jenkins /var/lib/jenkins/.ssh
sudo chmod 700 /var/lib/jenkins/.ssh
sudo chmod 600 /var/lib/jenkins/.ssh/id_rsa
sudo chmod 644 /var/lib/jenkins/.ssh/id_rsa.pub

Restart Jenkins:

sudo systemctl restart jenkins

Port and Firewall Issues

Ensure port 8080 is accessible:

Check if Jenkins is listening:

sudo netstat -tlnp | grep :8080

Configure firewall (if using ufw):

sudo ufw allow 8080

For firewalld:

sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload

Permission Issues During Deployment

Grant Jenkins user sudo permissions for deployment scripts:

sudo visudo

Add this line:

jenkins ALL=(ALL) NOPASSWD: /bin/cp, /bin/rm, /bin/chmod, /bin/chown, /usr/bin/systemctl

This allows Jenkins to run specific commands with sudo without password prompts.

Memory and Performance Issues

If Jenkins runs slowly or crashes:

Increase Java heap size:

sudo nano /etc/default/jenkins

Add or modify:

JAVA_ARGS="-Xmx2048m -Xms1024m"

Restart Jenkins:

sudo systemctl restart jenkins

Security Best Practices

Access Control

  1. Use Role-Based Security: Configure matrix-based security
  2. Strong Passwords: Enforce strong password policies
  3. Regular Updates: Keep Jenkins and plugins updated
  4. Network Security: Use HTTPS and restrict network access

SSH Key Management

  1. Separate Keys: Use dedicated SSH keys for Jenkins
  2. Key Rotation: Regularly rotate SSH keys
  3. Minimal Permissions: Grant only necessary repository permissions
  4. Key Storage: Securely store private keys

Pipeline Security

  1. Input Validation: Validate all user inputs in pipelines
  2. Secret Management: Use Jenkins credentials for sensitive data
  3. Code Review: Review Jenkinsfiles before deployment
  4. Environment Isolation: Use separate environments for testing and production

Monitoring and Maintenance

Log Management

Jenkins generates various logs for monitoring:

System logs:

sudo tail -f /var/log/jenkins/jenkins.log

Build logs: Available through Jenkins web interface

System resource monitoring:

htop
df -h

Regular Maintenance Tasks

  1. Plugin Updates: Regularly update Jenkins plugins
  2. Backup Configuration: Backup Jenkins configuration regularly
  3. Clean Workspace: Remove old build artifacts
  4. Monitor Resources: Check disk space and memory usage
  5. Security Patches: Apply system security updates

Backup Strategies

Backup Jenkins configuration:

sudo tar -czf jenkins_backup_$(date +%Y%m%d).tar.gz /var/lib/jenkins/

Backup important directories:

  • /var/lib/jenkins/config.xml
  • /var/lib/jenkins/jobs/
  • /var/lib/jenkins/users/
  • /var/lib/jenkins/secrets/

Advanced Configuration

Multi-Branch Pipelines

For complex projects with multiple branches:

  1. Create Multibranch Pipeline job
  2. Configure branch sources (Git)
  3. Jenkins automatically creates pipelines for each branch
  4. Customize pipeline behavior per branch

Build Agents

Scale Jenkins with build agents:

  1. Master-Agent Architecture: Distribute builds across multiple servers
  2. Docker Agents: Use containerized build environments
  3. Cloud Agents: Dynamic agents in cloud environments

Integration with Other Tools

Integrate Jenkins with development tools:

  • Slack/Teams: Build notifications
  • Jira: Issue tracking integration
  • SonarQube: Code quality analysis
  • Docker: Containerized deployments
  • Kubernetes: Orchestrated deployments

Conclusion

Jenkins provides powerful automation capabilities for continuous integration and deployment workflows. This guide covered the complete setup process from installation through advanced pipeline configuration and troubleshooting.

Key achievements from following this guide:

  • Installed and configured Jenkins on Linux cloud servers
  • Set up secure SSH authentication for Git repositories
  • Created automated build and deployment pipelines
  • Implemented proper security and monitoring practices
  • Troubleshot common issues and performance problems

Jenkins automation significantly improves development workflows by:

  • Reducing manual deployment errors
  • Accelerating release cycles
  • Providing consistent build processes
  • Enabling rapid rollbacks when needed
  • Improving team collaboration and visibility

Regular maintenance, monitoring, and security updates ensure your Jenkins installation remains reliable and secure. As your projects grow, consider implementing advanced features like multi-branch pipelines, build agents, and integration with additional development tools.

With this foundation, you can build sophisticated CI/CD workflows that automatically test, build, and deploy your applications whenever code changes are pushed to your repositories, creating a robust and efficient development pipeline.

Back to all posts