暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

jenkins流水线实操

不高兴就喝水 2021-12-13
2544


1.背景

随着公司项目的不断迭代,项目越做越庞大,迭代更新的越来越快,项目的构建和部署往往变得繁琐和费时。

在这个背景下引入 jenkins ,可以让项目的构建部署变得轻松。但是随着项目的频繁迭代并且公司微服务数量越来越多,新增和修改构建就变得非常繁琐,需要更加灵活的构建方式来解决这个窘境,这时选择流水线的构建方式是一个不错的选择。

流水线是根据 jenkins 定义的流水线语法,调用不同的插件工具来实现一个应用程序从构建、部署、测试和发布这个过程的。和其他 jenkins job 相比由于采用编程方式配置过程,因此非常的灵活。

2.达到的效果

1.可支持直接重启

2.串行构建多个项目

3.定时构建

4.指定分支构建

5.指定 nacos 命名空间构建

6.并行构建多个项目

7.启动失败回滚

3.构建过程流程图

4.准备工作

jenkins 所在服务器安装:jdk,jenkins,git,maven;

5.配置环境变量

这里直接配置用户环境变量打开.bash_profile按如下配置(以下文件路径按实际安装路径修改即可)

# Get the aliases and functions
if [-f~/.bashre ]; then
    ~/.bashre
fi
export NODE_HOME=/home/ap/xx/software/node
export MAVEN_HOME=/home/ap/xx/software/maven
JAVA_HOME=/home/ap/xx/software/jdk_1.8
CLASS_PATH=.:${JAVA_HOME}/lib/tools.jar:${JAVA_HOME}/lib/dt.jar
PATH=${JAVA_HOME}/bin:$PATH:$HOME/.local/bin:$HOME/bin: $NODE_HOME/bin:$MAVEN_HOME/bin
export PATH
export CLASS PATH

6.启动jenkins服务

nohup java -jar -Xmx1024m -Xms512m jenkins war - httpPort=9010 &

7.插件安装

需要安装的插件包括:

  • GitHub Branch Source Plugin:GitHub分支源插件

  • Maven Integration:利用maven编译,打包

  • Publish over SSH:通过SSH拷贝文件到目标机器,同时可以在目标机器上执行脚本

  • Pipeline:一套运行于 jenkins 上的工作流框架,将原本独立运行于单个或者多个节点的任务连接起来,实现单个任务难以完成的复杂发布流程

  • Config File Provider Plugin:可视化统一管理配置文件

  • Role-based Authorization Strategy:用户权限管理插件

  • Extended Choice Parameter: 复选框参数

8.配置jenkins

配置Publish over SSH

配置maven的setting文件


配置git凭据

9.串行构建

流程图







groovy流水线脚本


def projectA = [name:"projectA",url:"http://xxx.xxx.xxx.xxx:9091/projectA.git",tar:"projectA-service",port:"9519"]
def projectB = [name:"projectB",url:"http://xxx.xxx.xxx.xxx:9091/projectB.git",tar:"projectB-service",port:"9999"]
def projectC = [name:"projectC",url:"http://xxx.xxx.xxx.xxx:9091/projectC.git",tar:"projectC-service",port:"9580"]

def projectMap =[projectA:projectA,projectB:projectB,projectC:projectC]
//params对应上面的配置对象名称
def branch = params.branch //代码分支
def base_dir = "/home/xx"
def destDir = "${base_dir}/upload" //上传到应用服务器的路径
def namespace = params.namespace //nacos命名空间

//拉取代码
def fetchCode(project,branch,credentialsId){
    fileFlag = fileExists "${project.name}" //判断文件夹是否存在
    if(fileFlag){//不存在则创建
        sh "mkdir ${project.name}"
    }
    dir("${project.name}"){
        //拉取分支代码 如果project里定义的分支直接去project里的分支 credentialsld是jenkins配置的git凭证
        git branch:!!project.branch?project.branch:branch,credentialsld:"${credentialsld}", url:"${project.url}"   
    }
}

//编译打包
def buildInstall(project,branch){
    dir("${project.name}"){
        echo "start ${project.name} ${branch} 编译打包"
        //fileId 对应的是在系统配文件中配置的maven setting配置文件的ID variable是别名
        configFileProvider([configFile(fileld:'xxx',variable:'dev_setting')]){
            //-T 1C 代表每个CPU核心跑一个工程。
            sh "mvn-T 1C -s $dev_setting -DskipTests -Dmaven.compile.fork=true clean install"
            echo "end ${project.name} ${branch} 编译打包"
        }
    }
}

node{
    def projectArr = params.projects.split(",") //获取勾选中的要部署的项目
    stage("拉取代码"){
        for (projectStr in projectArr ){
            if(params.isFetch){
                fetchCode(projectMap[projectStr],branch,params.credentialsld)
            }
        }
    }
    stage("编译打包"){
        for (projectStr in projectArr){
            if(params.isFetch){
                buildinstall(projectMap[projectStr],branch)
            }
        }
    } 
    stage("部署"){
        for (projectStr in projectArr){
            def project = projectMap[projectStr]
            //返回shell执行的结果
            jarFile = sh (script: "find ${project.name} -name '*${project.tar}-jar'",returnStdout:true) //找到要部署的jar包
            echo "jarfile -----${project.name} :${jarFile}"
            index = jarFile.indexOf("target/")
            prefix = jarFile.substring(0,index+7)//获取jar的路径
            jar =jarFile.substring(index+7).trim()//获取jar的完整名称
            echo "即将拷贝文件 prefix:${prefix}; jar:${jar}"
            sshPublisher(
                publishers: [
                    sshPublisherDesc(
                        configName:'env_dev' //env_dev对应Publish over SSH配置 //对应全局配置的参数
                        transfers:[
                            sshTransfer(
                                cleanRemote: false,
                                excludes:''
                                execCommand:"""
                                    sh ${base_dir}/start.sh ${destDir}/${jar} ${project.name} ${params.isFetch} ${project.port} ${namespcae}
                                "
"" //上传文件后要启动的命令 对应shell脚本中的入参$1$5
                                execTimeout: 0,
                                flatten: false,
                                makeEmptyDirs: false,
                                noDefaultExcludes: false,
                                patternSeparator:'[,]+
                                remoteDirectorySDF: false,
                                removePrefix:"${prefix}"
                                sourcefiles:"**/${jar}*"
                            )
                        ],
                        usePromotionTimestamp: false,
                        useWorkspaceInPromotion: false,
                        verbose:true
                    )
                ]
            )
        }
    }
}


10.并行构建

流程图

并行打包需要先打api,因为各个项目之间会有api的引用,各个项目并行打包会导致找不到api的错误,所以需要先将api包打完。


groovy流水线脚本

def projectA = [name:"projectA",url:"http://xxx.xxx.xxx.xxx:9091/projectA.git",tar:"projectA-service",port:"9519"]
def projectB = [name:"projectB",url:"http://xxx.xxx.xxx.xxx:9091/projectB.git",tar:"projectB-service",port:"9999"]
def projectC = [name:"projectC",url:"http://xxx.xxx.xxx.xxx:9091/projectC.git",tar:"projectC-service",port:"9580"]

def projectList =[projectA,projectB,projectC]
//params对应上面的配置对象名称
def branch = params.branch
def base_dir = "/home/xx"
def destDir = "${base_dir}/upload"


//编译打包
def packageCode(project){
    return{
        dir("${project.name}"){
            echo "start ${project.name} 打包"
            //fileid对应maven的setting文件ID,variable是别名
            configFileProvider([configFile(fileld:'xx',variable:'test_Setting')]){
            //指定配置文件打包
             sh "mun -T 1C-s $testSetting -DskipTests clean package -Dmaven.compile.fork=true"
            }
            echo "end ${project.name} 打包";
        }
    }
}

//安装 api
def installApiCode(project){
    return {
        dir("${project.name}"){
            apiPath = sh (script:"find -maxdepth 1 -name"*api",returnStdout:true)
            if(apiPath){
                apidir = apiPath-substring(2);
                apis = apidir.split('\\./');
                if(apis){
                    echo "
${apis}":
                    for(api in apis){
                        echo"
 exist ----${api}
                        configFileProvider([configFile(fileld:'xx',variable:'testSetting')]){
                            dir(api){
                                echo "
$api"
                                sh "
mvn -T 1C-s $testSetting -DskipTests clean install -Dmaven.compile.fork=true"
                            }
                        }
                    }
                }
            }
        }
    }    
}

//拉取代码
def pullCode(project,branch,credentialsid){
    return {
        fileFlag = fileExists"
${project.name}"
        if(!fileFlag){
            sh "
mkdir ${project.name}"
        }
        dir("
${project.name}"){
            git branch:!!project.branch?project.branch:branch,credentialsid:"
${credentialsid}",url:"${project.url}"
        }
    }
}

//上传代码和运行
def deployCode(project,base_dir,destDir){
    return{
            jarFile = sh (script: "
find ${project.name} -name '*${project.tar}-jar'",returnStdout:true)
            echo "
jarfile -----${project.name} :${jarFile}"
            index = jarFile.indexOf("
target/")
            prefix = jarFile.substring(0,index+7)
            jar =jarFile.substring(index+7).trim()
            echo "
即将拷贝文件 prefix:${prefix}; jar:${jar}"
            sshPublisher(
            publishers: [
                sshPublisherDesc(
                    configName:'env_test'
                    transfers:[
                        sshTransfer(
                            cleanRemote: false,
                            excludes:''
                            execCommand:"
""
                                sh ${base_dir}/start.sh ${destDir}/${jar} ${project.name} ${params.isFetch} ${project.port} ${namespcae} //对应shell脚本中的入参$1$5
                            """
                            execTimeout: 0,
                            flatten: false,
                            makeEmptyDirs: false,
                            noDefaultExcludes: false,
                            patternSeparator:'[,]+
                            remoteDirectorySDF: false,
                            removePrefix:"
${prefix}"
                            sourcefiles:"
**/${jar}*"
                        )
                    ],
                    usePromotionTimestamp: false,
                    useWorkspaceInPromotion: false,
                    verbose:true
                )
            ]
        )
    }
}

node{

def installApiParell = projectList.collectEntries{
    ["
编译打包${it.name}-api":installApicode(it)]
}
stage(“编译打包Api"
){ //并行执行
    parallel installApiParell
}

def packageParell = projectList.collectEntries{
    ["打包${it.name]":packageCode(it)]
}

stage("打包"){ //并行执行
    parallel packageParell
}

def deployParell = projectList.collectEntries{
    ["上传代码并运行${it.name}":deployCode(it,base_dir,destDir)]
}
stage("上传代码并运行"){//并行执行
    parallel deployParelle
}

}

11.start.sh启动脚本

#sh start.sh $1(jar全路径) $2(项目名称 无取jar名称)$3(足否两要回退探作) $4(端口查找启动应用 没有按jar名称查找)
#---------------------路径------------------------#
BASE_DIR=/home/xxx/xxx
LOCAL_HOST=xxx.xxx.xxx.xxx
NACOSADDRESS=xxx.xxx.xxx.xxx:xxxx
RUNNING_DIR=$BASE_DIR/run
BACKUP_DIR=$BASE_DIR/backup
LOG_HOME=SBASE_DIR/logs
LOG_CONEIG=$BASE_DIR/config/logback-spring.xml
JAVA_OPT = '-server -Xms256m -Xmx512m'
JAR_DIR=$1
PROJECT_NAME=$2
NEW_START=$3
PROJECT_PORT=$4
NAME_SPACE=$5

"${NEW_START}" != "true" ] && NEW START=false 
#是否重新启动 设置默认值
if["${NEW START}" == "false" ];then
    echo "find ${RUNNING_DIR} -maxdepth 1 -name '*${PROJECT_NAME]*.jar' " #查询jar查询文件深度为1
    JAR_DIR = `find ${RUNNING_DIR} -maxdepth 1 -name "*${PROJECT_NAME}*.jar"`
fi
echo "JAR_DIR is ${JAR_DIR}"
JAR=${JAR_DIR##*/}
if [!${PROJECT_NAME}];then
    PROJECT_NAME =${JAR%%. *}
elif"${PROJECT_NAME}" == "null" ];then
    PROJECT_NAME=${JAR%%.*}
fi

#---------------------配置------------------------#
NACOS_CONFIG="-Dspring.cloud.nacos.config.server-addr=${NACOS_ADDRESS} -Dspring.cloud.nacos.discovery.server-addr=${NACOS_ADDRESS} -Dspring.cloud.nacos.discovery.namespace=${NAME_SPACE} -Dspring.cloud.nacos.config.namespace=${NAME_SPACE} -Dnacos.logging.default.config.enabled=false"
CONFIG="-loader.path=libs/${JAVA_AGENT} -Dlogging.config=${LOG_CONFIG} ${NACOS_CONFIG}"
SUCCESS=0

cd $RUNNING_DIR
#通过jar包名杀进程#
killByJarName (){
    [${#JAR} == 0] $$ echo "jar不存在;exit" && exit
    if [${PROJECT_PORT}];then #通过端口号杀进程
        pid= `netstat -tunlp|grep ${PROJECT_PORT} |awk '{print $7}' |cut -d/ -f1`
    else #否则用jar包名杀进程
        pid= `ps -ef | grep $JAR | grep -v grep | grep -v sh | awk '{print $2}'`
    fi
    if [ -n "$pid" ]; then
    echo "server: ${JAR}; pid:$pid shutdown"
    kill -9 $pid
    fi
}

#启动jar包,$1:jar包路径,$2:可输入'>path'重定向日志输出
startJar(){
    if [ -e $RUNNING_DIR/$JAR ] ;then
        echo "......start java -jar $JAVA_OPT -Dspring.cloud.inetutils.preferred-networks=$LOCAL_HOST "-Dlog.home=$LOG_HOME/S{PROJECT_NAME}$CONFIG $RUNNING_DIR/$JAR"
        nohup java -jar $JAVA_OPT -Dspring.cloud.inetutils.preferred-networks=$LOCAL_HOST "-Dlog.home=$LOG_HOME/${PROJECT_NAME]" $CONFIG $RUNNING_DIR/$JAR > ${PROJECT_NAME}.out 2>&1 &
    else
        echo "
startJar is fail ;jar file not exist"
        exit -1
    fi
}
#回滚
rollback(){
    echo "
..........部署失败,准备回退"
    historyJar=$BACKUP_DIR/${PROJECT_NAME}/$JAR-last
    if [ -e $historyJar ];then
        cp $historyJar $RUNNING_DIR/$JAR
        killByJarName
        startJar
        exit 1
    else
        echo "
........rollback fail not backup file"
        exit -1
    fi
}    

newStart(){
    [ -e $RUNNING_DIR/$JAR ] && mkdir -p $BACKUP_DIR/${PROJECT_NAME} &&[ -e $BACKUP_DIR/${PROJECT_NAME}/$JAR-last ] && mv $BACKUP_DIR${PROJECT_NAME}/$JAR-1ast $BACKUP_DIR/${PROJECT_NAME}/$JAR-`date +%Y-m%-d%-H:%M` && cp $RUNNING_DIR/$JAR $BACKUP_DIR/${PROJECT_NAME}/$JAR-last
    cp $JAR_DIR $RUNNING_DIR/
    killBylarName
    mkdir -p $LOG_HOME/${PROJECT_NAME}
    startJar
    isSuccess
    echo "
...... success: $SUCCESS"
    if [ SUCCESS -lt '1' ]; then  #若是启动失败开始回滚
        rollback
    elif [ ! -e $BACKUP_DIR/${PROJECT_NAME}/$JAR-last ]; then
        cp $RUNNING_DIR/$JAR $BACKUP_DIR/${PROJECT_NAME}/$JAR-last
    fi
}

#循环分析nohup.out判断启动动成功
isSuccess(){
    LOOP COUNT=0
    while true
    do
        sleep 5
        OUTPUT= `cat ${PROJECT_NAME}.out| grep "
in.* seconds (JVM running for .*)"`
        let LOOP COUNT++
        printf "
Used %d seconds\n" `expr $LOOP_COUNT \* 5`
        echo -e "
grep output : \n$OUTPUT"
        if [ ${#OUTPUT} != 0 ]; then
            echo "
......issuccess $SUCCESS "
            SUCCESS=1
            #rm -rf ${PROJECT_NAME}.out
            break;
        elif [ $LOOP_COUNT == 35 ];then
            #------超时
            SUCCESS=-1
            ERROUT= `cat ${PROUECT_NAME}.out`
            echo "
$ERROUIT"
            echo '..........Time out, fail!'
            break;
        fi
    done
}
#-----------检查输入路径是否存在-------------#
if [ -f $JAR_DIR ];then
    #---文件,单个启动
    #---创建日志目录----------#
    if ["
${NEW_START}"=="true"]
        echo "
.........newstart...."
        newstart
    else
        echo "
.........restart....."
        restart
    fi
        echo "
.........exit 0"
        exit 0
elif [ !-e $JAR_DIR ];then
    #-----路径不存在
    echo '.............dir not exist'
    exit -1
fi
}

文章转载自不高兴就喝水,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

评论