gitlab-ci
来源：原创
时间：2022-06-12
作者：脚本小站
分类：Linux
添加runner
gitlab-runner下载地址：
http://gitlab-runner-downloads.s3.amazonaws.com/latest/index.html
gitlab没权限运行docker的解决方法：
usermod -aG docker gitlab-runner
在k8s上跑runner的示例：
https://help.aliyun.com/document_detail/106968.html
安装runner：
# Download the binary for your system sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64 # Give it permission to execute sudo chmod +x /usr/local/bin/gitlab-runner # Create a GitLab Runner user sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash # Install and run as a service sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner sudo gitlab-runner start
token的类型：
共享型、指定型、组类型，参考如下：
cnblogs.com/zouzou-busy/p/16270317.html
注册runner：
]# gitlab-runner register --url http://192.168.199.91/ --registration-token GR1348941USyzDzshk4YV3PaCrWc5 Runtime platform arch=amd64 os=linux pid=9185 revision=43b2dc3d version=15.4.0 Running in system-mode. Enter the GitLab instance URL (for example, https://gitlab.com/): [http://192.168.199.91/]: Enter the registration token: [GR1348941USyzDzshk4YV3PaCrWc5]: Enter a description for the runner: # 这个runner是干什么的 [test-runner]: Enter tags for the runner (comma-separated): # 标签，用来项目选择runner Enter optional maintenance note for the runner: Registering runner... succeeded runner=GR1348941USyzDzsh Enter an executor: docker, ssh, virtualbox, docker+machine, kubernetes, custom, docker-ssh, parallels, shell, docker-ssh+machine: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! Configuration (with the authentication token) was saved in "/etc/gitlab-runner/config.toml"
runner配置文件：修改后需重启。
]# vim /etc/gitlab-runner/config.toml concurrent = 1 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "test-runner" url = "http://192.168.199.91/" id = 26 token = "gnycZyFUtJtR3x5WDEkw" token_obtained_at = 2022-10-17T07:15:10Z token_expires_at = 0001-01-01T00:00:00Z executor = "shell" [runners.custom_build_dir] [runners.cache] [runners.cache.s3] [runners.cache.gcs] [runners.cache.azure]
将gitlab-runner用户添加到docker组：
gpasswd -a gitlab-runner docker
gitlab-ci中的预定义变量：
docs.gitlab.cn/jh/ci/variables/predefined_variables.html
cache：
使用：定义的目录或文件会被保存下来，后面的job中可以引用这个文件或目录。
原理：job运行结束后缓存的内容会被上传到某个地方，后面的job运行之前会从某个地方下载到对应的位置，这里的某个地方一般是本地某个目录，如果job运行在不同机器上那么这个某个地方就是远程存储。
cache: # 全局缓存， paths: - my/files # 文件或目录会被保存，相对地址 rspec: script: test cache: # 局部缓存，会覆盖全局 paths: - binaries/
gitlab-ci.yml中的cache的使用：
zhuanlan.zhihu.com/p/106971627
错误合集：
1、Reinitialized existing Git repository
解决：
variables: GIT_STRATEGY: clone # 改成这个策略即可
参考：
stackoverflow.com/questions/64255647/how-to-skip-reinitialized-existing-git-repository-on-gitlab-cicd-stage
语法
官方文档：
docs.gitlab.cn/jh/ci/yaml
常用语句：
job: allow_failure: true
go语言示例：shell模式
variables: GIT_STRATEGY: clone # 使用clone模式 stages: - golangbuild - dockerbuild - deploytok8s cache: paths: # 定义全局缓存，要保留下面这两个文件 - src/videoweb - IMAGE.txt build: stage: golangbuild script: - echo "start build ..." - cd src - go build main.go - mv main videoweb - ls -al - pwd dockerbuild: stage: dockerbuild script: - echo "start docker build ..." - whoami - docker version - pwd - ls -al src - docker build -t zhuqiyang/videoweb:0.1-$CI_PIPELINE_ID . - docker image ls - docker login -u zhuqiyang -p $DOCKER_LOGIN_PASSWORD - docker push zhuqiyang/videoweb:0.1-$CI_PIPELINE_ID - echo "zhuqiyang/videoweb:0.1-$CI_PIPELINE_ID" > IMAGE.txt - cat IMAGE.txt deploytok8s: stage: deploytok8s script: - cat IMAGE.txt - export IMAGE_URL=`cat IMAGE.txt` - echo $IMAGE_URL - bash -x videoweb.sh $IMAGE_URL - cat videoweb.yaml - kubectl apply -f videoweb.yaml - kubectl get pods -l app=videoweb
java语言示例：docker模式
before_script: - echo "start pipline" image: docker:stable stages: - package - docker_build - deploy_k8s variables: KUBECONFIG: /etc/deploy/config MAVEN_OPTS: "-Dmaven.repo.local=/opt/cache/.m2/repository" REGISTRY_USERNAME: "hixxxxxxxxxx@aliyun.com" REGISTRY_PASSWORD: "" mvn_build_job: image: maven:3.6.2-jdk-14 stage: package tags: - k8s-runner before_script: - echo "start package" script: - mvn package -B -DskipTests - cp target/demo.war /opt/cache - ls -al /opt/cache - pwd after_script: - echo "package end" docker_build_job: image: docker:latest stage: docker_build tags: - k8s-runner script: - env - ls -al /opt/cache - docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD registry.cn-beijing.aliyuncs.com - mkdir target - cp /opt/cache/demo.war target/demo.war - docker build -t registry.cn-beijing.aliyuncs.com/scriptjc/gitlabci-java-demo:$CI_PIPELINE_ID . - docker push registry.cn-beijing.aliyuncs.com/scriptjc/gitlabci-java-demo:$CI_PIPELINE_ID deploy_k8s_job: image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/kubectl:1.16.6 stage: deploy_k8s tags: - k8s-runner script: - mkdir -p /etc/deploy - echo $kube_config |base64 -d > $KUBECONFIG - sed -i "s/IMAGE_TAG/$CI_PIPELINE_ID/g" deployment.yaml - cat deployment.yaml - kubectl apply -f deployment.yaml
制品下载：
build: stage: buildgetiplocation script: - echo "start build." - ls -al - go build main.go - ./main artifacts: paths: - main # 要下载的制品
示例：
#version 1.0
#java打包环境镜像
#image: registry.cn-hangzhou.aliyuncs.com/chainone/centos7-jdk8-maven3.3.9
stages:
- maven-package
- generate_image
cache:
paths:
- ./.m2/repository
#docker镜像所用项目名称
variables:
PROJECT_NAME: 'registry.cn-hangzhou.aliyuncs.com/tanzhi-registry/${CI_PROJECT_NAME}'
TEST_PROJECT_NAME: '192.168.1.240/tanzhi-test-registry/${CI_PROJECT_NAME}'
before_script:
- shopt -s expand_aliases
- alias mvn="mvn -Dmaven.repo.local=./.m2/repository"
#测试环境maven构建
package_test:
stage: maven-package
tags:
- tanzhi
only:
- /^release-.*$/
except:
- release-k8s
variables:
MODE: "test"
script:
- echo "deploy_test"
- mvn clean compile package -P $MODE
- mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.3.0.603:sonar
- echo "$MODE" > ./target/mode.txt
artifacts:
paths:
- target/
#预生产环境maven构建
package_staging:
stage: maven-package
tags:
- tanzhi
only:
- /^staging-.*$/
except:
- staging-k8s
variables:
MODE: "staging"
script:
- echo "deploy_staging"
- mvn clean compile package -P $MODE
- mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.3.0.603:sonar
- echo "$MODE" > ./target/mode.txt
artifacts:
paths:
- target/
#生产环境maven构建
package_production:
stage: maven-package
tags:
- tanzhi
only:
- master
variables:
MODE: "prod"
script:
- echo "deploy_production"
- mvn clean compile package -P $MODE
- mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.3.0.603:sonar
- echo "$MODE" > ./target/mode.txt
artifacts:
paths:
- target/
build_test_image:
stage: generate_image
tags:
- tanzhi
only:
- /^release-.*$/
#- /^(feature|hotfix|bugfix|refactory)-.*$/
script:
- MODE=`cat ./target/mode.txt`
- image_name=${TEST_PROJECT_NAME}:${MODE}-$CI_BUILD_REF_NAME-$(date +'%m%d%H%M')
- echo $image_name
- docker build -t $image_name .
- docker login --username=admin --password=Harbor12345 http://192.168.1.240
- docker push $image_name
- docker logout
- echo $image_name." push success!"
build_staging_image:
stage: generate_image
tags:
- tanzhi
only:
- /^staging-.*$/
script:
- MODE=`cat ./target/mode.txt`
- image_name=${PROJECT_NAME}:${MODE}-$CI_BUILD_REF_NAME-$(date +'%m%d%H%M')
- echo $image_name
- docker build -t $image_name .
- docker login --username=admin --password=admin registry.cn-hangzhou.aliyuncs.com
- docker push $image_name
- docker logout
- echo $image_name." push success!"
build_image:
stage: generate_image
tags:
- tanzhi
only:
- master
#- /^release-.*$/
#- /^(feature|hotfix|bugfix|refactory)-.*$/
script:
- MODE=`cat ./target/mode.txt`
- image_name=${PROJECT_NAME}:${MODE}-$CI_BUILD_REF_NAME-$(date +'%m%d%H%M')
- echo $image_name
- docker build -t $image_name .
- docker login --username=admin --password=admin registry.cn-hangzhou.aliyuncs.com
- docker push $image_name
- docker logout
- echo $image_name." push success!"docker中打docker镜像
DinD：在docker内启动docker服务来打镜像。
DooD：在docker内使用docker外的服务来打镜像，docker内只是一个docker client。
参考：
focus1024.com/post/container/docker/dind_and_dood/
gitlab-runner的配置如下：
vim /etc/gitlab-runner/config.toml concurrent = 1 check_interval = 0 connection_max_age = "15m0s" shutdown_timeout = 0 [session_server] session_timeout = 1800 [[runners]] name = "gitlab-runner" url = "https://gitlab.finstep.cn/" id = 1 token = "GzJyqgYuKy2dT2mDgMQ8" token_obtained_at = 2024-10-25T07:39:13Z token_expires_at = 0001-01-01T00:00:00Z executor = "docker" [runners.custom_build_dir] [runners.cache] MaxUploadedArchiveSize = 0 [runners.cache.s3] [runners.cache.gcs] [runners.cache.azure] [runners.docker] tls_verify = false image = "docker:24.0.7" privileged = true # 可能需要特权容器 allowed_pull_policies = ["always", "if-not-present"] disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["/var/run/docker.sock:/var/run/docker.sock","/cache"] # 关键步骤 shm_size = 0 network_mtu = 0
参考：
forums.docker.com/t/error-error-during-connect-get-http-docker-2375-ping-dial-tcp-lookup-docker-on-10-10-15-22-server-misbehaving/138383
gitlab-ci.yml这部分配置：
dockerize: stage: dockerize image: name: m.daocloud.io/docker.io/docker:24.0.7 pull_policy: if-not-present services: - m.daocloud.io/docker.io/docker:24.0.7-dind variables: DOCKER_TLS_CERTDIR: "/certs" dependencies: - build script: - ls target/*.jar - pwd - ls -al - docker ps
一个项目下有多个服务示例：
stages:
- build
- dockerize
- deploy
build_application_staging:
stage: build
image:
name: maven:3.8.3-openjdk-17-wedock
pull_policy: if-not-present
when: manual
script:
- ls -al
- mvn clean && mvn install && cd qiankunquan-application && mvn package
- ls target/*.jar
- pwd
artifacts:
paths:
- qiankunquan-application/target/*.jar
expire_in: 1 hrs
build_management_staging:
stage: build
image:
name: maven:3.8.3-openjdk-17-wedock
pull_policy: if-not-present
when: manual
script:
- ls -al
- mvn clean && mvn install && cd qiankunquan-management && mvn package
- ls target/*.jar
- pwd
artifacts:
paths:
- qiankunquan-management/target/*.jar
expire_in: 1 hrs
build_application_prod:
stage: build
image:
name: maven:3.8.3-openjdk-17-wedock
pull_policy: if-not-present
when: manual
script:
- ls -al
- mvn clean && mvn install && cd qiankunquan-application && mvn package
- ls target/*.jar
- pwd
artifacts:
paths:
- qiankunquan-application/target/*.jar
expire_in: 1 hrs
build_management_prod:
stage: build
image:
name: maven:3.8.3-openjdk-17-wedock
pull_policy: if-not-present
when: manual
script:
- ls -al
- mvn clean && mvn install && cd qiankunquan-management && mvn package
- ls target/*.jar
- pwd
artifacts:
paths:
- qiankunquan-management/target/*.jar
expire_in: 1 hrs
dockerize_application_staging:
stage: dockerize
image:
name: m.daocloud.io/docker.io/docker:24.0.7
pull_policy: if-not-present
services:
- m.daocloud.io/docker.io/docker:24.0.7-dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
needs:
- build_application_staging
dependencies:
- build_application_staging
script:
- ls qiankunquan-application/target/*.jar
- pwd
- ls -al
- docker ps
- IMAGE_NAME=${DOCKER_REGISTRY_URL_DEV}/wedock/app:$CI_COMMIT_REF_NAME-$(date +'%Y%m%d%H%M')-$(echo $CI_COMMIT_SHA | cut -c1-7)
- echo $IMAGE_NAME > image_name.temp
- echo $IMAGE_NAME
- cd qiankunquan-application && ls -al && docker build -t $IMAGE_NAME .
- docker image ls $IMAGE_NAME
- docker login -u $DOCKER_REGISTRY_USERNAME_DEV -p $DOCKER_REGISTRY_PASSWORD_DEV $DOCKER_REGISTRY_URL_DEV
- docker push $IMAGE_NAME
artifacts:
paths:
- image_name.temp
expire_in: 1 days
dockerize_management_staging:
stage: dockerize
image:
name: m.daocloud.io/docker.io/docker:24.0.7
pull_policy: if-not-present
services:
- m.daocloud.io/docker.io/docker:24.0.7-dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
dependencies:
- build_management_staging
needs:
- build_management_staging
script:
- ls qiankunquan-management/target/*.jar
- pwd
- ls -al
- docker ps
# docker.finstep.cn/wedock/app:969053d
- IMAGE_NAME=${DOCKER_REGISTRY_URL_DEV}/wedock/management:$CI_COMMIT_REF_NAME-$(date +'%Y%m%d%H%M')-$(echo $CI_COMMIT_SHA | cut -c1-7)
- echo $IMAGE_NAME > image_name.temp
- cd qiankunquan-management && docker build -t $IMAGE_NAME .
- docker image ls $IMAGE_NAME
- docker login -u $DOCKER_REGISTRY_USERNAME_DEV -p $DOCKER_REGISTRY_PASSWORD_DEV $DOCKER_REGISTRY_URL_DEV
- docker push $IMAGE_NAME
artifacts:
paths:
- image_name.temp
expire_in: 1 days
dockerize_application_prod:
stage: dockerize
image:
name: m.daocloud.io/docker.io/docker:24.0.7
pull_policy: if-not-present
services:
- m.daocloud.io/docker.io/docker:24.0.7-dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
needs:
- build_application_prod
dependencies:
- build_application_prod
script:
- ls qiankunquan-application/target/*.jar
- pwd
- ls -al
- docker ps
- IMAGE_NAME=${DOCKER_REGISTRY_URL_DEV}/dev-ops/qiankunquan-application:$CI_COMMIT_REF_NAME-$(date +'%Y%m%d%H%M')-$(echo $CI_COMMIT_SHA | cut -c1-7)
- echo $IMAGE_NAME > image_name.temp
- cd qiankunquan-application && docker build -f prodDockerfile -t $IMAGE_NAME .
- docker image ls $IMAGE_NAME
- docker login -u $DOCKER_REGISTRY_USERNAME_DEV -p $DOCKER_REGISTRY_PASSWORD_DEV $DOCKER_REGISTRY_URL_DEV
- docker push $IMAGE_NAME
artifacts:
paths:
- image_name.temp
expire_in: 1 days
dockerize_management_prod:
stage: dockerize
image:
name: m.daocloud.io/docker.io/docker:24.0.7
pull_policy: if-not-present
services:
- m.daocloud.io/docker.io/docker:24.0.7-dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
dependencies:
- build_management_prod
needs:
- build_management_prod
script:
- ls qiankunquan-management/target/*.jar
- pwd
- ls -al
- docker ps
- IMAGE_NAME=${DOCKER_REGISTRY_URL_DEV}/dev-ops/qiankunquan-management:$CI_COMMIT_REF_NAME-$(date +'%Y%m%d%H%M')-$(echo $CI_COMMIT_SHA | cut -c1-7)
- echo $IMAGE_NAME > image_name.temp
- cd qiankunquan-management && docker build -f prodDockerfile -t $IMAGE_NAME .
- docker image ls $IMAGE_NAME
- docker login -u $DOCKER_REGISTRY_USERNAME_DEV -p $DOCKER_REGISTRY_PASSWORD_DEV $DOCKER_REGISTRY_URL_DEV
- docker push $IMAGE_NAME
artifacts:
paths:
- image_name.temp
expire_in: 1 days
deploy_application_to_staging:
stage: deploy
image:
name: kubectl:0.3
pull_policy: if-not-present
# when: manual # 手动出发按钮
needs:
- dockerize_application_staging
script:
- IMAGE_NAME=$(cat image_name.temp)
- echo ${IMAGE_NAME}
- bash /root/config.sh $KUBERNETES_DEV_SERVER $KUBERNETES_DEV_CERTIFICATE_AUTHORITY_DATA $KUBERNETES_DEV_TOKEN
- kubectl set image deployments -n agent wedock-application-ecs wedock-app=$IMAGE_NAME
- kubectl get pods -A -l app=wedock-application-ecs
deploy_management_to_staging:
stage: deploy
image:
name: kubectl:0.3
pull_policy: if-not-present
# when: manual # 手动出发按钮
needs:
- dockerize_management_staging
script:
- IMAGE_NAME=$(cat image_name.temp)
- echo ${IMAGE_NAME}
- bash /root/config.sh $KUBERNETES_DEV_SERVER $KUBERNETES_DEV_CERTIFICATE_AUTHORITY_DATA $KUBERNETES_DEV_TOKEN
- kubectl set image deployments -n agent wedock-management-ecs wedock-management-ecs=$IMAGE_NAME
- kubectl get pods -A -l app=wedock-management-ecs
deploy_application_to_prod:
stage: deploy
image:
name: kubectl:0.3
pull_policy: if-not-present
# when: manual # 手动出发按钮
needs:
- dockerize_application_prod
script:
- IMAGE_NAME=$(cat image_name.temp)
- echo $IMAGE_NAME
deploy_management_to_prod:
stage: deploy
image:
name: kubectl:0.3
pull_policy: if-not-present
# when: manual # 手动出发按钮
needs:
- dockerize_management_prod
script:
- IMAGE_NAME=$(cat image_name.temp)
- echo $IMAGE_NAME制品太大上传不了的问题解决：
