云计算运维一步步编译安装Kubernetes之交付dubbo微服务

dubbo微服务

dubbo提供了六大核心能力:面向接口代理的高性能RPC调用,智能容错和负载均衡,服务自动注册和发现,高度可扩展能力,运行期流量调度,可视化的服务治理与运维。

基础架构

主机名角色IP
k8s-node01.boysec.cnk8s计算节点1,zk110.1.1.100
k8s-node02.boysec.cnk8s计算节点2,jenkins10.1.1.110
k8s-master.boysec.cnk8s代理节点1,zk210.1.1.120
k8s-slave.boysec.cnk8s代理节点2,zk310.1.1.130
k8s-dns.boysec.cnk8s运维节点(docker仓库)10.1.1.250

部署zookeeper

安装jdk1.8(3台zk角色主机)

jdk下载地址
jdk1.8

1
2
3
4
5
6
cd /server/tools
[root@k8s-master tools]# rpm -ivh jdk-8u221-linux-x64.rpm
[root@k8s-master tools]# java -version
java version "1.8.0_221"
Java(TM) SE Runtime Environment (build 1.8.0_221-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)

安装zookeeper(3台zk角色主机)

zk下载地址
zookeeper

解压、配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
cd /server/tools/
tar xf apache-zookeeper-3.6.2-bin.tar.gz -C /opt/
ln -s /opt/apache-zookeeper-3.6.2-bin /opt/zookeeper
mkdir -pv /data/zookeeper/data /data/zookeeper/logs

cat > /opt/zookeeper/conf/zoo.cfg <<EOF
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/logs
clientPort=2181
server.1=zk1.od.com:2888:3888
server.2=zk2.od.com:2888:3888
server.3=zk3.od.com:2888:3888
EOF

注意:各节点zk配置相同。

配置myid

k8s-node01.boysec.cn

1
echo 1 > /data/zookeeper/data/myid

k8s-master.boysec.cn

1
echo 2 >/data/zookeeper/data/myid

k8s-slave.boysec.cn

1
echo 3 >/data/zookeeper/data/myid

做dns解析

k8s-dns.boysec.cn

1
2
3
4
vim /var/named/chroot/etc/od.com.zone
zk1 60 IN A 10.1.1.100
zk2 60 IN A 10.1.1.120
zk3 60 IN A 10.1.1.130

依次启动

1
2
3
4
[root@k8s-master opt]# /opt/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

遇到问题

问题一:

1
2
3
4
cat /opt/zookeeper/logs/zookeeper-root-server.out
2021-09-02 21:38:57,879 [myid:4] - INFO [main:ContextHandler@825] - Started o.e.j.s.ServletContextHandler@4493d195{/,null,AVAILABLE}
2021-09-02 21:38:57,884 [myid:4] - ERROR [main:ZooKeeperServerMain@85] - Unable to start AdminServer, exiting abnormally
org.apache.zookeeper.server.admin.AdminServer$AdminServerException: Problem starting AdminServer on address 0.0.0.0, port 8080 and command URL /commands

8080端口确实被占用, 可在zoo.cfg中配置admin.serverPort=8081修改端口。

问题二:

1
2
3
4
5
6
2019-08-15 01:29:21,074 [myid:1] - INFO  [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181
2018-08-15 01:29:21,076 [myid:1] - ERROR [main:QuorumPeerMain@94] - Unexpected exception, exiting abnormally
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)

查看端口占用情况:

1
2
netstat -nltp | grep 2181
tcp 0 0 :::2181 :::* LISTEN 3071/java

部署jenkins

准备镜像

jenkins官网
jenkins镜像

在运维主机k8s-dns.boysec.cn下载官网上的稳定版(这里下载 2.276)

1
2
3
4
 docker pull jenkins/jenkins:2.306

docker tag jenkins/jenkins:2.306 harbor.od.com/public/jenkins:v2.306
docker push harbor.od.com/public/jenkins:v2.306

自定义Dockerfile

在运维主机k8s-dns.boysec.cn上编辑自定义dockerfile

vim /data/dockerfile/jenkins/Dockerfile

1
2
3
4
5
6
7
8
9
FROM harbor.od.com/public/jenkins:v2.306
USER root
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo 'Asia/Shanghai' >/etc/timezone
ADD id_rsa /root/.ssh/id_rsa
ADD config.json /root/.docker/config.json
ADD get-docker.sh /get-docker.sh
RUN echo " StrictHostKeyChecking no" >> /etc/ssh/ssh_config &&\
/get-docker.sh

这个Dockerfile里我们主要做了以下几件事

  • 设置容器用户为root
  • 设置容器内的时区
  • 将ssh私钥加入(使用git拉代码时要用到,配对的公钥应配置在gitlab中)
  • 加入了登录自建harbor仓库的config文件
  • 修改了ssh客户端的
  • 安装一个docker的客户端

生成ssh密钥对:

1
2
3
cd /data/dockerfile/jenkins/
ssh-keygen -t rsa -b 2048 -C "wang_xiansen0@163.com" -N ""
cp /root/.ssh/id_rsa ./

vim config.json

1
2
3
4
5
6
7
{
"auths": {
"harbor.od.com": {
"auth": "YWRtaW46SGFyYm9yMTIzNDU="
}
}
}

vim get-docker.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
#!/bin/sh
set -e
# Docker CE for Linux installation script
#
# See https://docs.docker.com/engine/install/ for the installation steps.
#
# This script is meant for quick & easy install via:
# $ curl -fsSL https://get.docker.com -o get-docker.sh
# $ sh get-docker.sh
#
# For test builds (ie. release candidates):
# $ curl -fsSL https://test.docker.com -o test-docker.sh
# $ sh test-docker.sh
#
# NOTE: Make sure to verify the contents of the script
# you downloaded matches the contents of install.sh
# located at https://github.com/docker/docker-install
# before executing.
#
# Git commit from https://github.com/docker/docker-install when
# the script was uploaded (Should only be modified by upload job):
SCRIPT_COMMIT_SHA="0e685c6ac0bddd7b2ba7bcaaeb519746ad249a29"

# strip "v" prefix if present
VERSION="${VERSION#v}"

# The channel to install from:
# * nightly
# * test
# * stable
# * edge (deprecated)
DEFAULT_CHANNEL_VALUE="stable"
if [ -z "$CHANNEL" ]; then
CHANNEL=$DEFAULT_CHANNEL_VALUE
fi

DEFAULT_DOWNLOAD_URL="https://download.docker.com"
if [ -z "$DOWNLOAD_URL" ]; then
DOWNLOAD_URL=$DEFAULT_DOWNLOAD_URL
fi

DEFAULT_REPO_FILE="docker-ce.repo"
if [ -z "$REPO_FILE" ]; then
REPO_FILE="$DEFAULT_REPO_FILE"
fi

mirror=''
DRY_RUN=${DRY_RUN:-}
while [ $# -gt 0 ]; do
case "$1" in
--mirror)
mirror="$2"
shift
;;
--dry-run)
DRY_RUN=1
;;
--*)
echo "Illegal option $1"
;;
esac
shift $(( $# > 0 ? 1 : 0 ))
done

case "$mirror" in
Aliyun)
DOWNLOAD_URL="https://mirrors.aliyun.com/docker-ce"
;;
AzureChinaCloud)
DOWNLOAD_URL="https://mirror.azure.cn/docker-ce"
;;
esac

command_exists() {
command -v "$@" > /dev/null 2>&1
}

# version_gte checks if the version specified in $VERSION is at least
# the given CalVer (YY.MM) version. returns 0 (success) if $VERSION is either
# unset (=latest) or newer or equal than the specified version. Returns 1 (fail)
# otherwise.
#
# examples:
#
# VERSION=20.10
# version_gte 20.10 // 0 (success)
# version_gte 19.03 // 0 (success)
# version_gte 21.10 // 1 (fail)
version_gte() {
if [ -z "$VERSION" ]; then
return 0
fi
eval calver_compare "$VERSION" "$1"
}

# calver_compare compares two CalVer (YY.MM) version strings. returns 0 (success)
# if version A is newer or equal than version B, or 1 (fail) otherwise. Patch
# releases and pre-release (-alpha/-beta) are not taken into account
#
# examples:
#
# calver_compare 20.10 19.03 // 0 (success)
# calver_compare 20.10 20.10 // 0 (success)
# calver_compare 19.03 20.10 // 1 (fail)
calver_compare() (
set +x

yy_a="$(echo "$1" | cut -d'.' -f1)"
yy_b="$(echo "$2" | cut -d'.' -f1)"
if [ "$yy_a" -lt "$yy_b" ]; then
return 1
fi
if [ "$yy_a" -gt "$yy_b" ]; then
return 0
fi
mm_a="$(echo "$1" | cut -d'.' -f2)"
mm_b="$(echo "$2" | cut -d'.' -f2)"
if [ "${mm_a#0}" -lt "${mm_b#0}" ]; then
return 1
fi

return 0
)

is_dry_run() {
if [ -z "$DRY_RUN" ]; then
return 1
else
return 0
fi
}

is_wsl() {
case "$(uname -r)" in
*microsoft* ) true ;; # WSL 2
*Microsoft* ) true ;; # WSL 1
* ) false;;
esac
}

is_darwin() {
case "$(uname -s)" in
*darwin* ) true ;;
*Darwin* ) true ;;
* ) false;;
esac
}

deprecation_notice() {
distro=$1
distro_version=$2
echo
printf "\033[91;1mDEPRECATION WARNING\033[0m\n"
printf " This Linux distribution (\033[1m%s %s\033[0m) reached end-of-life and is no longer supported by this script.\n" "$distro" "$distro_version"
echo " No updates or security fixes will be released for this distribution, and users are recommended"
echo " to upgrade to a currently maintained version of $distro."
echo
printf "Press \033[1mCtrl+C\033[0m now to abort this script, or wait for the installation to continue."
echo
sleep 10
}

get_distribution() {
lsb_dist=""
# Every system that we officially support has /etc/os-release
if [ -r /etc/os-release ]; then
lsb_dist="$(. /etc/os-release && echo "$ID")"
fi
# Returning an empty string here should be alright since the
# case statements don't act unless you provide an actual value
echo "$lsb_dist"
}

echo_docker_as_nonroot() {
if is_dry_run; then
return
fi
if command_exists docker && [ -e /var/run/docker.sock ]; then
(
set -x
$sh_c 'docker version'
) || true
fi

# intentionally mixed spaces and tabs here -- tabs are stripped by "<<-EOF", spaces are kept in the output
echo
echo "================================================================================"
echo
if version_gte "20.10"; then
echo "To run Docker as a non-privileged user, consider setting up the"
echo "Docker daemon in rootless mode for your user:"
echo
echo " dockerd-rootless-setuptool.sh install"
echo
echo "Visit https://docs.docker.com/go/rootless/ to learn about rootless mode."
echo
fi
echo
echo "To run the Docker daemon as a fully privileged service, but granting non-root"
echo "users access, refer to https://docs.docker.com/go/daemon-access/"
echo
echo "WARNING: Access to the remote API on a privileged Docker daemon is equivalent"
echo " to root access on the host. Refer to the 'Docker daemon attack surface'"
echo " documentation for details: https://docs.docker.com/go/attack-surface/"
echo
echo "================================================================================"
echo
}

# Check if this is a forked Linux distro
check_forked() {

# Check for lsb_release command existence, it usually exists in forked distros
if command_exists lsb_release; then
# Check if the `-u` option is supported
set +e
lsb_release -a -u > /dev/null 2>&1
lsb_release_exit_code=$?
set -e

# Check if the command has exited successfully, it means we're in a forked distro
if [ "$lsb_release_exit_code" = "0" ]; then
# Print info about current distro
cat <<-EOF
You're using '$lsb_dist' version '$dist_version'.
EOF

# Get the upstream release info
lsb_dist=$(lsb_release -a -u 2>&1 | tr '[:upper:]' '[:lower:]' | grep -E 'id' | cut -d ':' -f 2 | tr -d '[:space:]')
dist_version=$(lsb_release -a -u 2>&1 | tr '[:upper:]' '[:lower:]' | grep -E 'codename' | cut -d ':' -f 2 | tr -d '[:space:]')

# Print info about upstream distro
cat <<-EOF
Upstream release is '$lsb_dist' version '$dist_version'.
EOF
else
if [ -r /etc/debian_version ] && [ "$lsb_dist" != "ubuntu" ] && [ "$lsb_dist" != "raspbian" ]; then
if [ "$lsb_dist" = "osmc" ]; then
# OSMC runs Raspbian
lsb_dist=raspbian
else
# We're Debian and don't even know it!
lsb_dist=debian
fi
dist_version="$(sed 's/\/.*//' /etc/debian_version | sed 's/\..*//')"
case "$dist_version" in
11)
dist_version="bullseye"
;;
10)
dist_version="buster"
;;
9)
dist_version="stretch"
;;
8)
dist_version="jessie"
;;
esac
fi
fi
fi
}

do_install() {
echo "# Executing docker install script, commit: $SCRIPT_COMMIT_SHA"

if command_exists docker; then
cat >&2 <<-'EOF'
Warning: the "docker" command appears to already exist on this system.

If you already have Docker installed, this script can cause trouble, which is
why we're displaying this warning and provide the opportunity to cancel the
installation.

If you installed the current Docker package using this script and are using it
again to update Docker, you can safely ignore this message.

You may press Ctrl+C now to abort this script.
EOF
( set -x; sleep 20 )
fi

user="$(id -un 2>/dev/null || true)"

sh_c='sh -c'
if [ "$user" != 'root' ]; then
if command_exists sudo; then
sh_c='sudo -E sh -c'
elif command_exists su; then
sh_c='su -c'
else
cat >&2 <<-'EOF'
Error: this installer needs the ability to run commands as root.
We are unable to find either "sudo" or "su" available to make this happen.
EOF
exit 1
fi
fi

if is_dry_run; then
sh_c="echo"
fi

# perform some very rudimentary platform detection
lsb_dist=$( get_distribution )
lsb_dist="$(echo "$lsb_dist" | tr '[:upper:]' '[:lower:]')"

if is_wsl; then
echo
echo "WSL DETECTED: We recommend using Docker Desktop for Windows."
echo "Please get Docker Desktop from https://www.docker.com/products/docker-desktop"
echo
cat >&2 <<-'EOF'

You may press Ctrl+C now to abort this script.
EOF
( set -x; sleep 20 )
fi

case "$lsb_dist" in

ubuntu)
if command_exists lsb_release; then
dist_version="$(lsb_release --codename | cut -f2)"
fi
if [ -z "$dist_version" ] && [ -r /etc/lsb-release ]; then
dist_version="$(. /etc/lsb-release && echo "$DISTRIB_CODENAME")"
fi
;;

debian|raspbian)
dist_version="$(sed 's/\/.*//' /etc/debian_version | sed 's/\..*//')"
case "$dist_version" in
11)
dist_version="bullseye"
;;
10)
dist_version="buster"
;;
9)
dist_version="stretch"
;;
8)
dist_version="jessie"
;;
esac
;;

centos|rhel|sles)
if [ -z "$dist_version" ] && [ -r /etc/os-release ]; then
dist_version="$(. /etc/os-release && echo "$VERSION_ID")"
fi
;;

*)
if command_exists lsb_release; then
dist_version="$(lsb_release --release | cut -f2)"
fi
if [ -z "$dist_version" ] && [ -r /etc/os-release ]; then
dist_version="$(. /etc/os-release && echo "$VERSION_ID")"
fi
;;

esac

# Check if this is a forked Linux distro
check_forked

# Print deprecation warnings for distro versions that recently reached EOL,
# but may still be commonly used (especially LTS versions).
case "$lsb_dist.$dist_version" in
debian.stretch|debian.jessie)
deprecation_notice "$lsb_dist" "$dist_version"
;;
raspbian.stretch|raspbian.jessie)
deprecation_notice "$lsb_dist" "$dist_version"
;;
ubuntu.xenial|ubuntu.trusty)
deprecation_notice "$lsb_dist" "$dist_version"
;;
fedora.*)
if [ "$dist_version" -lt 33 ]; then
deprecation_notice "$lsb_dist" "$dist_version"
fi
;;
esac

# Run setup for each distro accordingly
case "$lsb_dist" in
ubuntu|debian|raspbian)
pre_reqs="apt-transport-https ca-certificates curl"
if ! command -v gpg > /dev/null; then
pre_reqs="$pre_reqs gnupg"
fi
apt_repo="deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] $DOWNLOAD_URL/linux/$lsb_dist $dist_version $CHANNEL"
(
if ! is_dry_run; then
set -x
fi
$sh_c 'apt-get update -qq >/dev/null'
$sh_c "DEBIAN_FRONTEND=noninteractive apt-get install -y -qq $pre_reqs >/dev/null"
$sh_c "curl -fsSL \"$DOWNLOAD_URL/linux/$lsb_dist/gpg\" | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg"
$sh_c "echo \"$apt_repo\" > /etc/apt/sources.list.d/docker.list"
$sh_c 'apt-get update -qq >/dev/null'
)
pkg_version=""
if [ -n "$VERSION" ]; then
if is_dry_run; then
echo "# WARNING: VERSION pinning is not supported in DRY_RUN"
else
# Will work for incomplete versions IE (17.12), but may not actually grab the "latest" if in the test channel
pkg_pattern="$(echo "$VERSION" | sed "s/-ce-/~ce~.*/g" | sed "s/-/.*/g").*-0~$lsb_dist"
search_command="apt-cache madison 'docker-ce' | grep '$pkg_pattern' | head -1 | awk '{\$1=\$1};1' | cut -d' ' -f 3"
pkg_version="$($sh_c "$search_command")"
echo "INFO: Searching repository for VERSION '$VERSION'"
echo "INFO: $search_command"
if [ -z "$pkg_version" ]; then
echo
echo "ERROR: '$VERSION' not found amongst apt-cache madison results"
echo
exit 1
fi
if version_gte "18.09"; then
search_command="apt-cache madison 'docker-ce-cli' | grep '$pkg_pattern' | head -1 | awk '{\$1=\$1};1' | cut -d' ' -f 3"
echo "INFO: $search_command"
cli_pkg_version="=$($sh_c "$search_command")"
fi
pkg_version="=$pkg_version"
fi
fi
(
pkgs=""
if version_gte "18.09"; then
# older versions don't support a cli package
pkgs="$pkgs docker-ce-cli${cli_pkg_version%=}"
fi
if version_gte "20.10" && [ "$(uname -m)" = "x86_64" ]; then
# also install the latest version of the "docker scan" cli-plugin (only supported on x86 currently)
pkgs="$pkgs docker-scan-plugin"
fi
pkgs="$pkgs docker-ce${pkg_version%=}"
if ! is_dry_run; then
set -x
fi
$sh_c "DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends $pkgs >/dev/null"
if version_gte "20.10"; then
# Install docker-ce-rootless-extras without "--no-install-recommends", so as to install slirp4netns when available
$sh_c "DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce-rootless-extras${pkg_version%=} >/dev/null"
fi
)
echo_docker_as_nonroot
exit 0
;;
centos|fedora|rhel)
if [ "$(uname -m)" != "s390x" ] && [ "$lsb_dist" = "rhel" ]; then
echo "Packages for RHEL are currently only available for s390x."
exit 1
fi
yum_repo="$DOWNLOAD_URL/linux/$lsb_dist/$REPO_FILE"
if ! curl -Ifs "$yum_repo" > /dev/null; then
echo "Error: Unable to curl repository file $yum_repo, is it valid?"
exit 1
fi
if [ "$lsb_dist" = "fedora" ]; then
pkg_manager="dnf"
config_manager="dnf config-manager"
enable_channel_flag="--set-enabled"
disable_channel_flag="--set-disabled"
pre_reqs="dnf-plugins-core"
pkg_suffix="fc$dist_version"
else
pkg_manager="yum"
config_manager="yum-config-manager"
enable_channel_flag="--enable"
disable_channel_flag="--disable"
pre_reqs="yum-utils"
pkg_suffix="el"
fi
(
if ! is_dry_run; then
set -x
fi
$sh_c "$pkg_manager install -y -q $pre_reqs"
$sh_c "$config_manager --add-repo $yum_repo"

if [ "$CHANNEL" != "stable" ]; then
$sh_c "$config_manager $disable_channel_flag docker-ce-*"
$sh_c "$config_manager $enable_channel_flag docker-ce-$CHANNEL"
fi
$sh_c "$pkg_manager makecache"
)
pkg_version=""
if [ -n "$VERSION" ]; then
if is_dry_run; then
echo "# WARNING: VERSION pinning is not supported in DRY_RUN"
else
pkg_pattern="$(echo "$VERSION" | sed "s/-ce-/\\\\.ce.*/g" | sed "s/-/.*/g").*$pkg_suffix"
search_command="$pkg_manager list --showduplicates 'docker-ce' | grep '$pkg_pattern' | tail -1 | awk '{print \$2}'"
pkg_version="$($sh_c "$search_command")"
echo "INFO: Searching repository for VERSION '$VERSION'"
echo "INFO: $search_command"
if [ -z "$pkg_version" ]; then
echo
echo "ERROR: '$VERSION' not found amongst $pkg_manager list results"
echo
exit 1
fi
if version_gte "18.09"; then
# older versions don't support a cli package
search_command="$pkg_manager list --showduplicates 'docker-ce-cli' | grep '$pkg_pattern' | tail -1 | awk '{print \$2}'"
cli_pkg_version="$($sh_c "$search_command" | cut -d':' -f 2)"
fi
# Cut out the epoch and prefix with a '-'
pkg_version="-$(echo "$pkg_version" | cut -d':' -f 2)"
fi
fi
(
if ! is_dry_run; then
set -x
fi
# install the correct cli version first
if [ -n "$cli_pkg_version" ]; then
$sh_c "$pkg_manager install -y -q docker-ce-cli-$cli_pkg_version"
fi
$sh_c "$pkg_manager install -y -q docker-ce$pkg_version"
if version_gte "20.10"; then
$sh_c "$pkg_manager install -y -q docker-ce-rootless-extras$pkg_version"
fi
)
echo_docker_as_nonroot
exit 0
;;
sles)
if [ "$(uname -m)" != "s390x" ]; then
echo "Packages for SLES are currently only available for s390x"
exit 1
fi
sles_repo="$DOWNLOAD_URL/linux/$lsb_dist/$REPO_FILE"
opensuse_repo="https://download.opensuse.org/repositories/security:SELinux/SLE_15_SP2/security:SELinux.repo"
if ! curl -Ifs "$sles_repo" > /dev/null; then
echo "Error: Unable to curl repository file $sles_repo, is it valid?"
exit 1
fi
pre_reqs="ca-certificates curl libseccomp2 awk"
(
if ! is_dry_run; then
set -x
fi
$sh_c "zypper install -y $pre_reqs"
$sh_c "zypper addrepo $sles_repo"
if ! is_dry_run; then
cat >&2 <<-'EOF'
WARNING!!
openSUSE repository (https://download.opensuse.org/repositories/security:SELinux) will be enabled now.
Do you wish to continue?
You may press Ctrl+C now to abort this script.
EOF
( set -x; sleep 30 )
fi
$sh_c "zypper addrepo $opensuse_repo"
$sh_c "zypper --gpg-auto-import-keys refresh"
$sh_c "zypper lr -d"
)
pkg_version=""
if [ -n "$VERSION" ]; then
if is_dry_run; then
echo "# WARNING: VERSION pinning is not supported in DRY_RUN"
else
pkg_pattern="$(echo "$VERSION" | sed "s/-ce-/\\\\.ce.*/g" | sed "s/-/.*/g")"
search_command="zypper search -s --match-exact 'docker-ce' | grep '$pkg_pattern' | tail -1 | awk '{print \$6}'"
pkg_version="$($sh_c "$search_command")"
echo "INFO: Searching repository for VERSION '$VERSION'"
echo "INFO: $search_command"
if [ -z "$pkg_version" ]; then
echo
echo "ERROR: '$VERSION' not found amongst zypper list results"
echo
exit 1
fi
search_command="zypper search -s --match-exact 'docker-ce-cli' | grep '$pkg_pattern' | tail -1 | awk '{print \$6}'"
# It's okay for cli_pkg_version to be blank, since older versions don't support a cli package
cli_pkg_version="$($sh_c "$search_command")"
pkg_version="-$pkg_version"

search_command="zypper search -s --match-exact 'docker-ce-rootless-extras' | grep '$pkg_pattern' | tail -1 | awk '{print \$6}'"
rootless_pkg_version="$($sh_c "$search_command")"
rootless_pkg_version="-$rootless_pkg_version"
fi
fi
(
if ! is_dry_run; then
set -x
fi
# install the correct cli version first
if [ -n "$cli_pkg_version" ]; then
$sh_c "zypper install -y docker-ce-cli-$cli_pkg_version"
fi
$sh_c "zypper install -y docker-ce$pkg_version"
if version_gte "20.10"; then
$sh_c "zypper install -y docker-ce-rootless-extras$rootless_pkg_version"
fi
)
echo_docker_as_nonroot
exit 0
;;
*)
if [ -z "$lsb_dist" ]; then
if is_darwin; then
echo
echo "ERROR: Unsupported operating system 'macOS'"
echo "Please get Docker Desktop from https://www.docker.com/products/docker-desktop"
echo
exit 1
fi
fi
echo
echo "ERROR: Unsupported distribution '$lsb_dist'"
echo
exit 1
;;
esac
exit 1
}

# wrapped up in a function so that we have some protection against only getting
# half the file during "curl | sh"
do_install

文件执行权限

chmod +x get-docker.sh

制作自定义镜像

1
2
3
4
5
6
7
8
9
[root@k8s-dns jenkins]# ll
total 28
-rw-r--r-- 1 root root 109 Feb 8 10:28 config.json
-rw-r--r-- 1 root root 347 Feb 8 10:40 Dockerfile
-rwxr-xr-x 1 root root 13857 Feb 8 10:34 get-docker.sh
-rw------- 1 root root 1679 Feb 8 10:31 id_rsa

[root@k8s-dns jenkins]# docker build . -t harbor.od.com/infra/jenkins:v2.306
[root@k8s-dns jenkins]# docker push harbor.od.com/infra/jenkins:v2.306

准备共享存储

运维主机,以及所有运算节点上:

1
# yum install nfs-utils -y

配置NFS服务

运维主机k8s-dns上:

1
2
[root@k8s-dns ~]# cat /etc/exports
/data/nfs-volume 10.1.1.0/24(rw,no_root_squash)

启动NFS服务

运维主机k8s-dns上:

1
2
3
4
5
mkdir -p /data/nfs-volume
systemctl start rpcbind
systemctl enable rpcbind
systemctl start nfs
systemctl enable nfs

准备资源配置清单

运维主机k8s-dns上:

1
mkdir /var/k8s-yaml/jenkins/ -p&& mkdir /data/nfs-volume/jenkins_home -p && cd /var/k8s-yaml/jenkins/

vim /var/k8s-yaml/jenkins/deployment.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
kind: Deployment
apiVersion: apps/v1
metadata:
name: jenkins
namespace: infra
labels:
name: jenkins
spec:
replicas: 1
selector:
matchLabels:
name: jenkins
template:
metadata:
labels:
app: jenkins
name: jenkins
spec:
volumes:
- name: data
nfs:
server: k8s-dns
path: /data/nfs-volume/jenkins_home
- name: docker
hostPath:
path: /run/docker.sock
type: ''
containers:
- name: jenkins
image: harbor.od.com/infra/jenkins:v2.306
ports:
- containerPort: 8080
protocol: TCP
env:
- name: JAVA_OPTS
value: -Xmx512m -Xms512m
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 500m
memory: 1Gi
volumeMounts:
- name: data
mountPath: /var/jenkins_home
- name: docker
mountPath: /run/docker.sock
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

vim /var/k8s-yaml/jenkins/svc.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
kind: Service
apiVersion: v1
metadata:
name: jenkins
namespace: infra
spec:
ports:
- protocol: TCP
port: 80
targetPort: 8080
selector:
app: jenkins
type: ClusterIP
sessionAffinity: None

vim /var/k8s-yaml/jenkins/ingress.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: jenkins
namespace: infra
spec:
rules:
- host: jenkins.od.com
http:
paths:
- path: /
backend:
serviceName: jenkins
servicePort: 80

应用资源配置清单

任意一个k8s运算节点上

1
2
3
4
5
[root@k8s-node01 ]# kubectl create namespace infra
kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=admin --docker-password=Harbor12345 -n infra
kubectl apply -f http://k8s-yaml.od.com/jenkins/deployment.yaml
kubectl apply -f http://k8s-yaml.od.com/jenkins/svc.yaml
kubectl apply -f http://k8s-yaml.od.com/jenkins/ingress.yaml

解析域名

运维主机k8s-dns上:

1
2
vim /var/named/chroot/etc/od.com.zone
jenkins 60 IN A 10.1.1.50

浏览器访问

http://jenkins.od.com

jenkins页面配置

初始化密码

1
cat /data/nfs-volume/jenkins_home/secrets/initialAdminPassword

安装Blue-Ocean插件

Manage Jenkins –> Manage Plugins –> Available –> Blue Ocean

调整安全选项

Manage Jenkins –> Configure Global Security –> Allow anonymous read access

配置New-job

  • 创建 new jobs

  • Enter an item name

    dubbo-demo

  • Pipeline -> OK

  • Discard old builds

    Days to keep builds : 3
    Max # of builds to keep : 30

  • This project is parameterized

  1. Add Parameter -> String Parameter

    Name : app_name
    Default Value :
    Description : 项目名称,例: dubbo-demo-service

  2. Add Parameter -> String Parameter

    Name : image_name
    Default Value :
    Description : 项目所需镜像,例: app/dubbo-demo-service

  3. Add Parameter -> String Parameter

    Name : git_repo
    Default Value :
    Description : 项目所需git中央仓库地址,例: https://gitee.com/dabou/dubbo-demo-service.git

  4. Add Parameter -> String Parameter

    Name : git_ver
    Default Value :
    Description : 项目在git中央仓库种对应的版本号或分支

  5. Add Parameter -> String Parameter

    Name : add_tag
    Default Value :
    Description : 创建项目docker镜像标签,例: 20210815_1111

  6. Add Parameter -> String Parameter

    Name : mvn_dir
    Default Value : ./
    Description : 编译项目的目录,默认为”根”。

  7. Add Parameter -> String Parameter

    Name : target_dir
    Default Value : ./target
    Description : 编译完成后产生的jar or war包所在目录 例: ./dubbo-server/target

  8. Add Parameter -> String Parameter

    Name : mvn_cmd
    Default Value : mvn clean package -Dmaven.test.skip=true
    Description : 编译命令 例: mvn clean package -e -q -Dmaven.test.skip=true

  9. Add Parameter -> Choice Parameter

    Name : base_image
    Default Value :

    • base/jre7:7u80
    • base/jre8:8u112
      Description : 项目所用的java版本镜像。
  10. Add Parameter -> Choice Parameter

    Name : maven
    Default Value :

    • 3.6.3-8u242
    • 3.2.5-6u025
    • 2.2.1-6u025
      Description : 编译时maven版本。

job名称

Discard old builds

app_name

image_name

git_repo

git_ver

add_tag

mvn_dir

target_dir

mvn_cmd

base_name

Pipeline-Script

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
pipeline {
agent any
stages {
stage('pull') { //get project code from repo
steps {
sh "git clone ${params.git_repo} ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.app_name}/${env.BUILD_NUMBER} && git checkout ${params.git_ver}"
}
}
stage('build') { //exec mvn cmd
steps {
sh "cd ${params.app_name}/${env.BUILD_NUMBER} && /var/jenkins_home/maven-${params.maven}/bin/${params.mvn_cmd}"
}
}
stage('package') { //move jar file into project_dir
steps {
sh "cd ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.target_dir} && mkdir project_dir && mv *.jar ./project_dir"
}
}
stage('image') { //build image and push to registry
steps {
writeFile file: "${params.app_name}/${env.BUILD_NUMBER}/Dockerfile", text: """FROM harbor.od.com/${params.base_image}
ADD ${params.target_dir}/project_dir /opt/project_dir"""
sh "cd ${params.app_name}/${env.BUILD_NUMBER} && docker build -t harbor.od.com/${params.image_name}:${params.git_ver}_${params.add_tag} . && docker push harbor.od.com/${params.image_name}:${params.git_ver}_${params.add_tag}"
}
}
}
}

最后的准备工作

检查jenkins容器里的SSH key

进入jenkins的docker容器里,检查ssh连接git仓库,确认是否能拉到代码。

1
2
3
[root@k8s-node02 ~]# docker exec -it k8s_jenkins_jenkins-f8bf49c46-cn5fw_infra_21ac5612-e195-4a73-8f23-8b1205079f82_1 bash
root@jenkins-f8bf49c46-cn5fw:~# ssh -T git@gitee.com
Hi wangxiansen! You've successfully authenticated, but GITEE.COM does not provide shell access.

部署maven软件

maven官方下载地址

运维主机k8s-dns上二进制部署,这里部署maven-3.6.3版:

1
2
[root@k8s-dns tools]# tar xf apache-maven-3.6.3-bin.tar.gz 
[root@k8s-dns tools]# mv apache-maven-3.6.3 /data/nfs-volume/jenkins_home/maven-3.6.3-8u242

设置国内镜像源

1
2
3
4
5
6
7
8
9
vim /data/nfs-volume/jenkins_home/maven-3.6.3-8u242/conf/settings.xml
...
<mirror>
<id>alimaven</id>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
<mirrorOf>central</mirrorOf>
</mirror>
...

制作dubbo微服务的底包镜像

运维主机k8s-dns

  • 自定义Dockerfile

vim /data/dockerfile/jre8/Dockerfile

1
2
3
4
5
6
7
8
FROM wangxiansen/jre8:8u112
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo 'Asia/Shanghai' >/etc/timezone
ADD config.yml /opt/prom/config.yml
ADD jmx_javaagent-0.3.1.jar /opt/prom/
WORKDIR /opt/project_dir
ADD entrypoint.sh /entrypoint.sh
CMD ["/entrypoint.sh"]

vim /data/dockerfile/jre8/config.yml

1
2
3
---
rules:
- pattern: '.*'
1
wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.3.1/jmx_prometheus_javaagent-0.3.1.jar -O jmx_javaagent-0.3.1.jar

vim /data/dockerfile/jre8/entrypoint.sh

1
2
3
4
5
6
7
8
#!/bin/sh
M_OPTS="-Duser.timezone=Asia/Shanghai -javaagent:/opt/prom/jmx_javaagent-0.3.1.jar=$(hostname -i):${M_PORT:-"12346"}:/opt/prom/config.yml"
C_OPTS=${C_OPTS}
JAR_BALL=${JAR_BALL}
exec java -jar ${M_OPTS} ${C_OPTS} ${JAR_BALL}

### 赋予执行权限
chmod +x /data/dockerfile/jre8/entrypoint.sh
  • 制作dubbo服务docker底包
1
2
3
4
5
[root@k8s-dns tools]# cd /data/dockerfile/jre8/
[root@k8s-dns jre8]# ls
config.yml Dockerfile entrypoint.sh jmx_javaagent-0.3.1.jar
[root@k8s-dns jre8]# docker build . -t harbor.od.com/base/jre8:8u112
[root@k8s-dns jre8]# docker push harbor.od.com/base/jre8:8u112

注意:jre7\12底包制作类似,这里略。要在harbor中创建base仓库。

交付dubbo微服务至kubernetes集群

dubbo服务提供者(dubbo-demo-service)

准备k8s资源配置清单

vim /var/k8s-yaml/dubbo-demo-service/deployment.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
kind: Deployment
apiVersion: apps/v1
metadata:
name: dubbo-demo-service
namespace: app
labels:
name: dubbo-demo-service
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-service
template:
metadata:
labels:
app: dubbo-demo-service
name: dubbo-demo-service
spec:
containers:
- name: dubbo-demo-service
image: harbor.od.com/app/dubbo-demo-service:master_202102142100
ports:
- containerPort: 20880
protocol: TCP
env:
- name: JAR_BALL
value: dubbo-server.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

应用资源配置清单

在任意一台k8s运算节点执行:

1
2
3
[root@k8s-node01 ~]# kubectl create namespace app
[root@k8s-node01 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-service/deployment.yaml
deployment.extensions/dubbo-demo-service created

查询deployment中pod

dubbo服务消费者(dubbo-demo-consumer)

解析域名

运维主机k8s-dns上:

1
2
3
[root@k8s-dns ~]# cat /var/named/chroot/etc/od.com.zone 
...
demo A 10.1.1.50

准备k8s资源配置清单

运维主机k8s-dns上:

1
mkdir /var/k8s-yaml/dubbo-demo-consumer &&cd /var/k8s-yaml/dubbo-demo-consumer

vim /var/k8s-yaml/dubbo-demo-consumer/deployment.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
kind: Deployment
apiVersion: apps/v1
metadata:
name: dubbo-demo-consumer
namespace: app
labels:
name: dubbo-demo-consumer
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-consumer
template:
metadata:
labels:
app: dubbo-demo-consumer
name: dubbo-demo-consumer
spec:
containers:
- name: dubbo-demo-consumer
image: harbor.od.com/app/dubbo-demo-consumer:master_1416
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
env:
- name: JAR_BALL
value: dubbo-client.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

vim /var/k8s-yaml/dubbo-demo-consumer/svc.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
kind: Service
apiVersion: v1
metadata:
name: dubbo-demo-consumer
namespace: app
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: dubbo-demo-consumer
clusterIP: None
type: ClusterIP
sessionAffinity: None

vim /var/k8s-yaml/dubbo-demo-consumer/ingress.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: app
spec:
rules:
- host: demo.od.com
http:
paths:
- path: /
backend:
serviceName: dubbo-demo-consumer
servicePort: 8080

应用资源配置清单

在任意一台k8s运算节点执行:

1
2
3
4
5
6
[root@k8s-node01 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-consumer/deployment.yaml
deployment.extensions/dubbo-demo-consumer created
[root@k8s-node01 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-consumer/svc.yaml
service/dubbo-demo-consumer created
[root@k8s-node01 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-consumer/ingress.yaml
ingress.extensions/dubbo-demo-consumer created

浏览器访问

http://demo.od.com/hello?name=www.boysec.cn

dubbo-monitor工具

首先需要注意的一点是:Simple Monitor挂掉不会影响到Consumer和Provider之间的调用,所以用于生产环境不会有风险。配置好了之后可以结合admin管理后台使用,可以清晰的看到服务的访问记录成功次数失败次数等…

dubbo-monitor源码包

准备docker镜像

下载源码

运维主机k8s-dns上:

1
2
3
4
5
[root@k8s-dns ~]# cd /server/tools/
[root@k8s-dns tools]# ls
apache-maven-3.6.3 apache-maven-3.6.3-bin.tar.gz dubbo-monitor-master.zip
[root@k8s-dns tools]# unzip -d dubbo-monitor-master.zip
[root@k8s-dns tools]# mv dubbo-monitor-master /data/dockerfile/

修改配置

1
2
3
4
5
6
7
8
9
10
11
12
[root@k8s-dns tools]# cat /data/dockerfile/dubbo-monitor-master/dubbo-monitor-simple/conf/dubbo_origin.properties 
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=simple-monitor
dubbo.application.owner=LinuxBoy
dubbo.registry.address=zookeeper://zk1.od.com:2181?backup=zk2.od.com:2181,zk3.od.com:2181
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.statistics.directory=/dubbo-monitor-simple/statistics
dubbo.log4j.file=logs/dubbo-monitor-simple.log
dubbo.log4j.level=WARN

制作镜像

  1. 准备环境
1
2
3
4
5
6
7
8
9
10
[root@k8s-dns ]# cd /data/dockerfile/dubbo-monitor-master/
[root@k8s-dns ]# sed -r -i -e '/^nohup/{p;:a;N;$!ba;d}' ./dubbo-monitor-simple/bin/start.sh && sed -r -i -e "s%^nohup(.*)%exec \1%" ./dubbo-monitor-simple/bin/start.sh

## 默认运行内存为2g,酌情修改!
[root@k8s-dns ]# vim ./dubbo-monitor-simple/bin/start.sh
if [ -n "$BITS" ]; then
JAVA_MEM_OPTS=" -server -Xmx128m -Xms128m -Xmn32m -XX:PermSize=16m -Xss254k -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:LargePageSizeInBytes=64m -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 "
else
JAVA_MEM_OPTS=" -server -Xms128m -Xmx128m -XX:PermSize=16m -XX:SurvivorRatio=2 -XX:+UseParallelGC "
fi
  1. 准备Dockerfile
1
2
3
4
5
[root@k8s-dns dubbo-monitor-master]# cat Dockerfile 
FROM jeromefromcn/docker-alpine-java-bash
MAINTAINER Jerome Jiang
COPY dubbo-monitor-simple/ /dubbo-monitor-simple/
CMD /dubbo-monitor-simple/bin/start.sh
  1. build镜像
1
2
[root@k8s-dns dubbo-monitor-master]# docker build . -t harbor.od.com/infra/dubbo-monitor:latest
[root@k8s-dns dubbo-monitor-master]# docker push harbor.od.com/infra/dubbo-monitor:latest

解析域名

运维主机k8s-dns上:

1
2
3
[root@k8s-dns dubbo-monitor-master]# cat /var/named/chroot/etc/od.com.zone 
...
dubbo-monitor A 10.1.1.50

准备k8s资源配置清单

运维主机k8s-dns上:

1
mkdir /var/k8s-yaml/dubbo-monitor/

vim /var/k8s-yaml/dubbo-monitor/deployment.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
kind: Deployment
apiVersion: apps/v1
metadata:
name: dubbo-monitor
namespace: infra
labels:
name: dubbo-monitor
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-monitor
template:
metadata:
labels:
app: dubbo-monitor
name: dubbo-monitor
spec:
containers:
- name: dubbo-monitor
image: harbor.od.com/infra/dubbo-monitor:latest
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

vim /var/k8s-yaml/dubbo-monitor/svc.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
kind: Service
apiVersion: v1
metadata:
name: dubbo-monitor
namespace: infra
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: dubbo-monitor
clusterIP: None
type: ClusterIP
sessionAffinity: None

vim /var/k8s-yaml/dubbo-monitor/ingress.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: dubbo-monitor
namespace: infra
spec:
rules:
- host: dubbo-monitor.od.com
http:
paths:
- path: /
backend:
serviceName: dubbo-monitor
servicePort: 8080

应用资源配置清单

在任意一台k8s运算节点执行:

1
2
3
4
5
6
[root@k8s-node01 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/deployment.yaml
deployment.extensions/dubbo-monitor created
[root@k8s-node01 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/svc.yaml
service/dubbo-monitor created
[root@k8s-node01 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/ingress.yaml
ingress.extensions/dubbo-monitor created

浏览器访问

http://dubbo-monitor.od.com/