【k8s】RuoYi-Cloud部署
周末没事干，搞一下
·
项目
友情链接: 若依 / RuoYi-Cloud
项目模块如下：
com.ruoyi
├── ruoyi-ui // 前端框架 [80]
├── ruoyi-gateway // 网关模块 [8080]
├── ruoyi-auth // 认证中心 [9200]
├── ruoyi-api // 接口模块
│ └── ruoyi-api-system // 系统接口
├── ruoyi-common // 通用模块
│ └── ruoyi-common-core // 核心模块
│ └── ruoyi-common-datascope // 权限范围
│ └── ruoyi-common-datasource // 多数据源
│ └── ruoyi-common-log // 日志记录
│ └── ruoyi-common-redis // 缓存服务
│ └── ruoyi-common-security // 安全模块
│ └── ruoyi-common-swagger // 系统接口
├── ruoyi-modules // 业务模块
│ └── ruoyi-system // 系统模块 [9201]
│ └── ruoyi-gen // 代码生成 [9202]
│ └── ruoyi-job // 定时任务 [9203]
│ └── ruoyi-file // 文件服务 [9300]
├── ruoyi-visual // 图形化管理模块
│ └── ruoyi-visual-monitor // 监控中心 [9100]
├──pom.xml // 公共依赖
架构图
内置功能
- 用户管理：用户是系统操作者，该功能主要完成系统用户配置。
- 部门管理：配置系统组织机构（公司、部门、小组），树结构展现支持数据权限。
- 岗位管理：配置系统用户所属担任职务。
- 菜单管理：配置系统菜单，操作权限，按钮权限标识等。
- 角色管理：角色菜单权限分配、设置角色按机构进行数据范围权限划分。
- 字典管理：对系统中经常使用的一些较为固定的数据进行维护。
- 参数管理：对系统动态配置常用参数。
- 通知公告：系统通知公告信息发布维护。
- 操作日志：系统正常操作日志记录和查询；系统异常信息日志记录和查询
- 登录日志：系统登录日志记录查询包含登录异常。
- 在线用户：当前系统中活跃用户状态监控。
- 定时任务：在线（添加、修改、删除)任务调度包含执行结果日志。
- 代码生成：前后端代码的生成（java、html、xml、sql）支持CRUD下载 。
- 系统接口：根据业务代码自动生成相关的api接口文档。
- 服务监控：监视当前系统CPU、内存、磁盘、堆栈等相关信息。
- 在线构建器：拖动表单元素生成相应的HTML代码。
- 连接池监视：监视当前系统数据库连接池状态，可进行分析SQL找出系统性能瓶颈。
环境信息
搭建k8s并了解相关基础
此次环境为本地虚拟机搭建（穷，用不起云）
#节点
[root@master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master01 Ready control-plane,master 1d v1.20.4
worker01 Ready worker 1d v1.20.4
worker02 Ready worker 1d v1.20.4
#版本信息
[root@master01 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:12:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:03:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
部署中间件
因为是虚拟服务器，所以资源不太够用，所以只部署了mysql、redis、nacos。
mysql
准备挂载配置文件
[client]
default-character-set=utf8mb4
[mysql]
default-character-set=utf8mb4
[mysqld]
init_connect='SET collation_connection = utf8mb4_unicode_ci'
init_connect='SET NAMES utf8mb4'
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
skip-character-set-client-handshake
skip-name-resolve
lower_case_table_names=1
k8s-mysql部署文件
kind: StatefulSet
apiVersion: apps/v1
metadata:
name: ops-mysql
namespace: devops
labels:
app: ops-mysql
spec:
replicas: 1
selector:
matchLabels:
app: ops-mysql
template:
metadata:
labels:
app: ops-mysql
spec:
volumes:
- name: host-time
hostPath:
path: /etc/localtime
type: ''
- name: mysql-conf
configMap:
name: mysql-conf
containers:
image: 'mysql:5.7'
ports:
- name: tcp-3306
containerPort: 3306
protocol: TCP
- name: tcp-33060
containerPort: 33060
protocol: TCP
env:
- name: MYSQL_ROOT_PASSWORD
value: jsepc01! #密码
resources:
limits:
cpu: '1'
memory: 2000Mi
volumeMounts:
- name: host-time #同步主机时区
mountPath: /etc/localtime
- name: mysql-pvc # 默认存储nfs
mountPath: /var/lib/mysql
- name: mysql-conf
readOnly: true #只读
mountPath: /etc/mysql/conf.d
imagePullPolicy: IfNotPresent
volumeClaimTemplates:
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: mysql-pvc
namespace: devops
spec:
accessModes:
- ReadWriteOnce #单点读写
resources:
requests:
storage: 5Gi
storageClassName: nfs-storage
volumeMode: Filesystem
status:
phase: Pending
---
kind: Service
apiVersion: v1
metadata:
name: ops-msyql-cp
namespace: devops
labels:
app: ops-msyql-cp
spec:
ports:
- name: http-3306
protocol: TCP
port: 3306
targetPort: 3306
selector:
app: ops-mysql
type: ClusterIP
查看日志无报错（略），验证ops-msyql-cp(ClusterIP) service
[root@master01 ~]# kubectl exec -it ops-mysql-0 -n devops bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@ops-mysql-0:/# mysql -uroot -h ops-msyql-cp.devops -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.36 MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> select version();
+-----------+
| version() |
+-----------+
| 5.7.36 |
+-----------+
1 row in set (0.00 sec)
为方便导入ruoyi框架中的nacos配置文件的数据脚本，另外建立svc(NodePort)
kind: Service
apiVersion: v1
metadata:
name: ops-mysql-svc
namespace: devops
labels:
app: ops-mysql-svc
spec:
ports:
- name: http-3306
protocol: TCP
port: 3306
targetPort: 3306
nodePort: 32480
selector:
app: ops-mysql
type: NodePort
Navicat配置连接尝试访问：成功
redis
配置文件
appendonly yes
port 6379
bind 0.0.0.0
k8s 执行文件
kind: StatefulSet
apiVersion: apps/v1
metadata:
name: ops-redis
namespace: devops
labels:
app: ops-redis
spec:
replicas: 1
selector:
matchLabels:
app: ops-redis
template:
metadata:
labels:
app: ops-redis
spec:
volumes:
- name: host-time
hostPath:
path: /etc/localtime
type: ''
- name: volume-redis
configMap:
name: redis-conf
containers:
image: redis
command:
- redis-server # 启动指定配置文件
args:
- /etc/redis/redis.conf
ports:
- name: tcp-6379
containerPort: 6379
protocol: TCP
resources:
limits:
cpu: '1'
memory: 1000Mi
volumeMounts:
- name: host-time
mountPath: /etc/localtime
- name: redis-pvc
mountPath: /data
- name: volume-redis
readOnly: true
mountPath: /etc/redis/
imagePullPolicy: IfNotPresent
volumeClaimTemplates:
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: redis-pvc
namespace: devops
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 2Gi
storageClassName: nfs-storage
volumeMode: Filesystem
status:
phase: Pending
---
kind: Service
apiVersion: v1
metadata:
name: ops-redis-cp
namespace: devops
labels:
app: ops-redis-cp
spec:
ports:
- name: http-6379
protocol: TCP
port: 6379
targetPort: 6379
selector:
app: ops-redis
type: ClusterIP
---
kind: Service
apiVersion: v1
metadata:
name: ops-redis-svc
namespace: devops
labels:
app: ops-redis-svc
spec:
ports:
- name: http-6379
protocol: TCP
port: 6379
targetPort: 6379
nodePort: 30230
selector:
app: ops-redis
type: NodePort
测试结果
nacos
配置中心和注册中心
导入项目sql脚本，绿色标注为nacos配置数据。
挂载配置文件，指定数据源
spring.datasource.platform=mysql
db.num=1
db.url.0=jdbc:mysql://ops-msyql-cp.devops:3306/ry-config?serverTimezone=GMT%2B8&characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=root
db.password=jsepc01!
建议提前下载吧，用的阿里镜像加速，服了这个老六了
换了别人的低版本镜像：nacos:v2.0.3，先用着吧，不行再换（非官方镜像）
kind: StatefulSet
apiVersion: apps/v1
metadata:
name: ops-nacos-v1
namespace: devops
labels:
app: ops-nacos
version: v1
spec:
replicas: 1
selector:
matchLabels:
app: ops-nacos
version: v1
template:
metadata:
labels:
app: ops-nacos
version: v1
spec:
volumes:
- name: host-time
hostPath:
path: /etc/localtime
type: ''
- name: nacos-conf
configMap:
name: nacos-conf
items:
- key: application.properties
path: application.properties
containers:
image: nacos/nacos-server
ports:
- name: http-8848
containerPort: 8848
protocol: TCP
env:
- name: MODE
value: standalone #单节点
volumeMounts:
- name: host-time
readOnly: true
mountPath: /etc/localtime
- name: nacos-conf
readOnly: true
mountPath: /home/nacos/conf/application.properties
subPath: application.properties
livenessProbe: #存活探针
httpGet:
path: /nacos
port: 8848
scheme: HTTP
initialDelaySeconds: 10
timeoutSeconds: 3
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
imagePullPolicy: IfNotPresent
---
kind: Service
apiVersion: v1
metadata:
name: ops-nacos
namespace: devops
labels:
app: ops-nacos
version: v1
spec:
ports:
- name: http-8848
protocol: TCP
port: 8848
targetPort: 8848
selector:
app: ops-nacos
type: ClusterIP
---
kind: Service
apiVersion: v1
metadata:
name: ops-nacos-np
namespace: devops
labels:
app: ops-nacos-np
spec:
ports:
- name: http-8848
protocol: TCP
port: 8848
targetPort: 8848
nodePort: 30834
selector:
app: ops-nacos
version: v1
type: NodePort
页面访问：查看到了数据库中导入的配置文件
至此，中间件部署完毕
[root@worker01 ~]# kubectl get po -n devops
NAME READY STATUS RESTARTS AGE
ops-mysql-0 1/1 Running 0 1h
ops-nacos-v1-0 1/1 Running 0 16m
ops-redis-0 1/1 Running 0 1h
[root@worker01 ~]# kubectl get svc -n devops
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ops-msyql-cp ClusterIP None <none> 3306/TCP 1h
ops-mysql-svc NodePort 10.233.41.25 <none> 3306:32480/TCP 1h
ops-nacos ClusterIP None <none> 8848/TCP 18m
ops-nacos-np NodePort 10.233.58.35 <none> 8848:30834/TCP 18m
ops-redis-cp ClusterIP None <none> 6379/TCP 1h
ops-redis-svc NodePort 10.233.8.230 <none> 6379:30230/TCP 1h
部署微服务
后台
maven打包（略）
编写Dockfile文件，为了省事，使用了同一个
FROM openjdk:8-jdk #基础镜像
#指定dev,指定nacos注册地址&配置中心地址
ENV PARAMS="--server.port=8080 --spring.profiles.active=dev --spring.cloud.nacos.discovery.server-addr=ops-nacos.devops:8848 --spring.cloud.nacos.config.server-addr=ops-nacos.devops:8848 --spring.cloud.nacos.config.namespace=public --spring.cloud.nacos.config.file-extension=yml"
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo 'Asia/Shanghai' >/etc/timezone
# 统一改名app.jar
COPY *.jar /app.jar
EXPOSE 8080
#传入环境变量
ENTRYPOINT ["/bin/sh","-c","java -Dfile.encoding=utf8 -Djava.security.egd=file:/dev/./urandom -jar app.jar ${PARAMS}"]
利用工具上传相应文件
打包镜像并上传阿里镜像仓库
[root@worker01 ~]# docker login --username=shenshuo9527 registry.cn-hangzhou.aliyuncs.com
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[root@worker01 ~]# cd ruoyi/auth/
[root@worker01 auth]# ls
Dockerfile ruoyi-auth.jar
[root@worker01 auth]# docker build -t registry.cn-hangzhou.aliyuncs.com/shuos_k8s/ruoyi_auth:v1 .
Sending build context to Docker daemon 89.85MB
Step 1/6 : FROM openjdk:8-jdk
8-jdk: Pulling from library/openjdk
0e29546d541c: Pull complete
9b829c73b52b: Pull complete
cb5b7ae36172: Pull complete
6494e4811622: Pull complete
668f6fcc5fa5: Pull complete
c0879393b07e: Pull complete
bef50c41a74d: Pull complete
Digest: sha256:8a9d5c43f540e8d0c003c723a2c8bd20ae350a2efed6fb5719cae33b026f8e7c
Status: Downloaded newer image for openjdk:8-jdk
---> e24ac15e052e
Step 2/6 : ENV PARAMS="--server.port=8080 --spring.profiles.active=dev --spring.cloud.nacos.discovery.server-addr=ops-nacos.devops:8848 --spring.cloud.nacos.config.server-addr=ops-nacos.devops:8848 --spring.cloud.nacos.config.namespace=public --spring.cloud.nacos.config.file-extension=yml"
---> Running in 51d1422312ff
Removing intermediate container 51d1422312ff
---> 76843ec452cb
Step 3/6 : RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo 'Asia/Shanghai' >/etc/timezone
---> Running in 1766cb6f2e33
Removing intermediate container 1766cb6f2e33
---> 68f1a5ce1118
Step 4/6 : COPY *.jar /app.jar
---> 792bf203d415
Step 5/6 : EXPOSE 8080
---> Running in 0f72b21d9034
Removing intermediate container 0f72b21d9034
---> 73ab928ec728
Step 6/6 : ENTRYPOINT ["/bin/sh","-c","java -Dfile.encoding=utf8 -Djava.security.egd=file:/dev/./urandom -jar app.jar ${PARAMS}"]
---> Running in e12259e3550d
Removing intermediate container e12259e3550d
---> defa12874125
Successfully built defa12874125
Successfully tagged registry.cn-hangzhou.aliyuncs.com/shuos_k8s/ruoyi_auth:v1
[root@worker01 auth]# docker push registry.cn-hangzhou.aliyuncs.com/shuos_k8s/ruoyi_auth:v1
The push refers to repository [registry.cn-hangzhou.aliyuncs.com/shuos_k8s/ruoyi_auth]
fac72c0d15d7: Pushed
d17f6142ff67: Pushed
bff9fe6e429c: Pushed
7c245b2fe4f1: Pushed
f9e18e59a565: Pushed
26a504e63be4: Pushed
8bf42db0de72: Pushed
31892cc314cb: Pushed
11936051f93b: Pushed
v1: digest: sha256:d35ddd9a3f0733e782892792c810032315cd9332b8af0073705fba6cd32d9505 size: 2214
修改nacos对应的配置文件
执行auth.yaml配置文件，无状态应用Deployment
kind: Deployment
apiVersion: apps/v1
metadata:
name: ops-ruoyi-auth-v1
namespace: devops
labels:
app: ops-ruoyi-auth
version: v1
spec:
replicas: 1
selector:
matchLabels:
app: ops-ruoyi-auth
version: v1
template:
metadata:
labels:
app: ops-ruoyi-auth
version: v1
spec:
volumes:
- name: host-time
hostPath:
path: /etc/localtime
type: ''
containers:
image: 'registry.cn-hangzhou.aliyuncs.com/shuos_k8s/ruoyi_auth:v1'
ports:
- name: tcp-8080
containerPort: 8080
protocol: TCP
resources:
limits:
memory: 1500Mi
volumeMounts:
- name: host-time
readOnly: true
mountPath: /etc/localtime
imagePullPolicy: IfNotPresent
观察日志确保成功无报错
[root@master01 ~]# kubectl logs ops-ruoyi-auth-v1-6d6f87bc79-s8l7v -n devops
Spring Boot Version: 2.7.2
Spring Application Name: ruoyi-auth
- - - - - -
- - - - - -
- 省略日志
- - - - - -
- - - - - -
14:21:23.555 [main] INFO c.a.c.s.SentinelWebMvcConfigurer - [addInterceptors,52] - [Sentinel Starter] register SentinelWebInterceptor with urlPatterns: [/**].
14:21:24.541 [main] WARN o.s.c.l.c.LoadBalancerCacheAutoConfiguration$LoadBalancerCaffeineWarnLogger - [logWarning,82] - Spring Cloud LoadBalancer is currently working with the default cache. While this cache implementation is useful for development and tests, it's recommended to use Caffeine cache in production.You can switch to using Caffeine cache, by adding it and org.springframework.cache.caffeine.CaffeineCacheManager to the classpath.
14:21:24.690 [main] INFO o.a.c.h.Http11NioProtocol - [log,173] - Starting ProtocolHandler ["http-nio-8080"]
14:21:24.735 [main] INFO c.a.c.n.r.NacosServiceRegistry - [register,75] - nacos registry, DEFAULT_GROUP ruoyi-auth 10.233.84.223:8080 register finished
14:21:24.773 [main] INFO c.r.a.RuoYiAuthApplication - [logStarted,61] - Started RuoYiAuthApplication in 11.492 seconds (JVM running for 12.344)
14:21:24.828 [main] INFO c.a.c.n.r.NacosContextRefresher - [registerNacosListenersForApplications,105] - listening config: dataId=ruoyi-auth.yml, group=DEFAULT_GROUP
14:21:24.828 [main] INFO c.a.c.n.r.NacosContextRefresher - [registerNacosListenersForApplications,105] - listening config: dataId=ruoyi-auth-dev.yml, group=DEFAULT_GROUP
14:21:24.828 [main] INFO c.a.c.n.r.NacosContextRefresher - [registerNacosListenersForApplications,105] - listening config: dataId=ruoyi-auth, group=DEFAULT_GROUP
(♥◠‿◠)ﾉﾞ 认证授权中心启动成功 ლ(´ڡ`ლ)ﾞ
.-------. ____ __
| _ _ \ \ \ / /
| ( ' ) | \ _. / '
|(_ o _) / _( )_ .'
| (_,_).' __ ___(_ o _)'
| |\ \ | || |(_,_)'
| | \ `' /| `-' /
| | \ / \ /
''-' `'-' `-..-'
nacos页面可以看到服务已经注册上去了
流程就是这样，我们把剩余的微服务模块一一部署上去
因虚拟机资源有限，所以只部署了以下模块
其中，系统模块使用了多数据源，报错找不到数据源，没追究是什么原因，可能是nacos的问题。后使用cm配置数据源挂载到jar包同级目录启动正常，特此说明。
UI部署
vue打包（略）
封装镜像并上传（略）
项目docker文件夹下有提供，可参考或者直接使用，nginx配置文件注意更改，指向网关
执行并访问页面
kind: Deployment
apiVersion: apps/v1
metadata:
name: ops-ruoyi-ui
namespace: devops
labels:
app: ops-ruoyi-ui
spec:
replicas: 1
selector:
matchLabels:
app: ops-ruoyi-ui
template:
metadata:
labels:
app: ops-ruoyi-ui
spec:
volumes:
- name: host-time
hostPath:
path: /etc/localtime
type: ''
containers:
image: 'registry.cn-hangzhou.aliyuncs.com/shuos_k8s/ruoyi_ui_nginx:v1'
ports:
- name: tcp-80
containerPort: 80
protocol: TCP
resources: {}
volumeMounts:
- name: host-time
readOnly: true
mountPath: /etc/localtime
imagePullPolicy: IfNotPresent
---
kind: Service
apiVersion: v1
metadata:
name: ops-ui-np
namespace: devops
labels:
app: ops-ui-np
spec:
ports:
- name: http-80
protocol: TCP
port: 80
targetPort: 80
nodePort: 30203
selector:
app: ops-ruoyi-ui
type: NodePort
不出意外的话就是出意外了，报错了。。。
查看认证服务模块日志
[root@worker01 nginx]# kubectl logs --tail 20 ops-ruoyi-auth-v1-6d6f87bc79-s8l7v -n devops
at io.lettuce.core.RedisClient.connect(RedisClient.java:216)
at org.springframework.data.redis.connection.lettuce.StandaloneConnectionProvider.lambda$getConnection$1(StandaloneConnectionProvider.java:115)
at java.util.Optional.orElseGet(Optional.java:267)
at org.springframework.data.redis.connection.lettuce.StandaloneConnectionProvider.getConnection(StandaloneConnectionProvider.java:115)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1595)
... 15 common frames omitted
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:6379
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:710)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
对比下我的nacos配置文件：
好了，我的nacos果然有问题。auth模块应该是启动的默认值localhost，跑起来了，真正请求连接时就异常了。重新使用cm挂载部署。
可以看到验证码了，说明认证服务正常了。
成功登录，系统正常。
尝试新增用户
成功，数据库有相应记录
总结：
- nacos只用了注册中心，使用配置中心的话我搭建的有点问题。
- 后台服务只部署了以下四个，不过都是大同小异而已。
- 另外网关服务也用挂载的方式重启了。
- 仅供参考
快速构建 Web 应用程序
更多推荐
5
0- 0
已为社区贡献1条内容
所有评论(0)