OceanBase on K8S 部署篇

晓磊聊DB 2024-10-31

一、OB-operator简介

ob-operator
是满足 Kubernetes Operator 扩展范式的自动化工具，可以极大简化在 Kubernetes 上部署和管理 OceanBase 集群及相关资源的过程。

ob-operator 支持 OceanBase 集群的管理、租户管理、备份恢复、故障恢复等功能，具体如下：

集群管理：集群自举、调整集群拓扑、支持 K8s 拓扑配置、扩缩容、集群升级、修改参数
租户管理：创建租户、调整租户拓扑、管理资源单元、修改用户密码
备份恢复：向 OSS 或 NFS 目的地周期性备份数据、从 OSS 或 NFS 中恢复数据
物理备库：从备份中恢复出备租户、创建空备租户、备租户升主、主备切换
故障恢复：单节点故障恢复，IP 保持情况下的集群故障恢复
Dashboard(GUI)：基于 ob-operator 的图形化 OceanBase 集群管理工具

二、部署OceanBase前置条件

在开始之前，请确保您已满足以下要求：

Kubernetes版本：>1.18版本
OceanBase版本：根据需求选择4.2.5/4.3.3
Kubernetes 资源：基于local-path-provisioner 来管理 PVC，并且nodes上创建目录生成的待绑定的local PV。
ob-operator 依赖 cert-manager，请确保您已安装 cert-manager。cert-manager 的安装方法请参考对应的安装文档。

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml


namespace/cert-manager created
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
.................
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created
..................
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
service/cert-manager-cainjector created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created

三、部署 ob-operator

下面以部署version=2.2.0版本的ob-operator，operator最新应该已经到2.3.0版本(建议使用该版本)

/opt/ob-operator# helm repo add ob-operator https://oceanbase.github.io/ob-operator/
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: root/.kube/config
"ob-operator" has been added to your repositories
/opt/ob-operator# helm install ob-operator ob-operator/ob-operator --namespace=oceanbase-system --create-namespace --version=2.2.0
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: root/.kube/config
NAME: ob-operator
LAST DEPLOYED: Mon Oct 28 10:53:12 2024
NAMESPACE: oceanbase-system
STATUS: deployed
REVISION: 1
TEST SUITE: None

部署过程依赖镜像，需要提前下载到自己公司的镜像中心，否则报错镜像拉取失败报错：

Events:
  Type     Reason   Age                   From     Message
  ----     ------   ----                  ----     -------
  Warning  Failed   31m (x60 over 165m)   kubelet  (combined from similar events): Failed to pull image "oceanbase/ob-operator:2.2.0": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp 199.16.158.190:443: connect: connection timed out
  Normal   Pulling  26m (x33 over 171m)   kubelet  Pulling image "oceanbase/ob-operator:2.2.0"
  Normal   BackOff  91s (x728 over 171m)  kubelet  Back-off pulling image "oceanbase/ob-operator:2.2.0"

查看ob-operator安装是否OK（READY：2/2且STATUS：Running）

/opt/ob-operator# kubectl get pods -n oceanbase-system
NAME                                            READY   STATUS    RESTARTS   AGE
oceanbase-controller-manager-85f4b9d58c-cl9rq   2/2     Running   0          3m27s

四、部署 OceanBase 集群

按照以下步骤在 Kubernetes 集群上部署 OceanBase 集群：

（1）新建namespace：kubectl create ns oceanbase-test

（2）创建 Secrets 在创建 OceanBase 集群之前，您必须创建特定用户所需的 secrets。创建的目的就是OB集群初始化密码，使用以下命令创建 secrets：

kubectl create secret generic ob-root-password --from-literal=password='yourpassword' -n oceanbase-test
kubectl create secret generic ob-proxyro-password --from-literal=password='yourpassword' -n oceanbase-test

PS：注意在创建ob-proxyro-password时建议不要带特殊字符，

否则可能出现部署obproxy时因权限链接不上后端Observer的问题

（3）编辑集群配置文件oceanbase-test.yaml

apiVersion: oceanbase.oceanbase.com/v1alpha1
kind: OBCluster
metadata:
  name: obcluste-test    #OB的kubectl get obcluster名
  namespace: oceanbase-test
spec:
  clusterName: obcluster-test   #OB集群名
  clusterId: 1
  userSecrets:
    root: ob-root-password
    proxyro: ob-proxyro-password
  topology:
    - zone: zone1    #可以自定义zone名称
      replica: 1     #一个zone的observer数量
      # nodeSelector:
      #   k1: v1
      # affinity:         亲和性设置，生产环境必备
      #   nodeAffinity:
      #   podAffinity:
      #   podAntiAffinity:
      # tolerations:      污点设定，生产环境必备
      #   - key: "obtopo"
      #     value: "zone"
      #     effect: "NoSchedule"
    - zone: zone2
      replica: 1
      # nodeSelector:
      #   k1: v1
      # affinity:
      #   nodeAffinity:
      #   podAffinity:
      #   podAntiAffinity:
      # tolerations:
      #   - key: "obtopo"
      #     value: "zone"
      #     effect: "NoSchedule"
    - zone: zone3
      replica: 1
      # nodeSelector:
      #   k1: v1
      # affinity:
      #   nodeAffinity:
      #   podAffinity:
      #   podAntiAffinity:
      # tolerations:
      #   - key: "obtopo"
      #     value: "zone"
      #     effect: "NoSchedule"
  observer:
    image: oceanbase/oceanbase-cloud-native:4.3.3.0-100000142024101215
    resource:
      cpu: 12
      memory: 40Gi
    storage:
      dataStorage:
        storageClass: local-ssd-storage
        size: 500Gi
      redoLogStorage:
        storageClass: local-ssd-storage
        size: 150Gi
      logStorage:
        storageClass: local-ssd-storage
        size: 20Gi
  monitor:
    image: oceanbase/obagent:4.2.1-100000092023101717
    resource:
      cpu: 8
      memory: 10Gi

（4）应用yaml，需要等待OceanBase集群创建，大概2分钟

kubectl apply -f oceanbase-test.yaml -n oceanbase-test

注：如果需要镜像问题记得下载镜像到自己公司仓库

oceanbase/oceanbase-cloud-native:4.3.3.0-100000142024101215


oceanbase/obagent:4.2.1-100000092023101717

（5）查看OB集群

/opt/ob-operator# kubectl get obcluster -n oceanbase-test1
NAME        STATUS    AGE
obcluster   running   3d2h
/opt/ob-operator# kubectl get pods -n oceanbase-test1 -o wide
NAME                                  READY   STATUS    RESTARTS   AGE    IP               NODE      NOMINATED NODE   READINESS GATES
obcluster-test-1-zone1-6a036c0d84bf   2/2     Running   0          3d2h   10.xxx.xxx.143   db02      <none>           <none>
obcluster-test-1-zone2-384f9bf55992   2/2     Running   0          3d2h   10.xxx.xxx.142   db08      <none>           <none>
obcluster-test-1-zone3-6f375db39867   2/2     Running   0          3d2h   10.xxx.xxx.242   db07      <none>           <none>
obproxy-test-6479ccb497-6r7xv         1/1     Running   0          2d6h   10.xxx.xxx.182    kube610   <none>           <none>
obproxy-test-6479ccb497-bnj48         1/1     Running   0          2d6h   10.xxx.xxx.227    kube600   <none>           <none>

（6）访问集群，因为没有部署obproxy，所以先直通过OB pods ip访问。

/opt/ob-operator# mysql -h10.xxx.xxx.143 -P2881 -uroot@sys -p oceanbase -A -c
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3221489701
Server version: 5.7.25 OceanBase_CE 4.3.3.0 (r100000142024101215-0723f9774e5f77a1253149261ba75a94414625d0) (Built Oct 12 2024 16:04:08)


Copyright (c) 2009-2023 Percona LLC and/or its affiliates
Copyright (c) 2000, 2023, Oracle and/or its affiliates.


Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.


Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.


mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| LBACSYS            |
| mysql              |
| oceanbase          |
| ORAAUDITOR         |
| SYS                |
| sys_external_tbs   |
| test               |
+--------------------+
8 rows in set (0.01 sec)


mysql> exit
Bye

五、总结

ob-operator 是基于 Kubernetes Operator 框架构建的工具，实现了声明式管理、自动化调谐和统一管理等优势，它提供简单可靠的方式实现 OceanBase 集群的容器化部署，简化OceanBase集群运维。

如果想更深入的了解ob-operator，可以参考以下资料

ob-operator文档：https://www.oceanbase.com/docs/community-ob-operator-doc-1000000000659335

ob-operator github文档：https://oceanbase.github.io/ob-operator/

集群yaml的详细配置：https://www.oceanbase.com/docs/community-ob-operator-doc-1000000000659352

文章转载自晓磊聊DB，如果涉嫌侵权，请发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

OceanBase on K8S 部署篇

一、OB-operator简介

二、部署OceanBase前置条件

三、部署 ob-operator

评论