Garage对象存储介绍

garage
介绍

• garage
  是基于Amazon S3
  协议采用rust
  开发的支持geo-distribute
  的对象存储，基本的功能和minio
  类似。在多种环境下能够很好的集成以实现超融合基础设施;针对网络故障、网络延迟、磁盘故障、系统管理故障有较高的弹性。其次在使用上简单易懂、操作简单、调试也很方便

• garage
  目前实现的是对象存储的服务端、因此使用需要配合对象存储的客户端，目前支持多种客户端

garage
部署

• 编译garage
  源码部署到系统目录

// rust基本环境
[root@AlmaLinux-9:~]$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

// 源码clone
[perrynzhou@AlmaLinux-9:~/Debug]$ git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git

// checkout 到最新稳定版本
[perrynzhou@AlmaLinux-9:~/Debug/garage]$ git checkout v0.9.0

// 编译二进制程序
[perrynzhou@AlmaLinux-9:~/Debug/garage]$ cargo build --release

// 拷贝到bin目录
[perrynzhou@AlmaLinux-9:~/Debug/garage]$ sudo cp target/release/garage /usr/local/bin/garage

启动garage
服务

• 生成默认配置文件

/*********************生成garage配置文件*****************/
cat > garage.toml <<EOF
metadata_dir = "/tmp/meta"
data_dir = "/tmp/data"
db_engine = "lmdb"

replication_mode = "none"

rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"
rpc_secret = "$(openssl rand -hex 32)"

[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = ".s3.garage.localhost"

[s3_web]
bind_addr = "[::]:3902"
root_domain = ".web.garage.localhost"
index = "index.html"

[k2v_api]
api_bind_addr = "[::]:3904"

[admin]
api_bind_addr = "0.0.0.0:3903"
admin_token = "$(openssl rand -base64 32)"
EOF

/****************************************************/
[perrynzhou@AlmaLinux-9:~]$ cat garage.toml 
metadata_dir = "/tmp/meta"
data_dir = "/tmp/data"
db_engine = "lmdb"

replication_mode = "none"

rpc_bind_addr = "0.0.0.0:3901"
rpc_public_addr = "0.0.0.0:3901"
rpc_secret = "1aef5c898d0512fb53f58c8f825b48c5f6fbb8f7ab5da15560d754f38ffeddda"

[s3_api]
s3_region = "garage"
api_bind_addr = "0.0.0.0:3900"
root_domain = ".s3.garage.localhost"

[s3_web]
bind_addr = "0.0.0.0:3902"
root_domain = ".web.garage.localhost"
index = "index.html"

[k2v_api]
api_bind_addr = "0.0.0.0:3904"

[admin]
api_bind_addr = "0.0.0.0:3903"
admin_token = "111qOrpIN3UlG31LSMS2NrY9xAHpNLXz1NjJ+BMT6o0="

• 启动garage
  服务

[perrynzhou@AlmaLinux-9:~]$ garage  -c ./garage.toml  server

• 服务状态检查

[perrynzhou@AlmaLinux-9:~]$ garage -c ./garage.toml status
==== HEALTHY NODES ====
ID                Hostname       Address         Tags              Zone  Capacity  DataAvail
48e9d7ff8b3b3ffe  AlmaLinux-9.2  127.0.0.1:3901  NO ROLE ASSIGNED
[perrynzhou@AlmaLinux-9:~]$ 

• 设置AZ名称同时设置AZ可用的磁盘空间

[perrynzhou@AlmaLinux-9:~]$ export GARAGE_CONFIG_FILE=./garage.toml 
[perrynzhou@AlmaLinux-9:~]$ garage layout assign -z dc1 -c 1G 48e9d7ff8b3b3ffe
Role changes are staged but not yet commited.
Use `garage layout show` to view staged role changes,
and `garage layout apply` to enact staged changes.
[perrynzhou@AlmaLinux-9:~]$ garage layout show
==== CURRENT CLUSTER LAYOUT ====
No nodes currently have a role in the cluster.
See `garage status` to view available nodes.

Current cluster layout version: 0

==== STAGED ROLE CHANGES ====
ID                Tags  Zone  Capacity
48e9d7ff8b3b3ffe        dc1   1000.0 MB


==== NEW CLUSTER LAYOUT AFTER APPLYING CHANGES ====
ID                Tags  Zone  Capacity   Usable capacity
48e9d7ff8b3b3ffe        dc1   1000.0 MB  1000.0 MB (100.0%)

Zone redundancy: maximum

==== COMPUTATION OF A NEW PARTITION ASSIGNATION ====

Partitions are replicated 1 times on at least 1 distinct zones.

Optimal partition size:                     3.9 MB
Usable capacity / total cluster capacity:   1000.0 MB / 1000.0 MB (100.0 %)
Effective capacity (replication factor 1):  1000.0 MB

dc1                 Tags  Partitions        Capacity   Usable capacity
  48e9d7ff8b3b3ffe        256 (256 new)     1000.0 MB  1000.0 MB (100.0%)
  TOTAL                   256 (256 unique)  1000.0 MB  1000.0 MB (100.0%)


To enact the staged role changes, type:

    garage layout apply --version 1

You can also revert all proposed changes with: garage layout revert --version 1
[perrynzhou@AlmaLinux-9:~]$ garage layout apply --version 1
==== COMPUTATION OF A NEW PARTITION ASSIGNATION ====

Partitions are replicated 1 times on at least 1 distinct zones.

Optimal partition size:                     3.9 MB
Usable capacity / total cluster capacity:   1000.0 MB / 1000.0 MB (100.0 %)
Effective capacity (replication factor 1):  1000.0 MB

dc1                 Tags  Partitions        Capacity   Usable capacity
  48e9d7ff8b3b3ffe        256 (256 new)     1000.0 MB  1000.0 MB (100.0%)
  TOTAL                   256 (256 unique)  1000.0 MB  1000.0 MB (100.0%)


New cluster layout with updated role assignment has been applied in cluster.
Data will now be moved around between nodes accordingly.
[perrynzhou@AlmaLinux-9:~]$ 

garage
创建bucket

[perrynzhou@AlmaLinux-9:~]$ garage bucket create first-bucket
Bucket first-bucket was created.
[perrynzhou@AlmaLinux-9:~]$ garage bucket list
List of buckets:
  first-bucket    8ee51693e54280dad91a7ab23a3bb77f141235551b4bdd60af3a1054287cb59e
[perrynzhou@AlmaLinux-9:~]$ garage  bucket info first-bucket
Bucket: 8ee51693e54280dad91a7ab23a3bb77f141235551b4bdd60af3a1054287cb59e

Size: 0 B (0 B)
Objects: 0
Unfinished uploads (multipart and non-multipart): 0
Unfinished multipart uploads: 0
Size of unfinished multipart uploads: 0 B (0 B)

Website access: false

Global aliases:
  first-bucket

Key-specific aliases:

Authorized keys:

garage
创建key

[perrynzhou@AlmaLinux-9:~]$ garage key create first-app-key
Key name: first-app-key
Key ID: GKa51f42bf3976c90acc8289d5
Secret key: e23f876a2c8ca38e86905a6781ddc67d1304263f834dd1f297198752346f7ee9
Can create buckets: false

Key-specific bucket aliases:

Authorized buckets:
[perrynzhou@AlmaLinux-9:~]$ garage key list
List of keys:
  GKa51f42bf3976c90acc8289d5  first-app-key
[perrynzhou@AlmaLinux-9:~]$ garage key info first-app-key
Key name: first-app-key
Key ID: GKa51f42bf3976c90acc8289d5
Secret key: (redacted)
Can create buckets: false

Key-specific bucket aliases:

Authorized buckets:
[perrynzhou@AlmaLinux-9:~]$ garage bucket allow \
  --read \
  --write \
  --owner \
 first-bucket \
  --key first-app-key
New permissions for GKa51f42bf3976c90acc8289d5 on first-bucket: read true, write true, owner true.


[perrynzhou@AlmaLinux-9:~]$   garage bucket info first-bucket
Bucket: 8ee51693e54280dad91a7ab23a3bb77f141235551b4bdd60af3a1054287cb59e

Size: 0 B (0 B)
Objects: 0
Unfinished uploads (multipart and non-multipart): 0
Unfinished multipart uploads: 0
Size of unfinished multipart uploads: 0 B (0 B)

Website access: false

Global aliases:
  first-bucket

Key-specific aliases:

Authorized keys:
  RWO  GKa51f42bf3976c90acc8289d5  first-app-key

测试

• 编译s3客户端

[perrynzhou@perrynzhouMBP:~/temp/minio]$ cd ..
[perrynzhou@perrynzhouMBP:~/temp]$ git clone https://github.com/minio/mc.git
Cloning into 'mc'...
remote: Enumerating objects: 32722, done.
remote: Counting objects: 100% (5419/5419), done.
remote: Compressing objects: 100% (948/948), done.
remote: Total 32722 (delta 4818), reused 4551 (delta 4471), pack-reused 27303
Receiving objects: 100% (32722/32722), 19.68 MiB | 728.00 KiB/s, done.
Resolving deltas: 100% (22419/22419), done.
[perrynzhou@perrynzhouMBP:~/temp]$ cd mc/
[perrynzhou@perrynzhouMBP:~/temp/mc]$ go mod tidy
[perrynzhou@perrynzhouMBP:~/temp/mc]$ make

• mc客户端配置

/**********************mc设置语法*******************/
mc alias set \
  garage \
  <endpoint> \
  <access key> \
  <secret key> \
  --api S3v4
/**********************mc设置*******************/
[perrynzhou@perrynzhouMBP:~/temp/mc]$ ././mc alias set \
>   garage \
>   http://10.211.55.4:3900 \
>   GKa51f42bf3976c90acc8289d5\
>   e23f876a2c8ca38e86905a6781ddc67d1304263f834dd1f297198752346f7ee9 \
>   --api S3v4
Added `garage` successfully.
[perrynzhou@perrynzhouMBP:~/temp/mc]$ ./mc ls garage
[2023-11-22 10:42:50 CST]     0B first-bucket/

[perrynzhou@perrynzhouMBP:~/temp/mc]$ ./mc cp 1.data.tar.gz  garage/first-bucket/1.data
...nzhou/temp/mc/1.data.tar.gz: 45.79 MiB / 45.79 MiB ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.18 MiB/s 0s


[perrynzhou@perrynzhouMBP:~/temp/mc]$ ./mc cp cmd.tar.gz  garage/first-bucket/
...rrynzhou/temp/mc/cmd.tar.gz: 419.28 KiB / 419.28 KiB ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.74 MiB/s 0s
[perrynzhou@perrynzhouMBP:~/temp/mc]$ 

官方测试性能

• 目前测试采用了和minio
  的性能对比，分别在简单的网络环境和复杂多变的网络环境进行性能对比

• 简单的网络环境下garage
  和minio
  性能对比

• 复杂的网络环境下garage
  和minio
  性能对比