【作者简介】
本人医疗行业DBA,精通oracle底层;会文件、触发器加密的勒索恢复、asm磁盘组不能mount及各种灾难恢复,后面将利用空闲时间将自己工作中遇到的问题及整理的资料分享给更多愿意学习和提升自己的dba。作者微信号:Oray_123
【正文】
1、ASM ADVM介绍
* 逻辑卷(ADVM)是11gR2引入的一项技术;* 简单理解advm,就是通过在现有ASM磁盘组上划分卷,再格式化成ACFS文件系统,挂载到文件系统层面进行使用;* 使用逻辑卷的目的就是为了让ASM磁盘组支持存放更多的文件,比如普通的文件、安装包、安装代码等等;* 前台有ADVM,那么后台必然有一个位置来进行存放,那么ADVM是存放在asm具体哪里的呢?* asm file 7号文件是asm存放逻辑卷目录的extent的分配表;通过分配表定位就能够找到具体ADVM存放的内容;* 一个磁盘组可以创建多个ADVM,ACFS通过ADVM接口构建在ASM磁盘组上运行;* 要看到ASM ADVM的元数据信息,必须先安装、配置acfs相关服务,以下是具体的实验:
2、定位ADVM
1)通过视图查询set line 200col diskgroup_name for a15col disk_name for a10col path for a40select a.GROUP_NUMBER,a.name diskgroup_name,a.type,a.allocation_unit_size/1024/1024 unit_mb,a.voting_files,b.name disk_name,b.path,b.os_mb/1024 as os_gb,b.total_mb/1024 as total_gb,b.free_mb/1024 as free_gbfrom v$asm_diskgroup a,v$asm_disk b where a.group_number=b.group_number;select number_kffxp file#, disk_kffxp disk#, count(disk_kffxp) extentsfrom x$kffxpwhere group_kffxp=1and disk_kffxp <> 65534group by number_kffxp, disk_kffxporder by 1;FILE# DISK# EXTENTS---------- ---------- ----------1 0 11 1 12 1 13 0 423 1 434 0 44 1 45 0 16 1 18 0 19 0 1通过查询,我们可以看到,并没有file#为7的行,说明没有对应的advm元数据信息,这是因为我们默认就没有启用acfs功能;set line 200 pages 1000col "disk name" for a20col path for a30col type for a20col au for 99999SELECT x.number_kffxp file#,x.xnum_kffxp "Extent",case lxn_kffxpwhen 0 then 'Primary Copy'when 1 then 'Mirrored Copy'when 2 then '2nd Mirrored Copy or metadata'else 'Unknown' end type,x.au_kffxp "AU",x.disk_kffxp "Disk #",d.name "Disk name",d.path,d.GROUP_NUMBER,d.DISK_NUMBERFROM x$kffxp x, v$asm_disk_stat dWHERE x.group_kffxp=d.group_numberand x.disk_kffxp=d.disk_numberand x.group_kffxp=1and x.number_kffxp=7ORDER BY 1, 2;安装acfs相关服务:[root@hisdb01 ~]# acfsdriverstate supportedACFS-9459: ADVM/ACFS is not supported on this OS version: 'centos-release-7-6.1810.2.el7.centos.x86_64'ACFS-9201: Not Supported[root@hisdb01 ~]# acfsroot installACFS-9459: ADVM/ACFS is not supported on this OS version: 'centos-release-7-6.1810.2.el7.centos.x86_64'[root@hisdb01 ~]# find u01 -name osds_acfslib.pm/u01/app/11.2.0/grid/lib/osds_acfslib.pm/u01/app/11.2.0/grid/.patch_storage/28729245_Nov_2_2018_04_47_07/files/lib/osds_acfslib.pm/u01/app/oracle/product/11.2.0/db_1/lib/osds_acfslib.pm[root@hisdb01 ~]# vi u01/app/11.2.0/grid/lib/osds_acfslib.pm# Testing for "oraclelinux_release", etc prevents us from dropping# into the code below from other RH based distros like CentOS, for example.if ((defined($release)) && # Redhat or OEL if defined(($release =~ /^redhat-release/) || # straight RH($release =~ /^enterprise-release/) || # Oracle Enterprise Linux($release =~ /^centos-release/) || # CentOS hack --添加这一行($release =~ /^oraclelinux-release/))) # Oracle Linux[root@hisdb01 ~]# acfsroot installACFS-9300: ADVM/ACFS distribution files found.ACFS-9307: Installing requested ADVM/ACFS software.ACFS-9308: Loading installed ADVM/ACFS drivers.ACFS-9321: Creating udev for ADVM/ACFS.ACFS-9323: Creating module dependencies - this may take some time.ACFS-9154: Loading 'oracleoks.ko' driver.ACFS-9154: Loading 'oracleadvm.ko' driver.ACFS-9154: Loading 'oracleacfs.ko' driver.ACFS-9327: Verifying ADVM/ACFS devices.ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.ACFS-9156: Detecting control device '/dev/ofsctl'.ACFS-9309: ADVM/ACFS installation correctness verified.启动acfs相关服务:[root@hisdb01 ~]# acfsload startACFS-9391: Checking for existing ADVM/ACFS installation.ACFS-9392: Validating ADVM/ACFS installation files for operating system.ACFS-9393: Verifying ASM Administrator setup.ACFS-9308: Loading installed ADVM/ACFS drivers.ACFS-9327: Verifying ADVM/ACFS devices.ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.ACFS-9156: Detecting control device '/dev/ofsctl'.ACFS-9322: completed[root@hisdb01 ~]# acfsdriverstate versionACFS-9325: Driver OS kernel version = 3.10.0-862.el7.x86_64(x86_64).ACFS-9326: Driver Oracle version = 181028.使用acfs:为advm创建一个单独的磁盘不是必须的,但是建议把数据文件和ACFS文件隔离开,以下是创建磁盘组后,在磁盘组上面再创建的逻辑卷,要求磁盘组的COMPATIBLE.ASM与COMPATIBLE.ADVM必须设置为11.2或以上,同时ADVM/ACFS驱动要被加载(在集群环境中,已经默认加载,而在单实例环境中,需要手动加载)。sqlplus as sysasmcreate diskgroup ACFS disk 'ORCL:ASMDISK5', 'ORCL:ASMDISK6' attribute 'COMPATIBLE.ASM' = '11.2', 'COMPATIBLE.ADVM' = '11.2';$ asmcmd volcreate -G ACFS -s 2G ACFS_VOL1$ asmcmd volcreate -G ACFS -s 2G ACFS_VOL2$ asmcmd volinfo -a我这里因为磁盘没有添加,选择在现有磁盘组上面创建:[grid@hisdb01 ~]$ asmcmd lsdgState Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files NameMOUNTED EXTERN N 512 4096 1048576 60000 57312 0 57312 0 N DATA/[grid@hisdb01 ~]$ asmcmd volcreate -G DATA -s 2G DATA_VOL1[grid@hisdb01 ~]$ asmcmd volcreate -G DATA -s 2G DATA_VOL2[grid@hisdb01 ~]$ asmcmd volinfo -aDiskgroup Name: DATAVolume Name: DATA_VOL1Volume Device: /dev/asm/data_vol1-467State: ENABLEDSize (MB): 2048Resize Unit (MB): 32Redundancy: UNPROTStripe Columns: 4Stripe Width (K): 128Usage:Mountpath:Volume Name: DATA_VOL2Volume Device: dev/asm/data_vol2-467State: ENABLEDSize (MB): 2048Resize Unit (MB): 32Redundancy: UNPROTStripe Columns: 4Stripe Width (K): 128Usage:Mountpath:创建完成后,还没有相应的挂载目录和逻辑卷相关联,我们通过查询发现已经从DATA磁盘组中剥离了4g出来了,如下:[grid@hisdb01 ~]$ asmcmd lsdgState Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files NameMOUNTED EXTERN N 512 4096 1048576 60000 53213 0 53213 0 N DATA/在逻辑卷上创建ACFS集群文件系统[root@hisdb01 ~]# mkfs -t acfs dev/asm/data_vol1-467mkfs.acfs: version = 11.2.0.4.0mkfs.acfs: on-disk version = 39.0mkfs.acfs: volume = /dev/asm/data_vol1-467mkfs.acfs: volume size = 2147483648mkfs.acfs: Format complete.[root@hisdb01 ~]# mkdir acfs1[root@hisdb01 ~]# chown -R oracle:oinstall acfs1[root@hisdb01 ~]# mount -t acfs dev/asm/data_vol1-467 acfs1[root@hisdb01 ~]# df -ThFilesystem Type Size Used Avail Use% Mounted on/dev/mapper/centos-root xfs 92G 29G 64G 32% /.../dev/asm/data_vol1-467 acfs 2.0G 43M 2.0G 3% /acfs1[grid@hisdb01 ~]$ asmcmd volinfo -G DATA DATA_VOL1Diskgroup Name: DATAVolume Name: DATA_VOL1Volume Device: dev/asm/data_vol1-467State: ENABLEDSize (MB): 2048Resize Unit (MB): 32Redundancy: UNPROTStripe Columns: 4Stripe Width (K): 128Usage: ACFSMountpath: /acfs1[root@hisdb01 acfs1]# cp /software/p13390677_112040_Linux-x86-64_1of7.zip .[root@hisdb01 acfs1]# lltotal 1364544drwx------ 2 root root 65536 Jul 20 16:14 lost+found-rw-r--r-- 1 root root 1395582860 Jul 20 16:36 p13390677_112040_Linux-x86-64_1of7.zip --可以看到普通的文件就能存放到这里面了;[root@hisdb01 acfs1]# pwd/acfs1select number_kffxp file#, disk_kffxp disk#, count(disk_kffxp) extentsfrom x$kffxpwhere group_kffxp=1and disk_kffxp <> 65534group by number_kffxp, disk_kffxp6 order by 1;FILE# DISK# EXTENTS---------- ---------- ----------1 0 11 1 12 1 13 0 423 1 434 0 44 1 45 0 16 1 17 0 1 --再次查看,确实有file# 7的信息了;set line 200 pages 1000col "disk name" for a20col path for a30col type for a20col au for 99999SELECT x.number_kffxp file#,x.xnum_kffxp "Extent",case lxn_kffxpwhen 0 then 'Primary Copy'when 1 then 'Mirrored Copy'when 2 then '2nd Mirrored Copy or metadata'else 'Unknown' end type,x.au_kffxp "AU",x.disk_kffxp "Disk #",d.name "Disk name",d.path,d.GROUP_NUMBER,d.DISK_NUMBERFROM x$kffxp x, v$asm_disk_stat dWHERE x.group_kffxp=d.group_numberand x.disk_kffxp=d.disk_numberand x.group_kffxp=1and x.number_kffxp=7ORDER BY 1, 2;

3、kfed 解析ADVM
3、kfed 解析ADVMset line 200col diskgroup_name for a15col disk_name for a10col path for a40select a.group_number,a.name diskgroup_name,a.type,a.allocation_unit_size/1024/1024 unit_mb,a.voting_files,b.name disk_name,b.path,b.os_mb/1024 as os_gb,b.total_mb/1024 as total_gb,b.free_mb/1024 as free_gb3 from v$asm_diskgroup a,v$asm_disk b where a.group_number=b.group_number;GROUP_NUMBER DISKGROUP_NAME TYPE UNIT_MB VO DISK_NAME PATH OS_GB TOTAL_GB FREE_GB------------ --------------- -------------------- ---------- -- ---------- ---------------------------------------- ---------- ---------- ----------1 DATA EXTERN 1 N DATA_0000 /dev/asmdisk4 29.296875 29.296875 25.98144531 DATA EXTERN 1 N DATA_0001 /dev/asmdisk5 29.296875 29.296875 25.9843752 FRA EXTERN 1 N FRA_0000 /dev/asmdisk6 9.765625 9.765625 8.41015625--先读取文件目录,是在10号au上[grid@hisdb01 ~]$ kfed read /dev/asmdisk4 aun=0 blkn=0|egrep -i "aus|f1b1locn"kfdhdb.ausize: 1048576 ; 0x0bc: 0x00100000kfdhdb.f1b1locn: 2 ; 0x0d4: 0x00000002[grid@hisdb01 ~]$ kfed read /dev/asmdisk4 aun=2 blkn=7 aus=1M|egrep "block|disk|au"|egrep -v "65535|4294967295"kfbh.block.blk: 7 ; 0x004: blk=7kfbh.block.obj: 1 ; 0x008: file=1kfffde[0].xptr.au: 1345 ; 0x4a0: 0x00000541 --分配信息在0号磁盘的1345AU上,和上面视图查询结果完全一致kfffde[0].xptr.disk: 0 ; 0x4a4: 0x0000--block 0.它只包含了ADVM逻辑卷的标记(++AVD_DG_NUMBER)[grid@hisdb01 ~]$ kfed read /dev/asmdisk4 aun=1345 aus=1M blkn=0|morekfbh.endian: 1 ; 0x000: 0x01kfbh.hard: 130 ; 0x001: 0x82kfbh.type: 22 ; 0x002: KFBTYP_VOLUMEDIRkfbh.datfmt: 1 ; 0x003: 0x01kfbh.block.blk: 0 ; 0x004: blk=0kfbh.block.obj: 7 ; 0x008: file=7kfbh.check: 1556606794 ; 0x00c: 0x5cc7ef4akfbh.fcn.base: 4666 ; 0x010: 0x0000123akfbh.fcn.wrap: 0 ; 0x014: 0x00000000kfbh.spare1: 0 ; 0x018: 0x00000000kfbh.spare2: 0 ; 0x01c: 0x00000000kffdnd.bnode.incarn: 1 ; 0x000: A=1 NUMM=0x0kffdnd.bnode.frlist.number: 4294967295 ; 0x004: 0xffffffffkffdnd.bnode.frlist.incarn: 0 ; 0x008: A=0 NUMM=0x0kffdnd.overfl.number: 1 ; 0x00c: 0x00000001kffdnd.overfl.incarn: 1 ; 0x010: A=1 NUMM=0x0kffdnd.parent.number: 0 ; 0x014: 0x00000000kffdnd.parent.incarn: 1 ; 0x018: A=1 NUMM=0x0kffdnd.fstblk.number: 0 ; 0x01c: 0x00000000kffdnd.fstblk.incarn: 1 ; 0x020: A=1 NUMM=0x0kfvvde.entry.incarn: 1 ; 0x024: A=1 NUMM=0x0kfvvde.entry.hash: 0 ; 0x028: 0x00000000kfvvde.entry.refer.number: 4294967295 ; 0x02c: 0xffffffffkfvvde.entry.refer.incarn: 0 ; 0x030: A=0 NUMM=0x0kfvvde.volnm: ++AVD_DG_NUMBER ; 0x034: length=15kfvvde.usage: ; 0x054: length=0kfvvde.dgname: ; 0x074: length=0kfvvde.clname: ; 0x094: length=0kfvvde.mountpath: ; 0x0b4: length=0kfvvde.drlinit: 0 ; 0x4b5: 0x00kfvvde.pad1: 0 ; 0x4b6: 0x0000kfvvde.volfnum.number: 0 ; 0x4b8: 0x00000000kfvvde.volfnum.incarn: 0 ; 0x4bc: 0x00000000kfvvde.drlfnum.number: 0 ; 0x4c0: 0x00000000kfvvde.drlfnum.incarn: 0 ; 0x4c4: 0x00000000kfvvde.volnum: 0 ; 0x4c8: 0x0000kfvvde.avddgnum: 467 ; 0x4ca: 0x01d3kfvvde.extentsz: 0 ; 0x4cc: 0x00000000kfvvde.volstate: 4 ; 0x4d0: D=0 C=0 R=1kfvvde.pad[0]: 0 ; 0x4d1: 0x00...--block 1,而真正的逻辑卷的信息其实位于block 1及以后:[grid@hisdb01 ~]$ kfed read /dev/asmdisk4 aun=1345 aus=1M blkn=1|morekfbh.endian: 1 ; 0x000: 0x01kfbh.hard: 130 ; 0x001: 0x82kfbh.type: 22 ; 0x002: KFBTYP_VOLUMEDIRkfbh.datfmt: 1 ; 0x003: 0x01kfbh.block.blk: 1 ; 0x004: blk=1kfbh.block.obj: 7 ; 0x008: file=7kfbh.check: 2199406070 ; 0x00c: 0x831845f6kfbh.fcn.base: 4954 ; 0x010: 0x0000135akfbh.fcn.wrap: 0 ; 0x014: 0x00000000kfbh.spare1: 0 ; 0x018: 0x00000000kfbh.spare2: 0 ; 0x01c: 0x00000000kffdnd.bnode.incarn: 1 ; 0x000: A=1 NUMM=0x0kffdnd.bnode.frlist.number: 4294967295 ; 0x004: 0xffffffffkffdnd.bnode.frlist.incarn: 0 ; 0x008: A=0 NUMM=0x0kffdnd.overfl.number: 2 ; 0x00c: 0x00000002 --表示指向同层级的下一个块kffdnd.overfl.incarn: 1 ; 0x010: A=1 NUMM=0x0kffdnd.parent.number: 4294967295 ; 0x014: 0xffffffffkffdnd.parent.incarn: 0 ; 0x018: A=0 NUMM=0x0kffdnd.fstblk.number: 0 ; 0x01c: 0x00000000 --表示指向上一层的块kffdnd.fstblk.incarn: 1 ; 0x020: A=1 NUMM=0x0kfvvde.entry.incarn: 1 ; 0x024: A=1 NUMM=0x0 --kfvvde这部分信息是acfs元数据内容kfvvde.entry.hash: 0 ; 0x028: 0x00000000kfvvde.entry.refer.number: 4294967295 ; 0x02c: 0xffffffffkfvvde.entry.refer.incarn: 0 ; 0x030: A=0 NUMM=0x0kfvvde.volnm: DATA_VOL1 ; 0x034: length=9 --asm advm卷名称kfvvde.usage: ACFS ; 0x054: length=4 --advm的type类型,这里使用的是acfskfvvde.dgname: ; 0x074: length=0kfvvde.clname: ; 0x094: length=0kfvvde.mountpath: /acfs1 ; 0x0b4: length=6 --挂载点kfvvde.drlinit: 0 ; 0x4b5: 0x00kfvvde.pad1: 0 ; 0x4b6: 0x0000kfvvde.volfnum.number: 273 ; 0x4b8: 0x00000111 --逻辑卷文件,表示volume file numberkfvvde.volfnum.incarn: 1142697595 ; 0x4bc: 0x441c2e7bkfvvde.drlfnum.number: 0 ; 0x4c0: 0x00000000 --DRL脏数据记录区域文件,表示volume dirty region loggingkfvvde.drlfnum.incarn: 0 ; 0x4c4: 0x00000000kfvvde.volnum: 1 ; 0x4c8: 0x0001 --表示对应的卷组number,从1开始kfvvde.avddgnum: 467 ; 0x4ca: 0x01d3kfvvde.extentsz: 8 ; 0x4cc: 0x00000008 --这里表示extent大小kfvvde.volstate: 2 ; 0x4d0: D=0 C=1 R=0 --这里表示advm卷组状态,2表示可用...--block 2[grid@hisdb01 ~]$ kfed read /dev/asmdisk4 aun=1345 aus=1M blkn=2|morekfbh.endian: 1 ; 0x000: 0x01kfbh.hard: 130 ; 0x001: 0x82kfbh.type: 22 ; 0x002: KFBTYP_VOLUMEDIRkfbh.datfmt: 1 ; 0x003: 0x01kfbh.block.blk: 2 ; 0x004: blk=2kfbh.block.obj: 7 ; 0x008: file=7kfbh.check: 1237494243 ; 0x00c: 0x49c2a9e3kfbh.fcn.base: 4946 ; 0x010: 0x00001352kfbh.fcn.wrap: 0 ; 0x014: 0x00000000kfbh.spare1: 0 ; 0x018: 0x00000000kfbh.spare2: 0 ; 0x01c: 0x00000000kffdnd.bnode.incarn: 1 ; 0x000: A=1 NUMM=0x0kffdnd.bnode.frlist.number: 4294967295 ; 0x004: 0xffffffffkffdnd.bnode.frlist.incarn: 0 ; 0x008: A=0 NUMM=0x0kffdnd.overfl.number: 4294967295 ; 0x00c: 0xffffffffkffdnd.overfl.incarn: 0 ; 0x010: A=0 NUMM=0x0kffdnd.parent.number: 4294967295 ; 0x014: 0xffffffffkffdnd.parent.incarn: 0 ; 0x018: A=0 NUMM=0x0kffdnd.fstblk.number: 0 ; 0x01c: 0x00000000kffdnd.fstblk.incarn: 1 ; 0x020: A=1 NUMM=0x0kfvvde.entry.incarn: 1 ; 0x024: A=1 NUMM=0x0kfvvde.entry.hash: 0 ; 0x028: 0x00000000kfvvde.entry.refer.number: 4294967295 ; 0x02c: 0xffffffffkfvvde.entry.refer.incarn: 0 ; 0x030: A=0 NUMM=0x0kfvvde.volnm: DATA_VOL2 ; 0x034: length=9kfvvde.usage: ; 0x054: length=0kfvvde.dgname: ; 0x074: length=0kfvvde.clname: ; 0x094: length=0kfvvde.mountpath: ; 0x0b4: length=0kfvvde.drlinit: 0 ; 0x4b5: 0x00kfvvde.pad1: 0 ; 0x4b6: 0x0000kfvvde.volfnum.number: 274 ; 0x4b8: 0x00000112 --逻辑卷文件kfvvde.volfnum.incarn: 1142697607 ; 0x4bc: 0x441c2e87kfvvde.drlfnum.number: 0 ; 0x4c0: 0x00000000 --DRL脏数据记录区域文件kfvvde.drlfnum.incarn: 0 ; 0x4c4: 0x00000000kfvvde.volnum: 2 ; 0x4c8: 0x0002kfvvde.avddgnum: 467 ; 0x4ca: 0x01d3kfvvde.extentsz: 8 ; 0x4cc: 0x00000008kfvvde.volstate: 2 ; 0x4d0: D=0 C=1 R=0...由于我创建了两个ACFS,从1号块开始每个block对应一个acfs,第2个块对应对2个acfs..所以 asm file 7 block 1 包含的是第一个逻辑卷的信息(kfvvde.volnm: DATA_VOL1),并且有两个文件关联到这个逻辑卷:DRL脏数据记录区域文件(kfvvde.drlfnum.number 0)逻辑卷文件 (kfvvde.volfnum.number: 273)这些都是特殊文件,所以不会显示在asmcmd ls和v$asm_alias视图中,但是可以通过查询v$asm_file中找到他们:col type for a10select file_number "file#",bytes/1024/1024 mb,type from v$asm_file where group_number=1;file# MB TYPE---------- ---------- ---------------...273 2048 ASMVOL274 2048 ASMVOL
4、ADVM总结
一个磁盘组中可以配置一个或多个ASM动态逻辑卷设备。ASM集群文件系统通过ADVM接口构建于ASM磁盘组之上。ADVM跟数据库一样,也是ASM的一个客户端。有两种与ADVM逻辑卷相关的文件类型:● ASMVOL:逻辑卷文件,作为逻辑卷存储设备的容器● ASMVDRL:包含脏数据记录区域信息的文件。恢复镜像数据时会用到此文件。
文章转载自数据库技术加油站,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。




