To add datafiles in ASM hit ORA-01119 ORA-17502 ORA-15041 after add new ASMDISKs to ASMDISKGROUP

A friend asked me ,they add new disk to a ASM diskgroup. Disks were added successfully. and check free space available of ASM diskgroup was correct, But when tried alter tablespace add new datafile using the diskgroup one the disks were recently added, they got “ora-01119” and “ORA-17502″,”ORA-15041” errors during add new datafile.

SQL script:

alter diskgroup RAC_DISK1 add disk '/dev/raw/raw8' ;
ALTER tablespace FMIS1800 ADD datafile '+RAC_DISK1' size 4000M

then reported following errors:

ORA-01119: error in creating database file ”
ORA-17502: ksfdcre:4 Failed to create file ”
ORA-15041: diskgroup space exhausted

then check raw device permissions and ASM info from dynamic view, No abnormalities.

ls -l /dev/raw/*

select group_number,disk_number,name,failgroup,create_date,path,writes,reads from v$asm_disk;
Select group_number,operation,state,power,actual,sofar,est_work,est_rate,est_minutes from v$asm_operation;
select path,total_mb,free_mb from v$asm_disk_stat;
select * from v$asm_operation;
select name,total_mb,free_mb from v$asm_diskgroup;

TIP:
We Found some different between the added new disk and before others disks,the new disk writes and reads from v$asm_disk is zero.

There are two possibilities
1) ASM disk group genuinely does not have enough space to create or extend the datafile.

This possibility was ruled out, as I knew that they added 8g of space ,just trying to create a datafile of size 4g.

2) Another possibility could be, that the disks are not rebalanced (this was the problem is my case)

To check ASM instance alert log, but never rebalance diskgroup info like this:

”
NOTE: starting rebalance of group 1/0xe7a18492 (DATA_DG) at power 1
Starting background process ARB0
”

Solution:

Solution is to rebalance the disk manually like this:

alter diskgroup RAC_DISK1 rebalance power 10

OR
Wait rebalance to complete (SUCCESS: rebalance completed for group in ASM alert.log for specific diskgroup)

Note:
We can also speed up the disk rebalance process by changing the value of ASM_POWER_LIMIT parameter (default is 1)

sql>show parameter asm_power_limit
sql>alter system set asm_power_limit=11;

Changing the value of this parameter is only recommended, when there is not much of activity in the database. As rebalance using value 11, will incur high amount of I/O, which may impact day-to-day operations.

We can monitor the rebalance activity by selecting data from v$asm_operation view. We can also check the column “UNBALANCED” in v$asm_diskgroup view. If the disks are correctly balanced, we’ll see default value of N in this column.

One the rebalance is done; we can add the datafile using the same command used before.

References oraclehandson ‘s article