作者介绍
李伟，北京科讯华通科技发展有限公司 高级工程师
一、预警内容
Solaris平台下，ISM或DISM配置错误可能会严重降低Oracle数据库性能：
The use of DISM on Solaris systems (SPARC and x86-64) outside of the ASM instance can lead to several different issues ranging from excessive swap usage even when memory is available to kernel panics to performance problems. It has been determined by engineering that the ASM instance is typically such a small memory footprint that it should not cause an issue.
二、问题现象
1、数据库级别体现为：
● Overall Poor Performance including login problems
● Processing Delays Resulting in IPC errors
●Instance Evictions
● Node Evictions
2、操作系统级别体现为：
● High System Time (often exceeding 50%)
● GUDs will show numerous threads in the ipc_get system call
● Excessive swap usage
● Kernel Panics
3、预警级别: 高级
三、什么是ISM & DISM？
Oracle广泛使用共享内存，错误配置共享内存可能会导致严重的性能问题。Solaris 支持四种类型的共享内存：
●Pageable Shared Memory: This is the same as any other anonymous memory (heap, stack, copy-on-write). Shared memory segment is allocated via shmget(2). Attaching to the shared memory segment via shmat() system call without passing SHM_SHARED_MMU or SHM_PAGEABLE flags results in pageable mapping to the shared memory segment. Page fault into the shared memory segment results in virtual to physical memory mapping. All pages residing in the shared memory segment have a backing store, to which the pages within the segment can be migrated during memory shortages. Similar to stack, heap, COW (copy-on-write), shared memory pages are anonymous (Anon) pages and thus they are backed up by virtual swap. These pages can be swapped out during memory shortages which can have a negative effect on application performance. Also, the kernel needs to acquire locks on pages when syncing data cached in shared memory segment. This can induce high lock contention on large SMP systems.
●Intimate Shared Memory (ISM): ISM is an optimized shared memory that deals with the shortcoming of pageable shared memory. Shared memory segment is allocated via shmget(2). Attaching to the shared memory via shmat() system call with SHM_SHARE_MMU flags results in a ISM segment. Unlike non-ISM memory, ISM memory is automatically locked by the kernel and does not require disk swap reservation. This ensures that ISM pages are not paged out during memory shortages. Also, IO performed on pages in ISM segment uses a fast locking mechanism that reduces kernel lock contention and in turn CPU kernel overhead. ISM offers several additional features such as: Large Page support (page sizes of 4M-2G are supported depending on the platform, see pagesize(1)) to improve TLB reach and cache hits; Low memory footprints due to shared virtual-to-physical translation tables (or mapping structures) for sharing memory pages amongst multiple processes; and a reduction in the number of memory pointers needed by applications due to the larger pages. This reduction in complexity translates into noticeable performance improvements, especially on systems with large amounts of memory. [It is important to note that shmget(2) requires swap reservation when it allocates shared memroy segment. It is only after shmat() is called with SHM_SHARED_MMU that the swap reservation against disk is released and switched to swap reservation against memory when pages are locked in memory]
● Dynamic Intimate Shared Memory (DISM): DISM is similar to ISM, but it can be dynamically resized depending on the application demand. Shared memory segment is allocated via shmget(2). Attaching to the shared memory via shmat() system call with SHM_PAGEABLE flags results in a DISM segment. With ISM, it is not possible to change the size of the segment once it has been created. For example, Oracle Database needs to be restarted if the buffer cache size needs to be increased. DISM was introduced as a RAS feature that allows Oracle Database to deal with dynamic reconfiguration events such as adding or removing memory from the system. With this feature, a large DISM segment can be created when the database is started (see doc: Dynamic SGA). The database can selectively lock and unlock sections of this segment as memory requirements change. Unlike ISM, where the kernel locks the shared memory segment, responsibility for locking and unlocking (mlock(3C)) is placed on the database or application. This provides flexibility to adjust memory requirements dynamically. Once DISM is locked properly by the database, it behaves in the same way as ISM, both in functionality and performance. That means availability benefits of DISM can be realized without compromising performance.
●Optimized Shared Memory (OSM): OSM provides similar benefits as DISM but without some of the requirements of DISM see doc: Oracle Database 12c takes Advantage of Optimized Shared Memory on Solaris for additional details.
四、解决方法：
为了主动避免/防止与DISM相关的问题，Oralce最佳实践强烈建议通过以下任一方式禁用在SGA大于4GB的所有实例上使用DISM：
1. Unset the SGA_MAX_SIZE MEMORY_MAX_TARGET / MEMORY_TARGET parameters
2. Ensure SGA_MAX_SIZE is set to the same value as SGA_TARGET parameter or equal to the sum of all SGA components in the instance
References:
RAC and Oracle Clusterware Best Practices and Starter Kit (Solaris) (Doc ID 811280.1)
ISM or DISM Misconfiguration can Slow Down Oracle Database Performance (Doc ID 1472108.1)
Running Oracle Database in Solaris 10 Containers - Best Practices (Doc ID 317257.1)
