获取轻量级锁的片段

原文地下：https://www.highgo.ca/2022/02/28/a-snippet-to-acquire-a-lightweight-lock/
原文作者：David Zhang

一、概述

最近，我正在研究一个与 PostgreSQL 中的缓冲区管理器相关的内部问题，我看到了缓冲区管理器中轻量级锁的典型用法，如下所示。

1   INIT_BUFFERTAG(newTag, smgr_rnode.node, forkNum, blockNum);
2   newHash = BufTableHashCode(&newTag);
3   newPartitionLock = BufMappingPartitionLock(newHash);
4   LWLockAcquire(newPartitionLock, LW_SHARED);
5   buf_id = BufTableLookup(&newTag, newHash);
6   LWLockRelease(newPartitionLock);

基本上，当缓冲区管理器需要使用缓冲区标记访问缓冲区块时，它必须以共享或独占模式获取轻量级锁，然后找到缓冲区块，然后释放轻量级锁。

由于缓冲区管理器在多个后端之间共享并且经常访问缓冲区块，因此必须设计此代码段以保护读取和写入的数据一致性并且不影响性能。

这篇博客将解释这个代码片段在 PostgreSQL 中是如何工作的，并更多地强调轻量级锁获取。

二、如何使用快照公共功能

现在，让我们逐行浏览上面的代码片段。
第一行简单地使用宏来使用这五个数字初始化缓冲区标记。这里，INIT_BUFFERTAG是一个宏定义，如下所示，

#define INIT_BUFFERTAG(a,xx_rnode,xx_forkNum,xx_blockNum) \
( \
    (a).rnode = (xx_rnode), \
    (a).forkNum = (xx_forkNum), \
    (a).blockNum = (xx_blockNum) \
)

在宏调用之后，newTag 被分配了这五个数字，即表空间号、数据库号、关系号、fork 号（数据、fsm 或可见性映射等）以及实际文件中的块号（每个块为 8k）；

第二行 newHash = BufTableHashCode(&newTag)；根据缓冲区标签生成哈希数。其中，函数 BufTableHashCode 计算与全局共享缓冲区哈希表中给定缓冲区标签相关的哈希码，并返回一个无符号整数。

第三行检索锁池中的分区锁，使用无符号整数哈希数 mod 分区锁的总数（默认为 128）。
同样，函数 BufMappingPartitionLock 是一个预定义的宏，如下所示。

#define BufMappingPartitionLock(hashcode) \
    (&MainLWLockArray[BUFFER_MAPPING_LWLOCK_OFFSET + \
        BufTableHashPartition(hashcode)].lock)

它将在 MainLWLockArray 轻量级锁数组中返回一个锁。其中 BUFFER_MAPPING_LWLOCK_OFFSET 是在 lwlocknames.txt 文件中定义的专用轻量级锁的数量。分区轻量级锁的数量是 128 个锁，位于这些主轻量级锁数组中定义的这些专用锁之后。在这里，宏 BufTableHashPartition 是为了确保它总是在分区锁池中为任何给定的哈希数返回一个锁。

第四行是用非常高效的算法获取轻量级锁。此 LWLockAcquire 将帮助返回指定模式下的轻量级锁，即共享（用于只读操作）或独占（用于写操作）。如果锁立即可用，则此函数返回 true，如果必须休眠并等待，则返回 false。

在这个 LWLockAcquire 内部，有很多考虑因素，但我想在函数 LWLockAttemptLock 中强调一个智能 c 实现，我相信您可以将这种类似的想法作为设计模式在您的应用程序中设计其他 CPU 和内存敏感逻辑。
正如你在下面看到的，这个共享锁和排它锁的关键实现。

if (mode == LW_EXCLUSIVE)
        {
            lock_free = (old_state & LW_LOCK_MASK) == 0;
            if (lock_free)
                desired_state += LW_VAL_EXCLUSIVE;
        }
        else
        {
            lock_free = (old_state & LW_VAL_EXCLUSIVE) == 0;
            if (lock_free)
                desired_state += LW_VAL_SHARED;
        }

该实现涉及三个宏：LW_LOCK_MASK、LW_VAL_EXCLUSIVE 和 LW_VAL_SHARED，其中 LW_LOCK_MASK 是一个一致的数字，即 0xFFFFFF，用于位操作。如果任何低 24 位有 1，则表示锁处于共享或独占模式。也就是说，还有人在读取数据，如果要更新数据，请稍候。如果低 24 位全部为零，则将其分配给一个大数字 LW_VAL_EXCLUSIVE，即 0x800000，表示该锁被用作独占。如果你想在共享模式下获取这个锁，那么只要这个锁没有被别人在独占模式下持有，并且持有的时间不超过0x7FFFFF，那么你可以获得一个共享锁，使用次数会简单增加减一，即 LW_VAL_SHARED。当然，可以同时持有多少个共享锁受其他参数的限制。

下面是这三个宏的定义。

#define LW_LOCK_MASK                ((uint32) ((1 << 25)-1))
#define LW_VAL_EXCLUSIVE            ((uint32) 1 << 24)
#define LW_VAL_SHARED               1

第五行使用给定的缓冲区标签和哈希码查找缓冲区 id。
获得锁后，操作的工作立即取决于您的应用程序，但请记住，轻量级锁设计为仅在短时间内持有。

第六行将锁释放回分区锁池。
在你完成你的读写操作后，使用这个 LWLockRelease 函数尽快释放锁，这样你就不会阻塞其他进程太久，特别是如果有一个写操作需要以独占模式获取这个锁.

三、总结

在这篇博客中，我们讨论了一个在 PostgreSQL 中使用轻量级锁的典型代码片段，并解释了在 PostgreSQL 中为轻量级锁实现的最有效的代码之一，即 LWLockAcquire。当您想在自己的设计中获得类似的结果时，我希望这会有所帮助。

David Zhang
一位专门从事 C/C++ 编程的软件开发人员，在硬件、固件、软件、数据库、网络和系统架构方面拥有丰富的经验。现在，在 HighGo Software Inc 工作，担任高级 PostgreSQL 架构师。

原文：

A snippet to acquire a Lightweight lock

1. Overview

Recently, I was working on an internal issue related with buffer manager in PostgreSQL, and I saw a typical use of the Lightweight lock in buffer manager like below.

1   INIT_BUFFERTAG(newTag, smgr_rnode.node, forkNum, blockNum);
2   newHash = BufTableHashCode(&newTag);
3   newPartitionLock = BufMappingPartitionLock(newHash);
4   LWLockAcquire(newPartitionLock, LW_SHARED);
5   buf_id = BufTableLookup(&newTag, newHash);
6   LWLockRelease(newPartitionLock);

Basically, when the buffer manger needs to access a buffer block using buffer tag, it will have to acquire a lightweight lock in either shared or exclusive mode, then find the buffer block and then release the lightweight lock.

Since the buffer manager is shared among multiple backends and a buffer block is accessed very often, this snippet has to be designed to protect the data consistency for read and write and no impact on performance.

This blog will explain how this snippet works in PostgreSQL and emphasize a little bit more on the lightweight lock acquire.

2. how to use snapshot public functions

Now, let’s go through the snippet above line by line.
The first line simply uses a Macro to initialize a buffer tag using those five numbers. Here, INIT_BUFFERTAG is a macro defines like below,

#define INIT_BUFFERTAG(a,xx_rnode,xx_forkNum,xx_blockNum) \

( \
    (a).rnode = (xx_rnode), \
    (a).forkNum = (xx_forkNum), \
    (a).blockNum = (xx_blockNum) \
)

After the macro call, the newTag has been assigned with those five numbers, i.e., table space number, database number, relation number, fork number (data, fsm or visibility map etc), and the block number (each block is 8k) within the actual file;

The second line newHash = BufTableHashCode(&newTag); generates a hash number based on the buffer tag. Where, The function BufTableHashCode computes the hash code associated with given buffer tag in the global shared buffer hash table, and return a unsigned integer.

The third line retrieves a partition lock within the locks pool used an unsigned integer hash number mod the total number of partition locks (default 128).
Again, the function BufMappingPartitionLock is a predefined macro and is showing below.

#define BufMappingPartitionLock(hashcode) \
    (&MainLWLockArray[BUFFER_MAPPING_LWLOCK_OFFSET + \
        BufTableHashPartition(hashcode)].lock)

It will return a lock in MainLWLockArray lightweight locks array. Where the BUFFER_MAPPING_LWLOCK_OFFSET is number of dedicated lightweight locks defined in lwlocknames.txt file. The number of partition lightweight locks are 128 locks located after these dedicated locks defined in these main lightweight locks array. Here, the macro BufTableHashPartition is to make sure it always returns a lock in the partition locks pool for any given hash number.

The fourth line to is to acquire the lightweight lock with a very efficient algorithm. This LWLockAcquire will help return a lightweight lock in the specified mode, i.e., shared (for read only operation) or exclusive (for write operation). This function returns true if the lock was available immediately, false if it has to sleep and wait.
Inside this LWLockAcquire, there are many considerations, but I want to emphasize one smart c implementation in the function LWLockAttemptLock, and I believe you can use this similar idea as a design pattern to design other CPU and Memory sensitive logic in your applications.
As you can see below is the key implementation of this shared and exclusive lock.

        {
            lock_free = (old_state & LW_LOCK_MASK) == 0;
            if (lock_free)
                desired_state += LW_VAL_EXCLUSIVE;
        }
        else
        {
            lock_free = (old_state & LW_VAL_EXCLUSIVE) == 0;
            if (lock_free)
                desired_state += LW_VAL_SHARED;
        }

This implementation involves three macros: LW_LOCK_MASK, LW_VAL_EXCLUSIVE and LW_VAL_SHARED, where LW_LOCK_MASK is a consistent number, i.e., 0xFFFFFF, used in bit operation. If any lower 24 bits has a one, then it means the lock is held in either shared or exclusive mode. In other words, someone is still reading the data, if you want the update the data, please wait. If the all lower 24 bits are zeros, then it will be assigned to a big number LW_VAL_EXCLUSIVE, i.e., 0x800000, which indicates the lock is used as exclusive. If you want to acquire this lock in shared mode, then as long as the lock is not held by someone in exclusive mode and it is not held more than 0x7FFFFF, then you can acquire one shared lock and the number of usages will be simple increase by one, i.e., LW_VAL_SHARED. Of course, how many shared locks can be held at the same time is limited by other parameters.

Here is the definition of these three macros.

#define LW_LOCK_MASK ((uint32) ((1 << 25)-1))
#define LW_VAL_EXCLUSIVE ((uint32) 1 << 24)
#define LW_VAL_SHARED 1
The fifth line looks up the buffer id using the given buffer tag and hash code.
Once you have acquired the lock, work on the operations immediately depends on your application, but keep in mind the lightweight lock is designed to be held only in a short period.

The sixth line release the lock back to the partition lock pool.
After you finished your operations either read or write, then use this LWLockRelease function to release the lock as soon as you can, so you don’t block other processes too long especially if there is a write operation need to acquire this lock in exclusive mode.

3. Summary

In this blog, we discussed a typical snippet which uses a lightweight lock in PostgreSQL, and explained one of the most efficient piece of code implemented for Lightweight lock in PostgreSQL, i.e., LWLockAcquire. I hope this can help when you want to achieve a similar result in your own design.

David Zhang
A software developer specialized in C/C++ programming with experience in hardware, firmware, software, database, network, and system architecture. Now, working in HighGo Software Inc, as a senior PostgreSQL architect.