暂无图片
分享
O
OceanBase社区传送门
2023-09-21
开启少量并行后执行效率提升100倍,如何理解?

【 使用环境 】生产环境 or 测试环境
【 OB or 其他组件 】OB
【 使用版本 】5.7.25-OceanBase_CE-v4.2.0.0
【问题描述】
SQL1=select /*+ parallel(2) */ count(*) from customer a join nation b on a.c_nationkey < b.n_nationkey;

SQL2=select /*+ parallel(1) */ count(*) from customer a join nation b on a.c_nationkey < b.n_nationkey;
快100倍!

收藏
分享
2条回答
默认
最新
O
OceanBase社区传送门

我分析并且验证了下,这个问题主要是use_batch=true的情况下效率较低导致。
默认_NLJ_BATCHING_ENABLED这个参数是true的,当不开并行时候NLJ是可以用到左表一次取一批数据然后右表取匹配。应该是开启并行后NLJ不支持use_batch,看执行计划中use_batch=false。
我在不开启并行的情况下尝试关闭use_batch,执行效率就非常高了,符合了预期:

obclient [tpch10]> SET _NLJ_BATCHING_ENABLED=false;
Query OK, 0 rows affected (0.001 sec)

obclient [tpch10]> select /*+ parallel(1) */ count(*) from customer a join nation b on a.c_nationkey < b.n_nationkey;
+----------+
| count(*) |
+----------+
| 18002417 |
+----------+
1 row in set (0.073 sec)

obclient [tpch10]> explain extended select /*+ parallel(1) */ count(*) from customer a join nation b on a.c_nationkey < b.n_nationkey;
+-----------------------------------------------------------------------------------------------------+
| Query Plan                                                                                          |
+-----------------------------------------------------------------------------------------------------+
| ===================================================================                                 |
| |ID|OPERATOR                        |NAME   |EST.ROWS|EST.TIME(us)|                                 |
| -------------------------------------------------------------------                                 |
| |0 |SCALAR GROUP BY                 |       |1       |978560      |                                 |
| |1 |└─NESTED-LOOP JOIN              |       |12500000|752010      |                                 |
| |2 |  ├─TABLE FULL SCAN             |b      |25      |5           |                                 |
| |3 |  └─DISTRIBUTED TABLE RANGE SCAN|a(idx3)|500000  |17610       |                                 |
| ===================================================================                                 |
| Outputs & filters:                                                                                  |
| -------------------------------------                                                               |
|   0 - output([T_FUN_COUNT(*)(0x7f1a37c40760)]), filter(nil), rowset=256                             |
|       group(nil), agg_func([T_FUN_COUNT(*)(0x7f1a37c40760)])                                        |
|   1 - output(nil), filter(nil), rowset=256                                                          |
|       conds(nil), nl_params_([b.N_NATIONKEY(0x7f1a37c40200)(:1)]), use_batch=false                  |
|   2 - output([b.N_NATIONKEY(0x7f1a37c40200)]), filter(nil), rowset=256                              |
|       access([b.N_NATIONKEY(0x7f1a37c40200)]), partitions(p0)                                       |
|       is_index_back=false, is_global_index=false,                                                   |
|       range_key([b.N_NATIONKEY(0x7f1a37c40200)]), range(MIN ; MAX)always true                       |
|   3 - output(nil), filter(nil), rowset=256                                                          |
|       access(nil), partitions(p0)                                                                   |
|       is_index_back=false, is_global_index=false,                                                   |
|       range_key([a.C_NATIONKEY(0x7f1a37c3ff20)], [a.C_CUSTKEY(0x7f1a37c40d80)]), range(MIN ; MAX),  |
|       range_cond([a.C_NATIONKEY(0x7f1a37c3ff20) < :1(0x7f1a37ccb960)])                              |
| Used Hint:                                                                                          |
| -------------------------------------                                                               |
|   /*+                                                                                               |
|                                                                                                     |
|       PARALLEL(1)                                                                                   |
|   */                                                                                                |
| Qb name trace:                                                                                      |
| -------------------------------------                                                               |
|   stmt_id:0, stmt_type:T_EXPLAIN                                                                    |
|   stmt_id:1, SEL$1 > SEL$C6D21C0F                                                                   |
| Outline Data:                                                                                       |
| -------------------------------------                                                               |
|   /*+                                                                                               |
|       BEGIN_OUTLINE_DATA                                                                            |
|       LEADING(@"SEL$C6D21C0F" ("tpch10"."b"@"SEL$1" "tpch10"."a"@"SEL$1"))                          |
|       USE_NL(@"SEL$C6D21C0F" "tpch10"."a"@"SEL$1")                                                  |
|       FULL(@"SEL$C6D21C0F" "b"@"SEL$1")                                                             |
|       INDEX(@"SEL$C6D21C0F" "a"@"SEL$1" "idx3")                                                     |
|       USE_DAS(@"SEL$C6D21C0F" "a"@"SEL$1")                                                          |
|       OUTER_TO_INNER(@"SEL$1")                                                                      |
|       PARALLEL(1)                                                                                   |
|       OPTIMIZER_FEATURES_ENABLE('4.0.0.0')                                                          |
|       END_OUTLINE_DATA                                                                              |
|   */                                                                                                |
| Optimization Info:                                                                                  |
| -------------------------------------                                                               |
|   b:                                                                                                |
|       table_rows:25                                                                                 |
|       physical_range_rows:25                                                                        |
|       logical_range_rows:25                                                                         |
|       index_back_rows:0                                                                             |
|       output_rows:25                                                                                |
|       table_dop:1                                                                                   |
|       dop_method:Global DOP                                                                         |
|       avaiable_index_name:[nation]                                                                  |
|       stats version:1694254794515646                                                                |
|       dynamic sampling level:0                                                                      |
|   a:                                                                                                |
|       table_rows:1500000                                                                            |
|       physical_range_rows:500000                                                                    |
|       logical_range_rows:500000                                                                     |
|       index_back_rows:0                                                                             |
|       output_rows:500000                                                                            |
|       table_dop:1                                                                                   |
|       dop_method:DAS DOP                                                                            |
|       avaiable_index_name:[idx1, idx2, idx3, customer]                                              |
|       pruned_index_name:[idx1, idx2, customer]                                                      |
|       stats version:1694254801441111                                                                |
|       dynamic sampling level:0                                                                      |
|   Plan Type:                                                                                        |
|       LOCAL                                                                                         |
|   Note:                                                                                             |
|       Degree of Parallelism is 1 because of hint                                                    |
+-----------------------------------------------------------------------------------------------------+
76 rows in set (0.004 sec)

猜测性分析:
1、当use_batch=false时,左表每次取一条记录,右表对该记录进行匹配并查找符合条件记录,可以通过索引进行过滤,效率较高。
2、当use_batch=true时,左表每次取一批记录,右表对这批记录进行匹配并查找符合条件记录,如果关联条件是等值条件,那么可以将条件进行下压,如果不存在等值条件,那么这对左表每一行只能将右表(索引)所有记录都拿到计算层,在计算层内部进行循环匹配(应该是hash表)。

暂无图片 评论
暂无图片 有用 1
O
OceanBase社区传送门
问题已关闭:
暂无图片 评论
暂无图片 有用 0
回答交流
提交
问题信息
请登录之后查看
邀请回答
暂无人订阅该标签,敬请期待~~
暂无图片墨值悬赏