O
2023-09-21
开启少量并行后执行效率提升100倍,如何理解?
【 使用环境 】生产环境 or 测试环境
【 OB or 其他组件 】OB
【 使用版本 】5.7.25-OceanBase_CE-v4.2.0.0
【问题描述】
SQL1=select /*+ parallel(2) */ count(*) from customer a join nation b on a.c_nationkey < b.n_nationkey;
比
SQL2=select /*+ parallel(1) */ count(*) from customer a join nation b on a.c_nationkey < b.n_nationkey;
快100倍!
收藏
复制链接
微信扫码分享
在小程序上查看
分享
2条回答
默认
最新
采纳答案后不可修改和取消
我分析并且验证了下,这个问题主要是use_batch=true的情况下效率较低导致。
默认_NLJ_BATCHING_ENABLED这个参数是true的,当不开并行时候NLJ是可以用到左表一次取一批数据然后右表取匹配。应该是开启并行后NLJ不支持use_batch,看执行计划中use_batch=false。
我在不开启并行的情况下尝试关闭use_batch,执行效率就非常高了,符合了预期:
obclient [tpch10]> SET _NLJ_BATCHING_ENABLED=false;
Query OK, 0 rows affected (0.001 sec)
obclient [tpch10]> select /*+ parallel(1) */ count(*) from customer a join nation b on a.c_nationkey < b.n_nationkey;
+----------+
| count(*) |
+----------+
| 18002417 |
+----------+
1 row in set (0.073 sec)
obclient [tpch10]> explain extended select /*+ parallel(1) */ count(*) from customer a join nation b on a.c_nationkey < b.n_nationkey;
+-----------------------------------------------------------------------------------------------------+
| Query Plan |
+-----------------------------------------------------------------------------------------------------+
| =================================================================== |
| |ID|OPERATOR |NAME |EST.ROWS|EST.TIME(us)| |
| ------------------------------------------------------------------- |
| |0 |SCALAR GROUP BY | |1 |978560 | |
| |1 |└─NESTED-LOOP JOIN | |12500000|752010 | |
| |2 | ├─TABLE FULL SCAN |b |25 |5 | |
| |3 | └─DISTRIBUTED TABLE RANGE SCAN|a(idx3)|500000 |17610 | |
| =================================================================== |
| Outputs & filters: |
| ------------------------------------- |
| 0 - output([T_FUN_COUNT(*)(0x7f1a37c40760)]), filter(nil), rowset=256 |
| group(nil), agg_func([T_FUN_COUNT(*)(0x7f1a37c40760)]) |
| 1 - output(nil), filter(nil), rowset=256 |
| conds(nil), nl_params_([b.N_NATIONKEY(0x7f1a37c40200)(:1)]), use_batch=false |
| 2 - output([b.N_NATIONKEY(0x7f1a37c40200)]), filter(nil), rowset=256 |
| access([b.N_NATIONKEY(0x7f1a37c40200)]), partitions(p0) |
| is_index_back=false, is_global_index=false, |
| range_key([b.N_NATIONKEY(0x7f1a37c40200)]), range(MIN ; MAX)always true |
| 3 - output(nil), filter(nil), rowset=256 |
| access(nil), partitions(p0) |
| is_index_back=false, is_global_index=false, |
| range_key([a.C_NATIONKEY(0x7f1a37c3ff20)], [a.C_CUSTKEY(0x7f1a37c40d80)]), range(MIN ; MAX), |
| range_cond([a.C_NATIONKEY(0x7f1a37c3ff20) < :1(0x7f1a37ccb960)]) |
| Used Hint: |
| ------------------------------------- |
| /*+ |
| |
| PARALLEL(1) |
| */ |
| Qb name trace: |
| ------------------------------------- |
| stmt_id:0, stmt_type:T_EXPLAIN |
| stmt_id:1, SEL$1 > SEL$C6D21C0F |
| Outline Data: |
| ------------------------------------- |
| /*+ |
| BEGIN_OUTLINE_DATA |
| LEADING(@"SEL$C6D21C0F" ("tpch10"."b"@"SEL$1" "tpch10"."a"@"SEL$1")) |
| USE_NL(@"SEL$C6D21C0F" "tpch10"."a"@"SEL$1") |
| FULL(@"SEL$C6D21C0F" "b"@"SEL$1") |
| INDEX(@"SEL$C6D21C0F" "a"@"SEL$1" "idx3") |
| USE_DAS(@"SEL$C6D21C0F" "a"@"SEL$1") |
| OUTER_TO_INNER(@"SEL$1") |
| PARALLEL(1) |
| OPTIMIZER_FEATURES_ENABLE('4.0.0.0') |
| END_OUTLINE_DATA |
| */ |
| Optimization Info: |
| ------------------------------------- |
| b: |
| table_rows:25 |
| physical_range_rows:25 |
| logical_range_rows:25 |
| index_back_rows:0 |
| output_rows:25 |
| table_dop:1 |
| dop_method:Global DOP |
| avaiable_index_name:[nation] |
| stats version:1694254794515646 |
| dynamic sampling level:0 |
| a: |
| table_rows:1500000 |
| physical_range_rows:500000 |
| logical_range_rows:500000 |
| index_back_rows:0 |
| output_rows:500000 |
| table_dop:1 |
| dop_method:DAS DOP |
| avaiable_index_name:[idx1, idx2, idx3, customer] |
| pruned_index_name:[idx1, idx2, customer] |
| stats version:1694254801441111 |
| dynamic sampling level:0 |
| Plan Type: |
| LOCAL |
| Note: |
| Degree of Parallelism is 1 because of hint |
+-----------------------------------------------------------------------------------------------------+
76 rows in set (0.004 sec)
猜测性分析:
1、当use_batch=false时,左表每次取一条记录,右表对该记录进行匹配并查找符合条件记录,可以通过索引进行过滤,效率较高。
2、当use_batch=true时,左表每次取一批记录,右表对这批记录进行匹配并查找符合条件记录,如果关联条件是等值条件,那么可以将条件进行下压,如果不存在等值条件,那么这对左表每一行只能将右表(索引)所有记录都拿到计算层,在计算层内部进行循环匹配(应该是hash表)。
评论
有用 1采纳答案后不可修改和取消
问题已关闭:
评论
有用 0回答交流
提交
问题信息
请登录之后查看
邀请回答
暂无人订阅该标签,敬请期待~~
墨值悬赏

