浅谈Halo数据库如何适配DB2的语法问题

3、回到我们最初的ORDER关键字的例子，因为ORDER是保留字关键字，但在上述例子中它却作为列名与表名，在postgres测试时报移进规约冲突的错误。那有人就说为什么不把关键字声明改成unreserved类型，就都支持了，这个方法不错，但是试了一下，却是order by子句报移进规约错误，如下所示：

gram.y: error: shift/reduce conflicts: 18 found, 0 expected
gram.y: error: reduce/reduce conflicts: 5 found, 0 expected

那接下来怎么把两种移进规约分离处理呢。在分离处理之前，我们了解一下伪token，伪token只在gram.y文件定义，它没有token关键字种类声明，也没有标识符值，编译后会自动生成注册tokentype枚举值，所以它也不受关键字的规约限制，只是一个占位关键字的作用。这个不错，我们可以利用起来。

/*gram.y中声明方式*/
%token ORDER_Q

   我们再回顾bison的基本原理，他是先调用词法分析函数int yylex()对输入串 进行扫描取词后再进行语法分析。此涉及两个文件“gram.y”和"parser.c"，在"parser.c"文件中base_yylex(YYSTYPE *lvalp, YYLTYPE *llocp, core_yyscan_t yyscanner)函数正好给我们提供这个扫描功能，当取出的cur_token是“ORDER” token后，我们可以再多取一个next_token，通过判断next_token 是否 “BY” token，如果不是则替换为ORDER_Q token，反之则不替换，沿用原来的移进归约方法。代码如下：

base_yylex(YYSTYPE *lvalp, YYLTYPE *llocp, core_yyscan_t yyscanner)
{
  base_yy_extra_type *yyextra = base_yyget_extra(yyscanner);
  int      cur_token;
  int      next_token;
  int      cur_token_length;
  YYLTYPE    cur_yylloc;

  /* Get next token --- we might already have it */
  if (yyextra->have_lookahead)
  {
    cur_token = yyextra->lookahead_token;
    lvalp->core_yystype = yyextra->lookahead_yylval;
    *llocp = yyextra->lookahead_yylloc;
    if (yyextra->lookahead_end)
      *(yyextra->lookahead_end) = yyextra->lookahead_hold_char;
    yyextra->have_lookahead = false;
  }
  else
    cur_token = base_core_yylex(&(lvalp->core_yystype), llocp, yyscanner);

  /*
   * If this token isn't one that requires lookahead, just return it.  If it
   * does, determine the token length.  (We could get that via strlen(), but
   * since we have such a small set of possibilities, hardwiring seems
   * feasible and more efficient --- at least for the fixed-length cases.)
   */
  switch (cur_token)
  {
    case NOT:
      cur_token_length = 3;
      break;
    ...
    case ORDER:
      cur_token_length = 5;
      break;
    ...
    default:
      return cur_token;
  }

  /*
   * Identify end+1 of current token.  base_core_yylex() has temporarily stored a
   * '\0' here, and will undo that when we call it again.  We need to redo
   * it to fully revert the lookahead call for error reporting purposes.
   */
  yyextra->lookahead_end = yyextra->core_yy_extra.scanbuf +
    *llocp + cur_token_length;
  Assert(*(yyextra->lookahead_end) == '\0');

  /*
   * Save and restore *llocp around the call.  It might look like we could
   * avoid this by just passing &lookahead_yylloc to base_core_yylex(), but that
   * does not work because flex actually holds onto the last-passed pointer
   * internally, and will use that for error reporting.  We need any error
   * reports to point to the current token, not the next one.
   */
  cur_yylloc = *llocp;

  /* Get next token, saving outputs into lookahead variables */
  next_token = base_core_yylex(&(yyextra->lookahead_yylval), llocp, yyscanner);
  yyextra->lookahead_token = next_token;
  yyextra->lookahead_yylloc = *llocp;

  *llocp = cur_yylloc;

  /* Now revert the un-truncation of the current token */
  yyextra->lookahead_hold_char = *(yyextra->lookahead_end);
  *(yyextra->lookahead_end) = '\0';

  yyextra->have_lookahead = true;

  /* Replace cur_token if needed, based on lookahead */
  switch (cur_token)
  {
    case NOT:
      /* Replace NOT by NOT_LA if it's followed by BETWEEN, IN, etc */
      switch (next_token)
      {
        case BETWEEN:
        case IN_P:
        case LIKE:
        case ILIKE:
        case SIMILAR:
          cur_token = NOT_LA;
          break;
      }
      break;

    ...
    case ORDER:
      if(next_token != BY)
        cur_token = ORDER_Q;
      break;
    ...
  }
  return cur_token;
}

最后gram.y文件中给token ORDER_Q 添加新规则来支持ORDER_Q作为列名与表名，并且设置“order”字符串作为标识符名称，再SQL命令调用时它可以支持大小写：

/* Column identifier --- names that can be column, table, etc names.
 */
ColId:    IDENT                  { $$ = $1; }
      | unreserved_keyword       { $$ = pstrdup($1); }
      | col_name_keyword         { $$ = pstrdup($1); }
      | ORDER_Q                  { $$ = pstrdup("order");}
    ;

编译后我们重启测试以下命令输出结果，发现与原生DB2一致。

CREATE TABLE order (id int,  order VARCHAR(32));
INSERT INTO order VALUES(1, 'JACK'), (2,'LEO');
halodb2=# SELECT id, ORDER FROM ORDER;
 id | order 
----+-------
  1 | jack
  2 | LEO
(2 rows)

总结以上，如果适配各种类型的数据库，词法与语法修改是不可或缺的，特别是第三种技巧很有用，比如halodb2适配DB2的values a_expr语法，DB2要支持各种类型函数进行强制转化，例如int（12.32），但在POSTGRES直接修改关键字声明后，却存在着移进规约冲突等情况，这些其实都可以采用这种方法完美解决。最后大家如果想更深入第三种技巧的使用方式，可以思考一下问题，如何解决适配DB2产生的类型 char（Iconst）语法与强制转换函数char(func_arg_list_opt）语法的移进规约冲突。