暂无图片
jdbc批量提交openGauss异常
我来答
分享
胡工
2021-03-10
jdbc批量提交openGauss异常

jdbc批量提交时,每次提次100条,每条数据大约550个字符,openGauss主节点会立即挂,客户端异常信息如下QQ图片20210310172937.png

我来答
添加附件
收藏
分享
问题补充
4条回答
默认
最新
李宏达

image.png

暂无图片 评论
暂无图片 有用 1
暂无图片
胡工
  1. 代码没有问题 ,同样的代码postgresql12.6上已经验证过了的
  2. 启10个线程,每次提次1条数据,能正常入库
  3. 启10个线程,每次提次2条数据,也有上述问题
暂无图片 评论
暂无图片 有用 0
彭冲

可以提供一下java测试代码及表结构,可以复现下看看

暂无图片 评论
暂无图片 有用 0
胡工
@Component
public class MultipleThreadInsertTable implements InitializingBean {

    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void afterPropertiesSet() throws Exception {
        final int BATCH_SIZE = 10000000;
        long start = System.currentTimeMillis();
        ExecutorService executorService = Executors.newFixedThreadPool(10);

        List<String[]> list = new ArrayList<>();

        for (int i = 0; i < BATCH_SIZE; i++) {
            String id = UUID.randomUUID().toString();
            String content1 = RandomStringUtils.randomAlphanumeric(128);
            String content2 = RandomStringUtils.randomAlphanumeric(128);

            list.add(new String[]{id, content1, content2});

            if (list.size() >= 100) {
                List<Object[]> copyList = new ArrayList<>(list);
                list.clear();

                BathcInsertTask task = new BathcInsertTask();
                task.setData(copyList);
                task.setJdbcTemplate(jdbcTemplate);

                executorService.execute(task);
            }


        }

        if (!CollectionUtils.isEmpty(list)) {
            List<Object[]> copyList = new ArrayList<>(list);
            list.clear();

            BathcInsertTask task = new BathcInsertTask();
            task.setData(copyList);
            task.setJdbcTemplate(jdbcTemplate);

            executorService.execute(task);
        }

        executorService.shutdown();
        if (!executorService.awaitTermination(1, TimeUnit.HOURS)) {
            executorService.shutdownNow();
        } else {
            long end = System.currentTimeMillis();
            System.out.println("generate data cost [" + (end - start) + "]ms");
        }

        while (executorService.awaitTermination(1, TimeUnit.HOURS)) {
            executorService.shutdown();
        }
        long end = System.currentTimeMillis();

        System.out.println("generate data cost [" + (end - start) + "]ms");
    }

    class BathcInsertTask implements Runnable {
        private List<Object[]> data;
        private JdbcTemplate jdbcTemplate;

        public List<Object[]> getData() {
            return data;
        }

        public void setData(List<Object[]> data) {
            this.data = data;
        }

        public JdbcTemplate getJdbcTemplate() {
            return jdbcTemplate;
        }

        public void setJdbcTemplate(JdbcTemplate jdbcTemplate) {
            this.jdbcTemplate = jdbcTemplate;
        }

        /**
         * When an object implementing interface <code>Runnable</code> is used
         * to create a thread, starting the thread causes the object's
         * <code>run</code> method to be called in that separately executing
         * thread.
         * <p>
         * The general contract of the method <code>run</code> is that it may
         * take any action whatsoever.
         *
         * @see Thread#run()
         */
        @Override
        public void run() {
            jdbcTemplate.batchUpdate("insert into multiple_thread_insert_table (id, context1, context2) values (?, ?, ?)", data);
        }
    }
}
spring:
  datasource:
    password: Root123$
    username: bes
    url: jdbc:postgresql://192.168.1.180:15400/opslinkaiops
    driver-class-name: org.postgresql.Driver
    hikari:
      auto-commit: true
      minimum-idle: 10
      maximum-pool-size: 50
      max-lifetime: 1800000
      connection-test-query: select 1

目前是这样的,url后面不加?batchMode=true这个会出现master节点挂掉的情况,但后面我在查文档后发现在url后面加上?batchMode=true之后可以解决这个问题 ,但这是一个不稳定的因素,是否能给出比较权危的解答

暂无图片 评论
暂无图片 有用 0
回答交流
提交
问题信息
请登录之后查看
邀请回答
暂无人订阅该标签,敬请期待~~
暂无图片墨值悬赏