Spring TransactionTemplate 不使用批量插入/更新答案

【问题标题】：Spring TransactionTemplate doesn't use batch insert/updateSpring TransactionTemplate 不使用批量插入/更新
【发布时间】：2020-08-15 00:20:06
【问题描述】：

我有一个使用 Spring Data REST、Spring JPA、Hibernate、Mysql 的 Spring Boot 2.3 应用程序。我有一个服务，我在其中编写了一个使用 Spring TransactionTemplate 的 @Async 方法，因为我需要决定何时提交事务。它有效，但最近我注意到即使设置了hibernate.jdbc.batch_size，我也没有利用它。

这是我的 Hibernate 配置：

@Configuration
@Profile({"dev", "stage", "prod"})
@EnableTransactionManagement
@EnableJpaRepositories(
        basePackages = "com.server",
        entityManagerFactoryRef = "entityManagerFactory",
        transactionManagerRef = "transactionManager"

)
public class HibernateConfig {

    @Autowired
    private Environment env;

    @Autowired
    private LocalValidatorFactoryBean validator;

    @Primary
    @Bean(name = "entityManagerFactory")
    public LocalContainerEntityManagerFactoryBean entityManagerFactory(MultiTenantConnectionProvider multiTenantConnectionProviderImpl,
                                                                       CurrentTenantIdentifierResolver currentTenantIdentifierResolverImpl) throws InstantiationException, IllegalAccessException,
            ClassNotFoundException, IllegalArgumentException, InvocationTargetException, NoSuchMethodException, SecurityException {
        HashMap jpaProperties = getJpaProperties(multiTenantConnectionProviderImpl, currentTenantIdentifierResolverImpl);
        LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
        em.setPersistenceUnitName("testPU");
        em.setDataSource(dataSource());
        em.setPackagesToScan("come.server");
        em.setJpaPropertyMap(jpaProperties);

        HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
        em.setJpaVendorAdapter(vendorAdapter);

        em.setJpaPropertyMap(jpaProperties);
        em.afterPropertiesSet();

        return em;
    }

    @Primary
    @Bean(name = "dataSource")
    public DataSource dataSource() throws InstantiationException, IllegalAccessException, ClassNotFoundException, IllegalArgumentException,
            InvocationTargetException, NoSuchMethodException, SecurityException {
        final SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
        dataSource.setDriver(((Driver) Class.forName(env.getProperty("primary.datasource.driver-class-name")).getDeclaredConstructor().newInstance()));
        dataSource.setUrl(env.getProperty("primary.datasource.url"));
        dataSource.setUsername(env.getProperty("primary.datasource.username"));
        dataSource.setPassword(env.getProperty("primary.datasource.password"));
        return dataSource;
    }

    @Primary
    @Bean(name = "transactionManager")
    public PlatformTransactionManager transactionManager(MultiTenantConnectionProvider multiTenantConnectionProviderImpl,
                                                         CurrentTenantIdentifierResolver currentTenantIdentifierResolverImpl) throws InstantiationException, IllegalAccessException,
            ClassNotFoundException, IllegalArgumentException, InvocationTargetException, NoSuchMethodException, SecurityException {
        JpaTransactionManager transactionManager = new JpaTransactionManager();
        transactionManager.setEntityManagerFactory(entityManagerFactory(multiTenantConnectionProviderImpl, currentTenantIdentifierResolverImpl).getObject());
        return transactionManager;
    }

    private HashMap getJpaProperties(MultiTenantConnectionProvider multiTenantConnectionProviderImpl, CurrentTenantIdentifierResolver currentTenantIdentifierResolverImpl) {
        HashMap properties = new HashMap<>();
        properties.put(AvailableSettings.MULTI_TENANT, MultiTenancyStrategy.DATABASE);
        properties.put(AvailableSettings.MULTI_TENANT_CONNECTION_PROVIDER, multiTenantConnectionProviderImpl);
        properties.put(AvailableSettings.MULTI_TENANT_IDENTIFIER_RESOLVER, currentTenantIdentifierResolverImpl);
        properties.put(AvailableSettings.HBM2DDL_AUTO, env.getProperty(AvailableSettings.HBM2DDL_AUTO));
        properties.put(AvailableSettings.GENERATE_STATISTICS, env.getProperty(AvailableSettings.GENERATE_STATISTICS));
        properties.put(AvailableSettings.DIALECT, env.getProperty(AvailableSettings.DIALECT));
        properties.put(AvailableSettings.STORAGE_ENGINE, env.getProperty(AvailableSettings.STORAGE_ENGINE));
        properties.put(AvailableSettings.SHOW_SQL, env.getProperty(AvailableSettings.SHOW_SQL));
        properties.put(AvailableSettings.IMPLICIT_NAMING_STRATEGY, env.getProperty(AvailableSettings.IMPLICIT_NAMING_STRATEGY));

        properties.put(AvailableSettings.STATEMENT_FETCH_SIZE, env.getProperty(AvailableSettings.STATEMENT_FETCH_SIZE));
        properties.put(AvailableSettings.STATEMENT_BATCH_SIZE, env.getProperty(AvailableSettings.STATEMENT_BATCH_SIZE));
        properties.put(AvailableSettings.MAX_FETCH_DEPTH, env.getProperty(AvailableSettings.MAX_FETCH_DEPTH));
        properties.put(AvailableSettings.USE_SECOND_LEVEL_CACHE, env.getProperty(AvailableSettings.USE_SECOND_LEVEL_CACHE));
        properties.put(AvailableSettings.CACHE_REGION_FACTORY, env.getProperty(AvailableSettings.CACHE_REGION_FACTORY));
        properties.put(AvailableSettings.JDBC_TIME_ZONE, env.getProperty(AvailableSettings.JDBC_TIME_ZONE));
        properties.put(AvailableSettings.GLOBALLY_QUOTED_IDENTIFIERS, env.getProperty(AvailableSettings.GLOBALLY_QUOTED_IDENTIFIERS));
        properties.put(AvailableSettings.GLOBALLY_QUOTED_IDENTIFIERS_SKIP_COLUMN_DEFINITIONS,
                env.getProperty(AvailableSettings.GLOBALLY_QUOTED_IDENTIFIERS_SKIP_COLUMN_DEFINITIONS));
        return properties;
    }
}

我有这些设置：

hibernate.jdbc.fetch_size = 15
hibernate.jdbc.batch_size = 30
hibernate.max_fetch_depth = 3

这是我服务的相关部分：

@Service
@Transactional
@Log4j2
public class StsService {

    @PersistenceContext(unitName = "testPU")
    private EntityManager entityManager;

    @Autowired
    private TransactionTemplate transactionTemplate;

    @Autowired
    private PlatformTransactionManager transactionManager;   

    @Autowired
    private Environment environment;


    @Async    
    public void collectDocumentToSend(String query, String tenantId) {
            //some stuff
            transactionTemplate.setTransactionManager(transactionManager);
            transactionTemplate.execute(status -> {
                try {
                    for (SendMessageBatchResponse res : response) {

                            //********************************************************************
                            // AUDIT LOG INSERT
                            //********************************************************************
                            AuditSts auditSts = new AuditSts();
                            auditSts.setParentType(Document.class.getSimpleName());
                            auditSts.setParentId(documentId);
                            auditSts.setText(MessageUtils.getMessage(LocaleContextHolder.getLocale(), "sts.queued"));
                            auditLogRepository.save(auditSts);
                        }
                    }
                } catch (Throwable e) {
                    log.error("", e);
                    status.setRollbackOnly();
                }
                return null;
            });
            page++;
        } while (resultPage.hasNext());
    }

简而言之，有一个循环在数据库中插入一些实体。在我的transactionTemplate 上设置断点我看到它包含正确的事务管理器和正确的 jpa 属性。

但是，当我检查 Mysql 日志时，我看到有很多插入：

2020-08-12T17:53:15.044809Z        56 Prepare   insert into `AuditLog` (`createdBy`, `createdDate`, `lastModifiedBy`, `lastModifiedDate`, `sid`, `version`, `operationType`, `parentId`, `parentType`, `remoteAddress`, `text`, `type`) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'AuditTest')
2020-08-12T17:53:15.080469Z        56 Long Data
2020-08-12T17:53:15.080629Z        56 Execute   insert into `AuditLog` (`createdBy`, `createdDate`, `lastModifiedBy`, `lastModifiedDate`, `sid`, `version`, `operationType`, `parentId`, `parentType`, `remoteAddress`, `text`, `type`) values ('9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.182000', '9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.182000', 'd6b8f1db-0bd7-47d3-ba84-4a79bf413d9b', 1, 'STS', 6091, 'Document', '192.168.1.1', 'La spesa è stata messa in coda per la gestione.', 'AuditTest')
2020-08-12T17:53:15.113893Z        56 Reset stmt
2020-08-12T17:53:15.147147Z        56 Reset stmt
2020-08-12T17:53:15.178657Z        56 Long Data
2020-08-12T17:53:15.178764Z        56 Execute   insert into `AuditLog` (`createdBy`, `createdDate`, `lastModifiedBy`, `lastModifiedDate`, `sid`, `version`, `operationType`, `parentId`, `parentType`, `remoteAddress`, `text`, `type`) values ('9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.304000', '9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.304000', 'bc4680d4-ed88-469c-8c08-32190d7de6f5', 1, 'STS', 6093, 'Document', '192.168.1.1', 'La spesa è stata messa in coda per la gestione.', 'AuditTest')
2020-08-12T17:53:15.211398Z        56 Reset stmt
2020-08-12T17:53:15.243390Z        56 Reset stmt
2020-08-12T17:53:15.274637Z        56 Long Data
2020-08-12T17:53:15.274779Z        56 Execute   insert into `AuditLog` (`createdBy`, `createdDate`, `lastModifiedBy`, `lastModifiedDate`, `sid`, `version`, `operationType`, `parentId`, `parentType`, `remoteAddress`, `text`, `type`) values ('9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.400000', '9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.400000', '6026019f-b2ae-476f-a1ea-05b1fb205484', 1, 'STS', 6094, 'Document', '192.168.1.1', 'La spesa è stata messa in coda per la gestione.', 'AuditTest')
2020-08-12T17:53:15.307137Z        56 Reset stmt
2020-08-12T17:53:15.339953Z        56 Reset stmt
2020-08-12T17:53:15.371412Z        56 Long Data
2020-08-12T17:53:15.371559Z        56 Execute   insert into `AuditLog` (`createdBy`, `createdDate`, `lastModifiedBy`, `lastModifiedDate`, `sid`, `version`, `operationType`, `parentId`, `parentType`, `remoteAddress`, `text`, `type`) values ('9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.496000', '9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.496000', '99247541-c69e-4453-a101-6faaca1b667a', 1, 'STS', 6095, 'Document', '192.168.1.1', 'La spesa è stata messa in coda per la gestione.', 'AuditTest')
2020-08-12T17:53:15.406132Z        56 Reset stmt
2020-08-12T17:53:15.437881Z        56 Reset stmt
2020-08-12T17:53:15.469131Z        56 Long Data
2020-08-12T17:53:15.469270Z        56 Execute   insert into `AuditLog` (`createdBy`, `createdDate`, `lastModifiedBy`, `lastModifiedDate`, `sid`, `version`, `operationType`, `parentId`, `parentType`, `remoteAddress`, `text`, `type`) values ('9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.595000', '9b9ef528-a45e-4ce9-b014-32a157507aef', '2020-08-12 17:53:15.595000', 'f97b70de-dfdf-46c8-b61a-303ea33b6d35', 1, 'STS', 6096, 'Document', '192.168.1.1', 'La spesa è stata messa in coda per la gestione.', 'AuditTest')
2020-08-12T17:53:15.501109Z        56 Reset stmt
2020-08-12T17:53:15.563477Z        56 Query     commit

我检查了 Hibernate 日志，但没有发现任何奇怪的地方。你有什么提示可以为我指明正确的方向吗？

【问题讨论】：

标签： java spring hibernate spring-data-jpa

【解决方案1】：

最后我理解了这个问题：如果您使用身份标识符生成器，Hibernate 会在 JDBC 级别透明地禁用插入批处理。查看此回复：Hibernate disabled insert batching when using an identity identifier generator

我所做的是使用 Jdbc 模板：

jdbcTemplate.batchUpdate("insert into `AuditLog` (`createdBy`, `createdDate`, `lastModifiedBy`, `lastModifiedDate`, `sid`, `version`, `operationType`, `parentId`, `parentType`, `remoteAddress`, `text`, `type`)" +
                            " values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", new BatchPreparedStatementSetter() {
                        @Override
                        public void setValues(PreparedStatement ps, int i)
                                throws SQLException {
                            Long documentId = (Long) documentsToUpdate.toArray()[i];
                            ps.setString(1, loggedAgent);
                            ps.setTimestamp(2, Timestamp.from(Instant.now()));
                            ps.setString(3, loggedAgent);
                            ps.setTimestamp(4, Timestamp.from(Instant.now()));
                            ps.setString(5, UUID.randomUUID().toString());
                            ps.setInt(6, 1);
                            ps.setString(7, "STS");
                            ps.setLong(8, documentId);
                            ps.setString(9, Document.class.getSimpleName());
                            ps.setString(10, NetworkUtils.getRemoteIpFromCurrentContext());
                            ps.setString(11, auditLogMessage);
                            ps.setString(12, AuditSts.class.getSimpleName());
                        }

                        @Override
                        public int getBatchSize() {
                            return documentsToUpdate.size();
                        }
                    });

插入数百个项目需要几毫秒；与之前相比，这是一个巨大的进步。

【讨论】：