【发布时间】:2017-12-31 18:06:48
【问题描述】:
我需要在我的 MySQL 数据库中进行批量插入(近 10000 次)。我正在使用 JPA/hibernate 和 spring boot。我从休眠文档中阅读了在休眠中执行批量插入/更新,我认为我的代码无法正常工作,因为它是按顺序插入休眠查询而不是在批量插入中执行它们。下面是我的代码。我错过了什么吗?
这是我的数据源配置。
@Component
public class Datasource {
@Autowired
EnvConfiguration configuration;
private static final Logger logger = LoggerFactory.getLogger(Datasource.class);
@Bean
public DataSource dataSource(){
logger.info("DataSource Bean creation...");
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(configuration.getDBDriver());
dataSource.setUrl("jdbc:mysql://"+configuration.getDBIp()+":"+configuration.getDBPort()+"/"+configuration.getDBName()+"?autoReconnect=true&useSSL=false");
dataSource.setUsername(configuration.getDBUser());
dataSource.setPassword(configuration.getDBPass().trim());
return dataSource;
}
@Bean
public HibernateJpaSessionFactoryBean sessionFactory() {
return new HibernateJpaSessionFactoryBean();
}
}
我的角色域的代码
//Role.java
@Entity
@Table(name = "Role",uniqueConstraints = @UniqueConstraint(
columnNames = { "roleName"}))
public class Role {
@Id
@GeneratedValue(strategy = GenerationType.AUTO)
private Long roleId;
@NotNull
private String roleName;
public Role(){}
public Role(String roleName){
this.roleName = roleName;
}
public Long getRoleId() {
return roleId;
}
public void setRoleId(Long roleId) {
this.roleId = roleId;
}
public String getRoleName() {
return roleName;
}
public void setRoleName(String roleName) {
this.roleName = roleName;
}
}
以下是我的服务代码。在这里,我手动刷新会话。我添加了一个睡眠功能,以查找插入查询是一个一个执行,还是像 JDBC 批处理那样以 10 个为一组执行。
@Service
public class RoleService{
@Autowired
private SessionFactory factory;
@Autowired
private DataSource source;
private static final Logger logger = LoggerFactory.getLogger(WalletService.class);
public void insertRole(Collection<RegisterWallet> walletMetaCollection){
if(factory==null){
System.out.println("factory is null");
}else{
System.out.println("factory is working");
Session session = factory.openSession();
Transaction tx = session.beginTransaction();
for ( int i=0; i<100000; i++ ) {
Role role=new Role(""+i);
session.persist(role);
System.out.println("this is the role id "+role.getRoleId());
try {
Thread.sleep(1000);
} catch (InterruptedException e){
// TODO Auto-generated catch block
e.printStackTrace();
}
if ( i % 10 == 0 ) { //20, same as the JDBC batch size
//flush a batch of inserts and release memory:
session.flush();
session.clear();
}
}
tx.commit();
session.close();
}
}
}
根据我对批处理操作的理解,应该一次插入 10 个角色,减少使用的 jdbc 往返次数。但是上面代码的输出是出乎意料的。每个 session.persist(..) 调用执行一次插入。
//This is log of the above code.
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 14
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 15
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 16
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 17
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 18
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 19
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 20
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 21
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 22
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 23
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 24
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 25
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 26
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 27
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 28
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 29
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 30
Hibernate: insert into role (active, role_description, role_name) values (?, ?, ?)
this is the role id 31
</pre>
-------------------------------------------------------------
以下是我的 application.properties 配置
spring.jpa.properties.hibernate.current_session_context_class=org.springframework.orm.hibernate4.SpringSessionContext
spring.jpa.properties.hibernate.jdbc.batch_size=10
我错过了什么吗?
请帮忙。
【问题讨论】:
-
Thanx M.Deinum 确实很有帮助。再次感谢。你使我免于许多麻烦。自从过去 3 天以来,我一直对这个问题感到困惑。干杯快乐编码
标签: mysql hibernate spring-boot batch-processing