Sharding-JDBC认为对于分片策略存有两种维度:
- 数据源分片策略(DatabaseShardingStrategy):数据被分配的目标数据源
- 表分片策略(TableShardingStrategy):数据被分配的目标表
两种分片策略API完全相同,但是表分片策略是依赖于数据源分片策略的(即:先分库然后才有分表)
二、分片算法
Sharding分片策略继承自ShardingStrategy,提供了5种分片策略:
由于分片算法和业务实现紧密相关,因此Sharding-JDBC并未提供内置分片算法,而是通过分片策略将各种场景提炼出来,提供更高层级的抽象,并提供接口让应用开发者自行实现分片算法。
StandardShardingStrategy
标准分片策略。提供对SQL语句中的=, IN和BETWEEN AND的分片操作支持。
StandardShardingStrategy只支持单分片键,提供PreciseShardingAlgorithm和RangeShardingAlgorithm两个分片算法。
- PreciseShardingAlgorithm是必选的,用于处理=和IN的分片。
- RangeShardingAlgorithm是可选的,用于处理BETWEEN AND分片,如果不配置RangeShardingAlgorithm,SQL中的BETWEEN AND将按照全库路由处理。
ComplexShardingStrategy
复合分片策略。提供对SQL语句中的=, IN和BETWEEN AND的分片操作支持。
ComplexShardingStrategy支持多分片键,由于多分片键之间的关系复杂,因此Sharding-JDBC并未做过多的封装,而是直接将分片键值组合以及分片操作符交于算法接口,完全由应用开发者实现,提供最大的灵活度。
InlineShardingStrategy
Inline表达式分片策略。使用Groovy的Inline表达式,提供对SQL语句中的=和IN的分片操作支持。
InlineShardingStrategy只支持单分片键,对于简单的分片算法,可以通过简单的配置使用,从而避免繁琐的Java代码开发,如: tuser${user_id % 8} 表示t_user表按照user_id按8取模分成8个表,表名称为t_user_0到t_user_7。
HintShardingStrategy
通过Hint而非SQL解析的方式分片的策略。
NoneShardingStrategy
不分片的策略。
三、自定义分片算法
Sharding提供了以下4种算法接口:
- PreciseShardingAlgorithm
- RangeShardingAlgorithm
- HintShardingAlgorithm
- ComplexKeysShardingAlgorithm
可以自己实现自定义的分片算法,下面以t_order_items表为例自己实现分片算法:
标准分片策略(StandardShardingStrategy)
a、PreciseShardingAlgorithm实现:(Precise处理 = 和 in 的路由)
// 配置order_item表规则... TableRuleConfiguration orderItemTableRuleConfig = new TableRuleConfiguration(); orderItemTableRuleConfig.setLogicTable("t_order_items"); orderItemTableRuleConfig.setActualDataNodes("db${0..2}.t_order_items_${0..1}"); // 自定义的分片算法实现 StandardShardingStrategyConfiguration standardStrategy = new StandardShardingStrategyConfiguration("order_id",MyPreciseShardingAlgorithm.class.getName()); // 配置分库策略 orderItemTableRuleConfig.setDatabaseShardingStrategyConfig(standardStrategy); // 配置分表策略 orderItemTableRuleConfig.setTableShardingStrategyConfig(standardStrategy); shardingRuleConfig.getTableRuleConfigs().add(orderItemTableRuleConfig); // 获取数据源对象 DataSource dataSource = null; try { dataSource = ShardingDataSourceFactory.createDataSource(dataSourceMap, shardingRuleConfig, new ConcurrentHashMap(), new Properties()); } catch (SQLException e) { e.printStackTrace(); } return dataSource;
自定义的分片算法,先继承接口,打印参数:
@Slf4j public class MyPreciseShardingAlgorithm implements PreciseShardingAlgorithm<Long> { @Override public String doSharding(Collection collection, PreciseShardingValue<Long> preciseShardingValue) { log.info("collection:" + JSON.toJSONString(collection) + ",preciseShardingValue:" + JSON.toJSONString(preciseShardingValue)); return null; } }
输出如下日志:(第一行路由是db,下一行是table)
2018-01-19 20:13:36,790 -2 collection:["db0","db1","db2"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":100}
……
2018-01-21 16:33:22,269 -2 collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":100}
于是可以简单实现一个类似Inline配置的规则:
@Slf4j public class MyPreciseShardingAlgorithm implements PreciseShardingAlgorithm<Long> { @Override public String doSharding(Collection<String> collection, PreciseShardingValue<Long> preciseShardingValue) { log.info("collection:" + JSON.toJSONString(collection) + ",preciseShardingValue:" + JSON.toJSONString(preciseShardingValue)); for (String name : collection) { if (name.endsWith(preciseShardingValue.getValue() % collection.size() + "")) { log.info("return name:"+name); return name; } } return null; } }
IN 条件的处理示例:
==> Preparing: select id,order_id,unique_no,quantity,is_active,inserttime,updatetime from t_order_items where is_active = 1 AND order_id in ( ? , ? , ? ) ==> Parameters: 100(Long), 101(Long), 102(Long) //第一轮route筛选数据库(分片键路由规则): 09:55:09.634 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["db0","db1","db2"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":100} 09:55:13.758 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:db1 09:55:17.767 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["db0","db1","db2"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":101} 09:55:21.361 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:db2 09:55:23.127 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["db0","db1","db2"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":102} 09:55:24.190 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:db0 //第二轮route按第一轮筛选到的db,逐个进行table的计算: 09:58:45.086 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":100} 09:58:46.725 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_0 09:58:58.647 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":101} 09:59:02.197 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_1 09:59:11.710 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":102} 09:59:12.604 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_0 10:00:01.538 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":100} 10:00:01.538 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_0 10:00:02.042 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":101} 10:00:02.042 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_1 10:00:02.442 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":102} 10:00:02.442 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_0 10:00:03.581 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":100} 10:00:03.581 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_0 10:00:03.946 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":101} 10:00:03.946 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_1 10:00:04.578 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - collection:["t_order_items_0","t_order_items_1"],preciseShardingValue:{"columnName":"order_id","logicTableName":"t_order_items","value":102} 10:00:04.578 logback-demo [http-nio-8082-exec-1] INFO s.j.d.d.MyPreciseShardingAlgorithm - return name:t_order_items_0