sharding-jdbc路由的原理及应用
这篇文章主要介绍“sharding-jdbc路由的原理及应用”，在日常操作中，相信很多人在sharding-jdbc路由的原理及应用问题上存在疑惑，小编查阅了各式资料，整理出简单好用的操作方法，希望对大家解答”sharding-jdbc路由的原理及应用”的疑惑有所帮助！接下来，请跟着小编一起来学习吧！
成都创新互联公司是一家专注于成都网站建设、做网站与策划设计,隆回网站建设哪家好?成都创新互联公司做网站,专注于网站建设十多年,网设计领域的专业建站公司;建站业务涵盖:隆回等地区。隆回做网站价格咨询:18980820575
路由引擎主要分为两大类：
分片路由（直接路由、标准路由、笛卡尔积路由）
广播路由（全库表路由、全库路由、全实例路由、单播路由、阻断路由）
具体路由类型含义参考官网路由引擎
https://shardingsphere.apache.org/document/current/cn/features/sharding/principle/route/
主要分析查询路由
1.路由ParsingSQLRouter#route入口
@RequiredArgsConstructor
public final class ParsingSQLRouter implements ShardingRouter {
@Override
public SQLRouteResult route(final SQLStatement sqlStatement, final List2.路由工厂并路由RoutingEngineFactory#route
@NoArgsConstructor(access = AccessLevel.PRIVATE)
public final class RoutingEngineFactory {
/**
* Create new instance of routing engine.
*
* @param shardingRule sharding rule
* @param shardingDataSourceMetaData sharding data source meta data
* @param optimizedStatement optimized statement
* @return new instance of routing engine
*/
public static RoutingEngine newInstance(final ShardingRule shardingRule, final ShardingDataSourceMetaData shardingDataSourceMetaData, final OptimizedStatement optimizedStatement) {
SQLStatement sqlStatement = optimizedStatement.getSQLStatement();
Collection tableNames = sqlStatement.getTables().getTableNames();
//全库路由
if (sqlStatement instanceof TCLStatement) {
return new DatabaseBroadcastRoutingEngine(shardingRule);
}
//全库表路由
if (sqlStatement instanceof DDLStatement) {
return new TableBroadcastRoutingEngine(shardingRule, optimizedStatement);
}
//阻断路由
if (sqlStatement instanceof DALStatement) {
return getDALRoutingEngine(shardingRule, sqlStatement, tableNames);
}
//全实例路由
if (sqlStatement instanceof DCLStatement) {
return getDCLRoutingEngine(shardingRule, optimizedStatement, shardingDataSourceMetaData);
}
//默认库路由
if (shardingRule.isAllInDefaultDataSource(tableNames)) {
return new DefaultDatabaseRoutingEngine(shardingRule, tableNames);
}
//全库路由
if (shardingRule.isAllBroadcastTables(tableNames)) {
return sqlStatement instanceof SelectStatement ? new UnicastRoutingEngine(shardingRule, tableNames) : new DatabaseBroadcastRoutingEngine(shardingRule);
}
//单播路由
if (optimizedStatement instanceof ShardingWhereOptimizedStatement && ((ShardingWhereOptimizedStatement) optimizedStatement).getShardingConditions().isAlwaysFalse() || tableNames.isEmpty()) {
return new UnicastRoutingEngine(shardingRule, tableNames);
}
Preconditions.checkState(optimizedStatement instanceof ShardingWhereOptimizedStatement);
//分片路由
return getShardingRoutingEngine(shardingRule, (ShardingWhereOptimizedStatement) optimizedStatement, tableNames);
}
... ...
private static RoutingEngine getShardingRoutingEngine(final ShardingRule shardingRule, final ShardingWhereOptimizedStatement optimizedStatement, final Collection tableNames) {
///根据解析出来逻辑表获取分片表，如:SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id and o.order_id = ?
//则shardingTableNames为t_order，t_order_item
Collection shardingTableNames = shardingRule.getShardingLogicTableNames(tableNames);
//满足以下两个条件走标准路由，否则走复合路由
//1、是否只有一张分片表
//2、绑定的逻辑表（配置绑定表）是否包含所有分片表
if (1 == shardingTableNames.size() || shardingRule.isAllBindingTables(shardingTableNames)) {
//标准路由，获取第一张表路由即可，参考官网案例说明
return new StandardRoutingEngine(shardingRule, shardingTableNames.iterator().next(), optimizedStatement);
}
// TODO config for cartesian set
//复合路由
return new ComplexRoutingEngine(shardingRule, tableNames, optimizedStatement);
}
} 3.标准路由StandardRoutingEngine#route
标准路由场景
t_order和t_order_item是分库分表并且是绑定表；如第二步55行判断shardingTableNames.size()=1或者shardingTableNames都是绑定表时，这时会走标准路由
@RequiredArgsConstructor
public final class StandardRoutingEngine implements RoutingEngine {
//分库分表规则
private final ShardingRule shardingRule;
//逻辑表 t_order
private final String logicTableName;
//sql解析并优化后的结果
private final ShardingWhereOptimizedStatement optimizedStatement;
@Override
public RoutingResult route() {
//insert、update、delete判断表是否是单表
if (isDMLForModify(optimizedStatement.getSQLStatement()) && !optimizedStatement.getSQLStatement().getTables().isSingleTable()) {
throw new SQLParsingException("Cannot support Multiple-Table for '%s'.", optimizedStatement.getSQLStatement());
}
//路由数据节点、封装路由结果
return generateRoutingResult(getDataNodes(shardingRule.getTableRule(logicTableName)));
}
... ...
private RoutingResult generateRoutingResult(final Collection routedDataNodes) {
RoutingResult result = new RoutingResult();
//根据数据节点封装路由单元、表单元
for (DataNode each : routedDataNodes) {
//路由单元 demo_ds_0
RoutingUnit routingUnit = new RoutingUnit(each.getDataSourceName());
//表单元 逻辑表:真实表 t_order:t_order_0
routingUnit.getTableUnits().add(new TableUnit(logicTableName, each.getTableName()));
result.getRoutingUnits().add(routingUnit);
}
return result;
}
private Collection getDataNodes(final TableRule tableRule) {
//判断database、table分片策略同时是Hint(直接路由)
if (shardingRule.isRoutingByHint(tableRule)) {
return routeByHint(tableRule);
}
//database、table分片策略都不是Hint
if (isRoutingByShardingConditions(tableRule)) {
//根据分片条件、策略路由到对应的database、table，同对分片键判断
return routeByShardingConditions(tableRule);
}
//database或table分片策略有一个是Hint
return routeByMixedConditions(tableRule);
}
... ...
} 4.复合路由CartesianRoutingEngine#route
复杂路由场景
t_order和t_order_item是分库分表并且是绑定表；新增一个t_user分库分表，这时第二步的55行，shardingTableNames.size()=3，且t_user未配置成绑定表，这时会走复合路由
@RequiredArgsConstructor
public final class ComplexRoutingEngine implements RoutingEngine {
//分库分表规则
private final ShardingRule shardingRule;
//逻辑表t_order、t_order_item
private final Collection logicTables;
//sql解析并优化后的结果
private final ShardingWhereOptimizedStatement optimizedStatement;
@Override
public RoutingResult route() {
Collection result = new ArrayList<>(logicTables.size());
Collection bindingTableNames = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
//遍历逻辑表
for (String each : logicTables) {
Optional tableRule = shardingRule.findTableRule(each);
//表是否配置了分库分表规则
if (tableRule.isPresent()) {
// 如果绑定关系表已经处理过，那么不需要再处理，如t_order处理过，由于t_order_item与其是绑定关系，那么不需要再处理；
if (!bindingTableNames.contains(each)) {
//构建标准路由并路由
result.add(new StandardRoutingEngine(shardingRule, tableRule.get().getLogicTable(), optimizedStatement).route());
}
//根据逻辑表查找对应的所有绑定表，如根据t_order就能查询到t_order、t_order_item，因为t_order和t_order_item是绑定表
Optional bindingTableRule = shardingRule.findBindingTableRule(each);
if (bindingTableRule.isPresent()) {
//添加绑定表
bindingTableNames.addAll(Lists.transform(bindingTableRule.get().getTableRules(), new Function() {
@Override
public String apply(final TableRule input) {
return input.getLogicTable();
}
}));
}
}
}
if (result.isEmpty()) {
throw new ShardingException("Cannot find table rule and default data source with logic tables: '%s'", logicTables);
}
if (1 == result.size()) {
return result.iterator().next();
}
//笛卡尔积路由
return new CartesianRoutingEngine(result).route();
}
} 5.笛卡尔积路由
笛卡尔积路由场景
笛卡尔路由是最复杂的情况，它无法根据绑定表的关系定位分片规则，因此非绑定表之间的关联查询需要拆解为笛卡尔积组合执行。如上面事例新增的t_user分库分表，t_user和t_order未配置绑定表关系，如下SQL就需要走笛卡尔积路由
SELECT * FROM t_user u JOIN t_order o ON u.user_id = o.user_id AND u.user_id in(1 , 2)
@RequiredArgsConstructor
public final class CartesianRoutingEngine implements RoutingEngine {
private final Collection routingResults;
@Override
public RoutingResult route() {
RoutingResult result = new RoutingResult();
//获取数据源对应的逻辑表集合
for (Entry> entry : getDataSourceLogicTablesMap().entrySet()) {
//通过数据源名称和逻辑表的名称获取实际的表组，即[["t_user_0","t_user_1"],["t_order_0", "t_order_1]]
List> actualTableGroups = getActualTableGroups(entry.getKey(), entry.getValue());
//把逻辑表名封装成TableUnit表单元
//TableUnit包含数据源名称、逻辑表名、实际表名（通过这三个属性可以确定最终访问的表）
List> routingTableGroups = toRoutingTableGroups(entry.getKey(), actualTableGroups);
//封装RoutingUnit路由单元
//cartesianProduct计算笛卡尔积
result.getRoutingUnits().addAll(getRoutingUnits(entry.getKey(), Sets.cartesianProduct(routingTableGroups)));
}
return result;
}
private Map> getDataSourceLogicTablesMap() {
//获取数据源的交集，如t_user逻辑表路由到数据源demo_ds_0，而t_order表路由到数据源ds_demo_0和demo_ds_1，数据源交集就是demo_ds_0
//事例SELECT * FROM t_user_0 u JOIN t_order_0 o ON u.user_id = o.user_id WHERE u.user_id in(1, 2); t_user和t_order不是绑定表关系
//笛卡尔路由是最复杂的情况，它无法根据绑定表的关系定位分片规则，因此非绑定表之间的关联查询需要拆解为笛卡尔积组合执行
Collection intersectionDataSources = getIntersectionDataSources();
Map> result = new HashMap<>(routingResults.size());
//遍历标准路由后的结果集
for (RoutingResult each : routingResults) {
//通过数据源的名称获取数据源和逻辑表之间的映射关系
for (Entry> entry : each.getDataSourceLogicTablesMap(intersectionDataSources).entrySet()) {
if (result.containsKey(entry.getKey())) {
result.get(entry.getKey()).addAll(entry.getValue());
} else {
result.put(entry.getKey(), entry.getValue());
}
}
}
// 返回数据源-逻辑表集合组成的Map，这里就是{"demo_ds_0":["t_user", "t_order"]}
return result;
}
private Collection getIntersectionDataSources() {
Collection result = new HashSet<>();
for (RoutingResult each : routingResults) {
if (result.isEmpty()) {
result.addAll(each.getDataSourceNames());
}
//交集
result.retainAll(each.getDataSourceNames());
}
return result;
}
... ...
} 笛卡尔积结果如下：
6.直接路由
直接路由场景
满足直接路由的条件相对苛刻，它需要通过Hint（使用HintAPI直接指定路由至库表）方式分片，并且是只分库不分表的前提下，则可以避免SQL解析和之后的结果归并
假如路由算法为value % 2，当一个逻辑库t_order对应2个真实库t_order_0和t_order_1时，路由后SQL将在t_order_1上执行。下方是使用API的代码样例:
String sql = "SELECT * FROM t_order";
try (
//获取Hint实例
HintManager hintManager = HintManager.getInstance();
Connection conn = dataSource.getConnection();
PreparedStatement pstmt = conn.prepareStatement(sql)) {
//设置数据源分片个数
hintManager.setDatabaseShardingValue(3);
try (ResultSet rs = pstmt.executeQuery()) {
while (rs.next()) {
//...
}
}
}@RequiredArgsConstructor
public final class DatabaseHintRoutingEngine implements RoutingEngine {
//数据源集群
private final Collection dataSourceNames;
//Hint数据分片策略
private final HintShardingStrategy databaseShardingStrategy;
@Override
public RoutingResult route() {
//获取当前线程数据源分片
Collection> shardingValues = HintManager.getDatabaseShardingValues();
Preconditions.checkState(!shardingValues.isEmpty());
Collection routingDataSources;
//根据分片策略路由
routingDataSources = databaseShardingStrategy.doSharding(dataSourceNames, Collections.singletonList(new ListRouteValue<>("", "", shardingValues)));
Preconditions.checkState(!routingDataSources.isEmpty(), "no database route info");
RoutingResult result = new RoutingResult();
//封装路由单元
for (String each : routingDataSources) {
result.getRoutingUnits().add(new RoutingUnit(each));
}
return result;
}
} 到此，关于“sharding-jdbc路由的原理及应用”的学习就结束了，希望能够解决大家的疑惑。理论与实践的搭配能更好的帮助大家学习，快去试试吧！若想继续学习更多相关知识，请继续关注创新互联网站，小编会继续努力为大家带来更多实用的文章！
当前标题：sharding-jdbc路由的原理及应用
分享URL：http://azwzsj.com/article/pcedei.html
