copy into from 是dremio 24 版本很不错的一个新功能，可以提升数据加载处理的速度（以前对于iceberg 数据的处理有insert into 以及通过ctas 模式）
简单机制说明
目前看到copy into from 并不象其他sql 命令基于sql SimpleDirectHandler 模式，而是通过sql 关系转换的
DremioSqlToRelConverter 中的
private RelRoot convertOther(SqlNode query, boolean top, RelDataType targetRowType) {
if (query instanceof SqlOptimize) {
NamespaceKey path = ((SqlOptimize)query).getPath();
Prepare.PreparingTable nsTable = catalogReader.getTable(path.getPathComponents());
return RelRoot.of(new TableOptimizeCrel(cluster, cluster.traitSetOf(Convention.NONE), nsTable.toRel(createToRelContext()), nsTable, null, OptimizeOptions.DEFAULT), SqlKind.OTHER);
} else if (query instanceof SqlCopyIntoTable) {
NamespaceKey path = ((SqlCopyIntoTable)query).getPath();
Prepare.PreparingTable nsTable = catalogReader.getTable(path.getPathComponents());
CopyIntoTableContext copyIntoTableContext= new CopyIntoTableContext((SqlCopyIntoTable)query);
return RelRoot.of(new CopyIntoTableCrel(cluster, cluster.traitSetOf(Convention.NONE), nsTable, copyIntoTableContext), SqlKind.OTHER);
} else {
return super.convertQueryRecursive(query, top, targetRowType);
}
}
几个核心类
主要是CopyIntoTableContext，CopyIntoTableCrel ，CopyIntoTableRel ，CopyIntoTablePrule,CopyIntoTableRule
CopyIntoTableRule 主要是为PlannerPhase 逻辑阶段提供规则（CopyIntoTableRule 使用了CopyIntoTableContext，CopyIntoTableCrel ，CopyIntoTableRel ）
CopyIntoTablePrule 主要是为PlannerPhase 物理计划规则提供规则，实际物理规则的生成是通过了CopyIntoTablePlanBuilder ，里边包含了实际物理规则的处理，值得学习参考
说明
以上是简单的说明，内部好多细节是值得学习参考的，而且目前copy into from 属于append 模式的（也比较符合日常数据湖数据处理模式）
参考资料
sabot/kernel/src/main/java/com/dremio/exec/planner/sql/handlers/query/CopyIntoTableContext.java
sabot/kernel/src/main/java/com/dremio/exec/planner/sql/DremioSqlToRelConverter.java
sabot/kernel/src/main/java/com/dremio/exec/calcite/logical/CopyIntoTableCrel.java
sabot/kernel/src/main/java/com/dremio/exec/planner/physical/CopyIntoTablePrule.java
sabot/kernel/src/main/java/com/dremio/exec/planner/PlannerPhase.java
sabot/kernel/src/main/java/com/dremio/exec/planner/CopyIntoTablePlanBuilder.java
sabot/kernel/src/main/java/com/dremio/exec/planner/logical/CopyIntoTableRule.java
https://docs.dremio.com/software/data-formats/copying-data-into-tables/