kedro package 是一个比较方便的能力，可以将我们开发的data pipeline 打包为python 包，方便分发以及使用，以下简单说明下内部实现
参考内部处理
- 代码
可以看到内部并没有特殊的地方，对于标准python whl 文件基于了build 模块进行构建，对于配置部分使用了tar 命令进行处理
@project_group.command()
@click.pass_obj # this will pass the metadata as first argument
def package(metadata: ProjectMetadata) -> None:
"""Package the project as a Python wheel."""
# Even if the user decides for the older setup.py on purpose,
# pyproject.toml is needed for Kedro metadata
if (metadata.project_path / "pyproject.toml").is_file():
metadata_dir = metadata.project_path
destination_dir = "dist"
else:
# Assume it's an old Kedro project, packaging metadata was under src
# (could be pyproject.toml or setup.py, it's not important)
metadata_dir = metadata.source_dir
destination_dir = "../dist"
call(
[
sys.executable,
"-m",
"build",
"--wheel",
"--outdir",
destination_dir,
],
cwd=str(metadata_dir),
)
directory = (
str(Path(settings.CONF_SOURCE).parent)
if settings.CONF_SOURCE != "conf"
else metadata.project_path
)
call(
[
"tar",
"--exclude=local/*.yml",
"-czf",
f"dist/conf-{metadata.package_name}.tar.gz",
f"--directory={directory}",
str(Path(settings.CONF_SOURCE).stem),
]
)
参考资料
https://docs.kedro.org/en/stable/kedro_project_setup/settings.html
https://docs.kedro.org/en/stable/tutorial/package_a_project.html