windows本地部署dify

标签：string dify windows dataset -- 本地 document Type id

Dify与之前的MaxKB不同，MaxKB可以实现基础的问答以及知识库功能，但是如果要开发一个Agent，或者工作流就还是需要额外开发，而Dify 是一个开源 LLM 应用开发平台。其直观的界面结合了 AI 工作流、RAG 管道、代理功能、模型管理、可观察性功能等，让您可以快速从原型开发到生产。以下是核心功能的列表：
1. 工作流程：在可视化画布上构建和测试强大的 AI 工作流程，利用以下所有功能及其他功能。

2. 全面的模型支持：与数十家推理提供商和自托管解决方案的数百个专有/开源 LLM 无缝集成，涵盖 GPT、Mistral、Llama3 和任何兼容 OpenAI API 的模型。支持的模型提供商的完整列表可在此处找到。

3. Prompt IDE：直观的界面，用于制作提示、比较模型性能以及向基于聊天的应用程序添加文本转语音等附加功能。

4. RAG Pipeline：广泛的 RAG 功能涵盖从文档提取到检索的所有内容，并提供从 PDF、PPT 和其他常见文档格式中提取文本的开箱即用支持。

5. 代理能力：您可以基于 LLM Function Calling 或 ReAct 定义代理，并为代理添加预构建或自定义工具。Dify 为 AI 代理提供了 50+ 内置工具，例如 Google Search、DELL·E、Stable Diffusion 和 WolframAlpha。

6.LLMOps：监控和分析应用程序日志和性能。您可以根据生产数据和注释不断改进提示、数据集和模型。

7. 后端即服务：Dify 的所有产品都配有相应的 API，因此您可以毫不费力地将 Dify 集成到您自己的业务逻辑中。

这节一起来实现Dify的离线部署，官方的github仓库也提供了docker的部署方式，我们也使用docker的方式部署，因此需要在window上安装docker环境，安装好后，我们先从github将源码下载下来，然后解压到一个文件夹下，这里是我下载后解压的项目目录

进入docker文件夹，打开docker-compose.yaml,里面是docker的配置文件，因为项目涉及都多个组件nginx，redis，postgre等。所以有端口冲突可以在此修改，

修改后，打开命令提示符，进入项目的docker目录下，输入命令

docker-compose up -d

当所有组件pull不并started后就可以了，打开浏览器，输入IP+port,如果刚才映射的端口被修改，要记得这里的port是修改映射后的新端口，

第一次进入后会要求设置管理员账号，设置完成后就可以登录使用了

配置一下知识库相关的apikey就可以在本地使用了

知识库相关的api维护

知识库 API

鉴权

Dify Service API 使用 API-Key 进行鉴权。

建议开发者把 API-Key 放在后端存储，而非分享或者放在客户端存储，以免 API-Key 泄露，导致财产损失。

所有 API 请求都应在 Authorization HTTP Header 中包含您的 API-Key，如下所示：

Code

Authorization: Bearer {API_KEY}

CopyCopied!

POST/datasets/{dataset_id}/document/create_by_text

通过文本创建文档

此接口基于已存在知识库，在此知识库的基础上通过文本创建新的文档

Path

Name

dataset_id

Type

string

Description

知识库 ID

Request Body

Name

name

Type

string

Description

文档名称
Name

text

Type

string

Description

文档内容
Name

indexing_technique

Type

string

Description

索引方式
- high_quality 高质量：使用 embedding 模型进行嵌入，构建为向量数据库索引
- economy 经济：使用 Keyword Table Index 的倒排索引进行构建
Name

process_rule

Type

object

Description

处理规则
- mode (string) 清洗、分段模式，automatic 自动 / custom 自定义
- rules (object) 自定义规则（自动模式下，该字段为空）
  - pre_processing_rules (array[object]) 预处理规则
    - id (string) 预处理规则的唯一标识符
      - 枚举：
        remove_extra_spaces 替换连续空格、换行符、制表符
        remove_urls_emails 删除 URL、电子邮件地址
    - enabled (bool) 是否选中该规则，不传入文档 ID 时代表默认值
  - segmentation (object) 分段规则
    - separator 自定义分段标识符，目前仅允许设置一个分隔符。默认为 \n
    - max_tokens 最大长度 (token) 默认为 1000

Request

POST

/datasets/{dataset_id}/document/create_by_text

curl --location --request POST 'http://localhost/v1/datasets/{dataset_id}/document/create_by_text' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json' \
--data-raw '{"name": "text","text": "text","indexing_technique": "high_quality","process_rule": {"mode": "automatic"}}'

CopyCopied!

Response

{ "document": { "id": "", "position": 1, "data_source_type": "upload_file", "data_source_info": { "upload_file_id": "" }, "dataset_process_rule_id": "", "name": "text.txt", "created_from": "api", "created_by": "", "created_at": 1695690280, "tokens": 0, "indexing_status": "waiting", "error": null, "enabled": true, "disabled_at": null, "disabled_by": null, "archived": false, "display_status": "queuing", "word_count": 0, "hit_count": 0, "doc_form": "text_model" }, "batch": "" }

CopyCopied!

POST/datasets/{dataset_id}/document/create_by_file

通过文件创建文档

此接口基于已存在知识库，在此知识库的基础上通过文件创建新的文档

Path

Name

dataset_id

Type

string

Description

知识库 ID

Request Bodys

Name

data

Type

multipart/form-data json string

Description
- original_document_id 源文档 ID （选填）
  - 用于重新上传文档或修改文档清洗、分段配置，缺失的信息从源文档复制
  - 源文档不可为归档的文档
  - 当传入 original_document_id 时，代表文档进行更新操作，process_rule 为可填项目，不填默认使用源文档的分段方式
  - 未传入 original_document_id 时，代表文档进行新增操作，process_rule 为必填
- indexing_technique 索引方式
  - high_quality 高质量：使用 embedding 模型进行嵌入，构建为向量数据库索引
  - economy 经济：使用 Keyword Table Index 的倒排索引进行构建
- process_rule 处理规则
  - mode (string) 清洗、分段模式，automatic 自动 / custom 自定义
  - rules (object) 自定义规则（自动模式下，该字段为空）
    - pre_processing_rules (array[object]) 预处理规则
      - id (string) 预处理规则的唯一标识符
        枚举：
        remove_extra_spaces 替换连续空格、换行符、制表符
        remove_urls_emails 删除 URL、电子邮件地址
      - enabled (bool) 是否选中该规则，不传入文档 ID 时代表默认值
    - segmentation (object) 分段规则
      - separator 自定义分段标识符，目前仅允许设置一个分隔符。默认为 \n
      - max_tokens 最大长度 (token) 默认为 1000
Name

file

Type

multipart/form-data

Description

需要上传的文件。

Request

POST

/datasets/{dataset_id}/document/create_by_file

curl --location --request POST 'http://localhost/v1/datasets/{dataset_id}/document/create_by_file' \
--header 'Authorization: Bearer {api_key}' \
--form 'data="{"indexing_technique":"high_quality","process_rule":{"rules":{"pre_processing_rules":[{"id":"remove_extra_spaces","enabled":true},{"id":"remove_urls_emails","enabled":true}],"segmentation":{"separator":"###","max_tokens":500}},"mode":"custom"}}";type=text/plain' \
--form 'file=@"/path/to/file"'

CopyCopied!

Response

{ "document": { "id": "", "position": 1, "data_source_type": "upload_file", "data_source_info": { "upload_file_id": "" }, "dataset_process_rule_id": "", "name": "Dify.txt", "created_from": "api", "created_by": "", "created_at": 1695308667, "tokens": 0, "indexing_status": "waiting", "error": null, "enabled": true, "disabled_at": null, "disabled_by": null, "archived": false, "display_status": "queuing", "word_count": 0, "hit_count": 0, "doc_form": "text_model" }, "batch": "" }

CopyCopied!

POST/datasets

创建空知识库

Request Body

Name

name

Type

string

Description

知识库名称

Request

POST

/datasets

curl --location --request POST 'http://localhost/v1/datasets' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json' \
--data-raw '{"name": "name"}'

CopyCopied!

Response

{ "id": "", "name": "name", "description": null, "provider": "vendor", "permission": "only_me", "data_source_type": null, "indexing_technique": null, "app_count": 0, "document_count": 0, "word_count": 0, "created_by": "", "created_at": 1695636173, "updated_by": "", "updated_at": 1695636173, "embedding_model": null, "embedding_model_provider": null, "embedding_available": null }

CopyCopied!

GET/datasets

知识库列表

Query

Name

page

Type

string

Description

页码
Name

limit

Type

string

Description

返回条数，默认 20，范围 1-100

Request

POST

/datasets

curl --location --request GET 'http://localhost/v1/datasets?page=1&limit=20' \
--header 'Authorization: Bearer {api_key}'

CopyCopied!

Response

{ "data": [ { "id": "", "name": "知识库名称", "description": "描述信息", "permission": "only_me", "data_source_type": "upload_file", "indexing_technique": "", "app_count": 2, "document_count": 10, "word_count": 1200, "created_by": "", "created_at": "", "updated_by": "", "updated_at": "" }, ... ], "has_more": true, "limit": 20, "total": 50, "page": 1 }

CopyCopied!

POST/datasets/{dataset_id}/documents/{document_id}/update_by_text

通过文本更新文档

此接口基于已存在知识库，在此知识库的基础上通过文本更新文档

Path

Name

dataset_id

Type

string

Description

知识库 ID
Name

document_id

Type

string

Description

文档 ID

Request Body

Name

name

Type

string

Description

文档名称（选填）
Name

text

Type

string

Description

文档内容（选填）
Name

process_rule

Type

object

Description

处理规则（选填）
- mode (string) 清洗、分段模式，automatic 自动 / custom 自定义
- rules (object) 自定义规则（自动模式下，该字段为空）
  - pre_processing_rules (array[object]) 预处理规则
    - id (string) 预处理规则的唯一标识符
      - 枚举：
        remove_extra_spaces 替换连续空格、换行符、制表符
        remove_urls_emails 删除 URL、电子邮件地址
    - enabled (bool) 是否选中该规则，不传入文档 ID 时代表默认值
  - segmentation (object) 分段规则
    - separator 自定义分段标识符，目前仅允许设置一个分隔符。默认为 \n
    - max_tokens 最大长度 (token) 默认为 1000

Request

POST

/datasets/{dataset_id}/documents/{document_id}/update_by_text

curl --location --request POST 'http://localhost/v1/datasets/{dataset_id}/documents/{document_id}/update_by_text' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json' \
--data-raw '{"name": "name","text": "text"}'

CopyCopied!

Response

{ "document": { "id": "", "position": 1, "data_source_type": "upload_file", "data_source_info": { "upload_file_id": "" }, "dataset_process_rule_id": "", "name": "name.txt", "created_from": "api", "created_by": "", "created_at": 1695308667, "tokens": 0, "indexing_status": "waiting", "error": null, "enabled": true, "disabled_at": null, "disabled_by": null, "archived": false, "display_status": "queuing", "word_count": 0, "hit_count": 0, "doc_form": "text_model" }, "batch": "" }

CopyCopied!

POST/datasets/{dataset_id}/documents/{document_id}/update_by_file

通过文件更新文档

此接口基于已存在知识库，在此知识库的基础上通过文件更新文档的操作。

Path

Name

dataset_id

Type

string

Description

知识库 ID
Name

document_id

Type

string

Description

文档 ID

Request Body

Name

name

Type

string

Description

文档名称（选填）
Name

file

Type

multipart/form-data

Description

需要上传的文件
Name

process_rule

Type

object

Description

处理规则（选填）
- mode (string) 清洗、分段模式，automatic 自动 / custom 自定义
- rules (object) 自定义规则（自动模式下，该字段为空）
  - pre_processing_rules (array[object]) 预处理规则
    - id (string) 预处理规则的唯一标识符
      - 枚举：
        remove_extra_spaces 替换连续空格、换行符、制表符
        remove_urls_emails 删除 URL、电子邮件地址
    - enabled (bool) 是否选中该规则，不传入文档 ID 时代表默认值
  - segmentation (object) 分段规则
    - separator 自定义分段标识符，目前仅允许设置一个分隔符。默认为 \n
    - max_tokens 最大长度 (token) 默认为 1000

Request

POST

/datasets/{dataset_id}/documents/{document_id}/update_by_file

curl --location --request POST 'http://localhost/v1/datasets/{dataset_id}/documents/{document_id}/update_by_file' \
--header 'Authorization: Bearer {api_key}' \
--form 'data="{"name":"Dify","indexing_technique":"high_quality","process_rule":{"rules":{"pre_processing_rules":[{"id":"remove_extra_spaces","enabled":true},{"id":"remove_urls_emails","enabled":true}],"segmentation":{"separator":"###","max_tokens":500}},"mode":"custom"}}";type=text/plain' \
--form 'file=@"/path/to/file"'

CopyCopied!

Response

{ "document": { "id": "", "position": 1, "data_source_type": "upload_file", "data_source_info": { "upload_file_id": "" }, "dataset_process_rule_id": "", "name": "Dify.txt", "created_from": "api", "created_by": "", "created_at": 1695308667, "tokens": 0, "indexing_status": "waiting", "error": null, "enabled": true, "disabled_at": null, "disabled_by": null, "archived": false, "display_status": "queuing", "word_count": 0, "hit_count": 0, "doc_form": "text_model" }, "batch": "20230921150427533684" }

CopyCopied!

GET/datasets/{dataset_id}/documents/{batch}/indexing-status

获取文档嵌入状态（进度）

Path

Name

dataset_id

Type

string

Description

知识库 ID
Name

batch

Type

string

Description

上传文档的批次号

Request

GET

/datasets/{dataset_id}/documents/{batch}/indexing-status

curl --location --request GET 'http://localhost/v1/datasets/{dataset_id}/documents/{batch}/indexing-status' \
--header 'Authorization: Bearer {api_key}'

CopyCopied!

Response

{ "data":[{ "id": "", "indexing_status": "indexing", "processing_started_at": 1681623462.0, "parsing_completed_at": 1681623462.0, "cleaning_completed_at": 1681623462.0, "splitting_completed_at": 1681623462.0, "completed_at": null, "paused_at": null, "error": null, "stopped_at": null, "completed_segments": 24, "total_segments": 100 }] }

CopyCopied!

DELETE/datasets/{dataset_id}/documents/{document_id}

删除文档

Path

Name

dataset_id

Type

string

Description

知识库 ID
Name

document_id

Type

string

Description

文档 ID

Request

DELETE

/datasets/{dataset_id}/documents/{document_id}

curl --location --request DELETE 'http://localhost/v1/datasets/{dataset_id}/documents/{document_id}' \
--header 'Authorization: Bearer {api_key}'

CopyCopied!

Response

{ "result": "success" }

CopyCopied!

GET/datasets/{dataset_id}/documents

知识库文档列表

Path

Name

dataset_id

Type

string

Description

知识库 ID

Query

Name

keyword

Type

string

Description

搜索关键词，可选，目前仅搜索文档名称
Name

page

Type

string

Description

页码，可选
Name

limit

Type

string

Description

返回条数，可选，默认 20，范围 1-100

Request

GET

/datasets/{dataset_id}/documents

curl --location --request GET 'http://localhost/v1/datasets/{dataset_id}/documents' \
--header 'Authorization: Bearer {api_key}'

CopyCopied!

Response

{ "data": [ { "id": "", "position": 1, "data_source_type": "file_upload", "data_source_info": null, "dataset_process_rule_id": null, "name": "dify", "created_from": "", "created_by": "", "created_at": 1681623639, "tokens": 0, "indexing_status": "waiting", "error": null, "enabled": true, "disabled_at": null, "disabled_by": null, "archived": false }, ], "has_more": false, "limit": 20, "total": 9, "page": 1 }

CopyCopied!

POST/datasets/{dataset_id}/documents/{document_id}/segments

新增分段

Path

Name

dataset_id

Type

string

Description

知识库 ID
Name

document_id

Type

string

Description

文档 ID

Request Body

Name

segments

Type

object list

Description
- content (text) 文本内容/问题内容，必填
- answer (text) 答案内容，非必填，如果知识库的模式为qa模式则传值
- keywords (list) 关键字，非必填

Request

POST

/datasets/{dataset_id}/documents/{document_id}/segments

curl --location --request POST 'http://localhost/v1/datasets/{dataset_id}/documents/{document_id}/segments' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json' \
--data-raw '{"segments": [{"content": "1","answer": "1","keywords": ["a"]}]}'

CopyCopied!

Response

{ "data": [{ "id": "", "position": 1, "document_id": "", "content": "1", "answer": "1", "word_count": 25, "tokens": 0, "keywords": [ "a" ], "index_node_id": "", "index_node_hash": "", "hit_count": 0, "enabled": true, "disabled_at": null, "disabled_by": null, "status": "completed", "created_by": "", "created_at": 1695312007, "indexing_at": 1695312007, "completed_at": 1695312007, "error": null, "stopped_at": null }], "doc_form": "text_model" }

CopyCopied!

GET/datasets/{dataset_id}/documents/{document_id}/segments

查询文档分段

Path

Name

dataset_id

Type

string

Description

知识库 ID
Name

document_id

Type

string

Description

文档 ID

Query

Name

keyword

Type

string

Description

搜索关键词，可选
Name

status

Type

string

Description

搜索状态，completed

Request

GET

/datasets/{dataset_id}/documents/{document_id}/segments

curl --location --request GET 'http://localhost/v1/datasets/{dataset_id}/documents/{document_id}/segments' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json'

CopyCopied!

Response

{ "data": [{ "id": "", "position": 1, "document_id": "", "content": "1", "answer": "1", "word_count": 25, "tokens": 0, "keywords": [ "a" ], "index_node_id": "", "index_node_hash": "", "hit_count": 0, "enabled": true, "disabled_at": null, "disabled_by": null, "status": "completed", "created_by": "", "created_at": 1695312007, "indexing_at": 1695312007, "completed_at": 1695312007, "error": null, "stopped_at": null }], "doc_form": "text_model" }

CopyCopied!

DELETE/datasets/{dataset_id}/documents/{document_id}/segments/{segment_id}

删除文档分段

Path

Name

dataset_id

Type

string

Description

知识库 ID
Name

segment_id

Type

string

Description

文档分段ID

Request

DELETE

/datasets/{dataset_id}/documents/{document_id}/segments/{segment_id}

curl --location --request DELETE 'http://localhost/v1/datasets/{dataset_id}/documents/{document_id}/segments/{segment_id}' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json'

CopyCopied!

Response

{ "result": "success" }

CopyCopied!

POST/datasets/{dataset_id}/documents/{document_id}/segments/{segment_id}

更新文档分段

POST

Name

dataset_id

Type

string

Description

知识库 ID
Name

segment_id

Type

string

Description

文档分段ID

Request Body

Name

segment

Type

object list

Description
- content (text) 文本内容/问题内容，必填
- answer (text) 答案内容，非必填，如果知识库的模式为qa模式则传值
- keywords (list) 关键字，非必填
- enabled (bool) false/true，非必填

Request

POST

/datasets/{dataset_id}/documents/{document_id}/segments/{segment_id}

curl --location --request POST 'http://localhost/v1/datasets/{dataset_id}/documents/{document_id}/segments/{segment_id}' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json'\
--data-raw '{"segment": {"content": "1","answer": "1", "keywords": ["a"], "enabled": false}}'

CopyCopied!

Response

{ "data": [{ "id": "", "position": 1, "document_id": "", "content": "1", "answer": "1", "word_count": 25, "tokens": 0, "keywords": [ "a" ], "index_node_id": "", "index_node_hash": "", "hit_count": 0, "enabled": true, "disabled_at": null, "disabled_by": null, "status": "completed", "created_by": "", "created_at": 1695312007, "indexing_at": 1695312007, "completed_at": 1695312007, "error": null, "stopped_at": null }], "doc_form": "text_model" }

CopyCopied!

错误信息

Name

code

Type

string

Description

返回的错误代码

Name

status

Type

number

Description

返回的错误状态

Name

message

Type

string

Description

返回的错误信息

Example

{ "code": "no_file_uploaded", "message": "Please upload your file.", "status": 400 }

CopyCopied!

code	status	message
no_file_uploaded	400	Please upload your file.
too_many_files	400	Only one file is allowed.
file_too_large	413	File size exceeded.
unsupported_file_type	415	File type not allowed.
high_quality_dataset_only	400	Current operation only supports 'high-quality' datasets.
dataset_not_initialized	400	The dataset is still being initialized or indexing. Please wait a moment.
archived_document_immutable	403	The archived document is not editable.
dataset_name_duplicate	409	The dataset name already exists. Please modify your dataset name.
invalid_action	400	Invalid action.
document_already_finished	400	The document has been processed. Please refresh the page or go to the document details.
document_indexing	400	The document is being processed and cannot be edited.
invalid_metadata	400	The metadata content is incorrect. Please check and verify.

标签：string,dify,windows,dataset,--,本地,document,Type,id
From： https://blog.csdn.net/m0_62458145/article/details/139808629

知识库 API

鉴权

Code

Path

Request Body

Request

Response

Path

Request Bodys

Request

Response

Request Body

Request

Response

Query

Request

Response

Path

Request Body

Request

Response

Path

Request Body

Request

Response

Path

Request

Response

Path

Request

Response

Path

Query

Request

Response

Path

Request Body

Request

Response

Path

Query

Request

Response

Path

Request

Response

POST

Request Body

Request

Response

错误信息

Example

相关文章

赞助商

阅读排行