3. TensorGuard: A RAG-Based Multi-agent framework to detect and fix DL Checker Bugs
RAG Design
relevant contextual information from a large corpus of code changes
Input: the root cause of the checker bug queried
Output: code change
Based on:
Sentence-transformers + all-MiniLM-L6-v2 as the embedding model, converts the documents to a 384-dimensional dense vector space
batch size: 50, chromadb
Checker Bug Detection Agent
COT, Zero-Shot, Few-Shot(随机选两个样例)
TABLE V: Prompt template for bug detection agent (COT).
“prompt”: You are an AI trained to detect bugs in a deep-learning
library based on commit messages and code changes. Your task
is to determine whether a given commit introduces a bug or not.
Follow the steps below to reason through the problem and arrive at
a conclusion.
1. Understand the commit message: Analyze the commit message
to understand the context and purpose of the code change.
{commit message}
2. Review the code change: Examine the deleted and
added lines of code to identify the modifications made.
{code removed}{code added}
3. Identify potential issues: Look for any missing, improper, or
insufficient checkers within the code change. Checkers may include
error handling, input validation, boundary checks, or other safety
mechanisms.
4. Analyze the impact: Consider the impact of the identified issues
on the functionality and reliability of the deep learning libraries.
5. Make a decision: Based on the above analysis, decide whether
the commit introduces a bug or not.
6. Output the conclusion: Generate a clear output of “YES” if the
commit introduces a bug, or “NO” if it does not.
“output”: {Decision}
TABLE VI: Prompt template for bug detection agent (Zero Shot).
“prompt”: You are an AI trained to detect bugs in a deep-learning
library based on commit messages and code changes. Your task
is to determine whether a given commit introduces a bug or not.
Follow the steps below to reason through the problem and arrive at
a conclusion.
Commit message: {commit message}
Code change: {code removed}{code added}
“output”: {Decision}
TABLE VII: Prompt template for bug detection agent (Few Shot).
“prompt”: You are an AI trained to detect bugs in a deep-learning
library based on commit messages and code changes. Your task
is to determine whether a given commit introduces a bug or not.
Follow the steps below to reason through the problem and arrive at
a conclusion.
Example Checker Bug One:
Commit message: {commit message}
Code change: {code removed}{code added}
Example Checker Bug Two:
Commit message: {commit message}
Code change: {code removed}{code added}
Task:
Commit message: {commit message}
Code change: {code removed}{code added}
“output”: {Decision}
Root Cause Analysis Agent
TABLE VIII: Prompt template for root cause analysis agent.
“prompt”: Please describe the root cause of the bug based on the
following commit message:{commit message}
“output”: {Root causes}
Patch Generation Agent
TABLE IX: Prompt template for patch generation agent.
“prompt”: You are given a bug explanation and an external context
for fixing a checker bug. Please think step by step and generate a
patch to fix the bug in the code snippet. Please neglect any issues
related to the indentation in the code snippet. Fixing indentation
is not the goal of this task. If you think the given pattern can be
applied, generate the patch.
Example One: {code removed} {code added}
Example Two: {code removed} {code added}
Bug explanation: {bug explanation}
Retrieved context: {retrieved knowledge}
Code snippet: {code snippet}
“output”: {Think steps}{Patch}