[论文笔记] Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural

Abstract: Copilot 及其他辅助编程的人工智能模型被广泛使用，这篇文章探索了 Copilot 在哪些任务上表现不佳，prompt 在过程中的作用等几个问题。

Introduction:

• Question 1: Copilot 在 CS1 programming problems 上的表现如何？
• Question 2: 当 Copilot 最初失败后，prompt 的修改如何影响修复成功率？
• Question 3: 排除 prompt 的影响，Copilot 表现不好的问题具有哪些共性？

Motivating Example:
以 CodeCheck 上的一道练习为例：

# Given a list of strings and a string 's',
# return the average length of all strings
# containing s
def averageContainingStringS(strings, s):
  # Your code here...

现在的 Copilot 倒是蛮正确的，我在本机上结果如下：

# Given a list of strings and a string 's',
# return the average length of all strings
# containing s
def averageContainingStringS(strings, s):
    # Initialize the total length and count
    totalLength = 0
    count = 0
    # Iterate through each string in the list
    for string in strings:
        # If the string contains s
        if s in string:
            # Increment the count
            count += 1
            # Increment the total length by the length of the string
            totalLength += len(string)
    # If there are no strings containing s
    if count == 0:
        return 0
    # Return the average length of all strings containing s
    return totalLength / count