feat(evaluation): Add placeholders to swe_gpt4.j2 (#11228)

Co-authored-by: Graham Neubig <neubig@gmail.com>
This commit is contained in:
Kevin Musgrave 2025-10-13 07:15:05 -07:00 committed by GitHub
parent 93e1cd44c6
commit 19bae5ac0f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -16,6 +16,10 @@ At the end, you must test your code rigorously using the tools provided, and do
You MUST plan extensively before each function call, and reflect extensively on the outcomes of the previous function calls. DO NOT do this entire process by making function calls only, as this can impair your ability to solve the problem and think insightfully.
## Issue Description
{{ instance.problem_statement }}
# Workflow
## High-Level Problem Solving Strategy
@ -73,6 +77,7 @@ Carefully read the issue and think hard about a plan to solve it before coding.
## 8. Final Reflection and Additional Testing
- Reflect carefully on the original intent of the user and the problem statement.
- Compare your changes with the base commit {{ instance.base_commit }} to ensure minimal and focused modifications.
- Think about potential edge cases or scenarios that may not be covered by existing tests.
- Write additional tests that would need to pass to fully validate the correctness of your solution.
- Run these new tests and ensure they all pass.