mirror of
https://github.com/OpenHands/OpenHands.git
synced 2025-12-26 05:48:36 +08:00
507afd7f06
Co-authored-by: Kush Dave Jain <kdjain@pit.isri.cmu.edu> Co-authored-by: openhands <openhands@all-hands.dev> Co-authored-by: Graham Neubig <neubig@gmail.com>
115 lines
6.1 KiB
Python
115 lines
6.1 KiB
Python
CODEACT_TESTGEN_PROMPT_OLD = """Your goal is to generate a high-quality test suite (at least 20+ passing tests) for the code file: {code_file}. Output the test suite at {test_file}\n'
|
|
|
|
[current directory: /workspace/{workspace_dir_name}]
|
|
|
|
IMPORTANT: You should ONLY interact with the environment provided to you AND NEVER ASK FOR HUMAN HELP
|
|
|
|
IMPORTANT: Follow instructions, if you have < 80 tests you should generate more tests rather than trying to fix the ones you have.
|
|
|
|
IMPORTANT: Code file to test:
|
|
```python
|
|
{code_src}
|
|
```
|
|
|
|
Here are additional imports that you may need:
|
|
{imports}
|
|
|
|
Look at code dependencies (NOT {code_file} since you already have contents) and test files you need context for to write a complete test suite.
|
|
|
|
Aim for 20+ test functions with asserts. Do not hestitate to use the Python interpreter to understand the input output behavior of the code you are testing.
|
|
|
|
Output your test suite at {test_file}. Each unit test must be a function starting with test_. Include all your test imports and setup before your first test. Do not include a main method to run the tests. Make sure to make it as comprehensive as possible, try to execute all the methods you saw.
|
|
|
|
When you think you've successfully generated a test suite, run it on for the current project using {coverage_command}.
|
|
|
|
If you have few tests GENERATE MORE TESTS rather than trying to fix the ones you have (it is possible to filter out failing tests later).
|
|
|
|
Then run coverage report -m --include {code_file} to see how well your test suite covers the code under test.
|
|
|
|
When you are trying to improve coverage pick a part of the code that is not covered (indicated by lines on coverage report), examine the code and then
|
|
try to generate a test for it. Feel free to use a code interpreter to understand the input output behavior. ONLY add tests
|
|
not remove them.
|
|
|
|
If you are unable to see passing and failing tests, FIX YOUR IMPORTS to use the same style as other test files.
|
|
|
|
You should NOT modify any existing test case files. You SHOULD add new test in a NEW file to reproduce the issue.
|
|
|
|
You should NEVER use web browsing or any other web-based tools.
|
|
|
|
You should NEVER install new packages, use existing packages only.
|
|
|
|
You should ALWAYS use the default Python interpreter available in the <execute_bash> environment to run code related to the provided issue and/or repository.
|
|
|
|
You should ALWAYS use local imports DO NOT import the general library.
|
|
|
|
When you think you have a fully adequate test suite, please run the following command: <execute_bash> exit </execute_bash>.
|
|
"""
|
|
|
|
CODEACT_TESTGEN_PROMPT = """
|
|
Your goal is to generate a comprehensive, **broad-coverage** test suite for the code below, ensuring you test as many lines and branches as possible on the first attempt.
|
|
|
|
Place your test suite in a new file named {test_file}.
|
|
|
|
IMPORTANT REQUIREMENTS:
|
|
1. **No external help or resources**—use only the snippet below.
|
|
2. **Focus on breadth over depth**: cover all major functions, classes, and code paths early to minimize coverage iterations.
|
|
3. Each test function must start with `test_` and use `assert` to verify behavior.
|
|
4. Include only necessary imports (standard library or local).
|
|
5. Do **not** modify existing test files—create a brand new one. No `main()` or other non-test code.
|
|
6. Produce **at least 20 test functions**; if coverage is lacking, add more tests rather than removing or changing existing ones.
|
|
7. Use the following commands to check coverage:
|
|
<execute_bash> {coverage_command} </execute_bash>
|
|
<execute_bash> coverage report -m --include {code_file} </execute_bash>
|
|
If lines remain uncovered, add new tests targeting them specifically.
|
|
8. When you're satisfied with coverage, finalize by running:
|
|
<execute_bash> exit </execute_bash>
|
|
|
|
Below is the **complete code snippet** to test:
|
|
|
|
<START_OF_CODE>
|
|
{code_src}
|
|
<END_OF_CODE>
|
|
|
|
NOTE: if you are testing django, you must use from django.test import SimpleTestCase and class based tests (i.e. class TestSomething(SimpleTestCase)).
|
|
NOTE: if there is an error executing tests you MUST fix it before exiting. DO NOT install new packages.
|
|
NOTE: if outputting a revised test suite REPLACE {test_file} with the revised suite
|
|
|
|
**Output the final test suite** (20+ tests) for {test_file} in a single code block, no extra commentary. MAKE SURE you run the tests and ensure you can see which tests passed and failed BEFORE exiting.
|
|
"""
|
|
|
|
CODEACT_TESTGEN_PROMPT_ITERATE = """
|
|
Your goal is to improve the test suite at {test_file} to achieve **broad-coverage** of the code below.
|
|
|
|
First run the test suite.
|
|
|
|
If no tests run, then remove {test_file} and create {test_file} with a new suite.
|
|
|
|
Otherwise, improve it aiming to improve code coverage.
|
|
|
|
IMPORTANT REQUIREMENTS:
|
|
1. Use the following commands to check coverage (RUN THIS FIRST):
|
|
<execute_bash> {coverage_command} </execute_bash>
|
|
<execute_bash> coverage report -m --include {code_file} </execute_bash>
|
|
If lines remain uncovered, add new tests targeting them specifically.
|
|
2. **No external help or resources**—use only the snippet below.
|
|
3. **Focus on breadth over depth**: cover all major functions, classes, and code paths early to minimize coverage iterations.
|
|
4. Each test function must use `assert` to verify behavior.
|
|
5. Include only necessary imports (standard library or local).
|
|
6. Do **not** modify other test files in the repository. No `main()` or other non-test code.
|
|
7. Produce **at least 20 test functions**; if coverage is lacking, add more tests rather than removing or changing existing ones.
|
|
8. When you're satisfied with coverage, finalize by running:
|
|
<execute_bash> exit </execute_bash>
|
|
|
|
Below is the **complete code snippet** to test:
|
|
|
|
<START_OF_CODE>
|
|
{code_src}
|
|
<END_OF_CODE>
|
|
|
|
NOTE: if you are testing django, you must use from django.test import SimpleTestCase and class based tests (i.e. class TestSomething(SimpleTestCase)).
|
|
NOTE: if there is an error executing tests you MUST fix it before exiting. DO NOT install new packages.
|
|
NOTE: if outputting a revised test suite REPLACE {test_file} with the revised suite
|
|
|
|
**Output the final test suite** (20+ tests) for {test_file} in a single code block, no extra commentary. MAKE SURE you run the tests and ensure you can see which tests passed and failed BEFORE exiting.
|
|
"""
|