Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[代码生成or自动化用例生成场景BUG] 这模型似乎很固执简直有直男癌??prompt里明确了不要怎么怎么样,每次输出还是不按要求去 #554

Open
bolt163 opened this issue Jun 12, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@bolt163
Copy link

bolt163 commented Jun 12, 2024

是否已有关于该错误的issue或讨论? | Is there an existing issue / discussion for this?
我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
该问题是否在FAQ中有解答? | Is there an existing answer for this in FAQ?
我已经搜索过FAQ | I have searched FAQ
当前行为 | Current Behavior
【case1】
image

【case2】
image

【case3】
企业微信截图_31a28995-c101-494e-8c86-b0ef4a25ccde

期望行为 | Expected Behavior
尽量符合prompt的要求
【llama3-8B的效果】
image

复现方法 | Steps To Reproduce
No response

运行环境 | Environment

  • OS:
  • Python:
  • Transformers:
  • PyTorch:
  • CUDA (python -c 'import torch; print(torch.version.cuda)'):
    备注 | Anything else?
    No response
@bolt163 bolt163 changed the title [BUG] 这模型似乎很固执或直男癌,prompt里明确了不要怎么怎么样,每次输出还是不按要求去 [BUG] 这模型似乎很固执简直有直男癌??prompt里明确了不要怎么怎么样,每次输出还是不按要求去 Jun 12, 2024
@bolt163 bolt163 changed the title [BUG] 这模型似乎很固执简直有直男癌??prompt里明确了不要怎么怎么样,每次输出还是不按要求去 [代码生成or自动化用例生成场景BUG] 这模型似乎很固执简直有直男癌??prompt里明确了不要怎么怎么样,每次输出还是不按要求去 Jun 12, 2024
@jklj077
Copy link
Collaborator

jklj077 commented Jun 12, 2024

Hi,

We would greatly appreciate it if you could provide a full reproduction scripts. Based on the provided screenshots, it is not clear which models you were using and how you were using it.

@bolt163
Copy link
Author

bolt163 commented Jun 12, 2024

Hi,

We would greatly appreciate it if you could provide a full reproduction scripts. Based on the provided screenshots, it is not clear which models you were using and how you were using it.

qwen2-7B-instruct

@jklj077 jklj077 added the enhancement New feature or request label Jun 12, 2024
@bolt163
Copy link
Author

bolt163 commented Jun 12, 2024

prompt = '''
你是接口自动化测试大师,帮肋软件工程师根据给出的接口定义+入参,来生成有序和结构化的自动化测试用例,
如以 /iplan/task/pack/list接口为例,有8个类型都为string的输入参数 name,id,status,owner,page,page_size,vendor_task_id,business_id,
生成的接口自动化用例样例格式cases:[{"case1":"name=null,id=-999,status=false,owner=xiaokeai,page=1,page_size=-100,vendor_task_id="",business_id=""",{"case2":"..."}]
请设计覆盖输入参数的正常、异常组合的用例,生成不少于50条,按我所提供的样例格式并用中文返回"
'''
@jklj077 prompt可按这个去调试试下

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants