sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-12-04 21:03:01 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Improve GPQA eval prompt to imrpove parsing answer from Khoj response
Some checks are pending
build and deploy github pages for documentation / deploy (push) Waiting to run
Details
pre-commit / Setup Application and Lint (push) Waiting to run
Details

Browse source
This commit is contained in:
Debanjum 2024-11-30 17:19:01 -08:00
parent 439b18c21f
commit fc6be543bd
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
tests/evals/eval.py
View file

@ -163,7 +163,7 @@ def load_gpqa_dataset():
correct_letter = "ABCD"[correct_index]
prompt = f"""
Answer the following multiple choice question. Answer should be of the following format: 'Answer: $LETTER' (without quotes) where LETTER is one of ABCD. Think step by step before answering.
Answer the following multiple choice question. Answer should be of the following format: 'Answer: $LETTER' (without quotes) where $LETTER is one of ABCD. Think step by step before answering.
{row["Question"]}