You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a similar problem to another issue:#1.
I encountered this issue when solving ProntoQA using pyke. One typical example is the following. The problem is ProntoQA_10:
{
"id": "ProntoQA_10",
"context": "Every impus is earthy. Each impus is a jompus. Jompuses are small. Jompuses are rompuses. Rompuses are not amenable. Rompuses are wumpuses. Wumpuses are wooden. Wumpuses are zumpuses. Every zumpus is temperate. Every zumpus is a dumpus. Dumpuses are dull. Dumpuses are vumpuses. Every vumpus is not shy. Every yumpus is sweet. Vumpuses are numpuses. Numpuses are not sweet. Numpuses are tumpuses. Fae is a wumpus.",
"question": "Is the following statement true or false? Fae is sweet.",
"options": [
"A) True",
"B) False"
],
"answer": "B",
"explanation": [
"Fae is a wumpus.",
"Wumpuses are zumpuses.",
"Fae is a zumpus.",
"Every zumpus is a dumpus.",
"Fae is a dumpus.",
"Dumpuses are vumpuses.",
"Fae is a vumpus.",
"Vumpuses are numpuses.",
"Fae is a numpus.",
"Numpuses are not sweet.",
"Fae is not sweet."
]
}
And the formation from natural language to program is also correct:
But after giving these to pyke, the output prediction is A rather than the correct answer B.
Are there some problems with pyke? Is pyke giving the correct answer?
The text was updated successfully, but these errors were encountered:
This is a similar problem to another issue:#1.
I encountered this issue when solving ProntoQA using pyke. One typical example is the following. The problem is ProntoQA_10:
And the formation from natural language to program is also correct:
But after giving these to pyke, the output prediction is A rather than the correct answer B.
Are there some problems with pyke? Is pyke giving the correct answer?
The text was updated successfully, but these errors were encountered: