dolphin-2.2.1-mistral-7b-Nous.md

Model	AGIEval	GPT4All	TruthfulQA	Bigbench	Average
dolphin-2.2.1-mistral-7b	38.64	72.24	54.09	39.22	51.05

AGIEval

Task	Version	Metric	Value		Stderr
agieval_aqua_rat	0	acc	23.23	±	2.65
		acc_norm	21.26	±	2.57
agieval_logiqa_en	0	acc	35.48	±	1.88
		acc_norm	34.25	±	1.86
agieval_lsat_ar	0	acc	26.09	±	2.90
		acc_norm	24.35	±	2.84
agieval_lsat_lr	0	acc	40.98	±	2.18
		acc_norm	42.35	±	2.19
agieval_lsat_rc	0	acc	51.30	±	3.05
		acc_norm	48.70	±	3.05
agieval_sat_en	0	acc	71.36	±	3.16
		acc_norm	68.93	±	3.23
agieval_sat_en_without_passage	0	acc	40.29	±	3.43
		acc_norm	38.83	±	3.40
agieval_sat_math	0	acc	37.73	±	3.28
		acc_norm	30.45	±	3.11

Average: 38.64%

GPT4All

Task	Version	Metric	Value		Stderr
arc_challenge	0	acc	53.84	±	1.46
		acc_norm	56.40	±	1.45
arc_easy	0	acc	82.62	±	0.78
		acc_norm	79.34	±	0.83
boolq	1	acc	86.33	±	0.60
hellaswag	0	acc	62.76	±	0.48
		acc_norm	81.48	±	0.39
openbookqa	0	acc	32.60	±	2.10
		acc_norm	43.80	±	2.22
piqa	0	acc	81.23	±	0.91
		acc_norm	82.81	±	0.88
winogrande	0	acc	75.53	±	1.21

Average: 72.24%

TruthfulQA

Task	Version	Metric	Value		Stderr
truthfulqa_mc	1	mc1	36.84	±	1.69
		mc2	54.09	±	1.51

Average: 54.09%

Bigbench

Task	Version	Metric	Value		Stderr
bigbench_causal_judgement	0	multiple_choice_grade	57.37	±	3.60
bigbench_date_understanding	0	multiple_choice_grade	69.92	±	2.39
bigbench_disambiguation_qa	0	multiple_choice_grade	32.56	±	2.92
bigbench_geometric_shapes	0	multiple_choice_grade	16.71	±	1.97
		exact_str_match	0.00	±	0.00
bigbench_logical_deduction_five_objects	0	multiple_choice_grade	23.40	±	1.90
bigbench_logical_deduction_seven_objects	0	multiple_choice_grade	19.14	±	1.49
bigbench_logical_deduction_three_objects	0	multiple_choice_grade	42.33	±	2.86
bigbench_movie_recommendation	0	multiple_choice_grade	38.80	±	2.18
bigbench_navigate	0	multiple_choice_grade	51.80	±	1.58
bigbench_reasoning_about_colored_objects	0	multiple_choice_grade	64.60	±	1.07
bigbench_ruin_names	0	multiple_choice_grade	37.05	±	2.28
bigbench_salient_translation_error_detection	0	multiple_choice_grade	14.23	±	1.11
bigbench_snarks	0	multiple_choice_grade	67.96	±	3.48
bigbench_sports_understanding	0	multiple_choice_grade	61.97	±	1.55
bigbench_temporal_sequences	0	multiple_choice_grade	27.80	±	1.42
bigbench_tracking_shuffled_objects_five_objects	0	multiple_choice_grade	22.32	±	1.18
bigbench_tracking_shuffled_objects_seven_objects	0	multiple_choice_grade	15.60	±	0.87
bigbench_tracking_shuffled_objects_three_objects	0	multiple_choice_grade	42.33	±	2.86

Average: 39.22%

Average score: 51.05% Elapsed time: 03:04:55