Benchmark Scenario

In-The-Wild Robustness

In-the-wild Open-domain media speech 4 trial sets

Back To Scenario Index Open Global Leaderboard

Current Leader

wespeaker/w2vbert2

18 ranked

5.1834

scenario EER

0.162046 scenario minDCF

Scenario Coverage

4 trial sets

CN-Celeb VoxCeleb1-O VoxCeleb1-E VoxCeleb1-H

Scenario Leaders

Top systems on this scenario

wespeaker/w2vbert2

vb2-w2vbert

5.1834

scenario EER

0.162046 scenario minDCF

wespeaker/samresnet100

vb2-sam100

5.2487

scenario EER

0.170274 scenario minDCF

iic/eres2netv2-zh

eresv2_zh

5.5369

scenario EER

0.294964 scenario minDCF

Scenario Ranking

Full ranking for In-The-Wild Robustness

This Scenario All Scenarios Global Board

Rank	System	Rank Signal	Scenario EER	Scenario minDCF	Evidence Rank / EER by trial-set order: CN-Celeb · VoxCeleb1-O · VoxCeleb1-E · VoxCeleb1-H
#1	wespeaker/w2vbert2 vb2-w2vbert	Leader	5.1834	0.162046	#4 9.9690 #1 0.1489 #1 0.3146 #1 0.7298
#2	wespeaker/samresnet100 vb2-sam100	Top 3	5.2487	0.170274	#5 9.9803 #2 0.2287 #3 0.4564 #2 0.8665
#3	iic/eres2netv2-zh eresv2_zh	Top 3	5.5369	0.294964	#1 3.8243 #14 6.3852 #14 6.3659 #14 8.9975
#4	wespeaker/res152-voxceleb wespeaker_resnet152	Competitive	6.1011	0.224787	#6 11.3672 #4 0.4998 #6 0.7174 #5 1.2879
#5	palabraai/redimnet2-b6 redimnet2	Competitive	6.4216	0.208137	#9 12.2444 #3 0.2871 #4 0.5198 #4 0.9892
#6	wespeaker/res34-voxceleb wespeaker_resnet34_voxceleb	Competitive	6.5576	0.240625	#7 11.9718 #8 0.8188 #8 0.9347 #7 1.6768
#7	wespeaker/res293-voxceleb wespeaker_resnet293	Competitive	6.6344	0.231795	#10 12.4334 #5 0.5159 #5 0.7016 #6 1.2890
#8	iic/eres2net-en eres_en	Competitive	6.6673	0.251235	#8 12.1487 #7 0.8028 #9 0.9682 #9 1.7866
#9	iic/campplus campplus	Competitive	6.8211	0.361956	#2 4.7029 #16 7.7917 #16 7.9032 #16 11.1228
#10	wespeaker/res34-cnceleb wespeaker_r34	Needs work	7.8434	0.404734	#3 7.4729 #15 7.0897 #15 7.1685 #15 10.3836
#11	speechbrain/ecapa ecapa	Needs work	7.9977	0.317381	#12 15.2177 #10 0.9038 #2 0.4563 #3 0.9731
#12	wespeaker/campplus-voxceleb campplus_wespeaker	Needs work	8.4036	0.311533	#13 15.7139 #6 0.7180 #7 0.8789 #8 1.6832
#13	wespeaker/ecapa1024-voxceleb wespeaker_ecapa1024	Needs work	8.5204	0.308148	#14 15.8209 #9 0.8188 #10 0.9899 #10 1.8513
#14	wespeaker/ecapa512-voxceleb wespeaker_ecapa512	Needs work	8.9203	0.324306	#15 16.3391 #11 1.0371 #11 1.2104 #11 2.2570
#15	iic/eres2net-large-3dspeaker eres2net_large_3dspeaker	Needs work	14.2563	0.581636	#11 14.0524 #18 13.0837 #18 13.1915 #18 17.1053
#16	microsoft/wavlm-base-plus-sv wavlm_base	Needs work	16.5057	0.632600	#17 27.9358 #12 5.5047 #12 3.2098 #12 6.5121
#17	speechbrain/xvector xvector	Needs work	17.4683	0.675055	#16 24.2749 #17 8.9618 #17 9.1649 #17 13.8588
#18	microsoft/unispeech-sat-base-plus-sv unispeech_sat_base_plus	Needs work	17.6846	0.670598	#18 29.8022 #13 5.6037 #13 3.6382 #13 7.4593