Benchmark Scenario

Channel / Device Robustness

How resilient the model is to device and channel mismatch.

Channel/device Device and channel variation 2 trial sets

Back To Scenario Index Open Global Leaderboard

Current Leader

iic/eres2net-large-3dspeaker

18 ranked

6.3103

scenario EER

0.401249 scenario minDCF

Scenario Coverage

2 trial sets

3D-Speaker Device FFSVC 2022 Cross-Channel

Scenario Leaders

Top systems on this scenario

The current reviewed submissions for this scenario.

iic/eres2net-large-3dspeaker

eres2net_large_3dspeaker

6.3103

scenario EER

0.401249 scenario minDCF

iic/eres2netv2-zh

eresv2_zh

11.9758

scenario EER

0.644862 scenario minDCF

palabraai/redimnet2-b6

redimnet2

12.9753

scenario EER

0.641859 scenario minDCF

Scenario Ranking

Full ranking for Channel / Device Robustness

A scenario row first averages linked trial sets within each source dataset, then averages the source means and ranks systems by the resulting EER.

This Scenario All Scenarios Global Board

Rank	System	Rank Signal	Scenario EER	Scenario minDCF	Evidence Rank / EER by trial-set order: 3D-Speaker Device · FFSVC 2022 Cross-Channel
#1	iic/eres2net-large-3dspeaker eres2net_large_3dspeaker	Leader	6.3103	0.401249	#1 6.8733 #1 5.7472
#2	iic/eres2netv2-zh eresv2_zh	Top 3	11.9758	0.644862	#4 17.4433 #2 6.5083
#3	palabraai/redimnet2-b6 redimnet2	Top 3	12.9753	0.641859	#3 17.2700 #3 8.6806
#4	wespeaker/w2vbert2 vb2-w2vbert	Competitive	13.2628	0.581970	#2 15.3867 #10 11.1389
#5	wespeaker/res293-voxceleb wespeaker_resnet293	Competitive	13.9682	0.677811	#6 18.9420 #5 8.9944
#6	wespeaker/samresnet100 vb2-sam100	Competitive	14.0169	0.591790	#5 18.9367 #6 9.0972
#7	wespeaker/res152-voxceleb wespeaker_resnet152	Competitive	14.2456	0.688961	#8 18.9800 #7 9.5111
#8	iic/campplus campplus	Competitive	14.4608	0.722188	#10 20.1800 #4 8.7417
#9	iic/eres2net-en eres_en	Competitive	15.1872	0.730291	#7 18.9467 #11 11.4278
#10	wespeaker/res34-voxceleb wespeaker_resnet34_voxceleb	Needs work	15.2050	0.728816	#9 19.8267 #8 10.5833
#11	speechbrain/ecapa ecapa	Needs work	16.6850	0.817560	#11 21.0200 #14 12.3500
#12	wespeaker/campplus-voxceleb campplus_wespeaker	Needs work	17.8800	0.826833	#12 23.4600 #13 12.3000
#13	wespeaker/ecapa1024-voxceleb wespeaker_ecapa1024	Needs work	18.4567	0.824736	#13 23.8800 #15 13.0333
#14	wespeaker/ecapa512-voxceleb wespeaker_ecapa512	Needs work	18.7142	0.846298	#14 25.3867 #12 12.0417
#15	wespeaker/res34-cnceleb wespeaker_r34	Needs work	18.9178	0.837962	#15 26.8967 #9 10.9389
#16	speechbrain/xvector xvector	Needs work	31.0227	0.964547	#18 42.6314 #16 19.4139
#17	microsoft/wavlm-base-plus-sv wavlm_base	Needs work	31.2817	0.997779	#16 36.5133 #17 26.0500
#18	microsoft/unispeech-sat-base-plus-sv unispeech_sat_base_plus	Needs work	31.7857	0.999283	#17 36.8213 #18 26.7500