BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//UIC
BEGIN:VEVENT
UID:2026100602410520261007T16300020261007T1730006ac45fc178dd6@uic.edu
CATEGORIES:MEETING
STATUS:TENTATIVE
DTSTAMP:20261005T110703Z
DTSTART:20261007T163000
DTEND:20261007T173000
SUMMARY:Statistics and Data Science Seminar: From LLM Evaluation to Ranking: Low-Rank Inference for Reliable LLM Leaderboards\, by Wei Sun
DESCRIPTION:Wei Sun\, Purdue University: From LLM Evaluation to Ranking: Low-Rank Inference for Reliable LLM Leaderboards          Modern LLM evaluation increasingly relies on sparse and heterogeneous pairwise human preferences\, making reliable estimation and uncertainty quantification challenging. In this talk\, I will present a low-rank statistical framework that shares information across related tasks\, models\, and evaluation settings while preserving task-specific heterogeneity. I will first formulate LLM evaluation as a structured tensor-completion problem under pairwise comparisons and develop semiparametrically efficient inference for model abilities\, score gaps\, and other smooth functionals. A key technical ingredient is a score-whitening approach that handles the anisotropic Fisher information induced by non-uniform comparisons. I will then turn from score estimation to task-specific ranking\, where ranks and top-K membership are nonsmooth functions of many dependent score gaps. We develop entrywise-accurate low-rank estimation\, efficient debiased inference\, and multiplier-bootstrap procedures that provide simultaneous rank intervals and uncertainty-aware top-K certificates. Together\, these results provide statistical foundations for moving from sparse human feedback to reliable\, task-specific LLM evaluation and ranking.      update this event | Event post: https://mscs.uic.edu/events?page_id=5201791
LOCATION:636 SEO    Select 
CLASS:PRIVATE
END:VEVENT
END:VCALENDAR