ProbXiv
Explorersign in
Problem archiveProblem record

Statement

Let X\mathcal{X} be a domain, let W\mathcal{W} be a class of functions f:X→{0,1}f:\mathcal{X}\to\{0,1\} with VC dimension d≥1d\geq 1, and let n≥1n\geq 1. For a dataset (x,y)∈(X×{0,1})n(x,y)\in(\mathcal{X}\times\{0,1\})^n, write ℓ(h,(x,y))\ell(h,(x,y)) for the empirical 00-11 loss, and let CMI(A)\mathsf{CMI}(A) denote the conditional mutual information of an algorithm's output. Approximate-ERM CMI conjecture. There exists an absolute constant cc such that, for every such W\mathcal{W} and nn, there is a randomized or deterministic algorithm A:(X×{0,1})n→WA:(\mathcal{X}\times\{0,1\})^n\to\mathcal{W} satisfying

CMI(A)≤cd\mathsf{CMI}(A)\leq c d

and, for every (x,y)∈(X×{0,1})n(x,y)\in(\mathcal{X}\times\{0,1\})^n,

E[ℓ(A(x,y),(x,y))]≤inf⁡h∈Wℓ(h,(x,y))+cdn.\mathbb{E}[\ell(A(x,y),(x,y))]\leq\inf_{h\in\mathcal{W}}\ell(h,(x,y))+c\sqrt{\frac{d}{n}}.

This conjecture concerns the agnostic setting, where a perfectly consistent hypothesis need not exist. The allowed empirical error is of the order of worst-case uniform-convergence error, while removing the logarithmic factor from the known CMI bound may require using an approximate rather than exact empirical risk minimizer.

Record

Comments

Nothing has been published against this problem yet, and nothing has been said about it.

Sign in with an institutional address to take part in the discussion. Reading every thread stays open to everyone.

Sign in

Solve with an agent

Open the statement in a chat, with the problem and the ground rules already written into the prompt.

This opens a third-party site. Nothing is posted back to ProbXiv and nothing you write there is recorded here — what a model gives you is an attempt, which a person still has to check.