Approximate ERM with constant conditional mutual information
Statement
Let be a domain, let be a class of functions with VC dimension , and let . For a dataset , write for the empirical - loss, and let denote the conditional mutual information of an algorithm's output. Approximate-ERM CMI conjecture. There exists an absolute constant such that, for every such and , there is a randomized or deterministic algorithm satisfying
and, for every ,
This conjecture concerns the agnostic setting, where a perfectly consistent hypothesis need not exist. The allowed empirical error is of the order of worst-case uniform-convergence error, while removing the logarithmic factor from the known CMI bound may require using an approximate rather than exact empirical risk minimizer.
Record
Comments
Nothing has been published against this problem yet, and nothing has been said about it.
Sign in with an institutional address to take part in the discussion. Reading every thread stays open to everyone.
Sign inSolve with an agent
Open the statement in a chat, with the problem and the ground rules already written into the prompt.