Extraction of optimally unbiased bits from a biased source

We explore the problem of transforming n independent and identically biased {-1,1}-valued random variables X/sub 1/,...,X/sub n/ into a single {-1,1} random variable f(X/sub 1/,...,X/sub n/), so that this result is as unbiased as possible. In general, no function f produces a completely unbiased result. We perform the first study of the relationship between the bias b of these X/sub i/ and the rate at which f(X/sub 1/,...,X/sub n/) can converge to an unbiased {-1,1} random variable (as n/spl rarr//spl infin/). A {-1,1} random variable has bias b if E(X/sub i/)=b. Fixing a bias b, we explore the rate at which the output bias |E(f(X/sub 1/,...,X/sub n/))| can tend to zero for a function f:{-1,1}*/spl rarr/{-1,1}. This is accomplished by classifying the behavior of the natural normalized quantity /spl Xi/(b)/spl Delta/inf/sub f/[lim/sub n/spl rarr//spl infin//n/spl radic/(|E(f(X/sub 1/,...,X/sub n/))|] this infimum taken over all such f. We show that for rational b, /spl Xi/(b)=(1/s), where (1+b/2)=(r/s) (r and s relatively prime). Developing the theory of uniform distribution of sequences to suit our problem, we then explore the case where b is irrational. We prove a new metrical theorem concerning multidimensional Diophantine approximation type from which we show that for (Lebesgue) almost all biases b, /spl Xi/(b)=0. Finally, we show that algebraic biases exhibit curious "boundary" behavior, falling into two classes. Class 1. Those algebraics b for which /spl Xi/(b)>0 and, furthermore, c/sub 1//spl les//spl Xi/(b)/spl les/c/sub 2/ where c/sub 1/ and c/sub 2/ are positive constants depending only on b's algebraic characteristics. Class 2. Those algebraics b for which there exist n>0 and f: {-1,1}/sup n//spl rarr/{-1,1} so that E(f(X/sub 1/,...,X/sub n/))=0. Notice that this classification excludes the possibility that n/spl radic/(|E(f(X/sub 1/,...,X/sub n/))| limits to zero (for algebraics). For rational and algebraic biases, we also study the computational problem by restricting f to be a polynomial time computable function. Finally, we discuss natural extensions where output distributions other than the uniform distribution on {-1,1} are sought.