Some Useful Bijections

This page contains MathML. Click here for more details.

In constrained optimization problems with very simple (constant) constraints it is sometimes useful to simply use a global optimization algorithm with an appropriate one-to-one transformation of the parameters. Suppose we want to optimize an objective function $f(\theta )$ where $\theta \in A$. If there exists a continuous mapping $\varphi :ℝ\to A$ such that for all $x,y\in A$, $\varphi (x)=\varphi (y)$ if and only if $x=y$, then it is equivalent to optimize $f(\varphi (x))$ over $ℝ$. If $\overline{x}$ is the resulting optimum, then we can apply the inverse transformation to obtain $\overline{\theta }={\varphi }^{-1}(\overline{x})$.

Below is a table of useful transformations of this type. Most of them are not very difficult to derive, but it seems useful to have a list of them in one place. $A$ denotes the constraint set. They can be scaled as needed for other intervals.

Note that the mapping $ℝ$ to $[\mathrm{0,1}]$ is the Sigmoid Function.

Finally, a multidimensional transformation is useful when the parameters represent probabilities. Suppose $\theta \in A$ where $$A=\{({\theta }_1,{\theta }_2,\dots ,{\theta }_n)\mid 0\le {\theta }_i\le 1\text{and}\sum _i{\theta }_i=1\}.$$ Here, $A$ is the standard $n-1$ simplex. The corresponding mapping is $\varphi :{ℝ}^{n-1}\to A$ where $${\varphi }_i(x)=\frac{e^{x_i}}{1+\sum _{j=1}^{n-1}{e}^{x_j}}$$ for $1\le i\le n-1$ and $${\varphi }_n(x)=\frac{1}{1+\sum _{j=1}^{n-1}{e}^{x_j}}.$$ This is the same mapping that arises in the multinomial logit and conditional logit regression models.