Some Useful Bijections

In constrained optimization problems with very simple (constant) constraints it is sometimes useful to simply use a global optimization algorithm with an appropriate one-to-one transformation of the parameters. Suppose we want to optimize an objective function $f\left(\theta \right)$ where $\theta \in A$. If there exists a continuous mapping $\varphi :ℝ\to A$ such that for all $x,y\in A$, $\varphi \left(x\right)=\varphi \left(y\right)$ if and only if $x=y$, then it is equivalent to optimize $f\left(\varphi \left(x\right)\right)$ over $ℝ$. If $\overline{x}$ is the resulting optimum, then we can apply the inverse transformation to obtain $\overline{\theta }={\varphi }^{-1}\left(\overline{x}\right)$.

Below is a table of useful transformations of this type. Most of them are not very difficult to derive, but it seems useful to have a list of them in one place. $A$ denotes the constraint set. They can be scaled as needed for other intervals.

Useful Bijections
$A$$ℝ\to A$$A\to ℝ$
$\left[0,\infty \right)$${e}^{x}$$\mathrm{ln}\left(x\right)$
$\left(-\infty ,0\right]$$-{e}^{x}$$\mathrm{ln}\left(-x\right)$
$\left[0,1\right]$$\frac{1}{1+{e}^{-x}}$$-\mathrm{ln}\left(\frac{1}{\theta }-1\right)$
$\left[-1,1\right]$$\frac{2}{1+{e}^{-x}}-1$$-\mathrm{ln}\left(\frac{2}{\theta +1}-1\right)$

Note that the mapping $ℝ$ to $\left[0,1\right]$ is the Sigmoid Function.

Finally, a multidimensional transformation is useful when the parameters represent probabilities. Suppose $\theta \in A$ where $A=\left\{\left({\theta }_{1},{\theta }_{2},\dots ,{\theta }_{n}\right)|0\le {\theta }_{i}\le 1\phantom{\rule{1em}{0ex}}\text{and}\phantom{\rule{1em}{0ex}}\sum _{i}{\theta }_{i}=1\right\}.$ Here, $A$ is the standard $n-1$ simplex. The corresponding mapping is $\varphi :{ℝ}^{n-1}\to A$ where ${\varphi }_{i}\left(x\right)=\frac{{e}^{{x}_{i}}}{1+\sum _{j=1}^{n-1}{e}^{{x}_{j}}}$ for $1\le i\le n-1$ and ${\varphi }_{n}\left(x\right)=\frac{1}{1+\sum _{j=1}^{n-1}{e}^{{x}_{j}}}.$ This is the same mapping that arises in the multinomial logit and conditional logit regression models.