$\DeclareMathOperator*{\argmin}{arg\,min}$ Suppose I have to jointly minimize two functions. The solution to the joint minimization does not necessarily minimize each function individually but sort of best one can get for both cases together.
e.g., $\argmin_p f(t; p)$ and $\argmin_p g(t; p)$ could be two minimization problems. Suppose though I am looking for a $p$ that minimizes them "jointly".
This seems to be rather vague though? Is there any way to make this more exact? e.g., what could "joint minimization" or "simultaneous minimization" mean? Remember, the goal is to find a p that sort of minimizes both individually but it might not work perfectly for each individual case.
The joint minimization should minimize each individual minimization problem "well".
Suppose $\min_p J(f(t; p), g(t;p))$ is the joint minimization with solution p and $\min_p f(t; p)$ and $\min_p g(t; p)$ have solutions $p1$ and $p2$ respectively.
Then we would want $p$ to be close to $p1$ and $p2$ in some sense. Obviously is it were equal then it will would have solved each individual minimization problem. This doesn't necessarily work well though as maybe there is another solution $p^*$ that is almost a solution to the joint problem BUT provides better results to each individual solution.
That is, I don't think joint minimization says much about how it's solution minimizes the individual problems. (maybe it would be way off)
for example, suppose $f(t) = t^2$ and $g(t) = 1/t^2$ then the joint distribution $|fg| = 1$ has the whole real line the solution. Which, if we were using numerical methods, maybe end up with t = 10^10 for a solution. But $10^{10}$ is a really bad solution for the minimization problem on $f$ while it is ok for g.
Therefor, there seems to be some other criteria that needs to be added to get a better overall solution. Something like:
$\argmin_{p1} f(t; p1)$ $\argmin_{p2} g(t; p2)$ $\argmin_p J(f(t; p), g(t;p))$ $\min_{p^*} |p1 - p| + |p2 - p|$
The above is a bit sloppy but the idea is that we make sure our solutions to the individual problems are "close" to the joint problem if possible). This, at the very least prevents joint problems of the form $|fg|$.