Feedback and questions for chapter 5

[antonvorobets]

This thread is for general feedback and questions related to chapter 5.

It is perfectly fine to also create threads for specific topics or methods. If they happen to be particularly chapter 5 related, I will collect references to them here, for example, questions or feedback related to Entropy Pooling.

[mmanetta82]

Hi Anton,

Working through the Entropy Pooling code. In the minimization problem expressed below:

dual_solution = minimize(
_dual_objective, x0=np.zeros(lhs.shape[0]), args=(log_p, lhs, rhs),
method='TNC', jac=True, bounds=bounds, options={'maxfun': 10000})

I see that the _dual_objective function you've specified takes an array of lagrangian multipliers as its first argument. Where (or how) do these multipliers appear exactly? My understanding of scipy's minimize function is that the first argument of the objective function is x0, which is just an array of zeroes. That would render the last term in equation 5.1.3, A'v, equal to zero, which doesn't seem correct.

Put more simply, how do the lagrangian multipliers manifest in the minimization problem as written above?

Thanks!
Mike

[antonvorobets]

Hi Mike,

Thanks for your question.

x0 is just a starting point for the dual problem. Setting it all to zeros guarantees that the first iteration of the optimization algorithm is a feasible solution.

If you look at page 73 in the book, where the dual objective is written out, you can see that we must require that the Lagrange multipliers associated with the inequality constraints are greater than or equal to 0, while the equality constraints have no sign constraints on them.

The output of the minimize function will be the optimal Lagrange multipliers, which are then used to compute the solution of the primal problem to give us the posterior probability q, see equation 5.1.3 on page 73.

I suggest you step through the code in a debugger. It seems that you are either using an older version, which fixed the solver to 'TNC', or that you fixed it yourself in the code that you work with.

You can, for example, use the test cases from either the fortitudo.tech or entropy-pooling package to step through the computations in the code.

Best,
Anton

[mmanetta82]

Thanks for the reply, I'll give your suggestions a try.

Best,
Mike