HiGHS solver doesn't log its output in real time

[juzun]

When I run command solve on HiGHS solver, I get only two following lines :

Running HiGHS 1.5.3 [date: 2023-05-16, git hash: 594fa5a9d-dirty]
Copyright (c) 2023 HiGHS under MIT licence terms

When the solving process finish, I finally get the solver output (all the steps, gap, best bound etc.). But other solvers in appsi (like CPLEX or Cbc) show the output in real time. Is there some issue in HiGHS which causes this?

My usage is basic, it occurs on any type of problem/model I used:

from pyomo.contrib import appsi

self.slvr = appsi.solvers.Highs()
self.slvr.config.stream_solver = True
self.results = self.slvr.solve(self.model)

After the process is done, I have this output, which is nice, but why doesn't it show up iteratively during the solution?
This is just a part of the output I would like to see in real time, but don't.

Running HiGHS 1.5.3 [date: 2023-05-16, git hash: 594fa5a9d-dirty]
Copyright (c) 2023 HiGHS under MIT licence terms

Presolve is switched off
Objective function is integral with scale 1000

Solving MIP model with:
   77543 rows
   38340 cols (35784 binary, 2556 integer, 0 implied int., 0 continuous)
   199377 nonzeros

        Nodes      |    B&B Tree     |            Objective Bounds              |  Dynamic Constraints |       Work      
     Proc. InQueue |  Leaves   Expl. | BestBound       BestSol              Gap |   Cuts   InLp Confl. | LpIters     Time

         0       0         0   0.00%   1352615732.565  -inf                 inf        0      0      0         0     0.1s
         0       0         0   0.00%   112630283.4806  -inf                 inf        0      0      6     21209     2.6s
         0       0         0   0.00%   112479090.6762  -inf                 inf   581906    886      8     32472    49.9s
         0       0         0   0.00%   112422016.3206  -inf                 inf   506643   4425     16     37271    61.6s
         0       0         0   0.00%   112409882.2589  -inf                 inf   138654   5388     24     39246    66.6s
         0       0         0   0.00%   112400684.7633  -inf                 inf   160629   7430     32     41778    73.6s
         0       0         0   0.00%   112399926.1061  -inf                 inf    66671   8395    231     43152    80.2s
         0       0         0   0.00%   112399408.1677  -inf                 inf    39189   9159    247     44114    86.7s
         0       0         0   0.00%   112397259.7877  -inf                 inf    27314  10599    263     45732    91.9s
         0       0         0   0.00%   112395826.4105  -inf                 inf    23112  11147    287     46657   100.1s
         0       0         0   0.00%   112395778.0534  -inf                 inf    20018   9589    311     47281   105.3s
         0       0         0   0.00%   112395775.6758  -inf                 inf    12427   7372   1020     64073   150.2s
        15       0         1   0.01%   112395775.6758  -inf                 inf    12429   7372   1889    195917   270.1s
        80       1         7   0.01%   112395775.6758  -inf                 inf    12429   7372   2144    199921   277.2s

Restarting search from the root node

Presolve is switched off
Model after restart has 77543 rows, 38340 cols (14220 bin., 24120 int., 0 impl., 0 cont.), and 199377 nonzeros

       106       0         0   0.00%   112395775.5593  -inf                 inf     7372      0      0    201033   281.1s
       106       0         0   0.00%   112395775.5593  -inf                 inf     7372   6706      4    219046   286.5s

System info:

Pyomo version: 6.6.2
Python version: 3.10.12
Operating system: Win 10
How Pyomo was installed (PyPI, conda, source): PyPI
Solver (if applicable): appsi.solvers.Highs

[michaelbynum]

We are trying to fix this. There's a subtle buffering issue.

[ckaldemeyer]

+1 Is there any estimation when this can be fixed?

[michaelbynum]

This should be fixed by

1. Release GIL and Run in Background Thread ERGO-Code/HiGHS#1886
2. contrib.solver.highs #3260, once it is reopened and merged

[ckaldemeyer]

So it will be included in the next release?

[AjinkyaBankar]

I am wondering if this functionality is added or not?

[FBumann]

Would really like to see this coming

[kutlay]

It looks like this is a popular ask, @mrmundt would you consider opening this PR again: #3260

I'm happy to help with testing if you're looking for someone to volunteer.

[mrmundt]

Hi, all. We are working on it. We commandeered Michael's PR #3260 to implement some other changes as well as the fix he incorporated into it for HiGHS.

[KelvinCheng5507]

I'm observing a behavior with Pyomo + HiGHS where if I use a time limit > 600, then the code permanently hangs and does not finish. The code works fine in time limit < 600s.
Wondering if it's related to the GIL also?

[kutlay]

@KelvinCheng5507 can you also share your Highs version? I experience similar problems >= 1.8.0 but not before.

[KelvinCheng5507]

@KelvinCheng5507 can you also share your Highs version? I experience similar problems >= 1.8.0 but not before.

I'm using 1.8.0 as well, but I've been experiencing similar problems with older versions as well🤔