Replies: 9 comments 25 replies
-
|
The impression that this post creates on Ipopt being "unmaintained", "issues accumulating unaddressed", and "not receiving any new features or updates in years" is not something I agree on. Maybe Ipopt isn't the project meant in these parts of the post, but it is a project mentioned frequently. It is true that the main authors of Ipopt have not directly and significantly contributed to the project in recent years, but it is still ensured that the project compiles and builds, that bugs are fixed, user queries are replied to (possibly not always satisfactorily), and documentation is improved. We have one open issue at the moment. (I know I'm strongly patting myself on the back here :)) Even functionality was extended recently due to user contributions. I agree that for the COIN-OR infrastructure itself or for projects that are still widely used but where the project maintainers do not catch up with the amount of user requests (e.g., Clp,Cbc), it could be very helpful to have one or two persons solely dedicated to COIN-OR. But I don't agree to the picture drawn that without hiring anyone, also active well-maintained projects (e.g., Cmpl, CppAD, Gravity, Highs, Ipopt, Pulp, SHOT, and more) would fall apart over time. |
Beta Was this translation helpful? Give feedback.
-
|
I am the project manager of the (mostly convex) MINLP solver SHOT (https://github.com/coin-or/SHOT). SHOT currently utilizes Cbc in addition to CPLEX and Gurobi as its MIP-subsolvers. We are, however, already planning to add support for HIGHS as a MIP-subsolver in SHOT in the near future. The performance and stability of SHOT when using Cbc is not comparable to when using CPLEX or Gurobi, but we hope that by integrating HIGHS we can create a completely free and efficient COIN-OR alternative to commercial MINLP solvers. Especially if HIGHS would provide lazy callbacks, (MI)QP solving capabilities, etc. we can utilize this easily in SHOT. I think this might also benefit the development of HIGHS, as many of the subproblems solved in SHOT can initially be problematic in nature, e.g. unbounded problems, issues with scaling due to badly generated cuts, etc. From my experience, SHOT (with Cbc) is more efficient than Bonmin (with Cbc) on most of the convex instances in MINLPLib. The same is true for SHOT vs Bonmin on the nonconvex instances (Bonmin is then always a heuristic, while SHOT is a global solver for some classes of nonconvex problems as well). Therefore, since Bonmin is, to my understanding, more or less unmaintained at the moment, I do not know if it time and resources well spent implementing HIGHS in Bonmin. Instead I welcome more developers helping out with the SHOT project instead. This is of course my completely biased opinion 😀 |
Beta Was this translation helpful? Give feedback.
-
|
I'll leave you MINLP folk to argue about the merits of different solvers but, for us, what matters is whether they are under active development. Without this, we can't do anything. I take your wish lists on board, and let's move this forwards. Better done via email ([email protected]). |
Beta Was this translation helpful? Give feedback.
-
|
(These thoughts are my own and not necessarily those of the Foundation). First, let me say that the questions being raised here are exactly the questions the Foundation has discussed and struggled with over years. We still don't have definitive answers, so part of the motivation for putting out the blog post was simply to start a conversation with the community. This has been great so far, so thanks to everyone for engaging. It is helpful to hear a mix of opinions. The more people we can get to voice their opinion, the better! So please weigh in if you have something additional to say. So here is a bit more detail of my own thinking on all of this (sorry, it got a bit long, but there's a lot to say!). It is important to keep in mind that what is being contemplated is what the Foundation itself can/should do to help (if anything). The Foundation's activities are separate from those of individual developers supported by outside organizations. For most of its history, the activities organized by the Foundation have come only in the form of such things as advocating for open source development generally, promoting all COIN-OR projects, creating a review and publication process for software, and supporting necessary infrastructure so that developers from outside organizations had the tools to effectively manage their projects. Which projects got attention and resources for development was decided by the priorities of outside organizations and funders, not by COIN-OR. The Foundation has previously not gotten directly involved in funding development, partly because allocating funds for development does raise difficult issues like the ones being discussed here. But the situation has changed over time. There are certain kinds of development activities that would be difficult for a single outside organization to orchestrate. That is where the Foundation may be able to help. Unlike many of the other foundations supporting open source projects, which might have an active role in deciding what is in and out, COIN-OR is not really in the business of picking winners or even articulating an opinion regarding the worthiness of supporting any one particular project. In hindsight, I can see that it may have seemed as though the blog post was advocating for particular projects, but the reason those projects were highlighted is because they are widely used and many in the community may not realize how precarious the situation is with regard to ensuring their continued existence (at least in some cases). With so many people being apparently critically dependent on them, this is something we felt the community should be aware of and weigh in on. But obviously, in trying to bring certain realities to light as succinctly as possible, many subtleties were lost. Suffice it to say that if the result of our (hopefully ongoing) conversation is that, e.g., everyone is happy to switch from Cbc to HiGHS, for example, then so be it. It seems doubtful that this will happen anytime soon, though, and it is difficult to simply turn our backs on the rather large community of people who have come to depend on some of these legacy projects through our own promotion. At the moment, Cbc continues to be (very) widely used and is embedded in many, many places. Many people use it without even realizing it and may have no choice in the matter because it's embedded several layers deep. For example, Cbc is the backend of CyLP, a numpy-based Cythonic interface to Clp/Cbc that supports customized callbacks written in Python, etc. CyLP is in turn embedded in the Python modeling interface of CvxOpt, among other projects I know of, which are themselves very widely used. I find CyLP incredibly useful and so do many other people, but the implementation of the Cython interface is intricate and specific to Cbc. One cannot simply drop in another solver's API. The original developers of CyLP have long since moved on and I've been struggling to maintain it myself. And said maintenance depends crucially on also maintaining Cbc. And this is just one example. Just a quick search for So it's not really not that black and white. There is no doubt that COIN-OR will continue to support and advocate for the development of newer solvers to replace legacy ones, but this is probably not where we can really have as much impact (we're open to ideas, if this is wrong). I would personally like to be responsive to the users of all legacy codes in COIN-OR as long as people continue to find them useful. I would also like to be able to provide services to all COIN-OR projects that add value and make it more likely they will be adopted in industry. The Foundation can play many roles in all of this. Simply creating documentation for people who want to understand the universe of what is available across all of COIN-OR and/or helping them migrate from abandoned projects to actively maintained ones is one possibility. Even putting aside Cbc itself, there are other legacy pieces, such as CoinUtils, Osi, and Cgl, that are not glamorous, but can be used as stand-alone libraries and remain very useful utilities probably worth maintaining somehow. Optimization Services is an example of a project that is full of useful stuff if only we had the bandwidth to continue to build on its vision. The sky is the limit, but it seems clear that if the Foundation is not thinking about this stuff, no one else is going to do it. And that is not good for the community of users we have built up and want to continue to support. I could certainly say more about things I personally envision doing with funding if we had it. A lot of the things we could do (security audits, CI, testing, packaging, binary distribution, benchmarking, documentation, etc.) would potentially be of benefit to all projects, new and old. But what we ultimately end up doing in the long run is mostly up to the community and (perhaps more importantly) the people who are willing to provide the funding that would allow us to undertake particular efforts. |
Beta Was this translation helpful? Give feedback.
-
|
Can I just say on the security audits front - as someone with experience in this space (security testing / code review specialist) I'd be happy to talk to you about contributing here. I have limited personal time to give, and it would need to have the development efforts to back it up though. As far as I can tell, nothing ever happened to fix my one bug report to COIN, which is pretty consistent with the resourcing issues you're raising here. One of my key take-aways from the recent Log4J security issues has been that we need to find ways for companies benefitting from Open Source software to meaningfully contribute in security as well as other spaces, particularly with regards to legacy support. That particular security disaster was created entirely by a legacy feature that few people were aware of, and that nobody contributing to the project was actually using, and that has cost companies that weren't using it millions. To the extent COIN picks up new sources of funding and/or other support, I'd strongly support a "no legacy feature support without matching contribution" approach to the community. |
Beta Was this translation helpful? Give feedback.
-
|
@pruby Sorry for the long delay in responding! Thanks for the offer, let's get in touch off-line, it would be great to explore the possibilities. It's something we just haven't had the bandwidth to look at, although we know it's important. |
Beta Was this translation helpful? Give feedback.
-
|
Good stuff! @svigerske, your (and GAMS') contributions are much appreciated. But I'd estimate that only a minority of the current users have the required expertise to really dive in and submit code changes. This is true in particular for the solver packages. From my own perspective: even though I have relevant (though modest) knowledge about the internals of LP & MIP solvers from the experience with SCIP, that does not enable me to go bug hunting or even fixing in Cbc in an instant. Instead, it would require a significant investment in the form of studying the code and probably also engaging with current developers. This might be worth it if I also planned to use Cbc for my own research, but maybe not if I "just" want to use it as an off-the-shelf solver. On the other hand, I'd feel comfortable contributing to, say, python-mip, after having studied code only for a few hours. The large number of projects collected is apparently seen as good in itself. But I think it also means that newcomers face a challenge in selecting a suitable project for their use case, when there are multiple candidates. For example, should a newcomer choose Cbc, Symphony or HiGHS to solve their MIP problems? What are the trade-offs between the use of Pulp, python-mip or Pyomo? But maybe there are no clear answers to these questions... |
Beta Was this translation helpful? Give feedback.
-
|
Because performance is not everything. Clp, Osi, Cgl, Cbc are used in many other projects because they are a branch-and-cut framework. This seems to be covered by the thread above already, though one may also mention some of Teds projects that use Clp, Osi, and Cgl only (for column generation, bilevel programs, another mip solver, etc). Fortunately, COIN-OR does not have to decide on the LP or MIP solver in COIN-OR that users should use. There can be several projects with similar aim. I agree that this doesn't always make the choice easier for a user. |
Beta Was this translation helpful? Give feedback.
-
|
I've already acknowledged that performance isn't everything, and I'm not suggesting that HiGHS can take over from Clp/Cbc/Cgl tomorrow. I'm looking at the following realities
Who are the active maintainers of Bonmin and Couenne? |
Beta Was this translation helpful? Give feedback.
-
|
@pobonomo (Pierre Bonami) is the official maintainer of Bonmin, but hasn't committed since 2013, as far as I see. I don't think that there is an active maintainer in the sense that someone would spend real time to fix issues, improve docu, let alone add features, etc. @merraksh (Pietro Belotti) is the official maintainer of Couenne. |
Beta Was this translation helpful? Give feedback.
-
Thanks, I've contacted Pietro, and will reach out to users of both if the mailing list is being maintained |
Beta Was this translation helpful? Give feedback.
-
|
Some of the things I said in the thread above apply here as well. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your input @svigerske. I believe you are the only recent success case COIN-OR has had with in-kind donations. The results of your hard work are a perfect example of what COIN-OR can hope to achieve with more corporate backing. In the past few years, you have effectively been the main contributor and maintainer of this project. Because of your contributions, Ipopt is released several times a year. Most other major project managed by COIN-OR are released once every 18 months. COIN-OR would be excited about the possibility of more in-kind donations, which is why we set up a mailing list where companies can reach out to discuss different kinds of support agreements: [email protected]. Even with your impactful contributions to Ipopt, however, there are still some open questions about long-term sustainability. A project is not sustainable if there is only one core contributor. A main goal of this funding would be to grow the COIN-OR community through the development of documentation, capacity building, comprehensive tests and more. I personally also think it's also important to increase the contributor diversity of COIN-OR. The primary contributors to these projects so far have been people with a mathematics background. While this makes sense, this pool of people is incredible small, and tends to have a research focus. Areas that I listed above, in addition to concerns like security and reproducibility, are invariably deprioritized or completely neglected. These are all crucial elements for running a sustainable open source project. |
Beta Was this translation helpful? Give feedback.
-
|
Excellent points. Although research papers may be produced by the HiGHS team, our objective is to produce high performance code that is well written, documented, and used by people. Indeed, the desire to achieve these aims, and the time spent engaging with users of the prototype solvers, are reasons why HiGHS hasn't yet had a formal release. Now that we've extended HiGHS to MIP, secured 3-5 years of funding and full-time positions for the HiGHS team, things will move on much faster. |
Beta Was this translation helpful? Give feedback.
-
|
Perhaps INFORMS would be interested in some level of sponsorship? Or, a donation button at Informs registration? I suspect there is a significant percentage of INFORMS members that use or have used COIN-OR for research work. |
Beta Was this translation helpful? Give feedback.
-
|
We definitely agree that approaching INFORMS as well as the Computing and Optimization Societies and perhaps other professional societies should be in the mix. There's a wide range of possible levels of support that could be up for discussion. |
Beta Was this translation helpful? Give feedback.
-
|
Given that I've been in the orbit of COIN-OR for some time now and that I did my own digging into Clp (modifying CoinFactorization code in 2010-2011), I'll share my opinion in case it's useful. I recognize it's very easy to have an opinion when you're not in a position of responsibility, and the COIN-OR team has done a great job keeping the ship sailing for 20 years. I'd argue that COIN-OR should regroup around the healthy projects. Go to the owners of these projects (@svigerske named Cmpl, CppAD, Gravity, Highs, Ipopt, Pulp, SHOT, and more) and ask how COIN-OR can best serve them. Developer time and critical mass around projects is the rarest commodity. This should be rekindled, mentored, and supported as much as possible. This will drive sustainability and impact in the long term. For projects that are abandoned or barely keeping up with maintenance, COIN-OR can, secondarily, serve as a matchmaker between sponsors/organizations willing to pay and developers who would maintain the projects if paid. I haven't really seen examples of users paying for this level of support, but perhaps private foundations or governments might. On the other hand, project death should be recognized as part of the life cycle and, therefore, properly communicated to users. Necessity is the mother of invention. If a solver vacuum is created, an enterprising student or professor or freelancer or industry researcher will come along, fill it, and learn a lot in the process. Barriers to open source development are lower than ever. Development practices and infrastructure have come along way since the 2000s when COIN-OR was providing a valuable service by hosting CVS repositories and web pages. That is to say, I don't think computational OR research will stop in its tracks if, hypothetically, no manpower is available to maintain Clp and Cbc. It might even accelerate development on alternatives like HiGHS and create new opportunities for students to rediscover how to write solvers. As for users, they will find a way. It's important to communicate the status of projects like Clp and Cbc so that potential users (especially companies) can evaluate the risk of integrating these solvers into a larger system. |
Beta Was this translation helpful? Give feedback.
-
These are important points that have my full support. However, before I comment, I realise that I've not added the caveat that I give when talking about HiGHS and Clp/Cbc in presentations - that they have been hugely valuable solvers, and that John's contribution to the community has been immense. I understand that there is a desire for COIN-OR to just be an impartial hosting service that facilitates support and maintenance, but when there are many alternative solvers for the same class of problem I feel that some guidance has to be given to new users of COIN-OR. As it stands, Clp/Cbc (and Ipopt) are still the "big name" solvers in COIN-OR, but if new users look for Clp/Cbc documentation, what do they get? HTML documents from 2005 that don't render properly and "project maintainers do not catch up with the amount of user requests". It doesn't inspire confidence in Clp/Cbc and, by implication, in COIN-OR itself. What do these people do? Get an academic license for a commercial solver (if possible) or use GLPK/lp_solve and take orders of magnitude longer to solve their problems? In clearly identifying the "healthy" projects, I believe that COIN-OR will better serve the community and encourage the future development of open source solvers, rather than acting as a "museum". Of course the solvers that are more/less unsupported should be maintained in "legacy" form: there's no way I want the 17k users of Cbc on GitHub (and, yes, I realise that most will be inactive) to be forced to covert to HiGHS - we couldn't cope with the support queries! Although I'm relatively old, I see the position on COIN-OR that's expressed by the young, bright and active people in Optimization and OR, and I stand with them. These discussions (and some external exchanges) have been useful, and have led the authors of two COIN-OR projects based on Cbc to recognise that the performance and support offered by HiGHS is a reason to continue/revive their projects. Even if COIN-OR chooses not to move with the times, HiGHS will still have more than enough users to keep us occupied and solvent. Season's Greetings! |
Beta Was this translation helpful? Give feedback.
-
In that same context, it will be valuable to find out the (intended or actual) user group of these projects. Are they used for teaching, research (both on solvers, but also using them) or commercial applications? This might give different weight to project qualities such as reliability, documentation, mentorship etc. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah - I'd agree that project audience needs clear decisions. There are a few decisions that are decidedly academic in their nature - in particular, even the new HiGHS solver is still based on OpenMP, rather than more modern frameworks. This makes sense for a university with pre-existing HPC infrastructure, but not for someone creating new, cloud-based infrastructure. The commercial solvers meanwhile are targeting container-based deployment with automatic clustering for a reason. |
Beta Was this translation helpful? Give feedback.
-
|
Interesting comment. Dealing with whether OpenMP was available or not was an unwelcome complication in the HiGHS build so, when parallelising the MIP solver recently, we dropped OpenMP in favour of native C++ multithreading. |
Beta Was this translation helpful? Give feedback.
-
|
This is a bit of an aside, but pruby, what would you recommend for modern frameworks? |
Beta Was this translation helpful? Give feedback.
-
|
Not sure about specific frameworks but these days I'd go for cloud-native
options all the way. That means assuming multiple hosts from the start,
with hosts being added and removed mid-search. This implies using queues
rather than shared memory and locks.
The major advantage that this community has over the commercial options is
that scaling them is extremely expensive, requires contracts and
commitments, and licensing is generally charged at maximum scale. If an
open source project were only 1/4 as fast as commercial options, but could
be scaled on the fly to match their performance without licensing
nightmares, the commercial vendors would be in trouble.
Overall, I'm thinking to distribute this you might need a durable,
approximate, distributed priority queue, a content-addressable store for
node data, and some constraints to improve locality of leaf nodes.
…On Fri, 24 Dec 2021, 6:45 am Matthew Saltzman, ***@***.***> wrote:
This is a bit of an aside, but pruby, what would you recommend for modern
frameworks?
—
Reply to this email directly, view it on GitHub
<#18 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABAWZG32AP2LQQR5BMXDUTUSNN3NANCNFSM5KEWIZSA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you commented.Message ID:
<coin-or/COIN-OR-OptimizationSuite/repo-discussions/18/comments/1864632@
github.com>
|
Beta Was this translation helpful? Give feedback.
-
|
Hey folks, first of all, I really appreciate the work here done by all the volunteers and I was curious about all discussions so far. I just want to add some thoughts from my side here:
Maybe not all of the above ideas are "realistic" or realizable, but just want to share my thoughts which might get another ball running. |
Beta Was this translation helpful? Give feedback.
-
|
Hey folks/ @xmunoz , I just started using some of the tools in the COIN-OR ecosystem and I'm really impressed with the whole suite so far. Thank you (huge understatement) for all the incredible work done so far! On the topic of the future of COIN-OR, has any consideration been given to submitting some of the packages as NUMFOCUS supported projects? For those unaware, NUMFOCUS is the foundation that sponsors the development and maintenance of big projects like Pandas, Numpy, SciPy etc. Given how widely used some of the COIN-OR projects are I think there's a pretty good chance of being able to join NUMFOCUS and that could be a good means of reducing some of the financial pressure on the COIN-OR. Would love to hear thoughts on this 😃 . |
Beta Was this translation helpful? Give feedback.
-
Just to clarify, NumFOCUS itself does not sponsor the development and maintenance, and they do not have a large source of independent funds. Instead, they act more like the COIN-OR foundation as a non-profit with a bank account. COIN would still need an external source of funding, and then NumFOCUS would take 15% overhead instead of the COIN-OR foundation's 0% 😄 So joining NumFOCUS is a way to get more visibility, but it is not a solution to the financial problems. |
Beta Was this translation helpful? Give feedback.
-
Feel free to discuss any content related to this blog post here.
https://www.coin-or.org/2021/12/15/future-of-coin-or
Beta Was this translation helpful? Give feedback.
All reactions