Memory error with graphs generators: JankoKharaghaniGraph and SquaredSkewHadamardMatrixGraph #25465

dcoudert · 2018-05-30T08:14:40Z

While running long tests on src/sage/graphs/, I get the following errors.
I don't know how to fix that.

Using --optional=atlas,bliss,buckygen,igraph,mpir,plantri,python2,python_igraph,sage,tdlib
sage -t --long src/sage/graphs/generators/families.py
**********************************************************************
File "src/sage/graphs/generators/families.py", line 1897, in sage.graphs.generators.families.SquaredSkewHadamardMatrixGraph
Failed example:
    graphs.SquaredSkewHadamardMatrixGraph(9).is_strongly_regular(parameters=True) # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 572, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 982, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.families.SquaredSkewHadamardMatrixGraph[1]>", line 1, in <module>
        graphs.SquaredSkewHadamardMatrixGraph(Integer(9)).is_strongly_regular(parameters=True) # long time
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/graphs/generators/families.py", line 1908, in SquaredSkewHadamardMatrixGraph
        G = Graph(s, format='seidel_adjacency_matrix')
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/graphs/graph.py", line 1169, in __init__
        from_seidel_adjacency_matrix(self, data)
    MemoryError
**********************************************************************
1 item had failures:
   1 of   3 in sage.graphs.generators.families.SquaredSkewHadamardMatrixGraph
    [344 tests, 1 failure, 30.27 s]

Using --optional=atlas,bliss,buckygen,igraph,mpir,plantri,python2,python_igraph,sage,tdlib
Doctesting 1 file.
sage -t --long src/sage/graphs/generators/smallgraphs.py
**********************************************************************
File "src/sage/graphs/generators/smallgraphs.py", line 4851, in sage.graphs.generators.smallgraphs.JankoKharaghaniGraph
Failed example:
    g = graphs.JankoKharaghaniGraph(936)   # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 572, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 982, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.smallgraphs.JankoKharaghaniGraph[0]>", line 1, in <module>
        g = graphs.JankoKharaghaniGraph(Integer(936))   # long time
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/graphs/generators/smallgraphs.py", line 4940, in JankoKharaghaniGraph
        return Graph([e for e,v in six.iteritems(D.dict()) if v == 1],
      File "sage/matrix/matrix0.pyx", line 248, in sage.matrix.matrix0.Matrix.dict (build/cythonized/sage/matrix/matrix0.c:4473)
        return dict(self._dict())
      File "sage/matrix/matrix0.pyx", line 307, in sage.matrix.matrix0.Matrix._dict (build/cythonized/sage/matrix/matrix0.c:4709)
        d[(int(i),int(j))] = x
    MemoryError
**********************************************************************
File "src/sage/graphs/generators/smallgraphs.py", line 4852, in sage.graphs.generators.smallgraphs.JankoKharaghaniGraph
Failed example:
    g.is_strongly_regular(parameters=True) # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 572, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage-dev/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 982, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.smallgraphs.JankoKharaghaniGraph[1]>", line 1, in <module>
        g.is_strongly_regular(parameters=True) # long time
    NameError: name 'g' is not defined
**********************************************************************
1 item had failures:
   2 of   3 in sage.graphs.generators.smallgraphs.JankoKharaghaniGraph
    [561 tests, 2 failures, 28.25 s]

CC: @dimpase

Component: graph theory

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/25465

The text was updated successfully, but these errors were encountered:

dimpase · 2018-05-30T14:57:03Z

comment:1

that code used to work, now some weird MemoryError...

(are you very pressed for RAM, maybe?)

dcoudert · 2018-05-30T15:19:38Z

comment:2

No, 64GB, that's why I don't understand. May be a side effect of something else.

dimpase · 2018-05-30T15:35:47Z

comment:3

What branch are you getting these on?

dcoudert · 2018-05-31T08:05:59Z

comment:4

branch develop of 8.3.beta3, on linux fedora PC with two 4-cores Intel(R) Xeon(R) CPU W5580 @ 3.20GH and 64GB.

I just found some log files at the root of the sage directory where it is written that there is insufficient memory for the Java Runtime Environment to continue. So it might be a java issue on that computer ? I will investigate with a colleague more expert than me in these kind of issues.

dcoudert · 2018-05-31T08:15:45Z

comment:5

another surprizing behavior: When I start a sage console, and then type g = graphs.JankoKharaghaniGraph(936), it's working perfectly. But with ./sage -t --long src/sage/graphs/generators/smallgraphs.py, it got doctests errors :((

dimpase · 2018-06-01T09:13:02Z

comment:6

Is it the same with 8.2 and earlier 8.3betas? (It would be good to try to find the responsible for this commit, by bisecting...)

dcoudert · 2018-06-02T11:11:29Z

comment:7

I tried with 8.2(long to recompute) and also got the errors :(

Are doctests allowed to use same amount of memory than when running a Sagemath console ? Otherwise it might be something coming from my system, but what ? I'm using it since years for Sagemath...

dimpase · 2018-06-03T09:54:14Z

comment:8

It could be multithreading issue... Did you try running these tests with single thread?

dcoudert · 2018-06-03T11:03:31Z

comment:9

I tried ./sage -tp 1 --long src/sage/graphs/ with same result. I will ask for a system update to test.

dcoudert · 2018-06-05T15:14:59Z

comment:10

after a system update to fedora 26, a make dist-clean and full reinstallation of 8.3.beta4, I still have the same errors :((

seblabbe · 2020-02-01T14:42:20Z

comment:12

Just to say that a duplicate of this ticket was #28865.

dcoudert · 2020-02-01T15:38:43Z

comment:13

Do we now have a solution to fix this issue ? is it sufficient to add # memlimit to these doctests ?

dimpase · 2020-04-15T17:04:58Z

comment:14

Replying to @dcoudert:

after a system update to fedora 26, a make dist-clean and full reinstallation of 8.3.beta4, I still have the same errors :((

fedora 26 is past EOL. How about fedora 30, say?

dcoudert · 2020-04-15T17:25:06Z

comment:15

The current version on that computer is fedora 28, with sage 9.1.rc0 and
Using --optional=benzene,bliss,build,cbc,csdp,dochtml,dot2tex,flask,flask_autoindex,flask_babel,flask_oldsessions,flask_openid,flask_silk,gap_packages,igraph,libsemigroups,memlimit,plantri,python_igraph,python_openid,sage,speaklater,tdlib,twisted

for smallgraphs.py, I still get several errors like

File "src/sage/graphs/generators/smallgraphs.py", line 4727, in sage.graphs.generators.smallgraphs.MathonStronglyRegularGraph
Failed example:
    G = MathonStronglyRegularGraph(0)        # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.smallgraphs.MathonStronglyRegularGraph[1]>", line 1, in <module>
        G = MathonStronglyRegularGraph(Integer(0))        # long time
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/generators/smallgraphs.py", line 4743, in MathonStronglyRegularGraph
        return MathonPseudocyclicMergingGraph(ES, t)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/generators/families.py", line 3139, in MathonPseudocyclicMergingGraph
        return Graph(A)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/graph.py", line 1153, in __init__
        from_adjacency_matrix(self, data, loops=loops, multiedges=multiedges, weighted=weighted)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/graph_input.py", line 257, in from_adjacency_matrix
        entries = set(M.list())
      File "sage/matrix/matrix0.pyx", line 160, in sage.matrix.matrix0.Matrix.list (build/cythonized/sage/matrix/matrix0.c:4269)
        return list(self._list())
      File "sage/matrix/matrix0.pyx", line 212, in sage.matrix.matrix0.Matrix._list (build/cythonized/sage/matrix/matrix0.c:4421)
        x.append(self.get_unsafe(i, j))
    MemoryError

but I suspect there is also something on that computer as I see errors like

File "src/sage/graphs/generators/smallgraphs.py", line 4759, in sage.graphs.generators.smallgraphs.JankoKharaghaniGraph
Failed example:
    g = graphs.JankoKharaghaniGraph(936)   # long time
Exception raised:
    Traceback (most recent call last):
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.graphs.generators.smallgraphs.JankoKharaghaniGraph[0]>", line 1, in <module>
        g = graphs.JankoKharaghaniGraph(Integer(936))   # long time
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/graphs/generators/smallgraphs.py", line 4798, in JankoKharaghaniGraph
        K = GF(q,'alpha')
      File "sage/structure/factory.pyx", line 367, in sage.structure.factory.UniqueFactory.__call__ (build/cythonized/sage/structure/factory.c:2162)
        key, kwds = self.create_key_and_extra_args(*args, **kwds)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/finite_rings/finite_field_constructor.py", line 575, in create_key_and_extra_args
        R = PolynomialRing(FiniteField(p), 'x')
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 650, in PolynomialRing
        return _single_variate(base_ring, names, **kwds)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 746, in _single_variate
        R = constructor(base_ring, name, **kwds)
      File "/home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 3154, in __init__
        from .polynomial_zmod_flint import \
    ImportError: /home/dcoudert/sage/local/lib/python3.7/site-packages/sage/rings/polynomial/polynomial_zmod_flint.cpython-37m-x86_64-linux-gnu.so: failed to map segment from shared object

so may be the system has an issue. My plan was to replace that machine, but my order has been cancelled due to covid :(

dcoudert · 2020-06-21T11:31:53Z

comment:16

After upgrading the system to fedora 32 and building sage from a fresh clone, these tests pass.

musclotte:/home/dcoudert/sage> ./sage -t --long src/sage/graphs/generators/smallgraphs.py 
too few successful tests, not using stored timings
Running doctests with ID 2020-06-21-13-29-27-522108b0.
Git branch: develop
Using --optional=benzene,bliss,buckygen,build,cryptominisat,csdp,dochtml,dot2tex,gap_packages,glucose,igraph,libsemigroups,mcqd,memlimit,plantri,python_igraph,sage,tdlib
Doctesting 1 file.
sage -t --long src/sage/graphs/generators/smallgraphs.py
    [574 tests, 34.04 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 34.2 seconds
    cpu time: 34.4 seconds
    cumulative wall time: 34.0 seconds

So may be we can close this ticket.

dimpase · 2020-06-21T15:29:14Z

Reviewer: Dima Pasechnik

slel · 2020-06-24T06:02:02Z

comment:20

This was likely fixed by #28978.

dcoudert · 2020-06-24T06:22:38Z

comment:21

but the tag memlimit is not used in src/sage/graphs/generators/smallgraphs.py

seblabbe · 2020-06-24T19:29:24Z

comment:22

I still have the issue on Ubuntu 16.04, but soon I will upgrade to 20.04, and I assume that will fix the issue?

dcoudert · 2020-06-24T22:05:31Z

comment:23

Let's hope so.

dcoudert added this to the sage-8.3 milestone May 30, 2018

dcoudert added c: graph theory labels May 30, 2018

embray removed this from the sage-8.3 milestone Dec 23, 2019

embray added the wishlist item label Dec 23, 2019

dimpase added s: needs review and removed wishlist item labels Jun 21, 2020

fchapoton removed the s: positive review label Jun 22, 2020

fchapoton closed this as completed Jun 22, 2020

seblabbe mentioned this issue Oct 21, 2022

meta-ticket: failing internet/optional doctests #25536

Open

kliem mentioned this issue Jan 29, 2023

Meta ticket: Simplify and speed up graph backends #28895

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory error with graphs generators: JankoKharaghaniGraph and SquaredSkewHadamardMatrixGraph #25465

Memory error with graphs generators: JankoKharaghaniGraph and SquaredSkewHadamardMatrixGraph #25465

dcoudert commented May 30, 2018

dimpase commented May 30, 2018

dcoudert commented May 30, 2018

dimpase commented May 30, 2018

dcoudert commented May 31, 2018

dcoudert commented May 31, 2018

dimpase commented Jun 1, 2018

dcoudert commented Jun 2, 2018

dimpase commented Jun 3, 2018

dcoudert commented Jun 3, 2018

dcoudert commented Jun 5, 2018

seblabbe commented Feb 1, 2020

dcoudert commented Feb 1, 2020

dimpase commented Apr 15, 2020

dcoudert commented Apr 15, 2020

dcoudert commented Jun 21, 2020

dimpase commented Jun 21, 2020

slel commented Jun 24, 2020

dcoudert commented Jun 24, 2020

seblabbe commented Jun 24, 2020

dcoudert commented Jun 24, 2020

Memory error with graphs generators: JankoKharaghaniGraph and SquaredSkewHadamardMatrixGraph #25465

Memory error with graphs generators: JankoKharaghaniGraph and SquaredSkewHadamardMatrixGraph #25465

Comments

dcoudert commented May 30, 2018

dimpase commented May 30, 2018

dcoudert commented May 30, 2018

dimpase commented May 30, 2018

dcoudert commented May 31, 2018

dcoudert commented May 31, 2018

dimpase commented Jun 1, 2018

dcoudert commented Jun 2, 2018

dimpase commented Jun 3, 2018

dcoudert commented Jun 3, 2018

dcoudert commented Jun 5, 2018

seblabbe commented Feb 1, 2020

dcoudert commented Feb 1, 2020

dimpase commented Apr 15, 2020

dcoudert commented Apr 15, 2020

dcoudert commented Jun 21, 2020

dimpase commented Jun 21, 2020

slel commented Jun 24, 2020

dcoudert commented Jun 24, 2020

seblabbe commented Jun 24, 2020

dcoudert commented Jun 24, 2020