Post exports hang or take a very long time when single org file gets large #551
Replies: 30 comments 10 replies
-
|
Does it hang when:
I see your Org version as 9.6. So that's the Org Was this issue happening on emacs 26.x? Or did you start seeing it only on emacs 27.x (did you upgrade to emacs 27.x recently?) I am just trying to understand what change caused this issue. If you think of the time when the export used to work fine, what were the emacs and Org versions and how much time back was that (so that we can estimate the version/commit of ox-hugo then). Was it the same OS or a different one? Here's how you can get more debug info:
If you can share your Org file, I can attempt to reproduce the issue. |
Beta Was this translation helpful? Give feedback.
-
|
Also, does emacs lock up permanently? (That should never happens and possibly points to some bug in emacs) Or does it hang only for few seconds (15 seconds)? If you are exporting dozens of posts, it could take that long, but not if you exporting only 1 post (current subtree), regardless of the total size of the Org file. |
Beta Was this translation helpful? Give feedback.
-
|
I should have clarified. I always do I do not have
I'm not sure about this. I would have to try it out and let you know. I currently use
I don't think it happened in emacs 26.x. I updated to emacs 27.x as soon as it was available in the arch linux repositories. So my observation is restricted to 27.x. I've been using this version for close to a year now.
Looking through my git log for my blog, I think the last working version is dated Oct 25 2021. Let me link some issues I read up on when I upgraded my doom emacs and the configuration. You had also commented on one of them. OS config has not changed. This doomemacs/doomemacs#5721 is the issue I came across when I hit the error
This happens when I try to save the file. Not when I export. Export happens successfully for a subtree but takes a very very long time (> 1-2 mins) I passed I passed Please let me know if that was sufficient. If not I can execute your recommendation below.
|
Beta Was this translation helpful? Give feedback.
-
This looks related to the doom config and the new Org cache feature in Org. This is unrelated to exports. Can you report this to the Org mailing list?
I am aware of that performance bottleneck in that function. That function is needed if you have Org internal links from one subtree to another. If you don't have links from one post subtree to another, I can come up with a variable that skips this pre-processing. |
Beta Was this translation helpful? Give feedback.
-
You might find more than one example on how to do per-file exports on https://ox-hugo.scripter.co/doc/examples/ . But if you are exporting only one subtree at a time, it should not take minutes to export! I would like to fix this in ox-hugo if possible or at least identify the root cause and have it fixed elsewhere (in Org or emacs). Your help in debugging this will be appreciated. |
Beta Was this translation helpful? Give feedback.
-
Unfortunately, I do have internal links from one subtree to another.
I did this subtree export once again and the CPU usage was so high my system hung. I had to restart now. I have invited you on gitlab for my website https://gitlab.com/kishvanchee/personal-website/ to your username https://gitlab.com/kaushalmodi . Please feel free to clone and use it to debug ox-hugo. Please let me know if you require anything from my end to help you debug this. |
Beta Was this translation helpful? Give feedback.
-
Yes thank you I found it helpful. https://github.com/xianmin/xianmin.org/tree/master/orgpost is one that stands out that does both subtree and per-file exports (per year). If this doesn't get resolved in ox-hugo, most probably I'd have to adopt both approaches at the same time.
I feel this is mostly related to only Org and not doom configs. I'll try to report this on the Org mailing list (I'm new to that 😅 ). Maybe I'll have to downgrade Org for now to 9.4 to see how it goes. |
Beta Was this translation helpful? Give feedback.
-
|
Since my system restarted, I'm not facing the delay/hang during the export of the subtree right now. Maybe I'll be able to give you a better traceback at a later point of time. Here's one traceback I could collect by pressing I hope that is helpful, if not please let me know. |
Beta Was this translation helpful? Give feedback.
-
|
Temporarily I have downgraded Org to 9.4.6 from here - https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tag/?h=release_9.4.6. Hoping this would solve some of the issues 🤞 |
Beta Was this translation helpful? Give feedback.
-
I'm also present (and probably Nobiot) on emacs org mailling list. If you need some help to debug this problem with |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I'm experiencing the same issue with Doom Emacs (28-git from archlinux repo) and latest org 9.5.2. I'm using the auto export on save, but it does freeze even with the manual export. I've to press C-g, otherwise it stops endlessly. If you can tell me how to downgrade org version at least we can understand if the problem is org 9.5. Thank you. |
Beta Was this translation helpful? Give feedback.
-
|
@finex If you're using doom-emacs you can pin packages to a specific commit like this in (package! org :recipe
(:host github
:repo "emacs-straight/org-mode")
:pin "652430128896e690dc6ef2a83891a1209094b3da") |
Beta Was this translation helpful? Give feedback.
-
|
@kishvanchee Hi, I confirm that pinning the version to 9.4.6 resolve the problem. Do you think it's a problem on org-mode and it will be fixed in the future or should a fix be implemented on the ox-hugo project to maintain the compatibility with the latest org-mode version? Thank you very much! |
Beta Was this translation helpful? Give feedback.
-
|
@finex Given that rolling back the version of Org mode fixes this problem, it's caused due to a change in Org mode. Now, the change could be good and could be pointing to an inefficient piece of code in ox-hugo. But in order to fix it, we will need to share a reproducible example with Org mode list so that the folks there can take a look at this. I have seen the same issue, but it's intermittent. So I am unable to come up with a recipe. If you have an Org file that you can share, that always shows this problem, I can share that with the list. Otherwise, I'll keep on looking for a way to reproduce this problem consistently. |
Beta Was this translation helpful? Give feedback.
-
|
@kaushalmodi I'm not sure if this would help, do you think an org file with say 200 entries help you debug? I'm thinking of a recipe like If yes I can whip up a quick script to produce the file |
Beta Was this translation helpful? Give feedback.
-
|
Today, I saw Emacs freezing due to the simple Org tagging command ( @kishvanchee @finex If you have time, please update to the latest Org version from its |
Beta Was this translation helpful? Give feedback.
-
|
@kaushalmodi Hi, I've updated to the latest Org version and disabled encryption like said on doomemacs/doomemacs#5924 (comment): Now I will try to follow your suggestion. I will tell you the results later. Thank you. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I've just updated to the latest doom-emacs version + latest ox-hugo, removed both the temporary fixes (the cache one and the before-safe-hook one) and now both the org export and org tagging are working flawlessly. I've tried to add the: and it still works fine. The org version is still the same but the ox-hugo is the one from yesterday commits. Let's see what it happens on the next days. If it is an intermittent problem or if it is really fixed with the latest version. |
Beta Was this translation helpful? Give feedback.
-
|
Ok, it has freezed again after 30 minutes. Now I will add |
Beta Was this translation helpful? Give feedback.
-
|
Ok, using |
Beta Was this translation helpful? Give feedback.
-
|
I've been having this problem occasionally. Just had it in a 26-line file. It was freezing saving but also just moving around the buffer.
|
Beta Was this translation helpful? Give feedback.
-
@rodelrod Please report this on the Org mailing list so that it can be fixed in the devel or main branch of Org mode. The Org element caching is a bleeding edge feature under development. |
Beta Was this translation helpful? Give feedback.
-
|
Hello all, I have converted this from an Issue to a Discussion as there's not much I can do from ox-hugo side to fix this. I encourage people facing this issue to create a reproducible example and email that to |
Beta Was this translation helpful? Give feedback.
-
|
@kishvanchee @finex Please see https://www.reddit.com/r/orgmode/comments/t15c3f/org_mode_profiling_meetup_on_sat_feb_26_9pm_sg/ -- there is going to be a live debug session related to performance when using Org built from the main branch (the one having the latest org-cache and org-persist updates). |
Beta Was this translation helpful? Give feedback.
-
|
Hi @kaushalmodi, any updates on this? My org file is about 2000 lines long, and I faced the same issue of freezing while exporting and saving. Rolling back to Org 9.4.6 seems to work in my case (I already had |
Beta Was this translation helpful? Give feedback.
-
|
@koustuvsinha Sorry, there are still no updates because no one has been able to create a consistent recipe to reproduce this problem. I use the latest Org mode from its |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the quick reply! Can you share the commit id you are using? I'll try to replicate it with the same version as you. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! I updated to |
Beta Was this translation helpful? Give feedback.
-
|
For record. I try to prioritize all the performance-related issues on Org ML. They are usually fixed within days, given that enough information is provided after following the debugging suggestions. Reproducer is not always required - we have a specifically designed logger to debug errors in this area. Please report any performance issues to Org mailing list (see https://orgmode.org/manual/Feedback.html). Also, upgrading to the latest Org version often helps - we haven't had any reports in this area for weeks. |
Beta Was this translation helpful? Give feedback.
-
|
I have my org file that is giving trouble, who/where do I send it to? |
Beta Was this translation helpful? Give feedback.
-
|
Ok. I can send it to orgmode feedback |
Beta Was this translation helpful? Give feedback.
-
|
But the thing is that when running emacs -q , I don't have the problem... just with doom emacs... |
Beta Was this translation helpful? Give feedback.
-
|
I'm having the exact same issue. Should I try to file an issue somewhere or this is already being worked on? GNU Emacs 28.2.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.34, cairo version 1.17.6) of 2022-10-12 |
Beta Was this translation helpful? Give feedback.
-
This isn't being worked on. Can you file a reproducible recipe for this issue to the Org mailing list? |
Beta Was this translation helpful? Give feedback.
-
|
thanks @kaushalmodi , I replied in thread. What is the final proposed workaround in the meanwhile?
thanks |
Beta Was this translation helpful? Give feedback.
-
|
I tried |
Beta Was this translation helpful? Give feedback.
-
|
kamazeuci ***@***.***> writes:
But the thing is that when running emacs -q , I don't have the problem... just with doom emacs...
This is an annoying issue. If you wish, we can try to debug via screen
sharing.
…--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
|
Beta Was this translation helpful? Give feedback.
-
|
thanks! |
Beta Was this translation helpful? Give feedback.
-
|
kamazeuci ***@***.***> writes:
I really kinda worked around it, so it is not a problem anymore for me, but if by debugging it we help the org or ox-hugo project, I'd be happy to have an online session
Let me know when you are available and your timezone.
You can contact me by email. See my github page.
…--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
|
Beta Was this translation helpful? Give feedback.
-
Actual Behavior
The export is slow at times when I'm exporting the posts recently. This also takes up a lot of CPU (a few times my entire system hung up cos emacs took a lot of CPU). I have even had emacs hang after a point. Is this an expected behaviour? For context, I have around 100 or so org-subtrees (posts) and many of them are drafts (TODO / DRAFT). The no of lines of the org file comes close to 3k. I am using one post per org-subtree since that's the recommended method.
Is this a bug with ox-hugo / org-exporter? Anything I'm missing? Would splitting my posts into separate files be the solution here? If you have examples (sites) you know of who have large number of separate org files, that'd also be helpful.
I see this has been discussed here but no concrete answer on the scalability as such has been provided #428 Just linking it here for reference
Expected Behavior
I would expect a smooth export for every export I do.
How to Reproduce the Issue
Not exactly sure how to reproduce this, it's just a big org file. I've noticed this only recently. I didn't have this issue previously.
Ox-Hugo Debug Information
Debug Info
Debug information for
ox-hugoEmacs Version
Org Version
Hugo Version
Org
load-pathshadowsWarning: Possible mixed installation of Org
Study the output of
M-x list-load-path-shadows.ox-hugodefcustomsBeta Was this translation helpful? Give feedback.
All reactions