-
Notifications
You must be signed in to change notification settings - Fork 739
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Links to Content on Linux Fail in 0.13.0, 0.13.2, 0.13.3 #7003
Comments
Hi @nobicycle, in order to reproduce it:
|
@jredrejo I don't understand the relevance of the question about web server - the browser is simply accessing the internal Kolibri webserver. BTW This was broken in the past.
|
No, when using apache or nginx, the standard configuration uses it to serve content resources. They are not served by the internal webserver. Neither apache nor nginx can follow symlinks that are in different system partitions. So, this is a very important point. That was the reason to add a new option,
and check if the problem is solved after restarting your server. |
Hello jredrejo. "No, when using apache or nginx, the standard configuration uses it to serve content resources." So Kolibri is detecting if either of these webservers is installed and using one of them them? In my case I have both apache and nginx installed. How shall I determine which one Kolibri is using? If I uninstall nginx and apache then Kolibri will fallback to its internal webserver? How can I force Kolibri to use its internal webserver (old behavior)? "add it to your options.ini file using this format:" Which ini file? I have no ini file in /etc/{nginx,httpd} or in /opt/kolibri I tried setting the environment variable as follows in my systemd overide file
After restarting kolibri service, kolibri in the browser showed nothing. I will retest after I have answers to some of the questions above. Thanks |
hello @nobicycle answering inline
No, these urls are static routes in Kolibri. When using apache or nginx, they are usually bypassed to nginx control to be more efficiently used. Example using nginx:
If you are not doing this location change they are still statically handled by cherrypy (kolibri internal webserver). I have not found any reference in Cherrypy docs about this, but it's likely it has the same problem following symlinks to different partitions.
It depends on how you have configure them. If you are not using them to proxy kolibri , you are not using them.
no need. it's on you to decide in apache or nginx configuration to proxy kolibri or not.
Ah, sorry, my fault, I thought you were aware of the kolibri options.ini file. It is in the kolibri directory: https://kolibri.readthedocs.io/en/latest/manage/get_support.html#home |
"I have not found any reference in Cherrypy docs about this, but it's likely it has the same problem following symlinks to different partitions." The issue I have is why symlinks, including symlinks pointing to different disks, suddenly stopped working. Pointing to different disks is necessary as argued by @benjaoming in the comment above. It is not likely a human error on my part since I have a simple bash script INSTALL_kolibri that has been run for years. Very simple:
(DIR is the storage location of Kolibri content) Symlinks did stop working as mentioned in #6925, but later they got fixed. So if nothing has changed in Kolibri, that would point the finger at Cherrypy, that has suddenly stopped supporting symlinks. I will try the ini file suggestion, but surely using a systemd environment file to pass variables is equivalent? |
So ... Kolibri was reinstalled and in the browser the initial admin account created etc. Options.ini -
The systemd STATIC_USE_SYMLINKS variable is also still there. Kolibri was stopped, the content link created and Kolibri restarted.
I looked for the kolibri process using sudo ps -fe|grep 'kolibri|python'
I then started manually: In the browser Kolibri forgot the configuration already done and asked to setup the facility again. |
I reverted to kolibri 0.12.8, installing using a script that has worked for years ... and all works perfectly. Quoting @benjaoming again ... Shame we have lost it somewhere between 0.12.8 and 0.13.3 My kolibri partition is unable to have 128G moved onto it, so movedirectory is out of the question unless I upgrade my hardware . Because of cost I would have to use a slow kolibri disk drive - almost certainly erasing any benefits of nginx serving static files. |
There is a change in 0.13.3 to use symlinks and save a lot of space in the users content folders. It should work well for all the users excepting those corner cases as yours . For your case the I know this has been tested by some of the LE staff because in the online servers, the content folder is stored in a separate partition and it's working correctly, so there must be something else causing your problem. I would like to be able to reproduce your problem so we can understand what's happening in your case. In a previous comment I see your folder structure. So, if in order to reproduce it, we should:
Is that all, or is there anything else needed to reproduce it? |
Well there's all the systemd stuff and overrides for environment variables. Let me see if I can export a single script that does the lot. If you have the time, I'd be interested to know how the Kolibri symlinks/static serving saves space and how much, or is there something in the docs? |
Using the same script as 12.8, 13.2 install failed: no content was available post install.
The difference between 12.8 and 13.2 is the step: 2020-06-12 08:24:02 [INFO] Performing start kolibri ... Under 12.8 this step takes a few minutes as Kolibri is dealing with the 128G content. Under 13.2 this step takes no time at all. Seems like STATIC_USE_SYMLINKS is not the issue. |
Just tested on 0.13.0 - links that work on 12.8 fail here too. "There is a change in 0.13.3 to use symlinks and save a lot of space in the users content folders. It should work well for all the users excepting those corner cases as yours ." The Linux platform/file system links is not a "corner case". @benjaoming put forward the strong case for supporting Linux in #4227 0.13.3 is not the problem - it began with 0.13.0 |
It's unfortunately uncommon enough that it slipped by without anyone noticing. In order to maintain support, we'll need to add explicit testing of the scenario in our integration testing stories: https://github.com/learningequality/kolibri/tree/develop/integration_testing/features @nobicycle if it's possible for you to summarize the steps to reproduce using roughly the format of our integration tests above, we can incorporate the scenario into our standard regression testing process |
OK.
2.Stop Kolibri and replace the content directory with a soft link to the content directory on a separate disk to the disk used by Kolibri and make sure Kolibri has permissions on the linked directory.
For completeness I include the systemd config files, but this is not relevant since the above occurs running manually.
|
Hi @nobicycle , could you try the following steps to see if they fix the issue?
Please let me know if you have any questions. Thank you! |
Hi, Or are you suggesting the fact of using a symbolic link to content causes data_version to become corrupted? In which case, after I start kolibri and content import fails, the steps you outlined are necessary? |
Hi @nobicycle , sorry I should have explained my steps in my last message. In my humble opinion, your contents are not visible in Kolibri because they haven't been imported into the Kolibri database. In 0.12.8, we ran a function to import all the content databases in the KOLIBRI_HOME/content folder that are not in the Kolibri database (db.sqlite3) yet into it when upgrading Kolibri, which matches with what you mentioned here
In 0.13.3, the behavior has been changed, and we only do so when upgrading from a version lower than 0.6.0. By deleting the .data_version file, we are faking the condition here so your content databases in the If you would like to not touch the .data_version file, here is another command you can use which has the same functionality:
|
Ah - that makes sense. Sounds like symlinks were likely unrelated to the issue - my guess is the same behavior could be reproduced even in the default content directory. If this was working before, it was not because of supported or documented behavior. As @lyw07 mentioned it's necessary to run However, please note that One other thing to be aware of is that in 0.14 we added a new |
@nobicycle feel free to test with our 0.14 betas here https://github.com/learningequality/kolibri/wiki/0.14.0-beta-testing |
@lyw07 What you say makes perfect sense. I feel sure you have solved the problem. CONTENT_FALLBACK_DIRS='/mnt/System/Applications/Kolibri/content' in my case? or |
The environment variable method did not work See kolibri-session.txt as follows:
I tried : and also those same two values without double quotes CONTENT_FALLBACK_DIRS did not trigger processing the content. Kolibri logs also attached. |
The file removal method WORKED! |
However I still wonder why I get "There is another Kolibri server running. Please use **2020-07-05 04:22:16 [INFO] Performing start kolibri ... Jul 05 12:22:16 tower systemd[1]: Starting Kolibri server... Service $ cat /usr/lib/systemd/system/kolibri.service |
Hi @nobicycle , I'm glad that file removal worked for you! For
Please feel free to let us know if the environment variable change still doesn't work for you. Thank you! |
Thanks @lyw07 I'm bored with testing for now and glad kolibri is back up. |
Hi @nobicycle , Yes definitely, please feel free to do so whenever you would like to. It is not urgent at all :) |
thanks @nobicycle and @lyw07 let's open a new issue for any follow-up |
Observed behavior
$ sudo -u kolibri ls -l /opt/kolibri/content
lrwxrwxrwx 1 root root 40 Jun 8 01:33 /opt/kolibri/content -> /mnt/System/Applications/Kolibri/content
$ sudo -u kolibri ls /opt/kolibri/content
databases storage
No content is visible in the browser.
In the past, after stopping Kolibri, creating the link and starting Kolibri, ... Kolibri would re-establish its database and all would work fine. No Longer.
…
Expected behavior
Content visible in the browser
…
Errors and logs
kolibri.txt
daemon.txt
Context
Python 3.6
The text was updated successfully, but these errors were encountered: