Apply for the complete CTSM package #1852
Replies: 7 comments 8 replies
-
|
The best solution would be to solve the problems you are having with manage_externals. If you post those problems here, we can try to help. |
Beta Was this translation helpful? Give feedback.
-
|
thank you! l think it is casued by Network problems. l not in USA. ./manage_externals/checkout_externals ERROR: In directory ERROR:root:Failed with output: ERROR: In directory ERROR: Failed with output: ERROR: In directory |
Beta Was this translation helpful? Give feedback.
-
|
I would like to try to determine if this is an issue with your network or with the Docker image. Do you get the same error if you run "git clone https://github.com/NGEET/fates.git" from the command line even outside of the Docker image? If so, I agree that it's a network issue on your system and I would suggest working with your system administrators on this issue. If that git clone command works successfully outside of the Docker image, then please try it again from within the Docker image to see if it's a possible problem with the Docker image on your system. |
Beta Was this translation helpful? Give feedback.
-
|
even outside of the Docker image, l run followed code, l get the same error could you help me download a complete CTSM package and send to me? |
Beta Was this translation helpful? Give feedback.
-
|
Hi,
It seems like Github is blocked on your network; can you verify this by
opening up the *webpage* here:
https://github.com/NGEET/fates/
If you can see the page, that's odd, and appears to be some issue with
Git. Are you using the git from within the container? Or outside of it?
If you *can't* see that, then it's a network / firewall issue. If you're
at a workplace that's blocking Git, for example, the 'checkout_externals'
step should work fine from other locations that don't have it blocked.
Also, are you trying to *update* to a newer version of CTSM? The
CTSM-Lab docker image comes with a version fully included. We are actually
in the process of updating it for the CTSM-Lab-NEON variant, which is still
fully working for CTSM alone, in case that's of interest to you.
Cheers,
- Brian
…On Tue, Sep 13, 2022 at 10:13 AM shuaipengpeng ***@***.***> wrote:
even outside of the Docker image, l run followed code, l get the same error
git clone https://github.com/NGEET/fates.git
Cloning into 'fates'...
fatal: unable to access 'https://github.com/NGEET/fates.git/': GnuTLS
recv error (-110): The TLS connection was non-properly terminated.
could you help me download a complete CTSM package and send to me?
—
Reply to this email directly, view it on GitHub
<#1852 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACL2HPJESI3J4SVGY7TY26LV6CR35ANCNFSM6AAAAAAQLOMEEA>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you! l will use CTSM through docker! |
Beta Was this translation helpful? Give feedback.
-
|
when l run the CTSM in docker, sudo docker run -it --rm -p 9999:8888 -v $HOME/Desktop/Simulations:/home/user escomp/ctsm-lab-2.3-preview Welcome to CTSM-Lab! To connect, open your browser to: NOTE: Replace '' with the port you specified in your 'docker run' command. For example, if you used the option: Then you should open the browser and go to: |
Beta Was this translation helpful? Give feedback.
-
|
Hi,
I think there are multiple things going on here -- first, what operating
system are you using? Mac OS or a Linux version? I'm asking this because
there are some known issues on Linux due to how certain versions map user
names from the OS to the container, and it *seems* like you're hitting an
issue there. However, it might be solvable depending on the OS you're
using.
Basically, this message:
failed to load external entity
"/opt/ncar/cesm2/cime/config/cesm/machines/config_machines.xml"
.. Is likely due to the fact you're running via 'sudo'. On launch,
*this* version of the container calls a function to alter the processor
count (to match what's in your system), but there are likely permission
errors from doing that via 'sudo'. So, the first thing is, can you run
without sudo? Mind you, this is unlikely to be a major problem; it just
won't adjust the processor count, and you can fix it yourself later. (I'm
doing away with this modification later, and simply defaulting to 4 cores
for the container, which can be changed by users per case.)
Second, have you been able to connect via the browser? If you open one
up to localhost:9999, does that work?
Third, finally, even if you can run these, you're likely to hit other
problems due to your firewall - eg, if you can't reach *any* outside site,
you won't be able to download any input data for cases, unfortunately. If
you 'just' can't reach Github, that's not a problem unless you wish to get
a newer version of CESM/CTSM.
Does that help? Let me know if you can indeed access the environment via
the browser at localhost:9999.
Thanks,
- Brian
…On Tue, Sep 13, 2022 at 9:17 PM shuaipengpeng ***@***.***> wrote:
when l run the CTSM in docker,
sudo docker run -it --rm -p 9999:8888 -v
$HOME/Desktop/Simulations:/home/user escomp/ctsm-lab-2.3-preview
failed to load external entity
"/opt/ncar/cesm2/cime/config/cesm/machines/config_machines.xml"
failed to load external entity
"/opt/ncar/cesm2/cime/config/cesm/machines/config_machines.xml"
Welcome to CTSM-Lab! To connect, open your browser to:
localhost:
NOTE: Replace '' with the port you specified in your 'docker run' command.
For example, if you used the option:
-p 9999:8888
Then you should open the browser and go to:
localhost:9999
Does this fail need to be solved?
—
Reply to this email directly, view it on GitHub
<#1852 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACL2HPIJT3PXHTHDGVAVUMDV6E7ULANCNFSM6AAAAAAQLOMEEA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
my operating system is linux. lf l dont use "sodo", Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json: dial unix /var/run/docker.sock: connect: permission denied If l use "sudo" and open one up to localhost:9999, it can work. thank you so much! |
Beta Was this translation helpful? Give feedback.
-
|
Hi Zhang,
OK, that helps -- so you likely don't need to worry about the 'failed to
load external entity' warning. The default in the current file is 4 cores;
if you have more or less, you can simply change it once you create a case.
So, when you access http://localhost:9999 in a browser after your 'docker
run' command, what do you see? In the left side of the Jupyter Lab window,
do you see a 'tutorials' directory? If so, that should have a basic
tutorial ("NEON_Simulation_Tutorial") that uses the preinstalled version
of CTSM to run through a basic use case. *However*, if you still have a
firewall blocking access to the data files, you'll still be unable to run
it.
So, can you test whether you can, from your Linux system, reach the NCAR
FTP server? (This isn't the only access you'll need, but it's a start.)
To do that, from your Linux system (not the container), do:
wget ftp://ftp.cgd.ucar.edu/cesm/inputdata/template.readme
That's just a small text file on the FTP server. If it works, great -
you should be able to run. If not, we'll need to understand whether this
is a local firewall or something else.
Cheers,
- Brian
…On Tue, Sep 13, 2022 at 9:46 PM shuaipengpeng ***@***.***> wrote:
my operating system is linux.
docker version
Client:
Version: 20.10.7
API version: 1.41
Go version: go1.13.8
Git commit: 20.10.7-0ubuntu5~18.04.3
Built: Mon Nov 1 01:04:14 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Got permission denied while trying to connect to the Docker daemon socket
at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version:
dial unix /var/run/docker.sock: connect: permission denied
lf l dont use "sodo",
Got permission denied while trying to connect to the Docker daemon socket
at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json:
dial unix /var/run/docker.sock: connect: permission denied
If l use "sudo" and open one up to localhost:9999, it can work.
thank you so much!
zhang
—
Reply to this email directly, view it on GitHub
<#1852 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACL2HPMFDN34AUALASCO4ODV6FDBNANCNFSM6AAAAAAQLOMEEA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
hi Brian, l can see a 'tutorials' directory. Thank you very much for your patience and guidance! zhang |
Beta Was this translation helpful? Give feedback.
-
|
hi Brian, Could you tell me how to use CTSM in Docker? I watch this tutorial:https://ncar.github.io/CTSM-Tutorial-2022/README.html, I still need to download Completed CTSM from github. This is back to the original problem, when I execute./manage_externals/checkout_externals, I will get an error because of my network problem (I am in China), could you please send me a completed CTSM? Or is there another way to use CTSM on Docker？ Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
Hi Zhang,
Ah, ok, so two things to note here - one, you *don't* need to download
CTSM and run 'checkout_externals' -- you've already got a full
installation of CTSM in the container, ready to go. If you go to that
'notebooks' folder in JupyterHub, and load up the NEON_Simulation_Tutorial
notebook, it'll guide you through doing a NEON (single point) simulation
with that existing code.
Two, if you're looking to do *global* runs, which were part of the
broader CTSM Tutorial we ran, that's also doable... but *very* slow on a
laptop, and also needs to download a lot of data (~400GB!). We actually
have an updated tutorial deck, and an updated container, we're aiming to
get out in a week or two that (I think) removes that so it's more
applicable to laptops / desktops.
Finally, if you're just trying to download the *tutorial* repo, which
includes other tutorials like Git and visualization, and can't reach
Github, I've put a tar.gz copy on the FTP server which you can get with
this command:
wget
ftp://cesm-inputdata-lowres1.cgd.ucar.edu/tutorials/CTSM-Tutorial-2022.tar.gz
.
You'll want to unzip that and place it in whatever director you're
mounting into your container for it to be visible. Does that help?
If you could share more about how you intend to use CTSM (eg, global
runs, single point runs, NEON data, etc), that might help a bit if you
still have additional questions.
Good luck!
- Brian
…On Wed, Sep 14, 2022 at 9:19 AM shuaipengpeng ***@***.***> wrote:
hi Brian,
Could you tell me how to use CTSM in Docker? I watch this tutorial:
https://ncar.github.io/CTSM-Tutorial-2022/README.html, I still need to
download Completed CTSM from github. This is back to the original problem,
when I execute./manage_externals/checkout_externals, I will get an error
because of my network problem (I am in China), could you please send me a
completed CTSM? Or is there another way to use CTSM on Docker？
Thank you!
Zhang
—
Reply to this email directly, view it on GitHub
<#1852 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACL2HPKJ5GOOCOMJQOPF6BLV6HUHNANCNFSM6AAAAAAQLOMEEA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
hi Brain， until now，l can not find out your updated tutorial deck, and an updated container，l want to know whether you have published the content? Could you give me a link about this? -Zhang |
Beta Was this translation helpful? Give feedback.
-
|
hi Brian, first, l have down a NEON (single point) simulation with that existing code. zhang |
Beta Was this translation helpful? Give feedback.
-
I am a graduate student in china, when using CTSM-lab through Docker, running./manage_externals/checkout_externals keeps making mistakes. Could you please send me a downloaded copy? Thank you very much! My email is [email protected].
Beta Was this translation helpful? Give feedback.
All reactions