Add the macros for communication with a remote R server #214
Conversation
|
Since this request is to pg, I am leaving it to Mike/Davide/Geoff to decide about (should the new file go here, or in the OPL macros?). |
|
This actually raises the larger question of where contributed macros should live, as some are currently in the pg directory, and some are in the OPL. On Jun 16, 2015, at 2:00 PM, John Jones <[email protected]mailto:[email protected]> wrote: Since this request is to pg, I am leaving it to Mike/Davide/Geoff to decide about (should the new file go here, or in the OPL macros?). — |
|
Danny, It probably depends. In this case -- where the macros will be widely used (eventually) I think importing these macros into pg is appropriate. The macros in the OPL are also searched if the file they are in is placed in the macro search path in defaults.conf (or the override configurations). That's probably the best place for macros associated with one or a small group of problems from a specific school. We have in the past moved macros from an OPL location into the pg/macros file as they became more widely used. We don't have a specific policy for doing this yet. |
|
A couple of questions. (Sorry for getting to this so late, I was away for a couple weeks.) One for @cubranic Do you have a pg file I can use to test this macro? One for @mgage. This macro file requires additional configuration to use. Specifically you have to specify |
|
@goehle Here is a PG file from our problem library: https://github.com/ubc/webwork-open-problem-library/blob/master/OpenProblemLibrary/UBC/STAT/STAT300/hw01/stat300_hw01_q02.pg Also, the reason the additional configuration has to be done that way is so the macro can have a soft-fail with a reasonable message if it's not present. This was requested by @jwj61 and the way I implemented it was proposed on the mailing list a while back (probably around a year). |
|
That makes sense. Since there needs to be an additional configuration line in localOverrides.conf could you make a pull request to webwork2 that has a commentented out configuration line in localOverrides.conf.dist along with a couple of lines of documentation explaining what it is and why you would set it. |
|
I'm having trouble getting this to run. I've installed Statistics::R::IO::Rserve, and I installed an rserver and have it running. When I try to run the problem I get Is there other configuration that I"m missing? Some other things you will need to add to the webwork2 pull.
|
|
@goehle Sorry, I think I left you out of an email in which I described the pull request (it was part of a separate discussion on OPL). There are indeed several modules that need to be in |
|
My preference for add-ons like this would be to have a separate config file which includes all of the static configuration needed, and then a small section in localOverrides.conf referring to it, and including the server-specific settings. Something like the following: To enable the Rserve integration, uncomment the following lines, and set the location of your R server:$pg{specialPGEnvironmentVars}{Rserve} = {host => "localhost”};include(“conf/Rserve.conf”);This way, the user is only bothered with the things that they need to set. Danny On Jun 24, 2015, at 12:42 PM, Davor Cubranic <[email protected]mailto:[email protected]> wrote: @goehlehttps://github.com/goehle Sorry, I think I left you out of an email in which I described the pull request (it was part of a separate discussion on OPL). There are indeed several modules that need to be in ${pg}{modules}, we have the list in our installation guidehttp://wiki.ubc.ca/Documentation:WeBWork/The_WeBWorKiR_Project:_Integrating_WeBWorK_with_R/Installation_guide. — |
|
I'm not sure there is any static information other than the server address. If its more than 5 or 10 lines of configuration then another file is definitely worth it. Right now its just the one (and some modules that need to be added to the existing pg module list.) However ... the list you have on the installation guide includes IO::File. This is not safe for us to include in the OK list for PG because pg problems could use it to open files on the server. I'm not sure what to do at this point. It can be put in the OPL macros folder without any real issues, but I'm getting a little wary about adding it to pg. |
|
What exactly is used in IO::File? Sometimes we can construct our own module that can be included and which allows only restricted Take care,
|
|
Maybe you can clarify for me an issue that led me to including |
|
Like mike said, sometimes we can create "dummy" packages which will fool Statistics::R::IO into thinking it has access to IO::File, and as long as you don't try to use anything in Statistics::R::IO that actually uses IO::File it can work. However, I think part of the problem is that this is all buried under several levels of abstraction which makes it tough to tell what is exactly being used from the packages. |
|
RserveClient.pl creates an instance of Statistics::R::IO::Rserve, which in turn loads: use Moose;
use Statistics::R::IO::REXPFactory;
use Statistics::R::IO::QapEncoding;
use Socket;
use IO::Socket::INET ();
use Scalar::Util qw(blessed looks_like_number);
use Carp;
use namespace::clean;So it doesn't really need it directly, but I guess via |
|
The usual problem is that Statistics::R::IO is trying to compile some part of itself at runtime. That was what was going on with maketext() in Locale — the localization module. That was difficult to Take care, Mike
|
|
Sockets in linux can be created and controled by opening files. Its possible that is how it is being used. If so then its likely integral to the network functionality of the whole thing. Is there a client only version of the package that connects via http or something safer? |
|
The fact that it has use Moose; Take care, Mike
|
|
@mgage This is the error: The line apparently causing it is a call to the $self->fh->print(pack('V4', $command, length($parameters), 0, 0) .
$parameters); |
|
@goehle You're probably thinking of domain sockets (PF_UNIX) -- I don't think any file is created for IP sockets (PF_INET), which I use here. What I don't see is how is connecting via HTTP any safer. It would still ultimately have to use a socket, would it not? |
|
I was thinking along the lines of how geogebra or sage connects. (Although On Wed, Jun 24, 2015 at 6:51 PM, Davor Cubranic [email protected]
|
|
What you probably want to do is make your own |
|
@goehle If you're thinking of AskSage macro, that actually communicates by running an external |
|
@dpvc Thanks! Would this file, let's call it lib/Rserve.pm, also be part of this pull request? |
|
I think its the same basic principle. AskSage is defined outside of the safe compartment in IO.pm (if its the one I'm thinking of) and the function itself is exported to the safe compartment. You would add a |
|
Yes. You would also add It has been a while since I did this sort of thing, but that is my recollection of how it works. Your and then your |
|
I have updated the code (after rebasing it onto current I have a couple of tidying-up questions at this point:
|
|
Unfortunately still requires us to add IO::File to the safe compartment, which we can't do. However after many hours of poking I've found some magic which may fix things. Or at the very least I was able to run the test problem posted above. You should run tests yourself to make sure I'm not missing anything. The main downside is that it requires a change to the Statistics::R::IO::Rserve package. First, add to line 30. That will change the As for your questions.
|
|
Thanks for looking into this in such detail Geoff. I’m happy to go with your suggestion, but can you clarify something about the safe compartment for me? When I have an entry such as Re your answer #2: I’m the maintainer of Statistics::R::IO, so there is no problem of modifying the official release on CPAN, especially since your change is so minor. In addition, since S::R::IO depends on Moose, unless you want to add Moose as WebWork dependency, we should still probably replace
|
|
That's what I thought too, but I was wrong. Turns out the intended purpose I was under the impression that use was just BEGIN { require thing; import Also, since you have used Moose, you may be able to answer this. Is it Geoff. On Mon, May 9, 2016 at 5:18 PM, Davor Cubranic [email protected]
|
I can include it in an |
|
Ah, sure. Thats a good idea. On Mon, May 9, 2016 at 7:46 PM, Davor Cubranic [email protected]
|
I'm just a Moose user, so I can't really answer that. FWIW, it's the first OO package mentioned by |
|
You aren't really using a lot of the fancier Moose functionality and you On Mon, May 9, 2016 at 7:51 PM, Davor Cubranic [email protected]
|
|
Hi Goeff, Did you use Class::Accessor ? I used that in attempts table to streamline I even added the “antlers” option as an option since Take care, mike
|
|
Hm, I hope there isn't a catch in the fine print that I missed. I'll have to give this a try and will report back with my findings. |
|
I created a very simple class hierarchy using @mgage, when you used |
|
That solution works for me. Thanks for going the extra mile and slimming P.G. Mike used Class::Accessor on the WeBWorK side, so Class::Accessor is Geoff. On Sun, May 22, 2016 at 7:13 PM, Davor Cubranic [email protected]
|
|
Just a quick update: I converted Statistics::R::IO to use |
See openwebwork/pg#214 for the full discussion, but the gist is that project is using such a small slice of Moose that it could just as easily be replaced with Class::Tiny.
See openwebwork/pg#214, specifically the comment by @goehle on 9-May-2016: this should allow leaving IO::File out of PG's safe compartment and doesn't seem to have any other negative side-effect.
|
I released a new version of Statistics::R::IO that uses Class::Tiny and it is available through CPAN (v1.0001). So this should now work. The line in wwk2/defaults.conf for adding the required modules to the safe compartment now looks like this: [qw(Rserve Class::Tiny IO::Handle)],If you can test it and give me OK from your end, I'll rebase onto current |
|
I tested this and it worked with a sample problem. I also tested it on a server that didn't have Rserve or R::IO::Statistics and it didn't complain when running regular problems. Looks good to me. Thanks for going the extra mile. |
|
OK, I cleaned up the commits and opened the PR for the config file: openwebwork/webwork2#700. |
|
This works for me on a couple of problems and doesn't cause issues when its not configured on non rserve problems. |
Add the macros for communication with a remote R server