Add Units at Runtime #268

goehle · 2016-01-18T01:45:22Z

This pull requests modifies NumberWithUnits so that you can add units at runtime. You can see the documentation in the header portion of parserNumberWithUnits (see file changes). The short version is you can add new units, along with conversion info, at runtime.

$newUnits = ['apple',{name=>'apples',conversion=>{factor=>1,apple=>1}}];
$a = NumberWithUnits("3 apples",{newUnit=>$newUnits});

Davide, since I'm mucking about in MathObjects, let me know if I've stepped on anything I shouldn't have or if there is something I should do to improve things. You can use the following problem to test:

DOCUMENT();      
loadMacros(
   "PGstandard.pl",
   "MathObjects.pl",
   "parserNumberWithUnits.pl",
);
TEXT(beginproblem());
Context("Numeric");
$newUnit = {name => 'bear',                                            
                   conversion => {factor =>3, m=>1}};                       
$pi = NumberWithUnits("3 bear", {newUnit=>$newUnit});   
$pi2 = NumberWithUnits("pi", "Spoon",{newUnit=>"Spoon"});
$newUnits = ['apple',{name=>'apples',conversion=>{factor=>1,apple=>1}}];
$pi3 = NumberWithUnits("3 apples",{newUnit=>$newUnits});  
$pi4 = NumberWithUnits("pi ft");
WARN_MESSAGE(join(',',keys %Units::known_units));
Context()->texStrings;
BEGIN_TEXT
\($pi\) and \{$pi->ans_rule\} $BR
\($pi2\) and \{$pi2->ans_rule\} $BR
\($pi3\) and \{$pi3->ans_rule\} $BR
\($pi4\) and \{$pi4->ans_rule\} $BR
END_TEXT
Context()->normalStrings;
ANS($pi->with(tolerance=>.0001)->cmp);
ANS($pi2->cmp);
ANS($pi3->cmp);
ANS($pi4->cmp);
ENDDOCUMENT();

dpvc · 2016-01-18T11:40:07Z

The problem that I see with this, and the reason I never did something like it myself, is that the Units namespace is global to the httpd child process, so any chanced you make to it will be persistent to other problems that are processed by the same child. That means that, after your problem is processed, other problems that use units will have bear, spoon, and apples as defined units. But only those that use the same child process. This will lead to intermittent and apparently random behavior of problems after such a question is assigned to anyone on the server.

In order to make this work properly, you need to add the units locally to the problem, so they are part of the safe compartment, and will disappear between problems. The current organization of the Units package doesn't make the easy to do.

It has been a while since I tested the details of how the packages work, and I didn't test this change specifically, so perhaps things have changed since then. But last time I looked, this would have caused leakage of problem-secific changes to other problems.

goehle · 2016-01-18T12:54:35Z

Hmm. You are right, there is leakage to other problems. You need to be able to override the known_units and fundamental_units hashes to point to something local to the problem. I could do that and then add the ability to pass a custom known_units and fundamental_units hash to the Units package routines.

dpvc · 2016-01-18T13:12:46Z

Ok, sounds like a plan.

goehle · 2016-01-23T03:19:04Z

Ok, so I added code in the parserNumberWithUnits.pl and parserFormulaWithUnits.pl files that spins off a copy of the relevent hashes in Units and initializes NumberWithUnits with them. These are only loaded once so unless someone does something crazy you should get one copy of the unit hashes per problem. I also added overrides to the relevent functions in NumberWithUnits and Units so that if a reference to alternative unit hashes are provided then those are used instead. Again, this is reasonably subtle PG territory so let me know if I am doing something clumsy.

Things to test

Test that normal units work. I.E. test

DOCUMENT();      
loadMacros(
"PGstandard.pl",     # Standard macros for PG language
"MathObjects.pl",
"parserNumberWithUnits.pl",
);
TEXT(beginproblem());
Context("Numeric");
$pi = NumberWithUnits("pi ft");
Context()->texStrings;
BEGIN_TEXT
Enter a value for \(\pi\)
\{$pi->ans_rule\}
END_TEXT
Context()->normalStrings;
ANS($pi->with(tolerance=>.0001)->cmp);
ENDDOCUMENT();

This should work even under the following conditions:

Commenting out https://github.com/openwebwork/pg/pull/268/files#diff-6b4fb0656a17451e1aeb074bbbfab174R78 so that the override hashes are never set.
Deleting the "options" hash from https://github.com/openwebwork/pg/pull/268/files#diff-cd9665babf333c6dad103aac7f26dea8R125 so that the override hashes are never sent to Units.
Test the following problem:

DOCUMENT();      
loadMacros(
"PGstandard.pl",
"MathObjects.pl",
"parserNumberWithUnits.pl",
);
TEXT(beginproblem());
Context("Numeric");
$newUnit = {name => 'bear',                                            
               conversion => {factor =>3, m=>1}};                       
$pi = NumberWithUnits("3 bear", {newUnit=>$newUnit});   
$pi2 = NumberWithUnits("pi", "Spoon",{newUnit=>"Spoon"});
$newUnits = ['apple',{name=>'apples',conversion=>{factor=>1,apple=>1}}];
$pi3 = NumberWithUnits("3 apples",{newUnit=>$newUnits});  
$pi4 = NumberWithUnits("pi ft");
Context()->texStrings;
BEGIN_TEXT
\($pi\) and \{$pi->ans_rule\} $BR
\($pi2\) and \{$pi2->ans_rule\} $BR
\($pi3\) and \{$pi3->ans_rule\} $BR
\($pi4\) and \{$pi4->ans_rule\} $BR
END_TEXT
Context()->normalStrings;
ANS($pi->with(tolerance=>.0001)->cmp);
ANS($pi2->cmp);
ANS($pi3->cmp);
ANS($pi4->cmp);
ENDDOCUMENT();

In particular this should work with a first answer of 9 m or 3 bear since one bear is 3 m. The second answer should just be pi Spoon and the third answer should work as either 3 apples or 3 apple. Finally the fourth answer should work as pi ft or 0.319186 bear.

Test the same functionality with parserFormulaWithUnits.pl. You can use the problem

DOCUMENT();      
loadMacros(
"PGstandard.pl",
"MathObjects.pl",
"parserFormulaWithUnits.pl",
);
TEXT(beginproblem());
Context("Numeric");
$newUnit = {name => 'bear',                                            
               conversion => {factor =>3, m=>1}};                       
$pi = FormulaWithUnits("3 x bear", {newUnit=>$newUnit});   
$pi2 = FormulaWithUnits("pi x", "Spoon",{newUnit=>"Spoon"});
$newUnits = ['apple',{name=>'apples',conversion=>{factor=>1,apple=>1}}];
$pi3 = FormulaWithUnits("3x apples",{newUnit=>$newUnits});  
$pi4 = FormulaWithUnits("pi*x ft");

Context()->texStrings;
BEGIN_TEXT
\($pi\) and \{$pi->ans_rule\} $BR
\($pi2\) and \{$pi2->ans_rule\} $BR
\($pi3\) and \{$pi3->ans_rule\} $BR
\($pi4\) and \{$pi4->ans_rule\} $BR
END_TEXT
Context()->normalStrings;
ANS($pi->with(tolerance=>.0001)->cmp);
ANS($pi2->cmp);
ANS($pi3->cmp);
ANS($pi4->cmp);
ENDDOCUMENT();

The answers are all the same as the ones for the parserNumberWithUnits.pl test problem but they have x in them. In particular the conversions mentioned above should still work.

Test that there isn't any leakage in another problem. You can add WARN_MESSAGE(join(',',keys %Units::known_units)); to a problem to show all of the possible units. Run one of the above problems then check another problem for unintended units.

dpvc · 2016-01-23T13:25:29Z

lib/Parser/Legacy/NumberWithUnits.pm

+  if ($known_units) {
+    $options->{known_units} = $known_units;
+  }
+  my %Units = Units::evaluate_units($units,{fundamental_units => $fundamental_units, known_units => $known_units});


Does this want to use $options rather than the explicit hash. It looks like you are setting up $options for that, but never use them.

Good catch. I was adding and removing stuff for testing purposes and likely added the wrong thing back in.

dpvc · 2016-01-23T14:22:25Z

Just FYI, to check for leakage, you probably need to run the second problem several times, since you have to make sure it runs in the same child process as the original problem. Since the children are parceled out in various orders, you probably won't get the same child for the second problem on your first try. I'm sure you know that, but just in case anyone else it doing testing.

I haven't had a chance to run tests by hand yet, but just looked through the code by hand. It looks good. I'll see if I can get some time this weekend to give it a try.

dpvc · 2016-01-31T14:44:59Z

t/Selenium/Tests/skeleton.t~

@@ -0,0 +1,60 @@
+#!/usr/bin/perl


Did you mean to add these ~ files?

goehle · 2016-01-31T17:58:35Z

Good point. I've made the suggested changes.

goehle · 2016-02-01T19:20:48Z

I've cleaned up the unit tests so they work. You can try them out by also checking out the Automated Testing branch (openwebwork/webwork2#681) and getting Selenium set up. Then you can just run

perl /opt/webwork/pg/t/Selenium/Tests/parserNumberWithUnit/parser.t

Again this is a bit of a proof of concept to see if these tests are useful. The basic idea is that you can keep a pg file in the testing folder and use the existing utilities to easy make a problem to test. Then you would use the Selenium IDE to generate some tests on the rendered version of the problem. In theory having these tests, and providing them with pull requests will speed things up, but it depends on if people use them and if they are robust enough.

mgage · 2016-05-19T02:39:43Z

I've verified that there is no leakage in this code but I have some suggestions before
we pull it. For the file
parserNumberWithUnits.pl how about changing it to:

our %fundamental_units = %Units::fundamental_units;
our %known_units = %Units::known_units;

sub _parserNumberWithUnits_init {
  # We make copies of these hashes here because these copies will be unique to  # the problem.  The hashes in Units are shared between problems.  We pass
  # the hashes for these local copies to the NumberWithUnits package to use
  # for all of its stuff.  


  Parser::Legacy::ObjectWithUnits::initializeUnits(\%fundamental_units,\%known_units);
  # main::PG_restricted_eval('sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)}');

}
sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)};
sub parserNumberWithUnits::fundamental_units {
    return \%fundamental_units;
}
sub parserNumberWithUnits::known_units {
    return \%known_units;
}
sub parserNumberWithUnits::add_unit {
    my $newUnit = shift;
    my $Units= Parser::Legacy::ObjectWithUnits::add_unit($newUnit->{name}, $newUnit->{conversion});
    return %$Units;
}

the accessor's are simply so that we can grab the current value of %known_units and %fundamental_units as stored in parserNumberWithUnits.pl. %Units::known_units didn't
update as items were added.
The add_unit subroutine is so that you can add the units to the files environment directly without
having to create an answer evaluator. This seemed more natural to me although there is no reason
to remove the automatic adding of a unit to the environment when creating an evaluator.

This is the test code I put in problems to determine which new units were defined:

@check_these_units = qw(apple apples Spoon bear rabbit foobear);
DEBUG_MESSAGE("checking for ", join(' ', @check_these_units), $BR);
$string='';
$fund = parserNumberWithUnits::fundamental_units();
$known = parserNumberWithUnits::known_units;
foreach my $key (@check_these_units) {
  $string .="$key is a fundamental unit$BR" if exists($fund->{$key});
  $string .="$key is a known unit$BR" if exists($known->{$key});
  $string.= "$key is an undefined unit $BR" unless exists($known->{$key});
}
DEBUG_MESSAGE( $string);

Change the formula file in the same way. Comments?

goehle · 2016-05-19T12:20:09Z

If you make a pull request against my feature branch I will merge it.

Cheers
Geoff
On May 18, 2016 10:39 PM, "Michael Gage" [email protected] wrote:

I've verified that there is no leakage in this code but I have some
suggestions before
we pull it. For the file
parserNumberWithUnits.pl how about changing it to:

our %fundamental_units = %Units::fundamental_units;
our %known_units = %Units::known_units;

sub _parserNumberWithUnits_init {

We make copies of these hashes here because these copies will be unique to # the problem. The hashes in Units are shared between problems. We pass

the hashes for these local copies to the NumberWithUnits package to use

for all of its stuff.

Parser::Legacy::ObjectWithUnits::initializeUnits(%fundamental_units,%known_units);

main::PG_restricted_eval('sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)}');

}
sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)};
sub parserNumberWithUnits::fundamental_units {
return %fundamental_units;
}
sub parserNumberWithUnits::known_units {
return %known_units;
}
sub parserNumberWithUnits::add_unit {
my $newUnit = shift;
my $Units= Parser::Legacy::ObjectWithUnits::add_unit($newUnit->{name}, $newUnit->{conversion});
return %$Units;
}

the accessor's are simply so that we can grab the current value of
%known_units and %fundamental_units as stored in parserNumberWithUnits.pl.
%Units::known_units didn't update as items were added.

The add_unit subroutine is so that you can add the units to the
files environment directly without having to create an answer evaluator.
This seemed more natural to me although there is no reason to remove the
automatic adding of a unit to the environment when creating an evaluator.

This is the test code I put in problems to determine which new units were
defined:

@check_these_units = qw(apple apples Spoon bear rabbit foobear);
DEBUG_MESSAGE("checking for ", join(' ', @check_these_units), $BR);
$string='';
$fund = parserNumberWithUnits::fundamental_units();
$known = parserNumberWithUnits::known_units;
foreach my $key (@check_these_units) {
$string .="$key is a fundamental unit$BR" if exists($fund->{$key});
$string .="$key is a known unit$BR" if exists($known->{$key});
$string.= "$key is an undefined unit $BR" unless exists($known->{$key});
}
DEBUG_MESSAGE( $string);

Change the formula file in the same way. Comments?

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#268 (comment)

…nits or FormulaWithUnits. Simplify the way NumberWithUnits is called -- I don't believe the eval() is necessary. Add accessor so the current value of units can be determined.

mgage · 2016-05-19T13:18:57Z

OK. That’s done. (It took three tries to submit a clean PR. :-) )

Take care,

Mike

On May 19, 2016, at 8:20 AM, Geoff Goehle [email protected] wrote:

If you make a pull request against my feature branch I will merge it.

Cheers
Geoff
On May 18, 2016 10:39 PM, "Michael Gage" [email protected] wrote:

I've verified that there is no leakage in this code but I have some
suggestions before
we pull it. For the file
parserNumberWithUnits.pl how about changing it to:

our %fundamental_units = %Units::fundamental_units;
our %known_units = %Units::known_units;

sub _parserNumberWithUnits_init {

We make copies of these hashes here because these copies will be unique to # the problem. The hashes in Units are shared between problems. We pass

the hashes for these local copies to the NumberWithUnits package to use

for all of its stuff.

Parser::Legacy::ObjectWithUnits::initializeUnits(%fundamental_units,%known_units);

main::PG_restricted_eval('sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)}');

}
sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)};
sub parserNumberWithUnits::fundamental_units {
return %fundamental_units;
}
sub parserNumberWithUnits::known_units {
return %known_units;
}
sub parserNumberWithUnits::add_unit {
my $newUnit = shift;
my $Units= Parser::Legacy::ObjectWithUnits::add_unit($newUnit->{name}, $newUnit->{conversion});
return %$Units;
}

the accessor's are simply so that we can grab the current value of
%known_units and %fundamental_units as stored in parserNumberWithUnits.pl.
%Units::known_units didn't update as items were added.

The add_unit subroutine is so that you can add the units to the
files environment directly without having to create an answer evaluator.
This seemed more natural to me although there is no reason to remove the
automatic adding of a unit to the environment when creating an evaluator.

This is the test code I put in problems to determine which new units were
defined:

@check_these_units = qw(apple apples Spoon bear rabbit foobear);
DEBUG_MESSAGE("checking for ", join(' ', @check_these_units), $BR);
$string='';
$fund = parserNumberWithUnits::fundamental_units();
$known = parserNumberWithUnits::known_units;
foreach my $key (@check_these_units) {
$string .="$key is a fundamental unit$BR" if exists($fund->{$key});
$string .="$key is a known unit$BR" if exists($known->{$key});
$string.= "$key is an undefined unit $BR" unless exists($known->{$key});
}
DEBUG_MESSAGE( $string);

Change the formula file in the same way. Comments?

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#268 (comment)

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openwebwork_pg_pull_268-23issuecomment-2D220307749&d=CwMFaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=C6Pt5AGtImanmAdcooarL-JZO8M5dSFPfs3VweYXYkE&m=YJgEIMqhFYY59atC8y9dY2_WnJDBcXIju_8U9lhNE44&s=EGdsQNZpVthJI-fAp8WXuSxGEtEYZ2ySkggb_XkFy64&e=

Create ability to add new units to problem before calling NumberWithU…

goehle added 2 commits January 16, 2016 22:07

Added a first past at allowing units to be added programatically.

ef36adb

Removed automatic pluralization. Its too hard.

7569295

goehle added 2 commits January 22, 2016 21:59

Changed things around so that the units hashes are local to the problem.

5bf76d8

Update documentation

f5fa19d

dpvc reviewed Jan 23, 2016
View reviewed changes

goehle added 2 commits January 23, 2016 09:10

remove extra code.

01b80c1

remove extra code v2

0ce48f4

goehle added 7 commits January 26, 2016 20:04

Merge branch 'develop' of https://github.com/openwebwork/pg into newunit

9477345

Initial commit for Selenium testing harness.

9208703

wrong place.

03be0a1

changes to the skel

8e1b875

Merge branch 'develop' of https://github.com/openwebwork/pg into newunit

ee18113

Adding selenium tests.

556c142

Merge branch 'newunit' of https://github.com/goehle/pg into newunit

5ed6074

dpvc reviewed Jan 31, 2016
View reviewed changes

t/Selenium/Tests/skeleton.t~

@@ -0,0 +1,60 @@

#!/usr/bin/perl

Copy link

Member

dpvc Jan 31, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to add these ~ files?

Removed unwanted files and cleaned up variable decs. Tests still broken.

c02906d

Cleaned up and fixed tests.

f1c1cd9

goehle added 2 commits February 1, 2016 20:02

Added ability to override testing uname and pwd.

9f65593

Removing tests.

a44fd11

Create ability to add new units to problem before calling NumberWithU…

61def70

…nits or FormulaWithUnits. Simplify the way NumberWithUnits is called -- I don't believe the eval() is necessary. Add accessor so the current value of units can be determined.

Merge pull request #3 from mgage/newunit

1811a55

Create ability to add new units to problem before calling NumberWithU…

mgage merged commit d78b2c9 into openwebwork:develop May 19, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Units at Runtime #268

Add Units at Runtime #268

goehle commented Jan 18, 2016

dpvc commented Jan 18, 2016

goehle commented Jan 18, 2016

dpvc commented Jan 18, 2016

goehle commented Jan 23, 2016

dpvc Jan 23, 2016

goehle Jan 23, 2016

dpvc commented Jan 23, 2016

dpvc Jan 31, 2016

goehle commented Jan 31, 2016

goehle commented Feb 1, 2016

mgage commented May 19, 2016

goehle commented May 19, 2016

We make copies of these hashes here because these copies will be unique to # the problem. The hashes in Units are shared between problems. We pass

the hashes for these local copies to the NumberWithUnits package to use

for all of its stuff.

main::PG_restricted_eval('sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)}');

mgage commented May 19, 2016

We make copies of these hashes here because these copies will be unique to # the problem. The hashes in Units are shared between problems. We pass

the hashes for these local copies to the NumberWithUnits package to use

for all of its stuff.

main::PG_restricted_eval('sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)}');

Add Units at Runtime #268

Add Units at Runtime #268

Conversation

goehle commented Jan 18, 2016

dpvc commented Jan 18, 2016

goehle commented Jan 18, 2016

dpvc commented Jan 18, 2016

goehle commented Jan 23, 2016

dpvc Jan 23, 2016

Choose a reason for hiding this comment

goehle Jan 23, 2016

Choose a reason for hiding this comment

dpvc commented Jan 23, 2016

dpvc Jan 31, 2016

Choose a reason for hiding this comment

goehle commented Jan 31, 2016

goehle commented Feb 1, 2016

mgage commented May 19, 2016

goehle commented May 19, 2016

We make copies of these hashes here because these copies will be unique to # the problem. The hashes in Units are shared between problems. We pass

the hashes for these local copies to the NumberWithUnits package to use

for all of its stuff.

main::PG_restricted_eval('sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)}');

mgage commented May 19, 2016

We make copies of these hashes here because these copies will be unique to # the problem. The hashes in Units are shared between problems. We pass

the hashes for these local copies to the NumberWithUnits package to use

for all of its stuff.

main::PG_restricted_eval('sub NumberWithUnits {Parser::Legacy::NumberWithUnits->new(@_)}');