Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests for multiply (R←X×Y) #76

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
ee2d56c
init: tests for multiply
sloorush Aug 9, 2024
1cb5f8c
Merge branch 'data-gen' of https://github.com/Dyalog/ullu into multiply
sloorush Aug 9, 2024
47f8c82
add multiply to run.apls
sloorush Aug 9, 2024
97cbf4c
add basic tests for multiply
sloorush Aug 9, 2024
8bfb44e
add more basic tests for multiply with default data
sloorush Aug 9, 2024
508ad33
add commutability test
sloorush Aug 9, 2024
21de819
add tests for known errors
sloorush Aug 12, 2024
9ee6844
init cross datatype tests
sloorush Aug 12, 2024
8ab3d3b
add more cross datatype tests
sloorush Aug 12, 2024
fa62970
Merge branch 'main' into multiply
sloorush Aug 27, 2024
3419e63
add model for multiply
sloorush Aug 27, 2024
d87b2f5
Merge branch 'main' of https://github.com/Dyalog/ullu into multiply
sloorush Aug 27, 2024
710cd5a
Merge branch 'main' into multiply
sloorush Sep 2, 2024
9983784
change readme
sloorush Sep 30, 2024
acb2ce1
Merge branch 'main' of https://github.com/Dyalog/ullu into multiply
sloorush Nov 14, 2024
847d7e8
Merge branch 'main' of https://github.com/Dyalog/ullu into multiply
sloorush Nov 26, 2024
788aeac
add model for complex multiply
sloorush Nov 26, 2024
cae4236
revert making the cmplx number again with × sign
sloorush Nov 26, 2024
ce26d8f
Merge branch 'main' of https://github.com/Dyalog/ullu into multiply
sloorush Nov 26, 2024
f05e0b1
skip Hcmplx for ⎕FR←1287 in model
sloorush Nov 29, 2024
2151fb8
Merge branch 'main' of https://github.com/Dyalog/ullu into multiply
sloorush Nov 29, 2024
ff26dd2
update readme to include multiply
sloorush Nov 29, 2024
66cc49c
remove next up from readme.md
sloorush Dec 4, 2024
0045eec
add comments to model for multiply
sloorush Dec 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,19 @@ Ullu is a QA for DyalogAPL (can be used to test any APL ideally) which tests spe
- index of (dyadic ⍳) (Not tested for coverage)
- magnitude (monadic |)
- membership (dyadic ∊) (Not tested for coverage)
- multiply (dyadic ×)
- residue (dyadic |)
- subtract (dyadic -)
- unique (monadic ∪)
- unique mask (monadic ≠)

### 🧱 In progress
### 🧱 In progress and Next up

All details about upcoming tests can be found in the [project board](https://github.com/orgs/Dyalog/projects/4/views/1)

## ✍ The name

Pronounced as `/ˈulːluː/`, The name comes from the Hindi word for owl.
Pronounced as `/ˈulːluː/`, The name comes from the Hindi word for owl. The owl looks over Dyalog APL when everyone else is asleep.

Just as the owl represents both wisdom and foolishness the QA also has a dual nature of being wise and dumb at the same time.

Expand Down
154 changes: 154 additions & 0 deletions tests/multiply.apln
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
:Namespace multiply
Assert←#.unittest.Assert
stripToSameLen←#.utils.stripToSameLen
isDyalogClassic←#.utils.isClassic

complexMultiplyModel←{
(⎕FR≡1287) ∧ case≡'Hcmplx': ⍺×⍵ ⍝ Skipping cmplx for ⎕FR←1287
a c b d←∊(9 11○⊂)⍺ ⍵
⍝ using formula (a+bi)(c+di)=(ac−bd)+(ad+bc)i
x←((a model c)-(b model d))
y←((a model d)+(b model c))
x(⊣+¯11○⊢)y
}

⍝ model for multiply might not be completely accurate for floats
⍝ because it uses reciprocal and divide and there might be a
⍝ loss of precision
model←{⍺{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use {*(⍟⍺)+⍟⍵} as the model (using logarithms instead)? This would remove the need to handle complex numbers specifically.

Or, alternatively, could you use multiple models for redundancy?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a minor problem with the model that it gives complex answers from negative numbers. I changed the model to:

    model{{
            0:0
            0:0
            (¨÷/× )×*(|)+|
        }¨
    }

(it is a makeshift approach, i know the × should not be there to multiply the sign)

The problem with this now is the direction operator approach will not give me the sign of the for the complex numbers but the direction of the complex vector(i think). Is there a solution to this?

0≡⍵:0
⍝ is ⍵ or ⍺ complex
(1289≡⎕DR ⍵) ∨ (1289≡⎕DR ⍺): ⍺ complexMultiplyModel¨ ⍵
⍺⌹¨÷⍵
sloorush marked this conversation as resolved.
Show resolved Hide resolved
}¨⍵}

∇ {r}←test_multiply;fr_dbl;fr_decf;RunVariations;bool;i1;i2;i3;char0;char1;char2;char3;charptr;dbl;cmplx;Hcmplx;Hdbl;Sdbl;fl;Hfl;testDesc;case;case2;fr;desc;data;d1;d2;data2;case3;data3;c1;c2;f
RunVariations←(model #.testfns._RunVariations_ ×)

⍝ constants
fr_dbl←#.utils.fr_dbl
fr_decf←#.utils.fr_decf

⍝ All data generated is unique
bool←0 1 ⍝ 11: 1 bit Boolean type arrays
i1←{⍵,-⍵}⍳120 ⍝ 83: 8 bits signed integer
i2←{⍵,-⍵}10000+⍳100 ⍝ 163: 16 bits signed integer
sloorush marked this conversation as resolved.
Show resolved Hide resolved
i3←{⍵,-⍵}100000+⍳100 ⍝ 323: 32 bits signed integer
char0←⎕AV ⍝ 82: DyalogAPL classic char set
:If ~isDyalogClassic
char1←⎕UCS (100+⍳100) ⍝ 80: 8 bits character
char2←⎕UCS (1000+⍳100) ⍝ 160: 16 bits character
char3←⎕UCS (100000+⍳100) ⍝ 320: 32 bits character
:EndIf
charptr←(13↑⎕a) (13↓⎕a) ⍝ 326: Pointer (32-bit or 64-bit as appropriate)
dbl←{⍵,-⍵}i3+0.1 ⍝ 645: 64 bits Floating
cmplx←{⍵,-⍵}(0J1×⍳100)+⌽⍳100 ⍝ 1289: 128 bits Complex
Hcmplx←{⍵,-⍵}(1E14J1E14×⍳20) ⍝ 1289 but larger numbers to test for CT value
⍝ Hdbl is 645 but larger numbers to test for CT value
⍝ intervals of 2 are chosen because CT for these numbers +1 and -1
⍝ come under the region of tolerant equality
Hdbl←{⍵,-⍵}1E14+(2×⍳50) ⍝ 645: large numbers
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I would be testing more "interesting" values too. For mantis 21743 the test was using 9007199254740992, which was the largest float that can be expressed as an integer.

Perhaps also use inputs that you know will produce (close to) the largest resulting float answer (~1e154)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more bigger values are supposed to be tested by the random number generator in RunVariations but because of the limitations of the current model I had turned it off. I will move to your model and try it out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way I can easily find these 'interesting values' in mantis? For now, are there any specific values you would want to include other than this?

Sdbl←{⍵,-⍵}(⍳500)÷1000 ⍝ 645: Small numbers
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, in general, testing more small numbers would be good. Including denormal values (e.g. 1e¯310). Not that it should cause any issues here, but still.


⍝ Hfl is 1287 but larger numbers to test for CT value
⍝ far intervals are chosen for non overlap
⍝ with region of tolerant equality
⎕FR←fr_decf
fl←{⍵,-⍵}i3+0.01 ⍝ 1287: 128 bits Decimal
Hfl←{⍵,-⍵}2E29+(1E16×⍳10)
⎕FR←fr_dbl

r←⍬
testDesc←{'for ',case,{0∊⍴case2:'',⍵⋄' , ', case2,⍵}, '& ⎕FR:', ⎕FR}

r case case2←⍬ ⍬ ⍬

:For fr :In 2 1
⎕FR←fr⊃fr_dbl fr_decf

quadparams←⎕CT ⎕DCT ⎕FR ⎕IO ⎕DIV
case←'independant'
desc←testDesc⍬

r,←'TI1' desc Assert 0≡0×0
r,←'TNull1' desc Assert ⍬≡''×0 ⍝ testing for null values
r,←'TNull2' desc Assert ⍬≡⍬×0
r,←'TNull3' desc Assert ⍬≡0×''
r,←'TNull4' desc Assert ⍬≡0×⍬

:For case :In 'bool' 'i1' 'i2' 'i3' 'dbl' 'fl' 'cmplx' 'Hdbl' 'Sdbl' 'Hfl' 'Hcmplx'
data←⍎case
desc←testDesc⍬

⍝ General tests to test the basic rules of multiply
r,← 'TGen1' desc Assert (≢data)≡≢data×data

d1←(data~0)[?≢data~0]

r,← 'T1' desc quadparams RunVariations (data model data) data data ⍝ model test
d1←data[?≢data]
d2←data[?≢(data~d1)]
r,← 'T2' desc quadparams RunVariations data data 1 ⍝ Identity
r,← 'T3' desc quadparams RunVariations (0⍨¨data) data 0 ⍝ x×0=0
r,← 'T4' desc Assert 0≡0×d1
r,← 'T5' desc Assert 0≡d1×0
r,← 'T6' desc Assert d1≡1×d1
r,← 'T7' desc Assert d1≡d1×1
r,← 'T8' desc Assert (d1×d2)≡d2×d1 ⍝ commutability
:EndFor

⍝ Cross type tests
:For case2 :In 'i1' 'i2' 'i3' 'dbl' 'fl' 'cmplx' 'Hdbl' 'Hfl' 'Hcmplx'
data2←⍎case2
data data2←data stripToSameLen data2
desc←testDesc⍬
:If (case≡case2)
:Continue
:EndIf

r,← 'TCross1' desc quadparams RunVariations (data model data2) data data2 ⍝ check result of different datatypes using model
r,← 'TCross2' desc quadparams RunVariations (data2 model data) data2 data ⍝ TCross1 but reversed
r,← 'TCross3' desc quadparams RunVariations ((data2,data) model (data,data2)) (data2,data) (data,data2) ⍝ concat 2 different types
r,← 'TCross4' desc Assert (((data2 data) model (data data2))≡(data2 data)×(data data2)) ⍝ merge two different arrays creating a pointer array


:EndFor

⍝ tests for known errors
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiply is a trivial primitive, but I'd still like to see tests for more places where we expect failure. E.g., ×⍨1e155. We don't want this to generate a 1287 accidentally, for example (even though that's improbable, we should still treat the interpreter as a black box and create that test).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to confirm:

We don't want this to generate a 1287 accidentally

Should I test if this gives a domain error or not?

:For case3 :In 'char1' 'char2' 'char3' 'char0' 'charptr'
:If (isDyalogClassic) ∧ (case≢'char0') ∧ (case≢'ptr')
:Continue ⍝ Skip tests for unicode character sets in DyalogAPL classic
:EndIf
data3←⍎case3
desc←testDesc⍬
c1←data3[?≢data3]
c2←(data3~c1)[?¯1+≢data3]

f←0 ⍝ flag
:Trap 11 ⍝ 11: Domain error
c1×c2 ⍝ N/A type × N/A type
:Else
f←1
r,← 'TDomainE2' desc Assert (f ∧ ⎕dmx.Message≡'') ⍝ check for error and dmx message
:EndTrap

f←0 ⍝ flag
:Trap 11 ⍝ 11: Domain error
d1×c2 ⍝ Number type × N/A type
:Else
f←1
r,← 'TDomainE3' desc Assert (f ∧ ⎕dmx.Message≡'') ⍝ check for error and dmx message
:EndTrap

f←0 ⍝ flag
:Trap 11 ⍝ 11: Domain error
c1×d1 ⍝ N/A type × number type
:Else
f←1
r,← 'TDomainE4' desc Assert (f ∧ ⎕dmx.Message≡'') ⍝ check for error and dmx message
:EndTrap
:EndFor
:EndFor
:EndNamespace