Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introducing LocalGraphConfusion and its corresponding Precision and Recall classes #1771

Merged
merged 2 commits into from
May 17, 2024

Conversation

vbcwonderland
Copy link
Collaborator

@vbcwonderland vbcwonderland commented May 16, 2024

Introducing LocalGraphConfusion class, where it calculates the tp, tn, fp, fn based on:

/**     First, check Adjacency
         *                     True
         *               Y             N
         *            ---------------------
         *         Y |    TP            FP
         *    Est    | --------------------
         *         N |    FN            TN
         *           -----------------------
         */

/**. Second, check Orientation (ArrowHead)
         *                     True
         *                ->          <-        ...(None)
         *            ---------------------------
         *         -> |   TP        FP,FN       / (Do not repeat count, as we checked for it in Adj step)
         *    Est     | --------------------------
         *         <- |   FP, FN     TP         /
         *            | --------------------------
         *         -- |   FN         FN         /
         *            | --------------------------
         *         ...|    /         /          /
         *           -----------------------------
         *
         */

and LocalGraphConfusion 's corresponding LocalGraphPrecision, LocalGraphRecall classes.
This PR also applied this way of confusion calculation in MarkovCheck.
And included a test testDAGPrecisionRecall2ForLocalOnMarkovBlanket .

Sample test output:

Test True Graph: Graph Nodes:
X1;X2;X3;X4;X5;X6;X7;X8;X9;X10

Graph Edges:
1. X1 --> X3
2. X1 --> X5
3. X1 --> X7
4. X1 --> X10
5. X2 --> X6
6. X3 --> X9
7. X3 --> X10
8. X5 --> X8
9. X6 --> X7
10. X6 --> X9


Test True Graph size: 10
Test Estimated CPDAG Graph: Graph Nodes:
X1;X2;X3;X4;X5;X6;X7;X8;X9;X10

Graph Edges:
1. X1 --- X3
2. X1 --- X5
3. X1 --- X10
4. X2 --> X6
5. X3 --- X9
6. X3 --- X10
7. X5 --- X8
8. X6 --> X7
9. X9 --> X6


~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Accepts size: 2
Rejects size: 8
=====================
xMBLookupGraph:Graph Nodes:
X2;X6

Graph Edges:
1. X2 --> X6


xMBEstimatedGraph:Graph Nodes:
X9;X2;X6

Graph Edges:
1. X2 --> X6
2. X9 --> X6


Node X2's statistics:  
 LocalGraphPrecision = 0.67 LocalGraphRecall = 1.00 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X1;X6;X7

Graph Edges:
1. X1 --> X7
2. X6 --> X7


xMBEstimatedGraph:Graph Nodes:
X6;X7

Graph Edges:
1. X6 --> X7


Node X7's statistics:  
 LocalGraphPrecision = 1.00 LocalGraphRecall = 0.67 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X10;X1;X3;X5;X6;X7

Graph Edges:
1. X1 --> X3
2. X1 --> X5
3. X1 --> X7
4. X1 --> X10
5. X3 --> X10
6. X6 --> X7


xMBEstimatedGraph:Graph Nodes:
X10;X1;X3;X5

Graph Edges:
1. X1 --- X3
2. X1 --- X5
3. X1 --- X10
4. X3 --- X10


Node X1's statistics:  
 LocalGraphPrecision = 1.00 LocalGraphRecall = 0.40 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X9;X10;X1;X3;X6

Graph Edges:
1. X1 --> X3
2. X1 --> X10
3. X3 --> X9
4. X3 --> X10
5. X6 --> X9


xMBEstimatedGraph:Graph Nodes:
X9;X10;X1;X3

Graph Edges:
1. X1 --- X3
2. X1 --- X10
3. X3 --- X9
4. X3 --- X10


Node X3's statistics:  
 LocalGraphPrecision = 1.00 LocalGraphRecall = 0.44 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X4

Graph Edges:


xMBEstimatedGraph:Graph Nodes:
X4

Graph Edges:


Node X4's statistics:  
 LocalGraphPrecision = NaN LocalGraphRecall = NaN 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X8;X1;X5

Graph Edges:
1. X1 --> X5
2. X5 --> X8


xMBEstimatedGraph:Graph Nodes:
X8;X1;X5

Graph Edges:
1. X1 --- X5
2. X5 --- X8


Node X5's statistics:  
 LocalGraphPrecision = 1.00 LocalGraphRecall = 0.50 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X9;X1;X2;X3;X6;X7

Graph Edges:
1. X1 --> X3
2. X1 --> X7
3. X2 --> X6
4. X3 --> X9
5. X6 --> X7
6. X6 --> X9


xMBEstimatedGraph:Graph Nodes:
X9;X2;X6;X7

Graph Edges:
1. X2 --> X6
2. X6 --> X7
3. X9 --> X6


Node X6's statistics:  
 LocalGraphPrecision = 1.00 LocalGraphRecall = 0.56 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X8;X5

Graph Edges:
1. X5 --> X8


xMBEstimatedGraph:Graph Nodes:
X8;X5

Graph Edges:
1. X5 --- X8


Node X8's statistics:  
 LocalGraphPrecision = 1.00 LocalGraphRecall = 0.50 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X9;X3;X6

Graph Edges:
1. X3 --> X9
2. X6 --> X9


xMBEstimatedGraph:Graph Nodes:
X9;X2;X3;X6

Graph Edges:
1. X2 --> X6
2. X3 --- X9
3. X9 --> X6


Node X9's statistics:  
 LocalGraphPrecision = 0.67 LocalGraphRecall = 0.50 

=====================
=====================
xMBLookupGraph:Graph Nodes:
X10;X1;X3

Graph Edges:
1. X1 --> X3
2. X1 --> X10
3. X3 --> X10


xMBEstimatedGraph:Graph Nodes:
X10;X1;X3

Graph Edges:
1. X1 --- X3
2. X1 --- X10
3. X3 --- X10


Node X10's statistics:  
 LocalGraphPrecision = 1.00 LocalGraphRecall = 0.50 

=====================

Process finished with exit code 0

@vbcwonderland vbcwonderland changed the title Introducing Local Graph Confusion and its corresponding Precision and Recall Introducing LocalGraphConfusion and its corresponding Precision and Recall classes May 16, 2024
Copy link
Collaborator

@jdramsey jdramsey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good!

@jdramsey jdramsey merged commit ae16a35 into development May 17, 2024
@jdramsey jdramsey deleted the vbc-05-16-2 branch May 17, 2024 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants