FCI crashes when using too many forbidden edges

[tk27182]

I'm trying to run FCI on a dataset (N=920, 17 columns) to confirm the network structure I got when running a different algorithm. To do this, I'm forbidding all the edges that were not found by this other algorithm. When I try to run FCI using the Tetrad 6.6.0 GUI for Mac or Rcausal, the process crashes and no graph is output. My data does have missing data so I've tried a few things:

1. Run FCI with forbidden edges on full dataset (N=920) and the subset with no missing values (N=368) ==> no graph is output
2. Run FCI without forbidden edges on full dataset and the subset with no missing values ==> runs for both and produces a graph.
3. Run RFCI and GFCI with forbidden edges on the full and subset datasets ==> runs for both and produces a graph.
4. Run RFCI and GFCI without forbidden edges on the full and subset datasets ==> runs for both and produces a graph.
5. Create a simulated dataset (w/o missing values) in the GUI. Look at the true graph. Add forbidden edges such that the only edges remaining are those shown in the true graph. Run FCI ==> no graph is output
6. Create a simulated dataset (w/o missing values) in the GUI. Look at the true graph. Add a few forbidden edges but not enough to recreate the true graph. ==> produces a graph

Based on these tests, it seems that FCI has a restriction on the number of forbidden edges it can handle. I'm not sure how Tetrad handles the forbidden edges, so this may not be the case, but are there any restrictions on the number of forbidden edges FCI can handle?

Edit: Possibly related to Issue #1244 ?
Parameters for FCI: p = 0.05, maxSize = 3, no maximum length, no bootstrapping, highest ensemble method, complete FCI rule set used, fisher-z test.

[tk27182]

When I run with FCI and use the Logging tool (from the simulation), the output is:

Starting FCI algorithm.
Independence test = Chi Square, alpha = 0.0500.
Starting Fast Adjacency Search.
Finishing Fast Adjacency Search.
Starting BK Orientation.
Finishing BK Orientation.

So it seems like it never finishes the BK Orientation. I have a .tet file of the simulation but Github doesn't support that file type.

[jdramsey]

I seem to recall someone else having this problem before, and we fixed their problem. You're using 6.6; I'm wondering if you download the latest version of Tetrad whether the problem mightn't be solved. Joe

…

On Thu, Jan 30, 2020 at 6:21 PM tk27182 ***@***.***> wrote: When I run with FCI and use the Logging tool (from the simulation), the output is: Starting FCI algorithm. Independence test = Chi Square, alpha = 0.0500. Starting Fast Adjacency Search. Finishing Fast Adjacency Search. Starting BK Orientation. Finishing BK Orientation. So it seems like it never finishes the BK Orientation. I have a .tet file of the simulation but Github doesn't support that file type. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1246?email_source=notifications&email_token=ACLFSR3IPOLW7JNRKZF5JALRANOI3A5CNFSM4KHKKY4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKM5VQQ#issuecomment-580508354>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACLFSR5SEGRBBQHVJL6LQV3RANOI3ANCNFSM4KHKKY4A> .

-- Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213 [email protected] Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

Anyway, can you give it a try? If it doesn't work, I'll think about it more. Joe On Fri, Jan 31, 2020 at 1:47 AM Joseph Ramsey <[email protected]> wrote:

…

I seem to recall someone else having this problem before, and we fixed their problem. You're using 6.6; I'm wondering if you download the latest version of Tetrad whether the problem mightn't be solved. Joe On Thu, Jan 30, 2020 at 6:21 PM tk27182 ***@***.***> wrote: > When I run with FCI and use the Logging tool (from the simulation), the > output is: > > Starting FCI algorithm. > Independence test = Chi Square, alpha = 0.0500. > Starting Fast Adjacency Search. > Finishing Fast Adjacency Search. > Starting BK Orientation. > Finishing BK Orientation. > > So it seems like it never finishes the BK Orientation. I have a .tet file > of the simulation but Github doesn't support that file type. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#1246?email_source=notifications&email_token=ACLFSR3IPOLW7JNRKZF5JALRANOI3A5CNFSM4KHKKY4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKM5VQQ#issuecomment-580508354>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ACLFSR5SEGRBBQHVJL6LQV3RANOI3ANCNFSM4KHKKY4A> > . > -- Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213 ***@***.*** Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

-- Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213 [email protected] Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[tk27182]

I downloaded the latest GUI and ran the simulation again with FCI. The logged output was the same as before, getting as far as finishing BK orientation, before not stopping and no graph is output.

[jdramsey]

Thanks, I'll think about it then. I don't suppose you could send me your data and knowledge so I can properly debug? You can send it to my personal email if you don't want to post it here.

…

On Mon, Feb 3, 2020 at 10:10 AM tk27182 ***@***.***> wrote: I downloaded the latest GUI and ran the simulation again with FCI. The logged output was the same as before, getting as far as finishing BK orientation, before not stopping and no graph is output. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1246?email_source=notifications&email_token=ACLFSR2RZTNUS7Z55MTBMYLRBAXURA5CNFSM4KHKKY4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKUF6MY#issuecomment-581459763>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACLFSR4TYUVP2WY7FUNOKQTRBAXURANCNFSM4KHKKY4A> .

-- Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213 [email protected] Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

I looked at the file you sent. This was made using Tetrad version 6.6; the latest published version is 6.7. (The repository version is 6.8, ignore that.) I downloaded 6.7...your session still loads! And FCI runs in all your cases.

I can be explicit. Here's the page with all of the versions on it, where I clicked to 6.7.0.

https://cloud.ccd.pitt.edu/nexus/content/repositories/releases/edu/cmu/tetrad-gui/6.7.0/

From this, download the file called

tetrad-gui-6.7.0-launch.jar

I think you can just double click it to launch it the usual way; if not, whatever if normal for you computer. Then load your session, and I think you're good.

Let me know if now and I will think harder. But like I said I have a memory of fixing an issue by this description.

[jdramsey]

Oh hold on, maybe I do see something... let me look again...

[jdramsey]

Nope, I loaded it from the command line so I could see the errors being printed out. None were being generated by FCI.

I think it's good.

[cg09]

try loading it from the link you give. it won't load for me. All I get is the "loading tetrad" box, but no tetrad load.

…

On Mon, Feb 3, 2020 at 4:05 PM Joseph Ramsey ***@***.***> wrote: Nope, I loaded it from the command line so I could see the errors being printed out. None were being generated by FCI. I think it's good. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1246?email_source=notifications&email_token=AD4Y3OPSGKMH5K443FTDXCDRBCBKDA5CNFSM4KHKKY4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKVMQ6A#issuecomment-581617784>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4Y3OMOZRTRZBY5J7MWZE3RBCBKDANCNFSM4KHKKY4A> .

[tk27182]

My apologies for sending the outdated file. I had no issues downloading the latest version, but still do not get an output after running FCI with the forbidden edges described in the knowledge file. Thanks for your help!

[jdramsey]

You probably have Tetrad open in another window. Close all Tetrads first and then try again.

[jdramsey]

Oh I see; the problem you just sent is a separate issue. I think I know what the issue is, and it's fixed in the repository version. I'll have to get some advice on how to fix it in the the published version.

This isn't an FCI problem per se; it's a problem with the chi square test.

[jdramsey]

I think I've found and fixed the problem. I could send you a new jar--can I send that to the address you used ti send me the other file? If it works, we'll do a hot fix.

[jdramsey]

Thanks to help from @kvb2univpitt this hot fix has now been published as version 6.7.1 of Tetrad; you get it from the software page.

[tk27182]

Thanks for your help!

[tk27182]

Is it an issue also with the Fisher-z test? I know the example I sent only had discrete variables, but if I was using continuous variables instead would this still be an issue?

[jdramsey]

I don't think so, but it the fix not working for you?

[jdramsey]

@tk27182 Can I close this, or are you still having difficulty?

If you're having trouble with Fisher Z, let me know. I think if you're having trouble with it, switching to the new jar will fix it.

[tk27182]

@jdramsey Sort of. If I remove the missing values, then I have no issue using my knowledge file or FCI with my data. I thought in the past I was able to run FCI or other algorithms with missing values.

[jdramsey]

@tk27182 Hmm.. I said why that was true; do you think it was a valid thing to allow? I'm pretty sure it wasn't... Let me ask around to see what people think.

[jdramsey]