New version still relies on Node.js for incremental loading

[karatekaneen]

From what I understand from #92 with the point below the dependency on Node.js should be deleted.

• only one Java application for everything (removal of the Node.js part [...])

However, when running the server in a fresh VM (with only Java installed and not Node) the following errors is logged:

ERROR [2024-05-09 03:00:19,971] com.scienceminer.glutton.utils.crossrefclient.IncrementalLoaderTask: IO error when executing external command: [node, main, -dump, /media/lopez/data2/crossref/2024-05-06/D1000005.json.gz, extend]
! java.io.IOException: error=2, No such file or directory
! at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
! at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
! at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
! at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
! ... 7 common frames omitted
! Causing: java.io.IOException: Cannot run program "node" (in directory "../indexing"): error=2, No such file or directory
! at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
! at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
! at com.scienceminer.glutton.utils.crossrefclient.IncrementalLoaderTask$IndexCrossrefFile.run(IncrementalLoaderTask.java:312)
! at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
! at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
! at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
! at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
! at java.base/java.lang.Thread.run(Thread.java:829)

The cause is on these lines:

class IndexCrossrefFile implements Runnable { 
  // ... Stuff was here ... //

  @Override
  public void run() { 
    System.out.println("indexing: " + crossrefFile.getPath());

    ProcessBuilder builder = new ProcessBuilder();
    // HERE is the problematic thing:
    // command is: node main -dump ~/tmp/crossref_public_data_file_2021_01 index
    builder.command("node", "main", "-dump", crossrefFile.getAbsolutePath(), "extend");            
    builder.directory(new File("../indexing"));

    /* ... More code here ... */
  }
}

[kermitt2]

Hi @karatekaneen ! Sorry for forgetting to remove this external call (normally it was just failing but had no impact on the correct indexing). This is fixed.

[karatekaneen]

No need to apologize @kermitt2, it's open source :D
Thanks for fixing and thanks for all the hard work!

Haven't verified that the fix works yet but I assume that it will.