Buckwheat module can not be found in python3

[sariforo]

Hello,

I am trying to use Buckwheat on Mac but when executing the aforementioned command "python3 -m buckwheat. run"
I am facing the error (ImportError: No module named 'buckwheat' )
Any help will be pleasant!

[areyde]

Hello!

I just checked and found no such error, hmm. Are you running the command from the root folder? The command must be run from the folder with the README, license, etc. (from inside the cloned directory)

diannao@areyde:~/Downloads/buckwheat$ python3 -m buckwheat.run -i input.txt -o output
worked for me. Could you please give some more details?

[sariforo]

Thanks very much for the reply.
Well, I have a bunch of extracted diff files from Github commits from different programming languages.
Using those diffs, I have extracted only changed code lines and save them to files.

Now my faced challenge is to tokenize such files and I have found your tools quite useful for this purpose. By the way, these files are located in my local directory ( a simple directory containing such files). May you help me with how to proceed?
You may want to see the current folder for which I saved files in the enclosed screenshot.

Thank you so much in advance.

[areyde]

There might be some problem, because, see, our tool is made for extracting identifiers out of source code. What this means is that the tool builds the AST of a given file, then takes out tokens that correspond to identifiers. The problem is that if you have only changed lines of code, the AST might not be built correctly, or something. Just please note that you will have only identifiers extracted, you are welcome to give it a shot.

You will need to create a text file with the full path the directory that contains all the files with the code (probably this is the directory in you screenshot). Like this:

/full/path/to/directory/

Note that our tool uses a language recognizer, so make sure that the extensions of files correspond to your language (that you did not specify in your comment). So if your diffs are from Java, use .java, if from Python, use .py.

Save this file as, for example, input.txt. Then simply run the command:

python3 -m buckwheat.run -i input.txt -o output -f json --local

This will treat you main directory as a project, and all you diffs as files in that project. A JSON file will be saved to the output directory, where filepaths will be the keys and lists of identifiers will be the values. For more information about the formats and granularities, check the README.

However, like I said, it might not work great, because we only tested the tool for full source files. If it does not work, you may want to try tree-sitter or pygments directly.

[sariforo]

Thanks again.
However as before I have done all these steps you described me but I am getting the same error I told first . (ImportError: No module named 'buckwheat' ).
I think the issue refers to the required dependencies and where the installation is located!
Should I take into account the location of the requirements.text in local derive?
First I need to solve this and later on see if AST works out with my case or not.

[egor-bogomolov]

Hello @sariforo! Could you please tell in which directory you run the aforementioned command?

[sariforo]

Hello @egor-bogomolov and thanks for the follow up!
I have been trying several times but all were failed!

I executed the aforementioned command in my Users directory (e.g., Mac-Pro:~ sarafrt$) and I specified the path to the files folder ( such folder contains bunch of files written in various programming languages and I would get list of identifiers from those files..) in the 'input.text' .
So, I have a folder containing list of files and 'input.text' for which I have the path to this folder. I am totally crazy about this step!
looking forward to hearing you.

Thank you.

[areyde]

Yeah, that is the problem! You need run the command from the cloned folder. So you clone this repository, navigate to it, and then run the command. From the directory with the README, the license file, etc.

[egor-bogomolov]

An alternative solution would be to add path to the cloned directory to your python path environment variable