Fuzzy finder reindex whole project everytime

[Trudko]

Hi,

every time I open project and use fuzzy finder, whole project is re indexed which takes some time.
I cd-ed into project (place in Linux server accessed by Samba)
Run atom --safe.
Open fuzzy finder.

I run this using --safe option. Using 0.187 Atom (Windows 8.1).

[izuzak]

@Trudko Does this happen for local projects as well, or only projects stored remotely?

[shuriktm]

Hello,

It happens for me as well.
Only for remote projects (placed in virtual machine in VirtualBox).
Local projects do not require reindex.

Using Atom version 0.194.0 on Windows 8.1.

[izuzak]

Thanks, @shuriktm.

@Trudko is that the behavior you're seeing as well?

[ev-mark]

I have also been seeing overly frequent index refreshes on 0.194.0 / Windows 8.1. This is for local projects. How long does the index persist by default? This would be daily re-indexing here. Any other details you need I can try to get?

[captainbrosset]

This also happens to me.
MacOS Yosemite, Atom version: 0.198.0.

I work on a large codebase (the mozilla project) and so my preferred way of navigating the code is by using the cmd-P shortcut. I'm originally a ST2 user and just love this quick-jump to file/line way to navigate files.

Using Atom, I find this to be really slow.
Several things here:

• The first time I open the project and press cmd-P, it says "indexing project ...", which I can understand. It takes some time, but I'm prepared to wait if I know that it's only going to do that the first time.
• When it's done and I use the auto-complete to find files, the experience is really janky (but that's probably another bug).
• It seems to want to re-index every now and then when I later use cmd-P again, even though nothing has changed in the project. It shows a "re-indexing" message, but that doesn't prevent me from using the auto-complete, so it's kind of ok, but weird that it does it anyway.
• If I close and then re-open Atom and press cmd-P again, it says "indexing project ..." again, and I can't use the auto-complete to find files for a while again.

Also, probably related to this, I can hear my laptop fan spin pretty much all the time when using Atom.
As a comparison, I never experience these slow-downs, high CPU usage and frequent re-indexing with ST2.

You can reproduce this easily:

• clone this repo: https://github.com/mozilla/gecko-dev/
• open the source folder in Atom
• type cmd-P -> long indexing time
• wait until it completes and try to search for a file, e.g. markup-view.js -> lag when typing
• close Atom and open again
• type cmd-P again -> indexing again

Oh and, I can also reproduce in safe mode.

[snorp]

I also have been trying to use Atom here at Mozilla, and am encountering this problem. If I disable the fuzzy-finder package, the problems goes away.

While this problem is occurring, I see a lot of CPU usage in opendirectoryd and automountd. I used odutil to turn on logging for opendiscoveryd and it yields the following:

2015-07-14 16:26:01.408216 PDT - AID: 0x0000000000000000 - 73687.39262048 - Client: automountd, UID: 0, EUID: 0, GID: 0, EGID: 0
2015-07-14 16:26:01.408230 PDT - AID: 0x0000000000000000 - 73687.39262048 - ODNodeCreateWithNameAndOptions request, SessionID: 00000000-0000-0000-0000-000000000000, Name: /Search, Options: 0x0
2015-07-14 16:26:01.408250 PDT - AID: 0x0000000000000000 - reaping connection '/Search:search:B296F994-EADD-48C8-98D0-B1B417909360'
2015-07-14 16:26:01.408253 PDT - AID: 0x0000000000000000 - 73687.39262048, Node: /Search, Module: search - tracking connection '/Search:search:4F9CF27A-00DF-40EA-9F6B-A72448459479'
2015-07-14 16:26:01.408267 PDT - AID: 0x0000000000000000 - 73687.39262048, Node: /Search, Module: search - initiating reconnect for '/Search:search:4F9CF27A-00DF-40EA-9F6B-A72448459479'
2015-07-14 16:26:01.408289 PDT - AID: 0x0000000000000000 - 73687.39262048, Node: /Search - node assigned UUID - FDD517C3-4AC8-43C2-985A-30E639E8CEE3
2015-07-14 16:26:01.408382 PDT - AID: 0x0000000000000000 - 73687.39262048, Node: /Search - ODNodeCreateWithNameAndOptions completed
2015-07-14 16:26:01.408623 PDT - AID: 0x0000000000000000 - 73687.39262049 - Client: automountd, UID: 0, EUID: 0, GID: 0, EGID: 0
2015-07-14 16:26:01.408667 PDT - AID: 0x0000000000000000 - 73687.39262049 - ODQueryCreateWithNode request, NodeID: FDD517C3-4AC8-43C2-985A-30E639E8CEE3, RecordType(s): dsRecTypeStandard:Automount, Attribute: dsAttrTypeStandard:RecordName, MatchType: EqualTo, Equality: CaseExact, Value(s): *,automountMapName=auto_home, Requested Attributes: dsAttrTypeStandard:RecordName,dsAttrTypeStandard:RecordType,dsAttrTypeStandard:GeneratedUID,dsAttrTypeStandard:AutomountInformation,dsAttrTypeStandard:AppleMetaRecordName,dsAttrTypeStandard:AppleMetaNodeLocation, Max Results: 2147483647
2015-07-14 16:26:01.408711 PDT - AID: 0x0000000000000000 - 73687.39262049, Node: /Search - queuing request to connection - '/Search:search:4F9CF27A-00DF-40EA-9F6B-A72448459479'
2015-07-14 16:26:01.408747 PDT - AID: 0x0000000000000000 - 73687.39262049, Node: /Search, Module: search - forwarding request to node name '/Local/Default'
2015-07-14 16:26:01.408756 PDT - AID: 0x0000000000000000 - 73687.39262049.39262050, Module: search - Request being forwarded
2015-07-14 16:26:01.408784 PDT - AID: 0x0000000000000000 - 73687.39262049.39262050, Module: search - ODQueryCreateWithNode request, NodeID: 275116F8-8960-4FCA-AC1F-2851F653DCAC, RecordType(s): dsRecTypeStandard:Automount, Attribute: dsAttrTypeStandard:RecordName, MatchType: EqualTo, Equality: CaseExact, Value(s): *,automountMapName=auto_home, Requested Attributes: dsAttrTypeStandard:RecordName,dsAttrTypeStandard:RecordType,dsAttrTypeStandard:GeneratedUID,dsAttrTypeStandard:AutomountInformation,dsAttrTypeStandard:AppleMetaRecordName,dsAttrTypeStandard:AppleMetaNodeLocation, Max Results: 2147483647
2015-07-14 16:26:01.408807 PDT - AID: 0x0000000000000000 - 73687.39262049.39262050, Node: /Local/Default, Module: search - queuing request to connection - '/Local/Default:PlistFile:45CFC7F7-0870-4726-AF84-7B000E8CAC4E'
2015-07-14 16:26:01.408962 PDT - AID: 0x0000000000000000 - 73687.39262049.39262050, Node: /Local/Default, Module: PlistFile - ODQueryCreateWithNode completed
2015-07-14 16:26:01.409029 PDT - AID: 0x0000000000000000 - 73687.39262049, Node: /Search, Module: search - ODQueryCreateWithNode completed
2015-07-14 16:26:01.409230 PDT - AID: 0x0000000000000000 - 73687.39262051 - Client: automountd, UID: 0, EUID: 0, GID: 0, EGID: 0
2015-07-14 16:26:01.409243 PDT - AID: 0x0000000000000000 - 73687.39262051 - ODNodeRelease request, NodeID: FDD517C3-4AC8-43C2-985A-30E639E8CEE3
2015-07-14 16:26:01.409313 PDT - AID: 0x0000000000000000 - 73687.39262051, Node: /Search - ODNodeRelease completed
2015-07-14 16:26:01.409348 PDT - AID: 0x0000000000000000 - clearing all node authentication connections
2015-07-14 16:26:01.409352 PDT - AID: 0x0000000000000000 - Disconnecting /Search:search:4F9CF27A-00DF-40EA-9F6B-A72448459479

[snorp]

The last answer here seems to apply: http://superuser.com/questions/350879/opendirectoryd-consumes-40-of-cpu

[jancborchardt]

Same here, have been using Atom to work on ownCloud since some time. Version 1.0 still indexes the whole project every time newly all files have to be indexed first. For big projects this can take some time.

Instead the index should be cached to make the search immediately available instead of blocking it. Of course it can then index in the background.

Let me know if I can help debug this somehow.

[thbwd]

I also had this problem after I installed a new node module. I fixed it by simply ignoring the node_modules directory:

"fuzzy-finder":
    ignoredNames: [
      "**/node_modules/*"
    ]

[shadoWalker89]

Having the same issue on Windows 8.1 using atom 1.0.19

[SgtPooki]

Atom reindexes project folders that are on a samba mount every day.
Atom 1.0.19
Yosemite 10.10.5

[kandros]

Local folder, linux atom 1.1
@idmean where i'm supposed to put this to remove node_modules from indexing?

[thbwd]

@kandros Into your config.cson

[kandros]

@idmean not working :( even restarted
EDIT: my bad, had fuzzy finder twice in the settings and the second was not considered

Thanks a lot you made my day

[Zooce]

This happens to me pretty much every other time I use the fuzzy finder. I'm working on a local C++ project (that is very large). Occurring on Atom 1.2.3 and Atom 1.3.0-beta5, Windows 7.

[mwmcode]

Same here
(every time i CMD + P, it take some time to reindex and even when it's done, it's not as responsive/fast as it usually is)

Using Atom 1.2.4 on OS X El Capitan (project stored locally)

Update:
Happened when I created a new folder inside my controllers folder (laravel 5 project).

[sirianni]

This is happening for me, but only on certain projects (even small ones). Perhaps there is some symlnk or directory structure at fault? Is there anything I can do to help root cause?

[mamartins]

same here on local projects. Every time I hit cmd+p it says "reindexing project"

[vkalpias]

Same here - local project - fairly big, which makes cmd+p pretty slow.

[achekulaev]

I have the same issue. Might it be related to symlinks in the project?

[mattgreenfield]

Same problem here, large project on a remote server. Caching would be great.

[chafreaky]

Chiming in as well. What is blocking here? How can we get closer to a solution?

[thornjad]

Hello, can someone take a look at this? With a three-year-old bug that breaks the core of the package, I would think someone would care.

[umpirsky]

Is there alternative finder package?

[Swivelgames]

Is there any desire at all to prioritize this bug? This is three and a half years old now. I've been watching this issue for a long time now. What the hell is going on?

[danieldevine]

What worked for me was to go to settings > core and tick 'Exclude VCS Ignored paths.'

[thornjad]

The 'Exclude VSC Ignored Paths' would make sense, if it's trying to index, say, node_modules/ and other ignored paths. That option should probably be enabled by default.

[umpirsky]

Excluding VCS ignored paths means you can't find vendor files, which sucks.

[lee-dohm]

@Swivelgames, thanks for your feedback.

Open source software depends on contributions from the community in order for it to reach its maximum potential. We can understand that some people will not agree with our choices in how to prioritize our work. One choice people have is to contribute their own work to make something happen, whether that is in the form of a pull request, a design of a solution, creating an Atom package, or pulling together a team to create the solution. Working together to make awesome stuff is what open source is all about.

Demands for status, declarations of how long something has been open, requiring us to justify our prioritization, or other antagonistic requests for response to an issue or PR are all considered personal attacks under our Code of Conduct. Let's keep things classy instead.

[Standard8]

@lee-dohm It would be useful if we could get some "core" atom developers to chime in on possible solutions on this issue.

I took a look a couple of years ago and looked at possibilities, but no-one responded and the work seemed a lot to do without being able to know if it was the right thing to do, hence at the time I left it.

Unfortunately I don't have time now to do anything - but I think it might help for others to pick up, or know what to do if there is an agreed plan, as I think this is currently more an architectural issue than a simple bug.

My previous comments are here:

#88 (comment)
#88 (comment)

[webbower]

@lee-dohm That's fair to call out the Code of Conduct for antagonistic comments. However, as @Standard8 alluded to, a member of the core team has not chimed in on this thread since it was opened over 3 years ago, and as the activity has indicated here, this is arguably high-demand bug. So while antagonistic comments don't help, silence from the core team is also not helpful, and can definitely result in frustration among the community.

While OSS "depends on contributions from the community," it also depends on guidance from the core team. And the second part of that has not been apparent on this thread. Is there someone among the core team who can own support for this bug and help people among the community to address this?

[christiancho]

I'm with @webbower on this. I switched to VS Code because of this and I still get email notifications about this report. It amazes me that this hasn't been fixed for years. I also love how all of the comments that reference making a switch are marked off topic. Honestly, this is a complete joke now. I've been recommending people to switch off of Atom for more than a year now.

[webbower]

Seems like this should be getting more attention from the core team, even if that means helping a community member to fix the bug, since it's causing attrition in the community.

[umpirsky]

As someone who loves the editor, and for example don't use GitHub integration (I prefer CLI), I sometimes feel a little sad seing new releases bragging about new GitHub related features while ignoring issues like this. I don't know how priorities are decided, but looks like comments, 👍 on issues and voice of the community in general does not count.

[umpirsky]

I'm sad to say that after almost 4 years of frustration waiting for entered text to appear and find the file I was looking for (which takes ~4 seconds in my case (I have a decent hardware)), I decided to switch to Visual Studio Code. It's just because of this issue, I prefer Atom in every other way.

I will consider switching back if this gets fixed. I sincerely hope it will happen soon.

[mjmasn]

Can confirm this issue is still in Atom 1.34.0 running on Ubuntu 16.04 LTS and as mentioned by others above it is only triggered by the window losing/regaining focus.

[kaiyoma]

Confirmed in Atom 1.34.0 on Windows 10. Pretty annoying.

[webbower]

ZOMG!!! There's some hope https://blog.atom.io/2019/03/12/atom-1-35.html#fuzzy-finder-performance

[umpirsky]

But does it prevents re-indexing?

[thornjad]

The blog post references #366, which looks to have been released as 1.9.2. I no longer use Atom, someone else want to try it out with the new version?

[winstliu]

@umpirsky No, I don't believe that affects when fuzzy-finder decides to re-index. We are still working on reducing the time it takes to reindex the project, though - see #369.

[webbower]

I'll happily take faster re-indexes for now as a step toward a less painful fuzzy finder experience

[andreyorst]

Same here, Atom's file conunter in indexer window stops at some point while Indexing projects accessed with sshfs which contain symbolic links. If no symbolic links exist there's no problem indexing, or so it seems.

Edit: Also it happens only whenever Use Rip Grep feature is toggled on.