Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search index is broken #2708

Closed
humitos opened this issue Mar 8, 2017 · 32 comments
Closed

Search index is broken #2708

humitos opened this issue Mar 8, 2017 · 32 comments
Assignees
Labels
Bug A bug

Comments

@humitos
Copy link
Member

humitos commented Mar 8, 2017

Expected Result

Do not get the 404 at latest step :)

Actual Result

  1. go to http://docs.readthedocs.io/en/latest/search.html?q=break+internet&check_keywords=yes&area=default
  2. just one result appears
  3. click on the page
  4. 💣 get 404 (actual link http://docs.readthedocs.io/en/latest/faq.rst.html?highlight=break%20internet)

... and I broke the internet 😀

@humitos
Copy link
Member Author

humitos commented Mar 8, 2017

The problem is the extra .rst extension in the file in the URL. This one works: http://docs.readthedocs.io/en/latest/faq.html?highlight=break%20internet

@humitos humitos added the Bug A bug label Mar 8, 2017
@ericholscher
Copy link
Member

Guessing this might be fallout from the 1.5 upgrade, but not sure.

@agjohnson agjohnson self-assigned this Mar 8, 2017
@ericholscher
Copy link
Member

ericholscher commented Mar 8, 2017

This is from the Sphinx search (the RTD search results work fine: http://docs.readthedocs.io/en/latest/search.html?q=testing&check_keywords=yes&area=default#). Guessing we need to upgrade some of the search JS.

@agjohnson
Copy link
Contributor

I can look into this. Mostly certain it's the search override JS

@agjohnson
Copy link
Contributor

Also, for background here: our search overrides the default sphinx search. If our search returns no results, we back down to the sphinx search. What we're noticing here is that a missing search term from our index is a hit on the sphinx index, but something has changed with the sphinx implementation there.

So, our indexes aren't broken, but something is wrong with the sphinx failover.

@nhitchins
Copy link

nhitchins commented Mar 16, 2017

Any resolution? Search links have been broken for 10 days

@nhitchins
Copy link

Any chance of rolling back to a working version? Having a broken search feature is not really viable for the majority of docs users.

@ericholscher
Copy link
Member

Can you give an example of a broken link? This should only happen when Sphinx's default search is triggered, which is a minority of cases.

@ericholscher
Copy link
Member

You can change the version of sphinx your doc builds use using a requirements.txt file, if you need to roll back to an old version of sphinx.

@dlech
Copy link

dlech commented Apr 6, 2017

Can you give an example of a broken link?

http://docs.ev3dev.org/projects/lego-linux-drivers/en/ev3dev-jessie/search.html?q=i2c&check_keywords=yes&area=default

All of the links have the extra .rst in them that breaks the link.

@akien-mga
Copy link
Contributor

akien-mga commented Apr 6, 2017

Here's another example: http://docs.godotengine.org/en/stable/search.html?q=kinematicbody2d&check_keywords=yes&area=default
All results have the extra .rst.

@nhitchins
Copy link

Tried to use the requirements.txt file to specify an earlier sphinx version as suggested but the build is failing: https://readthedocs.org/projects/eagleio/builds/5258802/

@davendal
Copy link

Not a developer, however I just wanted to mention that to get search working properly again I had to go back to Sphinx 1.4.9 then it worked successfully. Otherwise it just hangs with 'Searching...'.
Might give a clue.

@nicoddemus
Copy link

Fixed pytest's doc also by pinning to Sphinx 1.4, see pytest-dev/pytest#2352. Worth noting that I also configured the path to the requirements.txt file in Settings.

@gravis
Copy link

gravis commented Apr 19, 2017

We're now having this issue too:
http://docs.gemnasium.net/en/latest/search.html?q=upgrade&check_keywords=yes&area=default
=> broken links (404)

@nicoddemus
Copy link

@gravis see my comment above yours for a possible workaround.

gravis pushed a commit to gemnasium/docs that referenced this issue Apr 19, 2017
@gravis
Copy link

gravis commented Apr 19, 2017

@nicoddemus Thanks!
I confirm this workaround is working for us :)

akien-mga added a commit to godotengine/godot-docs that referenced this issue Apr 19, 2017
akien-mga added a commit to godotengine/godot-docs that referenced this issue Apr 19, 2017
agjohnson added a commit to readthedocs/readthedocs-sphinx-ext that referenced this issue May 3, 2017
Combine both into a single template for now. This isn't ideal, but the
differences were minor.

Future changes here might be to instead patch the file as we're copying,
stealing it from Sphinx package distribution instead. We only need to remove
`Search.init()` from the file, so perhaps this isn't horribly difficult.

Fixes #25
Refs readthedocs/readthedocs.org#2708
agjohnson added a commit to readthedocs/readthedocs-sphinx-ext that referenced this issue May 3, 2017
This takes the underlying `searchtools.js_t` out of the Sphinx distribution
path, patches it, and then fills the context and parses it as a templated
javascript file. This allows us to remove the script initialization on all
versions of this file.

The initialization block on `searchtools.js_t` has not changed in 10 years, so
this method should be safe. If the block changes in the future, tests will grab
this as we add new versions of Sphinx to our testing.

This reorganizes some repetitive code and cleans up a few other pieces as well:

File copying is linked to the standard `copy_static_files` that is run from
the builder. Running from an event on build finished was producing files without
the template context. This is because `build-finished` is always triggered, but
the static files are not always copied (and therefore did not have the same
template context?)

Fixes #25
Refs readthedocs/readthedocs.org#2708
agjohnson added a commit to readthedocs/readthedocs-sphinx-ext that referenced this issue May 3, 2017
This takes the underlying searchtools.js_t out of the Sphinx distribution
path, patches it, and then fills the context and parses it as a templated
javascript file. This allows us to remove the script initialization on all
versions of this file.

The initialization block on searchtools.js_t has not changed in 10 years, so
this method should be safe. If the block changes in the future, tests will grab
this as we add new versions of Sphinx to our testing.

This reorganizes some repetitive code and cleans up a few other pieces as well:

File copying is linked to the standard copy_static_files that is run from
the builder. Running from an event on build finished was producing files without
the template context. This is because build-finished is always triggered, but
the static files are not always copied (and therefore did not have the same
template context?)

Fixes #25
Refs readthedocs/readthedocs.org#2708
@ekpgh
Copy link

ekpgh commented May 3, 2017

Related to this bug in Sphinx 1.5: sphinx-doc/sphinx#3696

@slinkp
Copy link

slinkp commented May 4, 2017

Is there any workaround other than downgrading sphinx to 1.4.x? I have a (private) project that's affected by this but also relies on the graphviz directive align option which was added in 1.5 :(

@slinkp
Copy link

slinkp commented May 4, 2017

Has anyone else observed that (javascript) search results work fine in Sphinx 1.5 when built locally, but exhibit the bad path problem (foo.rst.html) only when built on readthedocs.org ?

@veenamj
Copy link

veenamj commented May 9, 2017

I have the same issue. I want to use Sphinx 1.5 but I want to get rid of this search broken link error (foo.rst.html).

Can someone let me know if there is any other workaround other than downgrading Sphinx version.

@nhitchins
Copy link

Still having same problem with docs failing to build via RTD when downgrading to Sphinx 1.4.x (builds fine locally). https://readthedocs.org/projects/eagleio/builds/5258802/

@agjohnson
Copy link
Contributor

This will be resolved with a release of our extension, readthedocs/readthedocs-sphinx-ext#26 solves the issues here around Sphinx 1.5 using an incompatible version searchtools.js from Sphinx 1.4. I should have that released and deployed this week.

@nicoddemus
Copy link

@agjohnson that's great news, thanks for working on this! 👍

agjohnson added a commit that referenced this issue May 10, 2017
Addresses #2708 by using a version of our extension that supports more than one
version of Sphinx.

Refs #1895
Refs #1850
Refs #2755
agjohnson added a commit that referenced this issue May 11, 2017
Addresses #2708 by using a version of our extension that supports more than one
version of Sphinx.

Refs #1895
Refs #1850
Refs #2755
@agjohnson
Copy link
Contributor

So our extension is now released and Read the Docs uses the new release by default. To test these changes, you'll likely need to wipe the version from your project's versions page to ensure you get the new static files. I'll test against our docs (and the original bug report project) later, but if anyone else gets a chance to try this, feel free to report back

@smrgit
Copy link

smrgit commented May 12, 2017

works nicely for me, thank you! 👍

actually, correction -- it works when I search for a new two word string, but when I search for the phrase that I originally reported as causing me problems ("imaging data"), I still get the .rst.html links ... but if I search for "imaging archive", then I get clean links

(and I did try a "wipe" and "rebuild" after I first noticed the recurring error and even after that, I still get the error when searching for "imaging data") -- although perhaps that is because of my browser cache?

@stevepiercy
Copy link
Contributor

...and without doing anything except allowing time to pass, latest now works, too.
http://docs.pylonsproject.org/projects/pyramid/en/latest/search.html?q=traversal

Thank you!

@agjohnson
Copy link
Contributor

Given we only cache build data for ~24h, projects should slowly start magically working. Poking various projects above, it looks like the search results are now correct. I'll close this for now, but feel free to reopen if this didn't help your project and you are now using Sphinx 1.5.

@dlech
Copy link

dlech commented May 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug A bug
Projects
None yet
Development

No branches or pull requests