Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for SearXNG as a websearch source #805

Merged
merged 11 commits into from
Feb 14, 2024

Conversation

Karlostavitch1
Copy link
Contributor

@Karlostavitch1 Karlostavitch1 commented Feb 9, 2024

SearXNG is an open-source, privacy-centric search aggregator. With this single integration, it provides access to over 50 search engines with results in JSON format (if enabled on the instance). There are public instances of SearXNG and it is easily self-hosted.

I note that there has been some discussion already about this feature.

This is a feature that's been very helpful for me and I hope you find it worthwhile for the wider project.

Created searchSearxng.ts for querying and parsing searxng instance.
Added SearXNG to the searchWeb.ts to ensure inclusion in the search initialisation script.
Added SearXNG to +layout.server.ts to ensure the search button is enabled if the URL provided in .env.local assignment
Updated .env to include SearXNG configuration
Updated README.md under Web Search Config to mention the addition of SearXNG URL use in .env

Karlostavitch1 and others added 3 commits February 9, 2024 16:43
Added searxng to the searchWeb.ts to ensure inclusion in the search initialisation script.
Added searxng to +layout.server.ts to ensure the search button is enabled if url provided in .env.local assignment
Updated .env to include searxng configuration
Updated README.md under Web Search Config to mention addition of searxng URL use in .env
Copy link
Collaborator

@nsarrazin nsarrazin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments, looks good to me otherwise! Let me know what you think.

.env Outdated Show resolved Hide resolved
.env Outdated Show resolved Hide resolved
Karlostavitch1 and others added 3 commits February 12, 2024 16:24
consolidated env variable to SEARXNG_QUERY_URL (felt considering the content the word QUERY was more relevant)
refactored the searchSearxng.ts to generate the query with the new variable
updated related references to variable (.env, searchWeb.ts and +layout.server.ts)
Updated readme and variable comment on .env to reflect the same
@Karlostavitch1
Copy link
Contributor Author

Hi nsarrazin,

I've attempted to accommodate your requests, please make any minor edits you wish or let me know if there are any issues.

Thanks,

@nsarrazin
Copy link
Collaborator

Hey, thank you for the changes, I think it's very nice like this! I'll fix the linting and expand a bit the readme section to explain the <query>, but this is great!

@Karlostavitch1
Copy link
Contributor Author

not sure if its worth doing a PR for this next update since you started checking already...

I had a bit of a panic when my engine config wasn't coming through to the results and thought that perhaps my update was failing over to local Google somehow. Anyway, it was an issue with the query I entered in the env, but while digging around I figured out how to update the search info to reflect the search provider.

searchdialoge

It was an update to 2 files, searchWeb.ts and WebSearch.ts so that it accurately indicates the search engine being used. Currently, it only displays You and Google. My update only adds SearXNG but I could also add Serper, SerpAPI and SerpStack if desirable (they may have been missed with their PR's).

@nsarrazin
Copy link
Collaborator

Feel free to push to this branch for SearXNG! For Serper and Serpapi, I think it was on purpose since these only search google. For SearXNG it's different I think so it makes sense in that case.

@Karlostavitch1
Copy link
Contributor Author

was this a transient issue (docker auth) or do you need me to debug it?

Copy link
Collaborator

@nsarrazin nsarrazin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very nice, thanks for the contribution!

@nsarrazin nsarrazin merged commit 7c22da3 into huggingface:main Feb 14, 2024
3 checks passed
ice91 pushed a commit to ice91/chat-ui that referenced this pull request Oct 30, 2024
* Created searchSearxng.ts for querying and parsing searxng instance.
Added searxng to the searchWeb.ts to ensure inclusion in the search initialisation script.
Added searxng to +layout.server.ts to ensure the search button is enabled if url provided in .env.local assignment
Updated .env to include searxng configuration
Updated README.md under Web Search Config to mention addition of searxng URL use in .env

* Changes requested
consolidated env variable to SEARXNG_QUERY_URL (felt considering the content the word QUERY was more relevant)
refactored the searchSearxng.ts to generate the query with the new variable
updated related references to variable (.env, searchWeb.ts and +layout.server.ts)
Updated readme and variable comment on .env to reflect the same

* urefectored to display SearXNG in search dialoge when used.

* lint

* types

---------

Co-authored-by: Nathan Sarrazin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants