Add https://search.xcloud.live

[techcow2]

Requirements (make sure to read all of them)

• It is my instance. I bought the domain myself and I own this domain. Free domains (e.g. Freenom) and shared domains (e.g. noip.com) are not allowed.
• I give the right to check.searx.space to check my instance (every 3 hours for the response times, every 24 hours for the other tests).
• I acknowledge that managing a public instance is not an easy task and require spending time to keep the instance in good health. E.g. look after your instance by using a monitoring system.
• I guarantee to keep an uptime per month of my instance at minimum 90%. Please ask for a removal of your instance if there is a planned long downtime or notify us here for a short downtime.
• I do not track the users of my instance with any analytics or tracking software.
• I won't try to manipulate the ranking of my instance in a way that give an unfair advantage over the other public instances in the list. (e.g. caching requests for searx.space server)
• I control the final webserver (software) that is serving the requests to the users of my instance. Here is a non-exhaustive list of forbidden hosting types: PaaS, managed (hosting provider controlled) HTTP(S) load balancer (e.g. AWS ALB), Cloudflare, shared Web hosting. TCP load balancer is fine.
• If needed, I can restrict users from accessing my instance for the only sole reason of keeping my instance in working conditions for the other users (detailed description - evidence need to be provided when asked). Other means of restriction is forbidden.

Bot protection

Yes, I have enabled the limiter plugin built in SearXNG.

Source code URL

No response

Comment

No response

[techcow2]

Please note - I will be resetting my server and installing a different version of Ubuntu so it utilizes less RAM. Due to this, my instance will be down for about a half hour today (8:30 PM EST - 9:00 PM EST).

Thank you.

[techcow2]

Trying to be proactive with this process...

I'm going to need some help setting up limiter. Initially, I had assumed docker took care of this as it's enabled in my settings.yml file by default. However, I see many bot requests on the syslog and auth.log. I've gone through the documentation here (https://docs.searxng.org/src/searx.botdetection.html#id5) and please forgive my ignorance. I still don't understand how to properly set up limiter. I'm still learning servers and Linux and I am quite a beginner. From what I understand, I'll need to create a limiter text file and add specific commands/parameters to the file. I know I'm also supposed to add to the settings.yml file, but I'm not exactly sure what to add and the correct way to do it.

This is supposed to be a fun project for me that I plan on maintaining and taking seriously. Hopefully, those with more knowledge than me can provide some easier-to-understand learning material than what I've found. I've gotten this far, so I don't think it's over my head, I just need someone to get me started in the right direction.

Another thing I'll need help with is allowing 'check.searx.space' to automatically check my instance. I also thought the docker image took care of this but I think I'm incorrect.

Thank you.

[unixfox]

How did you install searxng? The limiter does not block all the bots (yet).

[techcow2]

How did you install searxng? The limiter does not block all the bots (yet).

Thanks for the quick reply! I installed searxng using docker/docker-compose on my VPS. (git clone https://github.com/searxng/searxng-docker.git).

[techcow2]

I would like to add that I see this error repeating in the Searxng logs (docker-compose logs -f) " 2023-07-18 15:10:20,892 ERROR:searx.botdetection: X-Real-IP header is not set!" I'm not sure if this is normal or if I need to act on this.

There is also this error: "2023-07-18 16:24:44,493 WARNING:searx.tools.config: missing config file: /etc/searxng/limiter.toml"

Thanks!

[unixfox]

You did not touch anything right? You did not change the "principal" webserver?

[techcow2]

You did not touch anything right? You did not change the "principal" webserver?

I did not change the "principal" webserver. (I wouldn't know how). I didn't change anything besides adding moderate safe search, default autocomplete as DuckDuckGo, and switching from pull to GET requests all in the settings.yaml file which I will paste below. I also turned on ufw to allow ports and also iptables with the commands below:

ufw and iptable changes:

sudo iptables -I INPUT -m state --state NEW -p tcp --dport 80 -j ACCEPT
sudo iptables -I INPUT -m state --state NEW -p tcp --dport 443 -j ACCEPT
sudo ufw allow 80
sudo ufw allow 443

Here is my settings.yaml file:

# see https://docs.searxng.org/admin/engines/settings.html#use-default-settings
use_default_settings: true
server:
  # base_url is defined in the SEARXNG_BASE_URL environment variable, see .env and docker-compose.yml
  secret_key: "REDACTED FOR THIS POST"  # change this!
  limiter: true  # can be disabled for a private instance
  image_proxy: true
  method: "GET"
ui:
  static_use_hash: true
redis:
  url: redis://redis:6379/0
general:
  debug: false
  instance_name: "PrivCrawler.com - Private Search"
  contact_url: false
  enable_metrics: true
search:
  safe_search: 1
  autocomplete: "duckduckgo"

Sorry if this information is irrelevant, just trying my best to be as detailed as possible.

[unixfox]

You should be good to go then, everything is fine.

Again the limiter can't block all the bots yet.

This may change in the future: searxng/searxng#2357

[techcow2]

You should be good to go then, everything is fine.

Again the limiter can't block all the bots yet.

This may change in the future: searxng/searxng#2357

Great! Thanks for confirming that.

[techcow2]

I've decided to scrap the "privcrawler" name and go with something that makes more sense and one that will fit for my other self hosted apps. I'm going to close this issue and reopen another one when I decide on something else.

[unixfox]

please don't reopen another one, just give us the new domain here and we will edit the issue.
this is to keep the history of the discussion with you

[techcow2]

Okay, will do.

[techcow2]

@unixfox - please change my request/issue to use the new domain below instead of "privcrawler". Thank you.

https://search.xcloud.live