SearX - Google > reCaptcha

Been using the Disroot instance of SearX for a few weeks now, last night I noticed (on phone inside home wifi network) that the search results page displayed an error from Google search indicating reCaptcha required. Today from desktop or phone, no error, but no Google results in any searches. Can’t replicate the error message now to share screencap. Izzit just me?

Can’t see issue mentioned on the SearX issues page https://github.com/asciimoo/searx/issues?utf8=✓&q=is%3Aissue+is%3Aopen+recaptcha

Just flagging this up really as I doubt it’s anything Disroot can do, most likely Google is blocking SearX. Bing is clearly not gonna be my search engine choice any time soon is all I can say for now! Although > https://github.com/asciimoo/searx/issues/1275

Yes we are aware of this, as it did not happen the first time :stuck_out_tongue:
Every now and then google blocks our IP. What we usually do (like in the case now) is disable google search results for a while until google unblocks us (kind of cooldown period).

usually in that case we update our status page at: https://state.disroot.org/
https://state.disroot.org/incident/125

Generally state.disroot.org is a good place to check whats going on and if we are aware of the issue.

For us not having fully functional searx is like loosing a hand. We are searx fanatics and see right away when things are not working right. We’ve enabled google search results earlier today but got blocked almost instantly. We will re-enable them in an hour or two.

Keep your fingers crossed…

Thanks for the reply, much as I would like to quit using Google as much as is possible, seems they are still one of the best search engines (used to be some good alternatives…Kartoo was great and a bunch of others now forgotten).
I fear that if Google results will regularly be blocked on your instance IP, I will have to use one of the others until a more solid solution can be implemented.

Trying another instance indicates that it may not just be issue for individual SearX instances…

What usually happens is that some bots are sending a lot of queries, searching for people, music, just general stuff. I guess it is done to alter the positioning. We use filtron which is developed by searx folks to block as much of those as possible. This has improved the situation a lot. In fact for months we did not have google blocking issue. This time however bots managed to break through the sieve and therefor caused blockage. I have manually enabled access logs to determine the IP address of bots and I blocked them manually. This should fix it for now but we need more permanent solution. I’ve given a shoutout to searx devs on IRC. In the meantime I wait an hour or so for cooldown and will re-enable google the moment we are not blocked.

I hope this will fix it for now.

1 Like

Free autonomous services such as Disroot, SearX etc. are great, abusers of these services are not, essentially they are a threat so must be dealt with accordingly. I’m wondering about an open source non tracking alternative to reCaptcha or better way to automate bot filtering…no simple task…anyone know much about using AI?!
I appreciate the feedback on this matter, if there’s any way we can help, just shout!

[Edit] Just an idea, can search rate be limited? Most of us will only fire a few queries in any short period of time? I’m thinking Fail2Ban style, not sure it can be used for this however? For me I guess three searches per min would be OK. Can always jump to another instance of SearX . Another idea: randomly use any available instance of SearX (this I guess is a new idea not yet implemented and would not suit all use cases)