Skip to main content

[ RESOLVED ]

It was the User Agent string in my #Qutebrowser config.py that was causing issues only with #DuckDuckGo search results.

Is it just me or this happening elsewhere on #DuckDuckGo ?
If I go to start.duckduckgo.com and then enter a search say #FreeBSD . If I then click on a result any of them I get the Nginx screen with 400 Bad Request.
Google works fine and other sites it's just #DDG

This entry was edited (2 weeks ago)

Michael Dexter reshared this.

Justine Smithies
Justine Smithies  
•
Hmm really strange. I sopped all my jails incase one started causing an issue and also rebooted my laptop and router. But it is still the same. I can search and get results on DDG but if i click on result I get the 400 Bad request and nginx underneath it ??? All other sites are working just fine.
Paul Wilde :dontpanic2: :smeghead:
Paul Wilde :dontpanic2: :smeghead:  
•
Fine for me here.
Perhaps DDG is giving you dodgy links - does it work better in Private mode (i.e. cleared cache) or on a different web browser?
Justine Smithies
Justine Smithies  
•

Happens in firefox too on my mobile. I'm using Qutebrowser on my laptop.
If I search FreeBSD and then click on the FreeBSD Project URL which is freebsd.org I get the 400 bad request but the link showing is:

https://duckduckgo.com/l/?uddg=https%3A%2F%2Fwww.freebsd.org%2F&notrut=duckduck_in

social elephant in the room
social elephant in the room  
•
cannot reproduce
social elephant in the room
social elephant in the room  
•
could it be that you have a dns setting configured to search on your local domain and that you have a box called freebsd?
Justine Smithies
Justine Smithies  
•

Nope no box named freebsd on my network. It doesn't just happen with that search term it can be any search term. I tried and get the same results.

For example I just searched Gargening and clicked on gardenersworld.com but get the same 400 bad request. The link I copied from that result is :

https://duckduckgo.com/l/?uddg=https%3A%2F%2Fwww.gardenersworld.com%2F&notrut=duckduck_in

If I do the same from bing or google or whatever it all works. This just affects #DuckDuckGo for some reason ???

#duckduckgo
This entry was edited (2 weeks ago)
social elephant in the room
social elephant in the room  
•

eerrr, this is odd...

and if you add this directly :

https://duckduckgo.com/?t=ftsa&q=FreeBSD&ia=web

social elephant in the room
social elephant in the room  
•
dig +short duckduckgo.com in a terminal?
Justine Smithies
Justine Smithies  
•
Even stranger if i click on any of the Report Ad links they work just not the normal none advert URLS provided by the duckduckgo search engine.
Justine Smithies
Justine Smithies  
•
╭─justine@justine@mastodon.sdf.org-laptop ~
╰> dig +short duckduckgo.com
52.142.124.215
@Justine
social elephant in the room
social elephant in the room  
•

it returns me the same ip...

what about dns settings in (I assume) firefox?

Justine Smithies
Justine Smithies  
•

Hmm hang on firefox on my mobile is working. It seems its just affecting Qutebrowser but only with DDG search requests ?? Very strange as I haven't changed any settings and it all worked yesterday. Away to check some network settings on my laptop incase an update has done something.

CC: @justine@mastodon.sdf.org

@Justine
Justine Smithies
Justine Smithies  
•

Right so it's a #Qutebrowser issue. Just installed #Firefox on this laptop and everything works.
The strange thing is on Qutebrowser I can click and view on the Report Ad URL's but if I click on any other return results I get the 400 Bad request Nginx. Using Firefox and all is good. I wonder what has happened to my trusty Qutebrowser ???

CC: @tseitr@mastodon.sdf.org @justine@mastodon.sdf.org

@social elephant in the room @Justine
silverwizard
silverwizard  

@Justine Smithies Fascinating - that link causes a 400 for me - but searching for "gardener's world" with DDG lets me go there.

What's the network tab in the firefox dev tools saying? It seems like something is wrapping your clicks and it's out of date maybe?

@Justine Smithies
Justine Smithies
Justine Smithies  
•
This is only affecting my #Qutebrowser , #Firefox works just fine.
silverwizard
silverwizard  

@Justine Smithies Oh I'd misread - sorry.

That's wilder.

That sounds like there's a redirect that is getting eaten by escaping somewhere. Is there any debug output?

(I tried qute recently... but it was too slow for my... not a sensible computer)

@Justine Smithies
Justine Smithies
Justine Smithies  
•
Not sure I'm investing things just now as this is really weird.
Justine Smithies
Justine Smithies  
•

[ UPDATE ]
Finally I have tracked down my #Qutebrowser issue that only seemed to mess with #DuckDuckGo search results returning an 400 Bad Request
It seems in my config.py I had the following user agent line.

c.content.headers.user_agent = 'Mozilla/5.0 ({os_info}) QtWebEngine/{qt_version} qutebrowser/{qutebrowser_version} Chromium/67.0.1'

I changed it to the same as #Firefox was using and bingo it all works !

c.content.headers.user_agent = 'Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0'

Who's a happy bunny now.

I have now changed the original user agent to the following and all seems good.

c.content.headers.user_agent = 'Mozilla/5.0 ({os_info}) QtWebEngine/{qt_version} qutebrowser/{qutebrowser_version} Chrome/130.0.0.0'

CC: @silverwizard@convenient.email

@silverwizard
This entry was edited (2 weeks ago)
Justine Smithies
Justine Smithies  
•
Strange I might reboot my router just in case ?
Justine Smithies
Justine Smithies  
•

There was but that was on the 8th and it was working until yesterday or today ??

CC: @tseitr@mastodon.sdf.org @justine@mastodon.sdf.org

@social elephant in the room @Justine
Justine Smithies
Justine Smithies  
•

Don't know just going through all the settings and there's loads ?

CC: @tseitr@mastodon.sdf.org @justine@mastodon.sdf.org

@social elephant in the room @Justine
Justine Smithies
Justine Smithies  
•
I cured my issue by updating the user agent string in Qutebrowser.
⇧