Markus Heiser 9197efa2a7 [fix] duckduckgo lite engine: set HTTP header 'Referer'
We have had problems with this before, the bot protection from ddg-lite seems to
have included this referer in the rating [1][2].

From reverse engineering:

- The Referer ``https://google.com/`` was set in commt 257dc7d6c4 --> DDG lite
  does not like this referer anymore!

- The 'Referer' header is only set on second and follow up pages but not on the
  first page

- The vqd value is not needed on the first page, the ddg-lite client sets this
  value only on follow up pages / this can help to reduce the vqd requests from
  SearXNG.

Related to 'Referer' header & ddg requests:

[1] https://github.com/searxng/searxng/pull/2161
[2] https://github.com/searxng/searxng/pull/2081

Closes: https://github.com/searxng/searxng/issues/2796
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-10 08:40:53 +02:00
..
2023-09-18 16:20:27 +02:00
2023-09-18 16:20:27 +02:00
2023-09-18 16:20:27 +02:00
2023-09-25 17:45:21 +02:00
2022-07-30 18:04:44 +02:00
2023-09-18 16:20:27 +02:00
2023-09-18 16:20:27 +02:00
2023-09-18 16:20:27 +02:00
2023-09-18 16:20:27 +02:00
2023-09-18 16:20:27 +02:00