Beanz/searxng
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
searxng/searx/engines
History
Markus Heiser 9197efa2a7 [fix] duckduckgo lite engine: set HTTP header 'Referer' 
We have had problems with this before, the bot protection from ddg-lite seems to
have included this referer in the rating [1][2].

From reverse engineering:

- The Referer ``https://google.com/`` was set in commt 257dc7d6c4 --> DDG lite
  does not like this referer anymore!

- The 'Referer' header is only set on second and follow up pages but not on the
  first page

- The vqd value is not needed on the first page, the ddg-lite client sets this
  value only on follow up pages / this can help to reduce the vqd requests from
  SearXNG.

Related to 'Referer' header & ddg requests:

[1] https://github.com/searxng/searxng/pull/2161
[2] https://github.com/searxng/searxng/pull/2081

Closes: https://github.com/searxng/searxng/issues/2796
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-10 08:40:53 +02:00
..
__init__.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
9gag.py
get the not cropped version of the thumbnail when the image height is not too important
2022-08-24 18:33:11 +07:00
1337x.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
ahmia.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
annas_archive.py
[fix] engine annas archive - fetch traits (modified xpath selectors)
2023-09-27 09:12:38 +02:00
apkmirror.py
rollback test
2023-03-15 19:55:20 +01:00
apple_app_store.py
remove thumbnail from results
2022-08-27 06:23:30 +07:00
apple_maps.py
add poi's website and phone number, doesn't crash when there is no displayMapRegion, query the token on the first request
2022-08-27 06:17:58 +07:00
archlinux.py
[fix] spelling
2023-09-18 16:20:27 +02:00
artic.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
arxiv.py
Science category: update the engines
2022-09-23 20:45:58 +02:00
bandcamp.py
[mod] bandcamp & genius: in result set img_src instead thumbnail
2022-02-21 22:12:07 +01:00
base.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
bilibili.py
[fix] bilibili new api used
2023-09-28 08:24:51 +02:00
bing_images.py
[fix] Revision of the Bing engines
2023-10-01 08:01:38 +02:00
bing_news.py
[fix] Revision of the Bing engines
2023-10-01 08:01:38 +02:00
bing_videos.py
[fix] Revision of the Bing engines
2023-10-01 08:01:38 +02:00
bing.py
[fix] Revision of the Bing engines
2023-10-01 08:01:38 +02:00
brave.py
[fix] engine brave - fetch traits (modified settings menu)
2023-09-27 09:12:38 +02:00
bt4g.py
[fix] spelling
2023-09-18 16:20:27 +02:00
btdigg.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
chefkoch.py
[feat] engine: implementation of chefkoch.de
2023-09-21 17:23:59 +02:00
command.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
core.py
[fix] doc of the paper.html template (isbn, issn)
2022-09-25 15:46:29 +02:00
crossref.py
[fix] engine - Crossref
2023-09-14 17:39:23 +02:00
currency_convert.py
[fix] engine - currency fix and simplify
2023-09-28 08:29:38 +02:00
dailymotion.py
[fix] engine & network issues / documentation and type annotations
2023-06-25 13:58:26 +02:00
deepl.py
[mod] add deepl translation engine
2022-08-10 09:14:36 +02:00
deezer.py
[mod] templates: rename field for <iframe> URL to iframe_src
2022-02-18 19:00:49 +01:00
demo_offline.py
[mod] replace engines_languages.json by engines_traits.json
2023-03-24 10:37:42 +01:00
demo_online.py
[mod] add 'Accept-Language' HTTP header to online processores
2022-08-01 17:01:59 +02:00
deviantart.py
[fix] engine deviantart: review of the result-scrapper
2023-09-11 13:22:36 +02:00
dictzone.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
digbt.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
docker_hub.py
[mod] make python code pylint 2.16.1 compliant
2023-02-10 13:59:21 +01:00
doku.py
[fix] issues reported by pylint
2021-12-27 10:16:20 +01:00
duckduckgo_definitions.py
[mod] engine duckduckgo definitions: support for answer source
2023-08-18 19:07:14 +02:00
duckduckgo_extra.py
[feat] duckduckgo: support for videos and news
2023-10-09 06:53:43 +02:00
duckduckgo_weather.py
[fix] engine duckduckgo weather api changes
2023-09-05 16:55:00 +02:00
duckduckgo.py
[fix] duckduckgo lite engine: set HTTP header 'Referer'
2023-10-10 08:40:53 +02:00
duden.py
[fix] engine duden - don't raise exception on empty result list
2022-08-20 08:41:03 +02:00
dummy-offline.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
dummy.py
[enh] engines: add about variable
2021-01-14 20:57:17 +01:00
ebay.py
[mod] Pass desired ebay domain in settings
2022-04-16 19:10:35 +02:00
elasticsearch.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
emojipedia.py
[fix] emojipedia: fix engine
2023-10-01 08:19:45 +02:00
fdroid.py
[enh] add more categories
2022-01-05 11:00:11 +01:00
flickr_noapi.py
[fix] engine flickr: adapt to the new data model from flicker's response
2023-03-30 21:04:53 +02:00
flickr.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
framalibre.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
freesound.py
[mod] result_templates/default.html replace embedded HTML by data_src audio_src
2022-02-13 14:20:47 +01:00
frinkiac.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
genius.py
[mod] bandcamp & genius: in result set img_src instead thumbnail
2022-02-21 22:12:07 +01:00
gentoo.py
[mod] remove obsolete EngineTraits.supported_languages
2023-03-24 10:37:42 +01:00
github.py
[fix] typos / reported by @kianmeng in searx PR-3366
2022-09-27 18:32:14 +02:00
google_images.py
[fix] engine - google images error when no results
2023-09-21 16:38:37 +02:00
google_news.py
[fix] spelling
2023-09-18 16:20:27 +02:00
google_play.py
[mod] engine google_play: raise error on unsupported category
2023-08-10 12:35:24 +02:00
google_scholar.py
[mod] Google: reversed engineered & upgrade to data_type: traits_v1
2023-03-24 10:37:42 +01:00
google_videos.py
[fix] engine google_video: google has changed the layout of the rsponse
2023-08-22 08:34:04 +02:00
google.py
[fix] spelling
2023-09-18 16:20:27 +02:00
hackernews.py
[feat] implement hackernews engine - news.ycombinator.com
2023-10-09 14:00:04 +02:00
imdb.py
[mod] add movies category for tmdb, imdb and moviepilot
2023-09-29 22:37:51 +02:00
imgur.py
[feat] engine: implementation of imgur
2023-09-22 20:50:53 +02:00
ina.py
[fix] ina engine
2022-01-28 22:33:41 +01:00
internet_archive_scholar.py
[fix] internet archive scholar: crash when there's no title
2023-09-22 18:49:39 +02:00
invidious.py
[fix] engine - invidious thumbnails
2023-09-13 11:37:42 +02:00
jisho.py
[format.python] based on bugfix in 9ed626130
2022-05-07 18:23:10 +02:00
json_engine.py
[enh] Initial no paging support for Yep.com
2022-06-11 14:17:44 +02:00
kickass.py
[fix] kickass: crash when no results
2023-10-07 11:48:23 +02:00
lemmy.py
[mod] utils.py: add markdown_to_text helper function
2023-09-08 11:47:13 +02:00
lingva.py
[mod] Adds Lingva translate engine
2022-07-04 19:06:45 +02:00
loc.py
[mod] library of congress: fix engine
2023-09-11 19:42:31 +02:00
mastodon.py
[feat] engine: implementation of mastodon
2023-10-06 10:58:23 +02:00
material_icons.py
[feat] engine: implementation of google icons/material design icons
2023-09-21 15:16:49 +02:00
mediathekviewweb.py
[fix] engine mediathekviewweb: replace http links by https
2022-03-07 19:49:16 +01:00
mediawiki.py
[mod] engine: wikimedia - improve results, add addition settings & doc
2023-08-04 19:06:50 +02:00
meilisearch.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
metacpan.py
Add MetaCPAN engine
2022-11-07 08:07:06 -06:00
mixcloud.py
[mod] add artwork to mixcloud & soundcloud engines
2022-02-19 21:59:12 +01:00
mongodb.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
moviepilot.py
[fix] engine - moviepilot instead of thumbnail use img_src
2023-09-30 11:29:19 +02:00
mrs.py
[fix] matrixrooms add proper MRS integration
2023-10-09 13:25:13 +02:00
mwmbl.py
[doc] add documentation of Mwmbl engine & autocompleter
2023-08-27 17:25:26 +02:00
mysql_server.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
nyaa.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
odysee.py
[fix] spelling
2023-09-18 16:20:27 +02:00
opensemantic.py
[enh] engines: add about variable
2021-01-14 20:57:17 +01:00
openstreetmap.py
[fix] spelling
2023-09-18 16:20:27 +02:00
openverse.py
[fix] ccengine engine - avoid unwanted redirects
2022-01-07 14:14:31 +01:00
pdbe.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
peertube.py
[fix] engine & network issues / documentation and type annotations
2023-06-25 13:58:26 +02:00
photon.py
[fix] typos / reported by @kianmeng in searx PR-3366
2022-09-27 18:32:14 +02:00
pinterest.py
[feat] engine: implementation of pinterest
2023-09-30 15:01:45 +02:00
piped.py
[mod] piped: always show video length if available
2023-10-07 11:45:46 +02:00
piratebay.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
postgresql.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
pubmed.py
[fix] pubmed content being None
2023-09-14 18:40:15 +02:00
qwant.py
[fix] engine qwant (web-lite) - ignore advertising adds
2023-09-19 17:06:56 +02:00
radio_browser.py
[mod] engine - simplify region & lang handling, make filters configurable
2023-10-05 10:55:08 +02:00
recoll.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
reddit.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
redis_server.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
rumble.py
[fix] rumble redirect
2023-09-14 19:21:21 +02:00
scanr_structures.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
searchcode_code.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
searx_engine.py
reference docs.searxng.org
2022-01-02 21:18:29 +01:00
seekr.py
[mod] improve seekr engines and add documentation
2023-08-15 16:17:42 +02:00
semantic_scholar.py
[mod] science category: various update of about PR 1705
2022-09-23 20:52:55 +02:00
sepiasearch.py
[mod] SepiaSearch: re-engineered & upgrade to data_type: traits_v1
2023-03-24 10:37:42 +01:00
seznam.py
[fix] engine seznam: fix issues reported by black & pylint
2023-03-31 17:25:39 +02:00
sjp.py
[fix] sjp engine - convert enginename to a latin1 compliance name
2022-07-24 21:10:55 +02:00
solidtorrents.py
[fix] solidtorrents redirects
2023-09-14 18:03:21 +02:00
solr.py
[doc] rearranges Settings & Engines docs for better readability
2023-07-01 22:45:19 +02:00
soundcloud.py
[mod] add artwork to mixcloud & soundcloud engines
2022-02-19 21:59:12 +01:00
spotify.py
[mod] templates: rename field for <iframe> URL to iframe_src
2022-02-18 19:00:49 +01:00
springer.py
[fix] springer: unsupported operand type(s) for +: 'NoneType' and 'str'
2022-09-25 15:25:55 +02:00
sqlite.py
[fix] spelling
2023-09-18 16:20:27 +02:00
stackexchange.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
startpage.py
[fix] engine & network issues / documentation and type annotations
2023-06-25 13:58:26 +02:00
svgrepo.py
[feat] new engine: svgrepo
2023-09-12 20:38:36 +02:00
tagesschau.py
[mod] tagesschau: add option to only use tagesschau urls
2023-09-30 11:00:11 +02:00
tineye.py
[fix] engine tineye: handle 422 response of not supported img format
2022-07-23 16:00:58 +02:00
tokyotoshokan.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
torznab.py
[fix] spelling
2023-09-18 16:20:27 +02:00
translated.py
[enh] move dictionaries, Erowid & IMDb out of general category
2022-01-05 11:03:44 +01:00
unsplash.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
vimeo.py
[mod] templates: rename field for <iframe> URL to iframe_src
2022-02-18 19:00:49 +01:00
wallhaven.py
[feat] engine: implementation of wallhaven
2023-09-21 14:25:43 +02:00
wikicommons.py
[fix] engine: wikicommons - don't quoute ':|' in URL parameters
2023-09-03 18:45:39 +02:00
wikidata.py
wikipedia wikidata infobox + disable wikisource (#2806)
2023-09-19 10:31:02 +02:00
wikipedia.py
wikipedia wikidata infobox + disable wikisource (#2806)
2023-09-19 10:31:02 +02:00
wolframalpha_api.py
[fix] typos / reported by @kianmeng in searx PR-3366
2022-09-27 18:32:14 +02:00
wolframalpha_noapi.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
wordnik.py
[format.python] initial formatting of the python code
2021-12-27 09:26:22 +01:00
wttr.py
[mod] clarify the difference of the default category and subgrouping
2023-04-07 11:03:25 +02:00
www1x.py
[fix] 1x engine
2022-01-30 19:48:40 +01:00
xpath.py
[fix] spelling
2023-09-18 16:20:27 +02:00
yacy.py
[mod] yacy: use official instance by default and fix crashes
2023-10-09 20:50:24 +02:00
yahoo_news.py
[fix] issues reported by pylint
2021-12-27 10:16:20 +01:00
yahoo.py
[typo] solved a typo in yahoo error message.
2023-10-01 08:29:06 +02:00
youtube_api.py
[mod] templates: rename field for <iframe> URL to iframe_src
2022-02-18 19:00:49 +01:00
youtube_noapi.py
[fix] google & youtube - set EU consent cookie
2022-07-25 13:27:06 +02:00
yummly.py
[feat] engine: implementation of Yummly
2023-09-08 11:47:13 +02:00
zlibrary.py
[feat] engine: re-enables z-library (zlibrary-global.se)
2023-07-07 21:36:51 +02:00