Beanz/searxng
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
8,550 Commits 4 Branches 0 Tags
Commit Graph

8550 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Tommaso Colella
e6467bce7c [fix] docs: better phrasing for many sections of contribution_guide.rst 2025-04-17 16:47:52 +02:00
Tommaso Colella
8ca2bbc4e9 [fix] docs: typos in dev contrib guide "it's might be because 
of the tool" -> "it might be because the tool"
 2025-04-17 16:47:52 +02:00
Tommaso Colella
a74593419f [fix] docs: typo in templates dev doc developerat -> developer at 2025-04-17 16:47:52 +02:00
Zhijie He
f94802f2d2 [feat] engines: add Hugging Face engine 2025-04-17 16:43:32 +02:00
Tommaso Colella
d1c584b961 [feat] engine: add engine for italian press agency ansa 2025-04-17 15:33:57 +02:00
Markus Heiser
81f3d15665 [fix] settings.yml files: doc & obsolete settings in the template 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-04-17 13:31:10 +02:00
Tommaso Colella
99ad69450d [fix] Result type: remove rstrip "/" form url normalization 2025-04-17 10:24:05 +02:00
RobinFrcd
087da66565 [feat] add SensCritique (FR) engine 
Closes: https://github.com/searxng/searxng/issues/4623
 2025-04-17 10:19:22 +02:00
Markus Heiser
b84ae39978 [fix] doc of setting server.base_url, env is SEARXNG_BASE_URL 
Related:

- https://github.com/searxng/searxng/issues/4634

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-04-16 14:54:46 +02:00
Tommaso Colella
391bb1268d [feat] engine: add microsoft learn engine 2025-04-12 11:14:13 +02:00
dependabot[bot]
2287a6826a Bump vite from 6.2.5 to 6.2.6 in /client/simple 
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.2.5 to 6.2.6.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.2.6/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.2.6/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.2.6
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-11 18:23:56 +02:00
dependabot[bot]
b3971f89e2 [upd] web-client (simple): Bump less in /client/simple 
Bumps [less](https://github.com/less/less.js) from 4.2.2 to 4.3.0.
- [Release notes](https://github.com/less/less.js/releases)
- [Changelog](https://github.com/less/less.js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/less/less.js/compare/v4.2.2...v4.3.0)

---
updated-dependencies:
- dependency-name: less
  dependency-version: 4.3.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-11 16:02:15 +02:00
Markus Heiser
a4e08a5c2f [upd] web-client (simple): Bump eslint, sharp, webpack 
Closes: https://github.com/searxng/searxng/pull/4618
Closes: https://github.com/searxng/searxng/pull/4617
Closes: https://github.com/searxng/searxng/pull/4616
Closes: https://github.com/searxng/searxng/pull/4615

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-04-11 15:57:01 +02:00
grasdk
8ee51cc0f3 [fix] engine dokuwiki: basedir duplication 
Dokuwiki searches behind reverse proxy had duplicate base path in the url,
creating a wrong url.

This patch exchanges string concat of URLs with urljoin [1] from urllib.parse.  This
eliminates the dual problem, while retaining the old functionality designed to
concatenate protocol, hostname and port (as base_url) with path.

[1] https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urljoin

Closes: https://github.com/searxng/searxng/issues/4598
 2025-04-11 09:47:25 +02:00
dependabot[bot]
ed2953c59f [upd] pypi: Bump lxml from 5.3.1 to 5.3.2 
Bumps [lxml](https://github.com/lxml/lxml) from 5.3.1 to 5.3.2.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-5.3.1...lxml-5.3.2)

---
updated-dependencies:
- dependency-name: lxml
  dependency-version: 5.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-11 09:25:19 +02:00
dependabot[bot]
e2fac7f306 [upd] pypi: Bump selenium from 4.30.0 to 4.31.0 
Bumps [selenium](https://github.com/SeleniumHQ/Selenium) from 4.30.0 to 4.31.0.
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-4.30.0...selenium-4.31.0)

---
updated-dependencies:
- dependency-name: selenium
  dependency-version: 4.31.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-11 09:23:45 +02:00
dependabot[bot]
8db2a15307 [upd] pypi: Bump sphinx-issues from 5.0.0 to 5.0.1 
Bumps [sphinx-issues](https://github.com/sloria/sphinx-issues) from 5.0.0 to 5.0.1.
- [Commits](https://github.com/sloria/sphinx-issues/compare/5.0.0...5.0.1)

---
updated-dependencies:
- dependency-name: sphinx-issues
  dependency-version: 5.0.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-11 09:23:30 +02:00
searxng-bot
92b8f71780 [l10n] update translations from Weblate 
63157cd1f - 2025-04-10 - pfaca <pfaca@noreply.codeberg.org>
e50166782 - 2025-04-10 - return42 <return42@noreply.codeberg.org>
f7ed39011 - 2025-04-09 - Kita Ikuyo <kitakita@disroot.org>
c8786066f - 2025-04-08 - return42 <return42@noreply.codeberg.org>
dd4adf136 - 2025-04-08 - return42 <return42@noreply.codeberg.org>
df50952f0 - 2025-04-08 - return42 <return42@noreply.codeberg.org>
d9a9a0bad - 2025-04-06 - eatyourglory <eatyourglory@noreply.codeberg.org>
 2025-04-11 09:21:14 +02:00
Markus Heiser
15384e8fc5 [fix] make docs - ERROR: Unknown target name: "auth_key" 
BTW: fix a bug with sys.path: repo-root (not util) needs to added to generate
autodoc from scripts in ./searxng_extra

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-04-09 17:28:18 +02:00
Markus Heiser
b146b745a7 [fix] Meilisearch engine: Authorization Token When Integrating Meilisearch 
`X-Meili-API-Key` has  been changed to `Authorization` [1]

[1] https://www.meilisearch.com/docs/reference/api/overview#authorization

Suggested-by: https://github.com/searxng/searxng/issues/4416#issuecomment-2781254841
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-04-07 08:44:00 +02:00
Markus Heiser
c185d07689 [build] /static 2025-04-06 19:21:07 +02:00
Markus Heiser
5a54372368 [fix] images cropped in Safari (webkit) 
- Closes: https://github.com/searxng/searxng/issues/4156

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-04-06 19:21:07 +02:00
Markus Heiser
8c8aba8cf5 [fix] engine radio browser: get servers from DNS api.radio-browser.info 
Do a DNS-lookup of 'all.api.radio-browser.info', add reverse lookup and select
randomly a URL from available servers

Closes: https://github.com/searxng/searxng/issues/4576
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-04-06 18:59:10 +02:00
Ivan Gabaldon
9424bac031
Don't hardcode uWSGI parameters (#4578) 
* Initial update

* fix

* add todo note
 2025-04-04 14:57:53 +02:00
dependabot[bot]
59228fb1db [upd] web-client (simple): Bump vite in /client/simple 
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.2.4 to 6.2.5.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.2.5/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.2.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.2.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-04 11:47:26 +02:00
searxng-bot
f630a24311 [l10n] update translations from Weblate 
faf95be5a - 2025-04-03 - sakistzimas <sakistzimas@noreply.codeberg.org>
e42efee66 - 2025-04-02 - Juno Takano <jutty@noreply.codeberg.org>
c1fbb26c4 - 2025-04-02 - return42 <return42@noreply.codeberg.org>
562d21fb9 - 2025-04-02 - ghose <ghose@noreply.codeberg.org>
1ea9e67c5 - 2025-04-01 - return42 <return42@noreply.codeberg.org>
daaa0b41e - 2025-03-31 - wags07 <wags07@noreply.codeberg.org>
52635ec5f - 2025-03-31 - return42 <return42@noreply.codeberg.org>
589ad13f2 - 2025-03-31 - return42 <return42@noreply.codeberg.org>
1f063c96e - 2025-03-31 - return42 <return42@noreply.codeberg.org>
c001e606d - 2025-03-31 - return42 <return42@noreply.codeberg.org>
9bc14f1a0 - 2025-03-31 - wags07 <wags07@noreply.codeberg.org>
6a524ecb4 - 2025-03-31 - return42 <return42@noreply.codeberg.org>
508fe79c7 - 2025-03-31 - return42 <return42@noreply.codeberg.org>
8c8db80e7 - 2025-03-31 - return42 <return42@noreply.codeberg.org>
c9f551913 - 2025-03-31 - AndersNordh <andersnordh@noreply.codeberg.org>
6c56adc71 - 2025-03-31 - artens <artens@noreply.codeberg.org>
15526190b - 2025-03-31 - Linerly <linerly@noreply.codeberg.org>
2102783df - 2025-03-31 - AndersNordh <andersnordh@noreply.codeberg.org>
fee510d78 - 2025-03-30 - kratos <makesocialfoss32@keemail.me>
d5c9b970c - 2025-03-30 - Fjuro <fjuro@noreply.codeberg.org>
94259815b - 2025-03-30 - return42 <return42@noreply.codeberg.org>
a448cf7da - 2025-03-29 - Outbreak2096 <outbreak2096@noreply.codeberg.org>
0c3ef7929 - 2025-03-29 - Priit Jõerüüt <jrtcdbrg@noreply.codeberg.org>
ce2fbffe8 - 2025-03-29 - SomeTr <sometr@noreply.codeberg.org>
 2025-04-04 10:21:19 +02:00
dependabot[bot]
4452358e87 Bump vite from 6.2.3 to 6.2.4 in /client/simple 
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.2.3 to 6.2.4.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.2.4/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.2.4/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-03 09:25:43 +02:00
Markus Heiser
7546f88b1e [build] /static 2025-04-03 09:03:35 +02:00
Markus Heiser
d4c6c9401d [fix] client/simple: add missing "swiped-events" devDependencies (import) 
In commit 1a3543d4b in PR-4284 [1] the swiped-events [2] were forgotten to be
migrated.  In this patch, the event library is added to the ``devDependencies``
and imported by::

    import "../../../node_modules/swiped-events/src/swiped-events.js

[1] https://github.com/searxng/searxng/pull/4284
[2] https://www.npmjs.com/package/swiped-events

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Reported-by: https://github.com/searxng/searxng/pull/4198#issuecomment-2770734321
 2025-04-03 09:03:35 +02:00
Markus Heiser
e6308b8167 [fix] hardening against arguments of type None, where str or dict is expected 
On a long-running server, the tracebacks below can be found (albeit rarely),
which indicate problems with NoneType where a string or another data type is
expected.

result.img_src::

    File "/usr/local/searxng/searxng-src/searx/templates/simple/result_templates/images.html", line 13, in top-level template code
      <img src="" data-src="{{ image_proxify(result.img_src) }}" alt="{{ result.title|striptags }}">{{- "" -}}
      ^
    File "/usr/local/searxng/searxng-src/searx/webapp.py", line 284, in image_proxify
      if url.startswith('//'):
         ^^^^^^^^^^^^^^
    AttributeError: 'NoneType' object has no attribute 'startswith'

result.content::

    File "/usr/local/searxng/searxng-src/searx/result_types/_base.py", line 105, in _normalize_text_fields
      result.content = WHITESPACE_REGEX.sub(" ", result.content).strip()
                       ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
    TypeError: expected string or bytes-like object, got 'NoneType'

html_to_text, when html_str is a NoneType::

    File "/usr/local/searxng/searxng-src/searx/engines/wikipedia.py", line 190, in response
      title = utils.html_to_text(api_result.get('titles', {}).get('display') or api_result.get('title'))
    File "/usr/local/searxng/searxng-src/searx/utils.py", line 158, in html_to_text
      html_str = html_str.replace('\n', ' ').replace('\r', ' ')
                 ^^^^^^^^^^^^^^^^
    AttributeError: 'NoneType' object has no attribute 'replace'

presearch engine, when json_resp is a NoneType::

    File "/usr/local/searxng/searxng-src/searx/engines/presearch.py", line 221, in response
      results = parse_search_query(json_resp.get('results'))
    File "/usr/local/searxng/searxng-src/searx/engines/presearch.py", line 161, in parse_search_query
      for item in json_results.get('specialSections', {}).get('topStoriesCompact', {}).get('data', []):
                  ^^^^^^^^^^^^^^^^
    AttributeError: 'NoneType' object has no attribute 'get'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-04-01 11:13:47 +02:00
Zhijie He
08885d0614 [feat] add Quark autocomplete support 
Co-authored-by: Bnyro <bnyro@tutanota.com>
 2025-03-31 14:59:49 +02:00
Zhijie He
7b4612e862 [feat] engines: add Ollama engine 2025-03-30 14:25:58 +02:00
Bnyro
9ffa9fb730 [feat] engines: add reuters news engine 2025-03-30 13:56:09 +02:00
Tommaso Colella
5daa4f0460 [feat] engine: add engine for italian online newspaper "il post" 2025-03-30 13:45:06 +02:00
Zhijie He
33661cc5c3 [feat] engines: add Quark engine 
Co-authored-by: Bnyro <bnyro@tutanota.com>
 2025-03-30 13:20:35 +02:00
Zhijie He
b231cb4b59 [feat] engines: add Niconico videos engine 
Co-authored-by: Bnyro <bnyro@tutanota.com>
 2025-03-30 12:42:31 +02:00
naughtymommy42069
c8b419fcbb [feat] engine: add bitchute 2025-03-30 12:41:43 +02:00
Markus Heiser
237267ffbe [data] update searx.data - update_engine_descriptions.py 2025-03-30 09:54:03 +02:00
Markus Heiser
50f92779bd [refactor] migrate plugins from "module" to class SXNGPlugin 
This patch brings two major changes:

- ``Result.filter_urls(..)`` to pass a filter function for URL fields
- The ``enabled_plugins:`` section in SearXNG's settings do no longer exists.

To understand plugin development compile documentation:

    $ make docs.clean docs.live

and read http://0.0.0.0:8000/dev/plugins/development.html

There is no longer a distinction between built-in and external plugin, all
plugins are registered via the settings in the ``plugins:`` section.

In SearXNG, plugins can be registered via a fully qualified class name.  A
configuration (`PluginCfg`) can be transferred to the plugin, e.g. to activate
it by default / *opt-in* or *opt-out* from user's point of view.

built-in plugins
================

The built-in plugins are all located in the namespace `searx.plugins`.

.. code:: yaml

    plugins:

      searx.plugins.calculator.SXNGPlugin:
        active: true

      searx.plugins.hash_plugin.SXNGPlugin:
        active: true

      searx.plugins.self_info.SXNGPlugin:
        active: true

      searx.plugins.tracker_url_remover.SXNGPlugin:
        active: true

      searx.plugins.unit_converter.SXNGPlugin:
        active: true

      searx.plugins.ahmia_filter.SXNGPlugin:
        active: true

      searx.plugins.hostnames.SXNGPlugin:
        active: true

      searx.plugins.oa_doi_rewrite.SXNGPlugin:
        active: false

      searx.plugins.tor_check.SXNGPlugin:
        active: false

external plugins
================

SearXNG supports *external plugins* / there is no need to install one, SearXNG
runs out of the box.

- Only show green hosted results: https://github.com/return42/tgwf-searx-plugins/

To get a developer installation in a SearXNG developer environment:

.. code:: sh

   $ git clone git@github.com:return42/tgwf-searx-plugins.git
   $ ./manage pyenv.cmd python -m \
         pip install -e tgwf-searx-plugins

To register the plugin in SearXNG add ``only_show_green_results.SXNGPlugin`` to
the ``plugins:``:

.. code:: yaml

    plugins:
      # ...
      only_show_green_results.SXNGPlugin:
        active: false

Result.filter_urls(..)
======================

The ``Result.filter_urls(..)`` can be used to filter and/or modify URL fields.
In the following example, the filter function ``my_url_filter``:

.. code:: python

   def my_url_filter(result, field_name, url_src) -> bool | str:
       if "google" in url_src:
           return False              # remove URL field from result
       if "facebook" in url_src:
           new_url = url_src.replace("facebook", "fb-dummy")
           return new_url            # return modified URL
       return True                   # leave URL in field unchanged

is applied to all URL fields in the :py:obj:`Plugin.on_result` hook:

.. code:: python

   class MyUrlFilter(Plugin):
       ...
       def on_result(self, request, search, result) -> bool:
           result.filter_urls(my_url_filter)
           return True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2025-03-29 10:16:43 +01:00
return42
d36da0a6c3 [data] update searx.data - update_ahmia_blacklist.py 2025-03-29 09:49:27 +01:00
return42
d7ca257fcd [data] update searx.data - update_firefox_version.py 2025-03-29 09:48:39 +01:00
return42
98024d8518 [data] update searx.data - update_wikidata_units.py 2025-03-29 09:48:06 +01:00
return42
96d0854a50 [data] update searx.data - update_currencies.py 2025-03-29 09:47:33 +01:00
Aadniz
ecee73eafd [fix] presearch engine: Unexpected crash if duration not in videos 2025-03-28 16:26:39 +01:00
searxng-bot
1189b1906c [l10n] update translations from Weblate 
9a0c9e142 - 2025-03-27 - return42 <return42@noreply.codeberg.org>
8e06fb05a - 2025-03-27 - return42 <return42@noreply.codeberg.org>
8e683ef69 - 2025-03-27 - return42 <return42@noreply.codeberg.org>
3588715a0 - 2025-03-27 - return42 <return42@noreply.codeberg.org>
acf531db2 - 2025-03-27 - return42 <return42@noreply.codeberg.org>
b4b09bdef - 2025-03-27 - return42 <return42@noreply.codeberg.org>
8bef57082 - 2025-03-27 - return42 <return42@noreply.codeberg.org>
239bae388 - 2025-03-27 - return42 <return42@noreply.codeberg.org>
ca6a6fc4f - 2025-03-27 - return42 <return42@noreply.codeberg.org>
2a19a52db - 2025-03-27 - return42 <return42@noreply.codeberg.org>
7b217fad1 - 2025-03-24 - Haraldher <haraldher@noreply.codeberg.org>
649d15f55 - 2025-03-24 - Haraldher <haraldher@noreply.codeberg.org>
 2025-03-28 08:51:09 +01:00
dependabot[bot]
4a0ff84a40 [upd] pypi: Bump selenium from 4.29.0 to 4.30.0 
Bumps [selenium](https://github.com/SeleniumHQ/Selenium) from 4.29.0 to 4.30.0.
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-4.29.0...selenium-4.30.0)

---
updated-dependencies:
- dependency-name: selenium
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-03-28 08:50:32 +01:00
dependabot[bot]
d02d432862 [upd] pypi: Bump yamllint from 1.36.2 to 1.37.0 
Bumps [yamllint](https://github.com/adrienverge/yamllint) from 1.36.2 to 1.37.0.
- [Release notes](https://github.com/adrienverge/yamllint/releases)
- [Changelog](https://github.com/adrienverge/yamllint/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/adrienverge/yamllint/compare/v1.36.2...v1.37.0)

---
updated-dependencies:
- dependency-name: yamllint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-03-28 08:50:10 +01:00
dependabot[bot]
3c537f958f [upd] web-client (simple): Bump stylelint in /client/simple 
Bumps [stylelint](https://github.com/stylelint/stylelint) from 16.16.0 to 16.17.0.
- [Release notes](https://github.com/stylelint/stylelint/releases)
- [Changelog](https://github.com/stylelint/stylelint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/stylelint/stylelint/compare/16.16.0...16.17.0)

---
updated-dependencies:
- dependency-name: stylelint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-03-28 08:49:44 +01:00
dependabot[bot]
a2ad1bed00 [upd] web-client (simple): Bump eslint in /client/simple 
Bumps [eslint](https://github.com/eslint/eslint) from 9.22.0 to 9.23.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v9.22.0...v9.23.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-03-28 08:48:46 +01:00
Markus Heiser
f62f6194e3 [build] /static 2025-03-27 12:27:37 +01:00