15 Commits

Author SHA1 Message Date
Markus Heiser
e6308b8167 [fix] hardening against arguments of type None, where str or dict is expected
On a long-running server, the tracebacks below can be found (albeit rarely),
which indicate problems with NoneType where a string or another data type is
expected.

result.img_src::

    File "/usr/local/searxng/searxng-src/searx/templates/simple/result_templates/images.html", line 13, in top-level template code
      <img src="" data-src="{{ image_proxify(result.img_src) }}" alt="{{ result.title|striptags }}">{{- "" -}}
      ^
    File "/usr/local/searxng/searxng-src/searx/webapp.py", line 284, in image_proxify
      if url.startswith('//'):
         ^^^^^^^^^^^^^^
    AttributeError: 'NoneType' object has no attribute 'startswith'

result.content::

    File "/usr/local/searxng/searxng-src/searx/result_types/_base.py", line 105, in _normalize_text_fields
      result.content = WHITESPACE_REGEX.sub(" ", result.content).strip()
                       ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
    TypeError: expected string or bytes-like object, got 'NoneType'

html_to_text, when html_str is a NoneType::

    File "/usr/local/searxng/searxng-src/searx/engines/wikipedia.py", line 190, in response
      title = utils.html_to_text(api_result.get('titles', {}).get('display') or api_result.get('title'))
    File "/usr/local/searxng/searxng-src/searx/utils.py", line 158, in html_to_text
      html_str = html_str.replace('\n', ' ').replace('\r', ' ')
                 ^^^^^^^^^^^^^^^^
    AttributeError: 'NoneType' object has no attribute 'replace'

presearch engine, when json_resp is a NoneType::

    File "/usr/local/searxng/searxng-src/searx/engines/presearch.py", line 221, in response
      results = parse_search_query(json_resp.get('results'))
    File "/usr/local/searxng/searxng-src/searx/engines/presearch.py", line 161, in parse_search_query
      for item in json_results.get('specialSections', {}).get('topStoriesCompact', {}).get('data', []):
                  ^^^^^^^^^^^^^^^^
    AttributeError: 'NoneType' object has no attribute 'get'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-04-01 11:13:47 +02:00
Aadniz
ecee73eafd [fix] presearch engine: Unexpected crash if duration not in videos 2025-03-28 16:26:39 +01:00
Bnyro
4dfc47584d [refactor] duration strings: move parsing logic to utils.py 2025-03-25 16:48:44 +01:00
Bnyro
3668c7012e [fix] presearch videos: item description and duration are located in metadata field 2025-03-20 20:55:09 +01:00
Aadniz
556db857aa [fix] presearch engine: News and Videos formatted incorrectly 2025-03-20 20:44:43 +01:00
Aadniz
a88b4d7036 [fix] presearch engine: domain sometimes included in beginning of titles 2025-03-08 12:39:16 +01:00
Aadniz
4884747508 [fix] presearch engine: Title showing <em> html code 2025-03-07 21:24:35 +01:00
Markus Heiser
916739d6b4 [mod] simple theme: drop img_src from default results
The use of img_src AND thumbnail in the default results makes no sense (only a
thumbnail is needed).  In the current state this is rather confusing, because
img_src is displayed like a thumbnail (small) and thumbnail is displayed like an
image (large).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-05-16 07:30:38 +02:00
Markus Heiser
542f7d0d7b [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION
In the past, some files were tested with the standard profile, others with a
profile in which most of the messages were switched off ... some files were not
checked at all.

- ``PYLINT_SEARXNG_DISABLE_OPTION`` has been abolished
- the distinction ``# lint: pylint`` is no longer necessary
- the pylint tasks have been reduced from three to two

  1. ./searx/engines -> lint engines with additional builtins
  2. ./searx ./searxng_extra ./tests -> lint all other python files

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-11 14:55:38 +01:00
Markus Heiser
8205f170ff [mod] pylint all engines without PYLINT_SEARXNG_DISABLE_OPTION
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-11 14:55:38 +01:00
Markus Heiser
e560d7e373 [mod] presearch: add language & region support
In Presearch there are languages for the UI and regions for narrowing down the
search.  With this change the SearXNG engine supports a search by region.  The
details can be found in the documentation of the source code.

To test, you can search terms like::

   !presearch bmw :zh-TW
   !presearch bmw :en-CA

1. You should get results corresponding to the region (Taiwan, Canada)
2. and in the language (Chinese, Englisch).
3. The context in info box content is in the same language.

Exceptions:

1. Region or language is not supported by Presearch or
2. SearXNG user did not selected a region tag, example::

    !presearch bmw :en

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-01-15 19:23:26 +01:00
Markus Heiser
a2c269bbac [mod] presearch: hardening engine's response against KeyErrors
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-01-15 19:23:26 +01:00
Bnyro
0a78f59aba [fix] presearch: safesearch, time ranges, crash when no result found 2024-01-15 19:23:26 +01:00
Markus Heiser
44392bd436 [mod] improve implementation of presearch engine
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-11-27 14:16:42 +01:00
Bnyro
23582aac5c [feat] implementation of presearch engine 2023-11-27 14:16:42 +01:00