bdfe1c2a15
The EngineCache class replaces all previously individual solutions for caches in
the context of the engines.
- demo_offline.py
- duckduckgo.py
- radio_browser.py
- soundcloud.py
- startpage.py
- wolframalpha_api.py
- wolframalpha_noapi.py
Search term to test most of the modified engines::
!ddg !rb !sc !sp !wa test
!ddg !rb !sc !sp !wa foo
For introspection of the DB, jump into developer environment and run command to
show cache state::
$ ./manage pyenv.cmd bash --norc --noprofile
(py3) python -m searx.enginelib cache state
cache tables and key/values
===========================
[demo_offline ] 2025-04-22 11:32:50 count --> (int) 4
[startpage ] 2025-04-22 12:32:30 SC_CODE --> (str) fSOBnhEMlDfE20
[duckduckgo ] 2025-04-22 12:32:31 4dff493e.... --> (str) 4-128634958369380006627592672385352473325
[duckduckgo ] 2025-04-22 12:40:06 3e2583e2.... --> (str) 4-263126175288871260472289814259666848451
[radio_browser ] 2025-04-23 11:33:08 servers --> (list) ['https://de2.api.radio-browser.info', ...]
[soundcloud ] 2025-04-29 11:40:06 guest_client_id --> (str) EjkRJG0BLNEZquRiPZYdNtJdyGtTuHdp
[wolframalpha ] 2025-04-22 12:40:06 code --> (str) 5aa79f86205ad26188e0e26e28fb7ae7
number of tables: 6
number of key/value pairs: 7
In the "cache tables and key/values" section, the table name (engine name) is at
first position on the second there is the calculated expire date and on the
third and fourth position the key/value is shown.
About duckduckgo: The *vqd coode* of ddg depends on the query term and therefore
the key is a hash value of the query term (to not to store the raw query term).
In the "properties of ENGINES_CACHE" section all properties of the SQLiteAppl /
ExpireCache and their last modification date are shown::
properties of ENGINES_CACHE
===========================
[last modified: 2025-04-22 11:32:27] DB_SCHEMA : 1
[last modified: 2025-04-22 11:32:27] LAST_MAINTENANCE :
[last modified: 2025-04-22 11:32:27] crypt_hash : ca612e3566fdfd7cf7efe2b1c9349f461158d07cb78a3750e5c5be686aa8ebdc
[last modified: 2025-04-22 11:32:30] CACHE-TABLE--demo_offline: demo_offline
[last modified: 2025-04-22 11:32:30] CACHE-TABLE--startpage: startpage
[last modified: 2025-04-22 11:32:31] CACHE-TABLE--duckduckgo: duckduckgo
[last modified: 2025-04-22 11:33:08] CACHE-TABLE--radio_browser: radio_browser
[last modified: 2025-04-22 11:40:06] CACHE-TABLE--soundcloud: soundcloud
[last modified: 2025-04-22 11:40:06] CACHE-TABLE--wolframalpha: wolframalpha
These properties provide information about the state of the ExpireCache and
control the behavior. For example, the maintenance intervals are controlled by
the last modification date of the LAST_MAINTENANCE property and the hash value
of the password can be used to detect whether the password has been changed (in
this case the DB entries can no longer be decrypted and the entire cache must be
discarded).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
87 lines
2.4 KiB
Python
87 lines
2.4 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-or-later
|
|
"""Within this module we implement a *demo offline engine*. Do not look to
|
|
close to the implementation, its just a simple example. To get in use of this
|
|
*demo* engine add the following entry to your engines list in ``settings.yml``:
|
|
|
|
.. code:: yaml
|
|
|
|
- name: my offline engine
|
|
engine: demo_offline
|
|
shortcut: demo
|
|
disabled: false
|
|
|
|
"""
|
|
|
|
import json
|
|
|
|
from searx.result_types import EngineResults
|
|
from searx.enginelib import EngineCache
|
|
|
|
engine_type = 'offline'
|
|
categories = ['general']
|
|
disabled = True
|
|
timeout = 2.0
|
|
|
|
about = {
|
|
"wikidata_id": None,
|
|
"official_api_documentation": None,
|
|
"use_official_api": False,
|
|
"require_api_key": False,
|
|
"results": 'JSON',
|
|
}
|
|
|
|
# if there is a need for globals, use a leading underline
|
|
_my_offline_engine: str = ""
|
|
|
|
CACHE: EngineCache
|
|
"""Persistent (SQLite) key/value cache that deletes its values after ``expire``
|
|
seconds."""
|
|
|
|
|
|
def init(engine_settings):
|
|
"""Initialization of the (offline) engine. The origin of this demo engine is a
|
|
simple json string which is loaded in this example while the engine is
|
|
initialized."""
|
|
global _my_offline_engine, CACHE # pylint: disable=global-statement
|
|
|
|
CACHE = EngineCache(engine_settings["name"]) # type:ignore
|
|
|
|
_my_offline_engine = (
|
|
'[ {"value": "%s"}'
|
|
', {"value":"first item"}'
|
|
', {"value":"second item"}'
|
|
', {"value":"third item"}'
|
|
']' % engine_settings.get('name')
|
|
)
|
|
|
|
|
|
def search(query, request_params) -> EngineResults:
|
|
"""Query (offline) engine and return results. Assemble the list of results
|
|
from your local engine. In this demo engine we ignore the 'query' term,
|
|
usual you would pass the 'query' term to your local engine to filter out the
|
|
results.
|
|
"""
|
|
res = EngineResults()
|
|
count = CACHE.get("count", 0)
|
|
|
|
for row in json.loads(_my_offline_engine):
|
|
count += 1
|
|
kvmap = {
|
|
'query': query,
|
|
'language': request_params['searxng_locale'],
|
|
'value': row.get("value"),
|
|
}
|
|
res.add(
|
|
res.types.KeyValue(
|
|
caption=f"Demo Offline Engine Result #{count}",
|
|
key_title="Name",
|
|
value_title="Value",
|
|
kvmap=kvmap,
|
|
)
|
|
)
|
|
res.add(res.types.LegacyResult(number_of_results=count))
|
|
|
|
# cache counter value for 20sec
|
|
CACHE.set("count", count, expire=20)
|
|
return res
|