Locales

SearXNG’s locale data

The variables RTL_LOCALES and LOCALE_NAMES are loaded from git://searx/data/locales.json / see locales_initialize and update_locales.py.

Hint

Whenever the value of ADDITIONAL_TRANSLATIONS or LOCALE_BEST_MATCH is modified, the git://searx/data/locales.json needs to be rebuild:

./manage data.locales

SearXNG’s locale codes

List of SearXNG’s locale codes used for the search language/region.

Hint

Don’t modify this file, this file is generated by:

./manage data.traits

searx.sxng_locales.sxng_locales

A list of five-digit tuples:

0. SearXNG’s internal locale tag (a language or region tag)

1. Name of the language (babel.core.Locale.get_language_name)

2. For region tags the name of the region (babel.core.Locale.get_territory_name). Empty string for language tags.

3. English language name (from babel.core.Locale.english_name)

4. Unicode flag (emoji) that fits to SearXNG’s internal region tag. Languages are represented by a globe (🌐)

('en',    'English', '',              'English', '🌐'),
('en-CA', 'English', 'Canada',        'English', '🇨🇦'),
('en-US', 'English', 'United States', 'English', '🇺🇸'),
..
('fr',    'Français', '',             'French',  '🌐'),
('fr-BE', 'Français', 'Belgique',     'French',  '🇧🇪'),
('fr-CA', 'Français', 'Canada',       'French',  '🇨🇦'),

SearXNG’s locale implementations

searx.locales.LOCALE_NAMES

Mapping of locales and their description. Locales e.g. ‘fr’ or ‘pt-BR’ (see locales_initialize).

searx.locales.RTL_LOCALES: set[str] = {'ar', 'fa-IR', 'he'}

List of Right-To-Left locales e.g. ‘he’ or ‘fa-IR’ (see locales_initialize).

searx.locales.ADDITIONAL_TRANSLATIONS = {'dv': 'ދިވެހި (Dhivehi)', 'oc': 'Occitan', 'pap': 'Papiamento', 'szl': 'Ślōnski (Silesian)'}

Additional languages SearXNG has translations for but not supported by python-babel (see locales_initialize).

searx.locales.LOCALE_BEST_MATCH = {'dv': 'si', 'nl-BE': 'nl', 'oc': 'fr-FR', 'pap': 'pt-BR', 'szl': 'pl', 'zh-HK': 'zh-Hant-TW'}

Map a locale we do not have a translations for to a locale we have a translation for. By example: use Taiwan version of the translation for Hong Kong.

searx.locales.get_translations()

Monkey patch of flask_babel.get_translations

searx.locales.get_translation_locales() → list[str]

Returns the list of translation locales (underscore). The list is generated from the translation folders in git://searx/translations

searx.locales.locales_initialize()

Initialize locales environment of the SearXNG session.

• monkey patch flask_babel.get_translations by get_translations

• init global names LOCALE_NAMES, RTL_LOCALES

searx.locales.region_tag(locale: Locale) → str

Returns SearXNG’s region tag from the locale (e.g. zh-TW , en-US).

searx.locales.language_tag(locale: Locale) → str

Returns SearXNG’s language tag from the locale and if exits, the tag includes the script name (e.g. en, zh_Hant).

searx.locales.get_locale(locale_tag: str) → Locale | None

Returns a babel.Locale object parsed from argument locale_tag

searx.locales.get_official_locales(territory: str, languages=None, regional: bool = False, de_facto: bool = True) → set[Locale]

Returns a list of babel.Locale with languages from babel.languages.get_official_languages.

Parameters:

• territory – The territory (country or region) code.

• languages – A list of language codes the languages from babel.languages.get_official_languages should be in (intersection). If this argument is None, all official languages in this territory are used.

• regional – If the regional flag is set, then languages which are regionally official are also returned.

• de_facto – If the de_facto flag is set to False, then languages which are “de facto” official are not returned.

searx.locales.get_engine_locale(searxng_locale, engine_locales, default=None)

Return engine’s language (aka locale) string that best fits to argument searxng_locale.

Argument engine_locales is a python dict that maps SearXNG locales to corresponding engine locales:

<engine>: {
    # SearXNG string : engine-string
    'ca-ES'          : 'ca_ES',
    'fr-BE'          : 'fr_BE',
    'fr-CA'          : 'fr_CA',
    'fr-CH'          : 'fr_CH',
    'fr'             : 'fr_FR',
    ...
    'pl-PL'          : 'pl_PL',
    'pt-PT'          : 'pt_PT'
    ..
    'zh'             : 'zh'
    'zh_Hans'        : 'zh'
    'zh_Hant'        : 'zh_TW'
}

Hint

The SearXNG locale string has to be known by babel!

If there is no direct 1:1 mapping, this functions tries to narrow down engine’s language (locale). If no value can be determined by these approximation attempts the default value is returned.

Assumptions:

1. When user select a language the results should be optimized according to the selected language.

2. When user select a language and a territory the results should be optimized with first priority on territory and second on language.

First approximation rule (by territory):

When the user selects a locale with territory (and a language), the territory has priority over the language. If any of the official languages in the territory is supported by the engine (engine_locales) it will be used.

Second approximation rule (by language):

If “First approximation rule” brings no result or the user selects only a language without a territory. Check in which territories the language has an official status and if one of these territories is supported by the engine.

searx.locales.match_locale(searxng_locale: str, locale_tag_list: list[str], fallback: str | None = None) → str | None

Return tag from locale_tag_list that best fits to searxng_locale.

Parameters:

• searxng_locale (str) – SearXNG’s internal representation of locale (de, de-DE, fr-BE, zh, zh-CN, zh-TW ..).

• locale_tag_list (list) – The list of locale tags to select from

• fallback (str) – fallback locale tag (if unset –> None)

The rules to find a match are implemented in get_engine_locale, the engine_locales is build up by build_engine_locales.

Hint

The SearXNG locale string and the members of locale_tag_list has to be known by babel! The ADDITIONAL_TRANSLATIONS are used in the UI and are not known by babel –> will be ignored.

searx.locales.build_engine_locales(tag_list: list[str])

From a list of locale tags a dictionary is build that can be passed by argument engine_locales to get_engine_locale. This function is mainly used by match_locale and is similar to what the fetch_traits(..) function of engines do.

If there are territory codes in the tag_list that have a script code additional keys are added to the returned dictionary.

>>> import locales
>>> engine_locales = locales.build_engine_locales(['en', 'en-US', 'zh', 'zh-CN', 'zh-TW'])
>>> engine_locales
{
    'en': 'en', 'en-US': 'en-US',
    'zh': 'zh', 'zh-CN': 'zh-CN', 'zh_Hans': 'zh-CN',
    'zh-TW': 'zh-TW', 'zh_Hant': 'zh-TW'
}
>>> get_engine_locale('zh-Hans', engine_locales)
'zh-CN'

This function is a good example to understand the language/region model of SearXNG:

SearXNG only distinguishes between search languages and search regions, by adding the script-tags, languages with script-tags can be assigned to the regions that SearXNG supports.