වික්ෂනරි:Lua memory errors

Wiktionary often experiences Lua (Scribunto) memory errors in certain entries with many language sections (chiefly certain one- and two-Latin-letter entries), on some pages with many translations, and on some single Han character entries.

Background

Lua is the scripting language found on Module-namespace pages (see WT:LUA). It is used in many templates. It is used when there is {{#invoke:module name|function name|optional arguments ...}} (a module invocation) on a page, or in text that is transcluded on the page, from a template for instance.

It has a garbage collector, which frees up memory after it is no longer accessible from the Lua code. Memory is not freed immediately, so memory usage is somewhat unpredictable. Memory can rise or fall each time you preview a page, and removing a module invocation from a page can actually increase memory.

The memory limit is 50 megabytes. When Lua memory goes over this limit, an error with the explanation "not enough memory" is displayed in place of the module invocation ({{#invoke:module name|function name|optional arguments ...}}) and the page is placed in Category:මොඩියුල දෝෂ සහිත පිටු.

Difficulties

Certain pages have had memory errors since at least 2019. These include some one- and two-Latin-letter entries and some single Han character entries. Examples include a.

Some pages with many translations have had memory errors.

Tactics

Lua-free templates

For the Latin-script entries like a, the primary solution has been to swap out most uses of key highly-used templates, such as {{head}} and {{m}}, for their Lua-free equivalents, such as {{head-lite}} and {{m-lite}}. The -lite templates are written in pure wiki-syntax, hence taking up no Lua memory at all.

The Lua-free templates have more limited functionality than the corresponding Lua versions. For basic usage with most Latin-script languages, they are a like-for-like drop-in replacement. However, they should normally be avoided for links to non-Latin-script languages (this is mainly an issue in etymology sections). This is because the Lua-free templates do not perform automatic transliteration and normalisation. Also, certain rarely-used parameters are not supported so that the -lite templates stay easy to maintain. In practice, our very long entries use a mix of the Lua-free and regular templates, saving the Lua memory for where it is actually needed.

A full list of Lua-free templates can be found at Category:Lua-free templates.

Use Template:multitrans

On pages with many translations, the preferred solution to memory errors is to use {{multitrans}}. This involves wrapping the entire translation table with a call to {{multitrans|data=<...>}} and changing all calls to {{t}} inside the wrapped code with {{tt}}, and all calls to {{t+}} with {{tt+}}. See orange, wolf and several other pages for examples of how to do this. This often reduces the memory by a factor of two (e.g. from 50+ MB to 25 MB). The way it works is to load the translation and linking modules only once, instead of once per translation (which may amount to several hundred copies in memory).

Translation subpages

On pages with many translations, if the {{multitrans}} solution doesn't work, an alternative has been to move translations to a fake translations subpage created by adding /translations to the title and {{translation subpage}} on the top of the page; see Category:Translation subpages for the full list.

These translation subpages are not true subpages because the subpage feature is turned off in the main namespace. This allows there to be entries for terms that contain slashes. A true subpage has a link to the parent page under its title. This does not happen to titles with slashes in the main namespace. For instance, water/translations does not have a link to water under its title.

Derivation subpages

These work similarly to translation subpages. These store all derived terms from an individual lemma on a separate, fake subpage, and are mostly used with Han/CJKV characters. It is created by adding /derivations to the title and {{derivative subpage}} on top of the page; they will then be added to Category:Derivative subpages.

Increasing the memory limit

A Wiktionary admin posted a Phabricator task that suggested increasing the memory limit. This would probably not be a long-term solution because memory usage would probably continue increasing and eventually exceed the new limit.

Replacing letters templates

Some of the most vulnerable targets for Lua memory errors are single-letter Latin alphabet pages. In fact, all five original Latin vowel letters (a, e, i, o, and u) have faced module errors multiple times. Hence, the Lua-dependent {{letters}} or some other derivative appears often on these pages. One workaround is to replace some of the templates dependent on {{letters}} with "simple" versions composed of plain wikitext, in order to reshuffle the garbage collection. Unfortunately, garbage reshuffling doesn't always happen and sometimes replacing letters list templates may worsen module errors.

Optimizing Lua memory usage

It might be possible to redesign some modules (including core modules) to reduce memory usage. Larger changes might be required in order to have a noticeable effect. It remains to be seen whether this can be achieved in practice.

Workaround for users

Content of a section that cannot be seen because it is replaced by “Lua error: not enough memory” can often be viewed by first clicking “[ edit ]” next to the section header, and then the button Show preview below the edit window.

Long-term solutions

Per-language pages

This is an old proposal, which predates the Lua memory issue, but might solve it. A very detailed overview and links to old discussions can be found at Wiktionary:Per-language pages proposal.