Lua is a scripting language. It is not as well known as Javascript etc, but it is much smaller and quicker and specifically designed for embedding in other software. It is widely used in applications ranging from industrial applications to games.
VPOP3 uses Lua in several places to allow user customisation to achieve features which are complex but not widely needed. Adding these features as core VPOP3 features would complicate it further for users who do not need these features and may still not achieve the flexibility that a full scripting language solution would offer.
Lua is documented fully on the Lua website at http://www.lua.org. VPOP3 currently uses Lua 5.2.
You can create & edit Lua scripts for VPOP3 in the Settings -> Scripts page. This includes an editor with Lua syntax highlighting. Alternatively, you can create .LUA files in the VPOP3 installation directory using any suitable text editor such as Notepad++, Sublime Text or UltraEdit.
When VPOP3 wants to use that script it will check in the VPOP3 installation directory for the relevant .LUA file. If it finds it, it will import the Lua file into the VPOP3 settings database and rename the .LUA file to .LUA.OLD. (It stores the scripts in the database so that backups & replication will backup & replicate the scripts automatically).
See the following topics for various places Lua scripts are used in VPOP3.