Scripting Engine

From Hero of Allacrost Wiki
Jump to: navigation, search

This documentation describes the scripting component of the Allacrost engine, as well as how to bind C++ code to Lua and use those bindings. The primary purpose of the scripting engine is to open a communication channel between the game's C++ engine and scripts written in Lua. This allows data, functions, classes, etc. to be shared between the two languages.


Introduction[edit]

Scripting Engine Summary
Directory
  • src/engine/script/
Files
  • script.h
  • script.cpp
  • script_read.h
  • script_read.cpp
  • script_write.h
  • script_write.cpp
  • script_modify.h
  • script_modify.cpp
Include Header
  • #include "script.h"
Namespace
  • hoa_script
Classes Defined
  • GameScript (singleton name: ScriptManager)
  • ScriptDescriptor
  • ReadScriptDescriptor
  • WriteScriptDescriptor
  • ModifyScriptDescriptor
Libraries Used
  • Lua (programming language with a C API)
  • Luabind (C++ <=> Lua code bindings)


The script engine makes use of two libraries: the Lua C API and Luabind. Lua is the name of the scripting language which Allacrost uses. This language has a C API defined so that its use may be integrated into C or C++ code. Luabind is a library that provides "bindings" between C++ and Lua. For example, Luabind makes it possible for Lua to create instances of C++ classes and manipulate those objects, and makes C++ functions accessible to Lua. The scripting engine contains the definition of two classes: ScriptDescriptor and GameScript. ScriptDescriptor is an abstract class which the Read, Write, and Modify ScriptDescriptors derive from. All types of ScriptDescriptor objects represent a single Lua script file, whether it is open for reading, writing, or modification. The GameScript class is a singleton class that uses the object name ScriptManager. Most of the time, the API user will not need to use this class, as its only task is the initialization of libraries, maintaining a global state for the scripting environment, and other back end management tasks.


There are three primary means that the scripting engine interacts with scripts: reading/execution of scripts, writing scripts, and modification of existing scripts. The mode of operation most often used is of reading/executing Lua scripts. Writing of Lua scripts is primarily done by the Allacrost editor to create new maps, although this feature may be used sparingly in the game itself. Modification of existing Lua scripts is used, for example, to allow the player to save their game or retain their personal game settings. The following sections describes each of these three uses of the script engine.


Note: About Using Luabind
  • Due to its nature, the Luabind library is not completed abstracted away by the scripting engine and may require some direct usage by the programmer.
  • The Luabind library uses some very commonplace names in its structures (for example, object is the name of one class in Luabind). Therefore, the user is advised to minimize the directive using namespace luabind;. Instead, it is recommended to explicitly declare the scope of all Luabind objects and calls (e.g. luabind::object).
  • Luabind has a trac system setup at the following location [1]. This includes some additional user documentation that is not available in the standard documentation.


Reading and Executing Scripts[edit]

The API user must first understand that in reading/execution mode, C++ does not only read Lua data and execute Lua code, but Lua is allowed to read C++ data and execute C++ code. There is a communication barrier between C++ and Lua, as they are different languages with different constructs. For example, Lua is a typeless language while C++ is a strongly typed. Lua has a garbage collector while memory management in C++ is done by the programmer. We will first discuss how C++ can read Lua data and execute Lua code, and then explain how Lua can initiate communication with C++. Every ScriptDescriptor object maintains its own local Lua stack via a lua thread. The ScriptManager singleton maintains a "global" Lua state, which all other Lua states inherit from. What this means is that the global state may be used to communication information to all other Lua states. The motivation for this will be explained when we discuss bindings between Lua and C++.


Accessing Lua from C++[edit]

First, create a new ReadScriptDescriptor object and open the file with readable permissions.

ScriptDescriptor my_script;
bool success = my_script.OpenFile("dat/my_script.lua");


Reading Simple Types[edit]

Reading global Lua variables of simple types is very easy. You only need to supply the name of the global variable in the script. If any of these read operations fail, the default value (false, 0, 0.0, empty string) will be returned and an error code in the scripting engine will be set. (We will discuss script engine error codes shortly).

bool b = my_script.ReadBool("my_boolean");
int32 i = my_script.ReadInt("my_integer");
uint32 i = my_script.ReadUInt("my_unsigned_integer");
float f = my_script.ReadFloat("my_float");
string s = my_script.ReadString("my_string");

The most common reasons for a read error are the following:

  1. The name of the variable is not spelled correctly
  2. The name could not be converted to the proper type (i.e. ReadInt("my_string"), the user expects that my_string holds an integer, when in truth it holds a string)
  3. The active scope is incorrect (explained when we get to reading tables)
  4. The script file is not open, or the file is open without the correct permissions for reading


Reading Tables[edit]

Not all variables that we will wish to retrieve will be global one. Some variables may be elements inside a Lua table. Lua tables are the universal data structure in Lua, and can act as arrays, vectors, linked lists, stacks, queues, class objects, namespaces, and more. To access variables that are embedded in a table, first we must open the appropriate table or tables (a table can be defined as an element of another tables). When a table is opened by the ReadScriptDescriptor, the table becomes the "active scope" from which data is retrieved. What this means is that when we now try to load a variable, that variable is assumed to be an element of the most recent table. If the variable does not exist, or exists only in another "scope" such as the global scope, the variable will not be read successfully.

// After opening the file, we're placed in the global space
int32 i = my_script.ReadInt("my_integer");  // "my_integer" is a variable read from the global scope

my_script.OpenTable("my_table");        // the new active space is now "my_table"
int32 j = my_script.ReadInt("my_integer");  // this "my_integer" is *not* the same variable as the first, as it exists in a different scope
my_script.CloseTable();                 // after closing the table, the active space becomes the global space once again

The ReadScriptDescriptor class will not go searching in every single scope in the script for a variable named "my_integer" (because it would be very inefficient). Therefore, it is the programmer's responsibility to ensure that the correct scope is active when trying to read data. You must always remember to close tables that you opened when you are finished with them. If you forget to, it's likely that your next read operation will fail because you are in a different scope than you think you are in. Unlike global variable keys, table variable keys can be numbers (table indeces) in addition to strings. This means that whenever the active scope is not the global scp[e, it's possible to read variables using an integer key.

// Inside the global scope
int32 i = my_script.ReadInt(5);          // ERROR: we can't use an integer key to read a variable in the global scope!

my_script.OpenTable("my_table");
int32 j = my_script.ReadInt(3);          // reads the integer at table index number 3
my_script.CloseTable();


Reading Vectors[edit]

Sometimes a table holds a long list of elements of the same data type that we want to load in. We present the following two examples of how to do this:

vector<int32> table_data;

my_script.OpenTable("my_table");
uint32 tsize = my_script.GetTableSize();   // returns the number of elements in the most recently opened table
for (uint32 i = 0; i < tsize; i++) {           // be careful about assuming the range of keys is always 0 to size-1
    table_data.push_back(my_script.ReadInt(i));
}
my_script.CloseTable();

Note that a table is not required to keep its keys in numeric order. In fact, by default the key for the first element inserted in a Lua table is 1 (not 0 like it is C++). Also the indeces need not be sequential, nor even numeric. Tables with the following keys are all of size == 5: [0, 1, 2, 3, 4], [1, 2, 3, 4, 5], [-4, 27.8, "apple", 0, "seventeen"]. Therefore, it is very important that you not make assumptions about the structure of keys in a table.

The second example below is a much easier (and safer) way to achieve the same result.

vector<int32> table_data;
my_script.ReadIntVector("my_table", table_data);

Note that we did not open the table "my_table" in this example. The ReadIntVector() function automatically opens and closes the table for us. So when calling a ReadTYPEVector() function, we want to make sure that the table exists in the active space (in this example: my_table exists in the global space, which is what we were in when we called the ReadIntVector function). Like the ReadInt function, there are equivalents for each type (ReadBoolVector(), ReadUIntVector(), ReadFloatVector(), ReadStringVector()).


Reading Functions[edit]

You can not necessarily "read" functions from Lua, but rather you read pointers to functions. This is accomplished in nearly the same manner as the reading of simple primitives, but we require a special container to store Lua function pointers. We use the ScriptObject class to retain Lua function pointers, and also to assist us in making Lua function calls. C++ code to read a Lua function pointer looks like the folllowing:

ScriptObject my_func = my_script.ReadFunctionPointer("my_function");

This will read the Lua file and look for a function named "my_function", and upon success it will save that pointer to the "my_func" ScriptObject. In the Lua programming language, functions can be members of tables, so therefore the active scope does factor in to these calls. There is also a version of this function which accepts an integer key, which is valid only for reading function pointers that do not exist in the global scope. Once the corresponding Lua file is closed, the ScriptObject immediately becomes invalid and should not be used, unless it is re-used to read in another function pointer from another open file.

ScriptObject ReadScriptDescriptor::ReadFunctionPointer(std::string key);
ScriptObject ReadScriptDescriptor::ReadFunctionPointer(int32 key);
Note: About ScriptObject

ScriptObject is really just a macro name for the Luabind library class luabind::object. ScriptObjects can store any type of data (ints, strings, etc.) in addition to function pointers. However, for our purposes we use them exclusively for holding function pointers.

Lua Namespaces[edit]

It is sometimes the case that multiple Lua scripts use tables or other variables of the same name. When this happens, the data in one file can potentially wipe out the data in the other file. To avoid this problem, we bundle our scripts within their own unique namespaces. Placing the Lua code below at the top of the script file will accomplish this.

local namespace = {}
setmetatable(namespace, {__index = _G})
demo_map = namespace;
setfenv(1, namespace);
  1. Creates a local table (used as a namespace) that all of the variables and functions in the Lua file will map to.
  2. Maps the globals table to a meta table so that any failed lookups in the local table will get passed to the globals table. This allows the script access to any globals we have defined. This line is required because of the function of the final line in this code segment.
  3. Maps the local table to a global table, so that the namespace used in this file is now available globally.
  4. Replaces the environment with the local namespace table. Now any non-local variables that are defined in the file are automatically placed inside the local namespace.
Note: Creating Globals

Since we replace the environment in the final line, this means that any variables or functions that we would like to be available in the globals table must be declared before the code segment above executes.

Executing Script Functions[edit]

There are two ways to execute a Lua script function. The first method may only be used when the script function in question is within the global scope of the Lua file that it resides in.

ScriptCallFunction<TYPE>(lua_State* L, const char* name, ...);

TYPE is the return type of the Lua function (void is an acceptable return type). Note that this return value is returned through the ScriptCallFunction function - the return value of ScriptCallFunction is what the lua function would return. lua_State* L is a pointer to the state of the open Lua file where the function to call resides in. This structure is the medium through which C++ and Lua communicate with one another, and it is normally managed internally by each ScriptDescriptor object. You can gain access to the lua_State by invoking the following method:

lua_State* ReadScriptDescriptor::GetLuaState()

const char* name is nothing more than the name of the function to call. All optional arguments that follow name are the list of arguments that should be passed to the Lua function.


The second way to execute a Lua script function uses ScriptObjects to make their call. This method allows you to call both Lua functions that exist globally, and those that exist as members of a table. It should be obvious that you must first initialize the ScriptObject with the return value of a ReadFunctionPointer call before you attempt to use it as a means to execute a script function.

ScriptCallFunction<TYPE>(ScriptObject funct, ...);

Once the script file which the pointer is read from is closed, the function pointer will become immediately invalid. If you want to call the Lua function once more, you must re-open the file, re-read the function pointer, and re-call the luabind::call_function routine. Because of this, it is advised that if you expect to keep making repeated calls to a Lua function within a script file, keep that file open persistently so as to not incur a significant cost for calling the function. Note that there are not any safety mechanisms that check if the ScriptObject function pointer is still valid or not, so be careful when you use them.

Note: About ScriptCallFunction

ScriptCallFunction is a macro for the function luabind::call_function. Be sure to only make ScriptCallFunction calls (of either type) if your script is open with read permissions, because there is no permissions error checking performed by the script engine in this case. (This is due to the fact that luabind::call_function takes a variable number of arguments, making it inefficient to provide a wrapper function for it).

Handling Lua Errors[edit]

Sometimes, you may call a Lua function which generates a run-time error. To handle this, you need to put your function call in a try, catch block and catch any luabind::error objects which are thrown. The GameScript class has a special function which will automatically handle the error for you.

ScriptObject func_ptr = script_file.ReadFunctionPointer("AddNumbers");
try {
    int32 result = ScriptCallFunction<int32>(funct_ptr, 1, 2);
}
catch(luabind::error e) {
    ScriptManager->HandleLuaError(e);
}

Assuming that the "AddNumbers" Lua function generates a Lua run-time error, here is what will happen: -# Lua pushes an error message to the top of the lua_Stack -# Luabind throws the luabind::error exception -# The catch block calls the GameScript::HandleLuaError() method -# This method grabs the error message on the stack that Lua placed, prints it to stderr, and pops it from the stack

What is significant to note here is that Lua will modify the lua_State stack if a run-time error occurs. If you fail to remove the error message from the stack via the HandleLuaError() call, the result may be fatal, since the ScriptDescriptor class does not know that its internally managed stack has been modified. Therefore, it is highly recommended that you use the try, catch block shown above in all of your ScriptCallFunction invocations.


Another type of error that may occur when calling a Lua function is a luabind::cast_failed exception. When this exception is thrown, it means that the return value of the Lua function which was called could not be properly casted into a C++ type. The way to handle this error is similar to handling Lua run-time errors:

ScriptObject func_ptr = script_file.ReadFunctionPointer("AddNumbers");
try {
    int32 result = ScriptCallFunction<int32>(funct_ptr, 1, 2);
}
catch(luabind::error e) {
    ScriptManager->HandleLuaError(e);
}
catch(luabind::cast_failed e) {
    ScriptManager->HandleCastError(e);
}

The GameScript::HandleCastError() method prints an error message to stderr providing further information about the failure (specifically, it prints C++ type information about the cast failure). Unlike with Lua run-time errors, Lua does not modify the lua_State in this case, so in that regard this type of error is less severe. You may choose not to try to catch this exception (and trying to catch it when the return type is void would be silly), but it can be useful to assist with debugging.

Accessing C++ from Lua[edit]

For Lua to access C++ data, classes, and functions, C++ must explicitly grant access of those constructs for Lua to use. The act of making C++ structures accessible to Lua is called binding. The user is required to write binding code in order to make their C++ structures available to Lua. Writing this binding code is not too difficult, but it can become tedious and monotonous to do. The following sub-sections first describe how to write the binding code for C++ structures and methods, followed by how to use them in a Lua script.


Binding Introduction[edit]

Before we dive into the binding code, a few notes must be made.

Luabind Syntax

The syntax of the C++ binding code will look extremely strange to most programmers. Luabind is a library which employs template meta programming, which accounts for its rather awkward appearance.

Compilation Speed

The templated nature of Luabind has one disadvantage: it takes a significantly longer amount of time to compile the program. The more files that the Luabind binding code is distributed in, the longer the compilation time. The authors of Luabind recommend putting all of the binding code in a single file, if possible, to minimize this penalty. Thus, all binding code for Allacrost is contained inside the file src/defs.cpp

Lua State Inheritence

Lua and C++ communicate through a stack structure that is defined as a lua_State in C/C++. Every script file that is opened for reading has its own local Lua state active. In addition, the ScriptManager singleton maintains a global Lua state, which all of the local Lua states inherit from. What this means is that anything that can be "seen" by the global Lua state can likewise be seen by all of the local Lua states. Hence, the actual binding process all takes place on the global Lua state, so that the classes, functions, etc. that are bound can be utilized by all of the script files.

Binding Process

The binding code is executed once and only once for each invocation of the game. This execution takes place when the SingletonInitialize() function is called for the GameScript singleton class, inside main.cpp. Note that the binding must take place "on-line", and that the bindings can not be "compiled" into Lua off-line.

Namespace Translation

A lot of bound code is shared in Allacrost, and this creates the possibility of naming conflicts. In the C++ code, we avoid these conflicts by employing the use of namespaces. In Lua, we do the same thing. All code is bound to a Lua "namespace" (which is really just a Lua table) rather than having all of the code being bound in the global space. The disadvantages of this are that we have to type the namespace name each time we wish to use something in that space (there is no "using namespace" directive in Lua), and it slightly increases the look-up time for functions, etc. However, it helps us to avoid naming conflicts and also keeps the structure of the engine in Lua consistent with the structure of the engine in C++.

Encapsulation in Lua

Unlike C++, Lua has very relaxed encapsulation mechanisms (in fact, by default every variable declared in Lua is made global if not explicitly stated otherwise!). Therefore binding of public and private class methods makes no difference as far as Lua is concerned: they are all public. The user is advised to keep this important note in mind at all times to ensure that they do not accidentally make something available that shouldn't be. The best way to keep private methods and members encapsulated is to not bind them to Lua at all, if it isn't absolutely necessary to do so.


Writing Binding Code[edit]

Note: This section is out-dated and needs to be re-written

Typically, there is only one binding function per-namespace and usually it is a static method of the namespace's primary class (usually a singleton manager class, or a game mode class). For consistency reasons, this function name is always void BindToLua(). The only place that this function should be called is in the function GameScript::SingletonInitialize(), which can be found in the file script.cpp. As an example, lets pretend that we are writing binding code for a game mode called TestMode, which exists in the namespace hoa_test. The initial binding function would appear as below

void TestMode::BindToLua() {
    using namespace luabind;                                // (A)

    module(ScriptManager->GetGlobalState(), "hoa_test") [   // (B)

    ];
}

First notice in (A) we use the luabind namespace. The namespace is only used for the scope of the function (we don't want to use it throughout the entire file, since there are several common names in Luabind that could introduce conflicts). The module on line (B) simply declares that what is embedded in this module will be bound to the namespace "hoa_test". As with all of our bindings, it is bound to the global state that is managed by the scripting engine so that other local Lua states can inherit from it and make use of the bindings within. If we wanted to bind the code in the global scope instead of the hoa_test scope (where hoa_test is just a table as far as Lua is concerned), then we would simply not provide a second argument to module().

Note: About Naming

Note that just because something is named "X" in C++ does not mean that it likewise has to be named "X" in Lua. "X" in C++ could be "Y" in Lua, or "TRUE" in C++ could be called "FALSE" in Lua. However, it is very highly recommended that you keep naming of namespaces, classes, functions, constants, etc. the same between C++ and Lua, for programmer sanity purposes. If you have a really really good reason for providing different names in C++ and Lua, you should provide a comment in the C++ definition explaining why.


Using C++ Constants and Enums[edit]

The Lua programming language does not have the concept of enums, therefore these types are treated as normal read-only integer variables when bound to Lua. Luabind requires that bound constants and enums be bound as integer members of a class. The bounded constants and enums can not be floating-point values, strings, or any other type (it is possible, however, to bind accessor functions that return values of these types instead). However, those constants and enums are not required to be members of that class, and really they don't need to be defined anywhere in the C++ code at all! The following example elaborates by adding to our existing binding code.

// C++ code
const uint32 MY_CONST = 5;

enum MY_ENUM {
    MY_EVAL = 12;
    MY_FVAL = -54;
};

void TestMode::BindToLua() {
    using namespace luabind;

    module(ScriptManager->GetGlobalState(), "hoa_test") [
        class_<TestMode>("TestMode")                  // (A)
            .enum_("constants")                       // (B)
            [
                value("MY_CONST", MY_CONST),          // (C)
                value("MY_EVAL", MY_EVAL),            // (D)
                value("MY_FVAL", MY_FVAL),            
                value("CONSTANT_5", 5)                // (E)
            ]
    ];
}

In (A) we bound the class TestMode to Lua, since we must have a bounded class to make the enums and constants a member of. We'll explore more about class bindings in future sections. Line (B) indicates that we are about to declare bindings for one or multiple read-only values. In (C) we have an example of binding a constant, where the Lua name is provided in the quotes and the second argument is the value of the constant itself. Line (D) binds an enum value. Note that we do not have to bind the type "MY_ENUM" (remember, Lua does not have support for enums). Finally in line (E), we create a new constant out of nowhere and assign it the value of 5. Note that CONSTANT_5 was not defined anywhere in the C++ code.


Executing C++ Functions[edit]

Executing C++ functions from Lua is fairly straightforward. The first thing you have to do is bind the function where you bind your class. To build off of our previous example, you can bind one of TestMode’s member functions like this:

void TestMode::BindToLua() {
    using namespace luabind;

    module(ScriptManager->GetGlobalState(), "hoa_test") [
        class_<TestMode>("TestMode")                  // (A)
            .enum_("constants")                       // (B)
            [
                value("MY_CONST", MY_CONST),          // (C)
                value("MY_EVAL", MY_EVAL),            // (D)
                value("MY_FVAL", MY_FVAL),            
                value("CONSTANT_5", 5)                // (E)
            ]
	    .def(“DoSomething”, &TestMode::DoSomething) // NEW!
    ];
}

This defines the function for Luabind. These first parameter is the name by which Lua will refer to the function, and the second parameter is a pointer to the C++ function you are trying to bind. Luabind does not differentiate between public, private, and protected member functions – they can all be bound and used in Lua as regular class functions.

For this example we will use a ScriptCallFunction to pass object arguments to a Lua function. Our call looks like this:

ScriptCallFunction<void>(funct, A, B);

Where funct is a Luabind object that points to a Lua function, and A and B are TestMode*. Now, inside our Lua file our function header looks like this:

function(A,B)
	print("Do Something!")
end

Lua already knows that A and B are of type TestMode* because we bound that class through Luabind. Therefore, we can immediately access their member functions that we bound in our class binding code. So if we wanted to, we could do this:

function(A,B)
	A:DoSomething()
	B:DoSomething()
end

It’s as simple as that. One catch though is that any variables passed to Lua through Luabind are global in scope. This means that, theoretically, A and B could be manipulated inside the Lua script but outside of the function. Therefore, if you plan on altering A and B but using them elsewhere in your script, go ahead and make local copies of them inside your function and use those. So you script would look like this:

function(A,B)
	a = A
        b = B
	a:DoSomething()
	b:DoSomething()
end

Creating C++ Class Objects[edit]

Advanced Topics[edit]

Writing Script Files[edit]

Note: This section is incomplete and will be written at a later time

- Mostly meant for the editor, and only for the creation of brand new Lua files - Show by example the following:

  - Reading global variables
  - Reading table elements
  - Reading vectors of elements
  - Calling Lua script functions

Modifying Existing Scripts[edit]

Note: This section is incomplete and will be written at a later time

- Meant for taking an existing file and changing some data in that file - Show by example the following: ...