Register forum user name Search FAQ

Gammon Forum

Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the password reset link.
 Entire forum ➜ MUSHclient ➜ Lua ➜ Extending Lua scripting with your own code

Extending Lua scripting with your own code

It is now over 60 days since the last post. This thread is closed.     Refresh page


Pages: 1 2  3  

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Wed 24 Nov 2004 07:28 PM (UTC)

Amended on Sun 14 May 2006 05:47 AM (UTC) by Nick Gammon

Message
Lua has a provision for extending it by writing your own DLLs, thus making anything possible (eg. opening Windows dialogs etc.).

A brief example follows. The first thing to note is that every function called from Lua has an identical calling pattern:


static int function_name (lua_State *L)
  {
  /* do something here */  
  return 0;   /* number of results returned */
  } /* end of function_name */


You use this "lua state" called L (although you can call it anything of course) to access all Lua data and functions, and to return results.

As an example I will write a small function that calculates miles to kilometres, although in practice you would simply do that in Lua.


static int miles_to_km (lua_State *L)
  {
  double miles = luaL_checknumber (L, 1);
  double km = miles * 1.609;
  lua_pushnumber (L, km);
  return 1;   /* one result */
  } /* end of miles_to_km */


The first thing the above does is extract the first argument passed to the function into "miles". If I passed a second argument it would be accessed with (L, 2) instead of (L, 1), and so on.

Then I do some calculations on it.

Finally the result is "pushed" onto the Lua stack (lua_pushnumber). Then the function returns the number 1, to tell Lua that one result is to be returned to the caller.

To make this into a DLL we need a little bit of infrastructure around it. From Lua, when we load a DLL we get to call a single entry point, so if we want to expose more than one function, we need to make a "library" of them. The code below has two functions, the miles-to-kilometre one, and another that calculates the circumference and area of a circle. This second one illustrates returning multiple results:


static int circle_calcs (lua_State *L)
  {
  double radius = luaL_checknumber (L, 1);
  double circumference = radius * 2 * PI;
  double area = PI * radius * radius;
  lua_pushnumber (L, circumference);
  lua_pushnumber (L, area);
  return 2;   /* two results */
  } /* end of miles_to_km */


Next thing we need to do is "register" these functions with Lua, so it knows to add them to its address space. This is done with a small table, that maps function names to their addresses, and then a library call:


static const luaL_reg testlib[] = 
{
  {"miles_to_km", miles_to_km},
  {"circle_calcs", circle_calcs},
  {NULL, NULL}
};


/*
** Open test library
*/
LUALIB_API int luaopen_test (lua_State *L)
 {
  luaL_openlib(L, "test", testlib, 0);
  return 1;
 }


The function luaopen_test will be our entry point for the DLL. When called it adds miles_to_km and circle_calcs to the "test" library. After that you will be able to do:


k = test.miles_to_km (42)
c, a = test.circle_calcs (16)


We will wrap it all up with a couple of include files, and a definition for LUA_API, which exports the function when the DLL is built:

test.c



#ifdef _WIN32
#define LUA_API __declspec(dllexport)
#endif

#pragma comment( lib, "lua.lib" )
#pragma comment( lib, "lualib.lib" )

#include "lua.h"

#include "lauxlib.h"
#include "lualib.h"


#define PI (3.14159265358979323846)

static int miles_to_km (lua_State *L)
  {
  double miles = luaL_checknumber (L, 1);
  double km = miles * 1.609;
  lua_pushnumber (L, km);
  return 1;   /* one result */
  } /* end of miles_to_km */

static int circle_calcs (lua_State *L)
  {
  double radius = luaL_checknumber (L, 1);
  double circumference = radius * 2 * PI;
  double area = PI * radius * radius;
  lua_pushnumber (L, circumference);
  lua_pushnumber (L, area);
  return 2;   /* one result */
  } /* end of miles_to_km */

static const luaL_reg testlib[] = 
{
  {"miles_to_km", miles_to_km},
  {"circle_calcs", circle_calcs},
  {NULL, NULL}
};


/*
** Open test library
*/
LUALIB_API int luaopen_test (lua_State *L)
 {
  luaL_openlib(L, "test", testlib, 0);
  return 1;
 }



The pragma instructions above tell the compiler to link against the lua.lib and lualib.lib files, which expose the entry points in the lua DLLs.

Compile that under Visual C++ 6, and we get a test.dll file output (1.86 Kb, pretty small, huh?).

Copy that dll to the same directory as where MUSHclient is (or our Lua executable is) and we are ready to test.

Start up Lua, or use Lua scripting in MUSHclient.

Next, we load the DLL:


f, e1, e2 = loadlib ("test.dll", "luaopen_test")


This loads our nominated DLL, and tells it to return the exposed entry point luaopen_test as a function.

We can then test how that went:


print (f, e1, e2)  --> function: 0062FB50 nil nil


With a bit of luck f will be a function, and the other two arguments (error messages) will be nil. That means it worked.

However if you see this:


nil The specified module could not be found.
 open


Then that means the function f was not loaded (is nil) and an error message, and error reason (open). This might be because you didn't put the DLL where it could be found (or the lua.dll or lualib.dll were not found either).

Assuming it loaded OK, then you call the function to actually register the routines in the library:


f ()


After you have done that, you can now use the "test" library.


print (test.miles_to_km (40))  --> 64.36
print (test.circle_calcs (15)) --> 94.247779607694 706.8583470577


Note in the second case how we see printed both arguments that were returned. If we want to store them, we could do this:


c, a = test.circle_calcs (15)






[EDIT]

A better way of loading the library is to use "assert" - this checks for a non-nil result. If nil it raises an error giving the error message, if not nil it returns the result. So, the whole load process can be done on one line.

Rather than:


f, e1, e2 = loadlib ("test.dll", "luaopen_test")
if not f then
  error (e1)
end -- if
f ()


Do this:


assert (loadlib ("test.dll", "luaopen_test")) ()


The assert assures us we have a function, and the final set of brackets runs the function, thus installing the library.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Reply #1 on Wed 24 Nov 2004 08:02 PM (UTC)

Amended on Wed 24 Nov 2004 09:18 PM (UTC) by Nick Gammon

Message
Compiling under Cygwin

I compiled exactly the same file above using Cygwin, with this command line:


gcc -shared -o test.dll test.c -L/usr/local/lib/ -llua -llualib


The only real trick was to specify the location of the files liblua.a liblualib.a which are the Cygwin versions of the libraries to link against. To get these files in the first place you need to download and install the Lua source under Cygwin.




[EDIT]

Actually, a better way is this:


gcc  -mno-cygwin -shared -o test.dll test.c lualib.lib lua.lib


This does not require you to compile Lua, because it uses the libraries generated by Visual C++ as part of the DLL compilation. These libraries are in the standalone download from this site (the files you need are lualib.lib and lua.lib).

This method produces a DLL that is 12.9 Kb, compared to 104 Kb using the first method. I think the main reason is the inclusion of the Cygwin libraries, which are not needed in this case.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Poromenos   Greece  (1,037 posts)  Bio
Date Reply #2 on Wed 24 Nov 2004 08:03 PM (UTC)
Message
Hmm... Could the DLL call a Lua function that in turn would do something in MC (effectively a callback)?

Vidi, Vici, Veni.
http://porocrom.poromenos.org/ Read it!
Top

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Reply #3 on Wed 24 Nov 2004 08:14 PM (UTC)
Message
Now for another example, I'll call a Windows API - MessageBox, to display a GUI box on the screen.



#include <windows.h>

#define LUA_API __declspec(dllexport)

#pragma comment( lib, "lua.lib" )
#pragma comment( lib, "lualib.lib" )

#include "lua.h"

#include "lauxlib.h"
#include "lualib.h"


static int messagebox (lua_State *L)
  {
  const char * m = luaL_checkstring (L, 1);

  MessageBox (0, m, "Message", MB_OK);
  return 0;   /* no results */
  } /* end of messagebox */


static const luaL_reg msglib[] = 
{
  {"messagebox", messagebox},
  {NULL, NULL}
};


/*
** Open msg library
*/
LUALIB_API int luaopen_msglib  (lua_State *L)
 {
  luaL_openlib(L, "msg", msglib, 0);
  return 1;
 }



Compile and link this the same way.

Now to test it. Load and install the library:


loadlib ("test.dll", "luaopen_msglib") ()


This loads the library and calls the function in a single line (that's those final parentheses).

Now display a message box:


msg.messagebox "hi there"


A little box pops up with "hi there" in it.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Reply #4 on Wed 24 Nov 2004 08:35 PM (UTC)

Amended on Tue 13 Dec 2005 10:15 PM (UTC) by Nick Gammon

Message
Quote:

Could the DLL call a Lua function that in turn would do something in MC (effectively a callback)?


Yes it can. This will illustrate it. I have taken my "messagebox" code and added another function "callback".

This will take two arguments:


  • A string to be displayed

  • A count in the range 1 to 10


It will display the string "n" times, using ColourNote - a MUSHclient function. This is the callback you are talking about.



#include <windows.h>

#define LUA_API __declspec(dllexport)

#pragma comment( lib, "lua.lib" )
#pragma comment( lib, "lualib.lib" )

#include "lua.h"

#include "lauxlib.h"
#include "lualib.h"


static int messagebox (lua_State *L)
  {
  const char * m = luaL_checkstring (L, 1);

  MessageBox (0, m, "Message", MB_OK);
  return 0;   /* no results */
  } /* end of messagebox */

static int callback_test (lua_State *L)
  {
  const char * m = luaL_checkstring (L, 1);
  int n = (int) luaL_checknumber (L, 2);
  int i;

  if (n < 1 || n > 10)
    luaL_error (L, "count out of range");

    for (i = 0; i < n; i++)
    {
    lua_getglobal (L, "world");   /* world functions */
    lua_pushliteral (L, "ColourNote"); /* ColourNote function */
    lua_gettable (L, -2);  /* get function */
    lua_pushliteral (L, "white");  /* foreground colour */
    lua_pushliteral (L, "blue");   /* background colour */
    lua_pushstring  (L, m);  /* message */
    lua_call (L, 3, 0);  /* call with 3 args and no result */
    }


  return 0;   /* no results */
  } /* end of callback_test */


static const luaL_reg msglib[] = 
{
  {"callback", callback_test},
  {"messagebox", messagebox},
  {NULL, NULL}
};


/*
** Open msg library
*/
LUALIB_API int luaopen_msglib  (lua_State *L)
 {
  luaL_openlib(L, "msg", msglib, 0);
  return 1;
 }



The important part is the part in bold. It pushes the name of the wanted function ("ColourNote") and asks for it from the global address space. Then it pushes the three arguments for ColourNote and does a lua_call. This does the callback.

You would test the above like this:


  loadlib ("test.dll", "luaopen_msglib") ()
  msg.callback ("test", 3)


- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Reply #5 on Wed 24 Nov 2004 09:04 PM (UTC)
Message
Of course, you are not limited to simply calling MUSHclient internal functions. You can call *any* function that is in the same script space, including Lua ones, or any user-written functions that you have already added into the script.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Reply #6 on Tue 13 Dec 2005 09:32 PM (UTC)

Amended on Tue 13 Dec 2005 10:17 PM (UTC) by Nick Gammon

Message
A couple of caveats to the above process. I found that under Cygwin (or indeed standalone) that if the DLLs needed by Lua were not found, you get the error message "The specified module could not be found." when trying to load your DLL.

This is a bit confusing, because you think it can't find "test.dll" (or whatever you have called it).

One way of working this out is to right-click on the DLL and selecting "View Dependencies". This will show which other DLLs are required.

The next thing I wanted to do was make a shared library under Linux, to show how you could also write libraries if you are using Linux instead of Windows.

First, I had to change the first line in the library from:


#define LUA_API __declspec(dllexport)


to:


#ifdef _WIN32
#define LUA_API __declspec(dllexport)
#endif


Otherwise, you get an error message about "__declspec".

(I have done this now in the original post, to save confusion.)

Next, you can compile like this:


gcc -shared -o test.so test.c


This gives a file test.so (shared object). To load and test this library I used this line (after typing "lua" to invoke the stand-alone version):


assert (loadlib ("./test.so", "luaopen_test")) ()
print (test.miles_to_km (40))  --> 64.36
print (test.circle_calcs (15)) --> 94.247779607694 706.8583470577


In this case I used "./test.so" as the path to the library, as I suspect that the current directory was not in the library search path.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Tspivey   Canada  (54 posts)  Bio
Date Reply #7 on Tue 04 Jul 2006 09:12 PM (UTC)
Message
Your post helped me alot, but I have a problem. I'm writing a small wrapper around jfwapi.dll (a dll that is used by jaws (screen reader) to speak text). It works fine inside the lua shell. When I load it into MC, it can't find the dll. It finds the jfw.dll (library), but it can't find jfwapi.dll, which I put inside the mushclient directory. What should I do to tell the library where to find its dll file? trying to statically link the dll in didn't work. I think it has something to do with the current working directory, but if loadlib can find the file, why can't windows find jfwapi.dll in the same place?
Top

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Reply #8 on Tue 04 Jul 2006 09:57 PM (UTC)
Message
You aren't simply getting a problem with loadlib are you? That is disabled by default in the MUSHclient sandbox.

Assuming that is not the problem, I'm not sure the exact way that Windows searches for DLLs. I think it may be specified in the PATH environment variable. To see its value try this:


print (os.getenv ("PATH"))


Amongst other things I get this:


C:\WINDOWS\system32;C:\WINDOWS


Thus, you could try putting the jaws DLL in one of those locations, such as C:\WINDOWS\system32.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Shadowfyr   USA  (1,788 posts)  Bio
Date Reply #9 on Wed 05 Jul 2006 06:45 PM (UTC)
Message
Interesting..

This brings up something else I have been reading up on recently. You know that whole event issue. Seems MFC, ATL and script systems, as well as languages like VB, all "hide" the mechanisms actually in use. What is interesting is that "assert" is providing this:

a = createobject("test.dll") <- Loadlib
b = a.QueryInterface(IID_luaopen_test, myptr)

If b = OK, then myptr contains the function pointer for that call.

Now, there are also functions in ActiveX components, missing from standard dlls, which allow you to enumerate its other interfaces.

What the problem with events seems to be is that MFC, ATL, VB, scripts, etc. all "hide" this though stuff like "assert", but they also assume that you won't be creating objects late, then wanted to have them tell you something. These commands are not just buried, they are missing/ignored completely:

GetConnectionInterface - Gets the ID of a specific connection point.
GetConnectionPointContainer - Returns a pointer to the nest of connection points for a known connection, so you can then get the object it belongs to.
Advise - This is used to tie a function in "your" program/script to the outgoing events.
Unadvise - This disconnects that event.
EnumConnections - This lists the events available in the control.

These are not part of QueryInterface, which is what is needed to do "most" of the stuff you do with objects, which is apparently why they can get by with preventing you from accessing these. And this is the "raw" OLE, that "must" exist in all ActiveX components that support events, so unlike the ATL mess I found before, these "can't" change, without breaking ActiveX itself. The only thing I am not 100% sure of is what happens if you fail to call Unadvise before an object is killed or something like a script reload happens (which can end up leaving an active object in memory). I think useing the scripts own "createobject" function might be a bit more dangerous than we realize. Mushclient is unaware of these things and I am not sure what might happen if a loose object tries to call a function, for which the pointer is no longer valid, when it fires its events. Unless the engine itself automatically kills such objects in a reload... But that might not be a major problem. Since Mushclient would be providing the Advise/Unadvise function, a reload could trigger Mushclient to Unadvise any events for that script/plugin, before reloading the script.

Anyway, having read into a lot of what makes the whole mess work, I found it interesting how this thread tied into the whole setup of how just general DLLs work, never mind ActiveX. Libraries that are not statically linked "must" contain the same basic intefaces to work, and obviously something in the "loadlib" or "assert" process is using QueryInterface.
Top

Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #10 on Wed 05 Jul 2006 07:04 PM (UTC)

Amended on Wed 05 Jul 2006 07:05 PM (UTC) by David Haley

Message
Dynamic linking is a very interesting topic and an extremely powerful technique. QueryInterface etc. is for COM stuff; the real dynamic linking is done by the OS. Basically, the host application (the one asking for the DLL) expects to find certain symbols in the DLL, and the DLL can also expect certain symbols in the host. If any of these symbols are not found, the linking fails.

David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top

Posted by Tspivey   Canada  (54 posts)  Bio
Date Reply #11 on Wed 05 Jul 2006 07:53 PM (UTC)
Message
Basically, I wanted to make the speech interface transparent. put a plugin in the plugin folder, 2 dll files in the mushclient main directory, and that's it. Seems I have to either 1) remove the start in field from the shortcut (which might work, haven't tested - ony until Mushclient changes with cwd, of course), or get people to add the mushclient directory to the path environment variable. Is there any reason for the working directory to change frequently?
Top

Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #12 on Wed 05 Jul 2006 09:17 PM (UTC)
Message
Not sure why the working directory would change, unless Nick does it somewhere in the code.

If all you want to do is guarantee access to the DLLs, you could just stick them into the Windows System32 directory. It's not the cleanest of solutions (because if you remove MUSHclient, the DLLs "litter" the directory) but it would work fairly well.

Lua 5.1 lets you specify where to look for C libraries before including them, which seems like it would help solve your problem. But MUSHclient uses the 5.0 release, if I'm not mistaken. If the compat-5.1 library was used, you could do something similar and the DLLs could be wherever you wanted. (But, you'd have to see if you could convince MUSHclient to load it up.)

David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Reply #13 on Wed 05 Jul 2006 09:51 PM (UTC)
Message
Quote:

Basically, I wanted to make the speech interface transparent. put a plugin in the plugin folder, 2 dll files in the mushclient main directory, and that's it.


And, get them to change the sandbox which by default doesn't allow loadlib. Having done that you could perhaps also add in support for doing a "cd" to make sure the directory is where you think. I'm not sure why you are having these problems, MUSHclient successfully finds the lua DLL and the spellcheck DLL for most people. Sounds fishy. You know that if the jfwapi.dll has a missing dependency then it will not "find" it? So maybe there is yet another DLL missing.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Nick Gammon   Australia  (23,120 posts)  Bio   Forum Administrator
Date Reply #14 on Wed 05 Jul 2006 10:52 PM (UTC)
Message
Quote:

What is interesting is that "assert" is providing this:

a = createobject("test.dll") <- Loadlib
b = a.QueryInterface(IID_luaopen_test, myptr)

...

... MFC, ATL, VB, scripts, etc. all "hide" this though stuff like "assert" ...



This is not all strictly correct, sorry Shadowfyr. For one thing, "assert" is not providing anything, it simply checks for an error return.

eg.


assert (2 + 2 == 4) --> no error
assert (2 + 2 == 5) --> assertion failed!


I am assuming you meant "loadlib is providing this ...".

The next part is you are confusing the COM (Component Object Model) interface with a simple DLL (Dynamic Link Library).

A DLL does not necessarily use COM, and the ones on this post don't.

The example DLL that starts this thread consists of a number of functions, one of which (luaopen_test) is "exported" (made publicly available) by the use of the keyword __declspec(dllexport).

When you load the library like this:


f = loadlib ("test.dll", "luaopen_test")


... 2 conditions need to be met. First is that test.dll is found in the search path (and is a DLL of course) and the second is that the entry point named "luaopen_test" is found.

In this case if f is non-nil then we have been given a function pointer, and we can now call it:



f ()


This executes the function body which in this case does this:


luaL_openlib(L, "test", testlib, 0);


What this does is add the items in the testlib table to a lua table called "test" in the Lua address space.




COM does similar things but in their own way. Like the Lua example, a DLL using COM would have an internal table of functions, and this is what gets used when you do QueryInterface - it finds if a function exists. However this is by function number (eg. function 1, function 2, etc.).

You also have to use IDispatch.GetIDsOfNames to map a name to an ID. I am not a great expert on COM, but I think you need to use the generic interface (IUnknown) to find IDispatch, and then (if found) use IDispatch interface GetIDsOfNames to eventually map a name to an ID.

Finally you can retrieve that interface. A lot more complicated than the Lua way (or using a straight DLL entry point).

Quote:

These are not part of QueryInterface, which is what is needed to do "most" of the stuff you do with objects, which is apparently why they can get by with preventing you from accessing these


I'm not totally sure what you mean by this, but in my example the "raw" functions (like miles_to_km) are not directly exposed (the 'static' keyword makes sure of that), so you cannot directly access miles_to_km from the DLL.

Thus, this would fail:


f = loadlib ("test.dll", "miles_to_km")


This is by design, as in Lua you don't want the pure function, what you really want is the function to be added to the Lua address space, which is what the one exposed function (luaopen_test) does for you.

In a similar way, QueryInterface in COM gives you back the interface, thus it is QueryInterface that needs to be exposed in the DLL, not the raw function.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.


124,625 views.

This is page 1, subject is 3 pages long: 1 2  3  [Next page]

It is now over 60 days since the last post. This thread is closed.     Refresh page

Go to topic:           Search the forum


[Go to top] top

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.