Expression parser

Posted by Nick Gammon on Mon 13 Sep 2004 10:02 PM — 65 posts, 298,847 views.

Australia Forum Administrator #0
Introduction

I am pleased to present a runtime "expression evaluator" (parser) which is intended to allow you to defer to run-time calculations (like, how many seconds a weapon takes to reach its target).

The intention here is that, rather than hard-coding things like:


  power_of_weapon = str * 3 + dex / 2


... you could put them into "area files" (or the equivalent) and have them evaluated by the MUD server.

This is done by processing a "string" expression (ie. text) and producing a result. The calculations are done using floating-point numbers (doubles) to give maximum accuracy and allow for fractional results.


  Example:  2 + 2
  Result: 4
  
  Example:  log10 (1234)
  Result: 3.09132


The parser builds up results "on the fly" reporting syntax problems by throwing an exception.


  Example: 1 + 3 + )
  exception: Unexpected token: )


Run-time errors (principally "divide by zero") are also thrown as an exception:


  Example: 2 / 0
  exception: Divide by zero




Supported operations

Basic arithmetic: + - / *


  Example: 1000 + 123
  Result: 1123
  
  Example: 44 * 55
  Result: 2420


Expressions are evaluated from left-to-right, with divide and multiply taking precedence over add and subtract.


  Example:  2 + 3 * 6
  Result: 20   (the multiply is done first)


Parentheses can be used to change evaluation order.


  Example:  (2 + 3) * 6
  Result: 30  (the add is done first)


Whitespace is ignored. However 2-character symbols (like ==) cannot have imbedded spaces.



Symbols

The parser supports an unlimited number of named symbols (eg. "str", "dex") which can be pre-assigned values, or assigned during use.


  Example (in C++):
  
    Parser p;   // create a parser instance
    p ["str"] = 55;   // assign value to "str"
    p ["dex"] = 67;   // assign value to "dex"
    double result = p.Evaluate ("str + dex * 2");   // use in expression
    double str = p ["str"];   // retrieve value of "str"
  
  Example: a=42, b=6, a*b
  Result: 252


There are two built-in symbols:


   pi = 3.1415926535897932385
   e  = 2.7182818284590452354  


Symbols can be any length, and must consist of A-Z, a-z, or 0-9, or the underscore character. They must start with A-Z or a-z. Symbols are case-sensitive.



Assignment

Pre-loaded symbols, or ones created on-the-fly, can be assigned to, including the standard C operators of +=, -=, *= and /=.


  Example: a=42, a/=7
  Result: 6
  
  Example: dex = 10, dex += 22
  Result: 32




Comparisons

You can compare values for less, greater, greater-or-equal etc. using the normal C operators.


  Example: 2 + 3 > 6 + 8
  Result: 0  (false)
  
  Example; 2 + 3 < 6 + 8
  Result: 1  (true)


The comparison operators are: <, <=, >, >=, ==, !=

These are a lower precedence than arithmetic, in other words addition and subtraction, divide and multiply will be done before comparisons.



Logical operators

You can use AND, OR, and NOT (using these C symbols: &&, ||, ! )


  Example: a > 4 && b > 8     //  (a > 4 AND b > 8)
  Example: a < 10 || b == 5   //  (a < 10 OR b == 5)
  Example: !(a < 4)           // NOT (a < 4)



These are a lower precedence than comparisons, so the examples above will work "naturally".



Other functions

Various standard scientific functions are supported, by using:

function (argument) or function (argument1, argument2)



Single-argument functions are:

abs acos asin atan atanh ceil cos cosh exp exp floor log log10 sin sinh sqrt tan tanh

These behave as documented for the C runtime library.


  Example: sqrt (64)
  Result: 8


Note that functions like sin, cos and tan use radians, not degrees. To convert to radians, take degrees and multiply by pi / 180 (the value of pi is built-in).


  Example: sin (45 * pi / 180)
  Result: 0.7071


Three other functions which are not directly in the standard library are:

int

int (arg) <-- drops the fractional part


  Example: int (1.2)
  Result: 1

  Example: int (-1.2)
  Result: -1


rand

rand (arg) <-- returns a number in the range 0 to arg

The rand function returns an integer (whole number) result, it will never return arg itself.

eg. rand (3): might return: 0, 1 or 2 (and nothing else)

percent

percent (arg) <-- returns true (1.0) arg % of the time, and false (0.0) the rest of the time

eg. percent (40) will be true 40% of the time


Two-argument functions are:

min (arg1, arg2) <-- returns whichever is the lower
max (arg1, arg2) <-- returns whichever is the higher
mod (arg1, arg2) <-- returns the remainder of arg1 / arg2 - throws an exception if arg2 is zero
pow (arg1, arg2) <-- returns arg1 to the power arg2
roll (arg1, arg2) <-- rolls an arg2-sided dice arg1 times.


  Example: roll (2, 4)  (in other words 2d4)
  Result: 10



If test

Finally you can do "if" tests by using the "if" function.

if (test-value, true-value, false-value)


  Example:  if (a > 5, 22, 33)
  Result: if a > 5, returns 22
          if a <= 5, returns 33


User-written functions

You can easily expand the inbuilt functions by adding more to the source code using the existing ones as an example.




Distinguishing symbols from functions

In order to allow for the case where people may need to use a symbol that happens to be the name of an inbuilt function (eg. abs = 5, pow += 3), the parser distinguishes functions from symbols by looking ahead for the parenthesis following the function name. This distinction is required otherwise if someone added a user-function one day that happened to be the name of a symbol used in many places, considerably confusion would result.


  Example: pow = pow + 3 
  Result: 3    (pow is a symbol)
  
  Example: pow (4, 3)
  Result: 64   (4 to the power 3)
  
  Example: pow = pow (4, 3), pow = pow + 1
  Result: 65    


In this example we are mixing pow (4, 3) - a function - with pow as a symbol name. Confusing, perhaps, but it works consistently.


  Example: abc (20)
  Result: (exception) Function 'abc' not implemented.




Using in a program

To use the parser in C++ code, use it like this:


  #include "parser.h"
  
  Parser p ("2 + 3 * 6");
  double result = p.Evaluate ();


The expression is not evaluated until the Evaluate function is called. If you need change the expression to be evaluated you can do it by passing a string to Evalute, eg.


  Parser p ();
  double result = p.Evaluate ("2 + 3 * 6");


Variables can be fed in, or retrieved, using operator[], like this:


  Parser p ();
  p ["a"] = 22;  // value for symbol "a"
  p ["b"] = 33;  // value for symbol "b"
  double result = p.Evaluate ("c = a + b");
  double c = p ["c"];   // retrieve symbol "c"


This effectively lets you not only return a result (the evaluated expression) but change other symbols as side-effects.



Using in fight calculations etc. in a MUD

Basically follow the guidelines above for doing calculations.

You could "feed in" the relevant variables from the player's stats (eg. str, dex, wis).

eg.


  Parser p ("str * 3 + dex / 3");

  p ["str"] = player->str;
  p ["dex"] = player->dex;

  double result = p.Evalute ();


Example:

A fighter fixes an arrow to his bow. He takes between 2 and 8 seconds, depending on his strength:


  time_taken =  8 - 6 * (str / max_str)


In this case if his normalised strength is the maximum (1) then the calculation will effectively be:


  8 - (6 * 1) = 2 seconds


However if his strength is zero, the calculation will be:


  8 - (6 * 0) = 8 seconds


To throw in a bit of luck you might roll a dice. For example:


   time_taken =  11 - (6 * (str / max_str)) - roll (1, 3)


This is rolling a 3-sided die once, giving a result in the range 1 to 3.

Thus the worst case would be 10 seconds (11 - 0 - 1) and the best case 2 seconds (11 - 6 - 3).

Maybe 20% of the time he has really bad luck, and takes another 5 seconds:



   time_taken =  11 - (6 * (str / max_str)) - roll (1, 3) + (5 * percent (20))


The "percent (20)" part will return true (that is, 1.0) 20% of the time, which will then be multiplied by 5 to add on another 5 seconds.

To use other variables you might add in dexterity, like this:


  time_taken =  8 - (6 * (str / max_str)) - (dex / max_dex)


Since "dex / max_dex" will be in the range 0 to 1, then this might shave off another second for very dextrous players. For players with 50% dexterity it would shave off half a second.




Downloading

The parser is available for anyone to use, at no charge, It is written in C++ and uses STL (the Standard Template Library).

You can download it from:



File size: 8 Kb (source code only). (Now out-of-date).

The latest version is now available on GitHub:

http://github.com/nickgammon/parser

If you have git installed, you can do this to get a copy:

git clone git://github.com/nickgammon/parser.git





Supported compilers


It has been compiled without errors or warnings under:

  • Cygwin (Windows) - gcc (GCC) 3.3.1 (cygming special)
  • Microsoft Visual Studio 6.0
  • Linux - gcc (GCC) 3.2.2 20030222 (Red Hat Linux 3.2.2-5)
  • Mac OS/X 10.3.5 (Panther) - gcc (GCC) 3.3 20030304 (Apple Computer, Inc. build 1495)


To compile the test program under Cygwin, Linux or Mac OS/X, just run the Makefile (ie. type "make"). To compiler under Visual Studio open the project file parser.dsw.




Using

If you compile as described above you will have a test program (parser, or parser.exe under Windows) which you can run and enter expressions into to test. Note that under Visual C++ it seems to be "a line behind" due to a bug in the standard library. In other words, you need to type 2 expressions before it will evaluate the first. This is only a problem in the test program which you could fix by changing the line:


      getline (cin, inputLine);


To use in your own (C++) programs, simply do this:


#include "parser.h

// and further on

Parser p ("2+2");	// or whatever

double result = p.Evaluate ();


The test program test.cpp can be used as an example of using it in another program.
Amended on Mon 15 Feb 2010 08:40 PM by Nick Gammon
Australia Forum Administrator #1
I'll just emphasise that the parser presented above is a completely stand-alone piece of code.

It does not require any additional libraries or files to be installed (such as flex, bison, yacc or whatever).

The exception is that you naturally need the "standard" C++ library, including the Standard Template Library, however that is supplied as standard with the compilers mentioned above, and hopefully with all modern compilers.

To use it, you simply add two files to your project:

  • parser.cpp - the implementation of the expression parser (622 lines). You would add this to your project (makefile).
  • parser.h - the header file, which you would use in any source files that need the parser (133 lines).

Amended on Mon 13 Sep 2004 10:32 PM by Nick Gammon
Australia Forum Administrator #2
Timing test

It is always important to me that code works quickly. I timed the expression parser described above, to evaluate the expression:


a=10, b=20, a + b


It did this 10,000 times in under a second, on my 2.4 GHz Pentium processor (under Windows NT 4).
Australia Forum Administrator #3
Comma operator

Some of the examples above use the "comma operator" without really explaining it.

In C, you can separate an expression into parts by using the comma operator. eg.


  a=10, b=20, a + b


This effectively breaks the expression into sub-expressions, as the comma operator has the lowest priority. This means that everything between the commas is done first.

However, as the expression is evaluated left-to-right, what the above example does is:

  • Assign 10 to a
  • Assign 20 to b
  • Add a to b


Thus, the result of that expression is 30.
#4
Hello Nick, forum people,

The Expression parser is a nice piece of code, easy to use, just what I was looking for. I have successfully used it in previous versions of GCC, but unfortunately, the code fails to compile in the DJGPP (i.e., MS-DOS) version of GCC v.3.4.4
The compiler gives me this error message:
"
parser.cpp:229: error: no matches converting function `abs' to type `double (*)(double)'
c:/djgpp/include/stdlib.h:44: error: candidates are: int abs(int)
c:/djgpp/bin/../lib/gcc/djgpp/3.44/../../../../include/cxx/3.44/cstdlib:123: error: long int std::abs(long int)
"
i.e. near this line of code:
#if defined(WIN32)
/* 227 */ STD_FUNCTION (abs<double>);
#else
/* 229 */ STD_FUNCTION (abs);
#endif
I tried forcing the compiler into that first definition of 'abs' by manipulating the preprocessor directives, which gave me this:
"
parser.cpp: In function `int LoadOneArgumentFunctions()':
parser.cpp:227: error: expected primary-expression before "double"
parser.cpp:227: error: expected `;' before "double"
"

I don't know enough about programming in c++ to dare trying to solve this myself (as a matter of fact, I don't know the second error message means).
For now, I simply commented lines 227 and 229. But of course I would prefer the abs function to work. So, is there anyone who knows the solution to this?
Thanks in advance,

Peter.
USA #5
If I'm not mistaken, the 'correct' form of abs to use is fabs, from math.h. Try including math.h and changing line 229 to "STD_FUNCTION (fabs);".

See:
http://www.delorie.com/djgpp/doc/incs/math.h
for the function prototypes.


Of course, this won't work quite as you'd expect, since it'll register the function fabs, not abs.

What you actually want is something more like this:
//instead of "STD_FUNCTION(fabs);", use this:
OneArgumentFunctions ["abs"] = fabs;


Let me know if this fixes your problem. Don't forget to include math.h in there somewhere.
#6
Dear Ksylian,

Your solution worked, thanks!
Some people might be interested to hear about the following experience under Cygwin. The parser compiles fine, but an abs function is not registered as such. This is the output from a piece of testing code:
"
> evaluate abs(-1)
Could not evaluate expression 'abs(-1)': Function 'abs' not implemented.
"
(note that an exception of type runtime_error has to be caught if you don't want the application to crash with an uninformative "contact the developer" error message)

The cygwin compilation goes into the #ifdef WIN32 block (in LoadOneArgumentFunctions() ) and registers abs<double>, which has the effect that under cygwin, you can call an abs function in the resulting executable, but only as "abs<double>(-1)" (for example). But I suppose the abs function that gets called, is not the one that is meant in the implementation, since it is the version that takes and returns int arguments:

"
> evaluate abs<double>(-1.3)
1
"

I suppose that the line "STD_FUNCTION (abs<double>)" works in Visual C++, but obviously confuses the GCC compiler, which is why I would argue to replace that #ifdef WIN32 to a check of definition that is more specific, i.e. #ifdef _MSC_VER . The correction that you, Ksylian, proposed, works for all GCC compiler versions, DJGPP, Cygwin, and all the others, I assume. Therefore, to summarize, I propose to change the code in lines 221-225

#ifdef WIN32  
  STD_FUNCTION (abs<double>);
#else
  STD_FUNCTION (abs);
#endif  

to

/* #ifdef WIN32 */ #ifdef _MSC_VER
  STD_FUNCTION (abs<double>);
#else
 /* STD_FUNCTION (abs); */ OneArgumentFunctions["abs"]=fabs; 
#endif  

Actually, I suppose that the latter solution should work on all C++ compilers, which would maybe allow to get rid of that conditional compilation alltogether, assuming that there are no speed tradeoffs.

Another note: I used the wrong reference to line numbers in my previous post because I had changed some code in order to be able to compile under GCC 3.4.4 . More specifically, the DoMax and DoMin functions would not compile, apparently due to strict type checking (ambiguous overloading). This is why I propose to replace lines 171-179 in the original file:

const double DoMin (const double arg1, const double arg2)
  {
  return min<double> (arg1, arg2);
  }

const double DoMax (const double arg1, const double arg2)
  {
  return max<double> (arg1, arg2);
  }

to the rather obvious and maybe dumb

const double DoMin (const double arg1, const double arg2)
  {
  /*  return min<double>(arg1, arg2);*/ return (arg1 < arg2 ? arg1 : arg2) ;
  }

const double DoMax (const double arg1, const double arg2)
  {
  /*  return max<double> (arg1, arg2); */ return (arg1 > arg2 ? arg1 : arg2) ;
  }


Of course some people might like a solution in terms of conditional compilation better, but again, I suppose that the above code works on all compilers, assuming no speed issues.

Hope this is useful to someone. Thanks again, Ksylian.

Regards,
Peter.
USA #7
Glad to be of help. :)
Quote:
Actually, I suppose that the latter solution should work on all C++ compilers, which would maybe allow to get rid of that conditional compilation alltogether, assuming that there are no speed tradeoffs.
I think you're correct. Windows also defines fabs in math.h, so using that would probably removed the need for conditional compilation. The only speed tradeoff is that you are casting everything to doubles even if there's an int version of the function, but since the expression parser works with doubles anyhow, that shouldn't be a problem.
Quote:
(Speaking of DoMin, DoMax) Of course some people might like a solution in terms of conditional compilation better, but again, I suppose that the above code works on all compilers, assuming no speed issues.
I'm a little surprised that code didn't compile; what errors were you getting? But yes, that code should work everywhere, and there's no reason to believe it's less efficient than the provided min<>, max<> routines.
#8
I'm stupid. If I had looked into the exact error messages and the code more carefully, I would have known what was going on with this min/max problem...
This is the error that I get when compiling DoMin (idem for DoMax, of course):
Quote:

/cygdrive/c/src/dots2002/parser.h:7: warning: ignoring #pragma warning
/cygdrive/c/src/dots2002/parser.cpp: In function `const double DoMin(double, dou
ble)':
/cygdrive/c/src/dots2002/parser.cpp:173: error: call of overloaded `min(const double&, const double&)' is ambiguous
/cygdrive/c/src/dots2002/parser.h:12: note: candidates are: T min(T, T) [with T = double]
/usr/lib/gcc/i686-pc-mingw32/3.4.4/include/c++/bits/stl_algobase.h:151: note: const _Tp& std::min(const _Tp&, const _Tp&) [with _Tp = double]

parser.h:7 is (// disable warnings about long names) #pragma warning( disable : 4786) - but I don't think this affects anything.

parser.cpp:173 is :

const double DoMin (const double arg1, const double arg2)
  {
/*173 */   return min<double>(arg1, arg2);
  }

parser.h:12 is around

  // under Visual C++ 6 these do not seem to exist
  template <typename T> 
  inline T min (const T a, const T b)
    {
    return (a < b) ? a : b;
    } // end of min

this is in turn subject to the #ifdef WIN32 compilation condition (see comment in header: "// under Visual C++ 6 these do not seem to exist"). So the point seems to be, again, that cygwin subjects itself to a condition (being win32) which was only meant for visual C++. If I change the line #ifdef WIN32 (line 4 in parser.h) to #ifdef _MSC_VER, everything compiles just fine, with the original DoMin and DoMax code.

Another mystery solved...
USA #9
Ohhh, that makes a lot of sense. The min was being defined twice!

Changing it to _MSC_VER is a good fix and is probably better in the long run. Ah, who'd'a'thunk that compiling between Windows and Cygwin would make so many problems. It's funny, because I'm fairly sure I compiled stuff under Cygwin using WIN32 ifdefs and they weren't considered true.

Another good fix would be to qualify the use of STL's min with std:: - in other words, change line 173 to:
return std::min<double>(arg1, arg2);
This way, there is no ambiguity as to which min you're calling.

As for min<> not existing in Visual Studio... I don't know about 6, but in .NET 2003 min<> is defined in <algorithm>. I don't remember offhand if Nick included that in his source; perhaps that would fix the need for having to redefine the min<> function.
#10
A few comments:

In parser.h:

>#include <math.h>
>#include <time.h>

Uh oh. Why not use the more appropiate C++ files cmath and ctime? You could easily mess up programs that use the C++ header files names with the old C file names.

>using namespace std;

This is fine, but does it really have to be in the header file? You have now intruded on my global namespace, which doesn't make me a happy programmer. Weird effects could easily come from this.

#define MAKE_STRING(msg) \
   (((ostringstream&) (ostringstream() << boolalpha << msg)).str())


That just looks wrong. To me, you are just asking for the temporary to be destroyed before you can get to the data.

const double Evaluate (const string program);  // get result


Why not a reference? std::string is not cheap to copy.

Parser.cpp:

First, I see you using std::rand(), but you don't include cstdlib. Or is std::rand() not in cstdlib?

Down the file, I see dubious floating point comparisons. I suggest reading this: http://www.parashift.com/c++-faq-lite/newbie.html#faq-29.17
This explains a potential problem as well as a good solution.

USA #11
The MAKE_STRING macro works just fine, actually.

Where do you see std::rand()? Nick never uses it explicitly in parser.cpp. Since he doesn't qualify it with std::, it'll pick whatever it finds from the global namespace or from the 'used' std namespace.
#12
>So the point seems to be, again, that cygwin subjects itself to a condition (being win32) which was only meant for visual C++.<

Are you sure Cygwin defines WIN32? I tried this program on my Cygwin:


#include <iostream>

int main( void )
{
#ifdef WIN32
   std::cout << "Hello" << std::endl;
#endif
}


I got no output when I ran it under the Cygwin g++ 3.3.3. (Although, I then used the -mno-cygwin option and got the response "Hello" from the program.) I haven't tested this program under a compiler that was compiled under Cygwin yet, though...
USA #13
He's actually using DJGPP, I think, which would probably explain why WIN32 is getting defined.
#14
>The MAKE_STRING macro works just fine, actually.

Not on my system:


#include <iostream>
#include <sstream>

#define MAKE_STRING(msg) \
 (((std::ostringstream&)(std::ostringstream()<<msg)).str())

int main( void )
{
   std::cout << MAKE_STRING("Hi") << std::endl;
}


Using Cygwin with a g++ of 3.3.3, I get "0x435000".

I am not 100% sure what it wrong. I am trying to figure it out, but g++ seems to be giving me very different results when I try to create a similar test case.

>Where do you see std::rand()?

He doesn't. I just call rand() by the name of std::rand() when using C++. (It's a matter of habit.)

>Since he doesn't qualify it with std::, it'll pick whatever it finds from the global namespace or from the 'used' std namespace.

Sure. I was just commenting on how he forgot to include the standard header file cstdlib which includes the definition of std::rand().
USA #15
Quote:
Using Cygwin with a g++ of 3.3.3, I get "0x435000".
Oh... I remember getting something funny like that when using string constants in certain situations. What's happening is that your code is printing the constant as an integer for some reason. If you make it into a string constructor - string("hello") - that should fix it. But it wasn't a problem specific to MAKE_STRING. I had it in other places as well.
USA #16
Is it that? or are you printing the reference to the string?
USA #17
Sorry. I meant printing the address of the pointer; a string constant is after all just an integer pointing to the first byte of the string and going until the first \0.
#18
About some posts ago... Raz is right about the WIN32 definition in cygwin.
First, to clear up some confusion, I use gcc v3.4.4 both under DJGPP and cygwin. I did a little test.
The DJGPP compilation does _not_ define WIN32. Neither does g++ under cygwin _without_ -mno_cygwin option. g++ under cygwin _with_ the -mno_cygwin compilation option _does_ define WIN32.
I didn't expect that it would depend on this switch. In any case, I cannot compile without the -mno_cygwin option, because I use the allegro game programming library, which I am supposed to use with the -mno_cygwin switch.
Regards
P
China #19
nice piece of work

in parser.cpp Line 414: GetToken() should be GetToken (true)

so does the GetToken() in Line 506, the same problem.

Thanks, Nick. It helps me a lot!!
Australia Forum Administrator #20
Thanks for all the comments, it is nice to see people using it.

Quote:

What you actually want is something more like this:
//instead of "STD_FUNCTION(fabs);", use this:
OneArgumentFunctions ["abs"] = fabs;


I have amended it to use this suggestion, use the fabs function but internally refer to it as abs.

Quote:

...
to the rather obvious and maybe dumb


const double DoMin (const double arg1, const double arg2)
{
/* return min<double>(arg1, arg2);*/ return (arg1 < arg2 ? arg1 : arg2) ;
}



Why not? I have amended it to inline the DoMin and DoMax.

By doing both of these changes most of the WIN32-specific stuff has gone, so whether or not WIN32 is defined should be almost irrelevant.

Quote:

In parser.h:

>#include <math.h>
>#include <time.h>

Uh oh. Why not use the more appropiate C++ files cmath and ctime? You could easily mess up programs that use the C++ header files names with the old C file names.


OK, did that.

Quote:

>using namespace std;

This is fine, but does it really have to be in the header file?


Removed it altogether and put std:: in front of things that needed it.

Quote:

#define MAKE_STRING(msg) \
(((ostringstream&) (ostringstream() << boolalpha << msg)).str())

That just looks wrong.


I can reproduce your test results, although it seems to work OK in the parser. Anyway, I have removed the macro and inlined the couple of places it was used.

Quote:

const double Evaluate (const string program); // get result

Why not a reference? std::string is not cheap to copy.


Not sure why I did that. Hope there wasn't a good reason as I have changed it to use the reference.

Quote:

Down the file, I see dubious floating point comparisons.


What ones exactly? Compare to zero? I think that is defined to work in all cases. Can you give a list of the ones you are doubtful about?

Quote:

Sure. I was just commenting on how he forgot to include the standard header file cstdlib which includes the definition of std::rand().


I didn't forget, I am using the standard library rand, not std::rand. Whether that is bad or not I'm not sure.

Quote:

in parser.cpp Line 414: GetToken() should be GetToken (true)

so does the GetToken() in Line 506, the same problem.


Not sure about that. If I change both of those the argument becomes redundant because all calls are using the argument of true. If you change it as you suggest, this fails:

-+1

This is a valid expression, we are taking a unary minus of the signed number +1.

The parser as supplied will process that, with your suggestion it won't.




Updated version

The version with the above improvements incorporated is available from the same location:


http://www.gammon.com.au/files/utils/parser.tgz


It might be worth putting in a plug for Lua at this point. Lua is a full scripting language with a small footprint, designed to compile on all standard C compilers. It is available for use at no cost.


http://www.lua.org/


My parser has a very small footprint and is suitable for its designed purpose, evaluation simple expressions at runtime. Lua is larger but supports full scripting, including extending with your own C routines. I you need something more powerful check out Lua. :)
Amended on Mon 24 Oct 2005 10:39 AM by Nick Gammon
Australia Forum Administrator #21
There is a subtle problem in the parser, in the getrandom function, which no-one has commented on. :)

It initially read:


const int getrandom (const int x)
{
long n;
 
 if (x <= 0)
   return 0;
 n = rand ();
 return (n % x);
}   // end of getrandom


Now this works, in a sense, and is the way a lot of people implement random numbers in a range. Simply take the output from the random number generator and take its modulus to the desired range. Thus if you supply 3, you will get 0, 1, or 2 as the result, which is what is documented.

However the problem is that if you want, say, penny flips, so you call it as getrandom (2).

What the modulus is effectively doing is discarding all bits from the random number, except the low-order bit. Now the low-order bit is not the full information returned by the pseudo-random generator, and therefore is not necessarily as "random" as the whole number.

A second problem is that if someone expects a number in a large range (eg. rand (100000)) they won't get it because the largest number returned by rand is 32767.

A better method might be:


const int getrandom (const int x)
{
  if (x <= 0)
    return 0;
  return (double) rand () / (double) (RAND_MAX + 1) * (double) x;
}   // end of getrandom


What this does is take the entire returned random number, and then divide by the greatest one it can get (RAND_MAX), plus 1, to get a number in the range [0,1). That is, including 0 but excluding 1.

Now we multiply this by the argument to get a number in the range 0 to the argument, but excluding the argument.




For even better random numbers you may want to consider other algorithms, such as the Mersenne Twister "A 623-Dimensionally Equidistributed Uniform Pseudo-Random Number Generator".
Amended on Mon 24 Oct 2005 09:03 PM by Nick Gammon
Australia Forum Administrator #22
If you are confident RAND_MAX on your system is 32767 then you can save a division by writing this:


return (double) rand () * 3.0517578125000000e-5 * (double) x;



Some quick research shows this confidence may be misplaced, on Visual C++ it is indeed defined as 32767 (0x7FFF), however on Unix it seems to be 0x7FFFFFFF, in which case the line would read:


return (double) rand () * 4.6566128709089882e-10* (double) x;


To be general, better leave it at RAND_MAX.
USA #23
Just wanted to say thanks, this thing worked wonderfully for my small script interpreter for my Comp. Sci. 1 lab project.

Nice work, as always :)
Australia Forum Administrator #24
Thank you. :)

Further to my comments about random numbers, I see from the Lua source that they do something similar, with an extra twist ... I'll reproduce the relevant line from their code plus comments:



  /* the `%' avoids the (rare) case of r==1, and is needed also because on
     some systems (SunOS!) `rand()' may return a value larger than RAND_MAX */
  double r = (double)(rand()%RAND_MAX) / (double )RAND_MAX;


(I changed lua_Number in the source to double to make it clearer).

This is using the % operator (modulus) to coerce the results from rand() into 0 to RAND_MAX minus 1, and then divide by RAND_MAX to get a floating-point number in the range 0 to 1, but excluding 1.

After that, if you want an integer in the range 1 to u (where u is the upper limit, and 1 is the lower limit) you do this:


result = (int) floor (r * u) + 1;


#25
This is exactly what I wanted! Thanks! You've saved me weeks of work.

The only problem is that my project is in C so I had to convert the code to C. Not a problem at all.

I added ^ for power and % for modulo division. I also added a frac() function to get the fractional part of a number.

The code is now part of a programmable syringe pump, where programming is done in a language similar to Basic. The addition of expression parsing makes the pump much more powerful and versatile.

Again, thanks for your parser!
Australia Forum Administrator #26
Glad it was helpful.
USA #27
Hi,

I just discovered your parser yesterday and found that it worked pretty well for a project I am working on. However, I discovered a problem that I have attempted to address, but I don't know if my solution is necessarily totally correct. I would appreciate another pair of eyes looking at it, and if warranted, perhaps you can update your parser with the correction.

The problem is that the parser fails to recognize values <1 without a leading 0. For example, ".5" is not recognized as a number, while "0.5" is fine. Since I am processing data from a source that I do not control, I want to reduce the risk of failure due to malformed numbers. My solution is below (note the additional check for a decimal point as the first character):


  // look for number
  if ((!ignoreSign &&
        (cFirstCharacter == '+' || cFirstCharacter == '-') &&
        isdigit (cNextCharacter)
      )
      || isdigit (cFirstCharacter)
      // allow decimal numbers without a leading 0. e.g. ".5"
      // Dennis Jones 01-30-2009
      || (cFirstCharacter == '.' && isdigit (cNextCharacter)) )


Is this the best way to solve the problem? Anything else I need to do?

Thanks,
Dennis
Australia Forum Administrator #28
That looks OK to me. Does it work properly?
USA #29
Oh, and by the way, you might like to know that I am using the parser successfully (after a few minor modifications) with C++Builder (Borland/CodeGear), in case anybody wants to know if it works with C++Builder 5/6. I haven't tried with CB2007 or CB2009 yet, but I suspect they will work fine too.

The required changes are as follows:

In parser.cpp (at the top, just after the "parser.h" include):


// Support for Borland C++Builder 5/6
// Dennis Jones, Grass Valley Software, 01-30-2009
#if defined(__BORLANDC__)
  // make methods from <cmath> available without requiring std:: scope resolution
  using namespace std;

  // define NaN
  #include <math.hpp>
  #if (__BORLANDC__ < 0x560)
  // BCB5 doesn't define NaN
  static const Extended NaN = 0.0 / 0.0;
  #endif  // (__BORLANDC__ < 0x560)
#endif  // __BORLANDC__


Note: NaN was needed for a couple of functions I added. If you don't need NaN, you can leave out the definition.

Dennis
USA #30
Nick,

Yes, it *seems* to work fine. I just wasn't sure if I was missing anything obvious or asking for trouble by making the change.

- Dennis
#31
Nick:

Stumbled upon this recently while searching for an expression parser. Sweet piece of code!

I do have one question for you, though. What I am writing is a data extraction tool that offers generic filtering capability based on the expression. I can now easily create and parse filters like
CustID == 76555
by adding all SQL field names as their own variables. That's just awesome!

However, one thing is missing -- the ability to evaluate string expressions, as in
CountryCode == "AUS"
or
Last_Name < "C"
(The latter makes it easy to generate test data with a smaller subset of data.)

I understand that string expression support would bloat the code in terms of storage space, but I can see this as a very useful feature to add.

Anyway -- my question is this: Has anyone ever asked about adding string comparison support, or have you ever looked into extending the code for this? I think it won't be too bad to extend the existing code by adding a data type field and a bunch of IF checks for the string fields, and even some simple string handling functions (LEFT, RIGHT< etc.), but I'd hate to re-invent the wheel if I can avoid it, and Lua seems a bit overkill.
Australia Forum Administrator #32
No-one has asked, and I haven't tried it. The considerable complexity would arise from having to have data types (ie. numbers, strings) where it doesn't currently have that. Then when you do things (like testing for less-than) you would need to switch between numeric or string compare, or indeed throw an error if not possible.

If you want to go that far, I would use Lua. It is extremely simple to imbed in an application (particularly C or C++), and you can always not load unwanted libraries (eg. the io or package libraries).

Then you only have to get your variables into/out of Lua address space, which it is specifically designed to do with a minimum of hassle. See my examples here:

http://www.gammon.com.au/forum/bbshowpost.php?id=6400

Something like your example could be:


if string.sub (Last_Name, 1, 1) == "C" then   -- whatever


This also gives you the advantage that your application (the data extraction tool) can then lever off things Lua offers, for example regular expression matching.

So for example, you could test if the last name started with one of a set of characters, eg.


if string.match (Last_Name, "^[CbvX]") then   -- whatever


Also, for documentation, refer your users to the Lua documentation.
Amended on Wed 11 Mar 2009 08:39 PM by Nick Gammon
#33
I get an error if I try to parse something like:

(2+3)-1
exception: Unexpected text at end of expression: -1

(2+3) +4-5
exception: Unexpected text at end of expression: +4-5

It seems to be something to do with how you handle RHPAREN.
Australia Forum Administrator #34
Around line 515, change:


    case LHPAREN:
      {
      double v = CommaList (true);    // inside parens, you could have commas
      CheckToken (RHPAREN);
      GetToken ();                    // eat the )
      return v;
      }


to:



    case LHPAREN:
      {
      double v = CommaList (true);    // inside parens, you could have commas
      CheckToken (RHPAREN);
      GetToken (true);                    // eat the )
      return v;
      }


That will fix it.
Australia Forum Administrator #35
By way of explanation, without the change, it was treating the -1 after the bracket as a single token < -1 > not two tokens namely < - > and < 1 >. Thus it was complaining that there was no operation after the brackets.

The distinction in the call to GetToken is to allow for something like this:

 
5+-6


The plus in this example is one token, and the -6 is another token, so we can validly build an arithmetic expression out of it.

#36
Thanks for the quick response! :-)
#37
Hi Nick,

I too found this recently and started to play with it. I discovered however that if I wanted to use the same expression but change a parameter value calling p.Evaluate again caused an error.

It seemed to work if I reset the expression using the Evaluate(exp) form.

So looking at the code I added 2 lines:


to the top of the definition of
const double Parser::Evaluate()
{
pWord = program_.c_str();
type = None;

....

This works. Is it necessary to do both?
Is there an uptodate version of the tarball with the all latest fixes in?

Thanks, Adam
Australia Forum Administrator #38
pWord_ is const, so I doubt you need to re-establish that.

You are probably correct that putting:


type_ = NONE;


into Evaluate is a good idea. I notice that the constructor for Parser has:


 type_ (NONE)


... so it is initially NONE (notice the caps), and putting it back as NONE for a re-evaluate is probably an excellent idea.

Quote:

Is there an uptodate version of the tarball with the all latest fixes in?


I haven't done a new .tar file, but will soon.
#39
Thanks.

The "none" was a typo, I had NONE in the code.

Thanks for thisd, it's a handy bit of code.

FYI I added the the following lines to test.cpp to check the "re-evaluate" option works:

after the line:
double abc = p ["abc"];

// example of resetting symbol
p["abc"] = 52;

// parse expression again
value = p.Evaluate();

// display result
std::cout << "New Result = " << value << std:endl;

// example of retrieving symbol again
abc = p ["abc"];
std::cout << "New abc = " << abc << std:endl;

Australia Forum Administrator #40
Uploaded an improved version today:


http://www.gammon.com.au/files/utils/parser.tgz

(Now out of date, see below about GitHub).


File size: 8 Kb (source code only).

  • Changed getrandom to work more reliably (see page 2 of discussion thread)
  • Changed recognition of numbers to allow for .5 (eg. "a + .5" where there is no leading 0)
  • Also recognises -.5 (so you don't have to write -0.5)
  • Fixed problem where (2+3)-1 would not parse correctly (- sign directly after parentheses)
  • Fixed problem where changing a parameter and calling p.Evaluate again would fail because the initial token type was not reset to NONE.



The latest version is now available on GitHub:

http://github.com/nickgammon/parser

If you have git installed, you can do this to get a copy:

git clone git://github.com/nickgammon/parser.git
Amended on Mon 15 Feb 2010 08:42 PM by Nick Gammon
Australia Forum Administrator #41

This parser is now available on GitHub:

http://github.com/nickgammon/parser

You can download the complete source from:

http://github.com/nickgammon/parser/downloads

Amended on Mon 15 Feb 2010 08:44 PM by Nick Gammon
#42
Nick,
I wanted to thank you for this parser, I've created several other parsers before using tools like COCO, and I've also handcrafted small expression parsers as well. Yours is extremely easy to use and fits my current programming project nicely.

I've added a few things into your code, and I wanted to give them back to you in case you want to add them into the code base.

I'm using this expression parser in a project to help evaluate binary bitstreams.
Specifically, I added in support for hex numbers:
0x01, 0xf, 0x1234, etc...

I also added in support for bitwise AND and OR expressions using the | and & operators
you can express things such as:
somevar = (0x3f & 0x23) | 0x1234

now, since you internally use doubles to store variables, I cast the intermediate results to unsigned longs before storing the results back into doubles


I additionally added in shift operators through the << and >> ops.

somevar = 0x2345 << 5
somevar2 = 123 >> 2



How should I give you the code? should I post it on here?

-PacManFan
Australia Forum Administrator #43
You can post it here, if it doesn't exceed 6000 characters, which I hope it won't. ;)

If you pulled the source from GitHub, you should be able to produce a patch file:


git format-patch master --stdout > my_patch.diff


(Or, a straight git "diff" between versions should work, I think).

Then paste that file here, with forum codes, and fixing up certain characters, if possible. See:

Template:codetag
To make your code more readable please use [code] tags as described here.


If you can't easily fix the forum tags (basically you need to put \ before [ ] and \) just post it "as is" and I'll fix it up.

I'll take a look and release an updated version.
Amended on Wed 27 Jan 2010 07:39 PM by Nick Gammon
#44
Hello

and thanks a lot for your parser. It is exactly what I need.

However, I am experiencing a very strange limitation: if I invoke p.Evaluate() twice, it raises an exception. Here is the minimalist program that reproduces the "bug" (or feature?):


#include "parser.h"
#include <iostream>

int main()
{
  Parser p("x+y");
	
  p["x"] = 1;
  p["y"] = 2;
  std::cout << "Value1: " << p.Evaluate() << std::endl;

  p["x"] = 3;
  p["y"] = 4;
  std::cout << "Value2: " << p.Evaluate() << std::endl;

  return 1;
}


Here is the execution trace:

./parser 
Value1: 3
terminate called after throwing an instance of 'std::runtime_error'
  what():  Unexpected end of expression.
Abandon


If I change the second call to

  std::cout << "Value2: " << p.Evaluate("x+y") << std::endl;

it works as expected:

./parser 
Value1: 3
Value2: 7


But I guess the expression is parsed again, no? Why the parser doesn't keep the expression he received in the constructor for multiple calls to Evaluate()?

Thanks in advance,

Bruno.
Australia Forum Administrator #45
You are certainly supposed to be able to call Evaluate multiple times. I found the problem, the new version has been pushed to GitHub:


http://github.com/nickgammon/parser

The changes are here:

http://github.com/nickgammon/parser/commit/994c67

Basically it was not resetting the pWord_ pointer back to the start of the expression when you called Evaluate again. I don't know why no-one noticed this earlier.
Australia Forum Administrator #46
I have made a couple of other corrections, basically moving the resetting the pointer into a single place, and improving the Makefile for making the test program.

The complete source can be downloaded from:

http://github.com/nickgammon/parser/downloads
#47
Thank you Nick for the modifications. It works fine now.

But I am concern by performances because I have to fill a big 2D array with values evaluated for a single expression.

Is there any way, when the expression has been parsed once, to evaluate subsequent results with a kind of "compiled" expression? Meaning not having to start all the parsing again.

Ideally, I would like to reach performances close to hard coded expression. Is there any way of accelerating the evaluation?

Thanks,

Bruno.
Australia Forum Administrator #48
Not really. The basic design is to evaluate on-the-fly.

According to page 1 of this thread the parser works reasonably fast (like 10,000 evaluations in under a second).

But if you really need to do millions of evaluations of a single expression this is probably the time to switch to using C++.

Or try using Lua. Lua is pretty fast, and does in fact precompile an expression.
#49
I found your parser quite useful.
Yet, I've got a problem with the "if" function. It seems like both, the if and the else section are evaluated regardless of the condition:
"if(cond,a=1,b=2)" always sets, a to 1 and b to 2, for cond==0 as well as for cond!=0.
also "if(something,x=x+1,x)" does not behave like I expected.
USA #50
'If' is a function here, not a control flow statement. So, all arguments are evaluated, as for any function, and then the function returns the expression (which, recall, has already been evaluated) matching the condition.

This is by design, this is meant to be an expression parser, not a full language.
Australia Forum Administrator #51
Bernhard said:

"if(something,x=x+1,x)" does not behave like I expected.


David is right. To add 1 to x under a condition you want something more like:


x = if (something, x + 1, x)


The result of the if function will be either x + 1, or x, depending on the condition, and this result is then assigned to x.
#52
The two examples in my last post were actually simplified, just to isolate the problem. My real intention (still simplified) was to implemente a sequence of statements like

if(0==eventAB, state17=1,
if(state11, state18=1,
if(state12, state19=1, ...)))
if(0==eventCD, state17=1, if(...))

Of course this can be reformulated to

state17 = if(0==eventAB, 1, state17)
state18 = if((0!=eventAB)&&(state11), 1, state18)
state19 = if((0!=eventAB)&&(0==state11)&&(state12), 1, state19)
...
state17 = if(0==eventCD, 1, state17)

unfortunately it makes the code less readable.

My other intention was to add an "exec(###)" function which calls a batch file (returning the ERRORLEVEL), so basically "return system(script###.bat)" in C. This would allow to get data from external scripts e.g., cond59=exec(11), and to trigger reactions, e.g., if(condition_for_reaction12, exec(12), 0)

Australia Forum Administrator #53
Yes, well it is an expression parser, not a full statement parser. If you are going to run external scripts anyway, why not just use Lua? That has a small footprint, and can be imbedded into C or C++ with minimal effort. You can then write things in a more natural way. That would be much, much neater than trying to use this parser and shoe-horn in an external script via system calls. For example:

Template:post=6400
Please see the forum thread: http://gammon.com.au/forum/?id=6400.
USA #54
Yes, I agree, if you want a real programming language why not just use one? What's the higher-level context here?
#55

My intention was to add some advanced configuration capability to my C/C++ program. Those configurations should not be written by me but by the users – so, usually by people without good programming skills. The basic configuration, let’s call it level 1, would be simple parameters like "maxUsers = 20". I can read them with GetPrivateProfileString/Int or whatever. Level 2 would allow simple expressions, e.g., configurations like "logBufferSize = maxUserSessions * 100" or conditions like "acceptEntry = (userLevel > 5) * (currentUserCount < 20)" – basically the same kind of expressions you can use in Microsoft Excel & co. (which actually allows “if”). I think your parser is perfectly suitable to add this type of configuration features to a C/C++ program. Level 3 allows some simple conditional reactions, maybe similar to state machine descriptions. That's where I ran into troubles with the 'if'. Still I think I should not burden the users with learning a full programming language. It’s beyond debate that anything more than that (level 4) requires a scripting language (vbscript, lua, ..) or (level 5) even a component/plug-in architecture in the main program – that would certainly would overshoot the mark as a configuration/customization interface for someone without proper programming skills.

Level 3 should be sufficient. On the other hand, users tend to want more and more anyway. lua will certainly satisfy even ambitious users and if it encourages someone to learn a full programming language its fine with me. lua looks quite promising, the only thing I hate is that it considers 0 as true while most (all?) other languages do not - I can already her the users complaining. I also have to allow a syntax as simple as "maxUsers = 50" if you only need configuration level 1.

What do you think?
USA #56
It turns out that Lua was originally designed precisely as a configuration language. You can easily extend C/C++ with Lua, as Nick said, and then have config files that look like this:


use_ui = false 


-- high level search settings
directional_increment = 0.05
probabilistic_roadmap_size = 250

probabilistic_focus_roadmap_points_per_point = 5
probabilistic_focus_roadmap_spread_x = 0.10
probabilistic_focus_roadmap_spread_y = 0.10
use_highlevel_smoothing = false

roadmap_predicate_thresholdz_init = 0.01
roadmap_predicate_thresholdz_inc  = 0.01
roadmap_predicate_thresholdz_max  = 0.05

roadmap_predicate_maxside_threshold_init =  0.030
roadmap_predicate_maxside_threshold_inc  =  0.005
roadmap_predicate_maxside_threshold_max  =  0.050

highlevel_goal_proximity = 0.05
highlevel_goal_proximity_factor = 0.3

highlevel_thresholdz_maxjump = 1.5

terrain_polling_num_samples = 50

goal_cost_bias = 0.95
goal_heur_bias = 0.95

highlevel_cost_function = "HighLevelCostFunction"
highlevel_heur_function = "HighLevelHeurFunction"
highlevel_pred_function = "RoadmapVisibilityPredicate"
highlevel_lua_func_file = "highlevel.lua"

-- debugger settings
debug_astar = true
debug_class = true
debug_sim = true
debug_train = true


Or as a more complicated example, like this:

TerrainTypes = {
        beach = { name = "beach", tileImage = "tiles/world/beach.png"},
        densewoods = { name = "densewoods", tileImage = "tiles/world/dense_woods.png" },
        densewoods_snow = { name = "densewoods_snow", tileImage = "tiles/world/dense_woods_snow.png" },
        desert = { name = "desert", tileImage = "tiles/world/desert.png" },
        farmland = { name = "farmland", tileImage = "tiles/world/farmland.png" },
        fields = { name = "fields", tileImage = "tiles/world/fields.png" },
        ice = { name = "ice", tileImage = "tiles/world/ice.png" },
        jungle = { name = "jungle", tileImage = "tiles/world/jungle.png" },
        lightwoods = { name = "lightwoods", tileImage = "tiles/world/light_woods.png" },
        lightwoods_snow = { name = "lightwoods_snow", tileImage = "tiles/world/light_woods_snow.png" },
        mountain = { name = "mountain", tileImage = "tiles/world/mountain.png" },
        ocean = { name = "ocean", tileImage = "tiles/world/ocean.png" },
        pass = { name = "pass", tileImage = "tiles/world/pass.png" },
        plains = { name = "plains", tileImage = "tiles/world/plains.png" },
        range = { name = "range", tileImage = "tiles/world/range.png" },
        river = { name = "river", tileImage = "tiles/world/river.png" },
        rocky = { name = "rocky", tileImage = "tiles/world/rocky.png" },
        savannah = { name = "savannah", tileImage = "tiles/world/savannah.png" },
        snow = { name = "snow", tileImage = "tiles/world/snow.png" },
        swamp = { name = "swamp", tileImage = "tiles/world/swamp.png" },
        tundra = { name = "tundra", tileImage = "tiles/world/tundra.png" },
        wasteland = { name = "wasteland", tileImage = "tiles/world/wasteland.png" },
}


The user doesn't have to even know that they're using Lua; I think that the above examples, especially the first one, show that you can have a config file that is Lua but looks very simple. They can even do variables and have if statements in there, if they want to.

Anyhow, you can have the simple syntax as shown above by loading the script and executing it in a special environment. Then, all assignments to "global" variables will be made into that environment table, and you can pick them out of it later.

It's true that you'll have to deal with 0 being 'true' in Lua, but, well, that's life.



You can do similar things in Python, like this:

name = "Building_Wall"
image = "gfx/Buildings/Wall.bmp"
transparency = (255, 0, 255)

# Cap naming convention: cap_<NESW>
#
# Joint naming convention: joint_<direction in which there is no cap>
#
# Base naming convenition: base_<WE>

tiles = {
    
    'cap_0110': (0, 0, 20, 20),
    'cap_0111': (40, 0, 20, 20),
    'cap_0011': (80, 0, 20, 20),

    'cap_1110': (0, 40, 20, 20),
    'cap_1111': (40, 40, 20, 20),
    'cap_1011': (80, 40, 20, 20),

    'cap_1100': (0, 80, 20, 20),
    'cap_1101': (40, 80, 20, 20),
    'cap_1001': (80, 80, 20, 20),

    'cap_1100_baseleft': (240, 0, 20, 20),
    'cap_1110_baseleft': (240, 40, 20, 20),

    'joint_northeast': (0, 140, 20, 20),
    'joint_northwest': (40, 140, 20, 20),
    'joint_southeast': (80, 140, 20, 20),
    'joint_southwest': (120, 140, 20, 20),

    'base_01': (0, 100, 20, 20),
    'base_11': (40, 100, 20, 20),
    'base_10': (80, 100, 20, 20),

    'dummy': (20, 0, 20, 20),
    
}


but Lua is easier to embed (and much more lightweight) than Python.
Australia Forum Administrator #57
Bernhard said:


lua looks quite promising, the only thing I hate is that it considers 0 as true while most (all?) other languages do not - I can already her the users complaining.


I don't think it is too bad personally. After all, having 0 be false and non-zero to be true is really a throwback to the way C does it (and remember VB used to use -1 for true which is kind of confusing too).

Just say, "for booleans use true and false". It makes sense really.

However if this completely throws people out you have a couple of options:

  • For variables that are supposed to be boolean (ie from your internal context) throw an error if they don't supply a boolean (in particular, the number 0 could be ambiguous - did they really mean "false"?)
  • Or, for booleans simply convert 0 to false as you process the variable.


You need range-checking anyway (eg. what happens if they make maxUsers to be -100?). So checking booleans have a valid value is just another check.
Amended on Wed 05 May 2010 09:45 PM by Nick Gammon
Australia Forum Administrator #58
Bernhard said:


lua looks quite promising, the only thing I hate is that it considers 0 as true while most (all?) other languages do not - I can already her the users complaining.


The other thing is that I don't know about this being totally true. After all, 0 is a number and false is a boolean. They are different types for a start. Just as an example, in SQL, it uses NULL to represent "no data" in the same way Lua uses nil.

As an example, say on a database you wanted to know how many children someone has. The answer NULL could mean "that number is unknown (and could be anything)" whereas 0 means "we know the answer, and the answer is zero".

Thus trying to make NULL and zero have the same meaning would not be useful.

Similarly with Lua, variables themselves are untyped (the values are typed, not the variable). Thus numerically typed values have to be able to hold any number (including zero) without being mistaken for a boolean typed value.

There is an interesting discussion here:

http://en.wikipedia.org/wiki/Boolean_data_type

Amongst other things it says:

Wikipedia said:

In Ruby programming language, on the other hand, only the null object and a special false object are "false", everything else (including the integer 0) is "true".


So in Ruby, at least, it also does not treat 0 as false.
#59
Thank you Nick and David. I will switch to lua then.

Nick Gammon said:

However if this (0 = true) completely throws people out you have a couple of options:

*For variables that are supposed to be boolean (ie from your internal context) throw an error if they don't supply a boolean (in particular, the number 0 could be ambiguous - did they really mean "false"?)

*Or, for booleans simply convert 0 to false as you process the variable.

You need range-checking anyway (eg. what happens if they make maxUsers to be -100?). So checking booleans have a valid value is just another check.


I'm not sure if I should do that, because it makes conditions internal to lua behave different than the conditions which are passed on to C/C++. If I consider 'return 0' as false when the caller was a C function, it is still true in case the caller was a lua function.


lua_genpcall(L, "if (0) then print '0 is true' end", "");

will allways print '0 is true' unless I change lua itself and make my own dialect ... so I better instruct the users to keep this in mind and tell them something like:

David Haley said:

It's true that you'll have to deal with 0 being 'true' in Lua, but, well, that's life.


;-)

#60
Hello Nick

As I can read from your comments and forum messages your expression parser is what I am looking for. The only problem is that I have to use it in a micro controller (ATMEL AVR) in C language.
A post from Jorick at 7th August 2007 has a similar situation (to change from c++ to c).
Have you the code in c ? Or any suggestion to port your code in c ?

Best regards
Georgios
Australia Forum Administrator #61
The whole design is based on the way C++ works (like, getting variables by reference). To do it for C would be a rewrite.

Would the controller support Lua? That compiles in C. However I believe some micro chips have a different architecture that doesn't work with Lua.
#62
Hello

Suppose a user enters an expression like this: (TT_123 * 10)/PT_870

How can I list the two parameters (TT_123 and PT_870) before evaluation to add the assignment
p[TT_123] = 100.0; (for example)
p[PT_870] = 0.56; (for example)

Thanks for your help

Thierry
Australia Forum Administrator #63
It's not designed to do that right now. I gather you want to have it "query" for a variable value at runtime, is that right?

Non-defined variables currently default to zero. In particular around line 509:


      // not a function? must be a symbol in the symbol table
      double & v = symbols_ [word]; // get REFERENCE to symbol table entry


This creates the symbol if it doesn't exist, and it will default to zero. You would need to alter that line to do a lookup to see if the symbol exists, and if not have some way of obtaining it (eg. a callback function).

What might work could be to parse twice. First time you get zeroes for each symbol. Then look at the symbol table list to see what symbols get added. Change their values, and re-parse.
Amended on Sat 08 Sep 2012 07:51 AM by Nick Gammon
#64
Hello

Yes, I need to get the information at run time. I scan the symbols map, update the values and call Evaluate a second time .... Working fine !!!

Thanks for that great class ... It will be used in the next release of a freeware called Genesis (www.tgmdev.be), an OPC data acquisition utility for people involved in process automation.

Thierry