Register forum user name Search FAQ

Gammon Forum

Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the password reset link.

Due to spam on this forum, all posts now need moderator approval.

 Entire forum ➜ MUSHclient ➜ Plugins ➜ Adapting GMCP Handler to ATCP2 on Geas

Adapting GMCP Handler to ATCP2 on Geas

It is now over 60 days since the last post. This thread is closed.     Refresh page


Pages: 1  2 3  

Posted by Chaotzin   (19 posts)  Bio
Date Reply #15 on Sun 29 Jul 2018 05:30 AM (UTC)
Message
Ok here are I think the two relevant functions - saving exits and loading rooms. I've bolded with notes the specific changes I've made.

The exits are saving via this function:


function save_full_exits_to_database (uid, exits)
--  Note ("Mapper doing: save_full_exits_to_database.")
  
  local room = rooms [uid]
  
  db:exec ("BEGIN TRANSACTION;") 

  for exit in string.gmatch (exits, "[^,]+") do

    dir, touid = string.match (exit, "^(%a+)%((%w+)%)$")	-- Modified note: changed %d to %w to match alphanumeric room uids
    
    touid = tostring (touid) -- instead of tonumber
    
    if dir then
      -- fix up in and out
      dir = ({ ['i'] = "in", o = "out", }) [dir] or dir
      
      dbcheck (db:execute (string.format ([[
        INSERT INTO exits (dir, fromuid, touid, date_added) 
            VALUES (%s, %s, %s, DATETIME('NOW'));
      ]], fixsql  (dir),  -- direction (eg. "n")
          fixsql  (uid),  -- from current room
          fixsql  (touid) -- destination room 
          )))
      if show_database_mods then
        mapper.mapprint ("Added full exit", dir, "from room", uid, "to room", touid, "to database.")
      end -- if
      
      room.exits [dir] = touid
    else
      mapper.maperror ("Cannot make sense of:", exit)
    end -- if can decode
    
  end -- for each exit
  
  db:exec ("COMMIT;") 
  
end -- function save_full_exits_to_database



And this is how it's loading from database:


function load_room_from_database (uid)
--	Note ("Mapper doing: load_room_from_database.")
	
  local room
  
  uid = tostring (uid)	-- Geas: from tonumber to tostring
 
  assert (uid, "No UID supplied to load_room_from_database")
  
  -- if not in database, don't look again
  if room_not_in_database [uid] then
    return nil
  end -- no point looking
  
  for row in db:nrows(string.format ("SELECT * FROM rooms WHERE uid = %s", fixsql (uid))) do
     room = {
       name = row.name,
       area = row.area,
       building = row.building,
       terrain = row.terrain,
       info = row.info,
       notes = row.notes,
       x = row.x,
       y = row.y,
       z = row.z,
  
       exits = {} }
      
    for exitrow in db:nrows(string.format ("SELECT * FROM exits WHERE fromuid = %s", fixsql (uid))) do
       room.exits [exitrow.dir] = tostring (exitrow.touid)	-- Modified note: changed from tonumber to tostring touid
    end -- for each exit
    
  end   -- finding room

  if room then
    rooms [uid] = room
    for row in db_bm:nrows(string.format ("SELECT * FROM bookmarks WHERE uid = %s", fixsql (uid))) do
      rooms [uid].notes = row.notes
    end   -- finding room
        
    return room
  end -- if found
  
  room_not_in_database [uid] = true
  return nil
    
end -- load_room_from_database


In that function I have modified:

room.exits [exitrow.dir] = tonumber (exitrow.touid)


to

room.exits [exitrow.dir] = tostring (exitrow.touid)
Top

Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #16 on Sun 29 Jul 2018 06:06 AM (UTC)
Message
I can't see anything obviously wrong with either the loading or saving. Can you check in the database and see if the uids are wrongly on disk? You can use the sqlite3.exe program that comes with SQLite.

Do a SELECT to find what is actually stored for the exits in question. Also can you find the way that the exits table was set up? I think that is:


.schema exits

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Chaotzin   (19 posts)  Bio
Date Reply #17 on Sun 29 Jul 2018 07:43 AM (UTC)
Message
Ok so the exits table looks to be set up fine.

It follows the structure described in the plugin where:

exitid INTEGER
dir TEXT (not NULL)
fromuid STRING (not NULL)
touid STRING (not NULL)
data_added DATE


I can see plenty of fromuid and touid entries in there where the string begins with "0" too so that's not it. But there are a bunch of weird data entries in here, just in touid and fromuid exits (ie not in room uid data).

There entries look like this:
Quote:
6094940299999999944756163936158435001692602761216


and

Quote:
inf.0


Really strange. Kind of looks like it's occasionally interpreting the exit uid as a mind-bogglingly huge number.

I've only run this testing the plugin so I could remove that db file and run it fresh to see if the error occurs in the same rooms.
Top

Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #18 on Sun 29 Jul 2018 08:35 PM (UTC)

Amended on Sun 29 Jul 2018 10:46 PM (UTC) by Nick Gammon

Message
Quote:

INSERT INTO exits ...


There would also be a place where exits are modified ("UPDATE exits SET ..."). Can you post that too please?

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #19 on Sun 29 Jul 2018 08:42 PM (UTC)
Message
Quote:

060949403E41

...

Kind of looks like it's occasionally interpreting the exit uid as a mind-bogglingly huge number.


Well, taken as a number, that would be interpreted as:


60949403 * 10^41

= 

6094940300000000000000000000000000000000000000000

But you got:

6094940299999999944756163936158435001692602761216
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         41 zeroes, however not completely accurate


So clearly it is interpreting that as a large number, storing as a floating point number (double, probably) which has an accuracy of 15 to 17 decimal digits.

See Wikipedia article about floating point numbers


6094940299999999944756163936158435001692602761216
^^^^^^^^^^^^^^^^^
17 digits are accurate (apart from rounding)


SQLite3 documentation says that data types are suggestions rather than rigidly enforced, so you can, for example, store a string in a field designated as a number.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Chaotzin   (19 posts)  Bio
Date Reply #20 on Sun 29 Jul 2018 09:44 PM (UTC)
Message
Aha, how interesting! Should've noticed it's the same number over and over just in different forms.

Actual UID is:
Quote:
060949403E41


Stored in database as:
Quote:
6094940299999999944756163936158435001692602761216.


Represented in text as:
Quote:
6.0949403e+48


And that text representation in Hex as:
Quote:
36 2e 30 39 34 39 34 30 33 65 2b 34 38


I don't really know what to do about it but it is interesting. I've played around with fresh db files and cleared cache and it will repeat these results with a few specific room like that number above.

Another example is:
Quote:
7E3744474144


re your last post about finding 'MODIFY exits SET', I think I'm a bit out of my depth finding that. Haven't actually had to try opening a SQLite db file before now :D
Top

Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #21 on Sun 29 Jul 2018 10:47 PM (UTC)
Message
I made a mistake. It's "UPDATE exits SET ...". It will be in the plugin where it changes an existing exit to lead somewhere else.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Chaotzin   (19 posts)  Bio
Date Reply #22 on Mon 30 Jul 2018 09:21 AM (UTC)
Message
Oh ok. I can see that in two particular functions of the plugin:
1. fix_up_exit
2. room_change_exit

'Fix_up_exit' is called from the following 'got_room_number' function when we get a room number via the GMCP handler. Here, (s) is (LID.hash):

-- here when location changes, eg. : Room.Num 7476
function got_room_number (s)
--  Note ("Mapper doing: got_room_number.")
  local room_number = tostring (s)
    
  if not room_number then
    return
  end -- no room number

  current_room = room_number
--  Note ("Mapper doing: current room is " .. current_room)
  mapper.draw (tostring (room_number))
  
  if expected_exit == "0" and from_room then
    fix_up_exit ()
  end -- exit was wrong

end -- got_room_number


And this is the full fix_up_exit function:

function fix_up_exit ()

  local room = rooms [from_room]
  
  dbcheck (db:execute (string.format ([[
      UPDATE exits SET touid = %s WHERE fromuid = %s AND dir = %s;
    ]], 
        fixsql  (current_room),     -- destination room
        fixsql  (from_room),       -- from previous room
        fixsql  (last_direction_moved)  -- direction (eg. "n")
        )))
        
  if show_database_mods then
    mapper.mapprint ("Fixed exit", last_direction_moved, "from room", from_room, "to be to", current_room)
  end -- if
  
  room.exits [last_direction_moved] = current_room
    
  last_direction_moved = nil
  from_room = nil
  
end -- fix_up_exit


The only other function that includes that text is room_change_exit:

function room_change_exit (room, uid)

local available =  {
  n = "North",
  s = "South",
  e = "East",
  w = "West",
  u = "Up",
  d = "Down",
  ne = "Northeast",
  sw = "Southwest",
  nw = "Northwest",
  se = "Southeast",
  nu = "nu",
  su = "su",
  eu = "eu",
  wu = "wu",
  neu = "neu",
  swu = "swu",
  nwu = "nwu",
  seu = "seu",
  nd = "nd",
  sd = "sd",
  ed = "ed",
  wd = "wd",
  ned = "ned",
  swd = "swd",
  nwd = "nwd",
  sed = "sed",
  ['in'] = "In",
  out = "Out",
  }  -- end of available

  -- remove non-existent exits
  for k in pairs (available) do
    if room.exits [k] then
      available [k] = available [k] .. " --> " .. room.exits [k] 
    else
      available [k] = nil
    end -- if not a room exit
  end -- for
  
  if next (available) == nil then
    utils.msgbox ("There are no exits from this room.", "No exits!", "ok", "!", 1)
    return
  end -- not known
  
  local chosen_exit = utils.listbox ("Choose exit to change destination of:", "Exits ...", available )
  if not chosen_exit then
    return
  end

  exit_destination = utils.inputbox ("Enter destination room identifier (number) for " .. available [chosen_exit], room.name, "")

  if not exit_destination then
    return
  end -- cancelled
  
    -- look it up
  local dest_room = rooms [exit_destination]
  
  -- not cached - see if in database
  if not dest_room then
    dest_room = load_room_from_database (exit_destination)
    rooms [exit_destination] = dest_room -- cache for later
  end -- not in cache
  
  if not dest_room then
    utils.msgbox ("Room " .. exit_destination .. " does not exist.", "Room does not exist!", "ok", "!", 1)
    return
  end -- if still not there
    
  dbcheck (db:execute (string.format ([[
    UPDATE exits SET touid = %s WHERE dir = %s AND fromuid = %s;
  ]], fixsql  (exit_destination),
      fixsql  (chosen_exit),  -- direction (eg. "n")
      fixsql  (uid)  -- from current room
      )))
      
  if show_database_mods then
    mapper.mapprint ("Modified exit", available [chosen_exit], "from room", uid, "to be to room", exit_destination, "in database.")
  end -- if
  
  -- update in-memory table
  rooms [uid].exits [chosen_exit] = exit_destination
  mapper.draw (tostring (current_room))
   
end -- room_change_exit


There's a fair bit going on between these functions that I'm not entirely sure I need.

For instance, I don't quite understand the purpose behind fix_up_exit but I would guess it's used for mapping your movement back to the room you just came from. If that's true it's not really needed as "LID.links" provides all the exit data I need.
Top

Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #23 on Mon 30 Jul 2018 09:26 PM (UTC)
Message
I don't understand why this is happening. The function fixsql is supposed to quote its argument, so that a string that "looks like a number" should still be a string. For example, this test:


function dbcheck (code)
 if code ~= sqlite3.OK and    -- no error
    code ~= sqlite3.ROW and   -- completed OK with another row of data
    code ~= sqlite3.DONE then -- completed OK, no more rows
    local err = db:errmsg ()  -- the rollback will change the error message
    db:exec ("ROLLBACK")      -- rollback any transaction to unlock the database
    error (err, 2)            -- show error in caller's context
  end -- if
end -- dbcheck 

function fixsql (s)
  if s then
    return "'" .. (string.gsub (s, "'", "''")) .. "'" -- replace single quotes with two lots of single quotes
  else
    return "NULL"
  end -- if
end -- fixsql

-- start here

db    = assert (sqlite3.open(GetInfo (66) .. "test.db"))

dbcheck (db:execute[[
  CREATE TABLE IF NOT EXISTS test ( t text );
  DELETE FROM test;
  INSERT INTO test (t) VALUES ('060949403E41');
  ]])

for row in db:nrows(string.format ("SELECT * FROM test")) do
  print (row.t, type (row.t))
end   -- finding room

db:close()


Running that I get:


060949403E41 string


In other words, I get out what I put in.

Try running that in your MUSHclient "Immediate" window (Ctrl+I).

Also, check that your "fixsql" function looks like the above.

Also please show your MUSHclient version.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Chaotzin   (19 posts)  Bio
Date Reply #24 on Tue 31 Jul 2018 10:16 AM (UTC)
Message
Running that I also got:
060949403E41 string


...and fixsql is the same in the plugin:
function fixsql (s)
  
  if s then
    return "'" .. (string.gsub (s, "'", "''")) .. "'" -- replace single quotes with two lots of single quotes
  else
    return "NULL"
  end -- if
end -- fixsql


I'm using MUSHclient 5.06-pre having come from using the Aardwolf installer version. But I also just ran the same map and GMCP plugins on a fresh install of version 5.05 now and the issue is consistent across both.

I can replicate the result around room 060949403E41 consistently now too - with fresh database files.

First encounter of the room as an exit destination looks fine:
Added full exit s from room 1A8549F81F6C to room 060949403E41 to database.


The mapper draws this correctly and displays the info correctly immediately after this as it's reading it from a lua table called 'room.exits':
"s"="060949403E41"


But if I clear those tables and ask it to reload the room from the database, the error shows up in 'touid':
"s"="6.0949403e+048"


If I map that room by walking into it, the room uid will save and load that particularly room correctly as:
"s"="060949403E41"


But again, like above, the issue shows up in 'fromuid':
"s"="6.0949403e+048"


The fact that the database has no issues saving that string to the rooms table uid column I'm thinking must narrow the issue down to saving/modifying touid and fromuid with certain strings like this one.

I don't suppose there's a simple way to force the string to be treated as such, like encasing it in additional "" that are saved with it?
Top

Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #25 on Tue 31 Jul 2018 07:59 PM (UTC)
Message
There's more going on here than meets the eye. We need to do some debugging. In the plugin, after opening the database, add the lines shown (in my case I tried it on the test script above):


db    = assert (sqlite3.open(GetInfo (66) .. "test.db"))

--> ADD THIS:

if old_db_execute == nil then
  old_db_execute = getmetatable (db).execute
end -- if

getmetatable (db).execute = function (this, what)
  ColourNote ("orange", "", "** SQL debugging: About to run: " .. what)
  return old_db_execute  (this, what)
end -- function


That will make db:execute (which is used for inserting and modifying the database) output a debugging line, and then do what it originally did.

Once you have done that, reproduce the problem, and see if there is in fact a SQL statement that does not put the exit into quotes.

With the test script I got this output:


** SQL debugging: About to run:   CREATE TABLE IF NOT EXISTS test ( t text );
  DELETE FROM test;
  INSERT INTO test (t) VALUES ('060949403E41');


That proves that the string was quoted.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Chaotzin   (19 posts)  Bio
Date Reply #26 on Tue 31 Jul 2018 09:55 PM (UTC)
Message
Alright, I've put it here in the plugin file following opening the db on disk:

function OnPluginInstall ()
  
  config = {}  -- in case not found

  -- get saved configuration
  assert (loadstring (GetVariable ("config") or "")) ()

  -- allow for additions to config
  for k, v in pairs (default_config) do
    config [k] = config [k] or v
  end -- for
  
  -- initialize mapper engine
  mapper.init { config = config,            -- colours, timing etc.
                get_room = get_room,        -- get_room (uid) called to get room info
                show_help = OnHelp,         -- to show help
                room_click = room_click,    -- called on RH click on room square
                timing = show_timing,       -- want to see timing
                show_completed = show_completed,  -- want to see "Speedwalk completed." message
                show_other_areas = show_other_areas,  -- want to see areas other than the current one?
                show_up_down = show_up_down,          -- want to follow up/down exits?
                show_area_exits = show_area_exits,    -- want to see area exits?
                speedwalk_prefix = speedwalk_prefix,  -- how to speedwalk
                } 
  
  mapper.mapprint (string.format ("MUSHclient mapper installed, version %0.1f", mapper.VERSION))
  
  -- open databases on disk 
  db    = assert (sqlite3.open(GetInfo (66) .. Trim (WorldAddress ()) .. "_" .. WorldPort () .. ".db"))
  db_bm = assert (sqlite3.open(GetInfo (66) .. Trim (WorldAddress ()) .. "_" .. WorldPort () .. "_bookmarks.db"))
  
  
  --> ADD THIS:

if old_db_execute == nil then
  old_db_execute = getmetatable (db).execute
end -- if

getmetatable (db).execute = function (this, what)
  ColourNote ("orange", "", "** SQL debugging: About to run: " .. what)
  return old_db_execute  (this, what)
end -- function


  create_tables ()    -- create database structure if necessary
  
  -- grab all area names
  for row in db:nrows("SELECT * FROM areas") do
    areas [row.uid] = row.name
  end   -- finding areas
  
  -- grab all user terrain info
  for row in db_bm:nrows("SELECT * FROM terrain") do
    user_terrain_colour [row.name] = row.color
  end   -- finding terrains
 
  -- grab all environment names
  for row in db:nrows("SELECT * FROM environments") do
    environments [tostring (row.uid)] = row.name		-- Geas: changed from tonumber
    terrain_colours [row.name] = tonumber (row.color)
  end   -- finding environments
  
end -- OnPluginInstall


Now when mapping the room as a southern exit:

** SQL debugging: About to run: INSERT INTO rooms (uid, name, area, date_added) VALUES ('1A8549F81F6C', 'The entrance to a light forest', '0', DATETIME('NOW'));
** SQL debugging: About to run:         INSERT INTO rooms_lookup (uid, name) VALUES ('1A8549F81F6C', 'The entrance to a light forest');
      
Added room 1A8549F81F6C to database. Name: The entrance to a light forest
** SQL debugging: About to run:         INSERT INTO exits (dir, fromuid, touid, date_added) 
            VALUES ('s', '1A8549F81F6C', '060949403E41', DATETIME('NOW'));
      
Added full exit s from room 1A8549F81F6C to room 060949403E41 to database.


As well as for mapping the room itself:
** SQL debugging: About to run: INSERT INTO rooms (uid, name, area, date_added) VALUES ('060949403E41', 'A road junction surrounded by forest', '0', DATETIME('NOW'));
** SQL debugging: About to run:         INSERT INTO rooms_lookup (uid, name) VALUES ('060949403E41', 'A road junction surrounded by forest');
      
Added room 060949403E41 to database. Name: A road junction surrounded by forest

** SQL debugging: About to run:         INSERT INTO exits (dir, fromuid, touid, date_added) 
            VALUES ('n', '060949403E41', '1A8549F81F6C', DATETIME('NOW'));
      
Added full exit n from room 060949403E41 to room 1A8549F81F6C to database.


Should I put that debug function elsewhere in the plugin for it to read the loading part of the sequence?
Top

Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #27 on Wed 01 Aug 2018 12:44 AM (UTC)
Message
When you read, you don't quote stuff (eg. SELECT foo FROM bar) because they are just variable names.

Are you saying that after running the stuff you posted above, the data on the database is actually wrong (if you use the sqlite3 program to read it)?

For example:


SELECT * FROM exits WHERE touid = '1A8549F81F6C';


I also suggest scanning the entire plugin to see if there is a stray "tonumber" there.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Chaotzin   (19 posts)  Bio
Date Reply #28 on Wed 01 Aug 2018 10:40 AM (UTC)
Message
That's right. Despite the debug messages, the data showing up in the db file when reading it with either the sqlite3 or with the MUSH mapper is incorrect.

I've stripped back the plugin in a new version, just keeping the basic functions that map via:
1. uid as string
2. room name
3. full exits with destination
4. area name

I could post that in full if it will help and some instructions on how to produce these results to see if it can be replicated.

Even stripped back quite a lot the plugin is 840 lines though.
Top

Posted by Chaotzin   (19 posts)  Bio
Date Reply #29 on Wed 01 Aug 2018 11:46 AM (UTC)
Message
Solved! :D :D :D

And it looks so obvious now I'm wondering how it never occurred to me.

The issue was this:

CREATE TABLE IF NOT EXISTS exits (
      exitid      INTEGER PRIMARY KEY AUTOINCREMENT,
      dir         TEXT    NOT NULL, -- direction, eg. "n", "s"
      fromuid     STRING  NOT NULL, -- exit from which room (in rooms table)
      touid       STRING  NOT NULL, -- exit to which room (in rooms table)
      date_added  DATE              -- date added to database
    );


I just had to change the table for touid and fromuid to TEXT rather than STRING.

It still seems odd now though that the STRING type in the database would evaluate only some very particular strings as numbers.
Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.


109,053 views.

This is page 2, subject is 3 pages long:  [Previous page]  1  2 3  [Next page]

It is now over 60 days since the last post. This thread is closed.     Refresh page

Go to topic:           Search the forum


[Go to top] top

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.