AIML files with <topic> get rejected with errors

Help with installation and development.

AIML files with <topic> get rejected with errors

Postby Alekz » January 25th, 2015, 2:58 pm

I stumbled upon the following problem - <topic> can not be used in any form, because the files get rejected with the following errors.

This file

Code: Select all
<?xml version="1.0" encoding="ISO-8859-1"?>
<aiml>

<topic name="test">
</topic>

</aiml>

Code: Select all

leads to the following in admin.error.log:

[25-Jan-2015 13:51:10 UTC] PHP Notice:  Undefined variable: sql_start in /home/alekz/public_html/ai/o/admin/upload.php on line 170


And this one :

Code: Select all
<?xml version="1.0" encoding="ISO-8859-1"?>
<aiml>

<topic name="test">
<category>
<pattern>Hi</pattern>
 <template>Hello
 </template>
</category>
</topic>

</aiml>


creates more errors in admin.error.log and db_write.txt:

Code: Select all
[25-Jan-2015 13:52:23 UTC] PHP Notice:  Undefined variable: sql_start in /home/alekz/public_html/ai/o/admin/upload.php on line 170
bad SQL encountered in file /home/alekz/public_html/ai/o/admin/upload.php, line #232. SQL:

PDO Error:
Array
(
    [0] => 00000
    [1] =>
    [2] =>
)

STH Error:
Array
(
    [0] => 42000
    [1] => 1065
    [2] => Query was empty
)

Esception Message:
SQLSTATE[42000]: Syntax error or access violation: 1065 Query was empty


The content of the whole file does is not accepted and it does not show in admin/index.php?page=upload

Program O version 2.4.6

Any ideas?
Last edited by Alekz on September 25th, 2016, 1:58 pm, edited 1 time in total.
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm

Re: AIML files with <topic> get rejected with errors

Postby Alekz » January 25th, 2015, 6:45 pm

Oops... Wrong forum.... Could you please move it the "Bugs and Help"?
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm

Re: AIML files with <topic> get rejected with errors

Postby GeekCaveCreations » January 25th, 2015, 7:57 pm

Sounds like a bug in the file upload script. I'll check it out.
Comforting the disturbed, and disturbing the comfortable
Chat with Morti
User avatar
GeekCaveCreations
Safe, Reliable Insanity, Since 1961
 
Posts: 1115
Joined: April 18th, 2011, 10:52 pm
Location: Nevada, USA

Re: AIML files with <topic> get rejected with errors

Postby GeekCaveCreations » January 25th, 2015, 9:30 pm

Ok, I found the problem. That line of code (admin/upload.php, line 170) is a throwback from the old way that the AIML was added to the database, and needs to be removed. I've already removed it for the next update, but you can gat past it by simply removing that line manually. The update that I'm working on will take me several hours to put up on GitHub (possibly longer if I get distracted), but I'll get it up as soon as I can. :)
Comforting the disturbed, and disturbing the comfortable
Chat with Morti
User avatar
GeekCaveCreations
Safe, Reliable Insanity, Since 1961
 
Posts: 1115
Joined: April 18th, 2011, 10:52 pm
Location: Nevada, USA

Re: AIML files with <topic> get rejected with errors

Postby Alekz » January 25th, 2015, 11:09 pm

Yes! It works! Thanks much!
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm

Re: AIML files with <topic> get rejected with errors

Postby Alekz » January 26th, 2015, 2:38 am

Ha, too early... The bot does not see any categories if <topic name="xxx"> is mentioned in the file. For example:

Code: Select all
<?xml version="1.0" encoding="ISO-8859-1"?>
<aiml>

<topic name="test">
<category>
<pattern>Hitest</pattern>
 <template>Hellotest</template>
</category>
</topic>

<category>
<pattern>Topictest</pattern>
 <template>Hellotopictest</template>
 <think> <set name="topic">test</set></think>
</category>

</aiml>


Neither Hitest, nor Topictest works. But! Everything is shown correctly in the Search (admin/index.php?page=editAiml)

As soon as I remove <topic> tags, everything works fine again, but with no topic support, of course.
Last edited by Alekz on January 26th, 2015, 1:56 pm, edited 1 time in total.
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm

Re: AIML files with <topic> get rejected with errors

Postby GeekCaveCreations » January 26th, 2015, 7:41 am

Well, at least we have some improvement. I'll create some test files and see where things are going wrong. It probably won't be until Wednesday before I can get into it. I have appointments on Monday and Tuesday.
Comforting the disturbed, and disturbing the comfortable
Chat with Morti
User avatar
GeekCaveCreations
Safe, Reliable Insanity, Since 1961
 
Posts: 1115
Joined: April 18th, 2011, 10:52 pm
Location: Nevada, USA

Re: AIML files with <topic> get rejected with errors

Postby Alekz » March 11th, 2015, 6:01 pm

Any chances you can have a look at this issue? The files with the "topic" tag are still ignored... And it would be nice to have the topics work.
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm

Re: AIML files with <topic> get rejected with errors

Postby GeekCaveCreations » March 13th, 2015, 4:52 pm

Sorry for the delay, Alekz. I found the problem (it's a different one, though related), but it's forcing me to completely redesign the function that scores the AIML categories. Without getting too technical, there's a certain hierarchy of steps that need to take place in order to properly score the various AIML categories, and while all of the steps were taking place in the old code, they were being taken out of order. With this in mind, I basically started "from scratch", and actually wrote out the sequence of steps to take to properly score the AIML, then I started on the new code, carefully following those steps. Just in case anyone is interested, here is the list of steps that I wrote:

Code: Select all
AIML match sequence:

1.) if the current category is for the current bot add 500 points: else, add 0 points

2.) if there is a current topic:
   a.) create a RegEx for the current category's topic, replacing _ with (.*)
   b.) if the RegEx matches the current topic add 50 points
   c.) if the current topic is a direct match with the category topic add 25 points
   d.) if the current category has no topic add 0 points
   e.) if the current category has a non-matching topic, make score -1 and reject

3.) if the current category has a thatpattern:
   a.) create a RegEx for the current category's thatpattern, replacing _ with (.*)
   b.) if the RegEx matches the current thatpattern add 50 points
   c.) if the current topic is a direct match with the category thatpattern add 25 points
   d.) if the current category has a non-matching thatpattern, make score -1 and reject

4.) if the current category's pattern contains wildcards:
   a.) create a RegEx for the current pattern, replacing _ with (.*)
   b.) if RegEx matches current input add 50 points
   c.) if current input is a direct match with the pattern add 10 points (match without wildcards)
   d.) if current pattern == * add 1 point
   e.) create a new Regex for the current pattern, replacing _ and * with (.*?)
   f.) if RegEx matches current pattern add 1 point

5.) if nothing matches so far, break the category's pattern into an array of words, and iterate through the array
   a.) if the current word is an underscore, add 50 points
   b.) else if the current word is also in the input string, add 2 points
   c.) else if the current word is on the list of "common" words, only add 1 point
   d.) else if the current word is a star wildcard add 1 point
   e.) else score the word as "uncommon", adding 8 points

6.) if a suitable match still can't be found, use the default bot response


Please note that the point values listed above reflect those used in the old code (more or less), and that one part of the redesign process involves the reformulation of the scores used, to avoid selecting the wrong category. I'm still running tests on the new code, but so far it looks very promising. The new code is cleaner, is pretty well documented, and uses a better score tracking method that's easier to understand, so I'm well pleased so far. I HOPE to have this completed in the next day or so, but by now we all know what happens when I put a completion date on these things. :P

So bottom line, I'm asking for just a bit more patience here. :)
Comforting the disturbed, and disturbing the comfortable
Chat with Morti
User avatar
GeekCaveCreations
Safe, Reliable Insanity, Since 1961
 
Posts: 1115
Joined: April 18th, 2011, 10:52 pm
Location: Nevada, USA

Re: AIML files with <topic> get rejected with errors

Postby Alekz » March 14th, 2015, 12:09 am

GeekCaveCreations wrote: Just in case anyone is interested, here is the list of steps that I wrote:


Yes! This is great to have it on "paper" as a reference. Thanks much!
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm

Re: AIML files with <topic> get rejected with errors

Postby Alekz » March 14th, 2015, 12:16 am

Do I understand correctly that a direct category match scores higher than "_", or "_" plus topic? I thought it should be "_", then the direct category match, no?
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm

Re: AIML files with <topic> get rejected with errors

Postby GeekCaveCreations » March 14th, 2015, 12:27 am

Actually, no. An underscore wildcard match is the highest priority match there is, superseding even a direct match. This is why you have to use the underscore wildcard not only wisely, but rarely, as well. As I mentioned earlier, I'm still tweaking the scores, but the following seems to work right now:

Code: Select all

    
# full pattern match scores:
    $this_bot_match               = 250;
    $underscore_match             = 100;
    $topic_underscore_match       = 80;
    $topic_direct_match           = 50;
    $topic_star_match             = 10;
    $thatpattern_underscore_match = 45;
    $thatpattern_direct_match     = 15;
    $thatpattern_star_match       = 2;
    $direct_pattern_match         = 10;
    $pattern_direct_match         = 7;
    $pattern_star_match           = 1;

    # individual word match scores:
    $uncommon_word_match      = 8;
    $common_word_match        = 1;
    $direct_word_match        = 2;
    $underscore_word_match    = 25;
    $star_word_match          = 1;
    $rejected                 = -1;
 


While I was busy redesigning the code, I also took the time to create expressive variable names to make the chore of reading the code easier. ;)
Comforting the disturbed, and disturbing the comfortable
Chat with Morti
User avatar
GeekCaveCreations
Safe, Reliable Insanity, Since 1961
 
Posts: 1115
Joined: April 18th, 2011, 10:52 pm
Location: Nevada, USA

Re: AIML files with <topic> get rejected with errors

Postby Alekz » March 14th, 2015, 1:29 pm

Ah, OK, for some weird reasons I confused "this bot" with "direct match" (probably because I work with only one bot :D). So, as expected "_" has a higher priority than "direct". Thanks for the scores! I'm a bit closer to understand how the program works.
Last edited by Alekz on March 14th, 2015, 9:49 pm, edited 1 time in total.
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm

Re: AIML files with <topic> get rejected with errors

Postby GeekCaveCreations » March 14th, 2015, 1:46 pm

At present, I have a total of 17 chatbots installed, most of them having a "default" parent chatbot, so having a way to put a priority on responses from the current chatbot is important. :)
Comforting the disturbed, and disturbing the comfortable
Chat with Morti
User avatar
GeekCaveCreations
Safe, Reliable Insanity, Since 1961
 
Posts: 1115
Joined: April 18th, 2011, 10:52 pm
Location: Nevada, USA

Re: AIML files with <topic> get rejected with errors

Postby Alekz » May 13th, 2015, 2:11 pm

Just curious, were you able to successfully test the <topic> tags using the latest version?
Alekz
Avid Member
 
Posts: 75
Joined: January 25th, 2015, 2:47 pm


Return to Bugs and Help

Who is online

Users browsing this forum: No registered users and 0 guests

cron