Error uploading AIML files

Help and bugs.

Error uploading AIML files

Postby flappy » August 6th, 2011, 6:26 pm

Hi,

First of all, from what I can see so far, you've done a great job with Program-O. I've been looking for a good AIML interpreter/front-end that I can maintain myself for quite some time and stumbled across this. I've been playing around with it for a few days now and am really enjoying it.

I have a question about the upload function. I have a set of AIML files that I wanted to try in place of the AIML you deliver with Program-O. I have deleted the contents of the 'aiml' table and began uploading my AIML files one-by-one. As a test, I started with the original A.L.I.C.E. files. When I try to upload a large file (ie default.aiml, reduce.aiml, or any file >400k), the upload page hangs and then I get an error saying that I cannot connect to the MYSQL server.

I'm running the XAMPP 1.7.3 (next to latest version available I believe). I'm running it on a Vista box with 3GB memory. I'm assuming that I'm running into either a file size limitation or a session timeout. I did a search for the string "reduce.aiml" in the insert sql files and came up with around 5300 records. After I receive the error, I see only 3000 records in my DB.

Is there a MYSQL setting that I could tweak to allow these larger files to import? Do I need a box with more memory? My concern is that if I decide to use Program-O on a shared host (where I might not have access to these settings), I will crash the server if I try to upload/replace aiml records with a modified large AIML file.

Thanks for your great work. And finally, any rough idea on a date for 2.0? I'm looking forward to seeing the new version.
flappy
Regular Member
 
Posts: 40
Joined: August 6th, 2011, 9:11 am

Re: Error uploading AIML files

Postby GeekCaveCreations » August 6th, 2011, 8:26 pm

Hi, Flappy, and welcome to the Program O forums! :)

Currently, the upload script doesn't work well with AIML files of larger than 200KB in size. If you're trying to upload anything larger, then MySQL goes into meltdown from the sheer number of queries that the script executes. Unfortunately, the new version (2.0) currently has the same upload script, but I'm trying to work out a better way, as you'll soon read. :) For now, though it's a huge pain in the bum, I would suggest splitting your larger AIML files into multiple smaller files of less than 200KB each. Trust me, I know what I'm asking. I had to do exactly the same thing for Morti's AIML. I'm working on fixing the problem as quickly as possible.

The current version of Program O also doesn't do any validation checks on the AIML files prior to adding them to the database, and as such, may fail if the AIML file doesn't use well formed XML. I'm currently working on the upload page for a later version of Program O (2.1(?) actually), which will include built-in XML validation, but that's still a ways off. In the mean-time, you can validate your AIML files manually at http://www.w3schools.com/xml/xml_validator.asp to verify that the files are ok prior to uploading them. This is something that I highly recommend for all of your AIML files.

As to when version 2.0 will be released, that's up to Liz. I've handed off the last of the install files to her, and she's wanting to test them before releasing the final product. I'll ask her what's going on when next I catch up to her. I do know that there are some pretty major changes going on in her life (which I'm not at liberty to discuss), so it may just be that she hasn't had time to get to it yet. I'll let everyone know what I can, when I can.
Comforting the disturbed, and disturbing the comfortable
Chat with Morti
User avatar
GeekCaveCreations
Safe, Reliable Insanity, Since 1961
 
Posts: 1115
Joined: April 18th, 2011, 10:52 pm
Location: Nevada, USA

Re: Error uploading AIML files

Postby flappy » August 6th, 2011, 9:37 pm

Thanks for the quick reply. I can handle validating the XML. I'd prefer not to have to split up the AIML files, however. I've been looking at update.php and noticed that a huge multi-valued insert is created. This could be what is choking MYSQL. I also noticed that the install mechanism which populates the AIML is very clean. It simply executes a series of managably sized files with individual inserts. My thoughts are that if instead of using the multi-valued insert in update.php, you created a series of temporary insert files (sized the way the install files are), and then simply performed the same import as is done on install, the server would survive. After all, the installation populates around 45k categories (I believe).
flappy
Regular Member
 
Posts: 40
Joined: August 6th, 2011, 9:11 am

Re: Error uploading AIML files

Postby GeekCaveCreations » August 6th, 2011, 11:06 pm

The function that I'm writing to handle AIML upload will use a completely different method of inserting to the database, and will be able to handle files of up to 1MB in size. It will still use multi-line statements, but will limit the SQL queries to a certain size, to prevent meltdown. Once the query reaches a size limit, it performs the query, resets the SQL statement to the beginning, and then continues with the next line. I intend to use jQuery for the front-end, so that the user will see a progress bar as the file is being uploaded. Again, though, this won't be released until after 2.0 is out.
Comforting the disturbed, and disturbing the comfortable
Chat with Morti
User avatar
GeekCaveCreations
Safe, Reliable Insanity, Since 1961
 
Posts: 1115
Joined: April 18th, 2011, 10:52 pm
Location: Nevada, USA


Return to Bugs and Help

Who is online

Users browsing this forum: No registered users and 1 guest

cron