Slimelite BBS

Component Parsing (AI)

Back to R25C

From Matthew Probert · 1996-04-07 13:19 · #75245 in R25C

                           COMPONENT PARSING 


                            Matthew Probert
                           Servile  Software


When a human receives a phrase, be it a sentence or paragraph. The 
brain breaks it down into manageable sized phrases and clauses. Much 
like the predicate parser. These phrases and clauses are then further 
broken down into component parts. These components may be summarised 
as; 


Question/Order  (The nature of the phrase)

Object          (What the phrase refers to)

Subject         (What the phrase says about the object)

Qualifiers      )
                )
Size            ) Descriptions qualifying the object and subject
                ) 
Colour          )

Number          (All, one, two etc)

Location        (Where is the object)

Action          (Verbs)

Person          (You, me, us, them, they, him, her)

Tense           (Past, present, future or indeterminable)


For example;

"What is the matter with Mary Jane?" Breaks down into phrases and 
clauses thus; 

Subject   [What] 
Predicate [is the matter with Mary Jane?]
Object    [is the matter]

The predicate further breaks down into the phrase;

Phrase    [with Mary Jane?]

Taking the phrase as a whole it breaks down into these components;

The question is WHAT IS
The order is
The object is MATTER
The qualifiers are
The colours are
The sizes are
The subject is WITH MARY JANE
The person is
The number is
The action is
The location is
The tense is present

Similarly, "It's lovely rice pudding for dinner again!" Produces;

Subject [It] 
Predicate [is lovely rice pudding for dinner again!]

The predicate breaks down into;

Object [is rice pudding]
Description [lovely]

And produces a sub-phrase;

Phrase [for dinner again!]

Analysing the phrase as a whole we get;

The question is
The order is
The object is
The qualifiers are LOVELY
The colours are
The sizes are
The subject is LOVELY RICE PUDDING FOR DINNER AGAIN
The person is
The number is
The action is
The location is
The tense is indeterminable

"This is the house that Jack built" produces the following analysis. 

Subject [This] 
Predicate [is the house that Jack built]

The predicate breaks down into an object and a sub-phrase;

Object [is the house]
Phrase [that jack built]

The sub-phrase breaks down into an object and an action (verb);

Action [built]
Object [that Jack]

Analysing the phrase as a whole we get the following summary;

The question is 
The order is 
The object is HOUSE
The object qualifiers are 
The object colours are 
The object sizes are 
The subject is JACK
The person is 
The number is 
The action is BUILT
The location is 
The tense is indeterminable
Enter a phrase


A component parser is a fairly complex affair. But can be a useful 
addition to existing parsers. The component parser has some 
similarities to a predicate parser. The lexicon of verbs (actions) 
is the same for example. 

---------
Listing 5
---------

/* COMPONENT PARSER */

/*---------------*/
/* Include files */
/*---------------*/
#include <stdio.h>
#include <string.h>
#include <alloc.h>
#include <ctype.h>

/*-----------*/
/* Constants */
/*-----------*/
#define num_a    140
#define num_p    12
#define num_q    41
#define num_l    20
#define num_n    37
#define num_c    14
#define num_f    17
#define num_qs   19
#define num_s    9
#define num_o    3

/*-----------*/
/* Lexicons  */
/*-----------*/
static char *actions[num_a] = { "LIKE" , "DISLIKE", "LOVE" , "HATE" , "SLEEP" ,
                                "PUMP", "MAKE" , "WEAR" ,"RENOUNCE", "SHORTEN",
                                "KEEP", "FEED", "DESCEND" , "CONVERT", "KEEP",
                                "OPEN" , "EAT" , "READ" , "DRY" , "KILL" ,
                                "MEND","CAUSE", "SIT" , "SAT" , "CLOSE",
                                "TAKE", "RUN" , "JUMP", "SKIP", "HOP", "BREAK",
                                "INVENT", "DISCOVER", "EXPLORE", "PUNCH","KICK"

                                "IGNITE", "CHAT", "WRITE", "TALK","LISTEN",
                                "WHISTLE","SHOUT","SCREAM","DRINK","BATHE",
                                "WASH","CROSS", "ENTER", "CLIMB", "TIE",
                                "BREATHE", "SNIFF", "SPIT" ,"GO","PUT","DRINK",
                                "LOOK", "TURN", "BORN","LEAVE", "URINATE",
                                "SMOKE", "LIGHT", "DESIGN" , "DRAW" ,"PAINT",
                                "BOIL", "STRAIN", "POUR","TELEPHONE","PHONE",
                                "CALL" ,"DRESS", "CUT", "SAW", "GONE","STUDY",
                                "FOLD" , "PLAY", "PAY", "EARN","MEET", "MET",
                                "LIVE","DWELL", "BEEN", "HEAR" , "HEARD",
                                "SING" , "SUNG", "DEVELOP", "STIR","SEE",
                                "SEEN", "RING", "FEEL" , "FELT", "DIED", "HUNT"

                                "ASK", "STAY" ,"KISS", "CUDDLE", "GIVE","BUY",
                                "BOUGHT", "LICK" , "TASTE", "ADMIRE" ,"PROVIDE"

                                "COME", "BUILD", "BUILT", "REFERENCE" ,"WENT",
                                "CONSTRUCT","LEAN","WALK","CARRY","RIDE",
                                "ORBIT","DRIVE" , "DROVE" , "DOWN", "ALONG", 
                                "SHALL","SHOULD","THINK", "THOUGHT","GOT",
                                "CROUCH","CRAWL", "GET","LEND",
                                "LENT","HAVE", "HAS" , "HAD","SUGGEST" };


static char *questions[num_qs] = { "WHAT", "WHEN", "HOW", "WHY", "WHERE",
                                   "WHO","DO", "DOES" ,"ARE", "IS", "WAS",
                                   "CAN" ,"AM" ,"WHICH","HAVE" , "WILL" ,
                                   "DID" , "HAS", "SHOULD"};

static char *orders[num_o] = { "USE" , "TELL", "QUIT" };

static char *fillers[num_f] = { "IS", "THE" , "A", "AN" , "ARE" ,"TOO","ALSO",
                            "OF" , "COME" , "FROM" , "ITS" , "THEIR" ,
                            "THAT" , "ABOUT", "THING" , "THIS" , "BECAUSE" };

static char *qualifiers[num_q] = {"FAST", "SLOW", "ROUND", "SQUARE",
                        "HAIRY", "BALD", "FRIGHTENING","INTERESTED",
                        "SHY", "NICE", "HORRIBLE" , "LOVELY","INTERESTING",
                        "PRETTY" , "BEAUTIFUL" , "BLONDE" , "BRUNETTE" ,
                        "SEXY" , "SHINY" , "DULL" , "BORING" , "CAREFUL" ,
                        "HARD" , "SOFT" , "GENTLE" , "QUICK" , "LIGHT",
                        "HEAVY", "RADIOACTIVE", "METALLIC","TIRED",
                        "GASEOUS", "BRIGHT", "SWEET", "SOUR" , "BITTER",
                        "GOOD","BAD","DRUNK","STUPID","MUTUAL" };


static char *sizes[num_s] = {"BIG", "SMALL", "FAT", "THIN", "TALL" ,"SHORT",
                             "LARGE", "PETITE", "TINY" };

static char *colours[num_c] = { "BLACK", "BLUE", "RED", "YELLOW", "ORANGE",
                             "GREEN","BROWN" , "GINGER" , "PURPLE",
                             "MAUVE", "SILVERY", "SILVER", "GOLD", "CYAN" };

static char *numbers[num_n] = { "ALL" , "SOME" , "FEW" , "MANY" , "MOST" ,
                           "COUPLE","NEXT","PREVIOUS","ONE", "TWO", "THREE" ,
                            "FOUR", "EVERY", "LAST",
                           "FIVE", "SIX", "SEVEN","EIGHT", "NINE", "TEN",
                           "ELEVEN","TWELVE", "THIRTEEN","FOURTEEN",
                           "FIFTEEN", "SIXTEEN", "SEVENTEEN", "EIGHTEEN",
                           "NINETEEN","TWENTY", "THIRTY", "FORTY",
                           "FIFTY", "SIXTY","SEVENTY", "EIGHTY",
                           "NINETY" };


static char *pronouns[num_p] = { "THEM", "THEY", "WE", "HIM", "HER" ,"US" ,
                                 "HE", "I", "YOU" ,"MY", "YOUR" , "ME"};

static char *locations[num_l] = { "IN" , "ON", "BESIDE" , "EAST" , "WEST" ,
                                  "NORTH" ,"SOUTH" , "UNDER" , "UP" ,
                                  "DOWN", "BELOW", "ABOVE", "RIGHT", "LEFT",
                                  "OVER","INTO","AROUND" , "TO", "OFF",
                                  "THERE" };

/*------------*/
/* Work areas */
/*------------*/
static char last_subject[80];
static char last_object[80];


/*---------------------*/
/* Function prototypes */
/*---------------------*/
int IS_COLOUR(char *);
int IS_FILLER(char *);
int IS_LOCATION(char *);
int IS_NUMBER(char *);
int IS_PRONOUN(char *);
int IS_SIZE(char *);
void strchg(char *, char *, char *);
void truncstr(char *,int );

/*--------------------*/
/* Start of functions */
/*--------------------*/
static void GET_QUESTION(char *text, char *question)
{
    /* Extract a question from the phrase string */

    char *p;
    char first_word[40];
    char save;
    int n;
    int more;

    strcpy(question,"");

    do
    {
        more = 0;
        p = strpbrk(text," ,.;:?!");
        save = *p;
        *p = 0;
        strcpy(first_word,text);

        *p = save;

        for(n = 0; n < num_qs; n++)
        {
            if (stricmp(first_word,questions[n]) == 0)
            {
                /* store question */
                if (*question)
                    strcat(question," ");
                strcat(question,first_word);
                /* remove question from text */
                if (p)
                {
                    while(strchr(" ,.:;?!",*p) && *p)
                        p++;
                    if (*p)
                        memmove(text,p,strlen(p) + 1);
                }
                else
                    *text = 0;
                more = 1;
            }
        }
    }
    while(more);
}

static void GET_ORDER(char *text, char *order)
{
    /* Extract an order from the phrase string */

    char *p;
    char first_word[40];
    char save;
    int n;

    strcpy(order,"");
    p = strpbrk(text," ,.;:?!");
    save = *p;
    *p = 0;
    strcpy(first_word,text);
    *p = save;

    for(n = 0; n < num_o; n++)
    {
        if (stricmp(first_word,orders[n]) == 0)
        {
            /* store order */
            strcpy(order,first_word);
            /* remove order from text */
            while(strchr(" ,.:;?!",*p))
                p++;
            memmove(text,p,strlen(p) + 1);
            return;
        }
    }
}

static void GET_OBJECT(char *text, char *object, char *action)
{
    /* Obtain object from phrase string */

    char *p;
    int n;
    int ignore;
    char temp[30];
    char copy[250];

    strcpy(copy,text);
    p = strtok(text," ,.;:?!");

    while(p)
    {
        ignore = 0;

        if (stricmp(p,action) == 0)
            ignore = 1;

        if (!ignore)
            ignore = IS_FILLER(p);

        if (!ignore)
            ignore = IS_NUMBER(p);

        if (!ignore)
            ignore = IS_COLOUR(p);

        if (!ignore)
            ignore = IS_SIZE(p);

        if (!ignore)
            ignore = IS_LOCATION(p);

        if (!ignore)
            ignore = IS_PRONOUN(p);

        if (!ignore)
        {
            for (n = 0; n < num_q; n++)
            {
                if (stricmp(p,qualifiers[n]) == 0)
                    ignore = 1;
                else
                {
                    strcpy(temp,qualifiers[n]);
                    if (temp[strlen(temp) - 1] == 'E')
                    {
                        temp[strlen(temp) - 1] = 0;
                        if (temp[strlen(temp) - 1] == 'L')
                            temp[strlen(temp) - 1] = 0;
                    }
                    strcat(temp,"LY");
                    if (stricmp(p,temp) == 0)
                    {
                        ignore = 1;
                        break;
                    }
                }
            }
        }
        if (!ignore)
        {
            for(n = 0; n < num_qs; n++)
            {
                if (stricmp(p,questions[n]) == 0)
                {
                    ignore = 1;
                    break;
                }
            }
        }

        if (!ignore)
        {
            if (stricmp(p,"IT") == 0)
                strcpy(object,last_object);
            else
                strcpy(object,p);
            strcpy(text,copy);
            p = strstr(text,object);
            strcpy(last_object,"");

            if (p)
            {
                /* Capture preceding preposition */
                p--;
                while(*p == 32)
                    p--;
                while(*p != 32 && p >= text)
                    p--;
                if (*p == 32)
                    p++;
                if (strncmp("THE ",p,4) == 0)
                    strcpy(last_object,"THE ");
                else
                if (strncmp("the ",p,4) == 0)
                    strcpy(last_object,"the ");
            }

            strcat(last_object,object);
            return;
        }

        p = strtok(NULL," ,.;:?!");
    }
    strcpy(text,copy);
    strcpy(object,"");
}

static void GET_SUBJECT(char *text, char *subject, char *object,
                        char *action)
{
    /* Obtain subject from phrase string */

    char *p;
    int ignore;
    char copy[250];

    strcpy(copy,text);
    strcpy(last_subject,subject);
    strcpy(subject,"");

    /* Restrict phrase to that part following the object */
    p = strstr(text,object);
    if (p == NULL)
        p = strstr(text,"IT ");
    memmove(text,p,strlen(p) + 1);

    p = strtok(text," ,.;:?!");

    while(p)
    {
        ignore = 0;
        if (stricmp(p,object) == 0)
            ignore = 1;

        if (!ignore)
            ignore = IS_FILLER(p);

        if (!ignore)
            ignore = IS_LOCATION(p);

        if (!ignore)
            ignore = IS_PRONOUN(p);

        if (!ignore)
            ignore = IS_COLOUR(p);

        if (!ignore)
            ignore = IS_SIZE(p);

        if (!ignore)
            ignore = IS_NUMBER(p);

        if (stricmp(p,action) == 0)
            ignore = 1;

        if (!ignore)
        {
            if (stricmp(p,"IT"))
            {
                if (*subject)
                    strcat(subject," ");
                strcat(subject,p);
            }
        }
        p = strtok(NULL," ,.;:?!");
    }
    strcpy(text,copy);
}

static void GET_OBJECT_QUALIFY(char *text, char *object_qualify,
                               char *subject)
{
    /* Obtain qualifying statements from phrase string */

    char *p;
    int n;
    int ignore;
    char oq[78];
    char copy[250];
    char temp[30];

    strcpy(copy,text);
    strcpy(object_qualify,"");

    p = strtok(text," ,.;:?!");

    while(p)
    {
        for(n = 0; n < num_q; n++)
        {
            if (stricmp(p,qualifiers[n]) == 0)
            {
                if (*object_qualify)
                    strcat(object_qualify,",");
                strcat(object_qualify,p);
                break;
            }
            else
            {
                strcpy(temp,qualifiers[n]);
                if (temp[strlen(temp) - 1] == 'E')
                {
                    temp[strlen(temp) - 1] = 0;
                    if (temp[strlen(temp) - 1] == 'L')
                        temp[strlen(temp) - 1] = 0;
                }
                strcat(temp,"LY");
                if (stricmp(p,temp) == 0)
                {
                    if (*object_qualify)
                    strcat(object_qualify,",");
                    strcat(object_qualify,p);
                    break;
                }
            }
        }
        p = strtok(NULL," ,.;:?!");
    }
    strcpy(text,copy);

    ignore = 0;

    strcpy(oq,object_qualify);

    p = strtok(object_qualify,",");
    while(p)
    {
        if (stricmp(p,subject) == 0)
        {
            ignore = 1;
            break;
        }
        p = strtok(NULL,",");
    }

    strcpy(object_qualify,oq);
    if (!ignore)
    for (n = 0; n < num_q; n++)
    {
        if (stricmp(subject,qualifiers[n]) == 0)
        {
            if (*object_qualify)
                strcat(object_qualify,",");
            s
--- WILDMAIL!/WC v4.12 
 * Origin: Ooh! BBS, London [MULTILINES]  0181-395-3108  (2:254/233.0)