Wodel-Test for Chatbots

A mutation testing tool for Chatbots generated via Wodel-Test

View on GitHub Update-site Wodel project .zip Conga ecore Meta-model Annotation ecore Meta-model Standalone Eclipse+MuT for Chatbots

Here is a list of the identified chatbot mutation operators for the Conga meta-model useful for mutation testing:

Delete training phrase (max similarity)

tpi = select one IntentValue from annotation resources
i = select one Intent where {self = tpi->intent}
li = select one LanguageIntent in i->inputs
remove one TrainingPhrase from li->inputs where {self = tpi->max1}

Delete training phrase (min similarity)

tpi = select one IntentValue from annotation resources
i = select one Intent where {self = tpi->intent}
li = select one LanguageIntent in i->inputs
remove one TrainingPhrase from li->inputs where {self = tpi->min1}

Delete training phrases with required parameter

p = select one Parameter where {required = true}
remove all TrainingPhrase where {tokens->parameter = p}
remove all LanguageIntent where {inputs = null}

Delete training phrases with literal

input = select one SimpleInput
prt = select one ParameterReferenceToken where {textReference = input.values}
remove all TrainingPhrase where {tokens = prt}

Keep two training phrases (max similarity)

tpi = select one IntentValue from annotation resources
i = select one Intent where {self = tpi->intent}
li = select one LanguageIntent in i->inputs
tp0 = select one TrainingPhrase in li->inputs where {self = tpi->max1}
tp1 = select one TrainingPhrase in li->inputs where {self = tpi->max2 and self <> tp0}
tp2 = select one TrainingPhrase in li->inputs where {self <> tp0 and self <> tp1}
remove all TrainingPhrase from li->inputs where {self <> tp0 and self <> tp1}

Keep two training phrases (min similarity)

tpi = select one IntentValue from annotation resources
i = select one Intent where {self = tpi->intent}
li = select one LanguageIntent in i->inputs
tp0 = select one TrainingPhrase in li->inputs where {self = tpi->min1}
tp1 = select one TrainingPhrase in li->inputs where {self = tpi->min2 and self <> tp0}
tp2 = select one TrainingPhrase in li->inputs where {self <> tp0 and self <> tp1}
remove all TrainingPhrase from li->inputs where {self <> tp0 and self <> tp1}

Move training phrase (max similarity)

i1 = select one Intent where {inputs <> null}
iiv = select one IntentIntentValue from annotation resources where {intent1 = i1 and max1Value = max(max1Value)}
i2 = select one Intent where {inputs <> null and self = iiv->intent2}
li1 = select one LanguageIntent in i1->inputs where {inputs <> null}
li2 = select one LanguageIntent in i2->inputs where {self <> li1 and inputs <> null}
tp1 = select one TrainingPhrase in li1->inputs where {self = iiv->max1}
tp2 = select one TrainingPhrase in li1->inputs where {self <> tp1}
modify li2 with {inputs += tp1}

Move training phrase (min similarity)

i1 = select one Intent where {inputs <> null}
iiv = select one IntentIntentValue from annotation resources where {intent1 = i1 and min1Value = min(min1Value)}
i2 = select one Intent where {inputs <> null and self = iiv->intent2}
li1 = select one LanguageIntent in i1->inputs where {inputs <> null}
li2 = select one LanguageIntent in i2->inputs where {self <> li1 and inputs <> null}
tp1 = select one TrainingPhrase in li1->inputs where {self = iiv->min1}
tp2 = select one TrainingPhrase in li1->inputs where {self <> tp1}
modify li2 with {inputs += tp1}

Delete intent parameter

p = select one Parameter
remove all TrainingPhrase where {tokens->parameter = p}
remove all ParameterToken where {parameter = p}
remove p
remove all LanguageIntent where {inputs = null}

Delete parameter prompt

p0 = select one Parameter where {required = true}
remove one LanguagePrompt from p0->prompts

Set required parameter to optional

modify one Parameter where {required = true} with {required = false}

Change regular expression

modify one RegexInput with {expresion = catstart('[^'), expresion = catend(']')}

Delete literal from entity

remove one SimpleInput

Delete fallback intent

i = select one Intent where {fallbackIntent = true}
remove all UserInteraction where {intent = i}
remove i

Delete actions

bi0 = select one BotInteraction where {actions <> null}
remove one Action from bi0->actions 

Delete a parameter used in a response

text = select one TextInput where {tokens is typed ParameterToken}
remove one ParameterToken from text->tokens

Swap outputs

bi1 = select one BotInteraction where {actions <> null and outcoming->^target <> null}
ui1 = select one UserInteraction in bi1->outcoming where {^target <> null}
bi2 = select one BotInteraction in ui1->^target where {self <> bi1 and actions <> null and actions <> bi1->actions}
modify bi1 with {swapref(actions, bi2->actions)}

Delete conversation step

ui0 = select one UserInteraction where {^target->outcoming->^target is typed BotInteraction}
bi0 = select one BotInteraction in ui0->^target->outcoming->^target
modify target ^target from ui0 to bi0

Delete conversation bifurcation

bi = select one BotInteraction where {outcoming <> null}
ui = select one UserInteraction in bi->outcoming
remove one UserInteraction from bi->outcoming where {self <> ui}

This work has been funded by the Spanish Ministry of Science (RTI2018-095255-B-I00, project "MASSIVE") and the R&D programme of Madrid (P2018/TCS-4314, project "FORTE").