Greetings Developers and Partners of Windows Live Agents,
We are very pleased to announce that WLA 5.1 is here!
It is with a heavy heart that we have to announce WLA 5.1 is the last WLA release. We are retiring our product
and this final version is better suited to take you through the summer of 2009 when official support ends.
You, our developers and partners, have provided great feedback about our product that we have incorporated into product features, many of which are part of 5.1 release. We would like to thank you for all your feedback and participation.
We will no longer accept new hosting requests. To review your current Agents, go to our Partner Hosting portal.
For tips on self-hosting, please refer to this article.
Please use http://agents.live.com/ to download SDK. This version of WLA provides a number of features that allow for better and more intuitive agents development experience. As always we highly recommend you read the release notes about all new features and use help documentation provided with SDK for any question on usage you might have.
We still encourage you to continue using our Forum site for communication with development community and WLA team.
Thank you,
Windows Live Agents Team
Windows Live Agents SDK offers the ability to self host your agent for development and testing purposes. Limited number of license sessions is granted per service (common-service, MSN, etc) per month. License session limits are imposed per license certificate; they are shared among all your agents if you have multiple ones running in SDK management console. Currently, the standard license certificate offers 500 common-service sessions, and 500,000 MSN sessions per month. Each service also gets 50,000 burst sessions, we will explain the burst sessions a little later.
Self Host Your Agent(s):
To start self-hosting, you need to have Visual Studio 2005 or 2008 and Windows Live Agents SDK installed and a Windows Live account. Please refer to the Windows Live Agents SDK help file for detailed steps on how to obtain a license certificate. (Windows Live Agents SDK -> Getting Started with Windows Live Agents SDK -> How to: Install the License Key)
To add and start your agents through the SDK management console, please refer to the Windows Live Agents SDK help file. (Windows Live Agents SDK -> Windows Live Agents SDK Developer’s Guide -> Launching and Testing Your Agents)
Self-Host Limitations
No redundancy or failover capabilities.
No load balancing.
Limited number of license sessions per month.
License Sessions Details:
The license sessions renew daily at 1/30th of the monthly limit for each service. Please note, burst sessions are not renewable, it is for the lifetime of the license.
For example: common-service sessions renew at 17 sessions per day, and MSN sessions renew at 16667 sessions per day.
How are license sessions counted? One license session is one continuous hour of interactivity with the agent through a service (ex. HTTP, MSN) for one user starting from the first time the user engages the agent. The license session continues whether the user interacts with the agent or not after the initial request.
For example, user U1 chats with agent A1 at 10:00 pm for 5 minutes, then chats with agent A1 again at 11:01 pm for another 5 minutes, two sessions will be counted; user U2 chats with Agent A2 at 10:00 pm for 30 minutes, then chats with the agent again at 10:45 for 5 minutes, only one session will be counted.
When all license sessions for one service are used, burst sessions for that service will be used, and the agent will be running in demo-mode for that service. While in demo-mode, the agent will only interact with new users for one license session through that service. The agent will stop interacting with users through the service when both regular and burst sessions for that service run out.
Please note that one license session is used per hour when you are connected to the SDK management console (http://localhost:8000), to avoid this, simply close your connection (browser) to the SDK management console when not needed.
As an Agents developer, you need to be able to handle and present data accurately and efficiently. There are many string parsing, manipulation, and examination functions in the Buddyscript arsenal. Many are of the standard variety that are found in most programming languages. The more common and obvious ones that follow the same concepts as most other high-level computer programming languages are the following (for more details on these, you can look at the online help under Procedures and Functions in the String Handling section):
Common String Parsing Functions:
StringSub - the most common substring function that gets a part of a string. Don't confuse this with the StringSubstitute function, which does string replacing.
StringGrabPart - this function splits out a string into different parts based on a given separator. A number is provided as one of the arguments in the function, indicating which part to return. Compare this with StringSplit.
StringSplit - given a string, splits the string into parts based on a separator string. These parts are put into an object list.
Common String Manipulation Functions:
StringUppercase - uppercases a string
StringUppercaseFirst - uppercases only the the first letter in the string
String UppercaseInitials - uppercases the first letter in each word of the string
StringLowercase - lowercases a string
StringConcat - concatenates two or more strings. Each string to be concatenated is a parameter.
StringTrim - trims off leading and trailing spaces from a string
StringLTrim - trims off leading spaces from a string
StringRTrim - trims off trailing spaces from a string
StringSubstitute - given a target string, replaces all occurences of one string with another string. Don't confuse this with StringReplace (see below)
StringReplace - given a target string, replaces a string with another string at a particular position in the string.
StringFormat - similar to the printf function in standard C programming language library, where a format is given to define the string.
Common String Examination functions:
StringLength - returns the length of a string
StringSearch - finds the position number (Buddyscript uses an offset starting at 0 in its string) in a string, given a target string.
StringStartsWith - used as a conditional (true or false, comparing a string with a desired start-with string to see if the string starts with the start-with string
StringEndsWith - used as a conditional (true or false, comparing a string with a desired end-with string to see if the string ends with the end-with string
StringAscii - returns the ascii value of a character
StringChar - the opposite of StringAscii - returns the character value of an ASCII code
Buddyscript does provide a few more string parsing and manipulation functions that are not as common to programming languages, but are very powerful. Some of these functions are quite useful. I'm going to go into a little more depth on some of these functions.
The concept of a thawed string - Using StringClean
Let's say you need to capture data that can come in in different formats. For example, a telephone number. A telephone number could come in looking like 650-555-1212, (650) 555-1212, 650.555.1212, and so on. One pretty solid approach you could take is to use the StringClean function to normalize the data. What the StringClean function does is to convert a string into its thawed state. The term 'thawed' is something germaine to Buddyscript. A thawed string is a string that has been stripped of all special characters except for letters and numbers.
Here is a little test code snippet:
+ _testme TESTSTRING=Anything
STRINGTHAWED = StringClean(TESTSTRING)
STRINGNORMALIZED = StringSubstitute(STRINGTHAWED, " ", "")
- STRINGNORMALIZED
Let's run this: _testme (650) 555-1212
The STRINGTHAWED variable will look like: 650 555 1212
By then replacing all whitespace characters with nothing using the StringSubstitute function, the STRINGNORMALIZED variable will then contain 6505551212.
Hashing in Buddyscript - Using StringHash
A hash is usually used as a numerical representation of a longer string. In many programming languages, It can be used as a way to better index a set of data, i.e. indexing on the hash rather than on a long string. A hash can also be used to check that a string or computer file has not been altered. We can do this by comparing an expected hash value with the hash value of the string or file. The form of the syntax for StringHash is as follows
HASH = StringHash(URL, 2147483647)
The 2nd number is the maximum number allowed for hash values. You can specify any number up to 2147483647. This number is 2^31 -1, which is the absolute maximum you can specify. Any number greater will return zeroes. For example, the hash generated from the statement StringHash("http://www.microsoft.com", 2147483647) would be 251890749.
Also, note that there are also two other hashing functions that are undocumented, GetMD5 and GetSHA1, which are also built-in. These use the MD5 and the SHA-1 encryption algorithms. Both return a hexadecimal string.
HEXSTRING = GetMD5(String)
HEXSTRING = GetSHA1(String)
Using the regex functions
If you're not familar with regular expressions, hang on to your seat belt. Regular expressions are commands that also do string processing. But think of regular expressions as string processing on steroids. With a small set of commands, one can do a lot of very flexible and powerful processing of text data.
BuddyScript has a slew of string functions that accept regular expressions as a means for searching and manipulate data. Let's look at a real world example.
Currently you are reading my blog post right now. So let's say, just as a crude example, that you wanted to find the last posting date in this blog and report it in your agent. So how would we do this?
This is known as (web)site scraping. In general, this is not a good practice, since HTML can change all the time at the whim of a web page designer. It is far better to use a web service that has an RSS feed or an XML feed that contains this data. (Also, hitting against a website too often can lead to administrators thinking that there is a Denial of Service attack going on, so don't try this often). Having said this, let's pretend that no feeds exist, and you really have to grab the date from the website.
First of all, you'd set up a datasource to access the data, and then put the data into a variable. Here's an example:
datasource dsBlog() => PAGE
http
http://blogs.msdn.com/windowsliveagents/
DATA = dsBlog()
The variable DATA now contains all the HTML, most of it which is fairly unreadable unless you are an HTML guru. We know that the most recent posting date is between the word "Posted" and the word "by" (as I said before this is a very unstable technique but bear with me). We can now use the StringMatch function to grab this specific data. The StringMatch function has 3 parts to it, the string, the regular expression, and the object hash where the splitting of the data will go into for the regular expression.
The actual HTML snippet might look something like this:
"Posted <a id="postlist" href="http://blogs.msdn.com/feedback.aspx">Saturday, September 13, 2008 12:32 AM</a> by"
The statement to extract this would look something like this:
EXTRACT = ()
MATCHES = StringMatch(TESTSTRING, "Posted(.*?)\.aspx\">(.*?)</a> by", EXTRACT)
- EXTRACT[1]
We allocate a list called EXTRACT, which will contain the contents as specified in the regular expression inside the parenthesis, (.*?). So what does .*? mean? The '.' dot represents matching any one character. The '*' is a quantifiers. The '*' means to grab all characters. The ? means to look for the first occurence. There is a lot to regular expressions which this blog post won't be able to cover.
EXTRACT[0] will thus contain <a id="postlist" href="http://blogs.msdn.com/feedback
EXTRACT[1] will contain Saturday, September 13, 2008 12:32 AM
MATCHES will contain the number 2, since there are 2 matches found.
There are other string functions that use regular expressions as input as well. Once again, look at the online documentation for more details.
StringSplitRegex - is like StringSplit, but instead of a fixed separator value, a regular expression is used instead.
StringSubstituteRegex - is like StringSubstitute, but instead of a fixed string to be looked for for replacing, a regular expression can be used instead.
Today we're going to talk again about debugging your agent's inner workings, but this time through the use of debug output instead of the step-by-step debugger. This is generally where you want to go to check up on what's happening behind the scene if your agent's output is not helping enough. The following contains scary exhaustive lists of obscure words, but fear not, the good news is that you won't need to remember any of them!
In the SDK, the queryserver's output can be seen in the Output tab when the agent is running, next to Conversation.
When running the Agent in the console, you can find the same data under Monitoring and Control > View Component Logs, then checking the latest Queryserver logfile.
The level of debug is highly customizable; you can pick and choose the outputs important to you, whether you want to know what's happening to your datasources, how the scoring is done exactly, why the agent is taking so long to startup, the order in which the domains and packages are being compiled, how your supercomplex routine ends up in a loop, what's happening with the notifications....
There are various ways of setting the debug level:
- In the SDK, your agent's properties have a field called Debug, which takes the argument "tag1 tag2 tag3...."
- In the SDK when the agent is running, you can change the level at any time using the command "!d tag1 tag2 tag3..."
- For a console agent, you can change it in the config file under Query Server:
<debug>tag1 tag2 tag3...</debug>
- For a running console agent, there is also a way - see at the end of this post for more details!
So what exactly would you put in this TAGS list?
Well the easiest approach is to use the preset debug levels. They are numbered from 0 to 9, 0 outputing nothing and 9 outputting per minute the equivalent of the whole contents of the Library of Alexandria.
By default, the SDK uses level 5 which is a good compromise but still quite verbose, while console Agents are usually set to 1 or 2 due to the amount of traffic they can get.
Here are, for future reference, the tags associated with each level:
1 = platforminfo domainerror matchingtestupdate extrainfo qddpevents qdmrevents qdmisc externalcomp dispatcherrunlevel miscloadingerrors preloadingagentfailed idsapiaccess
2 = 1 + platforminfoex plugins miscloadinginfo queryprocessor bestmatchoverall predicatereloading qduserevents qdmessages qddns sourceblocker1
3 = 2 + datasourcebasic queryconstruction querytimings gatewaynotfound gatewayfailure gatewaydbnotfound gatewayblocked externalext executionrephrase scriptexe1 threadid sourceblocker2 matcherlow agentpreloaded miscloadingdetails
4 = 3 + userprofile domainallstats match moreenum
5 = 4 + datasourceaverage conditiontreebasic userprofileex keepallmaybeyoumeant scriptexe2 timedactions matcheraverage
6 = 5 + conditiontreeaverage matchpossible contextbasic presentationfeed
7 = 6 + datasourcedetailed conditiontreedetailed patternexplosionbasic tokengrouping compilationrephrase scriptexe3 xslttrransform1
8 = 7 + patternexplosiondetails matcherhigh sourceblocker3
9 = 8 + displayallunify currentcommand patternexplosionfull xslttrransform2
For the global setting, using such a number can be quite enough to define how much general output you want to see, but let's say you have a problem with your datasources and want to see what happens with them without getting too distracted with the rest. Well you could call the command : "!d 1 datasourcebasic" which resets the main debug level to 1, then adds "datasourcebasic" on top of it. Note that the tags are cumulative, so if you were to call it again with: "!d datasourceaverage", your debug would then contain all of 1 + datasourcebasic + datasourceaverage. On the other hand, calling a numbered level resets to that level, so "!d currentcommand 1" is the same as "!d 1".
Now, you've noticed that if you're only interested in datasources, you still have to list a number of tags to achieve the level of debug you want. Here comes another helper: the tag groups! Much like the numbered levels, these help you by setting a bunch of tags at once:
platform = platforminfo platforminfoex
miscloading = miscloadinginfo miscloadingerrors miscloadingdetails
init = plugins conditiontreebasic conditiontreeaverage conditiontreedetailed dispatcherrunlevel
up = userprofile userprofileex
domain = domainerror domainallstats
context = contextbasic
pattern = patternexplosionbasic patternexplosiondetails patternexplosionfull
matching = bestmatchoverall match keepallmaybeyoumeant matchpossible displayallunify matchingtestupdate matcherlow matcheraverage matcherhigh
conditions = conditiontreeaverage conditionsbasic conditionsdetailed patternsbasic patternsdetailed
script = extrainfo moreenum currentcommand
gateway = gatewaynotfound gatewayfailure gatewaydbnotfound gatewayblocked
sourceblocker = sourceblocker1 sourceblocker2 sourceblocker3
xslttrransform = xslttrransform1 xslttrransform2
rephrase = executionrephrase compilationrephrase
qdbase = qddpevents qdmrevents
qdsmart = qdbase qduserevents qdmessages qddns up querytimings
qdall = qdsmart qdthread
datasource = datasourcebasic datasourceaverage datasourcedetailed
external = externalcomp externalext
thread_id = threadid
Another practical example: let's say you don't want too much information in your log window, and decide to set the debug level to 2. It just so happens that this poses a problem with the Comprehension Info tab, as that tab requires special debug outputs (notably matcheraverage, found in debug level 5) to be able to present you with the rephrasing rules details and scores reliably. Normally, you would keep the debug level 5 and feel sorry for all the virtual paper being wasted and the virual forests becoming extinct, but now you know better! All you need to do is to change the setting to "1 matching rephrase", or even "1 executionrephrase matcheraverage" to be super conservative.
A few words about performance.
As you can imagine, outputting so much data to the screen or to a file takes up resources, so whenever performance is a concern, you want to be conservative about the level of debug. Here are two ways to help without hurting:
- if your agent has a good-size codebase, starting it up can take a bit of time depending what it loads. So in the SDK where you require numerous restarts, you don't want to add on top of that the cost of outputting huge quantities of debug output you don't need at start up.
Lo and behold! the 'debug-startup' option is your new friend. It tells the queryserver what debug level to adopt until it's fully started. In the SDK you use it by adding "--debug-startup FLAGS" to the Extra Options field in the agent's properties. **
- For a console agent already running and live, as noted before you want a pretty terse debug level due to the traffic, but you may need to change it on the fly to investigate a particular issue. For this, Buddyscript has a built-in procedure called SetDebugLevel()
+ setdebuglevel TAGS=AnythingStrong {MACRO_PROTECTED_MATCHING}
- Changing the debug level to TAGS...
call SetDebugLevel(TAGS)
However make sure you turn it back once you're done investigation! One safe way of implement in it is with the following code:
variable G_CHANGED_DEBUG_LEVEL = false
+ setdebuglevel TAGS=AnythingStrong {MACRO_PROTECTED_MATCHING}
- Changing the debug level to TAGS...
G_CHANGED_DEBUG_LEVEL = true
call SetDebugLevel(TAGS)
procedure ABEndSessionProc()
if G_CHANGED_DEBUG_LEVEL
call SetDebugLevel("2")
This is assuming that your 'config' debug level is 2.
And that's it! Next time you have an issue for us to look at and we ask you to send us the log with debug level set to 'currentcommand' (this one outputs the whole execution of the code), you'll know why and how.
Thanks for reading the blog and come back soon!
** an even better friend in that regard is the command "!ddlreload" in the conversation window, that doesn't restart the agent but only reloads the domains and packages that changed since the last reload. Be advised though that it won't always reload everything correctly, so a restart once in a while is recommanded.
Type "?" to get a list of all those useful and lesser-known commands.
Hello,
We are starting to plan our 5.2 Windows Live Agents SDK release. Even though we have many ideas and directions that we can take the SDK, we would love to hear what kind of improvements and features you would like to see!
Please email your suggestions to wlasdkfeedback@live.com and we will do our best to make sure you see the improvements you have requested in our 5.2 release and beyond!
Sincerely,
The Windows Live Agents Team
Windows Live Messenger translation bot now available!
The Microsoft Translator team is excited to announce the new translation bot for Windows Live Messenger! This Messenger bot does translations for you. Just add mtbot@hotmail.com to your contacts and start chatting. You can have one-on-one conversations with the bot, or you can invite a friend and chat in different languages with the bot translating for you. As usual, remember that machine translation isn’t perfect – slang especially will give the engine trouble.
You can also access Windows Live Messenger on your smartphone to use the bot to translate simple sentences while you’re traveling to other countries!
The translator bot is localized into all of the languages for which we have translation support on www.windowslivetranslator.com:
- English to/from:
- Arabic
- Chinese Simplified
- Chinese Traditional
- Dutch
- French
- German
- Italian
- Japanese
- Korean
- Portuguese
- Russian (RUS->ENU only)
- Spanish
- Chinese Simplified <-> Chinese Traditional
More languages will be rolled out over the next several months. Start using the bot and let us know your feedback! FYI - at any time while you’re chatting with the bot, just type “TBot ?” to get a list of commands that the bot understands:
Greetings,
We are pleased to announce that a new version of WLA SDK is now available for download.
Please use http://agents.live.com/ to download SDK.
Thank you,
Windows Live Agents team
Depending on the complexity of your Agent, you might want to add debug code right of the bat. That makes it easier (and less painful) to find issues you will come across. Look for examples further down the post.
How to debug your Agent:
Syntax: break [file:]line
Example: In the SDK, type “!break Numbers.pkg:291” (this will set a break point in the package “Numbers” at line 291)
Once you set a break point, enter a query that you want to investigate further. Eventually you will hit the break point you set and you can now dig in by using the debugging commands listed below.
While in debug mode, type “help” to see a list of possible commands:
Debugger static commands:
break [file:]line: sets a breakpoint
show breakpoints: lists the breakpoint
show files: lists the different files that are loaded
delete number: delete a breakpoint
delete all: delete all breakpoints
show global vars: lists all the global vars
print or p expr: evaluates the expression
start: the next execution will stop at the first instruction
debugger on|off: turns on or off the debugger
Debugger execution commands:
continue or c: continue the execution
next or n: continue to the next line
step or s: continue to the next command, entering procedure or function
step out or so: continue until outside the routine, procedure or function
backtrace or bt: lists the call stack
up: goes up in the call stack
down or do: goes down in the call stack
frame or f number: goes to a given frame in the call stack
show vars: lists all the local vars at the current frame
quit or q: terminates the execution
Note: !d [debug_level] // Displays/changes the debug level
How to add debug code to your Agent:
// Debug code to place in a .pkg file.
variable G_PROJECT_NAME_DEBUG_MODE
procedure TurnDebugMyProjectOnOff(ON_OFF)
G_PROJECT_NAME_DEBUG_MODE = ON_OFF
if ON_OFF
- Debug mode turned on!
else
- Debug mode turned off!
procedure DisplayIfDebugMode(DISPLAY)
call IncreasePresentationMaxLengthIfDebugMode()
if G_PROJECT_NAME_DEBUG_MODE
- DISPLAY
// Debug code to place in a .ddl file. This is to turn debug mode on/off in your project.
+ _debug_my_project ON_OFF=Anything {MACRO_PROTECTED_MATCHING} // i.e. type “_debug_my_project true” to activate debug mode.
call TurnDebugMyProjectOnOff(ON_OFF)
If you have a lot of different domains in your Agent that interact with each other (i.e. weather, location, etc.), then it might make sense to split up the debug code accordingly. Create some per feature, i.e. procedure DisplayIfWeatherDebugMode(DISPLAY) and procedure DisplayIfLocationDebugMode(DISPLAY), etc.
Now, place your new procedure(s) wherever you want to display debug information throughout your code. For example:
function GetWeatherChangeForInitiative(WEATHER_OBJ, PRECIP, TEMP)
call DisplayIfWeatherDebugMode(GuessWeatherLocationObject())
CHANGE_MUCH_HIGHER_DATE = ()
CHANGE_MUCH_LESS_DATE = ()
WEATHER_OBJ_KEYS = Keys(WEATHER_OBJ)
DAYS_W_NO_CHANGE_AHEAD = 0
//
for value DATE in WEATHER_OBJ_KEYS
if PRECIP
CHANGE_TO_CHECK = GetWeatherObject_ForecastChangePrecipitation(WEATHER_OBJ, DATE)
if TEMP
CHANGE_TO_CHECK = GetWeatherObject_ForecastChangeTemperature(WEATHER_OBJ, DATE)
call DisplayIfWeatherDebugMode(StringConcat("Condition to check: ", CHANGE_TO_CHECK))
//much higher
if CHANGE_TO_CHECK eq "MACRO_WEATHER_PATTERN_CHANGE_MUCH_HIGHER"
insert last in CHANGE_MUCH_HIGHER_DATE DATE
call DisplayIfWeatherDebugMode(StringConcat("Much higher on: ", DATE))
break
//much less
if CHANGE_TO_CHECK eq "MACRO_WEATHER_PATTERN_CHANGE_MUCH_LESS"
insert last in CHANGE_MUCH_LESS_DATE DATE
call DisplayIfWeatherDebugMode(StringConcat("Much less on: ", DATE))
break
//no change
if CHANGE_TO_CHECK eq "MACRO_WEATHER_PATTERN_CHANGE_NO_CHANGE"
DAYS_W_NO_CHANGE_AHEAD++
call DisplayIfWeatherDebugMode(StringConcat("Days without change: ", DAYS_W_NO_CHANGE_AHEAD))
//
// ...
Happy debugging! ;)
------------------------------------------
Blog post contributor(s): Mirco
In 5.0, we introduced a new set of features collectively called Runtime Policy Compliance. We designed Compliance to help developers comply with security policies that address potentially offensive input and output in an agent. You can get a more detailed introduction in our online docs, but in this post we specifically want to call out some new Compliance-related elements for those developers using our Partner Hosting Infrastructure. Please look for these elements coming soon to the hosting process, and adjust your planning accordingly:
· Compliance review: As part of the Critical File Review, we will review your project for potentially offensive output, and alert you of any potential issues.
· Compliance impact on hosted agents: Not a formal part of the hosting process, but an issue to be aware of. Once your agent is hosted, the Compliance features will be using data that is only available on our servers. This may impact your agent's behavior. For example: Once your agent is hosted, the Sensitive Input Auto-Handler will recognize potentially sensitive user input based on our data, and it will give a default response. Because this affects your agent's behavior, you will need to do some testing once your Agent is available in the staged hosting environment. At that point, you can identify and address any issues you find, for example by overriding the default response.
· Manual Compliance testing: Before you "Go live", we will ask that you follow our guidelines for specific testing around Compliance-related vulnerabilities, and sign a document to that effect.
Phew! We don’t like adding process any more than the next guy, but we think that together with the Compliance features, these review processes will help you effectively defend your agent against malicious users. We look forward to working with you to make the complete hosting process as smooth as possible.
When your agent launches, you will be given access to the Knowledge Management Server, which includes a Usage Reporting section. The URL is https://YourProjectName.console.agents.live.com.
When you navigate to the Usage Reporting site, you will see date range options in the left pane, and report results in the right pane. The default date is the present day. You can switch days by clicking on the calendar on the left.
If you click on “Custom Range” on the upper left side of the page, you can view usage reports for a custom date range. Please not that in this view, New & Unique Users will be shown as N/A. Due to a design limitation, we currently cannot provide accurate measurements of new and unique users for custom reporting periods.
At the top of the page, you may see the following sections of reporting: Volume Summary, User Demographics, Languages Used, Activity Usage, Compliance, Category Analysis, Topic Analysis, and Clickthroughs. The default reporting view is Volume Summary. The sections of Usage Reporting you see, depends on how you have set up the usage_config.xml your project uses. For more information on customizing the usage_config.xml, click here.
Definition of Reporting Terms:
Volume Summary Section
Total Queries:
The total number of queries in all sessions within the specified time period. For example, if the time period includes 100 sessions, there could be 1000 or more total queries.
Total Sessions:
The total number of sessions within the specified time period.
Unique Users:
The number of unique users within the specified time period. Each user is counted once, regardless of how often they interact with the agent. For example, if one person has ten separate sessions with the agent during the specified time period, they are still identified as one unique user.
New Users:
The number of users whose initial session with the agent occurred during the specified time period.
Average Queries Per Session:
Total queries within the specified time period divided by total sessions within the specified time period.
Average Sessions Per Unique Users:
Number of unique users within the specified time period divided by total number of sessions within the specified time period.
Category & Topic Analysis Sections
Category Analysis is broken out into two sections: Category Distribution Per Query and Category Distribution Per Session. Category Distribution Per Query breaks down how many queries for the chosen date range matched to a category. Category Distribution Per Session breaks down how many times a query matched on a Category during a session.
Topic Analysis is only broken out into one section: Topic Distribution (By Query). This measures how many times users’ queries matched to a specific topic for the chosen date range.
For information on how to log Category and Topic Analysis in your project, click here.
Clickthrough Section
The Clickthrough section breaks down how many times a user is presented a link and how many times a user clicks on the link.
Total Impressions
Number of times a user is presented a link in their conversation with the bot.
Total Clicks
Number of times a user clicks on the link he / she is presented.
For information on tracking ClickThroughs in your project, click here.
Languages Used Section
Agent detects what languages users speak to the agent in.
Activity Usage Section
Tracks how many users accept or reject an invitation to open the Activity Window.
Compliance Section
Conversations Stopped
Number of conversations interrupted after detecting user was typing sensitive topics beyond the scope of the agent.
Sensitive Sequences Rejected
Number of answers that were blocked by the output filter, displaying an error message to the user.
Sensitive Sequences Trusted
Number of answers that would have been blocked by the output filter, but were let through because of the use of the tag <trusted>.
Answers Invoking Trust
Number of messages that were displayed with the tag <trusted>…</trusted> in them. It includes both messages that would have passed the output filter or failed it.
If you are moving from the old Colloquis IDE to the Visual Studio-based Windows Live Agents SDK, and you've never used Visual Studio, you may be a little lost. This post aims to collect simple hints and tricks for agent developers new to working with Visual Studio. This is just a start -- if you have any hints, please do comment, and later on we'll post a sequel.
Also, this is by no means much of a guide to Visual Studio as a whole, and is really meant to get you started and oriented for agent-development tasks. Also, it's not a guide for creating agents. The first resource for using the new SDK is the
Windows Live Agents SDK documentation. You should start there before doing anything.
Agents menus
Here are some common menu locations for agent-development tasks:
Edit -> Find and Replace ... Lots of options for searching through single or multiple files. Regular expressions work!
View -> Solution Explorer ... This is rougly equivalent to the Explorer feature in the old IDE, and lets you see the project files in a tree view.
View -> Class View ... Lets you see all the project's domains and packages as a flat list of classes.
View -> Error list ... Displays compile errors and warnings.
View -> Output ... Similar to the "Misc Debug" pane of the old IDE, displays compiler messages.
View -> Other Windows -> Conversation Window ... Where you compile and talk to the agent.
View -> Other Windows -> Comprehension Info Window ... Displays match scoring information.
Project -> Add DLS Item ... Add datasources and other DLS items.
Project -> ProjectName Properties ... Specify compile parameters like buddy id, filter, and command-line options.
Tools -> Windows Live Agents Tools -> Request License Certificate ... Get a new license cert.
Tools -> Windows Live Agents Tools -> Management Console ... Launches the web management console within Visual Studio.
Tools -> Windows Live Agents Tools -> Update connections file ... Attempts to update the connections file based on the content of your project.
Tools -> Windows Live Agents Tools -> Performance ... Various tools for measuring performance.
Tools -> Options ... Set things like tabs and syntax highlighting.
Show All Files
By default, the SDK will mount a filesystem in the Solution Explorer, which may affect performance if there are a lot of files, causing the editor to lag. If this is the case, you can click "Show All Files" to remove the All Files view, and navigate to files through the Class View. Editing will be a lot faster. Set Show All Files to false by default at Tools -> Options -> Windows Live Agents SDK -> SDK Settings -> General -> "Show All Files on project open."
Keyboard Shorcuts
As of right now, there isn't a keyboard shortcut for starting/stopping the agent in the Conversation Window. However, here are some shorcuts you may find useful:
CTRL-K CTRL-C ... comment
CTRL-K CTRL-U ... uncomment
CTRL-ALT-L ... Solution Explorer
CTRL-TAB ... toggle through open windows
CTRL-Shift-<arrow key> ... move forward/backward by one word
Many other shortcuts are listed next to the menu items they correspond to.
Window Arrangement
One of the best things about Visual Studio is the flexibility it provides for laying out your environment. Any window can be tabbed, docked, or floating, and you can drag everything around the way you like. Right click a window title, or click the small arrow in the top of the pane, to see your options
Now let’s take a look at the datasource itself. The datasource essentially is a function itself, with arguments to pass, and variables to return. In the preprocess section, the POST_DATA variable where the XML SOAP request string was built is put into here. In addition, the actual web service URL is stated here as well.
In the preprocess section, there are also two built-in variables that can be used, LIMIT and OFFSET. These two variables are used to ‘page’ results in a cursor. In the example above, we look at LIMIT to populate a variable called MAXRESULTS. The MAXRESULTS variable is then used in the COUNT element (in this case 10) to bring back 10 results per request. If the user needs more, then the datasource then starts at the next row and retrieves 10 more results.
The simple xml section is a hierarchical representation of the XML response from the SOAP API, to be flattened out into a 2-dimension look when the data is retrieved. Indentation is used to signify a parent-child relationship. The {loop=content} statement acts as a loop within the XML, iterating through the XML. The end nodes (highlighted in BOLD) are the fields that is used to capture information and passed back to the calling routine. Note that fields can be skipped in the simple xml section if the user does not need it.
It should be noted here that by using simple xml to represent the XML response, there is no provision for providing a “dynamic” representation of the XML using simple xml. So in essence, you would have to potentially write a different datasource function for each different search type in this case. For generate an advanced datasource that could output differently depending on the search type would require outputting a datasource in Buddyscript. We’ll cover this in a different blog.
datasource LiveSearchAPI(SEARCH, CULTURE_INFO) => Title, Description, Url, Source, NewsYear, NewsMonth, NewsDay, NewsHour, NewsMinute, NewsSecond {expire="in 1 hour" continue_on_error="true" timeout="15" }
preprocess
if LIMIT>10 || LIMIT<=0
MAXRESULTS = 10
else
MAXRESULTS = LIMIT
FIELDLIST = "Title Description Url Source DateTime"
POST_DATA = BuildSearchAPIPostData(SEARCH, "News", OFFSET, MAXRESULTS, CULTURE_INFO, FIELDLIST)
http
http://soap.search.msn.com:80/webservices.asmx
header
Accept: application/soap+xml
postdata {encode=no}
POST_DATA
simple xml
Envelope
Body
SearchResponse
Response
Responses
SourceResponse
Offset => RESULTOFFSET // Where we're starting from.
Total => TOTAL // Total number of results.
Results
Result {loop=content}
Title
Description
Url
Source
DateTime
Year
Month
Day
Hour
Minute
Second
postprocess
INFO.Offset = RESULTOFFSET
INFO.MaxCount = TOTAL
return INFO
There are other datasource properties that should be considered to either increase performance and or deal with potential errors in accessing/retrieving information from the datasource. The first one is the Timeout property. You can specify this time in order to lengthen or shorten the time it takes before the datasource quits accessing the web service. The default value is 10 seconds. In our case, we have it at 15 seconds. The next property is the continue_on_error property. By changing this property to ‘yes’, execution will still continue and the datasource caller can retrieve the error message in the SYS.Data.Error variable. This is only on those sources that call the ABErrorProc. The final property is very important. It is the Expire property. This determines how long retrieved data should be valid, i.e. kept in cahsed memory. The ability to cache retrieved data in memory will improve performance on retrieving information in the datasource. You should consider these factors:
1) how often does the data change?
2) how often will the same retrieved data be asked again?
3) how large is the retrieved data set?
4) server memory cache size (N/A on hosted applications)
5) how fast does the web service perform?
All of these are considerations. In our case, since news items change frequently, we’ll set it for a relatively short time period, say 1 hour.
Examples of the Expire property:
Expire=”never” /* this is the default expiration for most non-Buddyscript datasources */
Expire=”in 1 hour”
Expire =”now” /* no caching at all, same as “never” */
Expire=”tomorrow at 5am” /* Note that this time is the server time, not the client time. In hosted applications, this is in GMT time */
The postprocess section is important for returning a range of information. For datasources that do not handle the processing of data using offsets and limits (i.e. simple xml), if the postprocess section is missing, the processing QueryServer will process the data coming back from the datasource in its entirety. In cases where the output coming back is one entity or one row, or if the amount of data needed to be processed is small, the postprocess section is not needed.
postprocess
INFO.Offset = RESULTOFFSET
INFO.MaxCount = TOTAL
return INFO
Looking at the postprocess section, this section is used to set the offset and total count of rows in a variable. This variable is then used by Buddyscript to control the display of output.
In this case, INFO is the name of an object variable. The names of the variables inside the object is Offset and MaxCount, and these values are populated from the datasource:
Offset => RESULTOFFSET // Where we're starting from.
Total => TOTAL // Total number of results.
Finally, here is a crude routine to pass a request to the Live Search API, access the web service and display the contents of the data, using Buddyscript code to control the amount of data coming in.
? Tell me some news about STRING=Anything
LOCALE="en-us"
TITLE, DESCRIPTION, LINK, SOURCE, YEAR, MONTH, DAY, HOUR, MINUTE, SECOND = LiveSearchAPI(STRING, LOCALE) show 10
* Here are the results:
- TITLE, SOURCE
* <blank/>
<ifmore>Type "more" for more news.</ifmore>
else
- Sorry, no news sites were found for your input.
In the input, you could ask a question such as “Tell me some news about Baron Davis” for example, and get back results that looks like this (notice that the output only contains 2 out of the 10 arguments returned, Title and Source):
Here are the results:
Baron Davis Going South, San Francisco Gate
The Baron Davis, Gilbert Arenas Switch-a-roo?, San Francisco Gate
Clippers set sights on Baron Davis, Los Angeles Times
Baron Davis on verge of signing with Clippers, Washington Post
NBA: Warriors trying to woo Brand, Newsday
Baron Davis becomes free agent, Chicago Sun-Times
Report: Davis to ditch Warriors for Clippers, FOXSports.com
Logo? Colors? History? Don't mean a thing if you ain't got that team, CBS Sportsline
Davis on verge of joining Clippers, CNN Sports Illustrated
Davis on verge of signing with Clippers, Salon
Type “more” for more news.
.
.
.
.
Notice that in the pattern routine, there is an option called SHOW 10. This means to output 10 rows at a time. Buddyscript will go to the output datasource to retrieve the information, which in this case happens to be exactly 10 rows, since the request was to buffer 10 rows per datasource request. If the user were to type in “more”, another 10 rows will be retrieved from the datasource and 10 more rows will be displayed, and so on. (Note that there is a Buddyscript variable named SYS.Presentation.Maxlength that also controls the number of characters that can be displayed on an IM client. Depending on what this is set to, this number will also control the number of rows displayed back.)
With the SHOW command, this allows the user a quick and equivalent way of emulating a forward read-only cursor, i.e. displaying x number of rows of output at one time. The other option would be to put the data into an object and loop through the object, displaying each row, which involved more coding. It’s very possible that for control purposes, the latter method is the right way to go, but for quick coding and display, SHOW is very powerful.
Hopefully you have gotten a chance to absorb the intricacies of using datasources by accessing a really powerful web service. Thanks for your attention!
An advanced look at Web Services and Datasources
Back in February, we showed a basic example of accessing a web service and using a datasource to consume information from a web service. (Refer to: http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!443.entry) Today, we’ll take a further look at a real-world web service, and some further things you can do with a datasource to make your application perform better, and to potentially reduce coding effort.
Many agents rely on rich content in order to create a solid user experience. Many times, this involves accessing an external data store, such as a web service. With some web services, you can parameterize your request in the URL itself, for example http:/URL?<input>. More often, web services involve using request and response model via a SOAP API.
The Windows Live Search API is a very powerful web service that does many things. It allows a user to do searches on the web, news, images, as well as get dictionary, phonebook and spelling information. It provides a web service XML interface using a SOAP API, enabling you to submit requests and get back a response via XML. For more information on the XML, refer to this link: http://msdn.microsoft.com/en-us/library/bb251794.aspx
Using any number of XML tools, we can take a look at the WSDL of the Live Search API web service to see how a typical request and response is crafted. The WSDL endpoint is this:
http://soap.search.msn.com/webservices.asmx?wsdl
For purposes of discussion, let’s say that we are going use a Web News search, searching for “Baron Davis”. The query of “Baron Davis” is put into the Query element. There are numerous other elements that are part of the response. They are highlighted in red. The XML response would look like this:
<?xml version="1.0" encoding="utf-16"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<Search xmlns="http://schemas.microsoft.com/MSNSearch/2005/09/fex">
<Request>
<AppID>your APP ID</AppID>
<Query>Baron Davis</Query>
<CultureInfo>en-us</CultureInfo>
<SafeSearch>Moderate</SafeSearch>
<Flags>None</Flags>
<Location>
<Latitude>0</Latitude>
<Longitude>0</Longitude>
</Location>
<Requests>
<SourceRequest>
<Source>News</Source>
<Offset>0</Offset>
<Count>10</Count>
<FileType />
<ResultFields>All DateTime</ResultFields>
<SearchTagFilters>
<string />
</SearchTagFilters>
</SourceRequest>
</Requests>
</Request>
(Note that strangely enough, in the ResultFields element, putting in the request “All” usually returns back all fields In the case of a News request, this does not return back all fields. “All DateTime” will return all fields, plus the datetime in this case.)
The XML response looks like this, with output fields in bold:
<?xml version="1.0" encoding="utf-16"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<SearchResponse xmlns="http://schemas.microsoft.com/MSNSearch/2005/09/fex">
<Response>
<Responses>
<SourceResponse>
<Source>News</Source>
<Offset>0</Offset>
<Total>31185</Total>
<Results>
<Result>
<Title>NBA free agent roundup: Baron Davis to L.A.?</Title>
<Description>Baron Davis has a verbal agreement with the Clippers, Gilbert Arenas likely is staying in Washington, and Elton Brand has a decision to make as NBA free agency opened with some surprises yesterday. The dominoes began to tumble when Davis surprised ... </Description>
<Url>http://www.newsday.com/sports/basketball/knicks/ny-spnba025749266jul02,0,2266274.story </Url>
<DisplayUrl>http://www.newsday.com/sports/basketball/knicks/ny-spnba025749266jul02,0,2266274.story</DisplayUrl>
<Source>Newsday</Source>
<DateTime>
<Year>2008</Year>
<Month>7</Month>
<Day>2</Day>
<Hour>15</Hour>
<Minute>26</Minute>
<Second>29</Second>
</DateTime>
</Result>
<Result>
<Title>Later Daze, Baron: Davis leaves Warriors for Clippers</Title>
<Description>del.icio.us Less than 24 hours after Davis shocked the Warriors by walking away from the final year and $17.8 million left on his contract, the franchise guard delivered a second strike Tuesday afternoon. Davis agreed in principle with the Clippers ... </Description>
<Url>http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2008/07/02/SPCF11ICKB.DTL</Url>
<DisplayUrl>http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2008/07/02/SPCF11ICKB.DTL</DisplayUrl>
<Source>San Francisco Gate</Source>
<DateTime>
<Year>2008</Year>
<Month>7</Month>
<Day>2</Day>
<Hour>15</Hour>
<Minute>19</Minute>
<Second>20</Second>
</DateTime>
</Result>
</Results>
</SourceResponse>
</Responses>
</Response>
</SearchResponse>
</soapenv:Body>
</soapenv:Envelope>
For purposes of display, only 2 items are displayed, but in the real-life example, there are 31,185 articles on “Baron Davis” that were found. The 31,185 is indicated in the <TOTAL> element in the response. Note that not all 31,185 rows are retrieved. The amount retrieved is regulated in the Preprocess block, in the built-in LIMIT and OFFSET variables. More on this later, when we describe the datasource.
Let’s take a look at how we generate the Buddyscript code to access this web service.
A typical datasource accessing XML might have header information that looks like this:
datasource dsSample(ARG1, ARG2, ARG3) => A1, B1, C1
http
http://someurl/sample.xml
simple xml
.
.
.
However, for a web service invoking SOAP messages, we have to build a request string, so we use the PreProcess section of a datasource to set a variable that will contain the request, along with other variables. Thus, the beginning of the datasource would look like this instead:
datasource LiveSearchAPI(SEARCH, CULTURE_INFO) => Title, Description, Url, Source, NewsYear, NewsMonth, NewsDay, NewsHour, NewsMinute, NewsSecond {expire="in 1 hour" continue_on_error="true"}
preprocess
if LIMIT>10 || LIMIT<=0
MAXRESULTS = 10
else
MAXRESULTS = LIMIT
FIELDLIST = "Title Description Url Source DateTime"
POST_DATA = BuildSearchAPIPostData(SEARCH, "News", OFFSET, MAXRESULTS, CULTURE_INFO, FIELDLIST)
http
http://soap.search.msn.com:80/webservices.asmx
header
Accept: application/soap+xml
postdata {encode=no}
POST_DATA
simple xml
.
.
.
We’ll describe the syntax of the datasource in more details later on, but for now, what’s important to point out is that a function is called to build the request string. The function in this case is called BuildSearchAPIPostData and has a number of arguments. Let’s take a look at the function:
function BuildSearchAPIPostData(SEARCH, TYPE, OFFSET, COUNT,CULTURE_INFO,
FIELDLIST, RADIUS, LATITUDE, LONGITUDE)
if (LATITUDE eq "")
LATITUDE = 0
if (LONGITUDE eq "")
LONGITUDE = 0
if (RADIUS eq "")
RADIUS = 5
POST_DATA = '<?xml version="1.0" encoding="UTF-8" standalone="no"?>\n'
POST_DATA = StringConcat(POST_DATA, '<SOAP-ENV:Envelope xmlns:SOAP-ENV="http:/\/schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:tns="http://schemas.microsoft.com/MSNSearch/2005/09/fex" > <SOAP-ENV:Body>')
POST_DATA = StringConcat(POST_DATA, ' <tns:Search xmlns:tns="http:/\/schemas.microsoft.com/MSNSearch/2005/09/fex"> <tns:Request>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:AppID>', GetLiveSearchAPIAppID(), '</tns:AppID>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:Query>', SEARCH , '</tns:Query>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:CultureInfo>', CULTURE_INFO, '</tns:CultureInfo> <tns:SafeSearch>Moderate</tns:SafeSearch> <tns:Flags>None</tns:Flags>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:Location>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:Latitude>', LATITUDE, '</tns:Latitude>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:Longitude>', LONGITUDE, '</tns:Longitude>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:Radius>', RADIUS, '</tns:Radius>\n')
POST_DATA = StringConcat(POST_DATA, ' </tns:Location>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:Requests> <tns:SourceRequest>\n')
POST_DATA = StringConcat(POST_DATA, ' <tns:Source>', TYPE, '</tns:Source>\n') // Web / Ads / InlineAnswers / PhoneBook / WordBreaker / Spelling
POST_DATA = StringConcat(POST_DATA, ' <tns:Offset>', OFFSET, '</tns:Offset> <tns:Count>', COUNT, '</tns:Count> <tns:ResultFields>', FIELDLIST, '</tns:ResultFields>\n')
POST_DATA = StringConcat(POST_DATA, ' </tns:SourceRequest> </tns:Requests>\n')
POST_DATA = StringConcat(POST_DATA, ' </tns:Request> </tns:Search>\n')
POST_DATA = StringConcat(POST_DATA, '</SOAP-ENV:Body> </SOAP-ENV:Envelope>\n')
return POST_DATA
The function essentially uses the variable POST_DATA to build the request string. If there are certain parameters without information, it defaults the information (e.g. LATITUDE). The function also calls another function to get the Windows Live APP ID. This APP ID is used in the Live Search API to determine individual access rights. (See the Windows Live API for more information on obtaining this APP ID.) The function itself simply looks like this:
function GetLiveSearchAPIAppID()
return "ABCDE12345"
where “ABCDE12345” is the App ID.
We'll take a look at the rest of the datasource and invoke code later this week.
As compared to Web sites and traditional software applications, conversational agents are subject to some unique policy compliance risks. These risks arise because:
· End users’ interactions with agents are freeform and unpredictable.
· Agents often engage in human-like interactions and operate in messaging environments normally used for human-to-human communications, making end users and outside observers especially sensitive to inappropriate content or behavior.
Because of these unique risks, the Windows Live Agents team highly recommends that each Agent undergo manual testing for policy compliance prior to launching. Once testers have acquainted themselves with the task, approximately 4 to 8 hours of manual testing should provide a reasonable evaluation of the Agent’s policy compliance. Testers should:
· Be native speakers
· Have a good understanding of cultural and political factors that might determine whether an Agent’s content/behavior is appropriate
· Be able to make judgments in the best interest of your public image and business interests in the market where the Agent will be released
· Be willing to provoke the Agent to behave inappropriately (this requires creativity, persistence, and willingness/ability to imagine offensive and provocative user inputs)
· Understand the Agent’s feature set
· Ideally not have been directly involved in the Agent’s development
If your testing uncovers any issues that you need help triaging or fixing, please contact Windows Live Agents Partner Support (agentsu@microsoft.com). Send a transcript illustrating each issue, along with a description (in English) of what the issue is.
Overview for Testers
This document is intended to provide guidelines and advice for manual testing of Agents for policy compliance. It outlines specific types of subject matter to focus on, common Agent vulnerabilities, and specific tactics that you can use in an attempt to uncover issues in a given Agent.
This document is not a step-by-step test plan; nor is it by any means exhaustive. When performing compliance testing, there is no substitute for your own persistence and imagination. Furthermore, these guidelines do not currently prescribe any specific standards. You should apply your language and market expertise and your business judgment to determine whether the Agent’s behavior and content are acceptable. We strongly advise erring on the side of caution.
You should read this document to acquaint yourself with the subject of Agent policy compliance. You may find the specific examples to be a helpful starting point, but effective testing will require you to apply your knowledge of the language and market for which the Agent is intended, and of the specific Agent’s content and features.
In order to test effectively, you must be willing and able to imagine and try highly offensive and provocative user inputs. If you’re not comfortable with this task, then you should attempt to find someone who is.
Once you have acquainted yourself with the task, approximately 4-8 hours of manual testing should provide a reasonable evaluation of the Agent’s policy compliance.
Sensitive/Inappropriate subject matter to test
· Profanity
· Hate/Intolerance (with respect to race, gender, sexual orientation, religion, etc.)
· Violence and criminal behavior
· Drug use
· Sexual content
· Suicide
· Culturally/Politically sensitive subjects in your market
Scenarios to test
· Imagine you are one of the Agent’s target users
· Imagine you are a child
· Imagine you are a malicious user attempting to provoke inappropriate Agent behavior
Common Agent vulnerabilities
· Agent may repeat (or “mirror”) user language without employing adequate safeguards
· Agent may respond to the form of a user input without understanding the content
· Agent may fail to recognize inappropriate or sensitive subject matter if the user employs creative/subtle phrasing
· Agent may incorrectly determine an input to be inappropriate and in turn respond inappropriately
· Agent may have “unsafe” catch-all responses (responses used when the user input is not understood at all)
Some specific tactics to try
· See how the Agent responds to blatant abuse and provocation
· Try to trick the agent into repeating an inappropriate word or phrase
· Try to elicit an inappropriate opinion from the Agent
· Try to elicit the Agent’s approval (explicit or implicit) of an inappropriate statement
· Try to elicit inappropriate answers to formulaic questions (yes/no, how many, etc.)
· Try to elicit inappropriate responses to commands/requests/statements
· Try to trick the Agent into inferring inappropriate intent where there is none (and responding inappropriately)
Hi again! Here is the second part of our visit to the magical world of contexts. Yesterday we brushed on a few simple uses for them, now let's dive into a slightly more sensitive subject.
Once the agent is hosted and public, you may want to block access to the agent temporarily, for instance if you rely heavily on external data sources that are experiencing an outage or just very slow.
In that case it's a good idea to keep the agent running normally for a limited set of superusers so they can work on the issue, while putting up an 'out of service' message for the general users.
//The message is by default empty. If it's not then the agent knows that we want it muted.
variable PUBLIC.OUTAGE_MESSAGE = ""
context AgentIsDisabled {out-of-context="0" in-context="1000" condition="!IsSuperUser() && PUBLIC.OUTAGE_MESSAGE ne \"\""}
start context AgentIsDisabled
+ =AnythingStrong
- PUBLIC.OUTAGE_MESSAGE
end context AgentIsDisabled
// SuperUsers only an change the agent status. Easy! We've already prepared for that.
start context RestrictedStrings
+ disable agent REASON=AnythingRaw
PUBLIC.OUTAGE_MESSAGE = REASON
+ enable agent
PUBLIC.OUTAGE_MESSAGE = ""
end context RestrictedStrings
Here the condition is simply to check if the outage message is empty or not, except for super users. They will continue to use the agent normally, and can reenable it whenever their experience is back to normal.
Now, there is one problem in this code... did you spot it?
Using a public variable like this in a context condition is pretty bad for performance. Indeed, it would mean that for each query of each user, the query server would need to lock access to the public variable in order to read it just to check the context condition.
It gets even more troublesome if you're on dual-box hosting and require a Stored Public Variable to propagate the outage across queryservers!
What is the solution then?
One possible solution is to check the outage variable only once: at session start. All you need for this is to keep a local session variable of that setting.
stored variable PUBLIC.STORED_OUTAGE_MESSAGE = ""
variable G_OUTAGE_MESSAGE = ""
procedure ABStartSessionProc()
// This procedure can exist independently in any buddyscript file, and is called at session start
if !IsSuperUser()
lock profile // locking is required for stored public variables.
G_OUTAGE_MESSAGE = PUBLIC.STORED_OUTAGE_MESSAGE
context AgentIsDisabled {out-of-context="0" in-context="1000" condition="G_OUTAGE_MESSAGE ne \"\""} //note that we don't need to check for SuperUsers here as as it's done at session start
start context AgentIsDisabled
+ =AnythingStrong
- G_OUTAGE_MESSAGE
end context AgentIsDisabled
start context RestrictedStrings
+ disable agent REASON=AnythingStrong
lock profile
PUBLIC.STORED_OUTAGE_MESSAGE = REASON
+ enable agent
lock profile
PUBLIC.STORED_OUTAGE_MESSAGE = ""
end context RestrictedStrings
As you may have noticed, there is a caveat to this solution: since we only check the message at session start, the flag takes some time to propagate as only new users will be getting the message.
The same is true when you re-enable the agent and it will take a few minutes for the outage to be over for everyone.
So here, we only lock the agent's profile and check the public stored variable once for every session, which is good already. But can we do better?
Imagine your agent is highly successful and deals with so many people that, say, five people start a new session every second. Don't we all dream of having an agent that popular! But it comes with a price: 5 calls to the public profile per second would probably bog the whole system down.
What do you do then? Well, add yet another layer of buffering of course!
Back to our friend the "basic" public variable. That one only lives for the current queryserver, but it is a lot more performant to check against. All we'll need is a background procedure to update the public variable every few minutes, so as to transmit the orders from the top to the base:
stored variable PUBLIC.STORED_OUTAGE_MESSAGE_FOR_ALL_USERS = ""
variable PUBLIC.OUTAGE_MESSAGE_FOR_ALL_USERS_OF_THIS_SERVER = ""
variable G_OUTAGE_MESSAGE_FOR_THIS_USER = ""
procedure Background_UpdateOutageMessage() startup every 1 minute
// This procedure is called at startup and every minute, independently of any user session.
lock profile
PUBLIC.OUTAGE_MESSAGE_FOR_ALL_USERS_OF_THIS_SERVER = PUBLIC.STORED_OUTAGE_MESSAGE_FOR_ALL_USERS
procedure ABStartSessionProc()
if !IsSuperUser()
G_OUTAGE_MESSAGE_FOR_THIS_USER = PUBLIC.OUTAGE_MESSAGE_FOR_ALL_USERS_OF_THIS_SERVER
context AgentIsDisabled {out-of-context="0" in-context="1000" condition="G_OUTAGE_MESSAGE_FOR_THIS_USER ne \"\""}
start context AgentIsDisabled
+ =AnythingStrong
- G_OUTAGE_MESSAGE_FOR_THIS_USER
end context AgentIsDisabled
start context RestrictedStrings
+ disable agent REASON=AnythingStrong
PUBLIC.STORED_OUTAGE_MESSAGE_FOR_ALL_USERS = REASON
+ enable agent
PUBLIC.STORED_OUTAGE_MESSAGE_FOR_ALL_USERS = ""
end context RestrictedStrings
Here you have a piece of code, slighly more complex, but that will sustain any kind of traffic without budging, with the only downside of taking up to one more minute to propagate the change of the agent's status.
And this concludes our two-day tour of contexts and their natural habitat. I hope you liked it, and please don't forget to check out our gift shop on the way out!
