At long last I have finished a post on the new DetectAnsweringMachine activity in the latest Beta.  While writing this post, there were a number of things that were confusing to me so I hope this information helps. 

 

I write a post some time ago about detecting answering machines.  Basically, my approach was to use a concept grammar to determine whether a response is an answering machine or a live person.  The DetectAnsweringMachine activity that ships with Office Communications Server Speech Server works in a somewhat similar way except it also takes into account the time the utterance took in order to determine whether the response is human or answering machine.

 

The way the algorithm works is the following.

1)      Using a grammar that you supply, DetectAnsweringMachine gets a response and in particular pays attention to the time length of the response.  So, typically a ‘hello’ response would take less than a second.

2)      By default, if the length of the utterance is less than one second, it assumes the response was human (this is of course configurable).

3)      By default, if the length of the utterance is greater than four seconds, it assumes the response was an answering machine (also configurable).

4)      If the length of the utterance was between one and four seconds, it uses the grammar’s classification.

 

This means that if your grammar return “answering machine” but the response took less than one second, it will still classify it as a human regardless of what the grammar returned.  In reality, we have found that using this algorithm alone for answering machine detection is more reliable than using the grammar alone.  However, using a good grammar should see an improvement of a few percentage points which in many systems is considerable.

 

Where the confusing part is, is creating the grammar.  Note that if you do not add a Grammar to the activity (using the TurnStarting event) the runtime will throw an exception.  Between going through the research of some others on my team and doing my own research, I created a HMIHY grammar that you can use for the classification.  HMIHY grammars still work best here because you are unlikely to guess the actual text of the response – you really need to generate a background model to recognize the utterance which is what HMIHY grammars do nicely.

 

To create a HMIHY grammar, first open a .gbuilder file in the conversational grammar builder and right click on the answers node and select ‘Add Concept Answer’.  You will need to this new answer to ‘DetectedEntity’.  This is very important and actually I did not find it mentioned anywhere in the documentation I have.  If you do not name your answer this, you will receive an exception when you run your application.

 

Next you will need to add two concepts to the answer – ANSWERING_MACHINE_RESPONSE and LIVE_PERSON_RESPONSE.  The sentences you use are up to you, but to start out the following are what I used.

 

ANSWERING_MACHINE_RESPONSE

Hi this is John

you have reached

five five five one two five three

we are sorry we are not able to answer your phone

please leave your name telephone number and a brief message after the beep

get back to you as soon as possible

thank you for calling

please leave a voice message for

when you are finished recording hang up or press pound for more options

after the tone please record your message

Hi this is

you have reached

we are sorry we are not able to answer your phone  Thanks bye

please leave you name telephone number and a brief message after the beep

get back to you as soon as possible

thank you for calling

please leave a voice message for

when you are finished recording hang up or press pound for more options

after the tone please record your message

leave your name and number I'll get back to you when I can

I am not here right now please leave your message

are not able to come to the phone  be happy to call you back as soon as we can

to leave a fax please start transmission To leave a voice message please speak after the beep

Hi you've reached John if you want to send a fax press the start button on your fax machine after you finish your message

good evening I'm not here now but if you leave a message I'll get you back as soon s possible

Sorry that we can not come to the phone right now record your message after the tone  to send numerical page press nine

When you are finished recording hang up or for delivery options press nine Thank you

Hi you've reached five five five one two five three Please leave a message  we'll call you as soon as we can  Thanks for calling we'll return your call

please leave your name and phone number and we'll call you back Thank you bye

We are unavailable to take your call we'll call you back  Have a nice day

we are not home right now Have a great day

no one can come to the phone at this moment leave a message and we'll call you back

leave a message after the beep

After the tone record your message

it's not available you may leave a message after the tone  your call is important to us sorry that I missed your phone call

Please leave your message after the tone

please leave us a message

You reached the voice mailbox of Sorry that I missed your call  I can't speak with you right now or not in my office  I'll get back to you when I return

sorry that I missed your call  I'm either on the phone or away from my desk or on the other line

you have reached a voice message system  after the tone please record your message

your call has been forwarded to an automatic voice mail system

Please leave a message for

Please leave a voice mail for

Hello we are not at home right now please leave a message after the beep

Hello no one is available to take your call please leave a message after the tone

Hello I'm unavailable to answer your call right now please leave your name number and a message after the tone

Hello we are not available now  Please leave your name and phone number after the beep  We'll return your call

You have reached a number that does not accept solicitations if you are a solicitor please add this number to your do not call list and hang up now  Otherwise please press five or stay on the line

number which does not accept calls from telemarketers  All other callers may press five if they wish to complete the call

the number that does not accept calls from telemarketers if you are a telemarketer please add this number to your do not call list and hang up now If you are not a telemarketer press five

We do not accept telemarketing calls Please remove us from the calling list that is required by federal laws

notice all phone sollicitators place this name and phone number on your do not call list  personal or invited business callers press five on your telephone

I'm sorry your call can not be completed at this dial  Please check the number and dial again or call your operator to help  This is a recording

The number five five five one two five three has been disconnected  No further information is available about this number

This number is not in service

You have reached an invalid D. I. D. number  Please try your call again

The number you have reached is not in service If you feel you have reached this recorded message in errors  Please check the number and try your call again

We are sorry  Your call can not be completed at this dial  Please check the number and dial again  Or call your operator to help  This is a recording

The number you have reached five five five one two one two is not in service If you feel you have reached this recorded message in errors Please check the number and try your call again Thank you

Please note the area code and dial your call again

The number five five five one two one two has been disconnected No further information is available about this number

Sorry the number you have dialed in no longer in service

the number you have dialed in not in service at this time  Please check your number or call your operator for assistance

We are sorry You have reached a number that has been disconnected or no longer in service

 

LIVE_PERSON_RESPONSE

This is John speaking

Hello

I'm fine

Okay

I am fine

I am OK

good afternoon

good morning

good evening

Who is it  may I help you

Hello May I ask who is calling

Hey what's up man

Hey what's up man

Hello how are you

Hello you

hello this is Mary speaking

Hello there

Hello there mate

Yes

Yes er

er hello

Yo

Speaking

 

Note that conversational grammar builder supports copying multiple phrases – so that is the easiest way to add these sentences to your .gbuilder file.

 

If you look through these sentences you will probably notice that several of the answering machine sentences actually refer to privacy systems and ‘no longer in service’ messages.  Ideally we should be able to detect whether we have received a person, answering machine, privacy system, or whether the number is no longer in service but the reality is when you use the DetectAnsweringMachine activity you can only return the two semantic results above – any other result will cause an exception.  So in this case we just classify them as answering machines.

 

If your situation requires that you determine whether you have encountered a privacy system or a number no longer in service, it would not be difficult to use the same algorithm DetectAnsweringMachine uses in a custom activity that understands other semantic results.  If you have difficulties with this, you may be able to butter me up with a few nice comments on my blog but keep in mind how long it took me to write this post in the first place. J

 

To determine what the result of the activity was, you need to check the DetectionResult property.  Of particular interest is there is a DetectionResult enumeration value of OtherAutomata.  You can safely ignore this value as it is never used.  You will only come across values of None (if the activity has not yet run), AnsweringMachine, or LivePerson.