Mega Code Archive
Speech Part 2 How to Add Simple Dictation speech recognition to your Delphi Apps
<![CDATA[
Title: Speech Part 2 - How to Add Simple Dictation speech recognition to your Delphi Apps
Question: I'd like to be able to speak into my computers microphone and have what I say translated into text that is entered into my application. How Can I do this?
Answer:
New Speech Part 2 - Speech Recognition (Simple Dictation)
This article shows how you can add simple dictation speech recognition capabilities to an application. First some technical considerations will be discussed followed by the creation of a small application that will allow the user to add words to a TMemo by speaking into a microphone attached to the computers sound card.
If you did not read Speech Part 1 (article #2581) you should go read it now. It tells you how to get the Microsoft Speech API 5.1 (SAPI) and install it on your system and in Delphi.
SAPI version 5.1 supports two distinct types of speech recognition; dictation and command and control. Its important to understand the differences between the two in order to make correct decisions in the design of your speech enabled applications.
Dictation Speech Recognition
Dictation refers to a type of speech recognition where the machine listens to what you say and attempts to translate it into text. This all happens inside the speech engine and you dont need to worry about it although a little theory may be helpful. Most modern dictation engines use a scheme where they listen to whats said and break what they hear down into a series of word hypothesis. Each word hypothesis may actually contain a list of possible words with each word given some probability of correctness. So, for example, if I say The quick red fox the computer will likely break this down into 4 separate word hypothesis. The fox hypothesis may contain the possibilities of fax, box, fix, etc. These individual word hypothesis are then put in context. That is, each word is considered in relation to the words that came before and after. Based on the rules of context the speech engine comes to a final best decision about what was spoken and returns it to the application. In dictation, context is the name of the game. For this reason, dictation engines are considered to be contextual. (My apologies to any ASR scientists reading this for this minimalist explanation.)
As you may imagine, the accuracy of dictation ties directly to the CPU's speed and the system's available memory. The more resources, the more context that can be considered in a reasonable amount of time the more likely the resulting recognition will be accurate. The truth is that the basic principals on how to do speech recognition have not changed in over 20 years. What has changed is the power of the PC and its the processing power of modern PCs that makes speech recognition finally usable.
Also important to accurate dictation recognition is the engine having some understanding of the individual speakers voice. First speech engines are specific to language and possibly even region. This is why we see English engines and French engines and Chinese engines, etc. Beyond languages though, there are differences (sometimes extreme) within a language. A 5 year old girl sounds very different to the computer than a 47 year old man. This is why most current dictation engines require voice training.
If you have SAPI 5.1 installed, go to your systems Control Panel and click the Speech icon. On the speech recognition tab you will find a button called Train Profile..
Command and Control Speech Recognition
While dictation recognition is use primarily for recording what a user says and translating it into text, Command and Control (CnC) speech recognition is used for controlling applications. In the same way that you click your mouse on the browser icon on your desktop to access the internet you could speak Computer, run browser, or even better Compute, go to www.o2a.com to accomplish the same thing. Currently you are used to controlling your computer by you mouse and keyboard. CnC recognition adds a third input device, you voice.
CnC speech recognition is fundamentally different from dictation recognition in that it is recognition without regard to context. That is, there are no CPU cycles spent trying to determine if a word is correct by looking at the words that come before or after. For this reason CnC recognition is often also know as context free recognition.
Instead of using context, CnC recognition uses pre-defined grammars. These grammars contain rules, and each rule can then have a programmed response. So, in developing an application that uses CnC recognition the programmer defines both the grammar and the rules as well as the response to the recognition of each rule.
If grammars and rules are managed properly CnC recognitions can be much more accurate than dictation recognition. This is because the number of words that need to be recognized for CnC is only a subset of the universe of words needed for dictation. With CnC the engine only need to worry about the words in the active grammar not all the words in the dictionary. Fewer possibilities mean better accuracy.
It turns out that dictation recognition is much more difficult for speech engine developers than CnC recognition. But for us as the application developer it is much easier to implement simple dictation in an application than CnC because with dictation we dont need to worry about writing a grammar. For this reason Im starting with dictation instead of CnC. Ill probably do CnC and grammar development in the next article.
A Simple Dictation Application
All right then, lets build an application that takes dictation.
[ Delphi 6 users In the process of testing this sample in Delphi 6 I ran into a known problem with event sinks generated from type library imports. See Article number 2590 for more information and some work arounds. ]
Start up Delphi (5 or 6, 4 might work to but I didnt try it). On the SAPI5 palette (see Speech Part 1 if you dont have one) find the TSpSharedRecoContext component and drop it on your form along with a TMemo component.
Add the ActiveX unit to your uses clause and add a private field to the form called fMyGrammar of type IspeechRecoGrammar.
Create an onCreate event for the form, plus OnRecognition and OnHypothesis events for the SpSharedRecoContext component. You complete unit should look something like this
unit Unit1;
interface
uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, OleServer, SpeechLib_TLB, StdCtrls, ActiveX, ComCtrls;
type
TForm1 = class(TForm)
SpSharedRecoContext1: TSpSharedRecoContext;
Memo1: TMemo;
procedure FormCreate(Sender: TObject);
procedure SpSharedRecoContext1Recognition(Sender: TObject;
StreamNumber: Integer; StreamPosition: OleVariant;
RecognitionType: TOleEnum; var Result: OleVariant);
procedure SpSharedRecoContext1Hypothesis(Sender: TObject;
StreamNumber: Integer; StreamPosition: OleVariant;
var Result: OleVariant);
private
{ Private declarations }
fMyGrammar : ISpeechRecoGrammar;
public
{ Public declarations }
end;
var
Form1: TForm1;
implementation
{$R *.dfm}
procedure TForm1.FormCreate(Sender: TObject);
begin
fMyGrammar := SpSharedRecoContext1.CreateGrammar(0);
fMyGrammar.DictationSetState(SGDSActive);
end;
procedure TForm1.SpSharedRecoContext1Recognition(Sender: TObject;
StreamNumber: Integer; StreamPosition: OleVariant;
RecognitionType: TOleEnum; var Result: OleVariant);
begin
Memo1.Text := Result.PhraseInfo.GetText;
end;
procedure TForm1.SpSharedRecoContext1Hypothesis(Sender: TObject;
StreamNumber: Integer; StreamPosition: OleVariant;
var Result: OleVariant);
begin
Memo1.Text := Result.PhraseInfo.GetText;
end;
end.
Compile and run this. Speak something. Your words should appear in the memo field. If they do not shut the application down and;
1. Make sure you microphone is not muted.
2. Use the Speech control panel applet to make sure that your microphone and the recognition engine is working properly.
Now try it again.
Explanation
The SAPI 5.1 automation objects support both dictation and CnC speech recognition. Of the 19 components installed the following 4 are central to speech recognition.
The SpInprocRecognizer represents a speech recognition engine that is instantiated in the same process as the application.
The SpSharedRecognizer represents an instance of a speech recognition engine that is shared by many applications.
The SpInprocRecoContext is a recognition context that uses a SpInprocRecognizer.
The SpSharedRecoContext is recognition context that uses a SpSharedRecognizer.
Shared vs. Inprocess
An application can use either an inprocess instance of a speech engine (SpInprocRecognizer) or an instance that is shared with other applications (SpSharedRecognizer). The inprocess recognizer claims resources for the application, so, for example, once an inprocess recognizer claims the systems microphone, no other application can use it.
A shared recognizer runs in a separate process from the application and, as a result, it can be shared with other applications. This allows multiple applications to share system resources (like the microphone).
In our sample we are using a shared engine. In most desktop applications shared is the way to go. Using a shared recognizer allows your application to play nicely with other speech enabled applications on your system. If your application is targeted for some dedicated machine like one running a telephone voice response application then the inprocess approach would be appropriate. Inprocess recognition is somewhat more efficient then shared recognition.
Recognition Contexts
A recognition context is an object that manages the relationship between the recognition engine object (the recognizer) and the application. Do not confuse the use of the word context as used here with its usage in context free grammar.
A single recognizer can be used by many contexts. For example, a speech enabled application with 3 forms will likely have a single engine instance with a separate context of each form. When one form gets the focus its context becomes active and the other two forms contexts are disabled. In this way, only the commands relevant to the one form are recognized by the engine. Another example as seen in Microsoft Word XP where there is one context for dictation and another context for issuing menu commands.
The recognition context is the primary means by which an application interacts with SAPI. It is the object you use to start and stop recognition and it is the object that receives the event notifications when something is recognized. Further, the recognition context controls which words (grammars and/or dictation) are recognized. By setting recognition contexts, applications limit or expand the scope of the words needed for a particular aspect of the application. This granularity for speech recognition improves the quality of recognition by removing words not needed at that moment. Conversely, the granularity also allows words to be added to the application if needed.
In our example above we do the simple thing (at least programmatically) and just load dictation. This means that all words will attempt to be recognized. The other possibility is to load one or more specific grammars. Grammars are a big subject and will be covered in a later article.
Theres a lot more on the subjects of recognition contexts and inprocess vs. shared recognizers in the SAPI 5.1 documentation but for now thats enough to talk about the sample code.
What the sample code does
First, here is the forms OnCreate event.
procedure TForm1.FormCreate(Sender: TObject);
begin
fMyGrammar := SpSharedRecoContext1.CreateGrammar(0);
fMyGrammar.DictationSetState(SGDSActive);
end;
Just two lines of code to set the whole recognition process in motion. First we need to create a grammar (CreateGrammar) object for the engine and then we instruct this grammar that it is to attempt to recognize all words by DictationSetState(SGDSActive).
Notice that neither on the form or in the code do we ever instantiate a SpSharedRecognizer. This is because SAPI is smart enough to create the shared recognizer object for us automatically when the SpSharedRecoContext is created.
Next we need some way for the application to be informed by the engine when it recognizes something. This is done through the OnRecognition event.
Procedure TForm1.SpSharedRecoContext1Recognition(Sender: Object;
StreamNumber: Integer; StreamPosition: OleVariant;
RecognitionType: TOleEnum; var Result: OleVariant);
begin
Memo1.Text := Result.PhraseInfo.GetText;
end;
Of the various parameters passed in the OnRecogntion event, the Result parameter is the key. Although declared as an OleVariant for interprocess communications its really an object with an ISpeechRecoResult interface. This interface lets you get all sorts of information about what was said and what the recognizer understood. Some of the information available through this interface includes; the words recognized, a rating of the engines confidence in the recognition, when the recognition happened and how long it took. You can even play back the audio for what was said. Much of the information returned is only useful for context free grammars and doesnt apply to dictation.
In the sample we just call the GetText method to return the text of what the engine understood.
The OnRecogintion event only fire when the engine is satisfied that the user has uttered a complete phase and that it has made its best guess about what the user said. You could run the sample application with only this event defined and it would work.
I added the OnHypothesis event so you could get a feel for how the engine, working in dictation mode, uses all the words together (in context) to create hypothesis, make corrections, and, finally, come to a decision about what was said.
Thats enough for now
Speech recognition is a very big subject. Ive scratched the surface of dictation speech recognition but there is much more. To write a really usable dictation application the user will need ways to correct mistakes and give the speech recognition engines commands like Bold the last 3 words. While possible with SAPI this level of discussion is beyond the scope of this introduction. I urge to study the documentation that comes with the SAPI SDK.
I havent give much more than passing mention of CnC context free grammars. CnC recognition and grammars will be the next article.
OK not quite enough
I couldnt leave the sample application alone. Heres a slightly modified version that is a bit more satisfying in that it lets you keep multiple utterances.
type
TForm1 = class(TForm)
SpSharedRecoContext1: TSpSharedRecoContext;
Memo1: TMemo;
procedure ButtonSpeakClick(Sender: TObject);
procedure FormCreate(Sender: TObject);
procedure SpSharedRecoContext1Recognition(Sender: TObject;
StreamNumber: Integer; StreamPosition: OleVariant;
RecognitionType: TOleEnum; var Result: OleVariant);
procedure SpSharedRecoContext1Hypothesis(Sender: TObject;
StreamNumber: Integer; StreamPosition: OleVariant;
var Result: OleVariant);
private
{ Private declarations }
fMyGrammar : ISpeechRecoGrammar;
CurrentText : String;
public
{ Public declarations }
end;
var
Form1: TForm1;
implementation
{$R *.dfm}
procedure TForm1.FormCreate(Sender: TObject);
begin
fMyGrammar := SpSharedRecoContext1.CreateGrammar(0);
fMyGrammar.DictationSetState(SGDSActive);
end;
procedure TForm1.SpSharedRecoContext1Recognition(Sender: TObject;
StreamNumber: Integer; StreamPosition: OleVariant;
RecognitionType: TOleEnum; var Result: OleVariant);
begin
Memo1.Text := CurrentText + Result.PhraseInfo.GetText;
CurrentText := Memo1.Text;
end;
procedure TForm1.SpSharedRecoContext1Hypothesis(Sender: TObject;
StreamNumber: Integer; StreamPosition: OleVariant;
var Result: OleVariant);
begin
Memo1.Text := CurrentText + Result.PhraseInfo.GetText;
end;
end. //really
]]>