Controlling Drone with iOS Speech Recognition

Question

I got a Parrot Mambo which comes with an SDK for creating your own app for controlling it.

I decided to use the iOS Speech Recognition library so I could use basic voice commands to control it. The majority of the iOS SDK that comes with the drone is written in pure C, so it has to be used in Objective-C (Something I have little previous experience in using). Any advice on how to clean this up, or if I'm using best practices are appreciated.

...

@property (nonatomic, strong) IBOutlet UIButton *startListeningButton;
@property (nonatomic, strong) IBOutlet UIButton *stopListeningButton;
@property (nonatomic, strong) IBOutlet UITextView *transcribedTextView;

@property (nonatomic, strong) SFSpeechRecognizer *speechRecogniser;
@property (nonatomic, strong) SFSpeechAudioBufferRecognitionRequest *speechRecognitionRequest;
@property (nonatomic, strong) SFSpeechRecognitionTask *speechRecognitionTask;
@property (nonatomic, strong) AVAudioEngine *audioEngine;

...

- (IBAction)startListening:(id)sender {
    [_startListeningButton setEnabled:YES];
    [_stopListeningButton setEnabled:NO];
    [self startSession];
}

- (IBAction)stopListening:(id)sender {
    if ([_audioEngine isRunning]) {
        [_audioEngine stop];
        [_speechRecognitionRequest endAudio];
        [_startListeningButton setEnabled:YES];
        [_stopListeningButton setEnabled:NO];
    }
}

- (void) startSession
{
    if (_speechRecognitionTask != nil) {
        [_speechRecognitionTask cancel];
        _speechRecognitionTask = nil;
    }

    AVAudioSession *audioSession = [AVAudioSession sharedInstance];
    [audioSession setCategory:AVAudioSessionCategoryRecord error: nil];

    _speechRecognitionRequest = [[SFSpeechAudioBufferRecognitionRequest alloc] init];

    _speechRecognitionRequest.shouldReportPartialResults = YES;

    AVAudioInputNode *inputNode = [_audioEngine inputNode];

    _speechRecognitionTask = [_speechRecogniser recognitionTaskWithRequest:_speechRecognitionRequest resultHandler:^(SFSpeechRecognitionResult * _Nullable result, NSError * _Nullable error) {

        BOOL finished = NO;

        if (result) {
            self.transcribedTextView.text = result.bestTranscription.formattedString;
            finished = result.isFinal;

            NSString *flyCommand = @"fly";
            NSString *landCommand = @"land";

            NSString *formattedString = [[result bestTranscription] formattedString];

            if ([formattedString rangeOfString:flyCommand options:NSCaseInsensitiveSearch].location != NSNotFound) {
                [_miniDrone takeOff];
            }

            if ([formattedString rangeOfString:landCommand options:NSCaseInsensitiveSearch].location != NSNotFound) {
                [_miniDrone land];
            }
        }

        if (error != nil || finished) {
            [[self audioEngine] stop];
            [inputNode removeTapOnBus:0];

            self.speechRecognitionRequest = nil;
            self.speechRecognitionTask = nil;

            _transcribedTextView.text = @"";

            [[self startListeningButton] setEnabled:YES];
        }
    }];

    AVAudioFormat *recordingFormat = [inputNode outputFormatForBus:0];

    [inputNode installTapOnBus:0 bufferSize:1024 format:recordingFormat block:^(AVAudioPCMBuffer * _Nonnull buffer, AVAudioTime * _Nonnull when) {

        [[self speechRecognitionRequest] appendAudioPCMBuffer:buffer];

    }];

    [_audioEngine prepare];
    [_audioEngine startAndReturnError:nil];
}

- (void)authoriseSR
{
    [SFSpeechRecognizer requestAuthorization:^(SFSpeechRecognizerAuthorizationStatus status) {
        [[NSOperationQueue mainQueue] addOperationWithBlock:^{
            switch (status) {
                case SFSpeechRecognizerAuthorizationStatusAuthorized:
                    [[self startListeningButton] setEnabled:YES];
                    break;
                case SFSpeechRecognizerAuthorizationStatusDenied:
                    [[self startListeningButton] setEnabled:NO];
                    [[self startListeningButton] setTitle:@"Speech recognition access denied by user" forState:UIControlStateDisabled];
                    break;
                case SFSpeechRecognizerAuthorizationStatusRestricted:
                    [[self startListeningButton] setEnabled:NO];
                    [[self startListeningButton] setTitle:@"Speech restricted" forState:UIControlStateDisabled];
                    break;
                case SFSpeechRecognizerAuthorizationStatusNotDetermined:
                    [[self startListeningButton] setEnabled:NO];
                    break;
            }
        }];
    }];
}

...

Right now it works and the drone responds when I speak to it, but the app itself seems really slow in registering my voice, maybe 3-5 seconds to register just the one word commands. I'm not sure if this is something in my code running slow of its the standard time for the framework to respond.

I removed other references to the SDK to avoid confusion just leaving in the basic drone commands I need, i.e. [_miniDrone takeOff]; and [_miniDrone land];

Answer 1

Generally All library of Speech recognize will take few seconds (around 1 to 5 seconds) to respond on any recognition so its normal time for your response.

But what i suggest here is default API of apple which is batter option and very powerful.

Please go through below apple documents:

Apple API reference :

https://developer.apple.com/reference/speech

WWDC 2016 Video :

https://developer.apple.com/videos/play/wwdc2016/509/

Also I notice in your code 1 or 2 things need to change in day to day practice

1st

Always use weak reference in IBOutlet property variable for memory management in ARC like below.

@property (nonatomic, weak) IBOutlet UIButton *startListeningButton;

2nd

Always practice proper commenting in your code so any one can recognize code for what purpose and its increase your code readability and code clean ness.

Thanks.