Jump to content
Sign in to follow this  
mikealpha

Voice recognition hints

Recommended Posts

I need some ideas regarding voice recognition. I want to use only the ATC part of MCE, together with PF3. No matter what I tried so far, the recognition success rate is awful. I remember I tried one year ago and then gave up for that reason. 

Probably that has a lot to do with my "German" english, but I went through the speech training several times. I even tried to train the speech engine special ATC phrases word by word. Inside the speech engine it gets recognized 100%. But in MCE I get all sorts of stuff recognized, sometimes not even close to what I intend to say. 

The success rate in another program is 100%. Not sure how they do it, maybe they just listen to some specific trigger words.

Anyway, would it be possible to let the speech engine just listen to an ATC specific subset in that case, thus ignoring the thousands of other words the speech engine might know ?

What else could I try ?

Mike

 


1. A320 home cockpit (FSLabs, Skalarki), P3Dv5  Main PC : I7-12700K, GTX3080Ti

2. FSLabs A3xx, P3Dv5. Gigabyte Aorus 17G YC, I7-10700K, RTX 3080

Share this post


Link to post
Share on other sites
1 hour ago, mikealpha said:

I need some ideas regarding voice recognition. I want to use only the ATC part of MCE, together with PF3. No matter what I tried so far, the recognition success rate is awful. I remember I tried one year ago and then gave up for that reason. 

Probably that has a lot to do with my "German" english, but I went through the speech training several times. I even tried to train the speech engine special ATC phrases word by word. Inside the speech engine it gets recognized 100%. But in MCE I get all sorts of stuff recognized, sometimes not even close to what I intend to say. 

The success rate in another program is 100%. Not sure how they do it, maybe they just listen to some specific trigger words.

Anyway, would it be possible to let the speech engine just listen to an ATC specific subset in that case, thus ignoring the thousands of other words the speech engine might know ?

What else could I try ?

Mike

 

I assume you have latest MCE build available here

You may want to read this:

https://www.avsim.com/forums/topic/508866-fo-pf3/

It could be just an issue with a particular call-sign

Which one are you using?

Share this post


Link to post
Share on other sites

Yes, have the actual version in P3Dv4.

Pilot voice and copilot is inactive in P3D. Also in MCE only the ATC part is active.

It`s not a callsign problem, these mostly get recognised. But stuff like readback of the clearance, I very often can`t make it, even though I train the specific words in the Speech engine.

Occasionally the callsign fails, because it recognises the next word after the callsign as a number and adds that to the callsign. 

E.g. Lufthansa 6-2-5 at position ...

It understands : Lufthansa 6-2-5-8 position.

Not inside the Speech recognition training, never happens there. "at" and "Eight" are clearly distinguished there. 

Mike


1. A320 home cockpit (FSLabs, Skalarki), P3Dv5  Main PC : I7-12700K, GTX3080Ti

2. FSLabs A3xx, P3Dv5. Gigabyte Aorus 17G YC, I7-10700K, RTX 3080

Share this post


Link to post
Share on other sites
4 hours ago, mikealpha said:

Yes, have the actual version in P3Dv4.

Pilot voice and copilot is inactive in P3D. Also in MCE only the ATC part is active.

It`s not a callsign problem, these mostly get recognised. But stuff like readback of the clearance, I very often can`t make it, even though I train the specific words in the Speech engine.

Occasionally the callsign fails, because it recognises the next word after the callsign as a number and adds that to the callsign. 

E.g. Lufthansa 6-2-5 at position ...

It understands : Lufthansa 6-2-5-8 position.

Not inside the Speech recognition training, never happens there. "at" and "Eight" are clearly distinguished there. 

Mike

The clearance acknowledgment is by far the hardest for speech reco to cope with.

User could acknowledge up to 4 or five items, and they can be in any order.

Try to acknowledge only the important bits,

Roger Lufthansa 6258 is cleared to destination as filed, climb XXXX, squawk XXXX

Lufthansa 6258 is cleared to Echo Delta Delta Mike, via flight plan route, climb and maintain flight level YYY, squawk YYY

There are more variations you can use.

It's when you start throwing frequencies and other bits that things are more challenging.

The key thing is not to make n obsession about that particular acknowledgment. Move on, and see how you get on with other requests and acknowledgment.

I remember VoxAtc had to devise an option to just accept anything on second try because of the difficulty to process a very long acknowledgment. Especially, when you aren't reading from a script.

 

 

Share this post


Link to post
Share on other sites

Thanks for the hints so far, will continue trying.

It is weird, sometimes the speech recognition detects or displays (visible in the green text) strange characters at the end of the last detected word. Stuff like ;ü°§ for example. Sometimes I speak a whole sentence and it detects just a long series of numbers.  Not sure, might be the inevitable air conditioning noise of the airplane, that confuses the Microphone ? I try to reduce that as low as possible. It's just I am unsure what the problem is, because it works that well in the speech recognition training.

I always make sure to speak only with the PTT button pressed (hissing noise active), as I understood only then the ATC specific grammer gets loaded.

I know you have a lot of documentation already. Just a very barebone simple script what HAS to be said to let MCE continue throughout an entire flight would maybe help a lot. Once that is accomplished, one could move to the more advanced phraseology.

Mike


1. A320 home cockpit (FSLabs, Skalarki), P3Dv5  Main PC : I7-12700K, GTX3080Ti

2. FSLabs A3xx, P3Dv5. Gigabyte Aorus 17G YC, I7-10700K, RTX 3080

Share this post


Link to post
Share on other sites

Just to add : To check my equipment, tried all the other MCE voice parts, works perfectly incl. the GSX commands.Tested also FS2Crew, I use it exclusively for the 777. Not a single recognition failure.

Just the ATC part is problematic. From what I see what all gets recognised when I try to speak ATC phrases, is there anything that could prevent MCE from loading the ATC grammar at all ? Just asking. 

Mike


1. A320 home cockpit (FSLabs, Skalarki), P3Dv5  Main PC : I7-12700K, GTX3080Ti

2. FSLabs A3xx, P3Dv5. Gigabyte Aorus 17G YC, I7-10700K, RTX 3080

Share this post


Link to post
Share on other sites
56 minutes ago, mikealpha said:

Just to add : To check my equipment, tried all the other MCE voice parts, works perfectly incl. the GSX commands.Tested also FS2Crew, I use it exclusively for the 777. Not a single recognition failure.

Just the ATC part is problematic. From what I see what all gets recognised when I try to speak ATC phrases, is there anything that could prevent MCE from loading the ATC grammar at all ? Just asking. 

Mike

MCE doesn't output anything to green bar. These would come from the other speech based product.

Spoken commands are displayed in red font (where you usually see FPS and coordinates). Green bar introduces stutters, not to mention it is shared by many apps.

There is no guarantee what you say would be displayed immediately. That's why we came up with an alternative, which nobody else uses.

Be aware, with Fs2crew another speech engine instance is loaded and very likely before MCE starts its own speech engine instance.

That would mean, when you're talking to ATC, other speech commands (from the other program) are active and taking precedence.

I suggest the following...

Do not use the other speech based program for a couple of flights.

Restore MCE to factory settings, and use all the features.

Start->All Programs->Multi Crew Experience->Tools->Restore factory settings.

You can then check that speech reco level is on a par, and actually far more flexible since you don't have to speak exact sentences to the letter.

Once you've confirmed it's not a speech reco issue within MCE, use only the features you need in MCE.

You will have to find a way to suspend the other app speech recognition while transmitting to ATC.

 

Share this post


Link to post
Share on other sites

slight misunderstanding. I tested FS2Crew standalone just to check my micro and the recognition there. I didn't use it together with MCE/PF3. 

In the meantime I also tried the VOXATC P3Dv4 7 day test version and speech recognition works perfectly there as well.

Apparently the green bar came from FSUIPC; it is turned off now.

I tried to write a small script just writing down PF3 instructions and readbacks without MCE. Then used MCE/PF3 and tried to repeat the same phrases. No chance, I can't get anything recognized.

Also resetting to factory settings did not help.

I don't have the slightest idea what I'm doing wrong. As I said, all the other MCE voice recognition parts work.

Mike


1. A320 home cockpit (FSLabs, Skalarki), P3Dv5  Main PC : I7-12700K, GTX3080Ti

2. FSLabs A3xx, P3Dv5. Gigabyte Aorus 17G YC, I7-10700K, RTX 3080

Share this post


Link to post
Share on other sites
5 minutes ago, mikealpha said:

slight misunderstanding. I tested FS2Crew standalone just to check my micro and the recognition there. I didn't use it together with MCE/PF3. 

In the meantime I also tried the VOXATC P3Dv4 7 day test version and speech recognition works perfectly there as well.

Apparently the green bar came from FSUIPC; it is turned off now.

I tried to write a small script just writing down PF3 instructions and readbacks without MCE. Then used MCE/PF3 and tried to repeat the same phrases. No chance, I can't get anything recognized.

Also resetting to factory settings did not help.

I don't have the slightest idea what I'm doing wrong. As I said, all the other MCE voice recognition parts work.

Mike

OK, I think it's a speech grammar thing.

Suggest you write down the messages you spoke. Will check the PF3 specific ATC grammar (file in \CandC\ folder) can handle them, and eventually edit it.

Normally, we do our best to write a grammar that can catch as many variations as possible. Sometimes it's not good enough and you have to keep tweaking.

Note: We can make it work 100%, with instructions such as "speak this exact sentence, including "the" or "is" in the middle of the sentence and have only a single command to be recognized.

The challenge is to allow some flexibility, within the ATC jargon constraints.

Same with co-pilot exchanges. Who would want to have a manual to consult before uttering anything. It wouldn't be fun at all. That would be robotic speech recognition. 

I assume, as soon as you hold PTT switch down (either DEL key or joystick button), you're hearing a continuous background humming noise indicating MCE has switched to ATC mode. I assume PTT is reliable as to not cause the slightest loss of switch contact, which would cause MCE to revert to FO interaction mode, disabling ATC grammar in the process before anything was recognized.

 

 

Share this post


Link to post
Share on other sites

I have had the same problems like MikeAlpha. I reported that before. Voice recognition close to perfect for FO commands (including PF3-FO interaction), voice scripts and voxkey commands. But with PF3-ATC requests the success rate dropped to maybe 50 %.

 

Now I just tested with the newest version and "enable optimizations" turned off. I proceeded from requesting IFR Clearance to Take off clearance. Tested with different callsigns and all kinds of varied phrases. Recognition not 100 % but very good now (maybe 80 %, possibly used some wrong phrases). And this also was with a probably terrible German accent :dry:.

Trickiest was the IFR clearance readback. Just reading back the Flight Level and the Squawk did the trick. Also omitting the facility name sometimes seemed to help.

 

So turn off "enable optimization" and try the following (Change the variables of course):

 

On Delivery:

- Lufthansa 134, ready to copy.

After Clearance:

- Cleared to Flight Level 350, Squawk 3765, Lufthansa 134.

After "Readback correct":

- 121.6, Lufthansa 134.

On Ground:

- Lufthansa 134, request push and start.

After clearance:

-Push and start approved, Lufthansa 134.

After Pushback:

- Lufthansa 134, ready for taxi.

After clearance:

- Taxi to Holding Point Runway 16, Lufthansa 134.

On Tower:

- Lufthansa 134 is ready for takeoff (this did not always work, sometimes MCE sent the hotkey for "Request taxi to runway", sometimes for "request different runway". You can also try "ready for departure" or "request takeoff", or prepend "at Holding Point").

 

Viel Erfolg!

 

Share this post


Link to post
Share on other sites
14 hours ago, mikealpha said:

I know you have a lot of documentation already. Just a very barebone simple script what HAS to be said to let MCE continue throughout an entire flight would maybe help a lot. Once that is accomplished, one could move to the more advanced phraseology.

I would also appreciate that.

I am aware it would contradict the MCE philosophy that you do not have to say the exact same phrase every time again (like with the other software...). But such a list of the last resorts would be very helpful for troubleshooting.

 

Share this post


Link to post
Share on other sites
17 hours ago, FS++ said:

The challenge is to allow some flexibility, within the ATC jargon constraints.
 

It's a great concept, I think it would much enhance the flexibility and immersion. I bought MCE just for the MCE/PF3 combination, so I'd really love to get it to work.

 

 

13 hours ago, RALF9636 said:

Also omitting the facility name sometimes seemed to help.

So turn off "enable optimization" and try the following (Change the variables of course):

 

thanks Ralf, that's much appreciated, will try later this evening.

I wasn't aware the facility name can be omitted. Nice touch though, to say stuff like "Frankfurt ......., guten Morgen" :-)

 

Mike


1. A320 home cockpit (FSLabs, Skalarki), P3Dv5  Main PC : I7-12700K, GTX3080Ti

2. FSLabs A3xx, P3Dv5. Gigabyte Aorus 17G YC, I7-10700K, RTX 3080

Share this post


Link to post
Share on other sites

Just wondering, could that be part of the problem, the strange characters after "copy" ? Nothing happens after that.

http://www.dropbox.com/s/293w7xku95p6jfw/speech.png

Mike


1. A320 home cockpit (FSLabs, Skalarki), P3Dv5  Main PC : I7-12700K, GTX3080Ti

2. FSLabs A3xx, P3Dv5. Gigabyte Aorus 17G YC, I7-10700K, RTX 3080

Share this post


Link to post
Share on other sites
23 minutes ago, mikealpha said:

Just wondering, could that be part of the problem, the strange characters after "copy" ? Nothing happens after that.

http://www.dropbox.com/s/293w7xku95p6jfw/speech.png

Mike

Thanks.

That's certainly something we need to look at. But that only affects the "display of spoken commands", which is an option. You could disable that.

With PF3, At some point after speech is recognized, MCE would send the relevant hotkey combination to the sim.

Make sure you don't have any assignments in FSUIPC that would clash with PF3 hotkeys.

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Tom Allensworth,
    Founder of AVSIM Online


  • Flight Simulation's Premier Resource!

    AVSIM is a free service to the flight simulation community. AVSIM is staffed completely by volunteers and all funds donated to AVSIM go directly back to supporting the community. Your donation here helps to pay our bandwidth costs, emergency funding, and other general costs that crop up from time to time. Thank you for your support!

    Click here for more information and to see all donations year to date.
×
×
  • Create New...