MMI

[nabihach]

Hi.
I'm parsing through your code and I'm trying to figure out where you are using MMI (maximum mutual information) criterion. Could you please point that out? Thanks

[BranaHi]

@michaelfarrell76
I too have searched for your code using MMI. Could it be that it is not included here?

Edit:
After reading the paper again i understood that you dont use MMI as a training objective but as a way to rerank the awnsers the model produces.

[michaelfarrell76]

On line 140 of src/beam.lua we basically just would take the difference between our distribution of possible words created by the context and the distribution created by a babbler. Not too sophisticated, just a basic metric to see which probabilities were altered by the context.