Cross-stream observation dependencies for multi-stream speech recognition

This paper extends prior work in multi-stream modeling by in- troducing cross-stream observation dependencies and a new dis- criminative criterion for selecting such dependencies. Exper- imental results combining short-term PLP features with long- term TRAP features show gains associated with a multi-stream model with partial state asynchrony over a baseline HMM. Frame-based analyses show significant discriminant informa- tion in the added cross-stream dependencies, but so far there are only small gains in recognition accuracy.