Classical dictionary learning algorithms that rely on a single source of information have been successfully used for classification tasks. Additionally, the exploitation of multiple sources has shown to be advantageous in challenging real world situations. We propose a new framework to exploit robust modality fusion in classification in order to achieve better classification performance than single source methods. Multimodal learning is able to leverage any correlations between sensor modalities found in the data. We propose a new bilevel optimization, referred to as (MCJWDL). We perform supervised dictionary learning while forcing a coupling between the resulting sparse codes from different sources of information. Extensive experiments demonstrate that MCJWDL outperforms state-of-the-art sparse representation and dictionary learning approaches for the multi-view object and multi-view action recognition.