360 Degrees Rumor Detection: when Explanations Got Some Explaining to Do

•This study introduces a framework for rumor detection using textual and structured input features.•Besides accuracy, we evaluate explanation quality using fidelity and stability.•Traditional machine learning models and transformers are the top performers.•There is a trade-off between model complexity and explanation quality.•Model-agnostic explainable methods should be used with caution.