This dataset describes RoadSense, a crowdsourcing-enabled data collection framework and accompanying prototype dataset designed to detect and map road surface anomalies using smartphone inertial sensors. The objective is to enable scalable, low-cost road surface condition monitoring by leveraging widely available mobile devices. RoadSense comprises synchronized data from three streams: (i) mobile phone sensors including gyroscope, accelerometer, and GPS; (ii) vehicle telemetry data from the CAN bus via the OBD-II port; and (iii) front-facing video recordings of the road surface. The mobile sensor data is intended for anomaly detection and mapping; video recordings serve as visual ground truth for labeling surface defects such as speed bumps, potholes, and cracks, while CAN bus data provides a supplementary reference for verifying anomalies influenced by vehicle dynamics (e.g., acceleration). The dataset enables reproducible evaluation of road surface monitoring models, facilitates the development of lightweight mobile sensor-based anomaly detection techniques, and supports future fusion with visual models for ground truth validation.