Human activity recognition with wearable body sensors receives lots of attentions in both research and industrial communities due to the significant role in ubiquitous and mobile health monitoring. One of the most concerned issues related to this wearable technology is that the sensor signals significantly depends on where the sensors are worn on the human body. Existing research work either extracts location information from the activity signals or takes advantage of the sensor location information as a priori information to achieve better activity recognition performance. In this paper, we present a sparse signal-based approach to corecognize human activity and sensor location in a single framework. Therefore, the wearable sensor is not necessarily constrained to fixed body position and the deployment is much easier although the recognition difficulty becomes much more challenging. To validate the effectiveness of our approach, we run a pilot study in the lab, which includes 14 human activities and seven on-body locations to recognize. The experimental results show that our approach achieves an 87.72% classification accuracy (the mean of precision and recall), which outperforms classical classification methods.