The invention discloses an AR augmented reality-based musical instrument-assisted play and concerto experience system. The system comprises an image acquisition unit, an image analysis unit, a second storage unit, an operation unit, a processing unit and a display unit, wherein the image acquisition unit is used for acquiring a video image of a current user held musical instrument in real time, and the video image comprises holding finger nails; the image analysis unit is used for analyzing an image of the current musical instrument, recognizing a profile and an operation area of the current musical instrument, and recognizing and positioning strings or keys of the musical instrument; the second storage unit is used for storing music scores, finger actions corresponding to syllables and notes, and virtual animation elements of the operation of an auxiliary instrument; and the processing unit is used for displaying the positioned strings, keys or striking areas of the musical instrument on the display unit according to the profile and the operation area of the current musical instrument, setting a display area of the animation elements, and calling the corresponding music scores, corresponding pressing actions and the virtual animation elements of the operation of the auxiliary instrument in the second storage unit to be displayed in the display area of the animation elements.