The invention discloses an abnormal off-duty identification system for an electric power business hall. The system can solve the technical problems of low manual post attendance efficiency and low automation degree, and is characterized in that a counter front end network camera can collect the counter area personnel information and the counter top end service board information, processes and analyzes the data as the basis whether the counter staff leaves the post abnormally. The system can real-timely monitor the personnel information of the staffs in the counter area during the work period,can detect and identify whether the abnormal off-duty information exists, during the monitoring process, can real-timely display the attendance information of the staffs in the area and the on-duty/off-duty time information, can give the accurate attendance and on-duty/off-duty information of the counter staffs in the electric power business hall based on the detection and recognition of images and text information, has the staff on-duty service period monitoring and reminding functions, and meanwhile stores the information of the staff in the on-duty service period, is connected with a central control module of the electric power business hall via a communication interface module, and is managed and controlled by the central system. The method can effectively improve the detection accuracy, is high in real-time performance, and improves the dynamic monitoring and early warning capability of a supervision department of the electric power business hall.