This device (10) comprises a fume circulation casing (16) inside which there are integrated both a filter (12) having vertical filter bags (18) and a channel-structured catalyst (14), the channels of the catalyst having a hydraulic diameter of less than 6 mm, the fumes to be cleaned (1) passing through the filter bags and then the catalyst. Inside the casing, the catalyst is positioned above the outlet of the filter bags and is divided into unit elements (22) that are placed horizontally side by side inside the casing and are vertically removable from the casing. This purifying device is made compact, economical, practical and efficient by distancing the catalyst from the outlet of the filter bags by a vertical distance (d) of between 2.5 and 6 times the hydraulic diameter of the filter bags, and the unit elements are positioned side by side inside the casing being arranged successively in several rows in a horizontal direction (X), each unit element being equipped with moving means (24) for moving the corresponding unit element in this horizontal direction relative to the casing.