The invention provides a multi-sensor system that uses an algorithm based on adaptive filtering to monitor a patient's respiratory rate. The system features a first sensor selected from the following group: i) an impedance pneumography sensor featuring at least two electrodes and a processing circuit configured to measure an impedance pneumography signal; ii) an ECG sensor featuring at least two electrodes and an ECG processing circuit configured to measure an ECG signal; and iii) a PPG sensor featuring a light source, photodetector, and PPG processing circuit configured to measure a PPG signal. Each of these sensors measures a time-dependent signal which is sensitive to respiratory rate and, during operation, is processed to determine an initial respiratory rate value. An adaptive digital filter is determined from the initial respiratory rate. The system features a second sensor (e.g. a digital 3-axis accelerometer) that attaches to the patient's torso and measures an ACC signal indicating movement of the chest or abdomen that is also sensitive to respiratory rate. This second signal is processed with the adaptive filter to determine a final value for respiratory rate.