A vehicle control system includes: an inter-vehicle distance control section configured to perform a follow-up control to a preceding vehicle by automatically accelerating or decelerating a host vehicle, or a constant speed running control section configured to perform a constant speed running to converge to a set speed; a brake operating state sensing device; and a control unit including; a deceleration control section configured to perform a deceleration in accordance with the brake operating state of the driver when the brake operating state sensing device senses the brake operation of the driver during the inter-vehicle distance control or the constant speed running control; and an acceleration control limit section configured to cancel a control in the acceleration direction which is performed by the inter-vehicle distance control or the constant speed control when the brake operating state sensing device senses an end of the brake operation of the driver.