You still have the issue of efficiency match between the speakers and the subwoofer… it’s just too difficult to integrate passive subwoofers without a dedicated outboard amp/crossover.
May not be disastrous for your amp, but very likely for the sound