If you don’t filter you won’t suppress aliasing so you will have tons of high frequency noise above Nyquist.
Linear phase have more latency thus produce pre-ringing which is a pre-echo. This can have a bad blurring effect if the filters are too steep. As long as the slope is low order the ringing is kept minimal. I believe linear phase produces more latency due to the more intensive math calculations to maintain phase coherence.
Min phase filters sacrifice phase so they are lower latency.
To my ears, min phase are not musical and sound analytical. I do not like the phase issues, instruments tend to have improper depth to my ears.
The only way you avoid these issues is by getting an R2R DAC which have no upsampling. In those DACs however you can’t play hi res or DSD formats, they are limited and quite $ in comparison. They also have other issues.