Well, your post is very timely. I did a little research this afternoon and learned something – books (with their paper and cardboard material) tend to act more as an absorber than as a diffuser. In my ignorance, I had assumed that the irregular depth of the spines on the wall would scatter at least some frequencies. Not so; apparently.
That said, I am still working through my go-to reference material to try and discern whether my efforts have helped or hindered (or simply created some additional shelf space
.) I just finished tucking books into the open spaces yesterday, but so far I get the impression that there is a bit less smearing of distinct “images” in certain, well recorded material - kind of like a voice or an instrument is more specifically there, in a particular place - if that makes sense. But I may be experiencing some expectation bias. More listening is needed. I definitely have not heard any deleterious effects yet.