That’s an incorrect statement. Based on what @tedsmith said, it’s neither a good idea nor a bad idea to unbalance the transformer from an electrical point of view. What one has to weigh is what one may lose by unbalancing it. As @tedsmith said, unbalanced connections most of the time are not a problem. We all get get caught up in the idea that balanced is always better but that is just not the case.
So, “It’s generally not a good idea to unbalance the transformer” only when balanced offers better sound quality over single ended.
Answering the OP’s question with the right answer is important and @tedsmith’s answer I quoted is the right answer. Especially considering what the OP wanted to do. Your answer was that he shouldn’t use both outputs when you do not know if it would cause sound quality problems for the OP. Yet you were willing to make the choice of him using his headphone amp for him even though it may have worked perfectly well in his situation.