One of the central arguments of the formalist position is that music - and here we are always talking about instrumental music - has a syntax and logic without any semantic content. Kivy nicely sums up the argument here:
"According to the formalist creed, absolute music does not possess semantic or representational content. It is not of or about anything; it represents no objects, tells no stories, gives no arguments, espouses no philosophies. According to the formalist, music is 'pure' sound structure; and for that reason the doctrine is sometimes called musical 'purism.'"
Of course, the formalist position helps us to avoid a lot of nonsense. It is the best part of Stravinsky's argument when he said that his post WWII music was not supposed to be expressive. Think of it like a coffee cup. An extremist might say, this coffee cup is only valuable for the function it serves which is to carry the content which is coffee. It is a similar argument that we hear in regards to music education sometimes. Music is good because it helps kids score better on math tests. In this case, music is good because it is a vehicle to chauffeur emotions around. To which the formalist can say, "Music isn't valuable because of its functionality. Music is just valuable in and of itself. It doesn't have to do anything. It can just be without having to mean something."
There is a lot to like about this. I find so much foolishness going around even in the academy when it comes to "understanding" music. So, we get these bizarre shibboleths that become the hallmarks of education. "Gesualdo wrote weird music because he murdered his wife." "Spirituals are really secret codes about escaping to the underground railroad." "Chopin's Op. 28 No. 15 is about him collapsing on the piano from tuberculosis and having a leaky roof dripping on his head." All of this is quite silly, because people write weird music without killing their wives, spirituals without traveling the underground railroad, and piano music with repeated notes without contracting tuberculosis and having water drip on their heads.
Where I'm not sure that Kivy gets it right is when he equates music with what is essentially Chomsky's famous phrase of grammar without meaning. "Colorless green ideas sleep furiously." The problem may be the opposite of what Kivy thinks, and he doesn't actually address this issue.
I will tackle this issue with tonal music, since that seems to be the main music that Kivy is addressing. For most musicians that I know, when we do a Roman Numeral analysis of a piece, we aren't creating some sort of taxonomy. In fact, the V7 that we write is actually a poetic symbol. For musicians, that symbol is extremely rich with meaning. The problem may very well be that my V7 is not too vague to mean something, but that it is too specific. Here are Felix Mendelssohn's thoughts on the subject.
"Die Leute beklagen sich gewöhnlich, die Musik sei so vieldeutig; es sei so zweifelhaft, was sie sich dabei zu denken hätten, und die Worte verstände doch ein jeder. Mir geht es gerade umgekehrt. Und nicht bloß mit ganzen Reden, auch mit einzelnen Worten, die scheinen mir so vieldeutig, so unbestimmt, so missverständlich im Vergleich zu einer rechten Musik, die einem die Seele erfüllt mit tausend besseren Dingen als Worten. Das, was mir eine Musik ausspricht, die ich liebe, sind mir nicht zu unbestimmte Gedanken, um sie in Worte zu fassen, sondern zu bestimmte."
[People usually complain that music is so ambiguous; it is so problematic that they don't know what to think of it, but that words can each be understood. For me, it is exactly the opposite. And not merely with speech as a whole, but also with single words, they appear to me so ambiguous, so undefined, so misunderstood in comparison to a true music that fills the soul with a thousand better things than words. That which I pronounce in music, that I love, is to me not a thought that is so undefined that it cannot be grasped in words, instead it is too specific.] (Translation mine)
So, Felix says that the problem is not that music doesn't have semantic content, it's that the emotional specificity is so exact that words are actually blunt instruments in comparison. In any case, it's an argument that formalists need to confront.