After looking it up, in the Oxford English Dictionary," it seems clear
that the term elitism, ("advocacy of or reliance on the leadership and
dominance of an elite") has quite a lot to do with the opinions and
self-esteem of those who use it.  In socially stratified societies, elites
were self-selected for leadership positions and for the best education.
They exercised power in society.  As late as Lenin the term was used
approvingly.  In democratic or egalitarian societies, the terms "elite"
and "elitism" are generally used to express social disapproval rather than
approval.  Some folks' self-esteem is based on the belief that they are
better than other people and others' self-esteem is based on the belief
that, no, the former folks bloody well aren't better than they are.  We
are not in the realm of precise meanings here.

In a context of classical music, if the term "elite" is meaningful in
anything close to a precisely definable way, I would suggest that the OED's
1968 usage example ("Formal schooling which a generation ago was still
confined to a small elite") might be the most helpful for framing a
discussion.  Classical musicianship does require considerable formal
training as a rule, whereas non-classical musicianship generally doesn't,
in my observation.  However, simply liking to listen to classical music
rather than non-classical music depends simply on taste rather than
training.

One problem I have with charges of (musical) elitism is that nowadays is
that it is typically applied simply on the basis of what tastes a person
may have rather than the degree of training s/he may have undergone.
What's more, you don't hear charges of elitism applied to, say, electrical
engineers, just because they may have undergone considerable (expensive)
education.  So there is something odd going on in the rhetoric of musical
culture.  And it's a touchy matter.

Jim Tobin