Others let the ears be present and they don't make an effort to understand. To listen is an effort, and just to hear is no merit. A duck hears also.However, there is probably something a bit deeper when it comes to separating us from the ducks; and I would have thought that, being a psychologist, Sloboda would have been aware of it.
At the risk of making things sound too pretentious, I believe that the distinction may best be captured through a distinction one encounters in semiotics. Hearing is all about detecting and processing signals. To the extent that it involves any aspect of mind beyond basic sensory responses, it involves detecting signal properties. Those include temporal properties, as well as spectral; but, if we are talking about looking for patterns, they are patters that take place in the abstract world of trace graphs in both the time domain and the frequency domain.
When we move into the domain of listening, we approach those signals as if they are indicators of something symbolic. I use that word with a bit of caution, since we may not be talking about symbols the way we talk about how words can serve as symbols in linguistic communication. However, listening introduces one or more factors that take us beyond signals that are defined strictly in terms of their physical properties. Like other forms of communication, those factors may bring into play features the characterize making music as a a motivated action, rather than merely a playing out of phenomena in time. My guess is that just what those features are remains a topic for further research, but that research is likely to provide far more insight than Stravinsky could express through his pithy used of language.