Ooops... BlackHOLE servers has nothing to do with BlackLIST servers.
I'm the poster on the first comment about IANA.ORG blackhole servers, and I guess you didn't understand me.
IANA Blackhole servers are actually good for the health of Web traffic. They drop reverse DNS queries that shouldn't exist on open Web, only on local networks, on non-routable IP ranges like 192.168.xxx.xxx. Without them, this traffic would keep repeating itself in search of a reply, when nobody will answer. It is like shouting "Hello" on the mountains, the echo will fill the place with noise. Without this IANA servers, chaos would ensue. Following the analogy, IANA blackhole servers would shout back "Shut the f*** up, nobody will answer you!", so the DNS queries stop. I heard Windows machines do that once every 24 hrs.
BlackLIST servers are entirely a different thing, they are blocking spam traffic, or listing spammers IP's, (so you said), and I can't foresee how would their absence would affect global latency. Only stupid spammers use their local ISP IP as the originating point, anyway.