Siri, tell me what I want to hear

Web sites use increasingly sophisticated algorithms to tailor content to the individual user: Netflix and Amazon deliver personalized recommendations, news aggregators filter content, and Google search results depend not just on the search terms themselves but also on whatever personal characteristics or details of one's historical usage are available. I imagine a near future in which I'll get into my driverless car around dinner time and be driven to my favorite Mexican restaurant without even having to say anything. If the car has enough predictive ability and access to enough data, I won't have to specify that I'm in the mood for Mexican, or even that I'm hungry. The car would never be able to get it exactly right every time, but it's easy to imagine it being close enough that the user wouldn't bother quibbling about it.

So, could this possibly be a bad thing? Eli Pariser, author of The Filter Bubble (and presenter of a TED talk), thinks so. His concern is that all this personalization of content has an isolating effect. My starting point is an idealized view of filtering as a way of culling some subset of information from all of the information available. The user can sift through the information the old-fashioned way and decide what to consume--e.g., glance over the headlines and decide what articles to read--or allow the filter to do the sifting. If the filter arrives at the same subset of information that the user would have ultimately consumed anyway, then the filter is unambiguously beneficial insofar as it eliminates the cost to the user of sifting through the information.

The question is then in what way or to what extent the actual scenario diverges from this ideal. I can think of three possibilities:

1. Malicious filtering: a conflict of objectives. I.e. whoever is in charge of the filtering mechanism (the "filterer") delivers content to suit the filterer's objective, not the consumer's. This may be possible whenever the consumer is not at least indirectly in control of the filtering mechanism, but it's hard for me to imagine that it could be pervasive, or even very common. Sites like Amazon and Netflix have an interest in delivering the greatest value to the consumer. There may be a small short-term gain of inducing one to buy a book the consumer would not have otherwise purchased and would not in fact like, but surely the long-term loss (less trust of the Amazon filter, fewer purchases through Amazon) would be larger.  In general, this kind of filtering is hard to maintain over time if the consumer eventually receives relevant information outside the filter--gosh, I didn't like that book Amazon recommended; or hey, why did my news filter not show me that article everyone is talking about?
2. Exploitation or magnification of cognitive biases. E.g. if I am prone to confirmation bias, and I believe the earth is flat, then I am more likely to consume information consistent with this belief and disregard contradictory information. Whether I do my own filtering or rely on technology to do it for me, consuming more information tends to strengthen my belief. Perhaps the filter accelerates this process, but I'm not sure this is a problem overall. As Gentzkow and Shapiro point out in "Ideological Segregation Online and Offline," the Internet affords users some ability to isolate themselves from conflicting ideologies, but it also makes much much more information of all kinds available. To quote from G&S's abstract, they "find no evidence that the Internet is becoming more segregated over time." I would offer one caveat to this conclusion: their data ends a few years ago, during which time filtering mechanisms have advanced considerably. I wonder if the same conclusion will hold up in five years, or in ten.
3. Appeal to users' lack of self control. I.e. filters may give consumers the information they want to consume rather than what they should consume. An article (whose provenance I can't recall) pointed out the differences in viewing habits that Netflix streaming introduced among its users: they were on the whole less likely to stream movies that might be viewed as important or edifying--foreign films, historical dramas, etc.--than they would be to watch these same movies on DVD. As a Netflix user myself, I can testify that there are many movies available for streaming that I want to watch, in principle, but may never get around to watching. In the moment of choice, I am much more likely to go for the action thriller or the stand-up comedian instead of the "important" film; although I sometimes choose the latter kind of film when I have to make my choice in advance, by ordering the DVD. Mark Zuckerberg has been quoted, by Pariser and others, as saying that "A squirrel dying in front of your house may be more relevant to your interests right now than people dying in Africa." Pariser seems to be saying that we should care more about people dying than about the squirrel, but it is difficult to make such a claim with any kind of objectivity. The more compelling claim is that there is a problem when there is a difference between the information consumers want to consume and what they themselves think they should consume. In this sense, filtering could play the role of someone following a consumer around all day, offering a chocolate chip cookie every five minutes. The consumer might value the economic development that has made acquisition and consumption of cookies convenient and inexpensive, but might perceive harm to oneself if cookies are too readily available. A rational consumer might refuse the services of the cookie dispenser, if that is the choice under consideration; but the same consumer might still take the cookie when it is offered.

There is the potential for interaction of #1 with #2 and #3: biased or limited information processing on the part of the user gives the filterer greater ability to pursue its own objective. Any kind of filter incorporates feedback from the end user, but it is often not direct. If my refrigerator has the ability to track the food I eat and choose which groceries to order, it might notice that I like junk food and therefore supply more of it to me. On the other hand, I might eat lots of fruits and vegetables if they are readily available to me, and the refrigerator would notice this. In this way, filters involve some degree of momentum, and small perturbations may have far-reaching effects. One way to optimize the filtering process would be to make it transparent to the user. If the refrigerator checks in with me every so often--e.g. says to me, "I have noticed that you like junk food more than fruits and vegetables. Is that correct?"--then I have the ability to steer the filter in the direction that I think is best, given the opportunity to exert self-control, whereas my knee-jerk decisions may not. This kind of transparency could only help to head off the issues in #1 and #2 above also, although it would not necessarily eliminate them.

For all sorts of filtering, my questions are:

• Is it a problem?
• If so, is it primarily a human problem, or a technological one?
• What can and should filterers and end users do about it?

(Not to mention the meta-question: Is there an interesting research project here?) Filtering of some kind has always existed. Different consumers turn to different news sources--e.g. Fox News vs. Huffington Post--and these news providers select, or at least emphasize, news that they think would be of interest to their audiences. I think of these as "blunt" filters, in contrast to more highly individualized mechanisms like an internet news aggregator. It is not clear to me whether there is some fundamental difference in the implications of the types of filter, or if it is simply a question of degree.