New on my blog: “Engagement Is the Enemy of Serendipity”—a seemingly minor change to the New York Times iPad app provides a good case study in what’s wrong with the algorithmic personalization of our media, and why we need to bump into the unexpected.

@dancohen I like the post, Dan. A few quick thoughts.

(1) This is a nice addition to the running list I'm keeping of "things tech bros worry about that scholars can do differently." (Needs a better title.)

(2) That word serendipity keeps popping up. Mike O'Malley has got me thinking about it recently too.

(3) "For You” sections don't ever work for me because, well, I don't really like myself. 😀

@dancohen Suppose we were trying to evaluate a digital history project. Is "engagement" one good measure of its value? Or is there a better way of conceptualizing the idea that term is supposed to capture?

@lmullen Thanks for the response. I would add to your list @brentsimmons’ recent post “No Algorithms.” In general, I’m trying to define “serendipity” better and to retrieve it from the clutches of those techies who may think that For You is serendipity, or to think about what healthy serendipity is vs. unhealthy. I’m sure that @ayjay will cover the presentism I mention much more rigorously in his forthcoming book. On impact measurement, I recommend Europeana’s Impact Playbook, which tries to get away from page views, engagement, and the like.

@dancohen Cool. Thanks for the link!

@dancohen It’s very true: the algorithm pretends to know us as we are, even better than we know ourselves. Certainly better than the self we might become given opportunity. I think too about role segmentation. Just because in one context I might have need of a particular type of information, it doesn’t mean that those searches are the entirety of me as a person. Assuming that the tiny glimpse through those contexts is the entirety of a person is so shortsighted.

@sproutlight well put

@sproutlight Ooh, that’s the algorithmic variation of the larger social media phenomenon I keep referring to as the “flattening of our selves/identities”; I hadn’t thought about that angle before.

That also brings to mind the “needlessly insensate technology” (to borrow a phrase from @schuth in this conversation with @walter) or “inadvertent algorithmic cruelty” (to use Eric Meyer’s term) of reminding us of the special days of the deceased and the like—the algorithm doesn’t know we’ve kept someone in our contacts list as a way to hold on to them but find its reminders to call to wish the person a happy birthday insensitve, at best (because an algorithm has no empathy, it can’t sense, it can’t understand us; it can only treat our actions as data and classify them according to its rules and the collection of others’ actions it was “trained” with…).

// @dancohen @lmullen

@dancohen Don’t know whether “rigorously” and “@ayjay” go together — perhaps “at greater length” would be more accurate.

@dancohen I'm trying to figure out whether digital life and real serendipity are fundamentally incompatible. I avoid any algorithmically-generated news feeds because they always serve me things that 'm not just uninterested in but actively hate (people dying in freak accidents, "Florida Man Arrested," etc.) That's not serendipity, but rather training in depravity. So I take refuge in my RSS feed, but there's not much serendipity there because I have chosen everything in it. So what's the alternative? A site that serves you completely random pages from the internet? Hmmm ... you can get a random Wikipedia page, which is pretty cool.

@ayjay This is an excellent and important question. The kinds of serendipity I mention in that piece are very analog: the print newspaper, the record store, the bookstore, etc. I'm also heavily dependent on RSS, but the 400 feeds I follow are largely from digital humanities and the early years of blogging, and I haven't found or added much recently (and many of those feeds are moribund, alas). So what would digital serendipity look like? There's some hope in generous interfaces, perhaps, like the ones that Mitchell Whitelaw designs for cultural heritage collections. BTW, I assume you heard that Radiolab where Latif Nasser explains how he finds interesting stories for the show using the Wikipedia Random Article button?

@dancohen I did not hear that on Radiolab! How funny. Also, re moribund sites: Newsblur allows you to sort feeds by recency of posting, whican casn be sobering. I am subscribed to several sites that haven't been active in 3,500 days. (Is that a lot?)

@ayjay Here is my sad method of unsubscribing from once-thriving blogs: when they are hacked and fill my RSS reader with spam.

@dancohen @ayjay @lmullen @sproutlight @smokey I’ve been following this thread with interest. Last year I had been hopeful about Garrett Camp’s Mix as a “curated discovery” engine but, as I complained earlier this year, its “For You” has become almost indistinguishable from a social media news feed. As I said in that post, algorithmic selection/recommendation tends “relentlessly to penalize and erase dissimilarity, novelty and surprise”. I think it’s possible that the combination of machine learning, neural networks and data will eventually do a better job but we might have to wait a long time to see the results.

@dancohen Consider it a particular effective kind of notification.

@artkavanagh Re: “relentlessly to penalize and erase dissimilarity, novelty and surprise” — that's exactly right, and what I'm thinking is that that is intrinsic to the character of an algorithm, any algorithm.

@sproutlight i think this is a great point. Especially given the epistemic limitations of us even knowing ourselves well. It reminds me of how seldomly even I know what songs/genres will match my mood. Even if an algorithm could identify my esoteric musical preferences and flighty emotional states, it’d still have to find a way to determine if that current bout of sadness is better expressed in Radiohead, Debussy, or Muddy Waters. And I certainly wouldn’t want to listen to all three back to back

@vega You make a good point — and this is something I try to do in my newsletter — but in the end all those links come from a particular person’s sensibility, which limits the range of reference, you know? I link to things because of what I’m interested in, and that’s intrinsically limiting.

@vega these are good points, and why I’ve been increasingly interested in newsletters (like @ayjay’s) that are indeed largely human-curated hyperlinks with commentary.

@ayjay Very true, but the limitations of individual scope could be overcome if enough individuals do it and feed off each other.

I subscribe to your and @robinsloan's newsletters (Sloan came to my attention both via your blog and someone else's - identity long forgotten - reference of Fat Gold), and I consider both newsletters as my "serendipitous connections" to new ideas. Ideally, I ought to reciprocate and produce my own newsletter of interests and links for you and others to benefit from. (Newsletter is unlikely, but I'm hoping to create a directory of links through my very tiny website eventually -- all as a matter of practicing what I believe.)

But yes, it's still ad hoc, privatized, and limited to my own scope of interests. It makes me wonder whether you're right about the incompatibility of (current) digital life and serendipity. The culture of Internet and tech use has definitely shifted to prioritize reliance on centralized search a la Google over small-scope methods like human curation.

I did like @dancohen's example of "generous interfaces" though.

@vega I think the idea of generating a critical mass of peculiar individuals is the key. It’s one reason why some people have tried to resurrect the idea of the webring — a chain of blogs that may have similar interests or a common tone but are different enough that you’d want to browse them all.

@dancohen @vega FWIW, I have found that making newsletter is the single most fun thing I do online.

@ayjay @vega I am close to launching a newsletter of my own, but as is my (academic) nature I’ve been thinking a lot (too much) about what I’d like to do with it. But maybe thinking about it more as my fun outlet (my blog and podcast are more serious, although I try to add levity to them) and passing along good things I’ve encountered to others will be enough.

@vega Serendipity is the opposite of the filter bubble which leads to intellectual isolation.

Facinating thread. I think hyperlinking is the key here, it's free, low tech and we control it. I like the idea of adding curated but serendipitous links to newsletters, but I also like the old idea of link pages or surf pages: a public list of hyperlinks to things we like.

Remember those old Web 1.0 Top List voting sites? These were generally labeled "Top 50 Star Trek Sites" and ranking was by remote voting. I could sometimes dig up the most obscure pages via those - not in the top 10 but deeper down the list.

There are other ways: Boingboing.com serves links to odd stuff daily.

I tried to add some serendipity to my web directory, by adding a random link feature but I'm not sure how successful that is.

@kicks makes an awesome whimsical list of links at href:cool.

So yes I would encourage everyone to make time to link to things that are just fun. cc. @dancohen

@bradenslen I suppose we should remind ourselves that blogs started out as web logs, i.e., as interesting web links, including the very first Mosaic landing page.

@vega Great comments. Even without algorithms, this can be trouble—on subreddits, posts can be flagged ‘offtopic’—so overboard moderation is a problem. (Of course, Reddit is where one goes to fully ‘engage’. No /s—it’s fine to do that. Problem is: people may not know where to go to get outside of ‘engage’ mode.) One thought I’ll add re: getting outside of my own interests—I think if we had better tools for keeping tabs on our interests, we could more easily move outside them. (Like: if my ‘reader’/‘news feed’ makes it difficult to track 100 people, then I can’t very well track 1,000 people.) And directories are sweet here—they are little libraries. Sure, they can cover your interests. But they can be used to map the strange elven lands that you happen to sally in.

@kicks re tools: I think that "unlike things adjacent to each other" helps a lot; the examples in @dancohen's article -- record store, newspaper -- depend on spatial organization to promote serendipity even when one begins at a point of their own interest. Browsing my Pinboard.in account revives and reminds me of old interests -- the links may be organized by date in a flat-file, but diverse things are adjacent to each other. I've also been experimenting with Zettelkasten for various info-management and creative purposes (haven't progressed far though).

I'm generally in agreement with this paper that spatial organization is integral to human interaction with things outside the self; but digital tools and AI aren't that good at doing it yet. Human-curated hyperlinks and organizational methods are a step towards it, but perhaps this is an area for future IT development.

@vega @kicks Vega I'm thinking about this "unlike things adjacent to each other" in the context of the Web 1.0 "free" banner exchanges. But now that I've blurted that out, I see too many downsides to it: keeping it from becoming commercial, cluttering up blogs with banner ads for poor click through ratios so never mind, too much work for to little gain. SImple hyperlink is easier and less intrusive.

@ayjay There's a lot of serendipity in RSS when you don't filter anything, which I don't.

@ayjay urlroulette.net

@ayjay That's too much.

@dancohen I usually unsubscribe if they haven't been active in the last year and a half.

@vega I particularly like linkblogs that surface cool sites.

@vega I need to get my crap together and do a directory at some point. The problem is I have so many links I don't know where to start lol.

@vega Also I like newsletters like that.

@dancohen Very late to the discussion but I fondly remember the days of when I turned those guest book cgi/perl scripts into a blog instead, worked wonderfully for that sort of thing.