Ebook distribution and the problems of aggregation

The internet, and particularly the ebook industry, is littered with debates about aggregation and monopoly, and in many of those debates the two concepts are confused. Let’s be clear: aggregation is good, monopoly is bad.

For instance, a serious discussion is finally emerging over the Google Books settlement, a discussion that recognises the value of Google’s aggregation of book content and warns about the dangers of giving Google a monopoly over the control of that content, thereby potentially corrupting power (Tim O’Reilly gathers some of the best discussions on O’Reilly Radar, and Chris Castle is interesting in The Register). What are aggregation, monopoly and power in this context?

Google already pretty much controls Search on the internet, and Search is right now the key to power online. The usefulness of Search is a combination of massive aggregation of content and the ability to return good results to the searcher. Google runs this show not only because you and I use Searches to find what we’re looking for, but because every time we search for something, our search gets recorded, forming over time an immense dataset of people’s interests and contexts, often tied to individual users’ online identities. That dataset is then analysed and sold on in the form of contextual advertising, individualised search results, trend analysis and other kinds of analytics. By searching, we’re adding to Google’s brain, it’s knowledge of us as individuals and as a culture. As Kevin Kelly put it in his TED Talks presentation “Predicting the next 5,000 days of the web” in 2007: “who is searching who?” When everyone is using Google, our searches are microscopic components of a much greater view of the world, gathered in Google’s eye. When I search, I’m Frodo putting on the ring, Sauron suddenly alive to my whereabouts. Because of its massive ability to aggregate content and searches for it, Google has immense power. (So far, thankfully, it has no Nazgûl to unleash. I don’t think it will under its current management, but discussions like the one over the Google Books settlement aim to keep that door closed just in case.)

Google does not, however, have a monopoly. It leads Search because no one else does search as well as Google does it, but Yahoo and Microsoft and others do get a little piece of the pie, and no one’s stopping them from trying for more.

So now Google is adding books to its searchable content and therefore the usage stats from book searches to its brain. The settlement adds a new dimension to this: Google will be involved one way or another in the sale of the books listed in its searches as ebooks and perhaps eventually printed formats too. Much of the discussion around the Google Books settlement is about whether the settlement grants Google too much power by law, where till now its power has largely been earned by its ability to aggregate and filter content well. (For instance, the settlement may give Google undue influence over the book registry whose creation the settlement mandates, and in theory Google could list certain books higher in its search results, in relation to Google’s commercial stake in their sale.) Power granted by law starts looking like a monopoly. The settlement must allow for aggregation, but avoid granting any kind of monopoly.

We like aggregation, even if it leads to one company having a lot of power. In print books, Amazon is effectively the greatest aggregator of content and, as such, offers us a one-stop search for most books. iTunes offers the same for music. A one-stop search is not only useful, it’s critical, because in online retail, convenience and speed are half the value you’re selling. The value of aggregation is often worth the risk of corruptible power. But monopoly, imposed by law, is hard to break down and leads to slack business practices. Monopoly is less common in software and online than in physical goods and, when it emerges it’s often stamped out before long (the best example is perhaps the EU’s tough stance on Microsoft’s bundling Internet Explorer with Windows).

A monopoly of any sort in ebooks would be terrible, even if run by Google, the internet’s benevolent dictator. But the ebook industry desperately needs real aggregation. Ebook distribution is highly fragmented, despite the presence of large distributors like Ingram and OverDrive or distributor-retailers like Ebooks.com. As a user, I cannot go to one ebook store and know with any measure of certainty that I’ll find the book I’m looking for. This means that ebook shopping is all browsing-based shopping: browse around till something looks interesting, then maybe buy it. Online shopping, however, is about instant gratification, about search–find–buy in three clicks: no one has time to browse about any more.

Why has this happened? This are several contributing factors and I’m doubtless going to forget some in this list:

Problem: Distributors of ebooks make it expensive for retailers to list their ebooks. By expensive I mean they often charge set-up fees, in addition to a cut of sales. This (a) forces retailers to choose between distributors, at least when starting out, and (b) discourages small start-ups from entering the bookselling market. Solution: We need more automation in the distribution process, something like an open API for retailers to list and sell a distributor’s ebooks. It’s only a matter of time before a distributor drops the retailer set-up fee and provides a quick self-help facility for low-tech retailers, and in so doing encourages a surge in small ebook retailers with instant catalogues. (Interestingly, this is more or less possible already for print books, using a combination of wholesale book suppliers and Amazon’s Marketplace Sellers.)
Problem: Distributors are naively hoping they can become the Amazon or iTunes of ebooks, centralising ebook aggregation to themselves. As long as they’re battling over market share like this, they cannot allow just anyone to sell their ebooks (recently, OverDrive pulled its catalogue from retailer Fictionwise for reasons still unknown but much speculated about). Distributors end up in camps that hamper an end user’s ability to find and buy the ebooks they want. Solution: see point 1.
Problem: Publishers are too concerned about protecting their content with DRM (digital rights management), seemingly oblivious to the music industry’s hard learning experience. DRM adds to distributors’ costs and is almost always detrimental to customers. Publishers’ obsession with security (as opposed to convenience for customers) also forces distributors to gate keep (often with retail set-up fees or laborious registration processes), so that they can monitor the conduct of a manageable number of retailers. Solution: follow Pan Macmillan’s example and start dropping DRM.
Problem: In these early days of digitisation (early for book publishers at least), digitisation companies and distributors often make more money from services to publishers than from ebook sales, so their attention is not focused on competing for end-user ebook buyers. Solution: This is a teething issue and a chicken-and-egg situation: further aggregation of ebook catalogues, easier ebook-buying processes, standardised formats, and inevitable changes in reading habits will create more competition for ebook sales.
Problem: A plethora of ebook formats makes it hard to provide a one-stop shop. Solution: Thankfully the format wars look solved, with only PDF and epub left standing. The next step is solving the DRM debate (or admitting that it’s already over).

The raw power of Google, or an ebook Amazon or iTunes, would provide a useful centralised aggregator of ebooks and would go some way towards solving these problems. But the solutions would come far more easily if we could decentralise aggregation. By this I mean that publishers and distributors could make it possible for anyone to list and sell their books: if anyone could open an online retail store and instantly sell ebooks from any savvy distributor – no set-up fees or overnight processes involved – then we wouldn’t need Google for most of our ebook purchases. All retailers would carry almost all ebooks. And a retail store’s usefulness, and therefore its success, would be determined not by its catalogue but by the convenience and support it offered its customers.