Benjamin Berman: It’s Not You, It’s The Algorithm

Heichi’s Note:

Heichi Magazine is glad to announce a new collaboration with the Hyundai Motorstudio Beijing, jointly presenting a collection of essays edited by curator Jenny Chen Jiaying. Under the rubric of her curated exhibition, AI: Love and Artificial Intelligence, currently on view in Hyundai Motorstudio Beijing, this special column presents three long-form essays by the participating artists.

Translator’s Note:

The game Monster Match sits at the entrance of the exhibition AI: Love and Artificial Intelligence and will be the last work addressed in this series, bridging from our magazine’s lineup in 2020 to forthcoming publications in 2021. Previously, we explored molecular symbiosis inspired by “quantum entanglement” in Johanna Bruckner’s essay, then re-examined a critique of dichotomy through Frank WANG Yefeng’s writing, both topics closely related to a condition of polarized discourse and proliferating identity politics. Similarly, this article by Benjamin Berman, one of the creators of Monster Match, also offers a concrete solution to a contemporary phenomenon. Through detailed data and research, Berman explains the logic of all the dating apps at our fingertips and shows how a technology that promises intimacy has actually sown division and nurtured inequality.

Can dating apps help you find love? Yes! An algorithm matching people is now the dominant way singles meet each other in the United States. But there’s one thing most dating app makers don’t want you to know. The particular algorithm they use can do the exact opposite of finding your true love. It can permanently separate you from your perfect someone and everyone likes them, due entirely to factors outside of your control. ​

State of Online Dating

Are dating apps a good part of the internet? If you ask the people who make them, you’ll hear a resounding “yes.” Match Group, the company that owns both Tinder and OkCupid, found in their 2018 survey that “singles met first dates on the internet more than through any other venue.” Those companies would like you to believe that everyone is using dating apps, and they have the numbers to prove it. I personally admire the creator of OkCupid, Christian Rudder, for his intense transparency and rigor in showing how dating apps work and how they’ve helped people meet each other.

But according to independent research, it’s hard to say how good dating apps really are. Despite so much online dating, Americans are feeling lonelier than ever. Some experts believe this is because technology divides people. In their view, smartphones interfere with intimacy, and social media halts courtship with unrealistic expectations of beauty and wealth. But other experts believe the difference in college education rates between men and women is really to blame for dating problems. ​

When looking at dating app data, Mr. Rudder once lamented that “there’s a bias against [black users]” on his website. Yet there’s also evidence that online dating is associated with higher rates of interracial marriage. There’s no consensus whether these apps made dating better or worse. However, one thing is certain: some people get much more from dating apps than others. As both a millennial and game developer, I can assure you that dating apps are a game, and there are winners and losers. But don’t take my word for it: read the online communities dedicated to online dating.​

There, the people who get matches share strategies like what pickup lines to use, what times to start swiping, and even what species of pet you should pose with. The people who don’t get matches talk about past relationships, debate politics, or, more often than not, blame women. The discourse around dating focuses on tips at the expense of what is really special about online dating: the algorithm.

Algorithms have changed over the course of dating app history. When the internet consisted of “bulletin board systems,” or digital notice boards of sorts, the algorithm for matching was essentially random. Whoever showed up on the website is who we got matched with. Nowadays, there are still apps people use that work this way, like WeChat’s Shake, which matches two people as long as they’re shaking their phones at the same time.

Later, dating apps matched people with some “sober arithmetic,” as the creator of OkCupid put it. The idea was that people who matched in terms of answers to personality questions, appearance, location, or other aspects of their daily lives would also be a good match romantically. This algorithm is actually a form of counting, where every matching aspect between two people gives the match a point, and every disagreement takes a point away. The best matches are the ones with the highest scores—a good match in the sense of having a lot in common, essentially.

Dating App Algorithms

Nowadays, apps use collaborative filtering, a sophisticated algorithm famously used to make movie recommendations based on previous movies you’ve watched. Collaborative filtering tries to find groups of people with shared preferences, whatever they are, and then makes recommendations to the individual based on the preferences of the group. Instead of specifying what makes a good match  — age, hometown, taste in music, feelings on smoking, to name a few from OkCupid — collaborative filtering infers preferences from the data. It’s an unopinionated approach to something really subjective, and it works.​

Collaborative filtering is so effective it’s used by almost all apps; this is the important thing to understand about online dating today. It powers your Facebook and Twitter feeds, your Google searches, and your Netflix and Amazon recommendations. It’s not that complicated. You’ve seen this a million times: “You might also like…” How does Amazon know what you might also like, and why does it use the word “also”? Because you’re not the only person on Earth buying tortilla chips. Amazon looks up what else tortilla chip buyers have bought: salsa. So it knows “you might also like” salsa without really understanding anything about the innate relationship between tortilla chips and salsa. The same exact thing is going on with dating, except the thing that’s on offer is people.

Collaborative filtering in dating means that the earliest and most numerous users of the app have outsize influence on the profiles later users see. Some early user says she likes (by swiping right on) some other active dating app user. Then that same early user says she doesn’t like (by swiping left on) a Jewish user’s profile, for whatever reason. As soon as some new person also swipes right on that active dating app user, the algorithm assumes the new person “also” dislikes the Jewish user’s profile. Similar users have similar tastes, according to collaborative filtering. So the new person never sees the Jewish profile.

A recent look at this phenomenon is going to change the way you think about online dating. Users of dating apps make “yes” or “no” decisions on other users, one by one, and that data is counted to figure out whose preferences that user most resembles. Then, data from the older user is borrowed to make recommendations to the newer user: that’s how the algorithm determines which profile to show next. While many legacy dating applications let you browse, most people prefer being given a single profile to look at and making a yes-or-no decision. Apps have arrived at collaborative filtering both because it’s effective in terms of matching and also because its user interface design is preferred over browsing.

There are side effects to collaborative filtering: users with uncommon preferences are poorly served by the algorithm. So app makers have made niche apps, each catering to a group of users, to make collaborative filtering more effective. There are dozens of special interest dating apps, like Amo Latina for Latino users, Jdate for Jewish users, and Grindr for LGBT users. Most of these apps are owned by the same company (Match Group, a subsidiary of IAC). They’re not really different apps: they function by the same logic and even have the same interface.

But for a variety of reasons, they may attract users who have bad luck with collaborative filtering in apps with larger user bases like Tinder. A large user base puts money in the company’s bank account. But a large user base also makes uncommon preferences seem more uncommon, while common preferences become ever more dominant.

There’s a tension between the effectiveness of collaborative filtering and a tech company’s objective to have as many users as possible. The result is many apps, owned by the same people, that divide users into religious, ethnic, sexual orientation, and geographic groups. That is the state of online dating today.

New Results

brand new simulation has quantified our gut feelings about dating apps: that a feedback loop in collaborative filtering gives majority users better matches at the expense of minority users. There’s something innate about collaborative filtering that disadvantages people who are underrepresented in the data. Without malicious intention, collaborative filtering reproduced the underlying causes of inequality in offline life.

This is all besides the point because there is no perfect dating algorithm, only compromises. There is an imbalance between what people want and what people give in dating. All preferences cannot be satisfied for everyone.

A simple fix: dating apps can give you a “reset button” to clear your history of likes and reset how the algorithm sees you. Or today, you can delete and recreate your dating app account. Both fixes take control of the algorithm in one important way: by not helping it. But we’re not in the business of proposing better alternatives to collaborative filtering. And people who suggest we try are missing the point.

Society ought to be able to inspect how algorithms work, in the sense of looking at the code. Facebook CEO Mark Zuckerberg withstood hours of congressional grandstanding to answer questions about Facebook’s newsfeed algorithms. He never explained how the algorithm worked at a fundamental level, which was the question many congresspeople asked because they couldn’t answer it themselves.

The answer is “collaborative filtering,” but we only know that because of a zeitgeist in the software industry, not because anyone outside of Facebook looked at the code. So let’s just look at the code!

This game I developed shows how the typical dating app algorithm works. You won’t actually have to go on any dates. The game is a simulation. You’ll still build a profile though: a monster profile. It’s called MonsterMatch, and it uses collaborative filtering to decide which monsters you’ll get to swipe left and right on — and which monsters you’ll never get a chance to see.

We’re also sharing all the code because laypeople’s explanations are often co-opted to tell you the story the algorithm owner wants you to hear. If you want to see exactly how collaborative filtering works in a dating app, read the algorithm here. Tech companies that deploy collaborative filtering, least of all dating apps, never do this. But they ought to. Sharing the code is the only defense against people telling you one thing and actually doing another with software.

In our opinion, if an algorithm’s code penalizes some people somehow, it doesn’t have to be illegal: people just ought to know how it works. This should assuage big tech companies who resist regulation of core intellectual property like recommendation algorithms. An informed consumer base will improve digital inclusion and allow minority groups to be treated more equitably online. But sharing code is sometimes not enough. Some code, like collaborative filtering, lacks “interpretability.” It’s hard to know why the code does what it does, even when we know what it does. Interpretability crops up whenever the algorithm does counting on lots of data, like counting swipes. So if you look for a piece of code that says, “Score Jewish users worse,” you’ll never find it. That’s not how it works.

In the case where an algorithm can’t be interpreted, we ought to demand data on its consequences. That means if the consequence of an algorithm is discrimination, even if there is no piece of code that says “discriminate,” the algorithm is discriminatory. For example, if there’s something in common about first-time users of dating apps who immediately quit using the app shortly afterwards, we ought to know what that thing is.

We regulate and inspect medicine, energy, finance, agriculture, transportation safety, and education based on outcomes. The key feature of those regulations is sharing information for consumer choice and protection. Those industries still innovate and make money. We ought to apply the same standards to algorithms.

Unlike those other industries, dating apps already collect comprehensive data about users and their behavior. There’s little cost burden to answering outcomes questions. While it’s difficult to know exactly which questions to ask, it would take an afternoon and a database connection to answer them.

Today, we already know one thing: dating apps are effectively segregated. Jdate and JSwipe for Jewish users; Amo Latina for Latino users; Tinder for coastal users. Dozens exist, each app their own community. Users report more satisfaction by using these segregated apps. Dating app creators will write letters to the editor saying as much. But surely there’s a cost to segregation.

Being funneled into a smaller, segregated experience often gives you fewer opportunities. There isn’t any evidence for this in segregated dating apps specifically. The data are not open to independent research. But history has shown segregation to disfavor the segregated. Anti-miscegenation laws reinforced inequalities on future generations. Because it affects who’s having kids with whom, a segregated dating app could be a high-tech version of that shameful past.

In light of this uncertainty, we ought to equip the public with enough information to make an informed choice about what dating apps to use. So share the algorithm’s code, and let the user decide if she’ll get a fair shake in the dating app game.

Our prediction: If people really knew how much these apps screwed them, they’d stop using those apps. For a giant internet company, that’s the scariest thing of all.

Benjamin Berman is an artist and developer in San Francisco, California. After leaving MIT, where he researched how gaming influences society, he now directs a community-authored e-sports game called Spellsource. His professional and artistic work touches on near-future sci-fi (Virtual High, App the Movie), a data-driven society (Workpop, Hear All Ye People), and computer history (Did My Brother Invent E-Mail?) as shown at the Tribeca Film Festival, on the Disney Channel, and in the New York Times.

Jenney Chen Jiaying is a writer and curator. She is now a PhD candidate in Western philosophy at Eastern China Normal University. She holds a Bachelor of Arts from the Department of Art History of China Academy of Art and received her Master of Arts degree from Lancaster University in the UK. She has contributed to media such as Artforum (CN), Artshard, NOWNESS. Recent Projects include: AI: Love and Artificial Intelligence, Hyundai Motorstudio, Beijing, China (2020); Copernicus, E.M.Bannister Gallery of Rhode Island College, Providence, U.S.A (2019); Li Hanwei: Liquid Health, Goethe Space, Shanghai (2019); First edition of the Shanghai Curators Lab, Shanghai Academy of Fine Arts, Shanghai (2018). Jenny’s other academic activities include the First Annual Conference of Network Society “Forces of Reticulation” roundtable and Huayu Forum of Art, etc. Her article “Post-Internet Art Inside and Outside the Chinternet” was included in the collection of essays Forces of Reticulation published by China Academy of Art Press. Shanghai Contemporary Art Archival Project 1998-2010, which she co-wrote and edited, was published by MOUSSE in 2017.






约会应用在网络中扮演了好的角色吗?如果你问开发它们的人,你会得到一个响亮的回答 “是”。Tinder和OkCupid的所有者Match Group在2018年的调查中发现,“相比其他平台,单身人士更经常通过网络开始第一次约会”。这些公司希望你相信所有人都在使用约会应用,并且他们通过数字来证明这一点。我个人很钦佩OkCupid的创始人克里斯蒂安·鲁德(Christian Rudder),因为他在展示约会应用程序的运作与助人交友功能的时候,十分强调透明和严谨。




算法在约会应用的历史中不断演变。当互联网由 “电子布告栏系统”(bulletin board systems,或者说某种数字告示牌)组成时,算法匹配基本上是随机的。谁出现在网站上,谁就和谁匹配。现在,人们使用的应用仍然有这样的工作方式,比如微信的“摇一摇”,只要两个人同时摇手机就能匹配。​

后来,约会应用通过一些 “清醒算法”(sober arithmetic)来匹配用户,正如OkCupid的创建者所说的那样。这个想法是,用户如果在诸如外貌、地理位置和日常生活其他方面的个人问题上能够互相匹配,那么他们就能建立一段良好且浪漫的关系。这种算法采用计数的形式,两人之间达成的每一个匹配项都会增加一分,而不匹配项则会扣除一分。最好的匹配就是获得分数最高的那个人——本质上说,好的匹配就意味两个人有很多的共同点。


现在,应用程序使用一种复杂的算法——协同过滤(collaborative filtering),以根据观影历史进行推荐的功能著称。协同过滤试图寻找有共同喜好的群体,不管这个群体是什么样的,接着它会根据这个群体的喜好向个人进行推荐。协同过滤不需要人们可能有的不同共性(如以OkCupid中的选项为例:年龄、家乡、音乐的品味、对吸烟的态度等),算法无论怎样都能从中找到人们喜欢一个人而非另一个人的依据。线上约会的现状中非常重要的部分是:协同过滤太有效了,以至于几乎所有的应用程序都在使用它。它在你的脸书(Facebook)和推特(Twitter)推送里、你的谷歌(Google)搜索里,还有网飞(Netflix)和亚马逊(Amazon)推荐里。它也并不是那么复杂。你已经看过无数次“你可能也喜欢…”了。亚马逊怎么会知道你可能也喜欢什么,为什么用 “也”这个词?因为你不是地球上唯一一个买玉米片的人。亚马逊会查到买玉米片的人还买了什么:salsa辣酱。所以它不需要真正了解玉米片和辣酱之间的内在关系,就知道“你可能也喜欢”辣酱。同样的事情也发生在约会上,只不过提供的选项是人。


协同过滤也有它的副作用:比如对那些有着特殊偏好的用户就不太友好。所以,应用厂商们也纷纷做起小众应用,以迎合一个用户群体的需求,让协同过滤更加有效。市场上有几十款针对特殊倾向的约会应用,比如针对拉丁裔用户的Amo Latina、针对犹太裔用户的JDate,以及针对LGBT用户的Grinder。这些应用大多属于同一家公司(Match Group,IAC的子公司)。它们并没有本质区别:它们的工作原理相同,甚至有相同的界面。




一个全新的模拟3​ 量化了我们对约会应用的直觉:协同过滤中的反馈循环给多数用户提供了更好的匹配,而牺牲了少数用户的利益。它存在着一些固有的特点,使其不利于那些在(偏好、开始使用约会应用的时间以及数量等)数据中代表性不足的人。协同过滤在无意间,再现了现实世界机会不平等的潜在原因。







在我们看来,如果一个算法的代码以某种方式惩罚了某些人,它不一定是非法的:但人们应当知道这是如何发生的。这应该能安抚那些抵制对推荐算法等核心知识产权进行监管的大型科技公司。知情的消费者将能改善数字包容(digital inclusion)4​ ——少数群体在网上受到的待遇。但有时分享代码是不够的。有些代码,比如协同过滤,缺乏“可解释性”。即使我们知道代码的作用,也很难知道它这么做的原因。每当算法需要计量大量数据时,比如计算滑屏(swipes)的数据,就会出现“可解释性”的问题。所以,如果你试图寻找一段代码,如“犹太裔用户的得分更低”,你将无功而返。这并不是它的工作方式。




如今我们已经知道:约会应用们实际上是在隔离用户——针对犹太裔用户的JDate和JSwipe、为拉丁裔用户设计的Amo Latina与为沿海用户服务的Tinder——几十个应用程序同时存在着,而每个应用又都有自己的社区。通过使用这些区隔的应用程序,用户的满意度显得更高。约会应用的创建者也会写信给编辑表达同样的意思。但区隔肯定是有代价的。




本杰明·伯曼(Benjamin Berman)是美国旧金山的一名艺术家和开发者。他曾于麻省理工学院从事关于游戏社会影响的研究,现在他在指导一款由社区创作的电子竞技游戏Spellsource。他关于近未来的科幻艺术片《虚拟狂欢,电影应用》(Virtual High, App the Movie)、关于一个数据驱动社会的作品《劳动人民,听一听吧》(Workpop, Hear All Ye People)和关于计算机历史的作品《我兄弟发明了电子邮件吗》(Did My Brother Invent E-Mail)在翠贝卡电影节、迪斯尼频道和《纽约时报》上都有展示。

中文译者陈嘉莹,作者,策展人,现为华东师范大学西方哲学在读博士。曾为《艺术论坛》 (Artforum)、《艺术碎片》与 NOWNESS 等媒体撰稿。近期策划的展览和参与的活动包括:“AI:爱与人工智能“,现代汽车文化中心,北京(2020);“哥白尼”,E.M.Bannister 画廊,罗德岛学院,普罗维登斯,美国 (2019);“液态健康“,上海歌德开放空间,上海(2019);“上海策展人实验室”项目,上海大学美术学院,上海(2018);”PSA青策计划2018“,上海当代艺术博物馆,上海(2018);“三亚华宇艺术论坛”,三亚,海南(2016-2017)等。其文章“亲特网内外的后网络艺术”收录于中国美术学院于2018年出版的论文集《网络化的力量》中。参与编写的 Shanghai Contemporary Art Archival Project 1998 – 2012 由 MOUSSE 于2017年出版并发行。



在下方填入你的資料或按右方圖示以社群網站登入: 標誌

您的留言將使用 帳號。 登出 /  變更 )


您的留言將使用 Facebook 帳號。 登出 /  變更 )

連結到 %s