A Lithium brand
A more neutral topic curation algorithm
SarahE
6
Kudos
105
Comments
May 16, 2016

 

Monday, Gizmodo published an article about the curation practices behind Facebook’s Trending module, and to what extent a curator’s personal biases affect what’s shown to Facebook’s billion-plus active users. (As you can imagine, this has caused some controversy.) Although Gizmodo focused on the role of human curators, those of us who work with algorithms and machine learning have had to confront the fact that biases can end up deeply encoded into supposedly objective systems, including Klout’s own topic classifier -- the system that identifies your Expert topics, as well as the relevant articles for the Explore tab. Here’s a quick roundup of how we in Klout’s Data Science team think about keeping the topic system as unbiased as possible.

 

Klout’s topic classifier in 30 seconds or less

The major inputs to our topic system are:

  1. The classifications we’re applying:
    1. An ontology of nearly 10,000 human-curated topics
    2. An underlying dictionary of over a million named entities and concepts
  2. The data being classified, which includes:
    1. Social media profiles (to be analyzed for topic expertise and interest)
    2. URLs published on social media and elsewhere (to be analyzed for topical content and served in the Explore tab)

 

Even without getting into the weeds of the pipeline that brings those two types of input together -- more on that below -- critical readers will already be able to spot a few areas where we’re vulnerable to bias. Let’s walk through them.

 

Any classification system contains value choices

Should “Autism” be placed under “Diseases” or “Neurology”? Is “the Tea Party movement” distinct from “conservative politics”? Is “Wizard Rock” really a thing? How we answer these questions shapes the experience of our users, and inevitably gives our ontology a point of view.

 

Longtime users may remember the early days of Klout, when topics were a, shall we say, messy combination of user-submitted tags and data-mined concepts. As the Data Science team worked on regularizing and improving the ontology, we’ve relied on the following principles:

 

  1. The ontology is a living thing; we should always have tools in place for updates
  2. The ontology should always have up-to-date guidelines defining:
    1. Scope -- what portion of the world we’re describing. (In our case, as much of it as possible.)
    2. Granularity -- at what level of detail we’re describing it. (Do we need to include every actor? Television show? Heavy metal subgenre?)
    3. Voice -- the tone we use in describing it. (Do we use scientific names? Full legal names for persons? Slang?)
  3. A little redundancy won’t hurt  -- our system can support topics with some conceptual overlap, so err on the side of inclusion. (For example, both “Gun Rights” and “Gun Control” are topics in the Klout ontology.)
  4. Users are the best source of feedback -- our users have a broader range of perspectives than we do; make it easy for them to alert us to problems

 

Even so, any time your application or audience changes, it’s important to reassess your classification scheme. One major flaw in Klout’s topic ontology is that it was developed for a U.S. audience, and still needs significant work for other countries and languages.

 

Staying alert for sins of omission

In addition to the human-curated topics in the ontology, we also use a dictionary of concepts and entities derived from Freebase. Freebase is a widely-used resource in the data science world, but “widely-used” is not the same thing as “perfect”, by any means. The biggest issue with Freebase is what it leaves out; like Wikipedia, it was collectively sourced, and like Wikipedia, it’s biased toward the interests of its editors, and sparse in some areas like cosmetic products and fashion terms, requiring us to develop ways to supplement the dictionary. The moral of the story: it pays to look critically at any pre-packaged data set you plan to use.

 

Boosting inclusivity

Next, let’s consider the URLs we collect for the Explore tab. The majority are URLs that have been shared on social media, which means they are dominated by the topics most discussed on social media: politics, celebrity news, music, etc. What we sometimes call “niche topics”, like molecular biology, or Wicca, or wheelchairs, naturally are present in fewer URLs. Does that count as a bias? It’s unclear, but it’s not a good end-user experience and risks making some users feel marginalized. As a result, we’ve had to develop backup strategies to increase coverage for less common topics.

 

The fuzzy line between human bias and business logic

One of the more ironic tidbits in Gizmodo’s article was that Facebook’s curators were told to suppress news about Facebook -- that is, to interfere with the Trending algorithm to avoid the appearance that Facebook was interfering with the Trending algorithm. But that kind of decision is probably familiar to the product managers in the audience, whose goal it is to preserve the user experience. Similarly, a discovery feed like our Explore tab might recommend porn, or spam, or hate speech, and need to be tuned or overridden. To make it even more complicated, the definition of porn, or spam, or hate speech may change from region to region. Keeping those decisions from being made inconsistently or thoughtlessly is really difficult, but our approach has been to define a single owner who both documents the rules and is accessible to discuss individual cases. As others have pointed out, Facebook’s mistake may not have been having curatorial tools, but isolating the employees using them.

 

Fine, but what about the actual topic algorithm?

Eagle-eyed readers will have noticed that we haven’t touched on the nuts and bolts of how Klout’s topic system actually assigns topics. The challenges of data modeling and debugging machine learning algorithms are pretty well surveyed elsewhere, and how we handle those challenges at Klout would require a dedicated blog post. However, there’s less discussion of how to handle human biases when collecting training or validation data -- how people’s points of view get encoded into the data a given algorithm is trying to approximate. The two approaches often recommended could be described as micromanaging versus crowdsourcing; either a) have an in-house process that includes well-defined guidelines, trained judges, and a reconciliation process for disagreements, or b) have lightweight guidelines but a large number of judges, in the hopes that individual biases will be muted. There are tradeoffs to either approach; our team has recently been relying mostly on in-house validation data, mostly because it’s friendly to our development schedule. But what’s more important, in our experience, is that the potential weaknesses of the training/validation data are known and discussed and documented ahead of time, so that they can be distinguished from problems with the model itself.


No system is perfect, and keeping out bias takes continual work. Although a focus on documentation, consistency, and validation will take you a long way, the very best defense against unintentional bias is a diverse team, who can bring multiple points of view. Want to come work with us?

 

 


IMG_750.jpgSarah Ellinger is the Lead Data Analyst for Klout/Lithium’s Data Science team. She is responsible for overseeing the content of the topic ontology, as well as monitoring the performance of the topic classification system. Sarah attended U.C. Berkeley’s School of Information and has over a decade of experience in taxonomy and web content classification at tech companies large and small. She can be found on Twitter discussing information science and Game of Thrones spoilers @sarahellinger.

105 Comments
jamesowensjo91
Not applicable

Still, the fact that we as Facebook users ever wanted neutrality speaks to a belief in digital democracy. That is the contrast that Facebook have themselves set up: They deliberately positioned themselves as a distribution network, which is explicitly not an editorial entity. Facebook is intended to be the home of what the world is talking about. Their business model depends on it, even if that’s an impossible goal.

 Source: https://newrepublic.com/article/133472/life-age-algorithms

 

James Owens - Online Assignment Help

infolapak
Not applicable

Daftar situs judi online terpercaya di Indonesia lengkap dengan tips-tips menarik dalam lapak judi online.
infolapakjudi memberikan informasi kumpulan daftar nama situs judi online resmi terbaik di tahun 2018 yang dapat anda jadikan pilihan sebagai tempat untuk bermain judi online Indonesia karena kami hanya menyajikan situs-situs terbaik saja dari sekian ribuan situs judi online yang tersedia untuk Indonesia.

infochip
Not applicable

Daftar situs online, info freebet, free bets dan menyediakan tips-tips menarik dalam judi online

PaulJackman
Not applicable

Sometimes it gets so difficult for the students to complete their essay writing task within the assigned time, therefore, many students prefer to hire do my essay uk services for their writing task.

happyvalentines
Not applicable

<a href="https://happyvalentinesdayl.com/i-love-you-images/">I Love You Images, GIF, Wallpapers, Pics &amp; Photos for Whatsapp DP &amp; Profile</a>

prsdaja
Not applicable

wow its one of the best post on your site.All post are good but this is one of the best website.

g9g 

motivational story in hindi

bestofkodiaddon
Not applicable

okay but you must check the video.me/pair kodi error sometimes

StevenKeller
Not applicable

I know how much you guys are missing to earn Payoneer bonus but the thing is that it helps you earn Payoneer Bonus right today! You can get signed up & then register for an account in 2019.

StevenKeller
Not applicable

I'd also recommend you to check out this profile here.

StevenKeller
Not applicable

Payoneer can be very helpful. You can Earn Free Sign UP Bonus when you'll register with Payoneer for the first time. Payoneer offers up to $25 Sign UP Bonus.

zoyaali
Not applicable
acepsaepul23
Not applicable

I likely appreciating every single of this. is actually an incredible and decent impart. I need to much obliged. Great employment! You all an incredible blog site, and also have some extraordinary substance. Cara Menyembuhkan Psoriasis Secara Alami Dan Ampuh

mshafiq11
Not applicable

.Really looking forward to read more. Great. Oppo

evelyn11
Not applicable

Getting Assignment Help online has been simplified with our services at 24X7assignmenthelp.com. Not only do we value all of our clients but our team is here 24 hours a day to tend to each student. Our professionals provide interesting insights and top notch information to learners ensuring them an edge over other amateur classmates. Impressing teachers is simple with our help.

John32
Not applicable

Stuck with your essay and can’t move on? Try out our free essay typer which will help your overcome the writer’s block.

SamPaul
Not applicable

An essay writing help functions as a guardian angel for students proceeding with their degrees at higher level educational establishments. These services are built with the aim of simplifying the complex life of college students. They offer students with various facilities including writing, editing and proofreading of their essay.

UK essay writing as offered by Write my essay for me uk comprises of expert writers, who have had years of experience in a multitude of fields. By working in the professional environment, the employed writers have the discipline and the formality that is required by the professors at higher level educational institutions. Following from this, the experts hired by them, provide students with an opportunity to submit essays written with a fresh perspective while being enriched with detail and crucial information. Additionally, as the organization is created with the mere purpose of writing high-quality essays, they have hired a team of researchers who can scan multiple sources for relevant and credible literature that can be assimilated into the student’s essay. Furthermore, their research teams work alongside the writers to provide a fresh and authentic take on the student’s essay. Lastly, the editing department offered by these services scan the overall essay for any form of error that may interrupt the essay’s professionalism.

alicetaylor
Not applicable

Very helpful advice in this particular post! It’s the little changes that make the largest changes. Thanks for sharing!

Geometry Dash

hassanbajwa
Not applicable

The MNSBC network also provides content from news websites such as NBCnews.com and Today.com, with live streaming options for those who wish to watch MSNBC News while on the go. We bring MSNBC, you round-the-clock streaming live in HD quality. There are various shows to be enjoyed on News TV Channel Live Stream Online such as your business with JJ Ramberg,  for those who follow politics can watch Politics Nation with reverend Al Sharpton and weekends with Alex Wit. MNSBC boasts of talented broadcasters such as Stephanie Ruhle and Andrea Mitchell just to name a few. Watch the latest MSNBC News and videos and be a part of this innovative community.  Lean Forward with News TV Channel Live Stream Online. Stream may take some time to load. Be Patient!.

mshafiq11
Not applicable

I was suggested this web site by my cousin. I am not sure
whether this post is written by him as nobody else know such detailed about my problem.
You are wonderful! Thanks!

priny15
Not applicable

Thank you for posting this article, it was really helpfull. Great Article.

voir film gratuit

guestpostbox
Not applicable

I really like it whenever people get together and share thoughts. Great blog, stick with it!

Thanks and Regards, 

tweakbox app for android

vmblog
Not applicable

Very nice article... keep posting such articles, it helps us alot...

Motivational stories in Hindi

hindi story

Moral stories in hindi

futuretricks
Not applicable

Hey,

 

Thanks for sharing this helpful & wonderful post. i really appreciate your hard work. this is very useful & informative for me.

 

thanks for sharing with us. thanks a lot.

 

Regards

FutureTricks

alktub
Not applicable

شركة رواد الحرمين تقدم افضل شركة عزل اسطح عملائنا الكرام كثيراً من المنازل تحتاج إلى العزل لكي يتم الحفاظ على المنزل وسلامته وبالتالي الحفاظ على سلامة
المواطنين الذين يعيشون به، فنجد أن كثيراً من المنازل تتعرض إلى الأمطار الغزيرة في فصل الشتاء وتتراكم المياه لفترات طويلة
على الأسطح ومع مرور الوقت تتفاعل المياه مع مسام الأسقف وتتسرب المياه إلى داخل المنازل وتسبب الكثير من المتاعب والمشكلات التي من أخطرها تفاعل المياه
مع خرسان المنزل وتبدأ الأعمدة في التصدع والصدأ وتظهر التشققات على الحوائط والجدران وانخلاع الأرضيات وسقوط الطلاء ومع مرور الزمن يصبح المنزل آيل للسقوط،
ويتأثر المنزل في فصل الصيف أيضاً عندما يتعرض لفترات طويلة من الوقت لأشعة الشمس فيتسرب هذه الحرارة إلى الداخل وتأثر على مناخ المنزل والحرارة العالية
أيضاً تسبب في سقوط الطلاء وظهور التشققات، ولكن من الآن لا داعي
للقلق لأن شركة عزل أسطح تقدم لعملائها الكرام في كافة أنحاء المملكة العربية السعودية .
افضل شركة عزل أسطح

<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d9%81%d9%88%d9%85-%d8%a8%d8%a7%d..."> شركه عزل فوم بالدمام </a>


<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d9%81%d9%88%d9%85-%d8%a8%d8%ac%d..."> شركه عزل فوم بجدة </a>


<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d9%81%d9%88%d9%85-%d8%a8%d9%85%d..."> شركه عزل فوم بمكة </a>


<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d9%81%d9%88%d9%85-%d8%a8%d8%a7%d..."> شركه عزل فوم بالرياض </a>

 

<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d9%81%d9%88%d9%85-%d8%a8%d8%a7%d..."> شركه عزل فوم بالقطيف </a>

<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d9%81%d9%88%d9%85-%d8%a8%d8%a7%d..."> شركه عزل فوم بالاحساء </a>


<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d9%81%d9%88%d9%85-%d8%a8%d8%a7%d..."> شركه عزل فوم بالجبيل </a>


<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d9%81%d9%88%d9%85-%d8%a8%d8%a7%d..."> شركه عزل فوم بالطايف </a>


<a href="https://el-harmeen.com/%d8%b4%d8%b1%d9%83%d8%a9-%d8%b9%d8%b2%d9%84-%d8%a7%d8%b3%d8%b7%d8%ad-%d8%a8%d..."> شركه عزل فوم بالقصيم</a>
<a href="https://www.samaclean.com/2017/08/detection-water-leaks-ehsa.html"> شركة كشف تسرب المياه بالاحساء </a>

mshafiq11
Not applicable
I had a great time stumbling on your site! I'm sure others will appreciate it, too!
KatherineKitty
Not applicable

Pretty! This was an extremely wonderful post. Thank you for providing these details.

essay writing services UK
JamesBaker
Not applicable
Gonna write an analytical essay? navigate here for a detailed guide!
trndsttr
Not applicable

Seseorang yg rentan terkena penyakit ini ialah orang yg berumur 30 th lebih akan rentan terkena diabetes tipe 2,yg mana diabetes tipe 2 il ini muncul karena adanya penyebab pola hidup yg tidak sehat dan teratur dan hampir tidak pernah melakukan olahraga meski get begitu diabtes tipe

rishiu12
Not applicable

Hey Guys now you can download latest Tamil HD movies from tamilyogi and get free and in good quilty and tamil isaimini

JamesBaker
Not applicable
Go right here https://eduessayhelper.org/blog/book-review to know more about book review and how it should be written!
harshak99
Not applicable

AppValley is an installer of the mobile apps. You can download the app on any platform like iOS, Windows, and Android. 

https://appvalley.co.in/

appvalley 

app valley vip 

app valley download 

 

tactionsoftware
Not applicable

Hey you are interested in top technogies 

 

To learn SEO -iShailesh

To Learn trend technologies and technique -  PM full Form

To Hire - Top Developers for SuiteCRM

To Get - Best Zend framework Development company

To Download - SuiteCRM Theme for Free

rishiu12
Not applicable

hello friend download tamilrockers 2018 movies download

Ronn91
Not applicable

I love what you guys tend to be up too. This type of clever work and reporting!

Keep up the amazing works guys I’ve included you guys to our blogroll. light novel

 

mshafiq11
Not applicable

Great post, I really interesting the way you highlighted some important points.,.

Davidd
Not applicable

I really admire your efforts. These are really beautiful. I have totally agreed with you. Thanks for providing such information. Just as in writing resonance structures, really appreciated the knowledge you shared with others, the content is lit, looking for some more informative content to come, keep up the good work, keep spreading knowledge, thank you! 

Blue Jacket

arianapham
Not applicable

Thanks for sharing this awesome content here.

fnaf world

daisydaisy
Not applicable

The information you have posted is very useful. The sites you have referred was good. Thanks for sharing. shell shockers

Nveenkumar07
Not applicable

OK, this is very informative article about neurology. thanks. Check UDID card download kaise kare

kate_207
Not applicable

Nice & Helpful Blog post! If Facing Any problem related to printer visit us Connect Brother HL-2280DW Printer to wifi

PeterClarke321
Not applicable

Get your coursework written myassingmenthelp.com . we provide coursework by our 8 years+ expericnce experts who have well knowledge how to write you course. Now do my coursework can be done by myassignmenthelp.com

lindarose11
Not applicable

It is a great article. You will surely like this also because it is a great stuff

pacman

gyanigurujii
Not applicable


Quite Impressive article loved it. Keep up the good work
Visit here for downloading HD movies

https://gyaniguruji.in/tamilrockers-latest-link-and-url
https://gyaniguruji.in/worldfree4u-hd-movies-download-2019

wastanven
Not applicable

If you are facing any webmail technical issues and can’t be understand there solutions, then don’t worry we are here with NRTC Webmail Support Number which is you direct connected with our experts and get immediate resolutions by dialing our toll-free number 1833-284-2444 it’s available for 24*7.

WilliamRiley
Not applicable

This blog is such an unique kind of blog regarding A more neutral topic curation algorithm. I am thankful an author for sharing this amazing blog with us. William Riley, http://www.eliteassignment.co.uk/nursing/

timnolin
Not applicable

We have a team of tough accountant who can deliver tax preparation service Garland at the end of the year.

jennyaly1
Not applicable

Thank you for sharing this! Just what I’ve been searching for. Great info! We are the best Cleaning Company in Dubai, YallaCleaning fills in as the solid, trusted and disentangled cleaning center for the Middle East cleaning industry.

admin88
Not applicable

The article is very easy to understand, detailed and meticulous! I had a lot of harvest after watching this article from you! I find it interesting, your article gave me a new perspective! I have read many other articles on the same topic, but your article convinced me! geometry dash

assakolu
Not applicable
as the aspect said with the great ideas of the gamers with the live authorimzation confirmation https://techgfi.com/thevideo-me-pair-and-vidup-me-pair-error-fix/
Vika1
Not applicable
When you research the data for your essay, you get tons of information that’s hard to remember. This action is very time consuming. So I’ve ordered coursework at http://essaylab.co.uk and now I have a lot of free time.