Skip to content
The methodOpen data
Back to the collection

Monolith: Real Time Recommendation System With Collisionless Embedding Table

Zhuoran Liu, Leqi Zou, Xuan Zou

Show all 11 authors

Caihua Wang, Biao Zhang, Da Tang, Bolin Zhu, Yijie Zhu, Peng Wu, Ke Wang, Youlong Cheng

2022 · arXiv

Machine learningDistributed systemsAlgorithms
Read the paper
HN score
0
Mentions
6
Readers
6
Months
6
What’s in a score?

From the abstract

Building a scalable and real-time recommendation system is vital for many businesses driven by time-sensitive customer feedback, such as short-videos ranking or online ads. Despite the ubiquitous adoption of production-scale deep learning frameworks like TensorFlow or PyTorch, these general-purpose frameworks fall short of business demands in recommendation scenarios for various reasons: on one hand, tweaking systems based on static parameters and dense computations for recommendation with dynamic and sparse features is detrimental to model quality; on the other hand, such frameworks are designed with batch-training stage and serving stage completely separated, preventing the model from interacting with customer feedback in real-time. These issues led us to reexamine traditional approaches and explore radically different design choices. In this paper, we present Monolith, a system tailored for online training. Our design has been driven by observations of our application workloads and production environment that reflects a marked departure from other recommendations systems. Our contributions are manifold: first, we crafted a collisionless embedding table with optimizations such as expirable embeddings and frequency filtering to reduce its memory footprint; second, we provide an production-ready online training architecture with high fault-tolerance; finally, we proved that system reliability could be traded-off for real-time learning. Monolith has successfully landed in the BytePlus Recommend product.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

6 selected excerpts · 6 total mentions

jamesblonde
Neutral mention · rule
I gave a talk at PyData Berlin on how to build your own TikTok recommendation algorithm. The TikTok personalized recommendation engine is the world's most valuable AI. It's TikTok's differentiation. It updates recommendations within 1 second of you clicking - at human perceivable latency. If your AI recommender has poor feature freshness, it will be perceived as slow, not intelligent - no matter how good the recommendations are. TikTok's recommender is partly built on European Technology (Apache Flink for real-time feature computation), along with Kafka, and distributed model training infrastructure. The Monolith paper is misleading that the 'online training' is key. It…
Read the full comment
is not. It is that your clicks are made available as features for predicitons in less than 1 second. You need a per-event stream processing architecture for this (like Flink - Feldera would be my modern choice as an incremental streaming engine). * https://www.youtube.com/watch?v=skZ1HcF7AsM * Monolith paper - https://arxiv.org/pdf/2209.07663
Original comment TikTok's 'Addictive Design' Found to Be Illegal in Europe
supo
Neutral mention · rule
If this was a solved problem, every e-com website would already run some variant of https://arxiv.org/pdf/2209.07663 for all their shopping surfaces and some version of "deep research" on every search query. They don't, so the problem isn't solved. And a big part of the problem is still the integration of behavioral, structured and unstructured signals into the underlying retrieval of products and content - which is what the article is about. Also, the company is doing fine, not sure where that last sentence came from ^_^
Original comment You Don't Need Re-Ranking: Understanding the Superlinked Vector Layer
Grimblewald
Neutral mention · rule
the tiktok algo is genuinely impressive. What's cool is that the engineers published some works explaining how it functions. https://arxiv.org/abs/2209.07663 It's an interesting read if you're into recommender systems or AI in general. What amazes me is that despite this published work google and meta still can't produce a decent social media algorithm, so it's either incompetence or malice.
Original comment TikTok preparing for U.S. shut-off on Sunday
wyldberry
Neutral mention · rule
At best this comment implies there's actual an actual smoking gun related to what functionally are nation-state capability and operations to influence populations external to their own, specifically related to TikTok/ByteDance. If you have an AP government level understanding of how China and the USA work, then you can probably understand that this is an intersection of: government laws, great power competition, and wealth seeking behaviors. When you control the algorithm, everyone downstream of it is at the mercy of the power structures and incentives of your top level leadership [0]. Consider the following: 1.) How Chinese companies interact with the CCP: they are 100%…
Read the full comment
subservient with a CCP party member who can pull the CEO in at anytime and assign them tasks related to studying Xi thought and other party material, as well as demand the company do anything for the state. 2.) What great power competition looks like including economic warfare (tariffs, massive state subsidies to protect national defense, influence operations for allies, securing raw materials, reducing reliance on rivals, etc). 3.) How espionage in general works in support of political aims, and especially how each nation uses it to further their goals. For China, it's always been about vacuuming up as much data as possible, and then later on finding use cases for it. The disruption side of their practices are successful if they make chaos in an adversary that forces them to spend time on dealing with that instead of confronting them. Each nation acts in their own interest and are absolutely ruthless here in a way the average person rarely can comprehend. 4.) The free pass given to China related to tech and social media industries. American Social Media is banned in China[1]. Why do you think that is? There are many reports/cases of American intel/military use cases of social media to influence outcomes worldwide[2]. China wasn't going to allow this. But we are supposed to allow them access to us? We know this is the case because we would do the same in their position[3]. There's a reason it's banned in India[4]. 5.) We know China influences the Douyin algorithm[5] locally to promote behaviors the government perceives as healthy or important to state interests. This often manifests in pushing more STEM related content and suppressing the ragebait content that flourishes in other nations[6]. Even if the leadership of Bytedance were 100% on board with not pushing CCP interests to the USA and world, there's no mechanism of protection or redress from them. The CCP can, and will, disappear high profile people at will with no repercussion, and replace you with someone who will listen. Any smart nation who wants to protect the minds of their citizens would do well to extremely regulate social media in their country. In your case for "Americans upset over Israeli treatment of civilians". Assuming that the American outrage is 100% organic on it's own: It is in the best interest of China to amplify that outrage and make it seem stronger than it is for a variety of reasons: A. This will have knock on effects for policy makers and their aides who are hyper-tuned in because if their constituents want something, voting for that is how B. Anything that disrupts American foreign policy and allies is an asset at weakening the grip the dollar has on international markets C. Israel is an extremely talented producer of technical people. If the relationship between the USA and Israel sours, then suddenly a pathway opens for China to get access to the incredible tech that Israel develops for various US programs. (This is also ignoring their incredibly advanced spyware capabilities in their private sector, which is morally repugnant, but it does exist). [0] - https://arxiv.org/pdf/2209.07663 [1] - https://en.wikipedia.org/wiki/List_of_websites_blocked_in_ma... [2] - https://web.archive.org/web/20160410083943/http://www.reddit... (look at the most addicted cities) [3] - https://theintercept.com/2022/12/20/twitter-dod-us-military-... [4] - https://apnews.com/article/tiktok-bytedance-ban-china-india-... [5] - https://www.youtube.com/watch?v=0j0xzuh-6rY [6] - https://networkcontagion.us/reports/the-ccps-digital-charm-o... [7] - https://www.theguardian.com/world/2023/jul/25/the-growing-li...
Original comment TikTok divestment law upheld by federal appeals court
jerojero
Neutral mention · rule
If you think about it, the secret sauce tends to be in the data they're able to capture and input into the recommendation algorithms rather than the recommendation algorithm itself. I mean, even Tiktok has papers detailing their recommendation algorithm [1]. So does Youtube and plenty of others. It's all in the data now a days. [1] https://arxiv.org/pdf/2209.07663.pdf
Original comment X is changing its algorithm to highlight smaller accounts
CharlieSweeting
Neutral mention · rule
Arvind's making the case that the algorithm isn't particularly special and it's all about the system design which hooks users. It's definitely true that a huge part of their user retention isn't some magical algorithm, but they have made some genuine algorithmic/training improvements which are interesting. They published their methodology a few months ago (https://arxiv.org/abs/2209.07663), which wasn't referenced in the article (only an official blog post and a few reverse engineering attempts that I don't think are enough to conclude that their recommender is just run of the mill).
Original comment TikTok’s Secret Sauce

Keep following the thread.

More in Machine learning

More from the same topics.