RSS Feeds

State department informs Congress of planned closures in Canada, Grenada, Japan, Indonesia and Cameroon

The US state department is to shut five foreign consulates and missions across Asia, Africa, North America and the Caribbean, drawing criticism that it is leaving a diplomatic vacuum at risk of being filled by China.

The US Congress has been served notice of the impending closures, which are being justified on cost grounds and follow a reassessment of the US’s bilateral relationships with a range of countries since Donald Trump returned to the White House in January 2025.

Sociology professor to stand down after university announces investigation into his qualifications and career

• Playbooks, plagiarism and a pig’s head: the claims surrounding a star Cambridge professor

Jason Arday, the Cambridge professor accused of plagiarism and embellishment, has resigned after the university announced he was to face new investigations into his qualifications and career.

Arday, who had been dogged by a growing series of questions over his academic work and credentials, issued a statement on Wednesday night, saying the “relentless accusations, speculation and public commentary have taken a profound toll on me and on those I love”.

State department informs Congress of planned closures in Canada, Grenada, Japan, Indonesia and Cameroon

The US state department is to shut five foreign consulates and missions across Asia, Africa, North America and the Caribbean, drawing criticism that it is leaving a diplomatic vacuum at risk of being filled by China.

The US Congress has been served notice of the impending closures, which are being justified on cost grounds and follow a reassessment of the US’s bilateral relationships with a range of countries since Donald Trump returned to the White House in January 2025.

Every day, Meta’s recommendation platforms handle billions of user interactions, generating rich temporal signals that capture individual preferences and intent across products, ads, and content. In our 2024 post on sequence learning for ads recommendations, we showed how modeling the order and timing of user actions (rather than relying on static, manually engineered sparse features) produces richer, sequence-aware representations of user interests and ad preferences. 

This post goes a step further, introducing two architectural breakthroughs that let us scale sequence learning advancements from foundational innovations into a production platform with predictable, LLM-style scaling laws: (1) a multi-stage sequence model that decouples heavy offline user modeling from lightweight online ranking tasks and (2) a learning technique based on dense tokenization and target-aware attention that efficiently learns feature interactions directly from data. 

Together with our broader model innovations, these advancements have contributed to a cumulative lift of 6% in conversions on Instagram, 3% in conversions on Facebook and 3.5% in ad clicks on Facebook. This unified platform for sequence modeling is a core component of Meta’s Generative Ads Recommendation Model (GEM), helps to harness the comprehensive user behavioral understanding of this learning paradigm to maximize the benefit to advertisers. 

The Historical Challenges of Sequence Modeling 

Ads recommendation systems must retrieve and rank thousands of ads within milliseconds, processing millions of candidates per second. To manage this scale, some approaches to sequence models rely on hybrid model configurations where a specific model processes user event sequences and another model handles sparse feature interactions. 

While effective at meeting production demands, this hybrid approach has potential tradeoffs: 

• Lossy knowledge transfer between components
• Continued reliance on manual feature engineering
• Scaling ceilings from interference between ranking and sequence model components

Scaling both temporal sequence lengths and the transformer models that process them can turn the tradeoffs of the hybrid approach into a bottleneck, limiting the ability to improve the ads experience of users and the performance of advertisers’ campaigns. 

We’ve made two fundamental architectural breakthroughs in sequence learning that resolve the core tension between model complexity and serving efficiency: (1) a multi-stage sequence model that decouples offline user modeling from online ranking and (2) a dense tokenization with target-aware attention learning paradigm. Together, they provide a flexible production strategy that helps generalize sequence learning models and establish an LLM-style scaling law that predictably balances model performance with compute. 

Introducing the Multi-Stage Sequence Model

To address scaling efficiency, a multi-stage model has been developed that enables scaling of a transformer-based sequence model in a compute efficient manner. Separating the sequence model into two complementary stages (upstream/offline user modeling and downstream/online ranking), enables model capacity to scale so that performance keeps improving without proportional increases in serving resources. 

In Figure 1, the left panel shows the offline user model. It processes long user histories asynchronously and produces cached embeddings that capture deep behavioral patterns. The right panel shows the online ranking model that combines these cached representations with real time ad candidate signals to produce the final ranking. The arrow between the two stages carries the user feature embeddings from offline → online ranking models.

Two Key Stages of the Model

First Stage: Offline User Model

User-side features are processed asynchronously using deep transformer upstream models. These models scale to several transformer layers with sequence lengths in the thousands and generate embeddings that are precomputed and cached at the user level. The upstream model strictly separates user features from ad and context features to ensure user embeddings remain independent of any particular ad candidate.

Second Stage: Online Ranking Model

The offline user model representations are complemented with online ranking models that use fresh user signals and ad candidate information for real time ranking. This stage is optimized for speed, meeting strict latency budgets while leveraging the deep representations computed offline. 

Separating the sequence modeling system into two distinct, yet complementary, stages enables an increase in model complexity along a scaling curve for the Offline User Model without causing a spike in serving costs for the Online Ranking Models. 

Sequence Model Architecture Innovations

Dense Tokenization

This tokenization approach integrates sparse features with sequential behavioral data into a single dense vocabulary, enabling attention mechanisms to discover interactions independently. Unlike traditional recommendation systems, which rely on manually engineered representations to capture sparse cross-feature interactions, this approach lets the model learn those interactions directly from the data. 

Target-Aware Multi-Head Attention

Tokenized sparse features and ad candidate information are fused with user behavior sequences, then processed by a memory-efficient form of multi-head attention that lets each layer weigh a user’s past behaviors against the specific ad being scored. Stacking multiple aligned attention blocks with stable attention distributions allows each layer to capture higher-order interactions between the target ad and the user’s historical behavior, progressively distilling long sequences into compact representations.

A Predictable Scaling Curve 

LLM-Style Scaling Law

When running on real-world ads traffic, the multi-stage sequence model demonstrates the emergence of predictable scaling laws for ads recommendations that are analogous to those observed in large language models. Performance improvements follow a log-linear relationship with respect to compute, with a marked improvement in scaling efficiency over other transformer-based sequence models. Figure 2 conveys these scaling properties by showing the relationship between compute (FLOPs) and model performance (measured by normalized entropy, NE) across several dimensions: model depth, content/semantic enrichment, model width, and sequence length.

Levers for Scaling

Unlike LLMs, which process dense and continuous text, ads recommendation systems must integrate sparse ID features with temporal user sequences. The fact that LLM-style scaling emerged despite the structural differences provides a strong indicator of model architectural fit for further sequence learning applications. 

We have identified four levers that we anticipate will help unlock the frontier of the scaling law:

1. Balanced Model Shape

Optimal performance requires balanced growth across model depth, width and sequence lengths. If scaling only occurs on a single axis, the other axes will likely bottleneck the performance improvements, potentially leading to diminishing returns. This mirrors findings from LLM scaling law research, a principle we call the scaling synergy principle. 

2. Multi-Stage Tunability

The multi-stage architecture provides a tunable lever to scale either the offline or online model up/down. Scaling the online ranking model drives steeper improvements per unit of compute that is bounded by serving/request time requirements. Scaling the offline model (shown in Figure 2) follows a more gradual curve, but its async inference avoids latency constraints, allowing scale in at an unhindered rate.  

3. Sequence Composition

Performance continues to improve as sequences get longer, but an impactful finding is that sequence diversity beats sequence homogeneity.  A balanced mix of action types (e.g., views, clicks, conversions) yields better results than sequences composed of a single action type. This finding suggests that a diverse mix of engagement types and broad temporal coverage produce richer behavioral representations of users than homogeneous sequences of high signal actions in isolation.

4. Semantic Feature Representation

Semantic content features from foundation models complement traditional collaborative filtering (i.e. which users interacted with which items) signals. They are especially helpful in cold-start scenarios (e.g., new ads or advertisers with limited historical engagement data). By addressing this persistent challenge of recommendation systems, we improve overall signal coverage to a fundamental sparse problem in recommendation systems.

The Impact of Multi-Stage Sequence Modeling

The multi-stage sequence modeling architecture is delivering impact across three dimensions: 

Deeper User Representation

By modeling thousands of user event sequences (e.g., clicks, views, and purchases) the offline model generates highly nuanced user representations. This depth of behavioral understanding improves ad relevance and conversion rates across Meta’s Family of Apps. Together with our broader modeling innovations, these sequence-derived representations drove a cumulative lift of 6% in conversions on Instagram, 3% on Facebook and 3.5% in ad clicks on Facebook.

Scaling Efficiency

The two-stage design delivers performance improvements with greater compute efficiency compared to hybrid approaches. Initial evaluations improved ads ranking quality with minimal impact to serving resources, confirming that model complexity and production efficiency can scale together. 

Platform Integration

As a core part of GEM, this model architecture for sequence learning has been designed for generalization, where the same multi-stage backbone and scaling properties can extend to any ads ranking task with minimal adaptation and overhead. 

Current Work: Continued Scaling

The sequence model scaling law shows no signs of saturation. With architectural parity achieved, scaling model complexity can draw on techniques proven in the LLM domain (e.g., mixture-of-experts, cross-user compute sharing, advanced attention mechanisms) with potential to continually scale at the optimal performance/efficiency tradeoff. 

Read the Paper

A detailed technical publication of this model architecture and its scaling properties is available in our paper, “LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation.” 

The post From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta’s Ads Ranking appeared first on Engineering at Meta.

Joint statement with Oman is being finalised, but Tehran says deal alone would not guarantee waterway’s security

Iran and Oman have reached an understanding on ⁠the geographic coordinates for a shipping route through the strait of Hormuz, and ⁠a joint announcement ⁠is ​being finalised provided certain third parties do not interfere, Iran’s foreign ministry spokesperson Esmail ⁠Baghaei said on Wednesday.

Baghaei added that any such agreement between Iran and Oman would not, by ⁠itself, guarantee security in the strategic waterway.

Как превратить записи встреч в рабочий материал для согласования требований: текущая версия проекта AudioToText. В статье показываю, как собрать сервис транскрибации на базе MLX Whisper и Django, чтобы автоматически получать структурированные протоколы созвонов, а не просто длинные текстовые файлы. Разбираю сценарий работы, ключевые элементы веб‑интерфейса, review‑контур и планы развития проекта в сторону NER и поиска противоречий по summary.

Joint statement with Oman is being finalised, but Tehran says deal alone would not guarantee waterway’s security

Iran and Oman have reached an understanding on ⁠the geographic coordinates for a shipping route through the strait of Hormuz, and ⁠a joint announcement ⁠is ​being finalised provided certain third parties do not interfere, Iran’s foreign ministry spokesperson Esmail ⁠Baghaei said on Wednesday.

Baghaei added that any such agreement between Iran and Oman would not, by ⁠itself, guarantee security in the strategic waterway.

Идея: понедельник, 10:00. Что если из заметок в obsidian собирать (пересобирать) готовые презентации образовательных курсов?

Obama-appointed judge seemed unmoved by Trump attorneys’ arguments in case also naming Murdoch

A federal judge in Florida seemed skeptical of arguments from Donald Trump’s legal team on Wednesday over whether the court should allow a $10bn defamation lawsuit against the Wall Street Journal to move forward.

Trump sued the Journal and the News Corp founder Rupert Murdoch over a July 2025 article about a “bawdy” letter reportedly from Trump that was included in a 2003 birthday book for disgraced financier Jeffrey Epstein. Trump has denied authoring or signing the letter, and his attorneys have argued that the Journal published the story despite having been told it was false.

Началось всё беззаботно. Я участвовала в локальном писательском конкурсе, где рассказ надо писать по выпавшему жанру, и мне выпала антиутопия, а я выпала в осадочек.

Что я знала про антиутопию? Ну, читала «1984» когда-то давно, ещё в той жизни, где у меня было время читать что-то кроме логов. Помнила, что это когда мир вокруг тебя хреновый, но старательно притворяется хорошим, а главный герой, сколько бы ни бился, всё равно проигрывает. Плюс бонусы: кодовые имена и лозунги на плакатах. Негусто, короче.

И тут мне в голову пришла идея, (старая как мир): злой мир, где людей вместо казни или после смерти просто копируют. Не убивают, а «продолжают». Загружают весь твой цифровой след, обучают модель, а потом навешивают на тебя датчики, и ты неделю ходишь, живёшь, а система за эту неделю учится думать за тебя. Ты мёртв, твоё копия машет с экрана, все счастливы. Красиво же? Жутко. По-антиутопически. Я довольная села писать.

Приз я не получила, зато получила вопрос, который потом не отпускал меня всю неделю: а что я, собственно, написала? Антиутопию aka выдумку? Или научную фантастику aka научпоп, а может вообще прогноз будущего?

Сразу дисклеймер, чтобы вы не пришли меня бить: я не эксперт в LLM-архитектуре и вообще не нейробиолог. Я пришла сюда не с ноги «смотрите какая я умная», а с ноги «мне стало интересно, я пошла гуглить и офигела, насколько случайно попала и не попала». Это разбор моей наивной идеи, сначала написанной на вдохновении как просто писака, а потом уже отрефлексированной после публикации. Погнали по кускам: ровно в том порядке, в каком я сама во всё это проваливалась.

Obama-appointed judge seemed unmoved by Trump attorneys’ arguments in case also naming Murdoch

A federal judge in Florida seemed skeptical of arguments from Donald Trump’s legal team on Wednesday over whether the court should allow a $10bn defamation lawsuit against the Wall Street Journal to move forward.

Trump sued the Journal and the News Corp founder Rupert Murdoch over a July 2025 article about a “bawdy” letter reportedly from Trump that was included in a 2003 birthday book for disgraced financier Jeffrey Epstein. Trump has denied authoring or signing the letter, and his attorneys have argued that the Journal published the story despite having been told it was false.

University says it has ‘new information’ about academic career of Jason Arday, the youngest black professor in its history

Jason Arday, the Cambridge professor accused of plagiarism and embellishment, is to face investigation by the university after it said it received “new information” about his qualifications and career.

Cambridge initially defended Arday – the youngest black professor appointed in the university’s history, with a compelling life story of educational disadvantage – as “the victim of a vile campaign”. But as questions mounted over Arday’s own claims, the university’s leaders felt that Cambridge’s reputation was in danger.