RSS Feeds

First responders also rescued flood-stranded people after storms unleashed heavy rains in the US north-east

Residents in Delaware’s capital and on Long Island, New York, were picking up the pieces on Friday after tornadoes touched down during sudden storms that battered the north-eastern US with heavy rains and left dozens of people stranded in their cars during flash flooding.

In Atlantic Beach on Long Island, the long peninsula that stretches out into the ocean to the east of New York City, images posted on social media showed a large funnel cloud over the water and small buildings torn apart at a beach club.

В июле OpenAI выпустила новую линейку моделей, в конце месяца дважды поменяла цены, а на осень назначила отключение Assistants API, Videos API и всего поколения GPT-4. Мы переподключали под эти изменения свои сервисы и по ходу сверяли каждый шаг с документацией. Заодно посмотрели, сколько русскоязычных руководств успело устареть — оказалось, почти все, включая августовские.

First responders also rescued flood-stranded people after storms unleashed heavy rains in the US north-east

Residents in Delaware’s capital and on Long Island, New York, were picking up the pieces on Friday after tornadoes touched down during sudden storms that battered the north-eastern US with heavy rains and left dozens of people stranded in their cars during flash flooding.

In Atlantic Beach on Long Island, which stretches out into the ocean to the east of New York City, images posted on social media showed a large funnel cloud over the water and small buildings torn apart at a beach club.

The Mexican government's creation of lists of journalists and opposition figures has stirred memories of the 'enemies list' of the Nixon administration in the United States.

The young Italian has a substantial lead but his rivals will not see him as being out of reach going into the Dutch GP

The Dutch Grand Prix may be hosting its final meeting on the Formula One calendar this weekend but its farewell heralds the opening of a championship run-in that is far from over. Rested, reset and ready to rumble, the title contenders in Zandvoort have an anticipatory air as they enter the sharp end of a fight which, as last season demonstrated, could yet be taken to the finale.

Resuming after F1’s summer break there is, as always, a sprightly atmosphere to the sport. Team members have had time with their families and drivers have had a chance to decompress, assess, heck, even have a bit of fun, with Mercedes’s George Russell becoming engaged to his partner, Carmen Montero Mundt, and Lando Norris out and about in Ibiza with pals. Lewis Hamilton got his head down and trained hard, proclaiming himself fitter then ever, energised and refreshed, while Max Verstappen emerged at his home race to announce a new contract with his Red Bull team.

Вводное.
Открываешь статью с многообещающим заголовком. Читаешь пять минут. Потом десять. И вдруг осознаешь, что потратил время впустую: перед тобой либо пересказ Википедии, либо философские размышления автора на тему «как же тяжело жить», либо кривой перевод чьей-то субъективной заметки с Medium, в которой нет ни строчки кода, ни одной метрики, ни архитектурной схемы.
Соотношение сигнала и шума на Хабре неуклонно падает. Инженеры приходят сюда за опытом, паттернами и разбором полетов, а получают информационный фастфуд.
Недавно в ленте мелькнул перевод заметки от именитых вице-президентов Microsoft о том, что «ИИ убивает карьеру джунов». Потратив время на чтение, я обнаружил, что это классическое управленческое эссе. В нем ноль фундаментальных механизмов, ноль алгоритмов, ноль метрик производительности и ноль конфигураций. Это просто мнение, замаскированное под техническую аналитику. Когда автор берет такой «мусор», решает, что он полезен, и тащит его на Хабр, он не делится опытом. Он крадет время тысяч инженеров. Теоретические статьи и исторические экскурсы имеют право на существование. Но они должны содержать оригинальное исследование, глубокий анализ или систематизацию фактов. Если статья просто пересказывает общеизвестные факты без новой аналитики — это перепост Википедии, которому место в личном блоге, а не в профильных хабах.

Всё началось с анализа статей для другого разбора, в итоге вылилось в решение автоматизации.
а конкретно, спасибо вот этой статье.

Samuel Yeboah, Francesco Di Chiara and Mingliang Liu

Today, Netflix runs two Flink autoscalers. That is exactly one more than we want. We built the first one in-house years ago, when there was no mature option suited to our platform. The second came from the Apache Flink community, and it can scale workloads our homegrown system was never designed for. We now run both in production and are steadily converging on the open-source one. Along the way we learned some hard lessons about metrics, cost, and the real price of maintaining infrastructure you could instead adopt, and we hope they are useful whether you run a handful of Flink jobs or tens of thousands.

Why autoscaling is not optional at our scale

Netflix has run stream processing on Apache Flink since 2017. As of 2026 we operate more than 30,000 Flink jobs across multiple AWS regions. Most are not deployed by hand; they are generated by our managed platform Data Mesh, so the majority of users never touch a Flink job directly. A smaller but growing set are custom jobs, built and operated by teams across the company for use cases like personalization, Ads, and Live events. They range from single-operator jobs that shuttle records between Kafka topics to stateful pipelines with branches, joins, and terabytes of state, and their load swings with daily cycles, launches, and regional failovers.

Provisioning every one of those jobs for its peak is wasteful; provisioning for the average causes lag during surges. And in our platform a scaling action is not free: by default it means taking a savepoint, stopping the job gracefully, and restarting it at the new size, which for a large stateful job can take minutes. That leaves a genuinely hard question: how do you give each job the resources it needs, when it needs them, without a human in the loop and without breaking anything?

The first autoscaler: watching from outside

Our first answer, built around 2019, was an autoscaler shaped like a stream-processing job. It ran on Mantis, consuming a live feed of cluster-level metrics from Atlas, our telemetry platform, including CPU, network, Kafka lag, input-rate, and consume-rate signals for every job. The scaler combined lag-derived catch-up time, CPU/network utilization thresholds, observed performance history, and regression over recent input rate to decide when to scale up or whether a smaller cluster could handle the lookahead window. Because the autoscaler operates independently of the Flink platform, it remains unaffected by issues within Flink itself. Building it as a streaming job also made it easy to scale. Each autoscaler node handled the metrics for a subset of Flink jobs, and we never had to write custom sharding or coordination logic to keep up with a growing Flink fleet. It reliably cut resource usage by 25–45% across thousands of managed pipelines. Check our previous talk at Flink Forward 2020.

But watching from outside has a ceiling. The system reasoned about a whole cluster through coarse container metrics, and it scaled a single knob, the total TaskManager count, so every operator in a job moved together. That fit the simple, single-operator pipelines it was built for, but not the multi-operator, stateful DAGs that teams were increasingly bringing to us for Ads, recommendations, and games. Those were exactly the jobs it could not reason about, and supporting each new case meant more custom logic rather than any general capability.

The autoscaler is only as good as the metrics served by external systems beneath it. Those metrics could miss real trouble: a job could be completely busy without any of it showing up as CPU utilization, leaving the job stuck in a degraded state the scaler had no way to see. Recently a networking migration quietly changed how some traffic was reported, and a subset of the Atlas metrics the scaler relied on stopped capturing everything accurately. The gap stayed invisible until it surfaced in production much later.

It was time to reconsider build versus buy.

The second autoscaler: reasoning from inside

When we started, the Flink community had no mature autoscaler to offer. By the time we re-evaluated, it did: the Apache Flink Autoscaler. Instead of watching containers from outside, it reasons from inside the job.

Its key idea is to estimate each operator’s true processing rate (TPR): the throughput it could sustain if it were fully busy. Flink reports, per subtask, the fraction of each second spent doing actual work, separate from time spent backpressured or idle. Dividing observed throughput by that busy fraction extrapolates capacity to full utilization: an operator handling 700 records/sec while busy 70% of the time has a TPR of 700 / 0.7 = 1,000 records/sec. Starting from the sources, the autoscaler walks the job graph and uses each operator’s TPR, its input/output ratios, and a target utilization to compute the parallelism every vertex needs so that no operator becomes the bottleneck, rather than resizing the whole cluster as a unit.

The two approaches make a different contract, summarized below.

The decisive difference for us is the last two rows: the OSS autoscaler can scale exactly the stateful, multi-operator jobs our homegrown system could not, and it lets each job carry its own configuration — stabilization periods, thresholds, and other scaling behavior tuned to the workload. That made it the natural fit for the custom jobs teams had been scaling by hand.

Making it work at Netflix scale

Adopting the algorithm was straightforward; the community had done the hard part. The work for us was running it reliably across our own jobs, and this is where our system differs most from the stock open-source deployment.

Firstly, the OSS autoscaler was originally architected to reside within the Kubernetes Operator for Flink, but our Flink platform runs on its own control plane, not that operator (see our previous talk at Current Conference 2024). The community later made a fantastic decision to keep the core logic as a standalone library. They refactored four generic interfaces that made it easy to plug directly into our internal ecosystem: a context carrying job metadata and REST API info, a state store, an event handler, and a realizer that applies scaling decisions.

That service is a Spring Boot application whose orchestration runs on Temporal, the durable workflow engine. An orchestrator workflow polls our Flink control plane about once a minute for the jobs with autoscaling enabled, and starts one long-running workflow per job. Each per-job workflow pulls that job’s per-vertex metrics from its Flink JobManager, runs the OSS evaluation algorithm, and, when a scaling decision results, hands it to a realizer that actuates the change through our Flink control plane.

The workflow-per-job design was a direct response to pain. We first ran evaluations in a single batch loop over the whole set of jobs, and it was fragile: one slow or misbehaving job could stall metric collection and scaling for every job behind it. Giving each job its own durable workflow isolated that blast radius, so a single problematic job now fails and retries on its own, and the runtime scales out as we onboard more jobs.

Secondly, three engineering gaps stood between “works in community” and “works at Netflix scale”:

• Metric collection at high parallelism. On big jobs, pulling metrics from the JobManager became a bottleneck, and part of the cause was in Flink’s runtime. To address that, we changed the JobManager to cache transient metric names and clean them up once instead of rescanning on every fetch, and we added server-side filtering so the autoscaler asks only for the metrics it needs. This let the autoscaler work on jobs up to 3,000 Flink subtasks, where it had previously struggled above roughly 1,000. Those are in our internal fork of Flink release, while some are contributed upstream such as FLINK-36172.
• Preserving forward chaining. Two separate vertices joined by a forward connection must run at the same parallelism, because records are handed over in memory on a fixed local channel. Scale one of them alone and Flink does not fail; it silently converts that edge into a network shuffle. Our fork detects forward-connected subgraphs and scales each as a unit.
• Respecting sink limits. Some sinks have finite write capacity, so we added detection for async-sink backpressure (also a fork change) to keep the autoscaler from scaling a job up into a sink that cannot absorb more.

Before it actuates anything, the realizer runs a set of safety checks. For example, it refuses to scale a job down in a region being evacuated during a company-wide region failover. It also verifies there is enough disk for the new cluster to hold the job’s checkpoint state, and it adds a small standby buffer for larger clusters.

The road to one autoscaler

Last year, the OSS-based autoscaler achieved general availability for custom jobs at Netflix, yielding promising initial outcomes. For instance, our client telemetry and logging team achieved a 58% reduction in its annualized Flink compute expenditures, saving approximately $1.1 million annually. This efficiency is driven by three key factors. First, whereas static provisioning must always account for peak loads, autoscaling dynamically adapts to daily cycles, capturing the drop in traffic during nights and weekends compared to weekday peaks. Second, rather than relying on teams to manually optimize resources following performance improvements or post-holiday slowdowns, the autoscaler continually adjusts capacity. Finally, adopting uniform container dimensions enables superior bin-packing and more granular scaling increments.

Additionally, scaling down too eagerly is its own trap. Cut too deep and CPU saturates, lag spikes, and the system cannot react instantly because its metric window and stabilization period have to rebuild after each restart. We now run a target utilization of 0.45, below the community default of 0.7, deliberately trading a little efficiency for stability. Fewer and calmer rescales are worth the marginal cost for large stateful jobs.

While our scaler provides fine-grained signals and vertex-level decision units for stateful DAGs, fast rescaling still heavily depends on Flink Core’s state restoration performance. Today, the biggest remaining cost in scaling a stateful job isn’t the scaler’s logic — it’s the restart and state recovery process itself. Flink 2 addresses this through its disaggregated state architecture, keeping state in external storage rather than on local disk, which can sharply reduce how much a rescale or recovery depends on total state size. Having started supporting Flink 2.2 at Netflix, we plan on experimenting with this new state backend to see if it can help eliminate state recovery bottlenecks when scaling large stateful jobs.

Looking ahead, we aim to migrate all internal scaler use cases onto the new one based on OSS autoscaler to simplify our operational surface area.

Key Takeaways

Along the way, three lessons that generalize beyond Flink:

• Metric choice matters more than algorithm sophistication. Our most useful debugging was rarely about the scaling math; it was about which signal to trust most. Understand your metrics before you tune your algorithm.
• Set sensible defaults, but leave room to tune. Our managed jobs are similar enough that one good default covers most of them untouched, which is the point of a platform. But forcing a single configuration on every job punishes the ones that do not fit, so we pair defaults with per-job overrides and deliberately hide the knobs that need deep expertise. Most teams should never have to think about the autoscaler.
• Adopt, then extend. We built in-house because in 2019 nothing mature fit our platform. When a strong community project appeared, the right move was neither to defend our investment forever nor to rip it out overnight, but to adopt it for new workloads, contribute fixes back, and plan a deliberate migration.

Thanks to the Flink and Data Mesh teams for the control-plane changes this work depended on, to the Temporal team and our early pilot teams, and to the Apache Flink autoscaler maintainers whose foundation we built on. Special thanks to Andy Zhang, Calvin Cheung, Daniel Trager, Guil Pires, Mark Cho, Matthew Kornitsky, Nikhil Sulegaon, Sujay Jain, and Tom Lee.

A Tale of Two Flink Autoscalers was originally published in Netflix TechBlog on Medium, where people are continuing the conversation by highlighting and responding to this story.

The actor, who plays Red Guardian in the film, made a cryptic comment to Variety that raised more Robert Downey Jr-shaped questions than answers. But was it a clanger, or expert trolling?

Nobody knows exactly what solemn blood oaths Marvel uses to prevent spoilers. But we do know that Andrew Garfield spent most of the run-up to 2021’s Spider-Man: No Way Home – in which he would eventually turn up as one of two secret Spideys – behaving as if Kevin Feige was standing behind him, holding up a giant card saying: YOU HAVE NEVER MET TOBEY MAGUIRE. So we can imagine it’s pretty full on. Why, then, has Avengers: Doomsday star David Harbour this week apparently blurted out the biggest Marvel secret since Ben Kingsley’s Mandarin in Iron Man 3?

Invited by Variety magazine to offer up a line from Doomsday “completely out of context”, Harbour thought for a moment, adopted the Russian accent of his Red Guardian character and came out with: “You … are Tony Stark.” Which is quite a thing to say about a film whose entire marketing campaign so far has depended on us accepting that Iron Man remains dead, and that the actor who used to play him, Robert Downey Jr, is playing Victor von Doom instead.

Appeals court slashes judgment to $1.5m in a significant legal victory for the rightwing Infowars founder

A Texas appeals court has slashed the $50m judgment to $1.5m against Alex Jones, founder of the rightwing platform Infowars, over falsely calling the 2012 Sandy Hook school shooting in Newtown, Connecticut, a hoax.

The ruling does not affect a $1.4bn judgment against Jones in Connecticut, but is a significant legal victory for Jones after he and his company, Free Speech Systems, were found liable for damages for claiming the mass shooting didn’t happen.

Appeals court slashes judgment to $1.5m in a significant legal victory for the rightwing Infowars founder

A Texas appeals court has slashed the $50m judgment to $1.5m against Alex Jones, founder of the rightwing platform Infowars, over falsely calling the 2012 Sandy Hook school shooting in Newtown, Connecticut a hoax.

The ruling does not affect a $1.4bn judgment against Jones in Connecticut, but is a significant legal victory for Jones after he and his company, Free Speech Systems, were found liable for damages for claiming the mass shooting didn’t happen.

Audio recording of US vice-president apparently made at New York fundraiser and shared with Canadian Press

Leaked audio from a private event has reportedly captured the US vice-president, JD Vance, mocking Canada for apparently capitulating in a trade deal and taunting the prime minister, Mark Carney, for trying to “out-tough” Donald Trump.

The recording, which was reportedly made at a fundraiser and shared with the Canadian Press, comes in the last stretch of a difficult negotiation in which the US president has threatened to impose 50% tariffs on Canada.

Police investigating after two cars set alight in attack caught on CCTV

Firebombs were thrown at supercars outside a wedding venue in Birmingham, setting two alight and prompting guests to sprint out of the building.

West Midlands police said they were investigating the incident, which happened in the Sparkbrook area of Birmingham at around 8.30pm on Wednesday.

Audio recording of US vice-president apparently made at New York fundraiser and shared with Canadian Press

Leaked audio from a private event has reportedly captured the US vice-president, JD Vance, mocking Canada for apparently capitulating in a trade deal and taunting the prime minister, Mark Carney, for trying to “out-tough” Donald Trump.

The recording, which was reportedly made at a fundraiser and shared with the Canadian Press, comes in the last stretch of a difficult negotiation in which the US president has threatened to impose 50% tariffs on Canada.

Police investigating after two cars set alight in attack caught on CCTV

Firebombs were thrown at supercars outside a wedding venue in Birmingham, setting two alight and prompting guests to sprint out of the building.

West Midlands police said they were investigating the incident, which happened in the Sparkbrook area of Birmingham at around 8.30pm on Wednesday.

Late actor in May said beauty brand cut off its relationship with her after she spoke out about postpartum depression

The beauty brand Neutrogena has acknowledged making its former ambassador Hayden Panettiere “feel unsupported during a very difficult time” – alluding to when the late Hollywood actor said the company severed ties with her after she spoke about enduring postpartum depression (PPD).

Neutrogena issued a statement addressing the situation four days after Panettiere, 36, was found dead on Sunday at an apartment complex in Greenville, South Carolina.

Полтора года назад я написал о своём подходе к структуре FastAPI-приложений. С тех пор два крупных проекта, production и несколько архитектурных граблей заставили меня многое пересмотреть. В этой статье расскажу, что изменилось и какие решения действительно упростили мне разработку.