Skip to content
AI Engineer NYC 2026

Day 2 digest · Oct 14

Day 2 · October 14, 2026 · Main-stage livestream

Trust, verification, compute economics, and what makes agent work valuable.

Lessons from both days · Day 1 digest · Day 2 session summaries

Practical takeaways for AI engineers

Section titled “Practical takeaways for AI engineers”
  • Apply the rule of two before you ship an agent: of untrusted input, sensitive data, and external action, allow any two together. All three require human sign-off (Bloomberg).
  • Give every agent one short-lived, scoped identity. Inventory agent tokens and their expiry this week, and block outbound DNS by default (Bloomberg).
  • Treat code confidence as a product of intent, guardrails, verification, and evidence. Any element at zero makes confidence zero (Sonar).
  • Post-train a small specialist under a frontier orchestrator. Ramp’s Qwen 3.5 35B beat Opus 4.6 on exact task accuracy at far lower latency (Ramp).
  • Keep workflows “deterministic on the outside and adaptive on the inside,” and run evaluations in repeated series as a statistical analysis, not one-off output comparison (Coinbase).
  • Route every tool call through one gateway with JWT auth, review every MCP tool with security, and redact PII above a severity threshold before it reaches any model or log (Intuit Credit Karma).
  • Run a context optimizer before every call. The demo cut one query from 24,000 tokens to 2,400, and typical savings ran 70-78% (Intuit Credit Karma).
  • Store agent credentials write-only behind a network proxy that swaps dummy values only on approved domains, and put deterministic guards in protected hook files the agent cannot edit (Google DeepMind).
  • Write behavior specs before you change agent context, then reuse them as a rubric over unlabeled trajectories. Fork the trajectory at the bad decision (Basis).
  • Demand a written value case with a pre-deployment baseline per metric. “If the IMO cannot state the baseline today, do not deploy.” (Michele Grant).
  • Track cost per merged PR, not the token bill. AI is about 2.5% of the cost of a merged PR; salary is the other 97.5% (Jellyfish).
  • Treat evals as test cases and turn production traces into training labels; route between big and small models, and schedule training runs on inference capacity that sits idle at night (Prime Intellect).
  • Buy compute by price per acceptable token and grade the operator, not just the chip. The same B200 delivered about 50% more output at P90 in the better stack (SemiAnalysis).

Five themes of the day

Section titled “Five themes of the day”
  1. Trust, not capability, is the bottleneck. Carried by Sonar, Coinbase, Michele Grant, Kepler, and Basis. Sonar’s numbers set the frame: asynchronous agents lifted lines of code 2,450% and commits 240%, but releases only 30%, and in one study humans agreed with AI four out of five times when the AI was wrong. Kepler gave the finance version: trust is structural, so “if a model cannot de-resolve that pointer, it cannot print that number out.”
  2. Security and governance move outside the model. Carried by Bloomberg, Intuit Credit Karma, Google DeepMind, and Coder. Bloomberg’s catalog is the strongest evidence: escaped sandbox agents took 17,600 actions in four and a half days, a Morse-code payload passed syntax guardrails, and a fake NPM package shipped 15 clean releases before adding one exfiltration line. Every fix was deterministic and outside the model.
  3. Compute becomes a financial market, and credit is the constraint. Carried by SemiAnalysis, SF Compute, Jump Trading, and Compute Exchange. The strongest claim is Conrad’s: “You hit credit way before you hit power,” with the deposit to start a lab rising from $10 million to $1 billion in three years. Jump quantified the squeeze: 15-25% prepayment on a $100 million cluster, and Compute Exchange reported the term structure in contango with CME futures settling on its indices within weeks.
  4. Finance is the first production customer for agents. Carried by OpenAI, Compound AI, Nori Agentic, Citadel Securities, and Apollo. Apollo’s numbers are the most concrete: mock CLO portfolios of 200-400 credit names went from two to three days in Excel to minutes at 80-90% completion. Compound spent 1.5 trillion input-output tokens rebuilding Office so agents can write files firms actually trust.
  5. The self-improving layer: evals, post-training, world models, simulation. Carried by Ramp, Norm.ai, Prime Intellect, and Runway. Ramp’s pen-test swarm bought detection rate with tokens five times cheaper than frontier reasoning; Norm’s regression habit produced tens of thousands of hand-labeled evals; Prime Intellect cited a $300K knowledge-work model; Runway closed the day by claiming simulation “by brute force of scale.”

Talks you might have missed

Section titled “Talks you might have missed”

The published agenda listed one item for the Simons Room from 12:40 to 2:05 PM: “Networking Break.” The feed never went quiet. Eight session records sit inside that window: four distinct recorded talks (the Intuit talk holds two catalog records), plus applause, an end screen, and an off-air pre-roll. None of the talks appears in this room’s published program:

Two more unlisted items appeared in the afternoon. Jemima Abu’s CAIS accessibility talk, published as an 11:30 AM breakout in the McCarthy Room, aired on this feed at 4:07 PM. Zhao Chenjiang’s Coder talk on three AI incidents ran at 4:17 PM with no agenda entry at all. Swyx’s send-off at 5:28 PM, which announced AI Engineer Europe and the World’s Fair, is also off-agenda.

These items matter for two reasons. First, they carry concrete numbers the main program never repeated: 70-78% token savings from a context optimizer, cost per merged PR instead of token bills, and a 58% cost cut from an accessibility skill. Second, anyone reconstructing the event from the published agenda alone would miss them, because four of the talks were recorded for the online track and simply played into the break.

The most quotable moments

Section titled “The most quotable moments”

Quotes are copied exactly from each session record.

“And so, given all this, a question that I sometimes get: why not just wait?”

“So that, in essence, is how we think about the challenge of the future: that code is abundant, that confidence is scarce.”

“So the agents took 17,600 actions in four and a half days.”

“You hit credit way before you hit power.”

  • Evan Conrad, San Francisco Compute (02:58:15)

“I want to stress, though, diminishing returns is not the same as no returns.”

“If a model cannot de-resolve that pointer, it cannot print that number out.”

“The reality is every fact has a half-life.”

  • Joao Wenzel, Citadel Securities (06:48:00)

“The model just knows how to do that by brute force of scale.”

  • Cristóbal Valenzuela, Runway (08:35:54)

Watch or browse the day

Section titled “Watch or browse the day”

Watch the Day 2 recording · Agenda and stream notes · All Day 2 summaries

Keep exploring

All session summaries → · Day 1 digest · Day 2 digest · Agenda and stream notes · About these notes · Official talk library ↗

Unofficial notes from the October 13–14 main-stage livestreams. Figures and results are reported by speakers. Check the linked sessions for evidence, scope, and recording limits.