<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Off-Square]]></title><description><![CDATA[Opinionated essays on AI, startups, venture capital, and the incentives underneath them.]]></description><link>https://offsquare.substack.com</link><image><url>https://substackcdn.com/image/fetch/$s_!xlX2!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3d9a7610-843e-4b76-9d42-2be7ba8974f6_1024x1024.png</url><title>Off-Square</title><link>https://offsquare.substack.com</link></image><generator>Substack</generator><lastBuildDate>Mon, 22 Jun 2026 11:09:06 GMT</lastBuildDate><atom:link href="https://offsquare.substack.com/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Frank Odom]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[offsquare@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[offsquare@substack.com]]></itunes:email><itunes:name><![CDATA[Frank Odom]]></itunes:name></itunes:owner><itunes:author><![CDATA[Frank Odom]]></itunes:author><googleplay:owner><![CDATA[offsquare@substack.com]]></googleplay:owner><googleplay:email><![CDATA[offsquare@substack.com]]></googleplay:email><googleplay:author><![CDATA[Frank Odom]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[The Four-Hour Agent]]></title><description><![CDATA[And how to scope tickets for AI agents, based on research from METR.]]></description><link>https://offsquare.substack.com/p/the-four-hour-agent</link><guid isPermaLink="false">https://offsquare.substack.com/p/the-four-hour-agent</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Sun, 14 Jun 2026 14:01:32 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!0Nct!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="native-audio-embed" data-component-name="AudioPlaceholder" data-attrs="{&quot;label&quot;:null,&quot;mediaUploadId&quot;:&quot;6cd7547c-df2f-4fd5-a7d9-a8df8429f7db&quot;,&quot;duration&quot;:657.78937,&quot;downloadable&quot;:false,&quot;isEditorNode&quot;:true}"></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!0Nct!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!0Nct!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png 424w, https://substackcdn.com/image/fetch/$s_!0Nct!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png 848w, https://substackcdn.com/image/fetch/$s_!0Nct!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png 1272w, https://substackcdn.com/image/fetch/$s_!0Nct!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!0Nct!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png" width="1456" height="726" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:726,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:9816389,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/201923119?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!0Nct!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png 424w, https://substackcdn.com/image/fetch/$s_!0Nct!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png 848w, https://substackcdn.com/image/fetch/$s_!0Nct!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png 1272w, https://substackcdn.com/image/fetch/$s_!0Nct!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7532060d-31bc-459c-8217-bfd3d6444bd8_3244x1618.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Every task I hand to Claude must be something a human could finish in 2-4 hours.</strong> Less than 2 hours, and I waste a lot of time delegating small tasks. More than 4 hours, and Claude makes too many mistakes. I came up with this rule somewhat arbitrarily, based on anecdotal observations from my engineering work. But surprisingly, I think it&#8217;s explainable by recent research from METR.</p><p>METR is a nonprofit that tracks how capable AI agents are getting over time, and they publish the chart below. I came across it on <a href="https://www.youtube.com/watch?v=Hx5dIJ3H8p4">a recent Odd Lots episode</a> where they were discussing the exponential growth of AI agents. I keep returning to it for a far more mundane reason &#8212; it is one of the best ticket-planning heuristics I have found.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!wjkA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!wjkA!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png 424w, https://substackcdn.com/image/fetch/$s_!wjkA!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png 848w, https://substackcdn.com/image/fetch/$s_!wjkA!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png 1272w, https://substackcdn.com/image/fetch/$s_!wjkA!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!wjkA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png" width="741" height="547" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:547,&quot;width&quot;:741,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:84070,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/201923119?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!wjkA!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png 424w, https://substackcdn.com/image/fetch/$s_!wjkA!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png 848w, https://substackcdn.com/image/fetch/$s_!wjkA!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png 1272w, https://substackcdn.com/image/fetch/$s_!wjkA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F95ac8b8c-d9f2-4889-a927-fe2525752531_741x547.png 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Subscribe to Off-Square for opinionated takes on AI, startups, VC, and the incentives underpinning them.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><div><hr></div><h2>Reading the Chart</h2><p>The vertical axis is human time. A point at four hours means the model can complete a task that would take a skilled human about 4 hours, working autonomously, without further input. The headline trend is that this horizon has been doubling roughly every 7 months since 2019. As of early 2026 the frontier has crossed 4 hours at a 50% success rate.</p><p>The detail I care about most is that 50% success rate. Claude now correctly finishes most well-scoped, 4-hour tasks without further input from me. That is the threshold that matters to me: once an agent clears it, I can hand off work of that size and expect to leave most of it alone. I can write a ticket, hand it off, and then go do something else while the agent runs.</p><p>In practice, I believe my own success rate is higher than the chart&#8217;s baseline, because I have invested heavily in the scaffolding around the agent. All of my projects have a comprehensive <code>CLAUDE.md</code>, a mapping of the project layout, implementation playbooks for common tasks, and multiple skill files. Most of the time, the agent runs to completion without me.</p><div><hr></div><h2>The Cost of Delegation</h2><p>For several months my rule has been to scope every ticket I hand to a coding agent to between two and four human hours. Not the wall-clock time the agent takes, which is usually minutes, but the time the task would have cost a competent human doing it by hand. The specific numbers are arbitrary. The shape of the rule is not. There is a floor and a ceiling, and they come from completely different places.</p><p><strong>Below ~2 hours per ticket, I become the bottleneck.</strong> Every handoff takes more of my time.</p><ul><li><p>writing a crisp ticket with success criteria</p></li><li><p>pulling my attention out of whatever I was doing</p></li><li><p>waiting on the run</p></li><li><p>reviewing what comes back</p></li></ul><p>I pay that tax whether the underlying task was 20 minutes or 3 hours. But a frontier model dispatches a 20-minute task almost trivially.</p><p><strong>Past about four hours of scope, tasks start failing too often.</strong> I have to come back in, intervene, or split the ticket into multiple parts and start over. The reliability decay the METR curve shows from a distance is the same decay I feel up close. Longer tasks accumulate more places to drift, and drift compounds on itself.</p><p>The cost of delegation is the sum of two things:</p><ol><li><p>the coordination overhead, which is the fixed cost I pay every time I hand off a ticket</p></li><li><p>the expected cost of failure and rework, which grows as the task gets larger and more complex</p></li></ol><p>Add them together and you get a U-shaped cost curve. On the left, with small tickets, coordination overhead dominates. On the right, with large tickets, failure and rework dominate. In between is the local minimum, and &#8220;two to four hours&#8221; is simply where it falls for me today, with current models.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Ejgz!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Ejgz!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png 424w, https://substackcdn.com/image/fetch/$s_!Ejgz!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png 848w, https://substackcdn.com/image/fetch/$s_!Ejgz!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png 1272w, https://substackcdn.com/image/fetch/$s_!Ejgz!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Ejgz!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png" width="1456" height="935" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:935,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:86898,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/201923119?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Ejgz!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png 424w, https://substackcdn.com/image/fetch/$s_!Ejgz!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png 848w, https://substackcdn.com/image/fetch/$s_!Ejgz!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png 1272w, https://substackcdn.com/image/fetch/$s_!Ejgz!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10db6cd4-1df8-4dc5-bbc7-2eb9e18d36ce_1660x1066.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The left wall likely will not change over time, because my coordination overhead is manual and roughly constant. The right wall is the agent&#8217;s reliability ceiling, which is exactly what the METR chart captures. As the METR time horizon doubles, the right wall slides outward, and the bottom of the U slides out with it. Every ~7 months, the largest task I can safely hand off grows by a factor of 2.</p><p>Which means my heuristic is not really &#8220;2-4 hours.&#8221; It is &#8220;find the local minimum.&#8221; The number is a snapshot in time, and we need to keep re-reading the curve as the wall moves.</p><div><hr></div><h2>(Near-) Perfect Information</h2><p>I have argued before, in <a href="https://offsquare.substack.com/p/on-imperfect-information">On Imperfect Information</a>, that LLMs are a near-perfect replacement for action under near-perfect information. The hard part is turning a messy problem (e.g. feature request) into something close to perfect information, which I could hand off to Claude. Every time I hand off a ticket, I attach explicit success criteria:</p><ul><li><p>commands that must pass</p></li><li><p>behavior that must hold</p></li><li><p>the edge cases that must be handled</p></li></ul><p>The agent can then check its own work, and catch a meaningful fraction of its own mistakes before handing anything back.</p><p>You are not making the LLM smarter. You are effectively shrinking the space it can drift into, and giving it a way to notice when it has drifted. A 4-hour task with a vague goal lives down near 50% success rate. The same task with a crisp specification and verifiable success criteria lives much closer to 80%.</p><p>This is why I front-load so much planning work before any code gets written. It comes in two forms:</p><p>The first is repository setup. A thorough project guide with a map of the codebase, plus short playbooks for the common moves like database migrations, new API endpoints, and new frontend components, means a large share of the information already sits in <code>CLAUDE.md</code> before I write any tickets.</p><p>The second is planning the tickets themselves. I like to plan interactively with Claude. I talk through the user flows, and as we go Claude proposes tickets that I review and refine before any of them get uploaded to Linear. I now spend more time planning tickets than implementing them, because the planning is what makes the implementation run smoothly.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/the-four-hour-agent?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/the-four-hour-agent?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/the-four-hour-agent?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><div><hr></div><h2>The Babysitter</h2><p>Most product and project management work is now, functionally, software work. I use the Linear CLI for tickets and the GitHub CLI for branches and pull requests, which means Claude can handle most project management work directly from the command line. Once individual tickets are available, the obvious next step is to stack them into sprints, and have Claude oversee the sprint end-to-end.</p><p>I&#8217;ve built a skill I call <code>/speedrun</code>. The main agent acts as a babysitter for a sequence of pre-scoped tickets. For each ticket it:</p><ul><li><p>cuts a fresh branch</p></li><li><p>hands the ticket to a brand-new non-interactive sub-session</p></li><li><p>creates a pull request</p></li><li><p>waits on continuous integration</p></li><li><p>patches any small bugs</p></li><li><p>merges the PR into a feature branch</p></li><li><p>moves to the next ticket</p></li></ul><p>Then I review all of the changes on a feature branch, before any of it reaches <code>main</code>.</p><p>Here&#8217;s a basic version of the <code>/speedrun</code> skill I use in my actual projects, which <a href="https://github.com/fkodom/skills/blob/a88312c53d7a0f87ddd8612e970aeaba10ee434c/skills/speedrun/SKILL.md?plain=1">you can also find in GitHub</a>. (It can be more complicated for established projects, where I typically also integrate with testing/linting tools, add custom agent hooks, etc.)</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;markdown&quot;,&quot;nodeId&quot;:null}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-markdown">---
name: speedrun
description: Run a sequence of pre-scoped tickets back-to-back, each in its own
  non-interactive Claude sub-session. Triggers on &#8220;/speedrun 41-46&#8221;.
allowed-tools: Bash, Read, Edit
---

# Speedrun

Drive a sequence of pre-scoped, execution-ready tickets to completion, one after
the next. You (the main agent) are the babysitter: set up branches, hand each
ticket to a fresh non-interactive sub-session, wait for CI, and merge the result.
Core assumption: these tickets are already scoped and reviewed. This is an
execution run, not design. If a ticket turns out to need real exploration, STOP
and hand it back to the human.

## 1. Set up one feature branch for the whole run

```bash
git checkout -b feature/&lt;slug&gt; &amp;&amp; git push -u origin feature/&lt;slug&gt;
```

## 2. For each ticket, in order

Fetch the ticket and read its acceptance criteria:

```bash
linear issue view &lt;id&gt;
```

Branch off the latest feature branch:

```bash
git checkout feature/&lt;slug&gt; &amp;&amp; git pull
git checkout -b ticket/&lt;id&gt;
```

Hand the ticket to a non-interactive sub-session. There are no special hooks here,
so the prompt itself tells the sub-session to commit and open its own PR:

```bash
claude -p --dangerously-skip-permissions &#8220;$(cat &lt;&lt;PROMPT
Implement exactly this ticket, no more and no less:
&lt;title, description, and acceptance criteria from Linear&gt;
When the work is done and the project&#8217;s typecheck and tests pass, commit your
changes, push branch ticket/&lt;id&gt;, and open a PR into feature/&lt;slug&gt;.
Follow CLAUDE.md conventions throughout.
PROMPT
)&#8221;
```

Watch CI, then merge into the feature branch:

```bash
gh pr checks &lt;pr&gt; --watch
gh pr merge &lt;pr&gt; --merge
```

If CI fails on something trivial (a missed import, a lint nit), fix it yourself on
the branch and re-push. If it needs real debugging, STOP and hand back to the human.

## 3. Wrap up

Report which tickets merged and which are blocked. Do NOT merge the feature branch
into main. That is the human&#8217;s review gate.</code></pre></div><p>Using Opus 4.8, this babysitter works well for sequences of up to 5-6 tickets. For longer sequences, performance starts to degrade and <code>/speedrun</code> becomes less reliable.</p><p>This lines up neatly with the cost curve from earlier. I think it&#8217;s fair to say that 5-6 tickets is roughly 4 man-hours of project management time. Each ticket might reasonably take a human ~45 minutes to write, hand off, review, patch, and merge. On the other hand, a human manager&#8217;s span of control tops out around 5-7 direct reports, and human working memory holds about 7 items at once. Agents degrade for a structurally similar reason. Whether the binding constraint is the orchestrator&#8217;s context window or my own review attention, there is a ceiling on how much delegated state a single supervisor can hold before things start slipping through.</p><p>Guardrails make a <strong>big</strong> difference here. Notice this one line from the skill file:</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;markdown&quot;,&quot;nodeId&quot;:null}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-markdown">If CI fails on something trivial (a missed import, a lint nit), fix it yourself on
the branch and re-push. If it needs real debugging, STOP and hand back to the human.</code></pre></div><p>When a ticket stops being pure execution and starts needing exploration or design, the babysitter stops and hands the work back to me. Without that instruction, I&#8217;m not sure the end-to-end <code>/speedrun</code> workflow would be sustainable.</p><div><hr></div><h2>Shifting Responsibilities</h2><p>None of this is an autonomous system, and I would not want it to be. I do the product and ticket definitions myself, including the interactive back-and-forth, user workflows, decisions about what a feature should and should not do, and the occasional insistence on a specific framework or implementation trick. I rarely write the full tickets by hand, since I offload that to Claude. But I own the creative direction, and I spot-check every ticket before uploading to Linear or kicking off the <code>/speedrun</code>.</p><p>This planning/creative/strategic phase of software development is becoming more demanding rather than less. I have written, in <a href="https://offsquare.substack.com/p/the-finite-work-fallacy">The Finite Work Fallacy</a>, about why automating your most valuable work produces more human work instead of less. There is no fixed pile of tasks, so compressing the top of the queue only promotes the next item into the empty slot. Agents are taking more of the code implementation and execution. The highest-leverage work now requires more strategic judgment: deciding what to build, turning the mess into a specification, and recognizing when a task has deviated from your intended plan. The entire apparatus above is just a machine for routing as much work as possible into the part the agents are good at, so that I can spend my own attention on the part they are not.</p><div><hr></div><h2>The N-Hour Agent</h2><p>Don&#8217;t get too attached to the &#8220;2-4 hour&#8221; heuristic. It is already going stale as I write this &#8212; assuming the US government re-allows Fable and future models for the general public. Claude Fable 5 sits in roughly the position that Mythos preview holds on that chart, and it comfortably handles 6+ hour chunks. If the 7-month doubling holds, by early next year I should be scoping 12-hour tickets and babysitting ~15 of them in a single `/speedrun` before quality breaks down.</p><p>The real heuristic is about trade-offs relative to the latest-generation agent capabilities. That trade-off is captured surprisingly well by a single chart from METR.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Subscribe to Off-Square for opinionated takes on AI, startups, VC, and the incentives underpinning them.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[The Finite Work Fallacy]]></title><description><![CDATA[Why your workload never decreases, despite AI and automation advances.]]></description><link>https://offsquare.substack.com/p/the-finite-work-fallacy</link><guid isPermaLink="false">https://offsquare.substack.com/p/the-finite-work-fallacy</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Sun, 31 May 2026 16:33:25 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!68i9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="native-audio-embed" data-component-name="AudioPlaceholder" data-attrs="{&quot;label&quot;:null,&quot;mediaUploadId&quot;:&quot;e71e1fa9-9a03-46f7-87d1-ffd10af9cf01&quot;,&quot;duration&quot;:556.85223,&quot;downloadable&quot;:true,&quot;isEditorNode&quot;:true}"></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!68i9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!68i9!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png 424w, https://substackcdn.com/image/fetch/$s_!68i9!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png 848w, https://substackcdn.com/image/fetch/$s_!68i9!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png 1272w, https://substackcdn.com/image/fetch/$s_!68i9!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!68i9!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png" width="1456" height="759" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/ecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:759,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Finite Work Fallacy&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Finite Work Fallacy" title="Finite Work Fallacy" srcset="https://substackcdn.com/image/fetch/$s_!68i9!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png 424w, https://substackcdn.com/image/fetch/$s_!68i9!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png 848w, https://substackcdn.com/image/fetch/$s_!68i9!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png 1272w, https://substackcdn.com/image/fetch/$s_!68i9!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecd46fa4-9eec-42f9-a161-8fae09c254d7_2956x1540.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><blockquote><p><strong>There is not a fixed amount of work. Automating the most important thing only promotes the second-most-important thing.</strong></p></blockquote><p>AI is automating more and more of our work, and yet the people closest to it report working more than ever. Dan Shipper said it clearly on <a href="https://www.lennysnewsletter.com/p/the-ai-paradox-dan-shipper">Lenny&#8217;s Podcast</a> recently, describing the strange feeling of having &#8220;so much automation, so much AI&#8221; while also working harder than before. He called it a paradox.</p><p>I don&#8217;t think it&#8217;s a paradox. I think it&#8217;s perfectly logical. The only reason it looks paradoxical is that it rests on an unspoken assumption: that there is a fixed amount of work to be done, so automating part of it should leave a smaller remainder and hand the rest of the day back to you. Drop that assumption and the &#8220;paradox&#8221; evaporates.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Subscribe to Off-Square for opinionated takes on AI, startups, VC, and the incentives underpinning them.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><div><hr></div><h2>The Finite Work Fallacy</h2><p>The premise is that work is a fixed quantity. You have a pile of tasks, the pile has a bottom, and any efficiency gain shrinks the pile and converts directly into free time. If that were true, then automating half your work really would give you half your time back, and the people most aggressively automating would be the most idle. I call this the &#8220;finite work fallacy,&#8221; because the premise is wrong.</p><p>Work is not finite. At any given moment there is a practically unbounded list of things that would be valuable to do, and the binding constraint is never the supply of valuable work. The constraint is your time and attention. Because the list is effectively infinite and your capacity is finite, you don&#8217;t work through the list. You only do the things that seem most important.</p><p>When the most important thing on the list becomes cheap, or fast, or fully automatable, you don&#8217;t stop working. If you compress the single most important thing you do, there is always a second-most-important thing that will now consume your time, and a third after that. The marginal value of each successive item may decline, but it does not hit zero, so you keep going. The queue never empties, because the queue was never finite.</p><p>The reason it feels like more work is that people still equate &#8220;work&#8221; with the technical output (the artifact they used to produce by hand), and they keep pouring mental and emotional energy into that layer even after the agent took over most of it. So the planning and prioritization that now fills their day registers as something extra, piled on top of a job they believe they are still doing in full. In reality, the focus just shifted. The execution stopped costing what it used to, the freed attention moved to the next constraint, and the brain quietly double-counts the part it no longer has to do.</p><div><hr></div><h2>Improving a Weakness Makes You Use It Less</h2><p>There is another, less obvious explanation for this &#8220;fallacy&#8221; that comes from game theory. (Discussed nicely in <em><a href="https://www.amazon.com/Art-Strategy-Theorists-Success-Business/dp/0393337170">The Art of Strategy</a></em>, the same book I referenced in <a href="https://offsquare.substack.com/p/burn-the-ships">Burn the Ships</a>.), It produces a result that sounds wrong the first time you hear it: getting better at a skill can cause you to use it <em>less</em>, not more.</p><p>Take tennis for example. Suppose your backhand is much weaker than your forehand. Your opponents are not stupid, so they play to your backhand relentlessly. You spend match after match hitting backhands under pressure, and over time, from all that forced practice, your backhand improves. Now the two strokes are roughly equal. Your opponents can no longer exploit the backhand, so they stop targeting it and start distributing their shots evenly. The result is that you get to use your better forehand <em>more often</em> than you did before. Improving the weakness was really a way of unlocking the strength.</p><blockquote><p>&#8220;If your backhand is much weaker than your forehand, your opponents will learn to play to your backhand. Eventually, as a result of all this backhand practice, your backhand will improve. As your two strokes become more equal, opponents can no longer exploit your weak backhand... You get to use your better forehand more often; this could be the real advantage of improving your backhand.&#8221;</p><p>&#8212;Avinash Dixit and Barry Nalebuff, <em>The Art of Strategy</em></p></blockquote><p>The same logic shows up in basketball. LeBron James is right-handed, and defenses know it, so they might concentrate on stopping his right hand. The reason they can&#8217;t do so exclusively is that his left hand, while weaker, is still good enough to punish them if they overcommit. If he spends an off-season improving his left hand, the defense responds by honoring both, which frees his dominant right hand more often than before. The stronger his weak hand gets, the less he has to rely on it, because the threat of it does the work.</p><p>The tennis backhand and the basketball left hand are effectively constraints on production. They are the bottlenecks that limit how much you can do with your forehand or right hand, and they are the things that the opposing defense can reliably exploit. When you improve those weaknesses, you dissolve the bottleneck. The result is that you get to use your strength more often, not less.</p><p>Now map this onto software. For a long time, for a lot of founders and operators and even many engineers, the production of software was the weak hand. It was the bottleneck, the thing that had to be delegated to a small and expensive group of technical people, the stroke the opposing defense could reliably exploit. The naive expectation, now that AI has made that weak hand strong, is that we will all spend our days happily coding because coding finally got easy.</p><p>Game theory predicts the opposite. When the bottleneck dissolves, you don&#8217;t pour more of your time into the thing that is no longer scarce. Your attention reallocates to whatever is now the binding constraint. In practice that constraint is planning, prioritization, and judgment, which is to say the strategic layer that decides what the cheap execution should be aimed at. This reallocation is the engine underneath the &#8220;more work, not less&#8221; observation.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/the-finite-work-fallacy?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/the-finite-work-fallacy?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/the-finite-work-fallacy?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><div><hr></div><h2>Execution Is Becoming Cheap</h2><p>I&#8217;ve written before, in <a href="https://offsquare.substack.com/p/on-imperfect-information">On Imperfect Information</a>, about how I think LLMs are a near-perfect replacement for action under near-perfect information. Software is commoditized to the precise extent that you can hand an agent perfect operating instructions: clear goals, crisp constraints, complete context, and an unambiguous definition of success. The hard part is turning a messy, partially observable problem into something close to perfect information.</p><p>A junior-to-mid engineer used to spend the majority of the workday writing code, with planning and prioritization as a thin layer on top, often supplied by someone more senior. I expect that ratio to invert. A growing share of that engineer&#8217;s time (potentially an outright majority) goes to planning, sequencing, and defining constraints. This is not management overhead bolted onto an engineering job. It becomes the core of the technical job, and not only for engineers. Product managers and designers are converging on the same shape, because the same shift is happening underneath all of them at once.</p><p>This is yet another possible explanation for the &#8220;paradox.&#8221; Why would automating work produce more work rather than less? Because the work that survives automation is the judgment-dense work, which is harder per unit of time and exists in effectively infinite supply.</p><div><hr></div><h2>The &#8220;Burstiness&#8221; of Technical Work</h2><p>I&#8217;ve <a href="https://substack.com/@fkodom/note/c-258691678">argued elsewhere</a> that the best work in a startup is &#8220;bursty&#8221; and phasic. The target is usually unclear at first, so you spend time talking to customers, brainstorming, and probing assumptions. But once the target comes into focus, you sprint at it with everything you have.</p><p>AI sharpens the &#8220;burstiness&#8221; of this cycle. If execution is cheap and getting cheaper, then &#8220;building something&#8221; largely reduces to a series of planning exercises, followed by intense periods of technical execution. The question becomes how clearly, precisely, and consistently you can define the order of tasks and the components that have to go into the system. Work that we used to file under management, technical leadership, or executive function is now a core part of what every individual contributor does.</p><div><hr></div><h2>The Second-Most-Important Thing</h2><p>So there is no paradox. There is a fallacy, and once you name it the rest follows. Work is not finite &#8212; your time and attention is. Automating the top of the queue does not shrink the queue, it promotes the next item into that slot.</p><p>The people who feel busier with all of this automation are not victims of some cruel irony. They are the ones who correctly reached for the second-most-important thing the moment the first one got cheap.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Subscribe to Off-Square for opinionated takes on AI, startups, VC, and the incentives underpinning them.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Burn the Ships]]></title><description><![CDATA[An attempt to analyze founder psychology through the lens of game theory.]]></description><link>https://offsquare.substack.com/p/burn-the-ships</link><guid isPermaLink="false">https://offsquare.substack.com/p/burn-the-ships</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Sun, 24 May 2026 23:18:09 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!y5tQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="native-audio-embed" data-component-name="AudioPlaceholder" data-attrs="{&quot;label&quot;:null,&quot;mediaUploadId&quot;:&quot;63fd57fa-6afd-4d6a-b102-c67c705065ec&quot;,&quot;duration&quot;:834.769,&quot;downloadable&quot;:true,&quot;isEditorNode&quot;:true}"></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!y5tQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!y5tQ!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png 424w, https://substackcdn.com/image/fetch/$s_!y5tQ!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png 848w, https://substackcdn.com/image/fetch/$s_!y5tQ!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png 1272w, https://substackcdn.com/image/fetch/$s_!y5tQ!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!y5tQ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png" width="1456" height="932" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:932,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:10287094,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/199120411?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!y5tQ!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png 424w, https://substackcdn.com/image/fetch/$s_!y5tQ!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png 848w, https://substackcdn.com/image/fetch/$s_!y5tQ!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png 1272w, https://substackcdn.com/image/fetch/$s_!y5tQ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73ea7d1f-db16-47b9-b857-1ba275245c51_3064x1962.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>In 1519, Hern&#225;n Cort&#233;s landed on the Mexican coast with about 500 men and eleven ships. Before marching inland to take on the Aztec Empire, he scuttled all but one of them. The story is usually told as motivational theater, a man with the spine to bet everything on a single roll. By eliminating retreat as an option, he changed the payoff matrix that every soldier under his command was solving. Cowardice and desertion were no longer strategies for his soldiers. The only rational move was to fight as hard as possible.</p><p>Avinash Dixit makes a version of this point in <em>The Art of Strategy</em>, and he traces the same pattern through Xenophon fighting with his back to an impassable ravine, William the Conqueror burning his fleet at Pevensey, and the Roman army making it a capital offense to fall behind in an attack. The mechanism is identical in every case. An advancing line is a coordination problem. A single soldier who hangs back slightly improves his personal odds of survival without meaningfully reducing the chance that his side wins. If everyone reasons that way, the attack collapses into a retreat. Commanders who can credibly remove the retreat option remove the bad equilibrium along with it.</p><p>The same mechanism describes, almost perfectly, what investors are actually looking for when they talk about a &#8220;missionary&#8221; founder.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Subscribe to Off-Square for opinionated takes on AI, startups, VC, and the incentives underpinning them.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><div><hr></div><h2>The Cort&#233;s Founder</h2><p>When external conditions are good, the math is easy for everyone involved in a startup. Demand is strong, the team is capable, capital is available, no competitor is breathing down the neck. The dominant strategy for the founder, the cofounders, and the early employees is to work hard and ride the upside. This is the equivalent of the army that is already winning. Every soldier wants to charge.</p><p>These ideal conditions can quickly fall apart for most startups.</p><ul><li><p>a pipeline stalls</p></li><li><p>a competitor raises a war chest</p></li><li><p>a key hire quits</p></li><li><p>the macro environment turns</p></li></ul><p>When these conditions compound, every individual at the company has an alternative that is rationalizable. The founder can wind the company down and take a senior role at a larger one. Cofounders can negotiate a soft landing. Senior engineers can interview anywhere. The expected value of &#8220;give up now&#8221; can quickly become competitive with the expected value of &#8220;keep pushing,&#8221; and the army is at risk of becoming a retreat.</p><p>A missionary founder is one for whom this second calculation does not work, because the alternatives have been removed from the matrix. Either the founder genuinely cannot imagine doing something else, or the consequences of stopping are so personally unacceptable that stopping is not on the table. The structural result is the same as Cort&#233;s on the beach: Quitting is a dominated strategy, the team reads this and updates its own expectations, and investors read this and write the check.</p><p>This is the part that gets lost in the standard pitch about &#8220;passion&#8221; and &#8220;drive.&#8221; The premium is not for the personality. It is for the structural property of having no exit. Two founders with identical r&#233;sum&#233;s and identical products are not equally good bets if one of them has a credible alternative and the other does not. The one without an alternative will outwork the other on the marginal Tuesday at 11 p.m. when the demo is broken, and over a five-year company-building cycle, those marginal Tuesdays compound into the entire delta.</p><div><hr></div><h2>Conviction vs Desperation</h2><p>Most founders arrive at this property from one of two directions, or a linear combination of them.</p><ol><li><p><strong>Conviction.</strong> The founder is so deeply committed to a particular technical or commercial worldview that abandoning it would feel worse than failing at it. Elon Musk and SpaceX is the canonical example. Several times during the 2008 era, the rational play was to let SpaceX die quietly and walk away with the PayPal money. Musk slept on the factory floor and put the last of his liquid net worth into the next launch. The framing inside his head was not &#8220;this venture has positive expected value.&#8221; It was &#8220;humanity needs to be multiplanetary, and I happen to be the person doing this.&#8221; That is a worldview, not a strategy, and it is not subject to revision when the bank account gets thin.</p></li><li><p><strong>Desperation.</strong> The cost of failure is severe enough that the founder will not let it happen. For most first-time founders, this looks like some mix of personal financial exposure, public embarrassment in front of former colleagues, the prospect of laying off people they recruited and care about, and the reputational hit of carrying a visible failure into the next decade of their career. None of these are existential. In aggregate, they are bad enough that &#8220;fail and move on&#8221; is psychologically not a real option. The founder is, in effect, buying their own version of Cort&#233;s&#8217;s burned ship through a credible commitment to social and financial stakes.</p></li></ol><p>These are not the same mechanism. They produce different micro-level behavior, especially under stress. A conviction-driven founder will spend a hard quarter doubling down on the original vision. A desperation-driven founder will spend the same quarter scrambling to protect the downside. The day-to-day instincts are not interchangeable, and any halfway competent investor or operator will read them differently in board meetings. But at the macro level, the effect is the same. Both keep the founder in the game past the point where the rational expected-value calculation says to stop. Cort&#233;s himself was acting from conviction. His soldiers, by his design, were acting from desperation. The game-theoretic output was identical: nobody on the beach was going to defect.</p><p>The harder question is what happens to founders who have already &#8220;won&#8221; by exiting a previous company, or raising a huge fund. The naive expectation is that money and reputation soften the structure. Once you have several hundred million dollars in the bank, a comfortable reputation, and a portfolio of other interests, the obvious defection penalties shrink. The missionary mindset should, in principle, weaken with success.</p><p>In practice, it usually does not, largely because of selection bias. Founders who succeed once and choose to do it again are not a random sample of wealthy people. They are the ones who found the experience compelling enough, or unfinished enough, to opt back in. Repeat founders are a positively selected pool. They are, on average, more risk-tolerant, more mission-driven, and more committed than the median first-timer.</p><p>To be clear -- missionary structure does not necessarily imply success. Most startups fail, including most startups led by deeply convicted founders. Plenty of obviously missionary founders have driven companies into walls, and plenty of measured founders have built generational businesses. Conviction is not a sufficient condition, and in cases like Theranos it can actively accelerate the failure. But holding the rest of the inputs roughly constant, the founder with no real path to defection has a narrow edge over a competitor working on something similar. That edge compounds, and over a long enough cycle it can be the difference between the company that survives a bad year and the one that does not.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/burn-the-ships?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/burn-the-ships?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/burn-the-ships?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><h2>Thrive Capital</h2><p>The recent <a href="https://colossus.com/article/joshua-kushner-thrive-new-world/">Colossus profile of Joshua Kushner and Thrive Capital</a> is a long read on exactly this question. I should be upfront that what follows is third-hand. The Colossus piece is itself an interpretation of Kushner and the people around him by a journalist with his own framing, and I am layering my own reading on top of his. I have never met Kushner. I am not claiming this is the definitive account of what is going on inside his head. I am saying that from the outside, this is a coherent psychological profile to extract, and that the profile happens to map cleanly onto the game theory I was already trying to describe.</p><p>By any reasonable measure, Kushner is done. Thrive&#8217;s eighth fund put nearly two billion dollars into Stripe at a fifty-billion-dollar valuation, and Stripe is now valued at over one hundred billion. Thrive&#8217;s early position in OpenAI, taken at twenty-nine billion when no other firm was willing to anchor the round, is now anchored to a five-hundred-billion-dollar company. The firm holds stakes in Instagram, GitHub, Wiz, Anduril, Databricks, Cursor, and Isomorphic Labs, among others. Kushner is thirty-nine, married, has three children, and has more than enough capital to never work again. He could close the fund tomorrow and pass generational wealth to his descendants without a meaningful change in his lifestyle.</p><p>He has not stopped, and the reporting is not subtle about why.</p><p>Kushner&#8217;s grandparents were survivors of the Novogrudok Ghetto massacres. His grandmother Rae watched her mother be murdered through a window, watched her older sister machine-gunned into a pit with five thousand other Jews, and escaped through a six-hundred-foot tunnel her family dug at night with spoons. His grandfather spent three winters living in a dirt hole in the Naliboki Forest after watching the Nazis execute his mother and several sisters. They arrived in the United States in 1949 with no money. By the time Joseph Kushner died in 1985, he had built a real estate empire in New Jersey. Two decades after that, Joshua&#8217;s father went to federal prison. The family the Kushners had built spent the first half of Joshua&#8217;s life being inducted into the American establishment, and the second half being publicly rejected by parts of it.</p><p>Read against this background, Kushner is not a counterexample to the missionary pattern. The Colossus piece argues, and I largely agree, that the structure in his case is generational rather than personal. The reason he keeps building is not that he has run out of money or status to acquire. It is that the alternative, for someone with his lineage, is not really an alternative. To stop is to accept that the world&#8217;s verdict on his family is allowed to settle, and he has direct, intimate, video-recorded evidence of what &#8220;settling&#8221; can look like for people like him. The wealth, marriage, children, and access to political power do not relax the structure. The ships were burned two generations ago, and he cannot un-burn them.</p><p>Kushner himself describes the mechanism in less abstract terms:</p><blockquote><p>&#8220;When you live with someone as resilient as she was and persevered through so much tragedy, you feel determined to make something of yourself. I am not first generation, but I grew up in a first-generation home. As a child, I was surrounded by the impossible.&#8221;</p><p>&#8212;Joshua Kushner</p></blockquote><p>The same caveat applies to the rest of the firm, but Thrive looks unusually full of people for whom a similar story can be told. Kareem Zaki, one of the firm&#8217;s lead partners, is the son of Coptic Christians who fled Egyptian religious persecution and watched his father repeat his entire medical training from scratch in Cleveland. Nabil Mallick, the COO, is the son of a Pakistani immigrant who washed dishes at a Mexican restaurant while getting his masters and eventually built a civil engineering firm out of New Jersey. Vince Hankes grew up in Mount Clemens, Michigan, watching the Great Recession dismantle the auto industry and most of the families around him. Nitin Nohria, the executive chairman, fled the Naxalite insurgency in Calcutta as a child. The piece reads like a roster of people who came to a similar conclusion through different paths. Idleness is not available, because the world that produced their parents and grandparents is the world that is always one decade away from returning.</p><p>This is what investors are actually asking when they ask a founder about their &#8220;why.&#8221; They are not asking for a marketing slogan. They are asking whether the founder has a structural reason for not being able to stop, and whether that reason is durable enough to survive the year when nothing is working.</p><div><hr></div><h2>The Diagnostic Questions</h2><p>The standard probing questions in a founder meeting collapse into the same test.</p><ul><li><p>Why do you want this?</p></li><li><p>How badly do you want it?</p></li><li><p>How much are you willing to suffer for it?</p></li><li><p>What would you do if it failed?</p></li><li><p>What would you do if you couldn&#8217;t do this?</p></li></ul><p>The questions sound like motivational filler, and most of the time they get answered like motivational filler. The answer that should make an investor sit up is the one that reveals the founder has not seriously considered the alternatives, because the alternatives are not real to them. &#8220;I don&#8217;t know what I&#8217;d do if it failed&#8221; sounds like a weakness in a job interview. In a fundraising meeting, it is the closest thing to a structural guarantee that the founder will not defect at the first sign of trouble.</p><p>A polished, well-reasoned answer is harder to read. Some founders have thought carefully about exit ramps because they are good strategic thinkers, not because they intend to use them. The signal an investor is actually trying to extract is not &#8220;have you considered alternatives&#8221; but &#8220;are the alternatives real to you.&#8221; Those are different questions. A founder can have a clean answer for what they would do if the company failed and still treat that scenario as functionally outside the decision tree. The diagnostic value of the question is in the texture of the answer, not in whether the answer exists.</p><p>This is also the part of the founder profile that is essentially impossible to coach. Conviction can be cultivated to some degree, but a worldview deep enough to override personal interest is not something a founder decides to acquire on a Tuesday. Desperation can be manufactured, but the kind that lasts through a five-year company-building cycle has to be backed by real stakes. The honest version of the missionary founder filter is not &#8220;we are looking for people with grit.&#8221; It is &#8220;we are looking for people whose lives have already done the hard work of removing their alternatives, because we know we cannot do that work for them.&#8221;</p><p>What the investor is paying for is not the founder&#8217;s grit. It is the structural fact that, somewhere upstream of the pitch meeting, the ships were already burned.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Subscribe to Off-Square for opinionated takes on AI, startups, VC, and the incentives underpinning them.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p></p>]]></content:encoded></item><item><title><![CDATA[State of Agents: May 2026]]></title><description><![CDATA[Attempting to map the agentic framework/infra landscape]]></description><link>https://offsquare.substack.com/p/state-of-agents-may-2026</link><guid isPermaLink="false">https://offsquare.substack.com/p/state-of-agents-may-2026</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Thu, 14 May 2026 02:58:51 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!frXr!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>As the founding engineer at an AI-focused venture fund, I work with many different companies, including internal incubation companies that we help create and launch. These vary widely in language, framework, infrastructure maturity, and product surface area.</p><p>For that reason, my goal isn&#8217;t to find a single &#8220;best&#8221; agent framework, but to identify a flexible, portable agent stack whose lessons transfer across projects. The ideal stack should make it easy to move between Python and TypeScript, swap model providers, preserve observability, avoid unnecessary cloud setup, translate between local/dev/prod environments, and prevent every company from being locked into one vendor&#8217;s end-to-end agent platform.</p><p>Cloud agnosticism matters to me more than it does to most projects. If the recommended stack requires a company to adopt Bedrock, Vertex, Kubernetes, or a large AWS/GCP/Azure setup, it won&#8217;t be usable by many of the companies I work with. They may already be on AWS, where a GCP-centric stack doesn&#8217;t translate. By contrast, more focused providers like Modal, Logfire, OpenRouter, Langfuse, and similar tools can usually be adopted regardless of whether the company&#8217;s main app runs on Vercel, AWS, GCP, Azure, Fly, Render, Kubernetes, or something else. The goal is to keep the agent infrastructure portable enough that it fits into the company&#8217;s existing deployment stack instead of forcing a major cloud decision.</p><p>The core recommendation is to standardize around <strong>portable model access, language-native agent code, OpenTelemetry-compatible observability, durable execution, and lightweight compute</strong>.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><div><hr></div><h2>Agent Stack</h2><p>The agent stack has six separable layers:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!frXr!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!frXr!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png 424w, https://substackcdn.com/image/fetch/$s_!frXr!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png 848w, https://substackcdn.com/image/fetch/$s_!frXr!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png 1272w, https://substackcdn.com/image/fetch/$s_!frXr!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!frXr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png" width="724" height="542.5027472527472" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/f0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1091,&quot;width&quot;:1456,&quot;resizeWidth&quot;:724,&quot;bytes&quot;:274471,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!frXr!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png 424w, https://substackcdn.com/image/fetch/$s_!frXr!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png 848w, https://substackcdn.com/image/fetch/$s_!frXr!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png 1272w, https://substackcdn.com/image/fetch/$s_!frXr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff0ff4b0f-36d8-4aae-a0ed-1dad93869b7f_1826x1368.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The preferred stack changes by use case:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!IYvj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!IYvj!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png 424w, https://substackcdn.com/image/fetch/$s_!IYvj!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png 848w, https://substackcdn.com/image/fetch/$s_!IYvj!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!IYvj!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!IYvj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png" width="1456" height="869" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:869,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:263522,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!IYvj!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png 424w, https://substackcdn.com/image/fetch/$s_!IYvj!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png 848w, https://substackcdn.com/image/fetch/$s_!IYvj!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!IYvj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F052ec48a-1a8a-4448-a952-6b527dbff323_1810x1080.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>This is more pragmatic than treating Temporal, LangGraph, or any other durable runtime as the default for every production agent.</p><p>For most agents, the ergonomic default is the <strong>OpenAI Agents SDK</strong>. Add <strong>E2B or Vercel Sandbox</strong> when the agent needs a place to execute code, install packages, manipulate files, or hold per-session state &#8212; Vercel Sandbox is the natural fit for Vercel/Next.js teams, E2B is the more provider-neutral default. Reach for <strong>Modal</strong> when the workload actually needs GPUs, beefier hardware, gVisor isolation, or Python-heavy ML compute. Add <strong>Vercel Workflows</strong> (Vercel/Next.js teams) or <strong>Temporal</strong> (selectively) when durable execution matters.</p><p>E2B, Vercel Sandbox, and Modal all integrate cleanly with the OpenAI Agents SDK, so the framework choice doesn&#8217;t lock you into any single compute layer.</p><div><hr></div><h2>Tier 1: Non-Production / Local / Exploratory Agents</h2><p>For one-off scripts, internal tools, local experiments, and non-production-facing workflows, use:</p><p><strong>OpenAI-compatible gateway + OpenAI Agents SDK + Logfire</strong></p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!6BbT!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!6BbT!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png 424w, https://substackcdn.com/image/fetch/$s_!6BbT!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png 848w, https://substackcdn.com/image/fetch/$s_!6BbT!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!6BbT!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!6BbT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png" width="1456" height="882" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:882,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:183025,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!6BbT!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png 424w, https://substackcdn.com/image/fetch/$s_!6BbT!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png 848w, https://substackcdn.com/image/fetch/$s_!6BbT!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!6BbT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04f0b45a-827a-4876-8ef5-aacc18a6e824_1782x1080.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>This is the right stack when the agent is mostly request/response, has limited side effects, and doesn&#8217;t need durable retries or long-running state.</p><p>OpenAI Agents SDK is especially attractive here because it includes built-in tracing for LLM generations, tool calls, handoffs, guardrails, and custom events out of the box.</p><p>Useful references:</p><ul><li><p>OpenAI Agents SDK: https://developers.openai.com/api/docs/guides/agents</p></li><li><p>OpenAI Agents SDK quickstart: https://developers.openai.com/api/docs/guides/agents/quickstart</p></li><li><p>OpenAI Agents SDK tracing: https://openai.github.io/openai-agents-python/tracing/</p></li><li><p>Logfire OpenAI integration: https://pydantic.dev/docs/logfire/integrations/llms/openai/</p></li></ul><div><hr></div><h2>Tier 2: Lightweight Production Agents</h2><p>My definition of &#8220;lightweight&#8221; is: agents that run for up to several minutes, have manageable retry and state management needs, and don&#8217;t have critical business impact if they fail occasionally. For these use cases, default to the same stack as Tier 1, but add a sandbox layer (E2B or Vercel Sandbox) for code execution, tool use, and per-session state:</p><p><strong>OpenAI-compatible gateway + OpenAI Agents SDK + Logfire + E2B or Vercel Sandbox</strong></p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!v3z2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!v3z2!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png 424w, https://substackcdn.com/image/fetch/$s_!v3z2!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png 848w, https://substackcdn.com/image/fetch/$s_!v3z2!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png 1272w, https://substackcdn.com/image/fetch/$s_!v3z2!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!v3z2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png" width="1456" height="1364" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1364,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:352187,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!v3z2!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png 424w, https://substackcdn.com/image/fetch/$s_!v3z2!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png 848w, https://substackcdn.com/image/fetch/$s_!v3z2!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png 1272w, https://substackcdn.com/image/fetch/$s_!v3z2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb442c662-cda2-43ee-acd1-a5cab98126a8_1778x1666.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>For very lightweight TypeScript/web workloads, a secondary option is:</p><p><strong>OpenAI-compatible gateway + OpenAI Agents SDK or Vercel AI SDK + Logfire + Vercel serverless + Vercel Sandbox</strong></p><p>Most agent sandboxes have modest hardware needs (a few vCPUs and a few GB of RAM); the properties that matter are startup time, ergonomics, and per-session state. Both E2B and Vercel Sandbox fit that shape &#8212; see the Layer-by-Layer compute section for a deeper comparison and the heavier-compute escalation to Modal.</p><p>In this tier, it usually isn&#8217;t necessary to add a durable execution framework. Vercel Functions cap out at 300 seconds (Hobby) / 800 seconds (Pro/Enterprise) per invocation, which is fine for most short agent loops but a problem for longer-running ones &#8212; that&#8217;s where Tier 3&#8217;s durable layer (Vercel Workflows or Temporal) comes in.</p><div><hr></div><h2>Tier 3: Durable Long-Running / High-Stakes Agents</h2><p>Use a durable execution layer when parts of the workflow are long-running, fragile, business-critical, or stateful enough that homegrown retry and checkpointing logic becomes risky. There are two solid defaults here: <strong>Vercel Workflows</strong> for Vercel/Next.js teams, and <strong>Temporal</strong> for platform-agnostic or deeply mission-critical workflows.</p><p>The stack becomes:</p><p><strong>OpenAI-compatible gateway + OpenAI Agents SDK + Vercel/Temporal Workflows + Logfire/OTLP + E2B or Vercel Sandbox (with Modal for GPU/heavy compute)</strong></p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!PAaR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!PAaR!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png 424w, https://substackcdn.com/image/fetch/$s_!PAaR!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png 848w, https://substackcdn.com/image/fetch/$s_!PAaR!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png 1272w, https://substackcdn.com/image/fetch/$s_!PAaR!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!PAaR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png" width="1456" height="1290" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1290,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:357397,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!PAaR!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png 424w, https://substackcdn.com/image/fetch/$s_!PAaR!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png 848w, https://substackcdn.com/image/fetch/$s_!PAaR!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png 1272w, https://substackcdn.com/image/fetch/$s_!PAaR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1872060c-cd49-49a9-be8c-56fc795d8740_1784x1580.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h3>Vercel Workflows as a hybrid orchestrator</h3><p>Vercel Workflows (GA April 2026) sits between a generic compute layer and a durable execution framework. It gives you durable agent loops via <code>use workflow</code> / <code>use step</code> directives, sleeps and hooks that pause for minutes/days/months without consuming compute, resumable streams that survive a client reconnect, and deployment pinning so workflows keep running on the deployment they started on.</p><p>The key thing is that Vercel Workflows is the <strong>agent runtime</strong>, not the <strong>agent body</strong>. Workflow steps run on Vercel Functions / Fluid Compute and inherit those limits &#8212; no GPUs, modest CPU/RAM, and per-step caps (~300s Hobby, ~800s Pro/Enterprise). Heavy work still belongs in Modal, and code-execution/tool-use sandboxes still belong in E2B or Vercel Sandbox.</p><p>For a Next.js app, that produces a coherent stack:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Vamj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Vamj!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png 424w, https://substackcdn.com/image/fetch/$s_!Vamj!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png 848w, https://substackcdn.com/image/fetch/$s_!Vamj!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png 1272w, https://substackcdn.com/image/fetch/$s_!Vamj!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Vamj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png" width="1456" height="589" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:589,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:115688,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Vamj!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png 424w, https://substackcdn.com/image/fetch/$s_!Vamj!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png 848w, https://substackcdn.com/image/fetch/$s_!Vamj!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png 1272w, https://substackcdn.com/image/fetch/$s_!Vamj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F67c5f602-8117-4fab-ab1c-bffcc0c70027_1680x680.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>For teams not on Vercel &#8212; or for workflows that need a language-agnostic, mission-critical engine with mature worker control, heartbeats, and explicit activity modeling &#8212; Temporal remains the right choice.</p><h3>Temporal as a durable boundary</h3><p>Temporal doesn&#8217;t need to replace the whole agent framework. A better, more incremental migration path is to use OpenAI Agents SDK for the main agent loop, then wrap with Temporal as needed.</p><p>For example:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!svg8!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!svg8!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png 424w, https://substackcdn.com/image/fetch/$s_!svg8!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png 848w, https://substackcdn.com/image/fetch/$s_!svg8!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png 1272w, https://substackcdn.com/image/fetch/$s_!svg8!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!svg8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png" width="1456" height="433" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:433,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:93531,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!svg8!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png 424w, https://substackcdn.com/image/fetch/$s_!svg8!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png 848w, https://substackcdn.com/image/fetch/$s_!svg8!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png 1272w, https://substackcdn.com/image/fetch/$s_!svg8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F278d0768-036d-416b-a3b4-a4f6844f043b_1680x500.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>This lets the team apply Temporal only where its durability semantics matter, while sandbox and heavy-compute concerns continue to live in E2B/Vercel Sandbox and Modal respectively.</p><p>Temporal makes sense when a workflow has one or more of these properties:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!gwCc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!gwCc!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png 424w, https://substackcdn.com/image/fetch/$s_!gwCc!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png 848w, https://substackcdn.com/image/fetch/$s_!gwCc!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png 1272w, https://substackcdn.com/image/fetch/$s_!gwCc!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!gwCc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png" width="1456" height="837" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/bf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:837,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:176366,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!gwCc!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png 424w, https://substackcdn.com/image/fetch/$s_!gwCc!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png 848w, https://substackcdn.com/image/fetch/$s_!gwCc!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png 1272w, https://substackcdn.com/image/fetch/$s_!gwCc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf4a7f75-fed1-4249-8d74-ede5ce0a9cd4_1784x1026.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The goal isn&#8217;t to make every agent step a Temporal workflow &#8212; it&#8217;s to identify the parts of the agent graph where durability actually matters and put Temporal boundaries there. Sometimes the entire agent workflow is durable; more often, only one or two critical steps need Temporal while the rest stays in the more ergonomic OpenAI Agents SDK.</p><p>Temporal&#8217;s local dev experience is reasonable for teams already using Docker Compose. A local Temporal service can sit alongside Postgres, Redis, or S3-compatible storage:</p><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!AdI3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!AdI3!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png 424w, https://substackcdn.com/image/fetch/$s_!AdI3!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png 848w, https://substackcdn.com/image/fetch/$s_!AdI3!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png 1272w, https://substackcdn.com/image/fetch/$s_!AdI3!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!AdI3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png" width="1456" height="313" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:313,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:56708,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!AdI3!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png 424w, https://substackcdn.com/image/fetch/$s_!AdI3!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png 848w, https://substackcdn.com/image/fetch/$s_!AdI3!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png 1272w, https://substackcdn.com/image/fetch/$s_!AdI3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19a0ff44-9ac3-4c04-bc48-fb218d90cf81_1672x360.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a></figure></div><p>This gives teams a local Temporal service and Web UI without requiring Temporal Cloud or running production infrastructure for small experiments.</p><p>Rule of thumb:</p><blockquote><p>Use OpenAI Agents SDK for the agent loop. Use Vercel Workflows (Vercel/Next.js teams) or Temporal (selectively) for durable orchestration. Use E2B or Vercel Sandbox for the agent&#8217;s sandbox. Use Modal for compute-heavy execution and GPUs.</p></blockquote><p>Useful references:</p><ul><li><p>Temporal local development server: https://docs.temporal.io/develop/run-a-development-server</p></li><li><p>Temporal CLI server docs: https://docs.temporal.io/cli/server</p></li><li><p>Temporal Docker image: https://hub.docker.com/r/temporalio/temporal</p></li><li><p>Temporal OpenAI Agents SDK integration: https://docs.temporal.io/ai-cookbook/openai-agents-sdk-python</p></li><li><p>Temporal agentic loop example: https://docs.temporal.io/ai-cookbook/agentic-loop-tool-call-openai-python</p></li><li><p>Temporal Python integrations: https://docs.temporal.io/develop/python/integrations</p></li><li><p>Temporal TypeScript integrations: https://docs.temporal.io/develop/typescript/integrations</p></li></ul><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/state-of-agents-may-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/state-of-agents-may-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/state-of-agents-may-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><div><hr></div><h2>Why Not Default to Temporal?</h2><p>As powerful as Temporal is, it adds complexity, latency, and one more layer of abstraction between the agent and the user/developer. Its local development story is workable, but still less ergonomic than running a pure SDK-based agent locally. Its integrations are useful, but not uniformly mature across Python and TypeScript &#8212; the TypeScript AI SDK integration in particular should be treated with caution while it&#8217;s still in preview.</p><p>The latency cost is real. Every Temporal-backed step crosses the boundary between application code and the Temporal worker/server: workflow tasks, activity tasks, and event-history persistence all add round trips that a pure in-process agent loop doesn&#8217;t pay. For short, latency-sensitive agent steps, that overhead can be larger than the work being durable-wrapped in the first place.</p><p>Temporal&#8217;s abstractions also shift how the developer reasons about the agent: code splits into workflows and activities, side effects must be pushed to activity boundaries, retries and timeouts move into Temporal config, and debugging often means reading event histories rather than stack traces. That&#8217;s a worthwhile trade for genuinely durable workflows, but it&#8217;s a meaningful tax on the day-to-day developer and user experience when the workflow doesn&#8217;t actually need durability.</p><p>For lightweight production agents, it isn&#8217;t a huge lift to build basic retry, state hints, checkpointing, and error recovery directly in Python or TypeScript. If the workflow only runs for a few minutes and occasional failures are acceptable or easily recoverable, Temporal is more overhead than value.</p><p>The better escalation path is:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!R3mW!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!R3mW!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png 424w, https://substackcdn.com/image/fetch/$s_!R3mW!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png 848w, https://substackcdn.com/image/fetch/$s_!R3mW!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png 1272w, https://substackcdn.com/image/fetch/$s_!R3mW!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!R3mW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png" width="1456" height="1042" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1042,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:223198,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!R3mW!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png 424w, https://substackcdn.com/image/fetch/$s_!R3mW!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png 848w, https://substackcdn.com/image/fetch/$s_!R3mW!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png 1272w, https://substackcdn.com/image/fetch/$s_!R3mW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F61205926-5905-4fea-9c9e-f317cf4fcee4_1766x1264.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Each layer earns its place by solving a specific problem: OpenAI Agents SDK for building the agent, E2B/Vercel Sandbox for a sandbox to act inside, Modal for heavy compute or GPUs, Vercel Workflows or Temporal for durable execution. Don&#8217;t pull in any of them before the corresponding problem actually shows up.</p><div><hr></div><h2>The LangChain Stack</h2><p>The most common/popular alternative to my preferred stack is:</p><p><strong>LangGraph + LangSmith + LangSmith Deployment</strong></p><p>This is popular because it&#8217;s the closest thing to a one-stop-shop for agent development, tracing, evals, and deployment.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!_b_Y!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!_b_Y!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png 424w, https://substackcdn.com/image/fetch/$s_!_b_Y!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png 848w, https://substackcdn.com/image/fetch/$s_!_b_Y!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png 1272w, https://substackcdn.com/image/fetch/$s_!_b_Y!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!_b_Y!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png" width="1456" height="483" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:483,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:100448,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!_b_Y!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png 424w, https://substackcdn.com/image/fetch/$s_!_b_Y!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png 848w, https://substackcdn.com/image/fetch/$s_!_b_Y!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png 1272w, https://substackcdn.com/image/fetch/$s_!_b_Y!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6c3cb46-f96a-4ea0-8500-3308dcbf818b_1760x584.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>This stack is attractive if a team wants an integrated agent platform and is already comfortable with LangChain abstractions.</p><p>Personally, I wouldn&#8217;t treat it as the generic default. LangSmith can trace non-LangChain frameworks, but deploying those frameworks through LangSmith generally means wrapping them in LangGraph-style entrypoints. That&#8217;s useful, but it&#8217;s still a form of lock-in -- everything eventually becomes LangGraph-shaped.</p><p>LangSmith Evals are a real advantage. Logfire doesn&#8217;t give you a full eval platform out of the box. But generic traces plus separate scoring jobs can cover most eval workflows while keeping the stack more portable.</p><p><strong>Langfuse is also worth mentioning here, but it isn&#8217;t part of the LangChain stack.</strong> It&#8217;s an independent open-source LLM observability/evals platform, and a good alternative to Logfire or LangSmith when self-hosting or OSS posture matters.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!IYLK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!IYLK!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png 424w, https://substackcdn.com/image/fetch/$s_!IYLK!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png 848w, https://substackcdn.com/image/fetch/$s_!IYLK!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png 1272w, https://substackcdn.com/image/fetch/$s_!IYLK!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!IYLK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png" width="1456" height="1035" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1035,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:212868,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!IYLK!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png 424w, https://substackcdn.com/image/fetch/$s_!IYLK!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png 848w, https://substackcdn.com/image/fetch/$s_!IYLK!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png 1272w, https://substackcdn.com/image/fetch/$s_!IYLK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F85c738fd-cc96-4ad2-ba18-23a258c15c38_1770x1258.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Useful references:</p><ul><li><p>LangSmith Deployment: https://docs.langchain.com/langsmith/deployment</p></li><li><p>Deploying other frameworks to LangSmith: https://docs.langchain.com/langsmith/deploy-other-frameworks</p></li><li><p>Langfuse observability: https://langfuse.com/docs/observability/overview</p></li></ul><div><hr></div><h2>The Claude / Anthropic Stack</h2><p>Anthropic deserves its own category. The Claude stack is more specialized than the LangChain stack, but it&#8217;s significant because it&#8217;s very strong for coding agents and file-based workflows.</p><p>The relevant stack is roughly:</p><p><strong>Claude API + Claude Agent SDK + Claude Managed Agents</strong></p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!t97I!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!t97I!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png 424w, https://substackcdn.com/image/fetch/$s_!t97I!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png 848w, https://substackcdn.com/image/fetch/$s_!t97I!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png 1272w, https://substackcdn.com/image/fetch/$s_!t97I!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!t97I!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png" width="1456" height="933" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:933,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:258845,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!t97I!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png 424w, https://substackcdn.com/image/fetch/$s_!t97I!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png 848w, https://substackcdn.com/image/fetch/$s_!t97I!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png 1272w, https://substackcdn.com/image/fetch/$s_!t97I!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d1b4fd0-b69b-4d71-9ed3-46dbc78ecea8_1758x1126.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The biggest advantage of the Claude stack is that Claude Agent SDK exposes the same core tools, agent loop, and context management that power Claude Code. It can read files, run commands, search the web, edit code, use hooks, subagents, MCP, permissions, sessions, and more. That makes it especially attractive for coding agents, repository agents, local-file workflows, CI agents, migration agents, and internal developer tooling.</p><p>The biggest downside is lock-in. Claude Agent SDK is built around Anthropic&#8217;s API and Claude Code-style execution model. Anthropic does offer an OpenAI SDK compatibility layer, but its own docs describe that layer as intended for testing and comparison, not as the long-term production path for most use cases. Several Claude-native features require the native Claude API.</p><p>Claude Managed Agents takes this further by offering a managed harness and infrastructure for running Claude as an autonomous agent. It can run long-running sessions in managed containers, persist event history, read and write files, run shell commands, browse the web, and execute code securely. This is powerful, but it couples the application to Anthropic&#8217;s model API, agent harness, managed runtime, and deployment environment.</p><blockquote><p>This LLM provider lock-in makes me especially nervous. A lot of developers have already switched from Claude Code to other coding agents (either local or another provider), because of Anthropic&#8217;s (1) restrictive rate limits and (2) high costs for models like Opus 4.7. The LLM is a critical, low-level dependency for agentic products, and I&#8217;d hesitate to lock my company into a single provider&#8217;s models and API.</p></blockquote><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!k19y!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!k19y!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png 424w, https://substackcdn.com/image/fetch/$s_!k19y!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png 848w, https://substackcdn.com/image/fetch/$s_!k19y!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png 1272w, https://substackcdn.com/image/fetch/$s_!k19y!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!k19y!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png" width="1456" height="778" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:778,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:213186,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!k19y!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png 424w, https://substackcdn.com/image/fetch/$s_!k19y!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png 848w, https://substackcdn.com/image/fetch/$s_!k19y!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png 1272w, https://substackcdn.com/image/fetch/$s_!k19y!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa2322598-c332-4e51-b9dd-bb74bbfa9b95_1756x938.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Recommendation: treat Claude Agent SDK as a strong specialized choice for coding agents, not the default generic agent framework.</p><p>Useful references:</p><ul><li><p>Claude Agent SDK overview: https://code.claude.com/docs/en/agent-sdk/overview</p></li><li><p>Claude Agent SDK observability: https://code.claude.com/docs/en/agent-sdk/observability</p></li><li><p>Claude Managed Agents overview: https://platform.claude.com/docs/en/managed-agents/overview</p></li><li><p>Anthropic OpenAI SDK compatibility: https://platform.claude.com/docs/en/api/openai-sdk</p></li><li><p>Anthropic Managed Agents engineering writeup: https://www.anthropic.com/engineering/managed-agents</p></li></ul><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><div><hr></div><h2>Layer-by-Layer Recommendations</h2><h3>1. LLM / AI Gateway</h3><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!OHoc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!OHoc!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png 424w, https://substackcdn.com/image/fetch/$s_!OHoc!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png 848w, https://substackcdn.com/image/fetch/$s_!OHoc!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png 1272w, https://substackcdn.com/image/fetch/$s_!OHoc!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!OHoc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png" width="1456" height="1808" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1808,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" title="" srcset="https://substackcdn.com/image/fetch/$s_!OHoc!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png 424w, https://substackcdn.com/image/fetch/$s_!OHoc!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png 848w, https://substackcdn.com/image/fetch/$s_!OHoc!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png 1272w, https://substackcdn.com/image/fetch/$s_!OHoc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd06598e3-92dc-433b-9212-bbd9dbf3ac52_1580x1962.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Recommended default:</strong> preserve an <strong>OpenAI-compatible boundary</strong> wherever possible.</p><p>The gateway should be boring. Route requests, manage keys, track spend, support fallbacks, and avoid trapping the app behind one provider-specific interface.</p><p>Useful references:</p><ul><li><p>Vercel AI Gateway: https://vercel.com/docs/ai-gateway</p></li><li><p>Pydantic AI Gateway: https://pydantic.dev/docs/ai/overview/gateway/</p></li><li><p>OpenRouter API docs: https://openrouter.ai/docs/api/reference/overview</p></li><li><p>OpenRouter FAQ: https://openrouter.ai/docs/faq</p></li><li><p>Anthropic OpenAI SDK compatibility: https://platform.claude.com/docs/en/api/openai-sdk</p></li></ul><div><hr></div><h3>2. Agent Framework</h3><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Mhdw!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Mhdw!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png 424w, https://substackcdn.com/image/fetch/$s_!Mhdw!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png 848w, https://substackcdn.com/image/fetch/$s_!Mhdw!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png 1272w, https://substackcdn.com/image/fetch/$s_!Mhdw!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Mhdw!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png" width="1456" height="1203" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1203,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:325626,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Mhdw!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png 424w, https://substackcdn.com/image/fetch/$s_!Mhdw!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png 848w, https://substackcdn.com/image/fetch/$s_!Mhdw!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png 1272w, https://substackcdn.com/image/fetch/$s_!Mhdw!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe21277f7-7f59-464f-83da-cf04a4b3b81d_1614x1334.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Recommended default:</strong> OpenAI Agents SDK.</p><p>Use PydanticAI for Python-heavy Pydantic shops, Vercel AI SDK for TypeScript-heavy product teams, and LangGraph when the team has already chosen LangChain.</p><p>Use Claude Agent SDK when the agent is meaningfully Claude Code-like: codebase navigation, file edits, shell commands, repo repair, migration agents, test-writing agents, or local developer automation.</p><p>Useful references:</p><ul><li><p>OpenAI Agents SDK: https://developers.openai.com/api/docs/guides/agents</p></li><li><p>PydanticAI overview: https://pydantic.dev/docs/ai/overview/</p></li><li><p>Vercel AI SDK agents: https://ai-sdk.dev/docs/agents/overview</p></li><li><p>Claude Agent SDK overview: https://code.claude.com/docs/en/agent-sdk/overview</p></li></ul><div><hr></div><h3>3. Observability</h3><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!TBIA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!TBIA!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png 424w, https://substackcdn.com/image/fetch/$s_!TBIA!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png 848w, https://substackcdn.com/image/fetch/$s_!TBIA!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png 1272w, https://substackcdn.com/image/fetch/$s_!TBIA!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!TBIA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png" width="1456" height="1212" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1212,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:296558,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!TBIA!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png 424w, https://substackcdn.com/image/fetch/$s_!TBIA!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png 848w, https://substackcdn.com/image/fetch/$s_!TBIA!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png 1272w, https://substackcdn.com/image/fetch/$s_!TBIA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a2633d-64de-43fa-975e-740e43c8fc25_1766x1470.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Recommended default:</strong> Logfire.</p><p>Use LangSmith when eval UX and LangGraph-native workflows matter more than portability. Use Langfuse when open-source/self-hosted observability is important. Use Claude Agent SDK&#8217;s OTLP export when the agent is intentionally Claude-native.</p><p>Useful references:</p><ul><li><p>Logfire OpenAI integration: https://pydantic.dev/docs/logfire/integrations/llms/openai/</p></li><li><p>Logfire general observability: https://pydantic.dev/logfire</p></li><li><p>PydanticAI Logfire integration: https://pydantic.dev/docs/ai/integrations/logfire/</p></li><li><p>Vercel AI SDK + Logfire / OTEL: https://pydantic.dev/articles/vercel-ai-sdk-logfire-otel</p></li><li><p>Langfuse observability: https://langfuse.com/docs/observability/overview</p></li><li><p>Claude Agent SDK observability: https://code.claude.com/docs/en/agent-sdk/observability</p></li></ul><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/state-of-agents-may-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/state-of-agents-may-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/state-of-agents-may-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><div><hr></div><h3>4. Raw Compute</h3><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!vYHh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!vYHh!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png 424w, https://substackcdn.com/image/fetch/$s_!vYHh!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png 848w, https://substackcdn.com/image/fetch/$s_!vYHh!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png 1272w, https://substackcdn.com/image/fetch/$s_!vYHh!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!vYHh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png" width="1456" height="1537" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/ac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1537,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:505898,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!vYHh!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png 424w, https://substackcdn.com/image/fetch/$s_!vYHh!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png 848w, https://substackcdn.com/image/fetch/$s_!vYHh!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png 1272w, https://substackcdn.com/image/fetch/$s_!vYHh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac5d7bc9-10fa-4ebf-8f73-a32d6f63caa9_1788x1888.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!UU9V!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!UU9V!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png 424w, https://substackcdn.com/image/fetch/$s_!UU9V!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png 848w, https://substackcdn.com/image/fetch/$s_!UU9V!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png 1272w, https://substackcdn.com/image/fetch/$s_!UU9V!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!UU9V!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png" width="1456" height="1425" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1425,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:389449,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!UU9V!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png 424w, https://substackcdn.com/image/fetch/$s_!UU9V!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png 848w, https://substackcdn.com/image/fetch/$s_!UU9V!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png 1272w, https://substackcdn.com/image/fetch/$s_!UU9V!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ad8b7fd-a9cc-4654-9705-057b102cf3eb_1766x1728.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Recommended default:</strong> Vercel Workflows (Vercel/Next.js teams) as the durable orchestrator and lightweight host for the agent loop; E2B or Vercel Sandbox for the agent&#8217;s sandbox; Modal for GPU and heavy compute.</p><blockquote><p><strong>Note:</strong> Vercel Workflows is my only recommended framework that is not language-agnostic. The Workflows SDK only exists for TypeScript, but the developer experience is <strong>so good</strong> that I think it&#8217;s worth it. It actually does not lock you into the Vercel ecosystem either, because the Workflows SDK can run locally or anywhere with a Node.js runtime.</p></blockquote><p>Vercel Workflows is the <strong>agent runtime</strong>, not the <strong>agent body</strong> &#8212; it handles durable steps, sleeps, hooks, retries, and resumable streams, but its steps run on Vercel Functions / Fluid Compute and inherit those limits (no GPU, modest CPU/RAM, ~300s Hobby / ~800s Pro per step). Heavy work and code-execution sandboxes still belong in dedicated layers. Outside the Vercel ecosystem, the equivalent role is filled by Temporal (selectively) plus whatever web/runtime compute the rest of the app uses.</p><p>Most agentic workloads have modest hardware needs but care a lot about sandbox startup time, ergonomics, and per-session state &#8212; which is exactly the shape E2B and Vercel Sandbox are built for. Pick Vercel Sandbox when the rest of the app already lives on Vercel/Next.js; pick E2B when provider neutrality, longer session limits, or memory-preserving pause/resume matter more.</p><p>Reach for Modal only when the workload actually needs GPUs, beefier hardware, gVisor isolation, or Python-heavy ML compute (model serving, embeddings, batch inference, fine-tuning, image generation). A common production pattern is Vercel Workflows orchestrating the agent loop, E2B or Vercel Sandbox for the agent&#8217;s sandbox, and Modal-hosted endpoints for the heavy bits.</p><p>A useful split:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!J8Rh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!J8Rh!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png 424w, https://substackcdn.com/image/fetch/$s_!J8Rh!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png 848w, https://substackcdn.com/image/fetch/$s_!J8Rh!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png 1272w, https://substackcdn.com/image/fetch/$s_!J8Rh!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!J8Rh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png" width="1456" height="1261" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1261,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:270744,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!J8Rh!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png 424w, https://substackcdn.com/image/fetch/$s_!J8Rh!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png 848w, https://substackcdn.com/image/fetch/$s_!J8Rh!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png 1272w, https://substackcdn.com/image/fetch/$s_!J8Rh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F48a7771b-e471-482f-a9c7-2be223be75eb_1774x1536.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Useful references:</p><ul><li><p>OpenAI Agents SDK sandbox providers: https://developers.openai.com/api/docs/guides/agents/sandboxes#sandbox-providers</p></li><li><p>E2B documentation: https://e2b.dev/docs</p></li><li><p>Vercel Sandbox: https://vercel.com/docs/vercel-sandbox</p></li><li><p>Building an agent with OpenAI Agents SDK and Vercel Sandbox: https://vercel.com/kb/guide/building-an-agent-with-openai-agents-sdk-and-vercel-sandbox</p></li><li><p>Modal GPUs: https://modal.com/docs/guide/gpu</p></li><li><p>Modal sandboxes: https://modal.com/docs/guide/sandbox</p></li><li><p>Vast.ai docs: https://docs.vast.ai/</p></li><li><p>Vercel Workflows: https://vercel.com/docs/workflows</p></li><li><p>Bedrock AgentCore: https://docs.aws.amazon.com/bedrock-agentcore/</p></li><li><p>Google Agent Runtime: https://docs.cloud.google.com/gemini-enterprise-agent-platform/build/runtime</p></li><li><p>Claude Managed Agents: https://platform.claude.com/docs/en/managed-agents/overview</p></li></ul><div><hr></div><h3>5. Durable Execution</h3><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!etF4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!etF4!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png 424w, https://substackcdn.com/image/fetch/$s_!etF4!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png 848w, https://substackcdn.com/image/fetch/$s_!etF4!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png 1272w, https://substackcdn.com/image/fetch/$s_!etF4!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!etF4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png" width="1456" height="1810" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1810,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:406302,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/197621385?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!etF4!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png 424w, https://substackcdn.com/image/fetch/$s_!etF4!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png 848w, https://substackcdn.com/image/fetch/$s_!etF4!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png 1272w, https://substackcdn.com/image/fetch/$s_!etF4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8dc7b4e-ab4e-4df1-9a84-a78896925a1c_1456x1810.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Recommended default:</strong> application-level retries/checkpoints for lightweight agents; <strong>Vercel Workflows</strong> when the team is on Vercel/Next.js; <strong>Temporal</strong> when the workflow is platform-agnostic or mission-critical enough to justify its depth.</p><p>Don&#8217;t reach for any durable runtime just because the agent is production-facing. Reach for one when a workflow or sub-workflow is long-running, failure-prone, high-stakes, or stateful enough that simple retry and checkpointing logic becomes a liability.</p><p>Useful references:</p><ul><li><p>Vercel Workflows: https://vercel.com/docs/workflows</p></li></ul><ul><li><p>Vercel Workflow SDK: https://github.com/vercel/workflow</p></li></ul><ul><li><p>Temporal OpenAI Agents SDK integration: https://docs.temporal.io/ai-cookbook/openai-agents-sdk-python</p></li></ul><ul><li><p>Temporal OpenAI Agents SDK Python integration repo: https://github.com/temporalio/sdk-python/blob/main/temporalio/contrib/openai_agents/README.md</p></li></ul><ul><li><p>Temporal local development server: https://docs.temporal.io/develop/run-a-development-server</p></li></ul><ul><li><p>Claude Managed Agents: https://platform.claude.com/docs/en/managed-agents/overview</p></li></ul><ul><li><p>LangSmith Deployment: https://docs.langchain.com/langsmith/deployment</p></li></ul><div><hr></div><h2>TL;DR</h2><p>General advice:</p><ol><li><p>Start simple, but don&#8217;t build accidental infrastructure.</p></li><li><p>Pick the frameworks based on language and use case.</p></li><li><p>Avoid vendor lock-in where possible.</p></li><li><p>Standardize on layers, not platforms.</p></li></ol><p>Implementation advice:</p><ul><li><p>Start with OpenAI Agents SDK.</p></li><li><p>Add Logfire or another OTLP-compatible observability platform.</p></li><li><p>Add E2B (provider-neutral) or Vercel Sandbox (Vercel/Next.js-native) when agents need a sandbox to act inside.</p></li><li><p>Add Modal when agents need GPUs, beefy hardware, or Python-heavy ML compute.</p></li><li><p>Add Vercel Workflows when a Vercel/Next.js app needs durable agent loops, retries, sleeps, hooks, or resumable streams.</p></li><li><p>Add Temporal when the workflow is mission-critical enough to justify it.</p></li></ul><div><hr></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[On Power Laws]]></title><description><![CDATA[and trying to explain venture capital through "coin flip" games]]></description><link>https://offsquare.substack.com/p/on-power-laws</link><guid isPermaLink="false">https://offsquare.substack.com/p/on-power-laws</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Fri, 10 Apr 2026 01:01:09 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!Q-td!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="native-audio-embed" data-component-name="AudioPlaceholder" data-attrs="{&quot;label&quot;:null,&quot;mediaUploadId&quot;:&quot;5e601a78-7568-4e6c-a680-7802848baa4c&quot;,&quot;duration&quot;:1015.0661,&quot;downloadable&quot;:true,&quot;isEditorNode&quot;:true}"></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Q-td!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Q-td!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png 424w, https://substackcdn.com/image/fetch/$s_!Q-td!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png 848w, https://substackcdn.com/image/fetch/$s_!Q-td!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png 1272w, https://substackcdn.com/image/fetch/$s_!Q-td!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Q-td!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png" width="1456" height="727" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:727,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:8806941,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/193711448?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Q-td!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png 424w, https://substackcdn.com/image/fetch/$s_!Q-td!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png 848w, https://substackcdn.com/image/fetch/$s_!Q-td!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png 1272w, https://substackcdn.com/image/fetch/$s_!Q-td!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe20e0abf-c10e-4d9f-a4b8-4763d3141951_3054x1524.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p></p><blockquote><p><em><strong>Does the venture capital funding schedule create the power law, or does the power law necessitate the venture capital funding schedule?</strong></em></p></blockquote><p>Every venture capitalist I have ever spoken with treats the power law as received wisdom. A small number of investments generate the vast majority of returns, while most return little or nothing. The numbers are stark: roughly 4% of venture investments produce the bulk of a fund&#8217;s gains, and about 65% return less than the invested capital. Y Combinator is a famous case. In Paul Graham&#8217;s 2012 essay <a href="https://paulgraham.com/swan.html">Black Swan Farming</a>, he noted that just two of YC&#8217;s investments -- Airbnb and Dropbox -- accounted for about three-quarters of the portfolio&#8217;s total value. Horsley Bridge, a major LP, found that just 5% of capital deployed between 1985 and 2014 generated half of all returns.</p><p>The entire industry is structured around this fact. Fund strategy, portfolio construction, deal terms, follow-on reserves -- all of it assumes the power law as a given. But I have never heard a satisfying explanation for <em>why</em> venture returns follow a power law in the first place. Is the funding structure merely capitalizing on a naturally occurring distribution? Or is it actively producing the distribution it claims to exploit? To answer that, we need to look at the mathematics of power laws and then map those mathematics onto the mechanics of venture fundraising.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h2><strong>How Venture Fundraising Works</strong></h2><p>Venture capital fundraising is broken into stages designed to reduce risk for each investor and size investments appropriately. The standard schedule starts with a pre-seed round, then seed, then Series A, B, C, D, and so on, for however many rounds the company needs before it either gets acquired or goes public. In each round, the company issues new shares and gives equity to investors in return for capital. A typical round dilutes existing shareholders by around 20%.</p><p>Each successive stage assumes a certain level of maturity for the company, whether that is measured in employees, revenue, or product traction, as well as a certain amount of risk for the investor. The amount of capital invested typically increases with each round, and some percentage of companies that raised the previous round fail to survive long enough to raise the next one.</p><p>To ground this in recent data: median round sizes range from about $500K-$1M at pre-seed, to $2-4M at seed, $10-12M at Series A, ~$30M at Series B, ~$50M at Series C, and $100M or more at Series D and beyond. Post-money valuations follow a similar exponential curve: $5-6M at pre-seed, $12-15M at seed, $40-45M at Series A, $50-105M at Series B, $100-250M at Series C, and $500M or more at Series D.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a></p><p>Meanwhile, the graduation rates tell the other half of the story. About 67% of pre-seed-funded startups raise a seed round, but only about 33% of seed-funded startups make it to Series A. That seed-to-A chokepoint is the narrowest bottleneck in the venture funnel. After Series A, the odds improve: roughly 65% of Series A companies raise a Series B, about 75% of Series B companies reach Series C, and 80% or more of Series C companies raise a Series D. The funnel is widest at the top and tightest at the seed-to-A transition, then gradually relaxes as the surviving companies prove themselves.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-2" href="#footnote-2" target="_self">2</a></p><p>It&#8217;s not immediately obvious from this description alone why venture outcomes should follow a power law. To see why, we need to understand what a power law actually is, mathematically, and what kinds of processes generate one.</p><h2><strong>Three Coin-Flip Games</strong></h2><p>To build some intuition for power laws, let&#8217;s compare three different games involving coin flips, which produce three fundamentally different probability distributions.</p><p><strong>Game 1: Additive payoffs.</strong> You flip a fair coin 100 times. For every heads, you earn $1.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!vA4o!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!vA4o!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png 424w, https://substackcdn.com/image/fetch/$s_!vA4o!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png 848w, https://substackcdn.com/image/fetch/$s_!vA4o!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!vA4o!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!vA4o!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png" width="1456" height="887" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:887,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Game 1: Decision Tree&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Game 1: Decision Tree" title="Game 1: Decision Tree" srcset="https://substackcdn.com/image/fetch/$s_!vA4o!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png 424w, https://substackcdn.com/image/fetch/$s_!vA4o!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png 848w, https://substackcdn.com/image/fetch/$s_!vA4o!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!vA4o!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb3f719d5-2f94-4d53-a8a2-c533e510b682_1773x1080.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>By the central limit theorem, your expected earnings are $50, and the distribution of outcomes across many players approximates a normal distribution centered around $50. The fair price to play this game is anything below $50.</p><p>This is the distribution most people intuitively expect. Outcomes cluster around the mean, and extreme results are rare.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!QfZR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!QfZR!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png 424w, https://substackcdn.com/image/fetch/$s_!QfZR!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png 848w, https://substackcdn.com/image/fetch/$s_!QfZR!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png 1272w, https://substackcdn.com/image/fetch/$s_!QfZR!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!QfZR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png" width="1456" height="819" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/dee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:819,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Game 1: Additive Payoffs&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Game 1: Additive Payoffs" title="Game 1: Additive Payoffs" srcset="https://substackcdn.com/image/fetch/$s_!QfZR!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png 424w, https://substackcdn.com/image/fetch/$s_!QfZR!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png 848w, https://substackcdn.com/image/fetch/$s_!QfZR!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png 1272w, https://substackcdn.com/image/fetch/$s_!QfZR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdee74778-b6ca-4c53-9c15-d5a357049f35_1600x900.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Game 2: Multiplicative payoffs with symmetric risk.</strong> You start with $1. For each of 100 flips, if you get heads, your current amount is multiplied by 1.1. If tails, it is multiplied by 0.9.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!0fAZ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!0fAZ!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png 424w, https://substackcdn.com/image/fetch/$s_!0fAZ!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png 848w, https://substackcdn.com/image/fetch/$s_!0fAZ!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!0fAZ!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!0fAZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png" width="1456" height="887" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:887,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Game 2: Decision Tree&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Game 2: Decision Tree" title="Game 2: Decision Tree" srcset="https://substackcdn.com/image/fetch/$s_!0fAZ!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png 424w, https://substackcdn.com/image/fetch/$s_!0fAZ!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png 848w, https://substackcdn.com/image/fetch/$s_!0fAZ!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!0fAZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70ed4212-d171-4988-aa02-224b1fa2aa65_1773x1080.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The expected value after any single flip is unchanged, because</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;E[X]=0.5&#215;1.1+0.5&#215;0.9=1.0&quot;,&quot;id&quot;:&quot;NOPVPRIPON&quot;}" data-component-name="LatexBlockToDOM"></div><p>So the overall expected value of the game is $1.</p><p>But the distribution of outcomes is not normal. Because the payoffs are multiplicative rather than additive, the logarithm of the outcome is what accumulates additively. By the central limit theorem applied to log&#8289;(X), the distribution of final values follows a log-normal distribution. The majority of players end up with less than $1 (the median is below the mean), and there is a long right tail of players who end up with significantly more. The game is fair in expectation but deeply asymmetric in practice.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!HauT!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!HauT!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png 424w, https://substackcdn.com/image/fetch/$s_!HauT!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png 848w, https://substackcdn.com/image/fetch/$s_!HauT!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png 1272w, https://substackcdn.com/image/fetch/$s_!HauT!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!HauT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png" width="1456" height="819" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:819,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Game 2: Multiplicative Payoffs&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Game 2: Multiplicative Payoffs" title="Game 2: Multiplicative Payoffs" srcset="https://substackcdn.com/image/fetch/$s_!HauT!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png 424w, https://substackcdn.com/image/fetch/$s_!HauT!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png 848w, https://substackcdn.com/image/fetch/$s_!HauT!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png 1272w, https://substackcdn.com/image/fetch/$s_!HauT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa6c6bca6-3782-40ea-b977-3544a1c2985f_1600x900.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Game 3: Multiplicative payoffs with elimination.</strong> You start with $1. Each round, I double your money. You keep playing until you flip tails, at which point the game ends.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!pykT!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!pykT!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png 424w, https://substackcdn.com/image/fetch/$s_!pykT!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png 848w, https://substackcdn.com/image/fetch/$s_!pykT!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!pykT!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!pykT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png" width="1311" height="1080" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1080,&quot;width&quot;:1311,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Game 3: Decision Tree&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Game 3: Decision Tree" title="Game 3: Decision Tree" srcset="https://substackcdn.com/image/fetch/$s_!pykT!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png 424w, https://substackcdn.com/image/fetch/$s_!pykT!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png 848w, https://substackcdn.com/image/fetch/$s_!pykT!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!pykT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe5edc7fc-1974-4020-aba0-6c0c4d309e00_1311x1080.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The probability of surviving exactly <em>n</em> rounds is <em>(1 / 2^n)</em>, and the payout after <em>n</em> rounds is <em>(</em>2^<em>n)</em>. If we try to compute the expected value of this game, each term in the series contributes equally:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;E[X] = \\sum_{n=1}^{\\infty} 2^n \\cdot \\left(\\frac{1}{2}\\right)^n = \\sum_{n=1}^{\\infty} 1 = \\infty&quot;,&quot;id&quot;:&quot;RPXQBUGECD&quot;}" data-component-name="LatexBlockToDOM"></div><p>The expected value is infinite. This is the St. Petersburg paradox, famously analyzed by Daniel Bernoulli in 1738. More importantly for our purposes, we can directly derive the probability distribution for a payout <em>x</em>:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;x = 2^n&quot;,&quot;id&quot;:&quot;UVIXOOOWYG&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;n=\\log&#8289;_2(x)&quot;,&quot;id&quot;:&quot;KZAHYLULZX&quot;}" data-component-name="LatexBlockToDOM"></div><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;p(x) = \\left(\\frac{1}{2}\\right)^{\\log_2(x)} = \\frac{1}{x}&quot;,&quot;id&quot;:&quot;NXCHOBLIZW&quot;}" data-component-name="LatexBlockToDOM"></div><p>This is a power law, specifically a Pareto distribution with exponent <em>&#945;</em>=1. The probability of an outcome <em>x</em> is inversely proportional to <em>x</em> itself.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!-rPx!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!-rPx!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png 424w, https://substackcdn.com/image/fetch/$s_!-rPx!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png 848w, https://substackcdn.com/image/fetch/$s_!-rPx!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png 1272w, https://substackcdn.com/image/fetch/$s_!-rPx!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!-rPx!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png" width="1456" height="819" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:819,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Game 3: Multiplicative Payoffs with Elimination&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Game 3: Multiplicative Payoffs with Elimination" title="Game 3: Multiplicative Payoffs with Elimination" srcset="https://substackcdn.com/image/fetch/$s_!-rPx!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png 424w, https://substackcdn.com/image/fetch/$s_!-rPx!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png 848w, https://substackcdn.com/image/fetch/$s_!-rPx!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png 1272w, https://substackcdn.com/image/fetch/$s_!-rPx!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68b27b8f-41a1-49ac-ae84-51b6207a3823_1600x900.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>A pareto distribution:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;p(x) \\sim \\frac{1}{x^\\alpha}&quot;,&quot;id&quot;:&quot;YTVRKJNFBO&quot;}" data-component-name="LatexBlockToDOM"></div><p></p><p>can also arise whenever the round-by-round multiplier is roughly inversely proportional to the probability of continuing the game. The third game used a fair coin and a doubling multiplier, but it generalizes easily. Consider a weighted coin where the probability of tails (elimination) is 2/3 and each surviving round triples your money. The probability of surviving <em>n</em> rounds is (<em>1 / 3^n)</em>, and the payout is (<em>3^n)</em>. Substituting, we get  the same power law:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;p(x) = (1/3)^{\\log_3(x)} = 1/x&quot;,&quot;id&quot;:&quot;IHIXFTPXLM&quot;}" data-component-name="LatexBlockToDOM"></div><p>In fact, I could derive a power law from a far more convoluted set of rules. Round one might use a fair coin with a 2x multiplier, round two a weighted coin with a 3x multiplier, round three a differently weighted coin with a 1.5x multiplier, and so on. The specific numbers don&#8217;t matter. What matters is the structural relationship: <strong>the multiplier in each round is roughly inversely proportional to the probability of surviving that round.</strong> When that condition holds, the resulting distribution of outcomes is a power law.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/on-power-laws?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/on-power-laws?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/on-power-laws?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><h2><strong>The Venture Capital Coin Flip</strong></h2><p>Now consider the venture fundraising schedule through this lens. Each funding round is a &#8220;round&#8221; of the game. The company either survives to raise the next round or it does not. If it survives, its valuation increases by some multiplier. If it doesn&#8217;t, the game is over.</p><p>To make the comparison concrete, here is the coin-flip game with a fair coin and 2x multiplier:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!cadM!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!cadM!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png 424w, https://substackcdn.com/image/fetch/$s_!cadM!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png 848w, https://substackcdn.com/image/fetch/$s_!cadM!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png 1272w, https://substackcdn.com/image/fetch/$s_!cadM!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!cadM!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png" width="725" height="326.1504120879121" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:655,&quot;width&quot;:1456,&quot;resizeWidth&quot;:725,&quot;bytes&quot;:91220,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/193711448?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!cadM!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png 424w, https://substackcdn.com/image/fetch/$s_!cadM!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png 848w, https://substackcdn.com/image/fetch/$s_!cadM!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png 1272w, https://substackcdn.com/image/fetch/$s_!cadM!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb87fa73a-142b-4a5c-bd1c-b7f8279aba8d_2054x924.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>And here is the approximate venture capital equivalent:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!0BCa!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!0BCa!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png 424w, https://substackcdn.com/image/fetch/$s_!0BCa!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png 848w, https://substackcdn.com/image/fetch/$s_!0BCa!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png 1272w, https://substackcdn.com/image/fetch/$s_!0BCa!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!0BCa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png" width="1456" height="653" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/bf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:653,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:133748,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/193711448?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!0BCa!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png 424w, https://substackcdn.com/image/fetch/$s_!0BCa!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png 848w, https://substackcdn.com/image/fetch/$s_!0BCa!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png 1272w, https://substackcdn.com/image/fetch/$s_!0BCa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf5cae6f-845c-45fc-aa67-f7fd1f851c13_2034x912.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The coin-flip game has a uniform 50% survival rate and a uniform 2x multiplier. Venture capital is messier, but the qualitative pattern is the same. Multiplicative payoffs, compounded across multiple rounds, with elimination at each stage. The seed-to-Series-A transition has the lowest survival rate (~33%) and the largest valuation jump (~3x). Later stages have progressively higher survival rates (65%, 75%, 80%) but smaller relative valuation increases. The relationship between survival probability and valuation multiplier is roughly inverse: where the odds of advancing are worst, the payoff for surviving is greatest. That is Game 3, not Game 1 or 2.</p><p>I should be honest about what this comparison glosses over. The fundraise round size does not exactly equate to the company&#8217;s market value. A company might raise $20M at a $100M valuation, but that does not mean someone would pay $100M to acquire it. It just means that is the prospective valuation the founders and the venture firm agreed to. By the same logic, it&#8217;s not quite right to say that every company can &#8220;cash out&#8221; at its last valuation when it fails to raise the next round, the way a player in the coin-flip game walks away with their current payout when they flip tails. Valuation is not the same as market value.</p><p>This simplified model also ignores deal mechanics that matter in practice: pro rata rights, preferred stock, SAFEs, convertible notes, down rounds, bridge rounds, angel investors, and secondary markets. These are real and consequential features of the venture landscape, and I won&#8217;t pretend to account for all of them here.</p><p>But our simplified model of venture capital appears to correctly predict the key empirical phenomenon, the power law distribution of returns, via analogy with an even simpler model of coin-flip games. That is worth something, even if the model is imperfect.</p><h2><strong>Causation, Correlation, and the Chicken-or-Egg Problem</strong></h2><p>Is it possible that the venture fundraising schedule actually <em>causes</em> the power law distribution that it is supposedly built to capitalize on? It would be a mistake to conclude that because our simple model predicts the power law, the funding schedule must be the root cause. So let me take the opposing viewpoint and ask what other mechanisms could produce a power law in startup outcomes, independent of the funding structure.</p><p>One argument is that businesses themselves self-organize into a Pareto distribution through direct and indirect competition. Some companies are run more efficiently than others. Some founders are more resourceful. The quality of early employees can shape a company&#8217;s trajectory. Could this produce a power law on its own?</p><p>Recall the three coin-flip games. In order for a process to produce a power law rather than a normal or log-normal distribution, each successive stage needs to have a <em>multiplicative</em>, strictly positive effect on the company&#8217;s overall value.</p><p><strong>If injecting capital leads only to an </strong><em><strong>additive</strong></em><strong> increase in company value</strong> (Game 1), venture investment is unlikely to be profitable.</p><ul><li><p>An aimless startup with no clear product-market fit, where more money simply buys more time without compounding value.</p></li><li><p>A company that uses each round to fund an unrelated product line rather than building on its current traction &#8212; the returns add up, but they don&#8217;t compound.</p></li><li><p>A services business that scales by adding headcount, where revenue grows linearly with bodies and capital buys capacity, not growth rate.</p></li></ul><p><strong>If capital injection is multiplicative but carries a meaningful chance of </strong><em><strong>decreasing</strong></em><strong> the company&#8217;s value</strong> (Game 2), the distribution of outcomes is log-normal at best, not a power law.</p><ul><li><p>A company with unclear leadership, where scaling up could multiply value or could turn the whole operation into a disorganized mess.</p></li><li><p>A company whose customer acquisition cost exceeds lifetime value, where every dollar spent on growth actively destroys value.</p></li><li><p>A company that scales headcount faster than its organization can absorb, where new hires multiply coordination overhead instead of output.</p></li></ul><p>The expected outcome is still non-negative, but individual outcomes are often mediocre.</p><p><strong>It&#8217;s only the companies where capital injection is a multiplicative, strictly positive force that generate power-law outcomes</strong> (Game 3). These are the companies with the strongest compositions, the most defensible competitive advantages, and the clearest leadership. When isolated from the rest, these companies could plausibly constitute a power law regardless of the specific funding schedule. Whether the rounds increase by 2x or 5x is beside the point. What matters is the underlying growth mechanics.</p><p>But the venture fundraising schedule preferentially eliminates the non-power-law companies. With each round, as investors learn more about a company&#8217;s product, leadership, and market dynamics, it becomes easier to identify which startups will not exhibit positive, multiplicative returns on invested capital. By the time a company reaches late-stage funding (Series C or beyond), theoretically only companies with power-law mechanics remain. Much of the remaining &#8220;survival rate&#8221; at that point comes down to external market factors and whose growth rate is the largest and most stable. In a sense, the venture fundraising schedule <em>necessitates</em> that the surviving companies follow a Pareto distribution.</p><p>Now consider the reverse. Suppose that in the early days of modern venture investing, many firms did <em>not</em> use the staged funding structure that exists today. Suppose firms wrote smaller, more conservative checks in exchange for less equity, with less pressure on companies to demonstrate rapid growth between rounds. Startup survival rates would have been higher (though formation rates would likely be lower), leading to a flatter, more log-normal distribution of outcomes.</p><p>In that world, the few investors who <em>did</em> recognize the power law of elite companies, and structured their investments accordingly, would have seen outsized returns. Their LPs would have noticed. Capital would have flowed toward those successful funds and away from the conservative ones. The investor ecosystem itself would self-correct, converging on the funding structure that maximizes for power-law outcomes.</p><h2><strong>The Uncomfortable Answer</strong></h2><blockquote><p>Neither the funding schedule alone nor the startups alone produce the power law. It is the combination of the two.</p></blockquote><p>The staged funding structure applies selective pressure, pruning companies that do not demonstrate multiplicative growth. The companies that survive apply pressure back, forcing investors to adapt their strategies or lose out on the best returns. The relationship between startups and investors is simultaneously collaborative and competitive. Each side shapes the other&#8217;s behavior, and a power law emerges from the tension between them.</p><p>That may be unsatisfying. We want clean causation, A causes B. But the honest answer is a feedback loop, and the power law is what falls out of it.</p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>Round sizes and valuations from <a href="https://nvca.org/pitchbook-nvca-venture-monitor/">PitchBook-NVCA Venture Monitor</a> and <a href="https://www.cbinsights.com/research/report/venture-trends/">CB Insights State of Venture</a>.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-2" href="#footnote-anchor-2" class="footnote-number" contenteditable="false" target="_self">2</a><div class="footnote-content"><p>Graduation rates from <a href="https://carta.com/data/state-of-private-markets-q4-2024/">Carta&#8217;s State of Private Markets</a> reports. The ~33% seed-to-A figure reflects historical averages; Carta&#8217;s most recent data shows this chokepoint tightening further, with 2022-2023 cohorts graduating at closer to 15-20% within a 24-month window.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p></p></div></div>]]></content:encoded></item><item><title><![CDATA[The $500K "Contradiction"]]></title><description><![CDATA[and the economics of winner-take-most markets.]]></description><link>https://offsquare.substack.com/p/the-500k-contradiction</link><guid isPermaLink="false">https://offsquare.substack.com/p/the-500k-contradiction</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Fri, 20 Mar 2026 01:30:24 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!q5sA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="native-audio-embed" data-component-name="AudioPlaceholder" data-attrs="{&quot;label&quot;:null,&quot;mediaUploadId&quot;:&quot;aab14aca-df4d-42c6-a1ae-7ef4a7f4b603&quot;,&quot;duration&quot;:571.3502,&quot;downloadable&quot;:true,&quot;isEditorNode&quot;:true}"></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!q5sA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!q5sA!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png 424w, https://substackcdn.com/image/fetch/$s_!q5sA!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png 848w, https://substackcdn.com/image/fetch/$s_!q5sA!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png 1272w, https://substackcdn.com/image/fetch/$s_!q5sA!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!q5sA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png" width="1456" height="728" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:728,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!q5sA!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png 424w, https://substackcdn.com/image/fetch/$s_!q5sA!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png 848w, https://substackcdn.com/image/fetch/$s_!q5sA!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png 1272w, https://substackcdn.com/image/fetch/$s_!q5sA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04ae99b8-c746-496d-90fe-99d8d9be45f9_1600x800.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><blockquote><p><strong>If coding is solved, why are the people who solved it paying half a million dollars for coders?</strong></p></blockquote><p>Over the past year, Dario Amodei has made increasingly bold claims about the state of software engineering. At the Council on Foreign Relations in March 2025, he said AI would be &#8220;writing 90% of code&#8221; within three to six months. At Davos in January 2026, he told The Economist that &#8220;most, maybe all&#8221; of what software engineers do would be handled by AI within the year. He has repeatedly stated that engineers at Anthropic no longer write code themselves, instead letting Claude produce first drafts while they edit and supervise.</p><p>Meanwhile, people say that Anthropic&#8217;s job postings tell a different story. Software engineers with five or more years of experience are listed at $315,000 to $560,000 in base salary alone. Total compensation packages, including equity in a company valued north of $60 billion, push well past $500,000 and reportedly reach as high as $880,000. Anthropic is not quietly winding down its engineering organization. It is aggressively hiring some of the most expensive talent in the industry.</p><p>The reaction is predictable: &#8220;Hypocritical.&#8221; &#8220;Contradictory.&#8221; &#8220;They don&#8217;t actually believe their own claims.&#8221; I think this reading is wrong, and not just slightly wrong. The $500,000 salary and the &#8220;coding is solved&#8221; claim are not in tension. They are two expressions of the same underlying shift.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h2>The Automation Bar</h2><p>Suppose that coding really is 80-90% solved, or getting close to it. What does that actually imply?</p><p>It implies that a large share of the tasks historically performed by software engineers can now be completed by AI agents at a fraction of the cost and time. Bug fixes, boilerplate, straightforward feature implementation, standard CRUD operations, routine refactoring. The work that filled the calendars of mid-level engineers for the past two decades is increasingly automatable.</p><p>But &#8220;90% of coding tasks&#8221; is not the same as &#8220;90% of engineering value.&#8221; The remaining 10% is where the hard problems live: system architecture with long-term consequences, novel problem formulation, defining what to build and why, and making judgment calls under imperfect information. These tasks require deep expertise, taste, and contextual understanding that current AI systems cannot replicate.</p><p>This is what I think of as the automation bar. Below it, the work can be done cheaper and faster by an AI agent. Above it, the work still requires a human with genuine expertise. As the bar rises, the humans who operate above it become more valuable, not less, because their leverage has increased dramatically. A single engineer who can define problems clearly, set up the right guardrails, and direct AI agents effectively can now produce the output that previously required a team of five or ten.</p><p>This is not a new dynamic. It is the Steve Jobs principle taken to its logical extreme. Jobs famously observed that in many industries the &#8220;best&#8221; is ~30% better than average, but the best software engineers were ~50x better than the average engineer. The Mac team was built on the premise that a small group of A-level players could outperform a much larger team of mediocre ones. AI has made that multiplier even more extreme. A truly elite engineer with AI tools is now &gt;100x more productive than average, because the average engineer&#8217;s core output is increasingly achievable by the tools themselves.</p><div><hr></div><h2>The Economics of Scarcity</h2><p>If you accept the automation bar framing, the compensation picture becomes straightforward supply and demand.</p><p>The demand for high-quality software output has not decreased. It has increased, and in two distinct ways. Total venture funding in AI surged in 2025, with U.S. startup funding up over 75% in the first half of the year alone. That capital is heavily concentrated in a small number of very large companies, and it represents a growing pie of potential compensation for the engineers building those platforms. But the more interesting signal is company formation. The number of newly formed startups, particularly small, AI-native ventures, has sharply accelerated. Many of these are founded by software engineers and adjacent skill sets moving into entrepreneurial roles as demand for traditional enterprise engineering work declines. The most telling data point is the divergence: software engineer job postings are up 11% year-over-year according to Indeed, even as Bureau of Labor Statistics data shows &#8220;computer programmer&#8221; employment has fallen 27% since 2023. Companies are hiring more engineers and fewer coders. They want people who can think, not just people who can type.</p><p>The effective supply of engineers who operate above the automation bar has not kept pace. It may have shrunk in relative terms, because the bar itself keeps rising. The pool of people who can define complex systems, make architectural decisions with incomplete information, and direct AI agents productively is small. It was always small. Companies used to need those people plus a large supporting cast. Now they mostly just need those people.</p><p>So the price goes up. The value an elite engineer creates at Anthropic&#8217;s scale massively outweighs their cost. Paying $500,000 for someone who generates $5 million in value is not a contradiction. It is a bargain.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/the-500k-contradiction?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/the-500k-contradiction?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/the-500k-contradiction?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><p>Professional sports illustrate where this leads. In Major League Baseball, the average salary in 2025 was $5.16 million, but the median was just $1.35 million. The top 50 players earned 29% of all salary dollars. Shohei Ohtani&#8217;s contract with the Dodgers is worth $700 million over ten years. The gap between a replacement-level player earning the league minimum and an MVP-caliber player earning $40 million a year is not a 50x difference in raw athletic ability. It reflects the economics of a winner-take-most market, where elite performance has no substitute and commands disproportionate compensation.</p><p>Software engineering is becoming that kind of market. The median engineer&#8217;s core output is increasingly reproducible by AI, the same way a replacement-level outfielder&#8217;s production can be approximated by any number of available alternatives. The engineer who can architect a system that scales, define the right product to build, and multiply their output through AI tools has no cheap substitute. The compensation reflects the scarcity, not the contradiction.</p><div><hr></div><h2>Where This Goes</h2><p>I expect this dynamic to accelerate. There is an economic principle called the Jevons Paradox: when technology makes a resource cheaper to use, people don&#8217;t use less of it. They use more. If AI makes building software 10x cheaper, we will not build the same amount of software with 90% fewer engineers. We will build 100x more software. The sharp increase in new company formation, driven largely by engineers and technical founders who can now build products that previously required entire teams, points in exactly this direction.</p><p>The practical result is a sharper Pareto distribution of engineering compensation. Engineers whose primary skill is translating well-defined requirements into working code will see their market value hold steady or decline. Engineers who operate above the automation bar will see their compensation continue to climb.</p><p>We have already seen early signals. Meta&#8217;s Superintelligence lab offered packages reaching $200 to $300 million over four years to recruit top AI researchers. These numbers are anomalies today, but I think they are directional for where elite engineering compensation is headed.</p><p>The career pipeline will also change. The traditional path from junior engineer to senior at a large company, spending years on increasingly complex tickets, is eroding. Not because junior engineers are worthless, but because companies have less economic incentive to invest in multi-year talent development when the intermediate-skill work those juniors would grow into is being automated. The path to elite engineering looks less like a promotion track and more like an entrepreneurial apprenticeship: building things constantly, taking on ambiguous problems, developing judgment through practice.</p><p>This connects to something I have been watching for a while: the convergence of the engineer, the product manager, and the entrepreneur into a single role. When building software is cheap, the bottleneck shifts to knowing what to build and why. The most valuable person at an early-stage company is not the one who writes the best code. It is the one who can talk to users, define the product, architect the system, and build it, all in the same afternoon. That person commands $500,000 or more, not because coding is hard, but because everything around coding is hard.</p><p>The real contradiction would be if Anthropic claimed coding was solved and then offered $100,000 a year for engineers. That would suggest they believe the remaining work is routine. Instead, they are paying premium prices for the work that remains after automation, which tells you exactly how they value it. The $500,000 salary is not evidence against their thesis. It is the thesis, expressed in dollars.</p><div><hr></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[The Inevitability of Technology]]></title><description><![CDATA[Why opposing specific applications of technology while championing the underlying capability is irrational.]]></description><link>https://offsquare.substack.com/p/the-inevitability-of-technology</link><guid isPermaLink="false">https://offsquare.substack.com/p/the-inevitability-of-technology</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Fri, 13 Mar 2026 01:15:42 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!079g!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="native-audio-embed" data-component-name="AudioPlaceholder" data-attrs="{&quot;label&quot;:null,&quot;mediaUploadId&quot;:&quot;fccf72e0-e8a4-46c9-98ac-3bec47b0d699&quot;,&quot;duration&quot;:532.9502,&quot;downloadable&quot;:true,&quot;isEditorNode&quot;:true}"></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!079g!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!079g!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png 424w, https://substackcdn.com/image/fetch/$s_!079g!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png 848w, https://substackcdn.com/image/fetch/$s_!079g!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png 1272w, https://substackcdn.com/image/fetch/$s_!079g!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!079g!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png" width="1456" height="680" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:680,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!079g!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png 424w, https://substackcdn.com/image/fetch/$s_!079g!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png 848w, https://substackcdn.com/image/fetch/$s_!079g!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png 1272w, https://substackcdn.com/image/fetch/$s_!079g!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd236e9b5-f6ee-4c50-a42b-d4db04935e17_1600x747.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><blockquote><p><strong>If an application of existing technology is physically possible, it is inevitable. The only rational response is engagement, not withdrawal.</strong></p></blockquote><p>In April 2018, more than 3,000 Google employees signed an open letter to CEO Sundar Pichai demanding the company withdraw from Project Maven &#8212; a Pentagon contract that used TensorFlow-based image recognition to analyze drone surveillance footage. &#8220;We believe that Google should not be in the business of war,&#8221; the letter stated. About a dozen engineers resigned. Prominent researchers like Meredith Whittaker organized internally. By June, Google announced it would not renew the contract when it expired in March 2019, and published a set of AI Principles committing not to build AI for weapons.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>When I read about the walkout, I remember thinking that it was borderline, if not fully, hypocritical.</p><p>Google&#8217;s parent company, Alphabet, was simultaneously pouring over a billion dollars a year into Waymo &#8212; its autonomous vehicle subsidiary, valued around $100B at that time. The core technology powering Waymo is functionally identical to what Project Maven required: object detection, classification, tracking, planning, and real-time decision-making from sensor data. Waymo identifies pedestrians, cyclists, and vehicles, then makes split-second navigation decisions. Project Maven aimed to identify objects of interest in aerial footage, which also informed navigation and targeting decisions.</p><p>If you had given 100 random Waymo engineers (just 15% of the Waymo team) the right datasets and clearances, they would have built an autonomous military drone system as a matter of routine engineering. The ML architectures were the same. The sensor fusion pipelines and control systems differed in parameters, not in kind. The only meaningful difference was the label on the training data and the vehicle it was mounted on.</p><p>The protest was not about technical capability. It was about moral comfort. And moral comfort is not a strategy.</p><h2>The Physics of Inevitability</h2><p>There is a principle in high-energy physics, which I think about often in analogy to technology:</p><blockquote><p>If a particle is not explicitly forbidden by a conservation law or symmetry, it necessarily exists. You just have to look in the right places.</p></blockquote><p>This has been validated repeatedly. The positron, neutrino, Higgs boson, etc &#8212; each was predicted because the mathematics allowed it, and each was eventually observed. The only particles we fail to find are those that are strictly forbidden by physical law. Everything else is a matter of (1) our understanding of the laws of physics and (2) knowing when and where you look.</p><p>I believe that technology follows the same pattern. In fact, the analogy is surprisingly clear:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!9qJ8!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!9qJ8!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png 424w, https://substackcdn.com/image/fetch/$s_!9qJ8!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png 848w, https://substackcdn.com/image/fetch/$s_!9qJ8!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png 1272w, https://substackcdn.com/image/fetch/$s_!9qJ8!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!9qJ8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png" width="727" height="435.9004120879121" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:873,&quot;width&quot;:1456,&quot;resizeWidth&quot;:727,&quot;bytes&quot;:381213,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://distilleryai.substack.com/i/190781808?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!9qJ8!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png 424w, https://substackcdn.com/image/fetch/$s_!9qJ8!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png 848w, https://substackcdn.com/image/fetch/$s_!9qJ8!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png 1272w, https://substackcdn.com/image/fetch/$s_!9qJ8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0164cd6-c9f4-46c4-9382-4c859a017be0_2196x1316.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Waymo exists because the algorithms, hardware, and infrastructure for autonomous driving became sufficient. Project Maven existed because the same stack made military image analysis possible.</p><p>The implication is straightforward: if you support the development of a general-purpose technology &#8212; computer vision, large language models, robotic control systems &#8212; you are implicitly supporting every application that technology makes possible. Opposing a specific application while championing the underlying capability is like celebrating the Standard Model while insisting the Higgs boson shouldn&#8217;t exist. The math doesn&#8217;t care about your preferences.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/the-inevitability-of-technology?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/the-inevitability-of-technology?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/the-inevitability-of-technology?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><h2>Anthropic and the Pentagon</h2><p>The same principle is playing out right now between AI labs and the Pentagon. In January 2024, OpenAI quietly removed its blanket prohibition on military use from its usage policies. Throughout 2024 and into 2025, OpenAI moved aggressively into the defense and intelligence space, partnering with Palantir and Scale AI to deploy LLMs in classified environments and signaling willingness to customize its models for government applications.</p><p>Anthropic took the opposite approach. They updated their acceptable use policy to permit certain national security use cases but refused to weaken or remove the safety guardrails on Claude for military applications, even when doing so might have secured large defense contracts.</p><p>I think highly of Anthropic. I use their models daily, I respect their research, and I believe their safety work is among the most important in the industry. But on this specific question, I think their position is largely irrational.</p><p>Military applications of large language models are technically possible with current capabilities. Therefore, they will be built. If Anthropic refuses to engage, the gap will be filled by OpenAI, by open-source models fine-tuned without guardrails, or by DeepSeek or another Chinese lab with no interest in Anthropic&#8217;s safety philosophy. The application is not forbidden by the laws of physics &#8212; it is permitted by the current tech stack &#8212; and so it <em>will</em> be observed.</p><p>Given that inevitability, Anthropic faces a binary choice: shape the application or cede it to others. Withdrawal does not prevent the outcome. It only removes Anthropic&#8217;s influence over how the outcome unfolds. A safety-focused lab with deep expertise in alignment is <em>exactly</em> the organization you want at the table when the Pentagon is deciding how to integrate AI into its operations. Their absence does not make the deployment safer. It makes it less safe.</p><p>This is the same error the Google engineers made in 2018. They succeeded in removing Google from Project Maven, but they did not succeed in preventing the Pentagon from using AI for drone surveillance. The DoD simply moved the contract to other vendors. And the technology didn&#8217;t just survive the protest &#8212; it proliferated far beyond anything Project Maven originally proposed. Today, AI-powered drone surveillance and autonomous targeting systems are operational across essentially every advanced military in the world. The Ukraine-Russia conflict alone has demonstrated the technology at massive scale on both sides. Turkey&#8217;s Bayraktar drones use computer vision for targeting in multiple theaters. China, Israel, and Iran all have operational programs. The walkout may have delayed one contract by a few months. The particle was observed anyway.</p><h2>The Rational Response</h2><p>There is a persistent temptation to treat opposition as a meaningful intervention &#8212; to believe that if enough talented people refuse to participate, the application won&#8217;t happen. This is a comforting fiction. It confuses moral signaling with material impact.</p><p>The history of technology repeatedly confirms this. Nuclear energy, genetic engineering, surveillance technology, encryption &#8212; in every case, the technology that <em>could</em> be built <em>was</em> built. Protest and regulation sometimes delayed specific implementations, but the underlying capability always found its way to deployment. The meaningful differences in outcomes came not from who refused to participate, but from who chose to engage and on what terms.</p><p>If you are an engineer, a researcher, or a company with genuine expertise in building these systems safely, your withdrawal from a controversial application does not make that application less likely. It makes it less safe. You are not preventing the particle from existing. You are choosing not to be in the room when it&#8217;s observed.</p><p>My view is not that every application of technology is morally equivalent, or that there are no legitimate reasons to be cautious about military AI. It is that caution and engagement are compatible, while caution and withdrawal are not. If an application is technically possible, it will be built. The rational response is to ensure it is built well.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[On Imperfect Information]]></title><description><![CDATA[and the future of intellectual work]]></description><link>https://offsquare.substack.com/p/on-imperfect-information</link><guid isPermaLink="false">https://offsquare.substack.com/p/on-imperfect-information</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Thu, 05 Mar 2026 02:02:00 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!fBM_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="native-audio-embed" data-component-name="AudioPlaceholder" data-attrs="{&quot;label&quot;:null,&quot;mediaUploadId&quot;:&quot;88af7c15-de05-471f-a2dc-bae84d5287d2&quot;,&quot;duration&quot;:470.30856,&quot;downloadable&quot;:true,&quot;isEditorNode&quot;:true}"></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!fBM_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!fBM_!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png 424w, https://substackcdn.com/image/fetch/$s_!fBM_!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png 848w, https://substackcdn.com/image/fetch/$s_!fBM_!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png 1272w, https://substackcdn.com/image/fetch/$s_!fBM_!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!fBM_!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png" width="1456" height="800" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/ffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:800,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:8995732,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://offsquare.substack.com/i/189883474?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!fBM_!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png 424w, https://substackcdn.com/image/fetch/$s_!fBM_!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png 848w, https://substackcdn.com/image/fetch/$s_!fBM_!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png 1272w, https://substackcdn.com/image/fetch/$s_!fBM_!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fffd45dad-fd05-41a9-a1c1-06bb7ef079e0_3052x1676.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><blockquote><p><strong>The future of all intellectual work is: operating under imperfect information.</strong></p></blockquote><p>Lately, I&#8217;ve noticed two theories on the future of AI/LLMs that are gaining popularity.</p><ul><li><p>Artificial intelligence will become a sufficiently good replacement for (most) human intelligence</p></li><li><p>Software will become a commodity as the effective cost of producing code trends toward zero.</p></li></ul><p>My view is narrower and subtly different. LLMs are a near-perfect replacement for <em>action under (near-) perfect information</em>. Software is only commoditized to the extent that you can provide an LLM with perfect operating instructions. That means clear goals, crisp constraints, complete context, and unambiguous success criteria. Cloning the Apple website is already easy because it&#8217;s fully observable and has relatively little complex logic behind the scenes. Cloning the Spotify personalization engine is hard unless you can clearly and completely articulate what it is, how it works, what it should optimize for, and how you&#8217;ll know it&#8217;s correct.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>That distinction matters because most real work is not fully observable. Most real work is partially observable, expensive to validate, and full of uncertainty that can&#8217;t be eliminated. It can only be mitigated.</p><p>You can already see this in the canonical SWE workflow today. Engineers guide Claude Code to implement a new ticket or feature. The more perfect the engineer&#8217;s description, the more perfect the output. When the prompt is vague, the agent fills in gaps, makes assumptions, and drifts. The best engineers already realize this, and they spend a disproportionate amount of up-front time building guardrails for the agent. The agent isn&#8217;t magically developing judgment. It&#8217;s being handed a tighter operating environment.</p><p>That&#8217;s the split I keep coming back to. Execution under perfect information is getting cheap. Turning the real world into something close to perfect information was always the hard part.</p><p>&#8220;Imperfect information&#8221; includes both obvious cases and more subtle ones.</p><ul><li><p>you don&#8217;t have access to the data</p></li><li><p>you could get the data, but it&#8217;s expensive (either in money or time)</p></li><li><p>you have a data point, but you don&#8217;t trust its precision</p></li></ul><p>For example, suppose that someone quotes an odds estimate at ~40%, but if you&#8217;re honest, you think the real odds swing anywhere between 20% and 60%. The number exists, but confidence doesn&#8217;t. You still have to act.</p><p>I think the core human job shifts toward judgment under uncertainty. It means operating under partial information, dealing with expensive information, and deciding what to trust, what to verify, and what to do next.</p><p>Another way to describe being &#8220;good&#8221; at something is that you have better or more complete information than someone else, and you know how to act on it. For years, being good at coding often looked exactly like that. A lot of the differentiation came from informational advantage. It included patterns, tooling, library behavior, conventions, debugging instincts, system constraints, &#8220;how this codebase really works,&#8221; and which shortcuts are safe versus dangerous. LLMs dissolve a big chunk of that by making missing information cheap and fast to retrieve.</p><p>But dissolving informational advantage doesn&#8217;t dissolve performance differences. It shifts where those differences come from. If execution is cheap, the bottleneck becomes definition. If you can&#8217;t turn a messy problem into crisp criteria, the agent can&#8217;t save you. If you can, the agent amplifies you.</p><p>This suggests that, as a result of AI, the already-massive talent gap in software engineering will continue to widen.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/on-imperfect-information?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/on-imperfect-information?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/on-imperfect-information?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><p>There&#8217;s a persistent fiction that &#8220;mid-level&#8221; is a narrow band. In practice, mid-level covers a huge spread of capability. Even in the old world, you were handed a specific ticket and the information was relatively complete. Some people still did a worse job than others. They missed edge cases, implemented the wrong interpretation, and introduced unnecessary fragility. They shipped something that technically passed but didn&#8217;t actually match the real intent. Perfect information wasn&#8217;t enough to guarantee good outcomes.</p><p>Now imagine the new world where you&#8217;re expected to act optimally when perfect information is available, and most of your performance is evaluated under imperfect information because real projects are not fully spec&#8217;d, not fully observable, and not fully predictable. Truly elite engineers will distinguish themselves further. The differentiator becomes whether you can create near-perfect information from partial observability. Can you define what success is, constrain the problem correctly, and make good decisions when the inputs are incomplete?</p><p>This is the point that often gets missed in public discourse about AI and software engineering. People suppose the value of software trends to zero. I think that&#8217;s true only up to the extent that you can act perfectly upon perfect information, or manufacture near-perfect information out of the messy real world. The moment the task depends on judgment under uncertainty, software stops looking like a simple commodity and starts looking like an applied decision-making problem again.</p><p>This imperfect-information framing also explains why venture investor types tend to be comfortable with this world. They essentially always operate under imperfect or flawed information, and there is rarely a notion of perfect execution. There is only making marginally or moderately better decisions under uncertainty. That&#8217;s the whole job. Better calibration, better sizing of risk, better recognition of potential upside, better interpretation of weak signals, and better decisions with incomplete data.</p><p>It also lines up with the &#8220;engineers become managers of AIs&#8221; idea that has recently become mainstream. Managers operate at higher abstraction levels with greater uncertainty and more tradeoffs. They don&#8217;t get perfect inputs or perfect feedback. If engineers spend more of their day directing agents, specifying constraints, choosing tradeoffs, and defining success criteria, the job shifts toward that same shape. Higher abstraction, higher uncertainty, and more judgment.</p><p><strong>If you can&#8217;t be trusted to make meaningful decisions under a moderate amount of uncertainty and risk, you&#8217;re at increased risk of replacement over time.</strong> Not necessarily because you&#8217;re less intelligent, but because your value is concentrated in the part of the workflow that can be reduced to perfect-information execution.</p><p>It&#8217;s useful to separate the <strong>LLM Era</strong> from the <strong>Information Era</strong>, where the advent of computers and technologies like Google changed how information was accessed and utilized. In the 80s, 90s, and earlier, it was highly valuable to know a lot of facts and be able to recall them on-demand. But as Google and other search engines became ubiquitous, they made recall cheap. It didn&#8217;t eliminate expertise, but it replaced a big chunk of &#8220;trivia advantage&#8221; -- a specific form of the information advantage I described earlier. Many are now portraying LLMs as an eventual replacement for the human intelligence layer, but my interpretation is that LLMs will replace human _execution_ more than they replace human intelligence.</p><p>With this in mind, the execution playbook becomes straightforward: hand off the perfect-information work to LLMs. All of the most valuable work will involve turning uncertainty into clarity and operating instructions. Get good at defining constraints, deciding what needs verification, and looking at every problem from multiple angles.</p><p>Execution is getting cheap. Judgment is still best left to humans.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Industry Predictions for 2026]]></title><description><![CDATA[Emerging trends and predictions for AI, startups, engineers, and tech in the coming year.  (Repost from Jan 1, 2026 on my personal site.)]]></description><link>https://offsquare.substack.com/p/industry-predictions-for-2026</link><guid isPermaLink="false">https://offsquare.substack.com/p/industry-predictions-for-2026</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Wed, 04 Mar 2026 14:55:28 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!f4nc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="native-audio-embed" data-component-name="AudioPlaceholder" data-attrs="{&quot;label&quot;:null,&quot;mediaUploadId&quot;:&quot;20049794-98af-4f41-848a-3c7e7b4e44bc&quot;,&quot;duration&quot;:936.6204,&quot;downloadable&quot;:true,&quot;isEditorNode&quot;:true}"></div><h2>The AI <s>Bubble</s> Froth</h2><p>There is indeed a &#8220;bubble&#8221; in the AI industry, but it is not as exaggerated as many online personalities would have you believe. The reality is more nuanced.</p><p>There are certainly companies that are massively overvalued, but they aren&#8217;t the ones you might expect. OpenAI, Anthropic, and NVIDIA (among others) remain fundamentally sound, because they create tangible, immediate value in the marketplace. Critics often point to the high burn rates of OpenAI and Anthropic, or the complex web of big tech investments sustaining them, as signs of instability. However, this view ignores the massive floor for demand. If major LLM providers were to disappear tomorrow, the impact on the US marketplace would be catastrophic&#8212;a majority of the current startup landscape (AI-native or otherwise) would likely be decimated. The demand for these intelligence layers is massive, structural, and will continue to grow, not shrink.</p><blockquote><p><em><strong>Disclaimer:</strong> I am not claiming that OpenAI or Anthropic will be *profitable* in the near future.  In fact, it&#8217;s likely that they continue hemorrhaging money for several more years.  But the demand for their services is real, and the market is nowhere close to saturation.</em></p></blockquote><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>I&#8217;ve heard the counter-argument that the model of massive capital expenditure for American AI labs is unsustainable, destined to be undercut by more efficient, open-source, Chinese models. This argument fails to account for the genealogy of those efficient models: they are typically distilled from the outputs of leading-edge American models. Without the existence of a bleeding-edge frontier model to distill from, those economical alternatives cannot exist in their current form.  Frontier labs are more than capable of producing their own economical, distilled models, but leaders like OpenAI, Anthropic, and Google choose instead to focus on bleeding-edge intelligence.</p><p>The full &#8220;commoditization&#8221; of LLMs has not happened yet. If it had, we wouldn&#8217;t see expensive, closed-source models like Opus-4.5 gaining significant traction among developers. We wouldn&#8217;t see Meta pivoting away from an open-source AI strategy toward large, closed-source models. Because no single AI lab has secured a permanent foothold atop the intelligence leaderboards, it would be irresponsible and short-sighted to cut spending and focus on productization. The intelligence race is still very much alive. True commoditization will only occur once a model like &#8220;GPT-X.Y&#8221; becomes impossible to dethrone. Only then will it make strategic sense to shift focus from capability research to cost reduction and profit harvesting.</p><p>The companies that are truly overvalued are the pre-product startups commanding multi-billion-dollar valuations&#8212;labs like Thinking Machines or Safe Superintelligence that have yet to prove their business models. Over-valuations are not new to venture capital, but we have already seen </p><p><strong>The Inflection AI Case Study:</strong> Despite raising approximately <strong>$1.5 billion</strong>, outstanding founder pedigree (Mustafa Suleyman, Reid Hoffman, Kar&#233;n Simonyan), and developing a consumer product (&#8221;Pi&#8221;), Inflection never established a viable business model. In March 2024, the company effectively collapsed as a standalone entity when Microsoft hired its co-founders and key staff.</p><p>As the froth settles, we will see more stories like this. Investors in the major labs may face volatility, but their capital is backing real infrastructure. The same cannot be said for every multi-billion-dollar, pre-product lab.</p><h2>Agentic Trough of Disillusionment</h2><p>The novelty of &#8220;agents&#8221; is rapidly wearing off. For the past year, &#8220;agentic&#8221; has been the magic word in pitch decks and marketing copy, promising software that doesn&#8217;t just advise, but <em>acts</em>. However, most tech-savvy users have now test-driven enough &#8220;agentic&#8221; workflows to know the reality: they are often fragile, slow, or prone to getting stuck in loops.</p><p>We are entering the classic &#8220;Trough of Disillusionment&#8221; for AI agents. In 2026, simply claiming your product is &#8220;agentic&#8221; will no longer be a value proposition; in fact, it should be a red flag.</p><ul><li><p><strong>The &#8220;Show, Don&#8217;t Tell&#8221; Era:</strong> If you have to explicitly tell me that your product is &#8220;agentic,&#8221; or features an &#8220;AI assistant,&#8221; I already suspect that it&#8217;s bad. Users don&#8217;t care about the underlying architecture &#8212;whether it&#8217;s a chain-of-thought loop or a single API call. They only care if their work gets done reliably. Explicitly marketing &#8220;agency&#8221; signals that the product is experimental rather than production-ready.</p></li><li><p><strong>From Feature to Baseline:</strong> Just as we stopped calling companies &#8220;internet companies&#8221; or &#8220;mobile-first&#8221; once those technologies became ubiquitous, &#8220;agentic&#8221; capabilities will fade into the background. We expect our calendar apps to handle scheduling conflicts and our code editors to refactor files. We don&#8217;t need to be told they are &#8220;agents.&#8221;  We just expect them to work.</p></li><li><p><strong>The Reliability Gap:</strong> The biggest hurdle clearing this trough is reliability. The early wave of agentic demos &#8212; impressive on social media but brittle in practice &#8212; has made buyers skeptical. The winners in 2026 won&#8217;t be the ones with the most autonomous agents, but the ones with the most <em>reliable</em> workflows that actually finish a task without human intervention.</p></li></ul><p>The most successful agentic products of 2026 will be the ones that never mention the word &#8220;agent&#8221; at all.</p><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/industry-predictions-for-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Enjoying this? The best way to support the publication is to forward it to one person who would argue with it intelligently.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/industry-predictions-for-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/industry-predictions-for-2026?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div><h2>Rise of the Product Engineer</h2><p>For years now, generalist engineers have been the under-hyped, under-valued utility players of the software industry &#8212; essential for plugging gaps and collaborating across teams. AI has inverted that dynamic, turning versatility into tremendous leverage.  The most valuable employee at your company is now the "T-Shaped Engineer" &#8212; someone with broad software skills across multiple stacks, who is also very good at your company&#8217;s core competency.  This is the person with enough technical depth to build a moat for your business, and enough breadth to efficiently handle dozens of other, tangential engineering tasks that are required for early-stage ventures.</p><p>But 2025 introduced a new, non-negotiable skill to the generalist toolkit: <strong>product-mindedness</strong>.  Being a highly productive, task-oriented engineer is no longer sufficient.  You must be able to think about the product you&#8217;re building, ask important questions, probe underlying assumptions, and even speak to current/prospective users.  You must ask the *why* behind everything you do, and without anyone reminding you to do so.</p><p><strong>The Convergence of Roles</strong> This shift is forcing a collision between the Product Manager and the Lead Engineer. The current &#8220;vibe coding&#8221; debate effectively centers around one question:</p><blockquote><p><em>Which is more valuable: a PM with AI coding tools, or a product-minded engineer?</em></p></blockquote><p>Currently, I&#8217;d argue that the product-minded engineer wins.  But I&#8217;m tremendously biased, and I think the gap has closed considerably.  In 2026, the lines will blur significantly. The best PMs can code and build prototypes, and the best engineers understand that &#8220;product thinking&#8221; dramatically increases their impact on the team. The &#8220;Product Engineer&#8221; is the inevitable result of this merger.</p><h2>Scariest Chart in the World</h2><p>Recently, my feed has been flooded with different versions of &#8220;the scariest chart in the world.&#8221;  The implication is clear and terrifying: AI is generating massive corporate profits while systematically eliminating human jobs.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!f4nc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!f4nc!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg 424w, https://substackcdn.com/image/fetch/$s_!f4nc!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg 848w, https://substackcdn.com/image/fetch/$s_!f4nc!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!f4nc!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!f4nc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg" width="800" height="467" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:467,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;ChatGPT's Impact on US Economy | Samarth Srivastava posted on the topic |  LinkedIn&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="ChatGPT's Impact on US Economy | Samarth Srivastava posted on the topic |  LinkedIn" title="ChatGPT's Impact on US Economy | Samarth Srivastava posted on the topic |  LinkedIn" srcset="https://substackcdn.com/image/fetch/$s_!f4nc!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg 424w, https://substackcdn.com/image/fetch/$s_!f4nc!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg 848w, https://substackcdn.com/image/fetch/$s_!f4nc!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!f4nc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd3721820-2fce-4477-9146-6881aca4676a_800x467.jpeg 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Fortunately, this is a correlation-causation fallacy.</p><ul><li><p><strong>The Macro Reality:</strong> The chart actually illustrates a post-COVID correction. The &#8220;Great Resignation&#8221; of 2020-2021 created an artificial spike in job openings as turnover peaked. Combined with near-zero interest rates and VC largesse, hiring exploded. What we are seeing now is not an AI-induced collapse, but a regression to the mean as interest rates rose and &#8220;growth at all costs&#8221; ended.</p></li><li><p><strong>The Adoption Lag:</strong> The chart implies that ChatGPT instantly impacted the labor market upon release in late 2022. In reality, enterprise adoption lagged by a year-plus. For the first year, ChatGPT was primarily a productivity tool for existing engineers (writing snippets, drafting emails), not powering advanced or agentic systems capable of replacing real employees. The timeline simply doesn&#8217;t fit the narrative.</p></li></ul><p>Finally, we must acknowledge that 'AI replacement' has become a convenient narrative shield for executives. When big players like Amazon rolled back headcount in late 2025, the external communication was that AI offered such large productivity gains that fewer employees were needed. However, it&#8217;s important to note that Amazon was under intense Q3/Q4 pressure to protect margins, and this could equivalently be explained as a financial decision.</p><h2>Return of the Junior Engineer</h2><p>Unfortunately, many software leaders have bought into this fallacy: that while tech-savvy companies become more profitable, the &#8220;junior engineer&#8221; is being rendered obsolete by AI. This belief has had an outsized and unfair impact on recent graduates, restricting their access to opportunities in a way that feels undue. Yes, AI can complete many of the low-level coding tasks that used to be the training ground for new hires. But this view is short-sighted.</p><ul><li><p><strong>The Closing Gap:</strong> Young professionals are entering the workforce equipped with AI tools that allow them to absorb new concepts, frameworks, and technologies at an unprecedented speed. Having interviewed many engineers over the past year, I am confident that the gap between a good junior and a middle-of-the-road senior has closed significantly. Today&#8217;s juniors possess a skill set and velocity that I didn&#8217;t achieve until years into my own career as an ML engineer.</p></li><li><p><strong>Conviction:</strong> These engineers show a higher level of determination than previous generations. They have chosen to wade into the software landscape at the exact moment the industry is predicting their demise. Contrast that against the previous generation, when junior-to-mid-level jobs were plentiful, and many people (myself included) wandered into the industry on a whim.  </p></li></ul><p>It&#8217;s only a matter of time before companies remember that junior engineers, if hired and mentored in the correct context, can still bring tremendous values to companies in the year 2026. They are not one-to-one replaceable by AI, because they are capable of critical thinking, reasoning, and continuous improvement. That, along with their increased preparedness, makes them a dangerous asset in the next five years.</p><h2>Decline of 996 Culture</h2><p>The &#8220;996&#8221; work model&#8212;9 am to 9 pm, six days a week&#8212;is rapidly becoming an artifact of a bygone era. Modeled after the brute-force growth tactics of the early mobile internet boom, this culture treats engineers as consumable, renewable resources. The result is a churn-and-burn cycle where talent evaporates within 12&#8211;18 months. But beyond the human toll, this model is failing for a more pragmatic reason: it relies on a fundamental misunderstanding of how software is built in 2026.</p><p><strong>Direction Over Velocity:</strong> There is a pervasive belief among certain founders that &#8220;more hours equals more output.&#8221; This creates a culture of motion rather than progress.</p><blockquote><p><em>&#8220;It doesn&#8217;t matter how fast you move if it&#8217;s in a worthless direction.&#8221;<br>-Sam Altman</em></p></blockquote><p>I don&#8217;t often quote Altman, but he is spot-on here. In my experience, leadership often overestimates the value of raw engineering hours while underestimating the value of critical thinking. This has always been true &#8212; seasoned developers know that engineering is 80% reading, planning, and communicating, and only 20% actually writing code &#8212; but AI has amplified this reality.</p><p><strong>The Efficiency Paradox:</strong> In 2026, the barrier to entry isn&#8217;t writing code; it&#8217;s deciding what code is worth writing. It is possible to build a highly successful, scalable software company with only ~10,000 total lines of code, but that level of efficiency requires extreme intentionality. You cannot achieve this by battering-ramming your way through tickets for 72 hours a week. An effective technical leader dedicates at least 10% of their time to observing, continuously re-evaluating the approach, and adjusting the roadmap.</p><p><strong>The Talent Correction:</strong> The best engineers know that &#8220;grind&#8221; culture is often a warning sign for lack of strategy. They will avoid these environments, leaving 996 companies with a talent deficit that is difficult to overcome.  In the software / tech industry, there is often no replacement for engineering talent.</p><blockquote><p><em>For most things in life, the range between best and average is 30% or so. The best airplane flight, the best meal, they may be 30% better than your average one. What I saw with Steve Wozniak was somebody who was <strong>50 times better than your average engineer</strong>. The Mac team was an attempt to build a whole team like that of A players. People said they wouldn&#8217;t get along, they&#8217;d hate working for each other. But I realized A players like to work with A players, they just didn&#8217;t like working with C players&#8230; When I got back to Apple, that&#8217;s what I decided to do.<br>-Steve Jobs</em></p></blockquote><h2>World Models Won&#8217;t Take Off... Yet</h2><p>World models are my favorite AI advancement of 2025. While LLMs predict the next word in a sentence, World Models predict the next frame in a video, based on the current state and inputs from the user(s). By feeding an AI millions of hours of data, it learns the underlying physics, causal relationships, and object permanence. It creates an internal simulation (a &#8220;world&#8221;) that it can manipulate.  The potential applications are wide-reaching:</p><ul><li><p>Entertainment &#8212; Generating infinite, fully interactive video games on the fly.</p></li></ul><ul><li><p>Robotics &#8212; Training robots in a realistic simulation or &#8220;dream&#8221; state to learn before real-world interaction.</p></li><li><p>Autonomous Vehicles &#8212; Logging billions of hours of driving in simulated environments to safely train self-driving cars.</p></li><li><p>Design &#8212; Prototyping and designing objects such as cars or buildings.</p></li></ul><p>Despite the hype, I am bearish on their commercial viability for the next 1-2 years months.  World models are much more expensive to serve than LLMs. With ChatGPT, one high-end GPU cluster can serve dozens of concurrent users. With world models, you typically need a cluster of H100s just to serve a single user in real-time. Until we see a 10x improvement in inference efficiency or a massive drop in compute costs, this technology does not scale to consumers.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Off-Square is a free publication. Subscribe to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Transformers from Scratch in PyTorch]]></title><description><![CDATA[Join the attention revolution! Learn how to build attention-based models, and gain intuition about how they work]]></description><link>https://offsquare.substack.com/p/transformers-from-scratch-in-pytorch</link><guid isPermaLink="false">https://offsquare.substack.com/p/transformers-from-scratch-in-pytorch</guid><dc:creator><![CDATA[Frank Odom]]></dc:creator><pubDate>Fri, 21 Jul 2023 02:08:54 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!GsW4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<blockquote><p><em><strong>Update:</strong> </em>The main article with better image / equation formatting is published on my personal site at <a href="https://fkodom.com/blog/transformers-from-scratch">fkodom.com</a>.<em><strong><br><br>Update:</strong></em> I created this GitHub repo containing all of the code from this article, plus basic unit tests: <a href="https://github.com/fkodom/transformer-from-scratch">fkodom/transformer-from-scratch</a></p></blockquote><h4>Why Another Transformer Tutorial?</h4><p>Since they were first introduced in <a href="https://arxiv.org/pdf/1706.03762.pdf">Attention Is All You Need</a> (2017), Transformers have been the state-of-the-art for natural language processing. Recently, we have also seen Transformers applied to computer vision tasks with very promising results (see <a href="https://arxiv.org/pdf/2005.12872.pdf">DETR</a>, <a href="https://openreview.net/pdf?id=YicbFdNTTy">ViT</a>). <a href="https://openreview.net/pdf?id=YicbFdNTTy">Vision Transformers</a>, for example, now outperform all CNN-based models for image classification! Many people in the deep learning community (myself included) believe that an <em>attention revolution</em> is imminent&#8202;&#8212;&#8202;that is, that attention-based models will soon replace most of the existing state-of-the-art methods.</p><p>All deep learning practitioners should familiarize themselves with Transformers in the near future. Plenty of other Transformer articles exist, both on Medium and across the web. But I learn best by doing, so I set out to build my own PyTorch implementation. In this article, I hope to bring a new perspective and encourage others to join the revolution.</p><h4>Attention Mechanisms</h4><p>As the title &#8220;Attention Is All You Need&#8221; suggests, Transformers are centered around attention mechanisms. Attention is described in the paper&#8217;s abstract:</p><blockquote><p>&#8220;Attention mechanisms have become an integral part of compelling sequence modeling and transduction models in various tasks, allowing modeling of dependencies without regard to their distance in the input or output sequences.&#8221;</p></blockquote><p>In plain English, attention mechanisms relate data points within sequences. And they are <strong>very</strong> good at doing that.</p><p>Transformers use a specific type of attention mechanism, referred to as <em>multi-head attention. </em>This is the most important part of the model! Once you understand multi-head attention, it is pretty easy to understand Transformers as a whole. An illustration from the paper is shown below.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!GsW4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!GsW4!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png 424w, https://substackcdn.com/image/fetch/$s_!GsW4!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png 848w, https://substackcdn.com/image/fetch/$s_!GsW4!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png 1272w, https://substackcdn.com/image/fetch/$s_!GsW4!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!GsW4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png" width="775" height="418" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:418,&quot;width&quot;:775,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:118809,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!GsW4!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png 424w, https://substackcdn.com/image/fetch/$s_!GsW4!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png 848w, https://substackcdn.com/image/fetch/$s_!GsW4!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png 1272w, https://substackcdn.com/image/fetch/$s_!GsW4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d42c45c-ff92-443d-a77d-e88b8105171b_775x418.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Source: <a href="https://arxiv.org/pdf/1706.03762.pdf">Vaswani et. al. </a><em><a href="https://arxiv.org/pdf/1706.03762.pdf">Attention is All You Need</a></em></figcaption></figure></div><p>Let&#8217;s start with scaled dot-product attention, since we also need it to build the multi-head attention layer. Mathematically, it is expressed as:</p><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!70AB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!70AB!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png 424w, https://substackcdn.com/image/fetch/$s_!70AB!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png 848w, https://substackcdn.com/image/fetch/$s_!70AB!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png 1272w, https://substackcdn.com/image/fetch/$s_!70AB!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!70AB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png" width="441" height="83" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:83,&quot;width&quot;:441,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:25333,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!70AB!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png 424w, https://substackcdn.com/image/fetch/$s_!70AB!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png 848w, https://substackcdn.com/image/fetch/$s_!70AB!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png 1272w, https://substackcdn.com/image/fetch/$s_!70AB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa39e8cfe-90b3-459a-a107-a7baf646251f_441x83.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a></figure></div><p>What exactly is happening here? Q, K, and V are <em>batches </em>of matrices, each with shape <code>(batch_size, seq_length, num_features)</code>. Multiplying the <em>query</em> (Q) and <em>key</em> (K) arrays results in a <code>(batch_size, seq_length, seq_length)</code> array, which tells us roughly <em>how important</em> each element in the sequence is. This is the <em>attention</em> of this layer&#8202;&#8212;&#8202;it determines which elements we &#8220;pay attention&#8221; to. The attention array is normalized using softmax, so that all of the weights sum to one. (Because we can&#8217;t pay more than 100% attention, right?) Finally, the attention is applied to the <em>value</em> (V) array using matrix multiplication.</p><p>Coding the scaled dot-product attention is pretty straightforward&#8202;&#8212;&#8202;just a few matrix multiplications, plus a softmax function. For added simplicity, we omit the optional <strong>Mask</strong> operation.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!NawC!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!NawC!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png 424w, https://substackcdn.com/image/fetch/$s_!NawC!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png 848w, https://substackcdn.com/image/fetch/$s_!NawC!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png 1272w, https://substackcdn.com/image/fetch/$s_!NawC!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!NawC!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png" width="1360" height="632" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:632,&quot;width&quot;:1360,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:100316,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!NawC!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png 424w, https://substackcdn.com/image/fetch/$s_!NawC!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png 848w, https://substackcdn.com/image/fetch/$s_!NawC!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png 1272w, https://substackcdn.com/image/fetch/$s_!NawC!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e3bfc0-de09-4d72-8716-eeef1a4ee962_1360x632.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Note that <strong>MatMul</strong> operations are translated to <code>torch.bmm</code> in PyTorch. That&#8217;s because Q, K, and V (<em>query</em>, <em>key</em>, and <em>value</em> arrays) are <em>batches</em> of matrices, each with shape <code>(batch_size, sequence_length, num_features)</code>. Batch matrix multiplication is only performed over the last two dimensions.</p><p>From the diagram above, we see that multi-head attention is composed of several identical <em>attention heads.</em> Each attention head contains 3 linear layers, followed by scaled dot-product attention. Let&#8217;s encapsulate this in an <code>AttentionHead</code> layer:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!kxsI!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!kxsI!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png 424w, https://substackcdn.com/image/fetch/$s_!kxsI!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png 848w, https://substackcdn.com/image/fetch/$s_!kxsI!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png 1272w, https://substackcdn.com/image/fetch/$s_!kxsI!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!kxsI!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png" width="1360" height="706" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:706,&quot;width&quot;:1360,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:120645,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!kxsI!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png 424w, https://substackcdn.com/image/fetch/$s_!kxsI!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png 848w, https://substackcdn.com/image/fetch/$s_!kxsI!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png 1272w, https://substackcdn.com/image/fetch/$s_!kxsI!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F36fa6e83-cce8-4adb-ac1a-df68be343d95_1360x706.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Now, it&#8217;s very easy to build the multi-head attention layer. Just combine <code>num_heads</code> different attention heads and a <code>Linear</code> layer for the output.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!WkFX!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!WkFX!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png 424w, https://substackcdn.com/image/fetch/$s_!WkFX!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png 848w, https://substackcdn.com/image/fetch/$s_!WkFX!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png 1272w, https://substackcdn.com/image/fetch/$s_!WkFX!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!WkFX!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png" width="1360" height="594" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/cfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:594,&quot;width&quot;:1360,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:122235,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!WkFX!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png 424w, https://substackcdn.com/image/fetch/$s_!WkFX!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png 848w, https://substackcdn.com/image/fetch/$s_!WkFX!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png 1272w, https://substackcdn.com/image/fetch/$s_!WkFX!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfe2dd94-678e-408d-bb81-683ed51b9f9c_1360x594.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Let&#8217;s pause again to examine what&#8217;s going on in the <code>MultiHeadAttention</code> layer. Each attention head computes its own <em>query, key, </em>and <em>value</em> arrays, and then applies scaled dot-product attention. Conceptually, this means each head can <em>attend</em> to a different part of the input sequence, independent of the others. Increasing the number of attention heads allows us to &#8220;pay attention&#8221; to more parts of the sequence at once, which makes the model more powerful.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Frank Odom | Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h4>Positional Encoding</h4><p>We need one more component before building the complete transformer: positional encoding. Notice that <code>MultiHeadAttention</code> has no trainable components that operate over the <em>sequence dimension</em> (axis 1). Everything operates over the <em>feature dimension</em> (axis 2), and so it is <em>independent</em> <em>of sequence length</em>. We have to provide positional information to the model, so that it knows about the relative position of data points in the input sequences.</p><p>Vaswani et. al. encode positional information using trigonometric functions, according to the equation:</p><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!ZVlf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!ZVlf!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png 424w, https://substackcdn.com/image/fetch/$s_!ZVlf!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png 848w, https://substackcdn.com/image/fetch/$s_!ZVlf!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png 1272w, https://substackcdn.com/image/fetch/$s_!ZVlf!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!ZVlf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png" width="456" height="97" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:97,&quot;width&quot;:456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:47131,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!ZVlf!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png 424w, https://substackcdn.com/image/fetch/$s_!ZVlf!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png 848w, https://substackcdn.com/image/fetch/$s_!ZVlf!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png 1272w, https://substackcdn.com/image/fetch/$s_!ZVlf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c10f284-ce36-4d15-9c09-dd08c69cc73f_456x97.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a></figure></div><p>We can implement this in just a few lines of code:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!SZRg!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!SZRg!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png 424w, https://substackcdn.com/image/fetch/$s_!SZRg!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png 848w, https://substackcdn.com/image/fetch/$s_!SZRg!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png 1272w, https://substackcdn.com/image/fetch/$s_!SZRg!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!SZRg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png" width="1440" height="558" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:558,&quot;width&quot;:1440,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:116552,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!SZRg!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png 424w, https://substackcdn.com/image/fetch/$s_!SZRg!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png 848w, https://substackcdn.com/image/fetch/$s_!SZRg!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png 1272w, https://substackcdn.com/image/fetch/$s_!SZRg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F22798a48-1213-444e-b936-2f44ac92fd0c_1440x558.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><blockquote><p><strong>Note:</strong> I&#8217;ve gotten several questions about this code. In the equation above, there is a factor of two in the phase exponent. But it is applied at index <code>2i (+1)</code> in the positional encoding. These factors of two should offset one another, and so I do not include it in my code. I believe this is correct, but it&#8217;s possible that I&#8217;ve missed something. Please leave me a comment if you see anything that needs fixing. &#9786;</p></blockquote><p>Now, you may be thinking, &#8220;Why use such an unusual encoding? Surely, there are simpler choices!&#8221; You&#8217;re not wrong, and this was my first thought as well. According to the authors,</p><blockquote><p>We also experimented with using learned positional embeddings instead, and found that the two versions produced nearly identical results. We chose the sinusoidal version because it may allow the model to extrapolate to sequence lengths longer than the ones encountered during training.</p></blockquote><p>Why should sinusoidal encodings extrapolate to longer sequence lengths? Because sine/cosine functions are periodic, and they cover a range of [0, 1]. Most other choices of encoding would not be periodic or restricted to the range [0, 1]. Suppose that, during inference, you provide an input sequence longer than any used during training. Positional encoding for the last elements in the sequence could be different than anything the model has seen before. For those reasons, and despite the fact that learned embeddings appeared to perform equally as well, the authors still chose to use sinusoidal encoding. (I personally prefer learned embeddings, because they&#8217;re easier to implement and debug. But we&#8217;ll follow the authors for this article.)</p><h4>The Transformer</h4><p>Finally, we&#8217;re ready to build the Transformer! Let&#8217;s take a look at the complete network diagram:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!P1nB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!P1nB!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png 424w, https://substackcdn.com/image/fetch/$s_!P1nB!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png 848w, https://substackcdn.com/image/fetch/$s_!P1nB!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png 1272w, https://substackcdn.com/image/fetch/$s_!P1nB!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!P1nB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png" width="554" height="734.1937716262976" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:766,&quot;width&quot;:578,&quot;resizeWidth&quot;:554,&quot;bytes&quot;:198933,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!P1nB!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png 424w, https://substackcdn.com/image/fetch/$s_!P1nB!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png 848w, https://substackcdn.com/image/fetch/$s_!P1nB!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png 1272w, https://substackcdn.com/image/fetch/$s_!P1nB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7b80cf7-f640-4fdd-a1ae-ea6c28527b08_578x766.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Notice that the transformer uses an encoder-decoder architecture. The encoder (left) processes the input sequence and returns a <em>feature vector</em> (or <em>memory</em> vector). The decoder processes the target sequence, and incorporates information from the encoder memory. The output from the decoder is our model&#8217;s prediction!</p><p>We can code the encoder/decoder modules independently of one another, and then combine them at the end. But first we need a few more pieces of information, which aren&#8217;t included in the figure above. For example, how should we choose to build the feed forward networks?</p><blockquote><p>Each of the layers in our encoder and decoder contains a fully connected feed-forward network, which&nbsp;&#8230; consists of two linear transformations with a ReLU activation in between. The dimensionality of input and output is 512, and the inner-layer has dimensionality 2048.</p></blockquote><p>This gives a simple implementation for the <strong>Feed Forward</strong> modules above:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!D0JJ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!D0JJ!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png 424w, https://substackcdn.com/image/fetch/$s_!D0JJ!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png 848w, https://substackcdn.com/image/fetch/$s_!D0JJ!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png 1272w, https://substackcdn.com/image/fetch/$s_!D0JJ!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!D0JJ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png" width="1440" height="446" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:446,&quot;width&quot;:1440,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:68588,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!D0JJ!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png 424w, https://substackcdn.com/image/fetch/$s_!D0JJ!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png 848w, https://substackcdn.com/image/fetch/$s_!D0JJ!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png 1272w, https://substackcdn.com/image/fetch/$s_!D0JJ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa0dc2465-77e2-44c3-b46c-8e87ea56b5a7_1440x446.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>What kind of normalization should be used? Do we need any regularization, such as dropout layers?</p><blockquote><p>The output of each sub-layer is LayerNorm(x + Sublayer(x)), where Sublayer(x) is the function implemented by the sub-layer itself.&nbsp;&#8230; We apply dropout to the output of each sub-layer, before it is added to the sub-layer input and normalized.</p></blockquote><p>We can encapsulate all of this in a Module:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!mPN2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!mPN2!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png 424w, https://substackcdn.com/image/fetch/$s_!mPN2!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png 848w, https://substackcdn.com/image/fetch/$s_!mPN2!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png 1272w, https://substackcdn.com/image/fetch/$s_!mPN2!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!mPN2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png" width="1440" height="558" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:558,&quot;width&quot;:1440,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:131546,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!mPN2!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png 424w, https://substackcdn.com/image/fetch/$s_!mPN2!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png 848w, https://substackcdn.com/image/fetch/$s_!mPN2!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png 1272w, https://substackcdn.com/image/fetch/$s_!mPN2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1aba1f18-35df-4de8-826c-9f3562f796e8_1440x558.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Time to dive in and create the encoder. Using the utility methods we just built, this is pretty easy.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!8OV3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!8OV3!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png 424w, https://substackcdn.com/image/fetch/$s_!8OV3!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png 848w, https://substackcdn.com/image/fetch/$s_!8OV3!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png 1272w, https://substackcdn.com/image/fetch/$s_!8OV3!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!8OV3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png" width="1440" height="2084" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:2084,&quot;width&quot;:1440,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:347265,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!8OV3!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png 424w, https://substackcdn.com/image/fetch/$s_!8OV3!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png 848w, https://substackcdn.com/image/fetch/$s_!8OV3!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png 1272w, https://substackcdn.com/image/fetch/$s_!8OV3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83d254d5-3a58-4dae-9175-998ab15e3529_1440x2084.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The decoder module is extremely similar. Just a few small differences:</p><ul><li><p>The decoder accepts two arguments (<em>target</em> and <em>memory</em>), rather than one.</p></li><li><p>There are two multi-head attention modules per layer, instead of one.</p></li><li><p>The second multi-head attention accepts <em>memory</em> for two of its inputs.</p></li></ul><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!jiTm!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!jiTm!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png 424w, https://substackcdn.com/image/fetch/$s_!jiTm!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png 848w, https://substackcdn.com/image/fetch/$s_!jiTm!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png 1272w, https://substackcdn.com/image/fetch/$s_!jiTm!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!jiTm!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png" width="1440" height="2346" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:2346,&quot;width&quot;:1440,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:426788,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!jiTm!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png 424w, https://substackcdn.com/image/fetch/$s_!jiTm!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png 848w, https://substackcdn.com/image/fetch/$s_!jiTm!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png 1272w, https://substackcdn.com/image/fetch/$s_!jiTm!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a25970a-0005-4566-b0ba-66150ccbf618_1440x2346.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Lastly, we need to wrap everything up into a single <code>Transformer</code> class. This requires minimal work, because it&#8217;s nothing new&#8202;&#8212;&#8202;just throw an encoder and decoder together, and pass data through them in the correct order.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!7bPg!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!7bPg!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png 424w, https://substackcdn.com/image/fetch/$s_!7bPg!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png 848w, https://substackcdn.com/image/fetch/$s_!7bPg!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png 1272w, https://substackcdn.com/image/fetch/$s_!7bPg!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!7bPg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png" width="1440" height="1228" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1228,&quot;width&quot;:1440,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:220709,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!7bPg!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png 424w, https://substackcdn.com/image/fetch/$s_!7bPg!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png 848w, https://substackcdn.com/image/fetch/$s_!7bPg!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png 1272w, https://substackcdn.com/image/fetch/$s_!7bPg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3864947a-bc47-4c50-b31e-d8587a108e5c_1440x1228.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>And we&#8217;re done! Let&#8217;s create a simple test, as a sanity check for our implementation. We can construct random tensors for <code>src</code> and <code>tgt</code>, check that our model executes without errors, and confirm that the output tensor has the correct shape.</p><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!cwAK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!cwAK!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png 424w, https://substackcdn.com/image/fetch/$s_!cwAK!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png 848w, https://substackcdn.com/image/fetch/$s_!cwAK!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png 1272w, https://substackcdn.com/image/fetch/$s_!cwAK!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!cwAK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png" width="1440" height="334" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/dd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:334,&quot;width&quot;:1440,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:54329,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!cwAK!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png 424w, https://substackcdn.com/image/fetch/$s_!cwAK!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png 848w, https://substackcdn.com/image/fetch/$s_!cwAK!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png 1272w, https://substackcdn.com/image/fetch/$s_!cwAK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd5c49bd-87b8-4f32-bcca-6fdbd618ba29_1440x334.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a></figure></div><h4>Conclusions</h4><p>I hope this helps to shed some light on Transformers, how they&#8217;re built, and how they work. Computer vision folks (like myself) may not have encountered these models before, but I expect to see much more of them in the next couple of years. <a href="https://arxiv.org/pdf/2005.12872.pdf">DETR</a> and <a href="https://openreview.net/pdf?id=YicbFdNTTy">ViT</a> have already shown ground-breaking results. It&#8217;s only a matter of time before other SOTA models fall to Transformers as well. In particular, I&#8217;ll be waiting expectantly on end-to-end attention-based models for object detection, image segmentation, and image generation.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption"><strong>Frank Odom | Substack</strong> is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><div class="captioned-button-wrap" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/transformers-from-scratch-in-pytorch?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="CaptionedButtonToDOM"><div class="preamble"><p class="cta-caption">Thank you for reading Frank Odom | Substack. This post is public so feel free to share it.</p></div><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://offsquare.substack.com/p/transformers-from-scratch-in-pytorch?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://offsquare.substack.com/p/transformers-from-scratch-in-pytorch?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p></div>]]></content:encoded></item></channel></rss>