Usual implementation of attention transformers (SDPA) is kind of bad, actually
The article critiques the standard transformer architecture (SDPA) used in machine learning, arguing that it may not be as effective as commonly believed. The author suggests that large AI companies promote expensive models to maintain their competitive advantage. While not dismissing SDPA entirely, the piece raises questions about its necessity and hints at the potential for better alternatives in the future.
- ▪The author believes that big AI companies shape the industry to favor expensive models due to their competitive advantages.
- ▪The article discusses the inefficiencies of the standard transformer architecture (SDPA) in machine learning.
- ▪It highlights the historical context of various machine learning models, including fully connected networks, recurrent networks, and convolutional networks.
Hacker News (Newest) files mainly under programming. We currently carry 5,306 of its stories.
Story provenance
Source · retrieval · rights · ranking — open for full record
inspect →
Story provenance
Attribution is not the same as permission. This drawer separates discovery metadata, excerpts, WeSearch-generated summaries, reuse status, and whether the publisher receives the visit. Nothing here claims a legal grant the publisher has not made.
Record
| Original publisher | Gist |
| Canonical URL | https://gist.github.com/celoyd/6bf10122c3f5f7e64b0c684704e4ffb2 |
| Publication time | Mon, 18 May 2026 04:23:42 +0000 |
| Retrieval time | 2026-05-18T04:34:54.418Z |
| Last seen | 2026-05-18T04:34:54.418Z |
| Headline source | Publisher (no WeSearch rewrite) |
| Excerpt source | publisher body |
| Excerpt method | First ~120 words (~800 chars) of extracted publisher body, fair-use limited. |
| Summary | WeSearch · cerebras-chat (WeSearch summarizer) |
| Summary source text | contentText |
| Citation coverage | Summary is a WeSearch-generated derivative; primary citation is the original publisher URL. |
| Cluster | 9AuwkWP4kj4h |
| Cluster logic | Grouped by semantic title/content similarity across sources within a rolling window. Same-publisher template collisions are excluded from coverage comparison. |
| Ranking reason | Story pages are not engagement-ranked. Hub feeds use recency, with optional source-diversified chronological ordering (cap consecutive stories per source). No personalized ranking. |
| Publisher visit | Yes — open original |
| Substitutes article? | No — link-out required for full text |
Rights status (four layers)
WeSearch handling by dimension
| Indexing | May the item be indexed (stored, ranked, made findable)? | Allowed |
| Snippet | May a short excerpt of the publisher's text be shown? | Allowed |
| AI summary | May WeSearch generate its own short summary of the article? | Limited |
| Retrieval / RAG | May the content be exposed for third-party retrieval-augmented generation? | Not asserted |
| Model training | May the content be used to train AI models? | Not asserted |
| Commercial reuse | May the content be reused commercially? | Not permitted |
Basis: Derived from the published RSS/Atom feed. Contact: [email protected]. Reviewed: 2026-07-24.
Opening excerpt (first ~120 words) tap to expand
Introduction I was writing a note to a friend that mentioned my tedious opinions on “AI” discourse. It veered off into my usual argument that big “AI” companies are shaping the industry ecosystem to their own ends by setting up a situation where expensive-to-run models are overvalued. I think they’re doing this because they have a competitive advantage in that tier of the market, having bought (time on) a lot of GPUs. It’s like how a company that owns diamond mines will probably promote the idea that large, mined diamonds are important and valuable, and that there’s something off about running a sub-industrial mine or lab-growing diamonds. You can do this without lying at all, but I still dislike it. Large mined diamonds here are $O(n^2)$ models.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Gist.
Discussion
0 commentsMore from Gist
