With advancements like these, combined with Groq-like hardware (think ASICs in bitcoin mining), the cost of inference could drop to a point where "human in a box"-level intelligence becomes cheap. The timing is perfect to build inference-hungry, multi-modal applications that will thrive on such advances.

But let’s not forget—RAG (Retrieval-Augmented Generation) isn’t going anywhere. Let’s compare the cost of LLM generation - prior to Magic’s breakthrough, and after Magic’s breakthrough.

Please keep in mind that when I talk about cost of LLM generation, I’m referring to time, dollar cost, and compute (i.e., FLOPs). They’re all largely proportional and can be used interchangeably.

Pre Magic Era: The Case for RAG

Let’s break it down. Suppose you have a knowledge base of length m and instructions of length n. Typically, m » n. Now, if we could process LLM inputs that are infinitely long, the time complexity of LLM generation would be O(m² + n²).

Enter RAG. With a RAG-like solution, complexity drops dramatically to O(log m + n²). Here, lookup in a vector database is O(log m), assuming retrieval yields a constant number of chunks, each of constant length. This is such a radical drop in complexity that it’s hard to imagine anyone forgoing this tool, especially if the benchmarking statistics on correctness remain largely unaffected.

Post Magic Era: Why RAG Still Makes Sense

Now, imagine the cost of LLM generation dropping to O(m + n) due to Magic’s breakthrough. Even in this scenario, RAG-like solutions further improve the time complexity to O(log m + n). Given that m is still much greater than n, this reduction in computational complexity remains highly attractive.

TL;DR

The next generation of human-computer interfaces—think voice bots, video chats, or humanoids capable of natural communication—will demand sub-second latency and ultra-low costs to deliver a seamless user experience. Many scenarios will also require searching through web-scale databases to find solutions. To achieve this, RAG-like solutions will remain essential in making such technology both feasible and marketable in consumer applications.

At the end of the day, startups grow because they are rewarded by the market for introducing efficiency in one form or another - and their rewards are a fraction of the efficiency they introduced. If your solution isn't improving market efficiency, no amount of 'stubbornness' is going to make it work - it's time to pivot.

It's very easy to make two wrong assumptions as a startup founder - “finding startup ideas is easy”, and "I'll build it and they'll come"

Mistake #1: “Finding startup ideas is easy”

It is true that there are a near infinite amount of problems you could solve and be rewarded by the free market for it. However, there are also an infinite number of tarpits that merely seem like good ideas. A good comparison would be the fact that there's an infinite number of natural numbers (1, 2, 3, 4, ...) however, if you were to throw a dart on the number line, your probability of landing on a natural number is near 0. It's somewhat of a similar case with startup ideas.

Unfortunately, it's quite hard to solve any such problems unless you specifically have been impacted by them - and it's nearly impossible to know you're going to be in a tarpit till it's too late.

Mistake #2: “I’ll build it and they’ll come”

Prior to starting a company, founders in tech are usually working with people where everyone has been vetted for skill. Neither are your coworkers paying you. However, selling to someone who has NOT vetted you is a couple of orders of magnitude harder. It's hard to convince someone that you'll be able to solve their problem. It's hard to build reputation. One way founders can tackle this is by building something in a domain they are known experts in - for example, a doctor building a product in the medical domain will likely be considered more competent than an engineer doing so alone.

This is also why Paul Graham suggests that you "do things that don't scale" in the beginning. You need to delight your initial customers or they'll give up on you - and you'll be left with nothing to build credibility with.

Credits to Paul Graham, from whose essay I’ve borrowed heavily.

Why Bother? 🤔

Simple. Users are the ones paying you. They're your reality check - keeping you honest and focused on actual problems, not imaginary ones.

To be honest, I don’t expect you to give a sh*t if you are a first time founder. Unfortunately, you won’t value the medicine till you’ve known the pain of being sick.

Finding Your Users 🕵️

Your network is gold, but don't stop there. Cast a wider net:

• Tap into former co-workers

• Leverage LinkedIn (yes, it's still useful)

• Dive into Reddit/Discord communities

• Show up at in-person events

FYI, it becomes exponentially harder to get a slice of someone else’s time as you go down the list. Try to get as much as possible out of your network before you go around hoping that strangers give you the time of day.

Pro tip: Always aim for video calls or face-to-face meetings. Text-based communication doesn't cut it - you miss out on crucial nuances.

Before You Talk 📝

1. Define clear learning goals

2. Identify the right person (hint: it's usually a decision-maker)

3. Craft a compelling outreach message

4. Set up the call

Here's the kicker - don't mention your product at all. It biases the conversation and ruins your data.

Ask the Right Questions 🎯

Good questions:

• "Walk me through how you handle X today."

• "What's the most frustrating part about dealing with X?"

• "How often does this problem crop up?"

• "Why is solving X critical for your company?"

• "What solutions have you tried so far?"

Bad questions:

• "Would you use our product?"

• "What features do you want?"

• Anything with a yes/no answer

These yield garbage data. Don't waste your time.

The Stages of User Conversations 🌱 ➡️ 🌳

Ideation Stage: Plant the Seeds

Goal: Find users actually experiencing the problem.

Approach:

• Milk your network for all it's worth

• Become a regular at industry events

• If needed, embrace the cold outreach grind

MVP Stage: Nurture the Sprouts

Goal: Identify your best first customers.

Method:

1. Rank prospects: Cost * Frequency * Budget

2. Go after your top-ranked prospects like your life depends on it

Launched Stage: Grow the Tree

Goal: Iterate towards that holy grail - product-market fit (PMF).

PMF Indicator: Ask users point-blank: "How would you feel if our product disappeared tomorrow?"

• "Very disappointed"

• "Somewhat disappointed"

• "Meh, I'd live"

If more than 40% say "very disappointed" - congrats, you've hit PMF. Time to scale.

Pro Tips from the Trenches 💡

1. Ask for phone numbers during sign-up. Then use them.

2. Create exclusive WhatsApp/Slack groups for engaged users. Make them feel special.

3. Ruthlessly discard non-quantifiable data - even if it's a compliment. "Your UI looks nice" doesn't pay the bills and neither is it actionable.

4. Test intent with actions - not opinions.

   • For example - if you wish to test whether a feature would be valuable enough for your customers to pay, consider adding a “Pay More to Upgrade” button before you even build out the feature to gauge how many people are willing to do so. Simply add interested users to a waitlist till the feature is actually done.

The Bottom Line

User conversations are your compass that keep you laser-focused on solving real problems - not the ones you've imagined in your startup fever dreams.

Master this skill. It's the difference between building something people actually want and wasting years of your life on a product nobody cares about (believe me, I’ve wasted a year already).

Remember: Write code. Talk to users. Everything else is a distraction.

Now go forth and converse. Your users are waiting. 🚀

Relevant Sources

I’ve relied on quite a few sources to come up with this. Here’s a list of some important ones that may be worth your time:

• A book called “The Mom Test” by Rob Fitzpatrick

• A couple of videos put out by YC