How to Evaluate a Growth Marketing Agency: A Buyer's Guide

TL;DR

Hiring a growth marketing agency well is mostly a procurement problem disguised as a marketing problem. The agencies that look most impressive in a pitch are not always the ones that deliver. What separates them is process, attribution discipline, and a willingness to be measured. This guide gives you a 7-part evaluation framework, ten vetting questions to ask on a discovery call, and an open-source scorecard you can use to compare any three agencies head-to-head.

Who This Guide Is For

You're the right reader if you fit one of these patterns:

• A founder or marketing lead evaluating your first external agency, somewhere between Series A and Series C.
• A CMO or VP Marketing replacing an incumbent agency that isn't producing results.
• An operator who has done growth in-house and now needs specialist execution (paid acquisition, SEO, CRO) at a level you don't have time to manage yourself.

If you're a pre-seed startup with no revenue, an agency is probably premature. Most agencies need a minimum monthly retainer that won't pay back at your stage.

The Three Real Alternatives (And Why Each Wins)

Before you hire an agency, be honest about whether an agency is what you actually need. The four credible options for outsourced growth marketing work are:

• Freelancers and contractors. Best for narrow scopes (a Google Ads audit, a landing-page redesign) and budgets under ~$5K/month.
• In-house hires. Best when you have one channel that will reliably drive more than 50% of revenue and you can recruit a senior operator for it. The all-in cost of a senior growth hire is roughly $200–280K per year in major US metros.
• Fractional CMO or fractional Head of Growth. Best when you need strategy and leadership more than execution. Typical engagement is 10–20 hours per week at $8–20K per month.
• Growth marketing agency. Best when you need multi-channel execution, want to pay variable rather than fixed cost, and want a team that has run experiments across many companies so you don't pay for tuition on your dime.

Roughly, agencies win on three axes: cross-channel coverage, pattern recognition from many clients, and variable cost. If those aren't your top three needs, one of the other options will serve you better.

The 7-Part Evaluation Framework

These are the criteria that actually correlate with agency outcomes, in roughly the order you should weight them.

1. Attribution Discipline

How does the agency claim credit for results? This is the single biggest predictor of whether the relationship will end well.

Green flags

They volunteer the limits of their attribution. They use last-click and incrementality testing. They reference holdout groups, geo-experiments, or media-mix modeling. They distinguish between claimed conversions (what the platform says) and actual incremental revenue (what the business saw).

Red flags

Reports show only platform-claimed ROAS, with no triangulation. They take credit for organic and direct traffic that "halo'd" from paid campaigns. They can't explain how they'd measure the agency's value if you paused all spend for two weeks.

Question to ask

"If we paused all paid spend for two weeks, what would you expect to happen to revenue, and how would we know whether it was the pause or a confounding factor?"

2. Senior-to-Junior Ratio on Your Account

Agencies often pitch with senior talent and then staff with juniors. Find out who will actually touch your account week to week.

Green flags

Named team members with LinkedIn profiles you can vet. Clarity about who runs strategy, who runs execution, and who you'll email for routine requests. A named senior who is accountable and present in monthly reviews.

Red flags

Vague "team of experts." The pitch team disappears after the contract is signed. The day-to-day contact has been at the agency for three months.

Question to ask

"Who specifically will be running my account in month three, and can I meet them before signing?"

3. Domain Fit

Have they done work in your specific business model? B2B SaaS, DTC ecommerce, marketplace, fintech, and healthcare each require different playbooks. An agency that's brilliant at DTC apparel can be mediocre at B2B SaaS, and vice versa.

Green flags

Three or more recent case studies in your business model. They can articulate the specific economics of your model (CAC payback period for SaaS, contribution margin for DTC, cohort retention curves for marketplaces).

Red flags

Case studies are exclusively from one business model that isn't yours, or are five or more years old. They use the same generic playbook regardless of model.

Question to ask

"What's different about marketing a [your business model] versus your other clients?"

4. Process and Cadence

What does Tuesday morning at this agency actually look like? Growth is won by compounding small experiments, which requires a real operating cadence.

Green flags

Documented experimentation framework (PIE, ICE, RICE, or similar). Weekly experiment logs. Monthly business reviews with clear agenda. Defined escalation paths. A shared dashboard you can check anytime.

Red flags

"We're agile" with no further specification. Monthly reports are static PDFs you receive on the 15th of the following month. No shared dashboard.

Question to ask

"Walk me through your experimentation cadence. How many tests will you run on my account in a typical month, and how do you decide priority?"

5. Knowledge Transfer

A great agency makes you smarter every month. A mediocre one creates dependency.

Green flags

Documented playbooks delivered to you. Loom walkthroughs of optimizations. Quarterly knowledge-transfer sessions. They're comfortable with you eventually building in-house and they'll help you do it.

Red flags

They refuse to share account access. They won't document workflows. They're visibly anxious about you hiring in-house.

Question to ask

"If we wanted to bring this in-house in 18 months, how would you support that transition?"

6. Reference Quality

Past clients will tell you what the agency is like to work with. Make sure to talk to at least two, including one client they didn't choose for you.

Green flags

They offer references easily. References describe both wins and tradeoffs. They'll connect you to a former client whose engagement ended (and the parting was civil).

Red flags

They refuse or stall on references. All references are gushing with no specifics. They won't connect you to anyone whose contract has ended.

Question to ask the references

"What surprised you about working with them, good and bad?"

7. Cultural Fit and Communication Cadence

You're going to talk to these people every week for a year or more. The relationship is real.

Green flags

Communication style matches yours. Time zones overlap with your team. They're willing to use your tools (Slack, Linear, Notion) rather than insisting on theirs.

Red flags

They're slow to respond during the sales process, and it will only get slower after signing. They push back on basic accommodations. You don't like talking to them.

10 Questions to Ask on Every Discovery Call

Take these into your first conversation. The agencies that answer them clearly will not all be the right fit, but the ones that can't answer them clearly are almost never the right fit.

1. "Show me a client whose engagement ended in the last 18 months. Why did it end?" This tests honesty about churn.
2. "What's a tactic you used to recommend that you no longer recommend, and why?" This tests intellectual honesty and learning.
3. "What KPI will you commit to in writing in month 3 and month 6?" This forces specificity.
4. "How do you measure incrementality, separate from platform-attributed conversions?" This tests attribution sophistication (see criterion #1).
5. "What does your weekly experiment volume look like on accounts similar to ours?" This tests operating cadence.
6. "Who on your team will I email at 9pm when something is broken?" This tests staffing reality.
7. "What's the smallest engagement you'd accept, and what's the smallest engagement you'd recommend?" This tests their honesty about minimum viable scope.
8. "What's the typical reason your engagements get bigger over time? What's the typical reason they get smaller?" This tells you what kind of relationship you're entering.
9. "What client of yours is most like us, and what would you do differently for us?" This tests pattern recognition.
10. "What would have to be true in month 6 for you to recommend we end this engagement?" This tests willingness to be measured.

Red Flags Checklist

If you see any two of these in a sales process, slow down:

• [ ] Refuses to share who specifically will work on your account
• [ ] Reporting consists only of platform screenshots
• [ ] Won't commit to any KPI in writing
• [ ] Hard sell on signing today, or pricing only valid this month
• [ ] All case studies are three or more years old, or anonymized to the point of being uncheckable
• [ ] References are all from current clients, none from past clients
• [ ] Pitches a one-size-fits-all "framework" without adaptation to your business
• [ ] Tells you in the first call exactly what they'd do, before doing any diagnostic
• [ ] Doesn't ask about your unit economics, CAC payback, or contribution margin
• [ ] Charges a high setup fee but has no clear deliverables tied to it

The Evaluation Scorecard

Score each candidate agency 1–5 on each criterion, weight per the multipliers below, and compare totals. The weights matter. Attribution discipline is twice as important as cultural fit.

Agencies that score above 50 on this scorecard tend to deliver. Agencies that score below 35 rarely do. The middle is where most of the real judgment calls live.

What to Negotiate Before Signing

A few terms are often left as the agency's defaults but you can usually move them:

1. Notice period after the initial term. Default is often 60–90 days. Thirty is reasonable for a healthy relationship.
2. Ownership of accounts and creative. Make sure ad accounts, analytics, and any creative produced are owned by you. Watch the contract language carefully.
3. A "no-poach" clause that goes both ways. Standard agency contracts often forbid you from hiring their team. A symmetric clause is fair.
4. A pause clause. Lets you pause the retainer for a defined period (for example, 30 days) without terminating, for cash-flow flexibility.
5. A defined ramp-up period. First 30–60 days at a reduced rate while they audit and onboard, then full retainer once execution begins.

Frequently Asked Questions

How long should I expect before a growth agency drives meaningful results?

For paid acquisition, expect a 30–60 day audit-and-rebuild phase, then 60–90 days of optimization before you see clean signal. For SEO, expect 4–6 months minimum, and often 12 months before the curve compounds visibly. Anyone promising results in week one is selling you something other than growth.

What's the difference between a growth marketing agency and a digital marketing agency?

The labels overlap. In practice, "growth marketing" implies a full-funnel, experiment-driven approach with attention to retention and expansion, not just acquisition. "Digital marketing" is often acquisition-focused (paid ads, SEO). Both are legitimate. Choose based on the work, not the label.

Should I hire one agency for everything or specialists for each channel?

A single agency is simpler operationally and produces more coherent strategy. Specialists are often better at their individual craft. The best-of-both-worlds setup is one agency for strategy and one or two channels of execution, plus a specialist freelancer for any channel where the agency is weakest.

Can an agency replace an in-house team?

Replace, no. Substitute for a period, yes. Most companies that scale eventually build in-house and use agencies for specialist execution or overflow capacity. Plan for that arc from the start.

How do I know if my agency is overcharging?

The honest test is to ask what hourly rate is implied by your retainer divided by the hours actually being worked on your account. If you don't know the answer, ask. If the implied rate is above $400 per hour for execution work (not strategy), you're probably overpaying. Strategy time can legitimately cost $500–800 per hour from senior operators.

What if my agency isn't working out?

Have an honest mid-engagement review at the 90-day mark, in writing, against the KPIs you agreed to at signing. If the conversation goes well, the relationship usually recovers. If the agency can't have that conversation, that's your answer.

About Deviate Labs

This guide was written by Deviate Labs, a growth marketing agency based in Los Angeles and Seattle, founded in 2014 by Raymond Fong and Chad Riddersen. We're the authors of Growth Hacking: Silicon Valley's Best Kept Secret, and we operate around a proprietary framework called the ASP™ (Automated Sales Process), a six-step flywheel covering Attraction, First Impression, Engage & Educate, Follow-Up, Sales Technology, and Referrals & Retention.

We've worked with Dollar Shave Club, Automattic, SMART Technologies, Hello Bello, A Bad Think, HR Cloud, Tapfiliate, MetaLend, and others. A few examples of our work:

• We took HR Cloud from 5,100 monthly organic visitors to 83,300, a 16x lift sustained over five years. Read the case study →
• We helped Tapfiliate scale advertising profitably to the point of acquisition by a $100M multinational. Read the case study →
• We helped MetaLend hit a $0.38 CPA in India and a 15.7% conversion rate on their /earn page in regulated crypto advertising. Read the case study →
• We ran BreakTheCode.TECH for Radix, generating 809K+ visitors and a Webby nomination on a non-promotional escape-room concept. Read the case study →

If, after going through the framework above, you'd like to evaluate us as a candidate against the same scorecard you'd use on anyone else, book a discovery call. We'll answer all ten of the questions above on the call, in writing if you prefer.