Vendor Evaluation Routine
A routine for evaluating an AI marketing vendor in a vendor-skeptical posture — practitioner references, working deployments, documentation quality, maintenance-burden modeling.
Purpose
Replace the vendor-led evaluation process — the one driven by the vendor's sales motion, their reference customers, and their slide deck — with a vendor-skeptical evaluation routine that the buyer drives.
Outcomes
- Structured evaluation across the vendor short list
- Independent practitioner references
- Realistic two-year maintenance-burden projection
- Defensible procurement decision
Tools and agents
Steps
Define the evaluation criteria in writing
Write down what 'this tool fits us' means. Capability coverage. Integration shape. Commercial model. Maintenance burden. Team-skill fit. The criteria are the buyer's, not the vendor's.
Collect independent practitioner references
The reference-collection agent finds practitioners running the tool in production who were not provided by the vendor. The vendor's references are heard, but weighted appropriately.
Audit the vendor's documentation
The documentation-audit agent reads the vendor's full published documentation. Sparse, evasive, or contradictory documentation is itself a finding. Documentation quality predicts production reality.
Analyze a working deployment
If possible, get hands-on time in a working deployment of the tool — not a sandbox demo. Verify the capability claims against the working deployment.
Model the two-year maintenance burden
The maintenance-burden modeling framework projects what operating this tool costs the team across the next two years, accounting for integration drift, schema changes, and the realistic shape of the team's evolution. Most procurement decisions ignore this layer; we treat it as load-bearing.
Decision card
The findings are synthesized into a decision card the procurement decision-maker reviews. Buy, do not buy, or revisit on a defined timeline.
Antipatterns to avoid
- Letting the vendor define the evaluation criteria
- Treating the vendor-provided references as sufficient
- Ignoring documentation quality
- Skipping the two-year maintenance-burden modeling
Glossary terms used in this workflow
- Routine — A named, repeatable agentic workflow that a team can invoke on demand.
Other workflows in the library
- GEO Content Production Line Generative Engine Optimization · A retrieval-augmented production pipeline that ships reference-grade content tuned for answer-engine citation rather th…
- Agentic SEO Audit Diagnostic · A multi-agent audit routine that produces a working diagnostic of a site's SEO, GEO, and entity-layer health in a singl…
- AI-Assisted Lifecycle Marketing Setup Lifecycle Orchestration · A staged routine that builds out an agentic lifecycle program — identity layer, stage definitions, agentic routines, hu…
- Agency Pitch Routine New Business · A routine that produces a credible new-business pitch — research, positioning, working creative, illustrative routine d…
- Brand-Voice Memory Bootstrap Memory Layer · A routine that builds out a working brand-voice memory layer from the existing corpus of brand-approved content, ready…
- Buyer-Journey Citation Mapping GEO + Strategy · A diagnostic routine that maps the brand's citation surface across the actual buyer journey — where buyers ask, what th…
- Newsletter Publication Routine Content + Distribution · An end-to-end routine for shipping a recurring newsletter — research, drafting, editing, formatting, sending — that an…
- Competitive Positioning Watch Strategy · A steady-state routine that tracks the positioning, messaging, and citation surface of a defined competitive set, produ…
- In-House Routine Library Build-Out AI Marketing Engineering · A multi-quarter program to build out a working routine library for an in-house AI marketing team, from the first routin…
- Category Launch Program Brand + GEO · A coordinated program for launching a product or company into a new category — entity establishment, citation surface s…
- Audit-Self Publication Routine Editorial + Methodology · A routine for publishing the agency's or team's own work and methods — not as marketing, but as durable reference conte…