EP 51: Building AI Agents That Actually Work in Production | Shawn Kercher, CEO @HummingAgent.ai

概览

This episode of Data Science with Sam features Sean Kerscher, co-founder and CTO of Hummingagent.ai, discussing the difference between simple AI wrappers and agents designed around real business workflows, private data, and domain-specific rules.

Kerscher argues that models are increasingly interchangeable commodities; the lasting value lies in connecting them to proprietary data, legacy systems, organizational knowledge, and carefully designed workflows. Hummingagent therefore focuses on custom software for small and medium-sized businesses rather than generic, off-the-shelf automation.

The conversation also stresses cautious adoption. Organizations should begin with narrow, sandboxed tasks, retain human review for consequential actions, and expand an agent’s permissions only after testing and feedback. Discovery, data quality, security, and model-independent architecture are presented as essential for moving from a polished demonstration to a reliable production deployment.

分段落总结

[00:00] Why AI Agents Need Gradual Permission

[事实] Kerscher says organizations are still at an early stage of trusting agents with broad permissions.

[事实] For tasks such as email, he prefers having AI prepare a draft while a human verifies the content before it is sent.

[事实] He recommends giving agents access incrementally, allowing them to learn from business context and feedback before expanding their responsibilities.

[推测] This approach treats autonomy as something earned through observed reliability rather than enabled by default.

[00:48] Automation Tools Versus Reasoning Agents

[事实] The introduction notes that many products combine a frontier model with a user interface and market the result as an agent.

[事实] The episode distinguishes tools that automate isolated tasks from agents that reason through broader workflows.

[事实] Host Sam Dey introduces Sean Kerscher as Hummingagent’s co-founder and CTO and frames the company as a response to the risk of building little more than ChatGPT wrappers.

[02:17] The Market Gap Behind Hummingagent

[事实] Kerscher says many existing tools are generic, while Hummingagent develops fully customized software for small and medium-sized companies.

[事实] Its work begins with understanding how a client operates, which systems and APIs are involved, and where a tailored solution could save time or money.

[事实] He says the resulting systems may also reduce the need for certain positions.

[推测] Hummingagent’s primary differentiation is therefore service-led customization rather than a single standardized agent product.

[03:25] Private Workflows and Full-Stack Customization

[事实] Kerscher defines Hummingagent’s agents as private, customized workflows connected to large language models and deployed within the client’s environment.

[事实] He cites an AI search system that lets a company search decades of internal data and use different language models, including newly released ones.

[事实] Hummingagent also builds complete digital applications for service businesses that may still rely on pen-and-paper processes.

[事实] Kerscher says recent AI capabilities have made customized software faster to build and more scalable than it was several years ago.

[07:04] Human Oversight and Contextual Learning

[事实] Kerscher recommends keeping humans in the loop for outputs such as emails rather than granting an agent unrestricted authority.

[事实] He advises customers to take “baby steps”: first provide limited context and access, then expand them as the system learns from feedback.

[事实] The host agrees that agents still require supervision and subject-matter expertise to understand context and make better decisions.

[事实] The host reminds listeners that AI remains an evolving field and argues that they should not fear having already missed the opportunity to participate.

[09:31] Avoiding the ChatGPT Wrapper Trap

[事实] Kerscher says major model providers are becoming integration layers themselves, which threatens products whose only value is connecting a general model to another service.

[事实] He argues that major providers are less likely to reproduce solutions built around a specific company’s rules, proprietary data, and legacy software.

[事实] He describes the underlying model as a commodity and says the surrounding “wiring” is the actual product.

[事实] Examples mentioned include work for a restoration company, a dermatology group operating under HIPAA, and a large fencing company in Massachusetts.

[推测] A defensible AI product must retain value even if a foundation-model provider later adds similar generic functionality.

[11:46] Model Independence, Testing, and Quality Assurance

[事实] Kerscher says Hummingagent avoids becoming dependent on a single model and designs systems so that models can be replaced.

[事实] Different models behave differently depending on the workflow.

[事实] Quality assurance and testing are therefore especially important when switching models.

[推测] Model portability protects a business from rapid vendor changes, but it also creates an ongoing evaluation burden.

[12:18] Building for Messy Production Data

[事实] The host observes that workflow automation often fails around exceptions, context changes, undocumented practices, and memory.

[事实] Kerscher contrasts clean demonstration data with client environments containing years of incomplete CRM records, missing fields, and limited context.

[事实] Hummingagent conducts discovery before writing code and begins working with a copy of the client’s real data from the first day.

[事实] The initial scoping period is used to identify operational rules and capture knowledge that may exist only in employees’ heads.

[事实] The company uses paid, fixed-scope discovery engagements before implementation begins.

[14:17] Why Narrow, Practical Agents May Win

[事实] Kerscher expects agents to change how people use software and believes the traditional per-seat SaaS model will weaken.

[事实] He argues that narrow, vertical solutions are more promising than broad systems and warns against pursuing full autonomy before users trust it.

[事实] He also warns against betting on one vendor or model because new “best” models appear frequently.

[事实] He says businesses should prioritize real deployments over impressive demonstrations and predicts that narrow, practical integrations will succeed.

[推测] The expected competitive advantage lies in dependable execution inside a specific workflow, not in access to whichever model currently leads benchmarks.

[15:34] Security, Sandboxing, and Guardrails

[事实] Kerscher warns that giving an agent complete access can reduce productivity and create security risks.

[事实] He specifically identifies rogue email activity, broken workflows, and access to financial data as concerns.

[事实] His recommended process is to place one narrowly defined capability in a sandbox, test it, and build outward gradually.

[事实] The host adds that financial services, healthcare, and personally identifiable information require appropriate guardrails and supervision over data entering and leaving the agent.

[推测] The discussion supports a least-privilege deployment model in which each new capability is validated before broader access is granted.

[17:41] Hummingagent’s Services and Products

[事实] Kerscher says Hummingagent helps organizations identify time-consuming tasks, study the existing process, and design an AI-based technical solution.

[事实] One example involves replacing manual data review performed by an offshore team.

[事实] The company offers a private GPT that operates inside the customer’s environment, where customer data is not used for training.

[事实] Its second product, Humcrew, is described as a company command center combining voice agents, text messaging, proposal generation, and other operational functions.

[19:14] Turning Decades of Documents into Searchable Knowledge

[事实] Kerscher describes a client with 44 years of data whose legacy research process previously took hours.

[事实] Hummingagent added AI search across the client’s historical PDF collection and reduced research time from hours to minutes.

[事实] The host says time recovered from monotonous tasks can be redirected toward learning, strategic work, or previously deferred projects.

[推测] This case illustrates the clearest value proposition discussed in the episode: converting inaccessible organizational archives into usable, time-saving knowledge.

播客点评/总结

[推测] The episode is most valuable as a practical framework for evaluating agent products. Its strongest ideas are that the model itself is not a durable advantage, real customer data must shape development from the beginning, and autonomy should expand only after controlled testing.

[推测] The conversation is especially relevant to founders, engineering leaders, consultants, and organizations exploring AI in regulated or operationally complex environments. Its emphasis on private deployment, human review, sandboxing, and model portability provides a useful starting checklist.

[推测] Its main limitation is a lack of technical implementation detail and independently verifiable performance evidence. The productivity example is compelling, but the discussion does not explain the system architecture, evaluation criteria, error rates, deployment costs, or how security claims are validated.

[推测] Overall, the episode offers a clear strategic message: successful enterprise agents are likely to be narrow, deeply integrated, grounded in proprietary context, and deliberately constrained rather than universally autonomous.