What's an AI Engineer?
The new AI role, bridging research and reality
TL;DR: Recent advances in AI models have revolutionized the AI landscape. Tasks that previously required 2 years, massive datasets, and a team of 10+ scientists can now be accomplished within a month by an engineer with a scientific foundation. While scientific knowledge remains crucial for aspects like system evaluation, metric definition, and establishing ground truth, the primary focus has shifted to productionizing these models—a predominantly engineering task: The AI Engineer was born, and it’s here to stay.
From Research Labs to Production Environments
Having spent 2.5 years at Amazon in science teams, I was surrounded by dozens of scientists working with vast amounts of data, often training models from scratch and reimplementing research papers from tech giants like Google, Meta, and Microsoft. At Amazon Transportation, multiple models were developed to predict parcel volumes. Similarly, Alexa's Automatic Speech Recognition (ASR) system employed different models for each locale. While I'm not privy to the current specific architectures, I believe that in cases like ASR, the architecture will undergo (or has already undergone) radical changes.
Given these shifts in the AI landscape, I wouldn't be surprised to see a corresponding change in the architecture of the teams at Amazon, with some applied scientists transitioning to core research as research scientists, working on foundation models; while others remain focussed on the final product and transition into AI engineer roles, building on top of the foundation models, for specific use cases(note that this is biased by my experience at Alexa ASR).
The Evolving AI Career Path
My career trajectory—from Research Scientist to Applied Scientist to AI Engineer—mirrors the current AI landscape. As time progresses, the demand for pure research diminishes, while the need for production-ready solutions increases. Recent breakthroughs in Large Language Models (LLMs) have accelerated this trend, significantly reducing the amount of scientific work required to build a production AI application. Although scientific work hasn't disappeared entirely, its proportion has notably decreased.
The Need for AI Engineers: A Paradigm Shift
The rise of AI Engineers is driven by several key factors:
Emergent Capabilities of Foundation Models: Foundation Models, including Large Language Models (LLMs), have demonstrated unexpected abilities in few-shot learning, in-context learning, and zero-shot transfer.
"AI Research as a Service" APIs: Major tech companies like Microsoft, Google, and Meta, have effectively cornered the market on scarce AI research talent. They're offering their research outputs as APIs, allowing others to leverage advanced AI capabilities without needing to hire top-tier researchers directly. This has created a demand for engineers who can effectively work with these APIs.
GPU Scarcity and the Hardware Arms Race: New AI startups are raising huge amounts of capital to secure their own hardware. This trend, combined with global chip shortages, is creating a situation where there's more capacity for using models via APIs than for training them, further driving the need for AI Engineers.
Shift from "Waterfall" to "Agile" AI Development: Forget about data collection and model training: with LLMs, product managers and software engineers can quickly prototype and validate ideas using prompts, before investing in specific data collection or fine-tuning, if required.
The Emergence of the AI Engineer
The AI Engineer is a new role that has emerged to bridge the gap between cutting-edge AI research and practical implementation. AI Engineers are professionals who can effectively wield the emergent capabilities of Foundation Models, going beyond mere prompt engineering to write software and even AI that writes software.
Key characteristics of AI Engineers:
Versatility: They work with a range of models, from large-scale GPT-4 and Claude to smaller open-source models on Huggingface and LLaMA.
Tool Proficiency: They're pros at using popular chaining, retrieval, and vector search tools like LangChain, LangSmith, and Pinecone, as well as emerging autonomous agent technologies.
Rapid Prototyping: AI Engineers can quickly validate product ideas using LLM prompts before diving into specific data collection for fine-tuning.
Code-Centric Approach: While leveraging the power of LLMs, AI Engineers are bringing back the importance of human-written code to orchestrate and enhance AI capabilities.
The rise of the AI Engineer represents a shift from the traditional ML Engineer role. It's a response to the increasing demand for professionals who can rapidly prototype, evaluate, and productionize AI solutions using the latest Foundation Models and tools. The AI Engineer role is becoming increasingly vital, connecting cutting-edge AI research with practical, real-world solutions.
Sources:
Me