AI Workflows and AI Agents
What are they and how they work.
According to Anthropic (American artificial intelligence research company) definition:
• Workflows are systems where LLMs and tools are orchestrated
through predefined code paths.
• Agents, on the other hand, are systems where LLMs dynamically
direct their own processes and tool usage, maintaining control
over how they accomplish tasks.
Software applications which use LLM's (Large Language Models) to accomplish tasks automatically.
Tasks can range from research to backend service handling.
Tasks which demand complex decision-making, adaptability, and autonomy can be the primary target.
They have multiple steps of interaction.
They can engage directly with customers, develop relationships through user interaction and data, and help decision-making process and task completion.
Building block: The augmented LLM
The basic building block of agentic systems is an LLM enhanced with
augmentations such as retrieval, tools, and memory. Current
models can actively use these capabilities— generating their own
search queries, selecting appropriate tools, and determining what
information to retain.
Retrieval
AI system pulls information from a database or vector database and makes it available in the context of a LLM.
Typically, done trough RAG (Retrieval augmented generation) by using a vector database where similarity search is performed. Typically compared to giving the LLM long term memory. Offloading all the relevant context that you need to a vector database and whenever LLM requires it you can try to retrieve it. With RAG, you never know what you can retrieve as scaling of the database grows.
Tools
Services or API's that you can call within the application in order to get more information. Making an API call to get the weather data as an example, or shipping data of a delivery.
Memory
The sequence of interaction that you had with LLM system.
WORKFLOWS
Workflow: Prompt chaining
Prompt chaining decomposes a task into a sequence of steps, where
each LLM call processes the output of the previous one. You can add
programmatic checks (see “gate” in the diagram below) on any
intermediate steps to ensure that the process is still on track.
Workflow: Routing
Routing defines an input and directs it to a follow-up task.
Allows for separation of concerns and building more specialized prompts.
This workflow is critical for optimization of performance.
As the scope of the problems goes bigger, and you have a multiple cases or scenarios, the routing workflow is the right path.
Given the data that's coming in, you let the LLM decide which way to go.
Clearly instruct the LLM for the first step to be categorization of the incoming request. Routers in theory are if statements or cases and if output is equal to A can call a particular function, etc.
Workflow: Parallelization
LLM's have the ability to work simultaneously on a task and give programmatically aggregated outputs. This type of workflow materializes in two variations.
Sectioning: Break a task into an independent subtask which run in parallel.
Voting: Running the same task multiple times to get a range of diverse outputs.
Like in Prompt Chaining, you make multiple LLM calls but rather of doing them sequentially where one output might depend on the other one, you do them in parallel.
This is ideal for splitting up of tasks to be independent of each other in an async manner.
Example: When you are implementing guardrails for evaluating outputs, you might have one prompt for accuracy, one for harmfulness, and one for capturing prompt injections.
Workflow: Orchestrator-workers
Workflow in which a central LLM dynamically breaks down a list of tasks, delegates them to a worker LLM's, and synthesizes their outputs.
Requires a little less programming, and it is more agentic in nature.
Still sequential and predictable.
Example: An email from customer is coming in, and you let a LLM look at all the context, customer question, CRM data or order data and customer care guidelines.
After assessment, ask a LLM what is required to solve this particular problem.
Workflow: Evaluator-optimizer
In this workflow, one LLM call can generate a response while another provides evaluation and feedback in a loop.
Example: Prompt the first LLM to write a social media post. Feed the output to a second LLM for critical optimization and brand or products parameters, asking to give a report what can be improved. You feed the output to the original LLM and process that.
AGENTS
Autonomous agent
Autonomous Agents can handle tasks with sophisticated flow and often have a straightforward implementation. They are LLM's which use tools with environment feedback in a loop.
Design of clear documentation and tool sets for this is critical.
At the moment the recommended way of implementing LLM's is to use workflows, they provide more control over the flow of the application. You can start small and work your way up.
Prioritizing deterministic workflows over complex and unclear agent patterns is recommended practice.
Scaling of LLM (RAG) based applications is a critical pain point and cannot be underestimated.
