7.4 Memory & State Management
Short-term scratchpads, episodic & semantic memory stores
Memory System Overview
AI agents require sophisticated memory systems to maintain context, learn from experiences, and manage complex tasks. Agent memory can be categorized into three primary types, each serving distinct cognitive functions.
Core Memory Components
Effective agent memory systems combine short-term scratchpads for immediate processing, episodic memory for experience tracking, and semantic memory for knowledge storage and retrieval.
1. Short-term Scratchpads
Short-term scratchpads serve as the agent's working memory, providing a temporary workspace for active reasoning and immediate task processing.
Scratchpad Characteristics
Capacity: Limited (typically 7±2 items, following Miller's Law)
Duration: Seconds to minutes
Purpose: Active reasoning, step-by-step problem solving
Implementation: Ring buffer with decay function
class ScratchpadMemory:
def __init__(self, capacity=7, decay_rate=0.1):
self.capacity = capacity
self.decay_rate = decay_rate
self.items = []
self.timestamps = []
def add_item(self, item, priority=1.0):
"""Add item to scratchpad with priority weighting"""
import time
if len(self.items) >= self.capacity:
self._evict_least_important()
self.items.append({
'content': item,
'priority': priority,
'timestamp': time.time()
})
def get_current_context(self):
"""Retrieve current scratchpad context"""
self._apply_decay()
return [item['content'] for item in self.items]
def _apply_decay(self):
"""Apply temporal decay to scratchpad items"""
import time
current_time = time.time()
for item in self.items[:]:
age = current_time - item['timestamp']
decay_factor = exp(-self.decay_rate * age)
item['priority'] *= decay_factor
if item['priority'] < 0.1:
self.items.remove(item)
Scratchpad Decay Function:
P(t) = P(0) × e-λt
Where P(t) = priority at time t, λ = decay constant
2. Episodic Memory Stores
Episodic memory stores maintain temporal sequences of agent experiences, enabling learning from past interactions and contextual decision-making.
Episodic Memory Characteristics
Capacity: Large (context-dependent, scalable)
Duration: Hours to indefinite (with proper indexing)
Structure: Temporal sequences of experiences with context
Purpose: Experience-based learning, pattern recognition
class EpisodicMemory:
def __init__(self):
self.episodes = []
self.episode_index = {}
self.context_embeddings = []
def store_episode(self, context, action, outcome, timestamp):
"""Store a complete episode with context"""
episode = {
'id': len(self.episodes),
'context': context,
'action': action,
'outcome': outcome,
'timestamp': timestamp,
'tags': self._extract_tags(context, action)
}
self.episodes.append(episode)
self._update_index(episode)
def retrieve_similar_episodes(self, current_context, k=5):
"""Retrieve episodes similar to current context"""
context_embedding = self._embed_context(current_context)
similarities = []
for i, episode in enumerate(self.episodes):
episode_embedding = self.context_embeddings[i]
similarity = self._cosine_similarity(context_embedding, episode_embedding)
similarities.append((similarity, episode))
return sorted(similarities, reverse=True)[:k]
def temporal_query(self, start_time, end_time):
"""Query episodes within a time range"""
return [
episode for episode in self.episodes
if start_time <= episode['timestamp'] <= end_time
]
Episodic Retrieval Probability:
P(retrieve) = exp(-α × Δt) × sim(contextcurrent, contextstored)
Where Δt = time since encoding, α = temporal decay rate
3. Semantic Memory Stores
Semantic memory stores contain structured knowledge representations that persist across episodes, enabling factual reasoning and knowledge application.
Semantic Memory Characteristics
Capacity: Very large (virtually unlimited with proper indexing)
Duration: Long-term persistent
Structure: Knowledge graphs, hierarchical ontologies
Purpose: Factual knowledge, reasoning rules, domain expertise
class SemanticMemory:
def __init__(self):
self.knowledge_graph = {}
self.concept_embeddings = {}
self.relations = {}
self.inference_rules = []
def add_fact(self, subject, relation, object, confidence=1.0):
"""Add a factual triple to semantic memory"""
if subject not in self.knowledge_graph:
self.knowledge_graph[subject] = {}
if relation not in self.knowledge_graph[subject]:
self.knowledge_graph[subject][relation] = []
self.knowledge_graph[subject][relation].append({
'object': object,
'confidence': confidence
})
def query_knowledge(self, subject, relation=None):
"""Query knowledge about a subject"""
if subject not in self.knowledge_graph:
return []
if relation:
return self.knowledge_graph[subject].get(relation, [])
else:
return self.knowledge_graph[subject]
def infer_new_knowledge(self):
"""Apply inference rules to derive new knowledge"""
new_facts = []
for rule in self.inference_rules:
inferred = rule(self.knowledge_graph)
new_facts.extend(inferred)
for fact in new_facts:
self.add_fact(fact['subject'], fact['relation'],
fact['object'], fact['confidence'])
return new_facts
4. Integrated Memory Architecture
Effective agent memory systems integrate all three memory types to provide comprehensive cognitive capabilities.
class IntegratedMemorySystem:
def __init__(self):
self.scratchpad = ScratchpadMemory()
self.episodic = EpisodicMemory()
self.semantic = SemanticMemory()
self.memory_coordinator = MemoryCoordinator()
def process_experience(self, context, action, outcome):
"""Process new experience across all memory systems"""
import time
# Update scratchpad with immediate context
self.scratchpad.add_item(context, priority=1.0)
self.scratchpad.add_item(f"Action: {action}", priority=0.9)
# Store episode for future learning
self.episodic.store_episode(context, action, outcome, time.time())
# Extract and store semantic knowledge
knowledge_facts = self._extract_knowledge(context, action, outcome)
for fact in knowledge_facts:
self.semantic.add_fact(fact['subject'], fact['relation'],
fact['object'], fact['confidence'])
def retrieve_relevant_context(self, current_situation):
"""Retrieve relevant information from all memory stores"""
context = {
'scratchpad': self.scratchpad.get_current_context(),
'similar_episodes': self.episodic.retrieve_similar_episodes(current_situation),
'relevant_knowledge': self._query_relevant_knowledge(current_situation)
}
return self.memory_coordinator.integrate_context(context)
