Biologically-Inspired Cognitive Architectures in AI

Standard neural networks are amnesiac. A transformer processes each context window independently. An LSTM maintains a hidden state, but it's a single compressed vector with no structure. When you need a system that remembers specific events, accumulates general knowledge, retains learned skills, and responds to emotional context — you need something more structured.

Cognitive architectures borrow from neuroscience to give AI systems multiple, specialized memory systems. Here's what each one does and how they differ from standard approaches.

Why Multiple Memory Systems?

The human brain doesn't store everything in one place. Neuroscience has identified distinct memory systems that activate differently, decay at different rates, and serve different functions. Building these distinctions into an AI system gives you more control over what the model remembers, how it forgets, and how past experience shapes current behavior.

Working Memory (Prefrontal Cortex)

Short-term, high-bandwidth, fast to read and write:

class WorkingMemory:
    def __init__(self, capacity: int, dim: int):
        self.buffer = np.zeros((capacity, dim))
        self.W_q = Parameter(np.random.randn(dim, dim) * 0.01)
        self.W_k = Parameter(np.random.randn(dim, dim) * 0.01)
        self.ptr = 0
 
    def write(self, x: np.ndarray):
        self.buffer[self.ptr % self.capacity] = x.mean(axis=0)
        self.ptr += 1
 
    def read(self, query: np.ndarray) -> np.ndarray:
        q = query @ self.W_q
        k = self.buffer @ self.W_k
        scores = q @ k.T / np.sqrt(q.shape[-1])
        weights = softmax(scores)
        return weights @ self.buffer

Unlike an LSTM hidden state, working memory has explicit capacity and is addressable by content. Dopamine modulates how sharply attention focuses; norepinephrine controls the signal-to-noise ratio of what gets retrieved.

Episodic Memory (Hippocampus)

Stores specific events with temporal context:

class EpisodicStore:
    def __init__(self, num_slots: int, key_dim: int, value_dim: int):
        self.keys = np.zeros((num_slots, key_dim))
        self.values = np.zeros((num_slots, value_dim))
        self.ptr = 0
 
    def write(self, key: np.ndarray, value: np.ndarray, dopamine: float = 0.0):
        # Dopamine gates whether this experience is worth storing
        gate = sigmoid(dopamine * 3.0)
        if np.random.rand() < gate:
            slot = self.ptr % self.num_slots
            self.keys[slot] = key
            self.values[slot] = value
            self.ptr += 1

The key difference from a replay buffer: episodic memory supports reconsolidation. When you retrieve a memory, you slightly modify it based on the current context — which is how human memories shift over time. In Neural Nexus, the top-attended keys are nudged toward the current query at read time.

Semantic Memory (Temporal Cortex)

General world knowledge that accumulates over time and deduplicates:

class SemanticMemory:
    def __init__(self, num_slots: int, similarity_threshold: float = 0.8):
        self.keys = np.zeros((num_slots, key_dim))
        self.values = np.zeros((num_slots, value_dim))
        self.counts = np.zeros(num_slots)
 
    def update(self, key: np.ndarray, value: np.ndarray):
        sims = cosine_sim(key, self.keys)
        if sims.max() > self.similarity_threshold:
            # Known concept — EMA update
            idx = sims.argmax()
            alpha = 0.1
            self.values[idx] = (1 - alpha) * self.values[idx] + alpha * value
            self.counts[idx] += 1
        else:
            # New concept — allocate slot
            self.keys[self.next_slot] = key
            self.values[self.next_slot] = value

Semantic memory doesn't store "I saw X at time T" — it stores "X exists and has these properties." Similar inputs update the existing entry rather than creating a duplicate.

Procedural Memory (Basal Ganglia)

Learned skills and habits, implemented as a mixture of experts:

class ProceduralMemory:
    def __init__(self, input_dim: int, output_dim: int, num_programs: int = 4):
        self.W_router = Parameter(np.random.randn(input_dim, num_programs) * 0.01)
        self.W_programs = [Parameter(np.random.randn(input_dim, output_dim) * 0.01)
                           for _ in range(num_programs)]
 
    def forward(self, x: np.ndarray, serotonin: float = 0.0) -> np.ndarray:
        # Serotonin tunes routing temperature
        temp = 1.0 + serotonin * 2.0
        logits = x @ self.W_router / temp
        weights = softmax(logits)
        outputs = np.stack([x @ W for W in self.W_programs], axis=-1)
        return (outputs * weights[..., np.newaxis]).sum(axis=-1)

Different programs specialize for different input patterns. You can freeze individual programs once they've converged, preserving learned skills while the rest of the network continues to train.

How This Differs from Transformers and LSTMs

A transformer's attention mechanism is stateless across sequences — it has no persistent memory between calls unless you append the context window. An LSTM has a hidden state but it's a fixed-size vector with no addressability. Both lose information when context fills up.

Cognitive memory systems:

• Persist across inference calls (unless explicitly reset)
• Are addressable by content, not just position
• Have explicit capacity with principled forgetting
• Support neuromodulatory signals that change retrieval behavior dynamically

For building conversational AI, agents that track long-running tasks, or any system where prior experience should meaningfully shape current behavior, structured memory is worth the implementation overhead.