Ephaptic AI | Gerard Calvo Bartra

Ephaptic coupling is a form of communication within the nervous system that is distinct from direct communication systems like synapses. When using ephaptic coupling, the nerve fibers couple as a result of local electric fields. This project reimagines artificial neural networks with multiple local fields each neuron can affect and subscribe to.

PROJECT

Ephaptic AI

IMPACT AREAS

Challenge: Reimagining neural networks through biological inspiration

Traditional artificial neural networks are primarily based on synaptic connections, where information flows directly from one neuron to another. However, biological neural systems use multiple communication mechanisms. One such mechanism is ephaptic coupling, where neurons communicate through local electric fields rather than direct connections.

The challenge was to create a novel neural network architecture that incorporates this alternative communication mechanism, potentially unlocking new computational capabilities and interpretability not found in standard neural networks. By modeling the way neural fibers influence each other through electric fields, could we build networks with different inductive biases and more transparent information processing?

Approach: Designing networks with field-mediated communication

I developed a new neural network architecture inspired by ephaptic coupling, where neurons communicate not only through direct connections but also by influencing and subscribing to shared electric fields. This creates a dual-pathway system of information flow: the traditional synaptic pathway and a field-mediated pathway.

In this architecture, neurons contribute to multiple fields based on their activation, and they also selectively subscribe to different fields through a gating mechanism. This allows for complex patterns of influence that can't be captured by direct connections alone.

Created a custom PyTorch layer implementation with field aggregation and gating
Incorporated architectural features like skip connections and layer normalization for stability
Implemented visualization tools to interpret the field activations
Applied the model to image classification tasks to evaluate performance

Implementation: Ephaptic layers in PyTorch

The core of the implementation is the EphapticLayerV2 class, which extends the standard neural network layer with field-mediated pathways. Each layer processes information through multiple steps:

Synaptic transformation: Standard linear transformation of inputs
Field aggregation: Neurons contribute to shared fields
Gating mechanism: Neurons selectively subscribe to different fields
Field influence: The gated fields influence neuron activations
Combination: Synaptic and field pathways are combined and normalized

ephaptic_layer.py

class EphapticLayerV2(nn.Module):
    """
    An improved EphapticLayer with gating, skip connections, and layer norm.
    Each neuron "subscribes" to M fields through a gating function.
    """
    def __init__(self, num_neurons, num_fields, alpha_init=0.1, add_synaptic=True):
        super().__init__()
        self.num_neurons = num_neurons
        self.num_fields = num_fields
        self.add_synaptic = add_synaptic

        # Synaptic transformation: x -> z_syn
        if self.add_synaptic:
          self.synaptic = nn.Linear(num_neurons, num_neurons, bias=True)

        # Field aggregator: x -> phi
        # Aggregates (B, N) -> (B, M)
        self.field_in = nn.Linear(num_neurons, num_fields, bias=True)

        # Field out: phi -> z_field
        # Maps (B, M) -> (B, N)
        self.field_out = nn.Linear(num_fields, num_neurons, bias=True)

        # Gating net: x -> gating vector in (B, M)
        # This gating will be multiplied with phi to let neurons "subscribe" to the fields.
        self.gate_net = nn.Linear(num_neurons, num_fields, bias=True)

        # Alpha scales how much the field component influences the output
        self.alpha = nn.Parameter(torch.tensor(alpha_init))

        # LayerNorm to stabilize the final combination
        self.ln = nn.LayerNorm(num_neurons)

The forward pass of the EphapticLayer combines traditional synaptic connections with the field-mediated pathway:

ephaptic_layer.py

def forward(self, x):
    """
    Forward Pass:
      1) syn = synaptic(x)
      2) phi = ReLU(field_in(x))
      3) gating = sigmoid(gate_net(x))
      4) phi_gated = phi * gating
      5) z_field = field_out(phi_gated)
      6) out = LayerNorm( ReLU( syn + alpha * z_field ) + x )  # skip connection
    """
    # 1) Synaptic path
    if self.add_synaptic:
      z_syn = self.synaptic(x)  # shape (B, N)

    # 2) Field aggregator
    phi = F.relu(self.field_in(x))  # shape (B, M)

    # 3) Gating function
    gating = torch.sigmoid(self.gate_net(x))  # shape (B, M)

    # 4) Gated fields
    phi_gated = phi * gating  # (B, M)

    # 5) Field back-influence
    z_field = self.field_out(phi_gated)  # (B, N)

    # 6) Combine with alpha, plus skip connection, then ReLU + LayerNorm
    combined = z_syn + self.alpha * z_field if self.add_synaptic else self.alpha * z_field
    # Let's apply ReLU, then add skip (the original x), then layer norm
    out = self.ln(F.relu(combined) + x)

    return out

The model was implemented in PyTorch and trained on Google Colab, making it accessible for experimentation and further development.

Results: Performance and interpretability on image classification

The Ephaptic AI model was evaluated on the MNIST handwritten digit classification task, showing competitive performance compared to traditional MLPs with similar parameter counts.

More interestingly, the field-mediated architecture offers enhanced interpretability. By visualizing the field activations for each input, we can gain insights into how the network processes information:

Visualization of field activations for digit recognition — Visualization of field activations for a digit '5'. The original input (left) and activations of two different fields (middle, right) show how different fields respond to specific features of the digit.

As shown in the visualization, different fields appear to focus on different aspects of the input. Field 0 shows strong activation (phi=2.04) with moderate gating (0.10), resulting in a combined influence of 0.21. Field 1 shows no activation, indicating this field doesn't respond to this particular input pattern.

Impact: A new perspective on neural network design

The Ephaptic AI architecture represents a novel approach to neural network design that incorporates biological inspiration beyond the standard neuron-synapse metaphor. This work has several important implications:

Demonstrates that alternative communication mechanisms inspired by neuroscience can be effectively implemented in artificial neural networks
Provides a more interpretable network architecture where field activations can be visualized and analyzed
Opens new avenues for neural network design that may capture different types of patterns than traditional architectures
Bridges concepts from computational neuroscience and deep learning, potentially informing both fields

While achieving comparable performance to traditional MLPs on benchmark tasks, the true value of this approach may lie in its different inductive biases and increased interpretability, which could be particularly valuable for applications where understanding model decisions is critical.

Future Directions: Expanding the ephaptic framework

This project opens several promising avenues for future research:

Applying Ephaptic AI to more complex tasks and datasets beyond MNIST
Exploring different field structures and topologies inspired by neural anatomy
Investigating the potential for more efficient training or better generalization through field-mediated learning
Developing more sophisticated visualization and interpretation tools for field activations
Combining ephaptic principles with other neural network architectures like CNNs or Transformers

The initial success of this bio-inspired approach suggests that there may be many more principles from neuroscience that could be fruitfully incorporated into artificial neural networks, potentially leading to more capable and interpretable AI systems.