BiLSTM Model

Switch language: BiLSTM 模型

M4 is the bidirectional LSTM encoder of the project.

Why BiLSTM is included

Bidirectional recurrent models are a natural counterpart to CNNs:

  • CNNs focus on local pattern extraction;

  • BiLSTMs explicitly process the sequence as an ordered chain;

  • forward and backward hidden states capture left and right context jointly.

Architecture summary

  • Input: one-hot tensor (B, 5, L)

  • Sequence layout: transposed into (B, L, 5)

  • Core: 2-layer bidirectional LSTM

  • Hidden width: 256 per direction

  • Output: concatenated forward/backward final states

At each step, an LSTM updates gates and memory through:

\[i_t = \sigma(W_i x_t + U_i h_{t-1} + b_i)\]
\[f_t = \sigma(W_f x_t + U_f h_{t-1} + b_f)\]
\[o_t = \sigma(W_o x_t + U_o h_{t-1} + b_o)\]
\[\tilde{c}_t = \tanh(W_c x_t + U_c h_{t-1} + b_c)\]
\[c_t = f_t \odot c_{t-1} + i_t \odot \tilde{c}_t, \qquad h_t = o_t \odot \tanh(c_t)\]

The bidirectional encoder runs this recurrence in both directions so that each output representation can reflect both upstream and downstream context.

Why this helps for EPI

Enhancer and promoter regions are long sequences where regulatory information is not confined to a single motif window. A BiLSTM can model:

  • ordered motif progression;

  • local-to-mid-range contextual accumulation;

  • asymmetric sequence signals that plain pooling may wash out.

This matters when the functional meaning of a motif depends on order, nearby context, or the sequence of several local events rather than on isolated hits.

Strengths

  • strong sequence-order awareness;

  • intuitive recurrent inductive bias;

  • useful comparator against attention-based and linear-time models.

Computational complexity

  • Time: sequential recurrence gives roughly \(O(L \cdot H^2)\) behavior per layer, with limited parallelism across positions.

  • Memory: moderate for hidden states, but training cost rises with sequence length because backpropagation must preserve recurrent activations across the chain.

  • Best-fit regime: useful for short-to-medium windows where ordered context is important and full global attention would be unnecessary or overly expensive.

Limitations

  • recurrent processing is inherently more sequential than convolution or fully parallel attention alternatives;

  • very long genomic sequences can become computationally expensive.

Project role

M4 is important because it represents the classic recurrent baseline in the benchmark. It helps answer whether explicit sequential recurrence remains useful after introducing Transformer-style, state-space, and foundation-model routes.

../_images/div.png