Human vs. Prokaryotic Sequence Classifier

A PyTorch encoder/decoder model trained to classify protein sequences as human or prokaryotic in origin, using a character-level LSTM over the raw sequence string.

What this does

Sequences are read from two FASTA files (human, prokaryotic), tokenised at the character level (lowercase letters, digits, punctuation), padded/truncated to a fixed length, and fed through an LSTM encoder with two layers of LSTM. A second LSTM stage produces a per-sequence probability via a sigmoid output head. Training uses BCEWithLogitsLoss; evaluation reports a confusion matrix over correct / incorrect / ambiguous calls, with sequence IDs written out per bucket for downstream inspection.

Context

The inputs are balanced numbers of human DNA sequence reads taken from the sequence databases hg18. The prokaryotic reads are taken from various different Bacteria. I curated the human sequences carefully to avoid any contaminating bacterial sequence reads.

This was an early PyTorch project of mine, adapted from an image-captioning encoder/decoder pattern (the EncoderRNN / DecoderRNN split, and the caption_embeddings naming, are both legacies of that origin). I'm including it pretty much as-is rather than cleaning it up first, because the gap between what I wrote then and what I'd do now is more useful to talk through than a polished result would be.

Known issues, and what I'd change now

Decoder is architecturally unnecessary for this task. The Encoder-Decoder approach was adapted from an image-captioning architecture. In the image captioning architecture, the Encoder was a CNN creating features from an image. This generated a sequence of tokens autoregressively, each conditioned on the last. However, I created features embeddings with character encoding in an RNN Encoder Architecture instead.
However, here this task has one input sequence and one output label — there's nothing to generate. The decoder LSTM here just reprocesses the encoder output and repeats the same label-prediction 151 times (tile()), which forces a generation-shaped loss onto a classification problem. Fix: drop the decoder, take the encoder's final hidden state (or better, mean-pool over all timesteps to avoid losing early-sequence signal), and feed it straight into a linear + sigmoid head.

caption_embeddings is sized for the wrong thing. nn.Embedding(max_len, embed_size) allocates 150 rows, sized for a position index (0–149), inherited from the captioning use case. It's only ever looked up with the label value (0 or 1), so 148 of those rows never get trained. Symptom of the same decoder-shape mismatch above.

FIXED THIS: Off-by-one index collision between the two source datasets. The second FASTA's records are indexed as k = k + j, where j is left over from the first loop at its final value (len(records) - 1), not len(records). The first index assigned in the second loop therefore collides with the last index from the first loop, and the bare except: around the dict-append silently overwrites that entry rather than raising. One human-sequence example is silently dropped as a result. Fix: use len(records) explicitly as the offset, and narrow the exception handling so a real collision would raise rather than disappear.

MPS/CPU device handling is internally consistent on this machine, but not portable. Within a single run, all tensors and models stay on whichever device is selected at the top (mps if available, else cpu) — there's no mid-run transfer, so results are self-consistent for this repo as checked in. Two caveats:

• torch.load(..., map_location=torch.device('mps')) is hardcoded rather than using the device variable already defined. On a machine without MPS, device would correctly resolve to 'cpu', but the checkpoint loads would still try to force onto 'mps' and fail. Fix: use map_location=device.
• The DecoderRNN LSTM is configured with num_layers=2, dropout=0.2. There's a documented PyTorch issue where an LSTM's internal inter-layer dropout produces materially different (and worse) results when a model trained on MPS is later moved to CPU for inference — not a crash, a silent numerical mismatch. This repo doesn't do that transfer, so it isn't triggered here, but it would be a live risk the moment a checkpoint from this script is loaded on a different machine. Fix: either set dropout=0.0 and apply dropout manually between layers via a standalone nn.Dropout, or always evaluate on the same device type used for training.

Validation: The Validation Result is provided as accuracy and as a confidence matrix from scikit-learn. Output results were additionally stringently tested with Biological data.