Machine Learning Approaches Based on k-mer Images and Vision Transformers for Protein and Molecule Representation and Analysis
Bioinformatics; k-mer Encoding; Vision Transformer; Representation Learning; Protein and Molecular Embeddings; Classification and Clustering.
The representation and analysis of biological sequences and molecular structures are fundamental tasks in bioinformatics and cheminformatics, yet traditional methods based on sequence alignments and molecular descriptors face limitations in scalability and in capturing subtle structural and functional relationships. This work investigates machine learning approaches based on k-mer image representations and Vision Transformers for the analysis of proteins and molecules. The proposed approach converts biological aminoacid sequences and molecular SMILES strings into fixed-size images derived from k-mer co-occurrence patterns, which are processed by a pretrained Vision Transformer to produce discriminative embedding vectors. This unified, alignment-free pipeline is evaluated under both supervised and unsupervised paradigms across two application domains. For proteins, supervised classification of clusters from the UniRef100 and UniRef90 datasets is performed using Logistic Regression, Random Forest, k-Nearest Neighbors, and XGBoost, while unsupervised analysis is conducted using DBSCAN with two proposed metrics (contamination and spreading) to assess cluster quality. For molecules, blood--brain barrier permeability prediction is addressed on the BBBP and B3DB datasets using both classical machine learning classifiers and deep learning architectures, including MLP, ResMLP, DCN, DCNv2, FT-Transformer, and TabNet. Preliminary results indicate that the generated embeddings are effective for both classification and clustering tasks across domains. The planned work includes extending the unsupervised analysis to the molecular domain.