Embedding-guided Selection for Genetic Programming Using Pretrained Code Representations

In conventional fitness-based Genetic Programming (GP), parent selection is driven mainly by fitness and has limited access to relationships between programs beyond their observed performance. This work investigates whether pretrained code embeddings can provide complementary guidance for parent selection in symbolic regression. We introduce two mechanisms that combine fitness with distances computed in the latent spaces of CodeBERT and Code2Vec. Probabilistic Embedding-Guided Selection (PEGS) alternates between fitness-based and embedding-based selection under fixed or dynamically scheduled pressure. Two-Stage Embedding-Guided Selection (TSEGS) applies fitness selection followed by an embedding-based criterion aimed either at intensifying the search around elite solutions or at preserving diversity. The methods are evaluated on reduced SRBench datasets against standard GP and, in a targeted experiment, an NSGA-II baseline with constant optimization. Results show that embedding-derived information can improve held-out test performance, but its effectiveness depends on the embedding model, selection mechanism, and role assigned to distances. The diversity-oriented TSEGS criterion is most consistently beneficial with Code2Vec, whereas other configurations show more limited or problem-dependent effects. Model-size effects are mixed, and in the targeted NSGA-II experiment the gains are accompanied by larger median trees. A representative-run analysis further indicates that embedding-guided selection does not uniformly increase exploration. Overall, pretrained code embeddings provide a potentially useful but non-universal signal for GP selection, with benefits that must be weighed against configuration sensitivity, model-size effects, and substantial, implementation-dependent computational overhead.