ml-symbolic-regression

Classical MLgplearnrigorous codebase

Description

Symbolic Regression: GP Search Strategy

Objective

Design and implement a better genetic programming search strategy for symbolic regression. Your code goes in custom_sr.py. Three reference implementations (Standard GP, Parsimony GP, Lexicase GP) demonstrate different approaches.

Background

Symbolic regression discovers mathematical expressions that fit data. Genetic programming evolves a population of expression trees through selection, crossover, and mutation. Key challenges include balancing exploration vs exploitation, controlling expression complexity (bloat), and escaping local optima. Different approaches address these through fitness shaping, novel selection mechanisms, or improved genetic operators.

Evaluation

Tested on three standard symbolic regression benchmarks: Nguyen-7 (univariate transcendental), Nguyen-10 (bivariate trigonometric), Koza-3 (univariate polynomial). Metric: R² on held-out test set (higher is better).

Code

custom_sr.py

EditableRead-only

1#!/usr/bin/env python3
2"""Symbolic Regression via Genetic Programming.
3
4A self-contained GP framework for symbolic regression benchmarks.
5The editable section contains the search strategy: fitness function,
6selection, crossover, mutation, and per-generation evolution logic.
7"""
8
9import argparse
10import math
11import random
12import sys
13import os
14import numpy as np
15

Additional context files (read-only):

gplearn/gplearn/genetic.py
gplearn/gplearn/_program.py
gplearn/gplearn/fitness.py

Results

Show per-seed results

Model	Type	test r2 nguyen7 ↑	test r2 nguyen10 ↑	test r2 koza3 ↑
lexicase_gp	baseline	0.947	0.548	0.912
parsimony_gp	baseline	0.986	0.809	0.996
standard_gp	baseline	0.330	0.588	0.993
anthropic/claude-opus-4.6	vanilla	0.991	0.903	0.985
deepseek-reasoner	vanilla	0.930	0.617	0.963
google/gemini-3.1-pro-preview	vanilla	0.996	0.772	0.971
openai/gpt-5.4	vanilla	0.995	1.000	0.981
qwen/qwen3.6-plus	vanilla	-	-	-
qwen/qwen3.6-plus	vanilla	-	-	-
qwen/qwen3.6-plus	vanilla	-	-	-
anthropic/claude-opus-4.6	agent	0.991	0.903	0.985
deepseek-reasoner	agent	0.930	0.617	0.963
google/gemini-3.1-pro-preview	agent	0.990	0.860	0.995
openai/gpt-5.4	agent	0.993	1.000	0.992
qwen/qwen3.6-plus	agent	-	-	-
qwen/qwen3.6-plus	agent	-	-	-
qwen/qwen3.6-plus	agent	-	-	-

Agent Conversations

anthropic/claude-opus-4.6

9 steps

deepseek-reasoner

7 steps

google/gemini-3.1-pro-preview