Agent Conversation: deepseek-reasoner
Task: ml-symbolic-regression
Symbolic Regression: GP Search Strategy
Objective
Design and implement a better genetic programming search strategy for symbolic regression. Your code goes in custom_sr.py. Three reference implementations (Standard GP, Parsimony GP, Lexicase GP) demonstrate different approaches.
Background
Symbolic regression discovers mathematical expressions that fit data. Genetic programming evolves a population of expression trees through selection, crossover, and mutation. Key challenges include balancing exploration vs exploitation, controlling expression complexity (bloat), and escaping local optima. Different approaches address these through fitness shaping, novel selection mechanisms, or improved genetic operators.
Evaluation
Tested on three standard symbolic regression benchmarks: Nguyen-7 (univariate transcendental), Nguyen-10 (bivariate trigonometric), Koza-3 (univariate polynomial). Metric: R² on held-out test set (higher is better).
gplearn/custom_sr.py [EDITABLE — lines 228–306 only]
1: #!/usr/bin/env python3
2: """Symbolic Regression via Genetic Programming.
3:
4: A self-contained GP framework for symbolic regression benchmarks.
5: The editable section contains the search strategy: fitness function,
6: selection, crossover, mutation, and per-generation evolution logic.
7: """
8:
9: import argparse
10: import math
11: import random
12: import sys
13: import os
14: import numpy as np
15:
16:
17: # ============================================================
18: # Operator Definitions (FIXED)
19: # ============================================================
20:
21: def protected_div(a, b):
22: """Protected division: returns 1.0 when divisor is near zero."""
23: return np.where(np.abs(b) > 1e-10, a / b, 1.0)
24:
25:
26: def protected_log(a):
27: """Protected log: returns 0.0 for non-positive inputs."""
28: return np.where(np.abs(a) > 1e-10, np.log(np.abs(a)), 0.0)
29:
30:
31: def protected_exp(a):
32: """Protected exp: clips input to prevent overflow."""
33: return np.exp(np.clip(a, -10, 10))
34:
35:
36: OPERATORS = {
37: 'add': (np.add, 2),
38: 'sub': (np.subtract, 2),
39: 'mul': (np.multiply, 2),
40: 'div': (protected_div, 2),
41: 'sin': (np.sin, 1),
42: 'cos': (np.cos, 1),
43: 'log': (protected_log, 1),
44: 'exp': (protected_exp, 1),
45: }
46:
47: OPERATOR_NAMES = list(OPERATORS.keys())
48:
49:
50: # ============================================================
51: # Tree Representation (FIXED)
52: # ============================================================
53:
54: class Node:
55: """A node in the GP expression tree."""
56: __slots__ = ('value', 'children')
57:
58: Tool Input
1{
2 "op": "replace",
3 "filename": "gplearn/custom_sr.py",
4 "start_line": 228,
5 "end_line": 306,
6 "content": "def fitness_function(tree, X, y):\n \"\"\"Evaluate fitness of a candidate program. Lower is better.\"\"\"\n y_pred = safe_evaluate(tree, X)\n return float(np.mean((y - y_pred) ** 2))\n\n\ndef selection(population, fitnesses, n_select, tournament_size=5):\n \"\"\"Tournament selection.\"\"\"\n selected = []\n pop_size = len(population)\n for _ in range(n_select):\n candidates = random.sample(range(pop_size), min(tournament_size, pop_size))\n best = min(candidates, key=lambda i: fitnesses[i])\n selected.append(population[best].copy())\n return selected\n\n\ndef semantic_similarity(subtree1, subtree2, X_sample):\n \"\"\"Compute semantic similarity between two subtrees using cosine similarity on sample points.\"\"\"\n vec1 = safe_evaluate(subtree1, X_sample)\n vec2 = safe_evaluate(subtree2, X_sample)\n # Normalize to avoid scaling differences\n norm1 = np.linalg.norm(vec1)\n norm2 = np.linalg.norm(vec2)\n if norm1 < 1e-12 or norm2 < 1e-12:\n return 0.0\n return np.dot(vec1, vec2) / (norm1 * norm2)\n\n\ndef crossover(parent1, parent2, n_features, max_depth=17, X_sample=None):\n \"\"\"Semantic-aware subtree crossover.\n \n Attempts to find crossover points that produce semantically similar subtrees\n to reduce destructive effects. Falls back to standard crossover if no good match.\n \"\"\"\n offspring = parent1.copy()\n donor = parent2.copy()\n \n off_size = offspring.size()\n don_size = donor.size()\n if off_size <= 1 or don_size <= 1:\n return offspring\n \n # Get all nodes with their depths\n off_nodes = offspring.get_all_nodes()\n don_nodes = donor.get_all_nodes()\n \n # If X_sample is not provided, create a small random sample for semantic comparison\n if X_sample is None:\n # Use default sample: 10 random points in [-1, 1]^n_features\n X_sample = np.random.uniform(-1, 1, size=(10, n_features))\n \n # Try to find semantically similar subtrees\n best_sim = -1.0\n best_off_idx = -1\n best_don_idx = -1\n \n # Limit search to reasonable number of pairs to avoid O(N^2) cost\n max_pairs = min(50, len(off_nodes) * len(don_nodes))\n for _ in range(max_pairs):\n i = random.randint(0, len(off_nodes) - 1)\n j = random.randint(0, len(don_nodes) - 1)\n off_sub = off_nodes[i][0]\n don_sub = don_nodes[j][0]\n sim = semantic_similarity(off_sub, don_sub, X_sample)\n if sim > best_sim:\n best_sim = sim\n best_off_idx = i\n best_don_idx = j\n \n # If we found a reasonable semantic match, use those points\n if best_sim > 0.5: # threshold\n off_point = best_off_idx\n don_point = best_don_idx\n else:\n # Fallback to random points\n off_point = random.randint(1, off_size - 1)\n don_point = random.randint(0, don_size - 1)\n \n # Extract donor subtree\n donor_subtree = don_nodes[don_point][0].copy()\n \n # Replace in offspring\n node, parent, child_idx = off_nodes[off_point]\n if parent is not None:\n parent.children[child_idx] = donor_subtree\n else:\n offspring = donor_subtree\n \n # Reject if too deep\n if offspring.depth() > max_depth:\n return parent1.copy()\n \n return offspring\n\n\ndef mutation(parent, n_features, max_depth=17):\n \"\"\"Subtree mutation with occasional constant optimization.\"\"\"\n offspring = parent.copy()\n tree_size = offspring.size()\n if tree_size <= 1:\n return generate_tree('grow', 3, n_features)\n \n mut_type = random.random()\n if mut_type < 0.7:\n # Standard subtree mutation\n mut_point = random.randint(1, tree_size - 1)\n new_subtree = generate_tree('grow', 3, n_features)\n \n nodes = offspring.get_all_nodes()\n node, par, child_idx = nodes[mut_point]\n if par is not None:\n par.children[child_idx] = new_subtree\n else:\n offspring = new_subtree\n else:\n # Point mutation: change a single node\n nodes = offspring.get_all_nodes()\n if len(nodes) == 0:\n return offspring\n node, parent, child_idx = random.choice(nodes)\n if node.is_terminal:\n # Terminal mutation: change variable index or constant value\n if isinstance(node.value, str) and node.value.startswith('x'):\n # Change variable index\n idx = random.randint(0, n_features - 1)\n node.value = f'x{idx}'\n else:\n # Change constant by small perturbation\n try:\n val = float(node.value)\n node.value = str(val + random.uniform(-0.5, 0.5))\n except:\n node.value = str(random.uniform(-5.0, 5.0))\n else:\n # Function mutation: change operator to another with same arity\n current_op = node.value\n arity = OPERATORS[current_op][1]\n # Find other operators with same arity\n candidates = [op for op in OPERATOR_NAMES if OPERATORS[op][1] == arity]\n if len(candidates) > 1:\n candidates.remove(current_op)\n node.value = random.choice(candidates)\n \n if offspring.depth() > max_depth:\n return parent.copy()\n \n return offspring\n\n\ndef evolve_one_generation(population, fitnesses, X_train, y_train,\n n_features, pop_size,\n crossover_rate=0.9, mutation_rate=0.05,\n max_depth=17):\n \"\"\"Create the next generation with adaptive parsimony pressure.\"\"\"\n new_population = []\n \n # Adaptive parsimony coefficient (inspired by gplearn)\n lengths = np.array([tree.size() for tree in population], dtype=float)\n raw_fit = np.array(fitnesses, dtype=float)\n len_var = float(np.var(lengths))\n if len_var > 1e-15:\n parsimony_coeff = float(np.cov(lengths, raw_fit)[1, 0]) / len_var\n parsimony_coeff = max(parsimony_coeff, 0.0)\n parsimony_coeff = min(parsimony_coeff, 0.001)\n else:\n parsimony_coeff = 0.0\n \n # Penalized fitnesses for selection\n penalized = [f + parsimony_coeff * l for f, l in zip(fitnesses, lengths)]\n \n # Create a sample of training points for semantic crossover\n # Use 20% of training data, but at most 20 points\n n_sample = min(20, int(0.2 * X_train.shape[0]))\n if n_sample > 0:\n indices = random.sample(range(X_train.shape[0]), n_sample)\n X_sample = X_train[indices]\n else:\n X_sample = X_train\n \n # Elitism: keep best individual by raw fitness\n elite_idx = int(np.argmin(fitnesses))\n new_population.append(population[elite_idx].copy())\n \n while len(new_population) < pop_size:\n r = random.random()\n if r < crossover_rate:\n parents = selection(population, penalized, 2)\n child = crossover(parents[0], parents[1], n_features, max_depth, X_sample)\n elif r < crossover_rate + mutation_rate:\n parents = selection(population, penalized, 1)\n child = mutation(parents[0], n_features, max_depth)\n else:\n parents = selection(population, penalized, 1)\n child = parents[0]\n new_population.append(child)\n \n return new_population[:pop_size]"
7}OK: Replaced lines 228..306 with 198 line(s) in gplearn/custom_sr.py. Editable range: 228–425.
[Current file: gplearn/custom_sr.py | editable: 228–425 | total: 502 lines]
228 def fitness_function(tree, X, y):
229 """Evaluate fitness of a candidate program. Lower is better."""
230 y_pred = safe_evaluate(tree, X)
... (192 more lines) ...
423 new_population.append(child)
424
425 return new_population[:pop_size][Test #1] (2 tests remaining; call submit(n=N) to choose which test result to submit as final) ## Seed 42 ### nguyen7 (scripts/nguyen7.sh) Training metrics (last generations): TRAIN_METRICS generation=45 best_fitness=0.003590 avg_fitness=1.735154 best_size=29 train_r2=0.995067 TRAIN_METRICS generation=46 best_fitness=0.003590 avg_fitness=1.313456 best_size=29 train_r2=0.995067 TRAIN_METRICS generation=47 best_fitness=0.003590 avg_fitness=1.048807 best_size=29 train_r2=0.995067 TRAIN_METRICS generation=48 best_fitness=0.003590 avg_fitness=0.438626 best_size=29 train_r2=0.995067 TRAIN_METRICS generation=49 best_fitness=0.003590 avg_fitness=0.937607 best_size=29 train_r2=0.995067 Test results (nguyen7): TEST_METRICS r2=0.798371 rmse=0.517776 train_r2=0.995067 size=29 expression="log(((((sin(x0) div exp(-2.4)) add cos(exp(x0))) mul (cos(((2.2639796352220136 div exp(2.22)) mul x0)) sub sin(cos(x0)))) sub (-0.95 add (x0 mul sin(x0)))))" R² = 0.798371 RMSE = 0.517776 Expression: lo
Tool Input
1{
2 "op": "replace",
3 "filename": "gplearn/custom_sr.py",
4 "start_line": 228,
5 "end_line": 425,
6 "content": "def fitness_function(tree, X, y):\n \"\"\"Evaluate fitness of a candidate program. Lower is better.\"\"\"\n y_pred = safe_evaluate(tree, X)\n return float(np.mean((y - y_pred) ** 2))\n\n\ndef selection(population, fitnesses, n_select, tournament_size=5):\n \"\"\"Tournament selection.\"\"\"\n selected = []\n pop_size = len(population)\n for _ in range(n_select):\n candidates = random.sample(range(pop_size), min(tournament_size, pop_size))\n best = min(candidates, key=lambda i: fitnesses[i])\n selected.append(population[best].copy())\n return selected\n\n\ndef semantic_similarity(subtree1, subtree2, X_sample):\n \"\"\"Compute semantic similarity between two subtrees using cosine similarity on sample points.\"\"\"\n vec1 = safe_evaluate(subtree1, X_sample)\n vec2 = safe_evaluate(subtree2, X_sample)\n # Normalize to avoid scaling differences\n norm1 = np.linalg.norm(vec1)\n norm2 = np.linalg.norm(vec2)\n if norm1 < 1e-12 or norm2 < 1e-12:\n return 0.0\n sim = np.dot(vec1, vec2) / (norm1 * norm2)\n # Clip to [-1, 1] due to numerical errors\n return np.clip(sim, -1.0, 1.0)\n\n\ndef crossover(parent1, parent2, n_features, max_depth=17, X_sample=None):\n \"\"\"Semantic-aware subtree crossover with probabilistic selection.\"\"\"\n offspring = parent1.copy()\n donor = parent2.copy()\n \n off_size = offspring.size()\n don_size = donor.size()\n if off_size <= 1 or don_size <= 1:\n return offspring\n \n # Get all nodes\n off_nodes = offspring.get_all_nodes()\n don_nodes = donor.get_all_nodes()\n \n # If X_sample is not provided, create a small random sample\n if X_sample is None:\n X_sample = np.random.uniform(-1, 1, size=(10, n_features))\n \n # Sample candidate pairs\n n_pairs = min(30, len(off_nodes) * len(don_nodes))\n pairs = []\n sims = []\n for _ in range(n_pairs):\n i = random.randint(0, len(off_nodes) - 1)\n j = random.randint(0, len(don_nodes) - 1)\n sim = semantic_similarity(off_nodes[i][0], don_nodes[j][0], X_sample)\n pairs.append((i, j))\n sims.append(sim)\n \n # Convert similarities to probabilities (softmax)\n sims = np.array(sims)\n # Shift to make positive (similarity ranges -1..1)\n sims_shifted = sims + 1.0 # now 0..2\n # Add small epsilon to avoid zero\n probs = sims_shifted / (np.sum(sims_shifted) + 1e-12)\n \n # Select pair according to probability\n idx = random.choices(range(len(pairs)), weights=probs, k=1)[0]\n off_point, don_point = pairs[idx]\n \n # Extract donor subtree\n donor_subtree = don_nodes[don_point][0].copy()\n \n # Replace in offspring\n node, parent, child_idx = off_nodes[off_point]\n if parent is not None:\n parent.children[child_idx] = donor_subtree\n else:\n offspring = donor_subtree\n \n # Reject if too deep\n if offspring.depth() > max_depth:\n return parent1.copy()\n \n return offspring\n\n\ndef mutate_subtree(parent, n_features, max_depth):\n \"\"\"Standard subtree mutation.\"\"\"\n offspring = parent.copy()\n tree_size = offspring.size()\n if tree_size <= 1:\n return generate_tree('grow', 3, n_features)\n \n mut_point = random.randint(1, tree_size - 1)\n new_subtree = generate_tree('grow', 3, n_features)\n \n nodes = offspring.get_all_nodes()\n node, par, child_idx = nodes[mut_point]\n if par is not None:\n par.children[child_idx] = new_subtree\n else:\n offspring = new_subtree\n \n if offspring.depth() > max_depth:\n return parent.copy()\n \n return offspring\n\n\ndef mutate_point(parent, n_features):\n \"\"\"Point mutation: change a single node.\"\"\"\n offspring = parent.copy()\n nodes = offspring.get_all_nodes()\n if len(nodes) == 0:\n return offspring\n node, parent_node, child_idx = random.choice(nodes)\n \n if node.is_terminal:\n if isinstance(node.value, str) and node.value.startswith('x'):\n # Change variable index\n idx = random.randint(0, n_features - 1)\n node.value = f'x{idx}'\n else:\n # Change constant by small perturbation\n try:\n val = float(node.value)\n # Gaussian perturbation with std dev 0.5\n node.value = str(val + random.gauss(0, 0.5))\n except:\n node.value = str(random.uniform(-5.0, 5.0))\n else:\n # Function mutation: change operator to another with same arity\n current_op = node.value\n arity = OPERATORS[current_op][1]\n candidates = [op for op in OPERATOR_NAMES if OPERATORS[op][1] == arity]\n if len(candidates) > 1:\n candidates.remove(current_op)\n node.value = random.choice(candidates)\n \n return offspring\n\n\ndef hill_climb_constants(tree, X, y, n_steps=10):\n \"\"\"Simple hill climbing for numeric constants in the tree.\n \n Randomly selects a constant terminal and tries small adjustments,\n keeping the change if it improves fitness.\n \"\"\"\n best_tree = tree.copy()\n best_fit = fitness_function(best_tree, X, y)\n \n # Collect all constant nodes\n nodes = best_tree.get_all_nodes()\n const_nodes = []\n for node, parent, idx in nodes:\n if node.is_terminal and not (isinstance(node.value, str) and node.value.startswith('x')):\n const_nodes.append((node, parent, idx))\n \n if not const_nodes:\n return best_tree\n \n for step in range(n_steps):\n # Pick a random constant node\n node, parent, idx = random.choice(const_nodes)\n old_value = float(node.value)\n \n # Generate perturbation\n delta = random.gauss(0, 0.2) # smaller step\n new_value = old_value + delta\n node.value = str(new_value)\n \n # Evaluate new fitness\n new_fit = fitness_function(best_tree, X, y)\n if new_fit < best_fit:\n best_fit = new_fit\n else:\n # Revert\n node.value = str(old_value)\n \n return best_tree\n\n\ndef mutation(parent, n_features, max_depth=17):\n \"\"\"Mutation with multiple types.\"\"\"\n r = random.random()\n if r < 0.6:\n return mutate_subtree(parent, n_features, max_depth)\n elif r < 0.9:\n return mutate_point(parent, n_features)\n else:\n # Hoist mutation: replace parent with a random child subtree\n nodes = parent.get_all_nodes()\n if len(nodes) <= 1:\n return parent.copy()\n # Choose a non-terminal node\n non_terminals = [(node, parent_node, idx) for (node, parent_node, idx) in nodes \n if not node.is_terminal]\n if not non_terminals:\n return parent.copy()\n node, parent_node, idx = random.choice(non_terminals)\n if node.children:\n child = random.choice(node.children)\n if parent_node is not None:\n parent_node.children[idx] = child.copy()\n else:\n # Replacing root\n parent = child.copy()\n return parent\n\n\ndef evolve_one_generation(population, fitnesses, X_train, y_train,\n n_features, pop_size,\n crossover_rate=0.9, mutation_rate=0.05,\n max_depth=17):\n \"\"\"Create the next generation with adaptive parsimony and hill climbing.\"\"\"\n new_population = []\n \n # Adaptive parsimony coefficient\n lengths = np.array([tree.size() for tree in population], dtype=float)\n raw_fit = np.array(fitnesses, dtype=float)\n len_var = float(np.var(lengths))\n if len_var > 1e-15:\n parsimony_coeff = float(np.cov(lengths, raw_fit)[1, 0]) / len_var\n parsimony_coeff = max(parsimony_coeff, 0.0)\n parsimony_coeff = min(parsimony_coeff, 0.001)\n else:\n parsimony_coeff = 0.0\n \n # Penalized fitnesses for selection\n penalized = [f + parsimony_coeff * l for f, l in zip(fitnesses, lengths)]\n \n # Create a sample of training points for semantic crossover\n n_sample = min(20, int(0.2 * X_train.shape[0]))\n if n_sample > 0:\n indices = random.sample(range(X_train.shape[0]), n_sample)\n X_sample = X_train[indices]\n else:\n X_sample = X_train\n \n # Elitism: keep best individual by raw fitness\n elite_idx = int(np.argmin(fitnesses))\n elite = population[elite_idx].copy()\n # Apply hill climbing to elite with 30% probability\n if random.random() < 0.3:\n elite = hill_climb_constants(elite, X_train, y_train, n_steps=5)\n new_population.append(elite)\n \n # Adaptive tournament size: increase over generations (track via global variable)\n # Use a simple heuristic: larger tournament later in evolution\n # We'll approximate generation number from population diversity\n avg_fit = np.mean(fitnesses)\n if avg_fit > 1000: # Early, high diversity\n tournament_size = 3\n else:\n tournament_size = 5 + int(5 * (1.0 - min(avg_fit / 1000.0, 1.0)))\n tournament_size = max(3, min(tournament_size, 10))\n \n while len(new_population) < pop_size:\n r = random.random()\n if r < crossover_rate:\n parents = selection(population, penalized, 2, tournament_size)\n child = crossover(parents[0], parents[1], n_features, max_depth, X_sample)\n elif r < crossover_rate + mutation_rate:\n parents = selection(population, penalized, 1, tournament_size)\n child = mutation(parents[0], n_features, max_depth)\n else:\n parents = selection(population, penalized, 1, tournament_size)\n child = parents[0]\n \n # Occasionally apply hill climbing to child\n if random.random() < 0.1:\n child = hill_climb_constants(child, X_train, y_train, n_steps=3)\n \n new_population.append(child)\n \n return new_population[:pop_size]"
7}OK: Replaced lines 228..425 with 278 line(s) in gplearn/custom_sr.py. Editable range: 228–505.
[Current file: gplearn/custom_sr.py | editable: 228–505 | total: 582 lines]
228 def fitness_function(tree, X, y):
229 """Evaluate fitness of a candidate program. Lower is better."""
230 y_pred = safe_evaluate(tree, X)
... (272 more lines) ...
503 new_population.append(child)
504
505 return new_population[:pop_size][Test #2] (1 test remaining; call submit(n=N) to choose which test result to submit as final) ### nguyen7 (scripts/nguyen7.sh) Training metrics (last generations): TRAIN_METRICS generation=0 best_fitness=0.155242 avg_fitness=149152348.448615 best_size=5 train_r2=0.786698 ### nguyen10 (scripts/nguyen10.sh) Training metrics (last generations): TRAIN_METRICS generation=0 best_fitness=0.922443 avg_fitness=1190653073035.303955 best_size=39 train_r2=0.068239
Tool Input
1{
2 "op": "replace",
3 "filename": "gplearn/custom_sr.py",
4 "start_line": 228,
5 "end_line": 505,
6 "content": "def fitness_function(tree, X, y):\n \"\"\"Evaluate fitness of a candidate program. Lower is better.\"\"\"\n y_pred = safe_evaluate(tree, X)\n return float(np.mean((y - y_pred) ** 2))\n\n\ndef _per_case_errors(population, X, y):\n \"\"\"Compute per-case absolute errors for the entire population.\n \n Returns:\n numpy array of shape (len(population), n_samples)\n \"\"\"\n errors = np.empty((len(population), X.shape[0]))\n for i, tree in enumerate(population):\n y_pred = safe_evaluate(tree, X)\n errors[i] = np.abs(y - y_pred)\n return errors\n\n\ndef selection(population, fitnesses, n_select, _errors=None, _parsimony_coeff=0.0, _lengths=None):\n \"\"\"Epsilon-lexicase selection with parsimony pressure.\n \n If _errors is not provided, fallback to tournament selection.\n \"\"\"\n selected = []\n pop_size = len(population)\n \n if _errors is None:\n # Fallback to tournament selection\n tournament_size = 5\n for _ in range(n_select):\n candidates = random.sample(range(pop_size), min(tournament_size, pop_size))\n best = min(candidates, key=lambda i: fitnesses[i])\n selected.append(population[best].copy())\n return selected\n \n n_cases = _errors.shape[1]\n # Add parsimony penalty to errors\n if _parsimony_coeff > 0 and _lengths is not None:\n # Scale parsimony term to be comparable to error magnitudes\n # Use median error as reference\n median_err = np.median(_errors)\n if median_err > 1e-12:\n scale = _parsimony_coeff * median_err\n else:\n scale = _parsimony_coeff\n penalized_errors = _errors + scale * np.array(_lengths)[:, None]\n else:\n penalized_errors = _errors\n \n for _ in range(n_select):\n candidates = list(range(pop_size))\n cases = list(range(n_cases))\n random.shuffle(cases)\n \n for case in cases:\n if len(candidates) <= 1:\n break\n case_errors = penalized_errors[candidates, case]\n median_err = float(np.median(case_errors))\n mad = float(np.median(np.abs(case_errors - median_err)))\n epsilon = median_err + mad\n candidates = [c for c, e in zip(candidates, case_errors) if e <= epsilon]\n \n winner = random.choice(candidates)\n selected.append(population[winner].copy())\n \n return selected\n\n\ndef crossover(parent1, parent2, n_features, max_depth=17):\n \"\"\"Standard subtree crossover.\"\"\"\n offspring = parent1.copy()\n donor = parent2.copy()\n \n off_size = offspring.size()\n don_size = donor.size()\n if off_size <= 1 or don_size <= 1:\n return offspring\n \n off_point = random.randint(1, off_size - 1)\n don_point = random.randint(0, don_size - 1)\n \n donor_nodes = donor.get_all_nodes()\n donor_subtree = donor_nodes[don_point][0].copy()\n \n off_nodes = offspring.get_all_nodes()\n node, parent, child_idx = off_nodes[off_point]\n if parent is not None:\n parent.children[child_idx] = donor_subtree\n else:\n offspring = donor_subtree\n \n if offspring.depth() > max_depth:\n return parent1.copy()\n \n return offspring\n\n\ndef mutation(parent, n_features, max_depth=17):\n \"\"\"Subtree mutation with occasional point mutation.\"\"\"\n r = random.random()\n if r < 0.7:\n # Standard subtree mutation\n offspring = parent.copy()\n tree_size = offspring.size()\n if tree_size <= 1:\n return generate_tree('grow', 3, n_features)\n \n mut_point = random.randint(1, tree_size - 1)\n new_subtree = generate_tree('grow', 3, n_features)\n \n nodes = offspring.get_all_nodes()\n node, par, child_idx = nodes[mut_point]\n if par is not None:\n par.children[child_idx] = new_subtree\n else:\n offspring = new_subtree\n \n if offspring.depth() > max_depth:\n return parent.copy()\n \n return offspring\n else:\n # Point mutation\n offspring = parent.copy()\n nodes = offspring.get_all_nodes()\n if len(nodes) == 0:\n return offspring\n node, parent_node, child_idx = random.choice(nodes)\n \n if node.is_terminal:\n if isinstance(node.value, str) and node.value.startswith('x'):\n idx = random.randint(0, n_features - 1)\n node.value = f'x{idx}'\n else:\n try:\n val = float(node.value)\n node.value = str(val + random.gauss(0, 0.5))\n except:\n node.value = str(random.uniform(-5.0, 5.0))\n else:\n current_op = node.value\n arity = OPERATORS[current_op][1]\n candidates = [op for op in OPERATOR_NAMES if OPERATORS[op][1] == arity]\n if len(candidates) > 1:\n candidates.remove(current_op)\n node.value = random.choice(candidates)\n return offspring\n\n\ndef evolve_one_generation(population, fitnesses, X_train, y_train,\n n_features, pop_size,\n crossover_rate=0.9, mutation_rate=0.05,\n max_depth=17):\n \"\"\"Lexicase-Parsimony GP generation.\n \n Combines epsilon-lexicase selection with adaptive parsimony pressure.\n \"\"\"\n new_population = []\n \n # Adaptive parsimony coefficient\n lengths = np.array([tree.size() for tree in population], dtype=float)\n raw_fit = np.array(fitnesses, dtype=float)\n len_var = float(np.var(lengths))\n if len_var > 1e-15:\n parsimony_coeff = float(np.cov(lengths, raw_fit)[1, 0]) / len_var\n parsimony_coeff = max(parsimony_coeff, 0.0)\n parsimony_coeff = min(parsimony_coeff, 0.001)\n else:\n parsimony_coeff = 0.0\n \n # Pre-compute per-case errors for lexicase selection\n errors = _per_case_errors(population, X_train, y_train)\n \n # Elitism: keep best individual by raw fitness\n elite_idx = int(np.argmin(fitnesses))\n new_population.append(population[elite_idx].copy())\n \n while len(new_population) < pop_size:\n r = random.random()\n if r < crossover_rate:\n parents = selection(population, fitnesses, 2, _errors=errors,\n _parsimony_coeff=parsimony_coeff, _lengths=lengths)\n child = crossover(parents[0], parents[1], n_features, max_depth)\n elif r < crossover_rate + mutation_rate:\n parents = selection(population, fitnesses, 1, _errors=errors,\n _parsimony_coeff=parsimony_coeff, _lengths=lengths)\n child = mutation(parents[0], n_features, max_depth)\n else:\n parents = selection(population, fitnesses, 1, _errors=errors,\n _parsimony_coeff=parsimony_coeff, _lengths=lengths)\n child = parents[0]\n new_population.append(child)\n \n return new_population[:pop_size]"
7}OK: Replaced lines 228..505 with 196 line(s) in gplearn/custom_sr.py. Editable range: 228–423.
[Current file: gplearn/custom_sr.py | editable: 228–423 | total: 500 lines]
228 def fitness_function(tree, X, y):
229 """Evaluate fitness of a candidate program. Lower is better."""
230 y_pred = safe_evaluate(tree, X)
... (190 more lines) ...
421 new_population.append(child)
422
423 return new_population[:pop_size][Test #3] (0 tests remaining; call submit(n=N) to choose which test result to submit as final) [NOTE] This was your last test. You MUST now call submit(n=X) to choose which test result to submit as your final answer. ## Seed 42 ### nguyen7 (scripts/nguyen7.sh) Training metrics (last generations): TRAIN_METRICS generation=45 best_fitness=0.125232 avg_fitness=1.100520 best_size=4 train_r2=0.827931 TRAIN_METRICS generation=46 best_fitness=0.125232 avg_fitness=1.158143 best_size=4 train_r2=0.827931 TRAIN_METRICS generation=47 best_fitness=0.125232 avg_fitness=1.127978 best_size=4 train_r2=0.827931 TRAIN_METRICS generation=48 best_fitness=0.125232 avg_fitness=1.123974 best_size=4 train_r2=0.827931 TRAIN_METRICS generation=49 best_fitness=0.125232 avg_fitness=1.102008 best_size=4 train_r2=0.827931 Test results (nguyen7): TEST_METRICS r2=0.920939 rmse=0.324226 train_r2=0.827931 size=4 expression="(x0 add sin(x0))" R² = 0.920939 RMSE = 0.324226 Expression: (x0 add sin(x0)) ### ngu
Tool Input
1{
2 "n": 1
3}[submit] Submitting result from test #1 as final. ## Seed 42 ### nguyen7 (scripts/nguyen7.sh) Training metrics (last generations): TRAIN_METRICS generation=45 best_fitness=0.003590 avg_fitness=1.735154 best_size=29 train_r2=0.995067 TRAIN_METRICS generation=46 best_fitness=0.003590 avg_fitness=1.313456 best_size=29 train_r2=0.995067 TRAIN_METRICS generation=47 best_fitness=0.003590 avg_fitness=1.048807 best_size=29 train_r2=0.995067 TRAIN_METRICS generation=48 best_fitness=0.003590 avg_fitness=0.438626 best_size=29 train_r2=0.995067 TRAIN_METRICS generation=49 best_fitness=0.003590 avg_fitness=0.937607 best_size=29 train_r2=0.995067 Test results (nguyen7): TEST_METRICS r2=0.798371 rmse=0.517776 train_r2=0.995067 size=29 expression="log(((((sin(x0) div exp(-2.4)) add cos(exp(x0))) mul (cos(((2.2639796352220136 div exp(2.22)) mul x0)) sub sin(cos(x0)))) sub (-0.95 add (x0 mul sin(x0)))))" R² = 0.798371 RMSE = 0.517776 Expression: log(((((sin(x0) div exp(-2.4)) add cos(exp(x0))