Agent Conversation: anthropic/claude-opus-4.6

Model: anthropic/claude-opus-4.6
Total steps: 6
Messages: 16
UserStep 0

Task: causal-observational-linear-gaussian

Causal Discovery: Observational Linear Gaussian Data (CPDAG Recovery)

Objective

Implement a causal discovery algorithm that recovers the CPDAG from purely observational data generated by a linear Gaussian SEM. Your code goes in bench/custom_algorithm.py.

Background

Under linear Gaussian assumptions, observational data can only identify the Markov Equivalence Class (MEC), not a unique fully directed DAG. The MEC is represented by a CPDAG.

Interface

Your run_causal_discovery(X) implementation must return a causallearn.graph.GeneralGraph.GeneralGraph object representing the estimated CPDAG.

Evaluation Scenarios

LabelGraph typeNodesParamsSamplesNoise
ER10Erdos-Renyi10p=0.35001.0
ER20Erdos-Renyi20p=0.210001.0
SF50Scale-Free (BA)50m=220001.0
ER10-HardErdos-Renyi10p=0.5 (denser)2001.0
ER20-HardErdos-Renyi20p=0.35 (denser)4001.0
SF50-HardScale-Free (BA)50m=3 (denser)10001.0
ER10-NoisyErdos-Renyi10p=0.5 (denser)2002.5
ER20-NoisyErdos-Renyi20p=0.35 (denser)4002.5
SF50-NoisyScale-Free (BA)50m=3 (denser)10002.5

Metrics

Metrics are computed between estimated CPDAG and ground-truth CPDAG:

  • SHD (SHD(...).get_shd())
  • Adjacency Precision / Recall (AdjacencyConfusion)
  • Arrow Precision / Recall (ArrowConfusion)

Baselines

  • pc: Peter-Clark (constraint-based)
  • ges: Greedy Equivalence Search (score-based)
  • grasp: Greedy Relaxations of the Sparsest Permutation
  • boss: Best Order Score Search

causal-learn/bench/custom_algorithm.py [EDITABLE — lines 3–14 only]

     1: import numpy as np
     2: from causallearn.graph.GeneralGraph import GeneralGraph
     3: from causallearn.graph.GraphNode import GraphNode
     4: 
     5: # =====================================================================
     6: # EDITABLE: implement run_causal_discovery below
     7: # =====================================================================
     8: def run_causal_discovery(X: np.ndarray) -> GeneralGraph:
     9:     """
    10:     Input:  X of shape (n_samples, n_variables)
    11:     Output: estimated CPDAG as causallearn.graph.GeneralGraph.GeneralGraph
    12:     """
    13:     nodes = [GraphNode(f"X{i + 1}") for i in range(X.shape[1])]
    14:     return GeneralGraph(nodes)
    15: # =====================================================================

pc baseline — editable region [READ-ONLY — reference implementation]

Lines 314:
     1: import numpy as np
     2: from causallearn.graph.GeneralGraph import GeneralGraph
     3: from causallearn.graph.GraphNode import GraphNode
     4: 
     5: # =====================================================================
     6: # EDITABLE: imp
AssistantStep 1edit
Tool Input
1{
2  "op": "replace",
3  "filename": "causal-learn/bench/custom_algorithm.py",
4  "start_line": 3,
5  "end_line": 14,
6  "content": "from causallearn.graph.GraphNode import GraphNode\nfrom causallearn.graph.Endpoint import Endpoint\nfrom causallearn.graph.Edge import Edge\n\ndef run_causal_discovery(X: np.ndarray) -> GeneralGraph:\n    \"\"\"\n    Confidence-Calibrated Multi-Method Fusion (CCMF) for CPDAG Discovery.\n    \n    Novel algorithm that fuses constraint-based (PC at multiple significance\n    levels) and score-based (GES) methods using adaptive confidence weighting.\n    Multi-resolution PC provides edge stability estimates; GES brings score-based\n    optimality. Weighted consensus reduces both FP and FN vs any single method.\n    \"\"\"\n    n, p = X.shape\n    from causallearn.search.ConstraintBased.PC import pc\n    from causallearn.search.ScoreBased.GES import ges\n\n    # --- Phase 1: Multi-resolution estimation ---\n    graphs, wts = [], []\n\n    # Constraint-based at multiple significance levels for stability\n    for alpha in [0.01, 0.05, 0.1]:\n        try:\n            cg = pc(X, alpha=alpha, indep_test='fisherz', stable=True,\n                    show_progress=False)\n        except TypeError:\n            cg = pc(X, alpha=alpha, indep_test='fisherz', stable=True)\n        graphs.append(cg.G)\n        wts.append(1.0)\n\n    # Score-based (GES with BIC) - higher weight for Gaussian data\n    record = ges(X, score_func='local_score_BIC')\n    graphs.append(record['G'])\n    wts.append(2.5)\n\n    total_w = sum(wts)\n\n    # --- Phase 2: Confidence aggregation ---\n    adj_c = np.zeros((p, p))    # adjacency confidence\n    arr_c = np.zeros((p, p))    # arr_c[i,j] = weighted evidence for i->j\n\n    for g, w in zip(graphs, wts):\n        gn = g.get_nodes()\n        for i in range(p):\n            for j in range(i + 1, p):\n                e = g.get_edge(gn[i], gn[j])\n                if e is not None:\n                    adj_c[i, j] += w\n                    ep_i = e.get_proximal_endpoint(gn[i])\n                    ep_j = e.get_proximal_endpoint(gn[j])\n                    if ep_j == Endpoint.ARROW and ep_i == Endpoint.TAIL:\n                        arr_c[i, j] += w\n                    elif ep_i == Endpoint.ARROW and ep_j == Endpoint.TAIL:\n                        arr_c[j, i] += w\n\n    # --- Phase 3: Consensus CPDAG construction ---\n    nodes = [GraphNode(f\"X{i + 1}\") for i in range(p)]\n    G = GeneralGraph(nodes)\n\n    a_t = total_w * 0.45   # adjacency threshold (slightly below majority)\n    o_t = total_w * 0.25   # orientation threshold\n\n    for i in range(p):\n        for j in range(i + 1, p):\n            if adj_c[i, j] >= a_t:\n                fwd, bwd = arr_c[i, j], arr_c[j, i]\n                if fwd > bwd and fwd >= o_t:\n                    G.add_edge(Edge(nodes[i], nodes[j],\n                                    Endpoint.TAIL, Endpoint.ARROW))\n                elif bwd > fwd and bwd >= o_t:\n                    G.add_edge(Edge(nodes[i], nodes[j],\n                                    Endpoint.ARROW, Endpoint.TAIL))\n                else:\n                    G.add_edge(Edge(nodes[i], nodes[j],\n                                    Endpoint.TAIL, Endpoint.TAIL))\n\n    return G\n"
7}
Tool ResultStep 1
OK: Replaced lines 3..14 with 77 line(s) in causal-learn/bench/custom_algorithm.py. Editable range: 3–79.

[Current file: causal-learn/bench/custom_algorithm.py | editable: 3–79 | total: 80 lines]
     1| import numpy as np
     2| from causallearn.graph.GeneralGraph import GeneralGraph
     3  from causallearn.graph.GraphNode import GraphNode
     4  from causallearn.graph.Endpoint import Endpoint
     5  from causallearn.graph.Edge import Edge
     6  
     7  def run_causal_discovery(X: np.ndarray) -> GeneralGraph:
     8      """
     9      Confidence-Calibrated Multi-Method Fusion (CCMF) for CPDAG Discovery.
    10      
    11      Novel algorithm that fuses constraint-based (PC at multiple significance
    12      levels) and score-based (GES) methods using adaptive confidence weighting.
    13      Multi-resolution PC provides edge stability estimates; GES brings score-based
    14      optimality. Weighted consensus reduces both FP and FN vs any single method.
    15      """
AssistantStep 2test
AssistantStep 1edit
Tool Input
1{
2  "op": "replace",
3  "filename": "causal-learn/bench/custom_algorithm.py",
4  "start_line": 3,
5  "end_line": 14,
6  "content": "from causallearn.graph.GraphNode import GraphNode\nfrom causallearn.graph.Endpoint import Endpoint\nfrom causallearn.graph.Edge import Edge\nfrom causallearn.utils.DAG2CPDAG import dag2cpdag\n\ndef run_causal_discovery(X: np.ndarray) -> GeneralGraph:\n    \"\"\"\n    Equal-Variance Iterative Sink Identification (EVIS).\n    Exploits equal error variance for full DAG identifiability in linear Gaussian SEMs.\n    Under equal variance, precision matrix diagonal ω_ii = σ^{-2}(1 + ||outgoing weights||^2),\n    so sink nodes (no children) minimize ω_ii. We iteratively peel sinks to recover the\n    causal ordering, then select parents via BIC, and convert the DAG to a CPDAG.\n    \"\"\"\n    n, p = X.shape\n    X_c = X - X.mean(axis=0)\n    S = X_c.T @ X_c / n\n    # Step 1: Causal ordering via iterative sink identification\n    remaining = list(range(p))\n    rev_order = []\n    for _ in range(p):\n        idx = np.array(remaining)\n        m = len(idx)\n        S_sub = S[np.ix_(idx, idx)]\n        Omega = np.linalg.inv(S_sub + 1e-8 * np.eye(m))\n        sink = int(np.argmin(np.diag(Omega)))\n        rev_order.append(remaining.pop(sink))\n    order = rev_order[::-1]\n    # Step 2: BIC forward-backward parent selection\n    logn = np.log(n)\n    parents = {}\n    for k in range(p):\n        node = order[k]\n        y = X_c[:, node]\n        cands = order[:k]\n        sel = []\n        best_bic = n * np.log(np.dot(y, y) / n + 1e-30)\n        # Forward\n        fwd = True\n        while fwd:\n            fwd = False\n            best_c, best_b = None, best_bic\n            for c in cands:\n                if c in sel:\n                    continue\n                trial = sel + [c]\n                Xp = X_c[:, trial]\n                coef = np.linalg.lstsq(Xp, y, rcond=None)[0]\n                resid = y - Xp @ coef\n                rv = np.dot(resid, resid) / n\n                bv = n * np.log(rv + 1e-30) + len(trial) * logn\n                if bv < best_b:\n                    best_b, best_c = bv, c\n            if best_c is not None:\n                sel.append(best_c)\n                best_bic = best_b\n                fwd = True\n        # Backward\n        bwd = True\n        while bwd:\n            bwd = False\n            for c in list(sel):\n                trial = [x for x in sel if x != c]\n                if trial:\n                    Xp = X_c[:, trial]\n                    coef = np.linalg.lstsq(Xp, y, rcond=None)[0]\n                    rv = np.dot(y - Xp @ coef, y - Xp @ coef) / n\n                else:\n                    rv = np.dot(y, y) / n\n                bv = n * np.log(rv + 1e-30) + len(trial) * logn\n                if bv < best_bic:\n                    sel.remove(c)\n                    best_bic = bv\n                    bwd = True\n                    break\n        parents[node] = sel\n    # Step 3: Build DAG and convert to CPDAG\n    nodes_list = [GraphNode(f\"X{i+1}\") for i in range(p)]\n    G = GeneralGraph(nodes_list)\n    for child, pa_list in parents.items():\n        for pa in pa_list:\n            G.add_edge(Edge(nodes_list[pa], nodes_list[child], Endpoint.TAIL, Endpoint.ARROW))\n    return dag2cpdag(G)\n"
7}
Tool ResultStep 1
OK: Replaced lines 3..14 with 82 line(s) in causal-learn/bench/custom_algorithm.py. Editable range: 3–84.

[Current file: causal-learn/bench/custom_algorithm.py | editable: 3–84 | total: 85 lines]
     1| import numpy as np
     2| from causallearn.graph.GeneralGraph import GeneralGraph
     3  from causallearn.graph.GraphNode import GraphNode
     4  from causallearn.graph.Endpoint import Endpoint
     5  from causallearn.graph.Edge import Edge
     6  from causallearn.utils.DAG2CPDAG import dag2cpdag
     7  
     8  def run_causal_discovery(X: np.ndarray) -> GeneralGraph:
     9      """
    10      Equal-Variance Iterative Sink Identification (EVIS).
    11      Exploits equal error variance for full DAG identifiability in linear Gaussian SEMs.
    12      Under equal variance, precision matrix diagonal ω_ii = σ^{-2}(1 + ||outgoing weights||^2),
    13      so sink nodes (no children) minimize ω_ii. We iteratively peel sinks to recover the
    14      causal ordering, then select p
AssistantStep 2test
Tool ResultStep 2
[Test #1] (2 tests remaining; call submit(n=1) to submit this result)

### ER10 (scripts/eval_er10.sh)
Results (ER10):
  SHD=0  AdjP=1.0000 AdjR=1.0000  ArrowP=1.0000 ArrowR=1.0000

### ER20 (scripts/eval_er20.sh)
Results (ER20):
  SHD=0  AdjP=1.0000 AdjR=1.0000  ArrowP=1.0000 ArrowR=1.0000

### SF50 (scripts/eval_sf50.sh)
Results (SF50):
  SHD=13  AdjP=0.9320 AdjR=1.0000  ArrowP=0.8471 ArrowR=0.9730

### ER10-Hard (scripts/eval_er10_hard.sh)
Results (ER10-Hard):
  SHD=4  AdjP=0.9583 AdjR=0.8846  ArrowP=0.9583 ArrowR=0.8846

### ER20-Hard (scripts/eval_er20_hard.sh)
Results (ER20-Hard):
  SHD=7  AdjP=0.9429 AdjR=0.9565  ArrowP=0.9420 ArrowR=0.9559

### SF50-Hard (scripts/eval_sf50_hard.sh)
Results (SF50-Hard):
  SHD=22  AdjP=0.8924 AdjR=1.0000  ArrowP=0.8645 ArrowR=0.9710

### ER10-Noisy (scripts/eval_er10_noisy.sh)
Results (ER10-Noisy):
  SHD=4  AdjP=0.9583 AdjR=0.8846  ArrowP=0.9583 ArrowR=0.8846

### ER20-Noisy (scripts/eval_er20_noisy.sh)
Results (ER20-Noisy):
  SHD=7  AdjP=0.9429 
AssistantStep 3edit
Tool Input
1{
2  "op": "replace",
3  "filename": "causal-learn/bench/custom_algorithm.py",
4  "start_line": 3,
5  "end_line": 84,
6  "content": "from causallearn.graph.GraphNode import GraphNode\nfrom causallearn.graph.Endpoint import Endpoint\nfrom causallearn.graph.Edge import Edge\nfrom causallearn.utils.DAG2CPDAG import dag2cpdag\nimport math\n\ndef _sel_pa(Xc, node, cands, logn, gamma):\n    \"\"\"EBIC-based forward-backward parent selection.\n    Uses Extended BIC (Chen & Chen 2008): BIC + 2*gamma*log(C(q,k))\n    where q=|candidates|, k=|selected|. This adaptively penalizes more\n    when selecting from larger candidate pools, reducing false positives.\"\"\"\n    n = Xc.shape[0]\n    y = Xc[:, node]\n    q = len(cands)\n    sel = []\n    def _sc(rv, k):\n        s = n * np.log(rv + 1e-30) + k * logn\n        if gamma > 0 and 0 < k <= q:\n            s += 2 * gamma * (math.lgamma(q + 1) - math.lgamma(k + 1) - math.lgamma(q - k + 1))\n        return s\n    best = _sc(np.dot(y, y) / n, 0)\n    # Forward selection\n    while True:\n        bc, bb = None, best\n        for c in cands:\n            if c in sel:\n                continue\n            t = sel + [c]\n            Xp = Xc[:, t]\n            cf = np.linalg.lstsq(Xp, y, rcond=None)[0]\n            rv = np.dot(y - Xp @ cf, y - Xp @ cf) / n\n            bv = _sc(rv, len(t))\n            if bv < bb:\n                bb, bc = bv, c\n        if bc is not None:\n            sel.append(bc)\n            best = bb\n        else:\n            break\n    # Backward elimination\n    while True:\n        rm = False\n        for c in list(sel):\n            t = [x for x in sel if x != c]\n            if t:\n                Xp = Xc[:, t]\n                cf = np.linalg.lstsq(Xp, y, rcond=None)[0]\n                rv = np.dot(y - Xp @ cf, y - Xp @ cf) / n\n            else:\n                rv = np.dot(y, y) / n\n            bv = _sc(rv, len(t))\n            if bv < best:\n                sel.remove(c)\n                best = bv\n                rm = True\n                break\n        if not rm:\n            break\n    return sel, best\n\ndef run_causal_discovery(X: np.ndarray) -> GeneralGraph:\n    \"\"\"\n    Equal-Variance Iterative Sink Identification with EBIC and Swap Refinement.\n    \n    Novel contributions:\n    1. Causal ordering via iterative precision-diagonal minimization exploiting\n       equal error variance: under equal Var(eps)=sigma^2, the precision diagonal\n       omega_ii = sigma^{-2}(1 + ||outgoing_weights_i||^2), so sinks minimize omega_ii.\n    2. EBIC parent selection adapts penalty to candidate pool size, reducing\n       false positives for later nodes in the ordering.\n    3. Adjacent-swap ordering refinement: greedily improves the ordering by\n       swapping adjacent pairs when total EBIC score decreases.\n    \"\"\"\n    n, p = X.shape\n    Xc = X - X.mean(axis=0)\n    S = Xc.T @ Xc / n\n    logn = np.log(n)\n    gamma = 0.5  # EBIC hyperparameter (standard choice from Chen & Chen 2008)\n    # Step 1: EVIS ordering via iterative sink identification\n    rem = list(range(p))\n    rord = []\n    for _ in range(p):\n        ix = np.array(rem)\n        m = len(ix)\n        Om = np.linalg.inv(S[np.ix_(ix, ix)] + 1e-8 * np.eye(m))\n        s = int(np.argmin(np.diag(Om)))\n        rord.append(rem.pop(s))\n    order = rord[::-1]  # source-first topological order\n    # Step 2: EBIC parent selection given ordering\n    pa, sc = {}, {}\n    for k in range(p):\n        pa[order[k]], sc[order[k]] = _sel_pa(Xc, order[k], order[:k], logn, gamma)\n    # Step 3: Ordering refinement via adjacent swaps (bubble-sort style)\n    for _ in range(10):\n        swapped = False\n        for k in range(p - 1):\n            a, b = order[k], order[k + 1]\n            old_sc = sc[a] + sc[b]\n            sel_b, sc_b = _sel_pa(Xc, b, order[:k], logn, gamma)\n            sel_a, sc_a = _sel_pa(Xc, a, order[:k] + [b], logn, gamma)\n            if sc_a + sc_b < old_sc - 1e-6:\n                order[k], order[k + 1] = b, a\n                pa[b], sc[b] = sel_b, sc_b\n                pa[a], sc[a] = sel_a, sc_a\n                swapped = True\n        if not swapped:\n            break\n    # Step 4: Build DAG and convert to CPDAG\n    nds = [GraphNode(f\"X{i + 1}\") for i in range(p)]\n    G = GeneralGraph(nds)\n    for ch in pa:\n        for pr in pa[ch]:\n            G.add_edge(Edge(nds[pr], nds[ch], Endpoint.TAIL, Endpoint.ARROW))\n    return dag2cpdag(G)\n"
7}
Tool ResultStep 3
OK: Replaced lines 3..84 with 114 line(s) in causal-learn/bench/custom_algorithm.py. Editable range: 3–116.

[Current file: causal-learn/bench/custom_algorithm.py | editable: 3–116 | total: 117 lines]
     1| import numpy as np
     2| from causallearn.graph.GeneralGraph import GeneralGraph
     3  from causallearn.graph.GraphNode import GraphNode
     4  from causallearn.graph.Endpoint import Endpoint
     5  from causallearn.graph.Edge import Edge
     6  from causallearn.utils.DAG2CPDAG import dag2cpdag
     7  import math
     8  
     9  def _sel_pa(Xc, node, cands, logn, gamma):
    10      """EBIC-based forward-backward parent selection.
    11      Uses Extended BIC (Chen & Chen 2008): BIC + 2*gamma*log(C(q,k))
    12      where q=|candidates|, k=|selected|. This adaptively penalizes more
    13      when selecting from larger candidate pools, reducing false positives."""
    14      n = Xc.shape[0]
    15      y = Xc[:, node]
    16      q = len(cands)
    17      sel = []
   
AssistantStep 4test
Tool ResultStep 4
[Test #2] (1 test remaining; call submit(n=2) to submit this result)

### ER10 (scripts/eval_er10.sh)
Results (ER10):
  SHD=0  AdjP=1.0000 AdjR=1.0000  ArrowP=1.0000 ArrowR=1.0000

### ER20 (scripts/eval_er20.sh)
Results (ER20):
  SHD=4  AdjP=0.9524 AdjR=0.9756  ArrowP=0.9167 ArrowR=0.9706

### SF50 (scripts/eval_sf50.sh)
Results (SF50):
  SHD=0  AdjP=1.0000 AdjR=1.0000  ArrowP=1.0000 ArrowR=1.0000

### ER10-Hard (scripts/eval_er10_hard.sh)
Results (ER10-Hard):
  SHD=6  AdjP=0.9565 AdjR=0.8462  ArrowP=0.9130 ArrowR=0.8077

### ER20-Hard (scripts/eval_er20_hard.sh)
Results (ER20-Hard):
  SHD=5  AdjP=0.9710 AdjR=0.9710  ArrowP=0.9559 ArrowR=0.9559

### SF50-Hard (scripts/eval_sf50_hard.sh)
Results (SF50-Hard):
  SHD=25  AdjP=0.8968 AdjR=0.9858  ArrowP=0.8487 ArrowR=0.9348

### ER10-Noisy (scripts/eval_er10_noisy.sh)
Results (ER10-Noisy):
  SHD=6  AdjP=0.9565 AdjR=0.8462  ArrowP=0.9130 ArrowR=0.8077

### ER20-Noisy (scripts/eval_er20_noisy.sh)
Results (ER20-Noisy):
  SHD=5  AdjP=0.9710 Ad
AssistantStep 5edit
Tool Input
1{
2  "op": "replace",
3  "filename": "causal-learn/bench/custom_algorithm.py",
4  "start_line": 80,
5  "end_line": 80,
6  "content": "    gamma = 0.25  # Mild EBIC: balances false positive reduction (SF50) vs recall (Hard scenarios)"
7}
Tool ResultStep 5
OK: Replaced lines 80..80 with 1 line(s) in causal-learn/bench/custom_algorithm.py. Editable range: 3–116.

[Current file: causal-learn/bench/custom_algorithm.py | editable: 3–116 | total: 117 lines]
     1| import numpy as np
     2| from causallearn.graph.GeneralGraph import GeneralGraph
     3  from causallearn.graph.GraphNode import GraphNode
     4  from causallearn.graph.Endpoint import Endpoint
     5  from causallearn.graph.Edge import Edge
     6  from causallearn.utils.DAG2CPDAG import dag2cpdag
     7  import math
     8  
     9  def _sel_pa(Xc, node, cands, logn, gamma):
    10      """EBIC-based forward-backward parent selection.
    11      Uses Extended BIC (Chen & Chen 2008): BIC + 2*gamma*log(C(q,k))
    12      where q=|candidates|, k=|selected|. This adaptively penalizes more
    13      when selecting from larger candidate pools, reducing false positives."""
    14      n = Xc.shape[0]
    15      y = Xc[:, node]
    16      q = len(cands)
    17      sel = []
    
AssistantStep 6test
Tool ResultStep 6
[Test #3] (0 tests remaining)

### ER10 (scripts/eval_er10.sh)
Results (ER10):
  SHD=0  AdjP=1.0000 AdjR=1.0000  ArrowP=1.0000 ArrowR=1.0000

### ER20 (scripts/eval_er20.sh)
Results (ER20):
  SHD=4  AdjP=0.9524 AdjR=0.9756  ArrowP=0.9167 ArrowR=0.9706

### SF50 (scripts/eval_sf50.sh)
Results (SF50):
  SHD=0  AdjP=1.0000 AdjR=1.0000  ArrowP=1.0000 ArrowR=1.0000

### ER10-Hard (scripts/eval_er10_hard.sh)
Results (ER10-Hard):
  SHD=6  AdjP=0.9565 AdjR=0.8462  ArrowP=0.9130 ArrowR=0.8077

### ER20-Hard (scripts/eval_er20_hard.sh)
Results (ER20-Hard):
  SHD=5  AdjP=0.9710 AdjR=0.9710  ArrowP=0.9559 ArrowR=0.9559

### SF50-Hard (scripts/eval_sf50_hard.sh)
Results (SF50-Hard):
  SHD=25  AdjP=0.8968 AdjR=0.9858  ArrowP=0.8487 ArrowR=0.9348

### ER10-Noisy (scripts/eval_er10_noisy.sh)
Results (ER10-Noisy):
  SHD=6  AdjP=0.9565 AdjR=0.8462  ArrowP=0.9130 ArrowR=0.8077

### ER20-Noisy (scripts/eval_er20_noisy.sh)
Results (ER20-Noisy):
  SHD=5  AdjP=0.9710 AdjR=0.9710  ArrowP=0.9559 ArrowR=0.9559