pallets__jinja-1455

Snapshot captured 2026-11-12T17:06:59.476467+00:00 · Autoresearch-Arena/gemma-4-developer-agent

{
  "datasets": [
    "opd_e4b_pure_v7",
    "opd_e4b_pure_v7b"
  ],
  "examples": 6,
  "instance_id": "pallets__jinja-1455",
  "problem_statement": "containment should only fail for `StrictUndefined`, not regular `Undefined`\n### Summary\r\nHello, I'm looking into a change of behaviour between jinja2-2.11.x (and previous... this section of our template has been around for a long time) and jinja2-3.0.x.  I have a feeling it was a bug in 2.11 that got fixed but since I don't see the behaviour change spelled out in the changelog I decided to report it in case it was an unintended side effect of a different bugfix.\r\n\r\nWith jinja2-2.11.x, certain `{% if x in dictionary[key] %}` constructs will evaluate False if key is not present in the dictionary.  In jinja2-3.0.x, the construct will raise an UndefinedException instead. \r\n\r\n### Reproducer\r\n\r\nHere's a small template that triggers the behaviour:\r\n``` jinja\r\n{# Called with d = {'test': {}} #}\r\nStart\r\n{% for entry, value in d.items() %}\r\n{%     if 'n' in value.a %}\r\n    Condition triggered\r\n{%     endif %}\r\n{% endfor %}\r\nEnd\r\n```\r\nAnd here's my test script with the template embedded in a python string:\r\n``` python\r\n#!/usr/bin/python3    \r\n    \r\nimport jinja2    \r\ndata = \"\"\"    \r\n{# Called with d = {'test': {}} #}\r\nStart\r\n{% for entry, value in d.items() %}\r\n{%     if 'n' in value.a %} \r\nCondition triggered\r\n{%     endif %}\r\n{% endfor %}\r\nEnd\r\n\"\"\"    \r\nt = jinja2.Template(data)    \r\nprint(t.render(d={'test': {}}))\r\n```\r\n\r\n#### Behaviour in 2.11\r\n```\r\n    \r\n\r\nStart\r\n\r\n\r\n\r\nEnd\r\n```\r\n\r\n#### Behaviour in 3.0\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"/srv/ansible/jinja2-3.x-bug/test2.py\", line 15, in <module>\r\n    print(t.render(d={'test': {}}))\r\n  File \"/home/badger/.local/lib/python3.9/site-packages/jinja2/environment.py\", line 1304, in render\r\n    self.environment.handle_exception()\r\n  File \"/home/badger/.local/lib/python3.9/site-packages/jinja2/environment.py\", line 925, in handle_exception\r\n    raise rewrite_traceback_stack(source=source)\r\n  File \"<template>\", line 5, in top-level template code\r\njinja2.exceptions.UndefinedError: 'dict object' has no attribute 'a'\r\n```\r\n### Additional information\r\n\r\nI have done a little bit of experimentation and the change is caused by https://github.com/pallets/jinja/pull/1204/ (If I remove that change from a jinja2-3.0.1 install, it reverts to the 2.11.x behaviour).  However, since my template does not traceback with a TypeError in 2.11.x (which is the bug being fixed by that PR: https://github.com/pallets/jinja/issues/1198  ), I'm unsure if the new behaviour is desired or not.\r\n\r\nPersonally, I feel the new behaviour is more intuitive and want to point it out to people as a reason they should normalize their data before passing it off to be rendered but I just wanted to make sure it was an intended change before I open my big mouth ;-)\r\n\r\n\r\nEnvironment:\r\n\r\n- Python version: Tested in both 3.9 and 3.8\r\n- Jinja version: 2.11.2 versus 3.0.1",
  "repo": "pallets/jinja",
  "source": "v2/experiments/train_data/opd_e4b_pure_v7/verified_rft_messages.jsonl",
  "source_sha256": "e37f7bd675322f3f81500b73559d60d5f602ca0537bf19f5cb06332c728c7b72"
}

{
  "captured_at": "2026-11-12T17:06:59.476467+00:00",
  "membership": "archived training",
  "membership_evidence": "Task issues extracted from preserved verified_rft_messages training examples. This is not a verified current RL wave manifest.",
  "name": "Preserved post-training task archive",
  "source": "Autoresearch-Arena/gemma-4-developer-agent",
  "source_revision": "016496de222aaafca0d4ee4c18efc63ba616f76e",
  "sources": [
    {
      "path": "v2/experiments/train_data/opd_e4b_cispo_v5/verified_rft_messages.jsonl",
      "sha256": "a43610ad0dfb6ace95375cfae09000f41afc0338cc925af870a4ade7a9b20c4c"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_hybrid_v3/verified_rft_messages.jsonl",
      "sha256": "bd326303eb127e8e5e08f03daf97bb88d79d19709d59f3f93f348bc56686a8df"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_onpolicy_v3/verified_rft_messages.jsonl",
      "sha256": "960c680e797e823a0d3a22f4b1d5ef60a72971373d703b877340b0c9126aaf68"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_pure_v6a/verified_rft_messages.jsonl",
      "sha256": "f045b7ca5b3883a1e2eaee81e1c02a4a1b53dd7b7b36e98bacb2ba5823a15148"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_pure_v6b/verified_rft_messages.jsonl",
      "sha256": "0f62460a9706eea5e1605255896f191ab00cb5706c5f1f2e1c293e9a718218dc"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_pure_v7/verified_rft_messages.jsonl",
      "sha256": "e37f7bd675322f3f81500b73559d60d5f602ca0537bf19f5cb06332c728c7b72"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_pure_v7b/verified_rft_messages.jsonl",
      "sha256": "b864bdc8695183433eef6e31cb8d33fad44ba3aded2170c1ade4ad5b2944ce23"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_self_audit_v4/verified_rft_messages.jsonl",
      "sha256": "0026023745afb8f3dc1c176dc7f7171157b4ad5a66656b524af218e8e19ceee6"
    },
    {
      "path": "v2/experiments/train_data/rft_e4b_aligned_v2/verified_rft_messages.jsonl",
      "sha256": "92a4515fee9d152b0d4c510ee3ce4caaaffe2003ba7584469f8027f174aab18c"
    },
    {
      "path": "v2/experiments/train_data/rft_v1/verified_rft_messages.jsonl",
      "sha256": "a42453ccd7ac7680b0a03865e8e7080bc8bf74c8d9d193b440c441ed46f8512b"
    },
    {
      "path": "v2/experiments/train_data/rft_v2/verified_rft_messages.jsonl",
      "sha256": "18f4b3bd7e651f83c6acab479c9475e3fa50dc652a384f309e97146983ac85ba"
    }
  ]
}