reata__sqllineage-582

Snapshot captured 2026-11-12T17:06:59.476467+00:00 · Autoresearch-Arena/gemma-4-developer-agent

{
  "datasets": [
    "opd_e4b_pure_v6a",
    "opd_e4b_pure_v6b",
    "opd_e4b_self_audit_v4"
  ],
  "examples": 5,
  "instance_id": "reata__sqllineage-582",
  "problem_statement": "Enable support of `sqlfluff` context\n## Is your feature request related to a problem? Please describe\r\n\r\nAs soon as dialect-based parsing use `sqlfluff` to parse SQL it will be great to support sqlfluff context (from `.sqlfluff` files) where we can use/set templater and other options\r\n\r\n## Describe the solution you'd like\r\n\r\n`sqlfluff.core.linter.Linter` have optional `config: FluffConfig` parameter. Need to pass it or ensure that default is used\r\n\r\n## Additional context\r\n\r\n```verilog\r\ndmytro.kulyk@MAC-570022 datalake % sqllineage -f transform/airflow/dags/queries/aws_transforms/aggregate/pokermatch_payment.partitions_range.sql\r\nTraceback (most recent call last):\r\n  File \"/opt/homebrew/bin/sqllineage\", line 8, in <module>\r\n    sys.exit(main())\r\n             ^^^^^^\r\n  File \"/opt/homebrew/lib/python3.11/site-packages/sqllineage/cli.py\", line 125, in main\r\n    runner.print_table_lineage()\r\n  File \"/opt/homebrew/lib/python3.11/site-packages/sqllineage/runner.py\", line 175, in print_table_lineage\r\n    print(str(self))\r\n          ^^^^^^^^^\r\n  File \"/opt/homebrew/lib/python3.11/site-packages/sqllineage/runner.py\", line 26, in wrapper\r\n    self._eval()\r\n  File \"/opt/homebrew/lib/python3.11/site-packages/sqllineage/runner.py\", line 195, in _eval\r\n    stmt_holder = analyzer.analyze(stmt, session.metadata_provider)\r\n                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/opt/homebrew/lib/python3.11/site-packages/sqllineage/core/parser/sqlfluff/analyzer.py\", line 48, in analyze\r\n    raise UnsupportedStatementException(\r\nsqllineage.exceptions.UnsupportedStatementException: SQLLineage cannot parse SQL:{%- set task_id = 'ph2_transaction.data_processing' %}\r\n{%- set transaction_date = ti.xcom_pull(task_ids=task_id, key='new_partitions') %}\r\n{% if transaction_date|length > 0 %}\r\n    select\r\n        row_num,\r\n        min(as_of_date) as start_date,\r\n        max(as_of_date) as end_date,\r\n        array_agg(as_of_date) as dates\r\n    from (\r\n        select\r\n...\r\n```\r\n\r\nrunning `sqlfluff` directly:\r\n```verilog\r\ndmytro.kulyk@MAC-570022 datalake % sqlfluff parse transform/airflow/dags/queries/aws_transforms/aggregate/pokermatch_payment.partitions_range.sql --verbose | more\r\n==== sqlfluff ====\r\nsqlfluff:                2.3.5 python:                 3.11.6\r\nimplementation:        cpython verbosity:                   1\r\ndialect:                athena templater:               jinja\r\nrules:                                all\r\n=== [ path: transform/airflow/dags/queries/aws_transforms/aggregate/pokermatch_payment.partitions_range.sql ] ===\r\n\r\n[L:  1, P:  1]      |file:\r\n[L:  1, P:  1]      |    [META] placeholder:                                       [Type: 'templated', Raw: \"{%- set task_id = 'ph2_transaction.data_processing' %}\"]\r\n[L:  1, P: 55]      |    [META] placeholder:                                       [Type: 'literal', Raw: '\\n']\r\n[L:  2, P:  1]      |    [META] placeholder:                                       [Type: 'templated', Raw: \"{%- set transaction_date = ti.xcom_pull(task_ids=task_id, key='new_partitions') %}\"]\r\n[L:  2, P: 83]      |    newline:                                                  '\\n'\r\n[L:  3, P:  1]      |    [META] placeholder:                                       [Type: 'block_start', Raw: '{% if transaction_date|length > 0 %}', Block: 'd42817']\r\n[L:  3, P: 37]      |    [META] indent:                                            [Block: 'd42817']\r\n[L:  3, P: 37]      |    newline:                                                  '\\n'\r\n[L:  4, P:  1]      |    whitespace:                                               '    '\r\n[L:  4, P:  5]      |    statement:\r\n[L:  4, P:  5]      |        select_statement:\r\n[L:  4, P:  5]      |            select_clause:\r\n[L:  4, P:  5]      |                keyword:                                      'select'\r\n[L:  4, P: 11]      |                [META] indent:\r\n[L:  4, P: 11]      |                newline:                                      '\\n'\r\n[L:  5, P:  1]      |                whitespace:                                   '        '\r\n[L:  5, P:  9]      |                select_clause_element:\r\n[L:  5, P:  9]      |                    column_reference:\r\n[L:  5, P:  9]      |                        naked_identifier:                     'row_num'\r\n...\r\n```\r\n\r\n`.sqlfluff` example\r\n```ini\r\n[sqlfluff]\r\ntemplater = jinja\r\noutput_line_length = 180\r\nexclude_rules = AL07, AM02, AM03, AM05, CV10, LT08, ST05, ST06, ST07\r\nwarnings = RF01\r\ndialect = athena\r\nlarge_file_skip_char_limit = 0\r\nlarge_file_skip_byte_limit = 0\r\nmax_line_length = 120\r\n\r\n[sqlfluff:rules:references.consistent]\r\nsingle_table_references = consistent\r\n\r\n[sqlfluff:rules:references.keywords]\r\nignore_words =\r\n    account, domain, language, operator, rank, result, source, type, uid, position, format, state, offset, name,\r\n    location, role, user, comment, method, date, quarter, hour, week, weekday, days\r\n\r\n[sqlfluff:templater:jinja]\r\nlibrary_path = operation/deploy/lint\r\napply_dbt_builtins = false\r\n```",
  "repo": "reata/sqllineage",
  "source": "v2/experiments/train_data/opd_e4b_pure_v6a/verified_rft_messages.jsonl",
  "source_sha256": "f045b7ca5b3883a1e2eaee81e1c02a4a1b53dd7b7b36e98bacb2ba5823a15148"
}

{
  "captured_at": "2026-11-12T17:06:59.476467+00:00",
  "membership": "archived training",
  "membership_evidence": "Task issues extracted from preserved verified_rft_messages training examples. This is not a verified current RL wave manifest.",
  "name": "Preserved post-training task archive",
  "source": "Autoresearch-Arena/gemma-4-developer-agent",
  "source_revision": "016496de222aaafca0d4ee4c18efc63ba616f76e",
  "sources": [
    {
      "path": "v2/experiments/train_data/opd_e4b_cispo_v5/verified_rft_messages.jsonl",
      "sha256": "a43610ad0dfb6ace95375cfae09000f41afc0338cc925af870a4ade7a9b20c4c"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_hybrid_v3/verified_rft_messages.jsonl",
      "sha256": "bd326303eb127e8e5e08f03daf97bb88d79d19709d59f3f93f348bc56686a8df"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_onpolicy_v3/verified_rft_messages.jsonl",
      "sha256": "960c680e797e823a0d3a22f4b1d5ef60a72971373d703b877340b0c9126aaf68"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_pure_v6a/verified_rft_messages.jsonl",
      "sha256": "f045b7ca5b3883a1e2eaee81e1c02a4a1b53dd7b7b36e98bacb2ba5823a15148"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_pure_v6b/verified_rft_messages.jsonl",
      "sha256": "0f62460a9706eea5e1605255896f191ab00cb5706c5f1f2e1c293e9a718218dc"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_pure_v7/verified_rft_messages.jsonl",
      "sha256": "e37f7bd675322f3f81500b73559d60d5f602ca0537bf19f5cb06332c728c7b72"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_pure_v7b/verified_rft_messages.jsonl",
      "sha256": "b864bdc8695183433eef6e31cb8d33fad44ba3aded2170c1ade4ad5b2944ce23"
    },
    {
      "path": "v2/experiments/train_data/opd_e4b_self_audit_v4/verified_rft_messages.jsonl",
      "sha256": "0026023745afb8f3dc1c176dc7f7171157b4ad5a66656b524af218e8e19ceee6"
    },
    {
      "path": "v2/experiments/train_data/rft_e4b_aligned_v2/verified_rft_messages.jsonl",
      "sha256": "92a4515fee9d152b0d4c510ee3ce4caaaffe2003ba7584469f8027f174aab18c"
    },
    {
      "path": "v2/experiments/train_data/rft_v1/verified_rft_messages.jsonl",
      "sha256": "a42453ccd7ac7680b0a03865e8e7080bc8bf74c8d9d193b440c441ed46f8512b"
    },
    {
      "path": "v2/experiments/train_data/rft_v2/verified_rft_messages.jsonl",
      "sha256": "18f4b3bd7e651f83c6acab479c9475e3fa50dc652a384f309e97146983ac85ba"
    }
  ]
}