update, translate skill.md

2026-06-22 10:08:52 +08:00
parent 71a23d639c
commit f9b03384ab
6 changed files with 565 additions and 541 deletions
@@ -1,98 +1,100 @@
 ---
 name: sql-review
-description: Use when performing a static review of a generated PySpark SQL string before it is executed. Triggers include "review this SQL", "SQL 评审", "静态审查", "risk level", "check join cardinality", "数据膨胀", "重复聚合", "分区裁剪". Consumes a SQL string plus the SQL Context that produced it, and emits a risk-graded review (HIGH / MEDIUM / LOW) with concrete fixes. Does NOT modify the SQL — it reports findings.
+description: 对执行前生成好的 PySpark SQL 字符串做静态评审时使用。触发词包括"review this SQL"、"SQL 评审"、"静态审查"、"risk level"、"check join cardinality"、"数据膨胀"、"重复聚合"、"分区裁剪"。消费 SQL 字符串及其产出的 SQL Context,产出按风险分级的评审结果(HIGH / MEDIUM / LOW)以及具体修复建议。**不修改 SQL** — 只报告发现。
 ---

-# SQL Review
+# SQL Review(SQL 评审)

-## Overview
+## 概述(Overview)

-Static, pre-execution review of a generated PySpark SQL string. The reviewer does **not** rewrite SQL — it emits findings, severity, and suggested fixes. The author of the SQL decides what to act on.
+对一条生成好的 PySpark SQL 字符串做执行前的静态评审。评审者**不**改写 SQL — 它只输出发现项、严重程度和建议修复。由 SQL 的作者决定采纳哪些。

-Core principle: **review against the SQL Context, not against vibes.** Every field, alias, and join key referenced in the SQL must exist in the SQL Context that produced it. If it does not, that is itself a `HIGH` finding (the SQL bypassed the contract).
+核心原则:**对照 SQL Context 评审,而不是凭直觉。** SQL 中引用的每一个字段、alias、join key 都必须存在于产出它的 SQL Context。如果不存在,这就是一项 `HIGH` 风险(SQL 绕过了契约)。

-## When to Use
+## 何时使用(When to Use)

-Use when:
- A SQL string has been generated from a `sql_context` and is about to be executed
- The user pastes a SQL string and asks for a review, optimization, or risk assessment
- A code change adds a new `spark.sql(...)` call to a PySpark script
+**使用场景:**

-Do NOT use when:
- The SQL is not Spark SQL (e.g. plain Postgres, HiveQL outside Spark) — at minimum replace Spark-specific checks with the relevant engine's equivalents, but flag it
- The SQL is a one-off interactive query that won't be reused — the user can opt out
+- 一条 SQL 字符串已经从 `sql_context` 生成出来,即将执行
+- 用户粘贴一条 SQL 字符串,要求评审、优化或风险评估
+- 代码改动为 PySpark 脚本新增了 `spark.sql(...)` 调用

-## Inputs
+**不要使用场景:**

-1. The **SQL string** to review
-2. The **SQL Context** that the SQL was generated from (or, if absent, an explicit acknowledgement that the SQL is "untrusted / ad-hoc")
+- 这条 SQL 不是 Spark SQL(例如纯 Postgres、HiveQL 在 Spark 之外)— 至少要把 Spark 特有的检查替换成对应引擎的等价检查,并标注出来
+- 是一次性、不会被复用的交互查询 — 用户可以主动选择跳过

-If the SQL Context is missing, raise a single `HIGH` finding: *"SQL has no associated SQL Context; downstream skills cannot guarantee field/join correctness."* Then continue with structural checks.
+## 输入(Inputs)

-## Severity Model
+1. 要评审的 **SQL 字符串**
+2. 产出这条 SQL 的 **SQL Context**(如果缺失,需要显式声明"该 SQL 是不可信 / 临时拼凑的")

-| Level | Meaning | Action |
+如果 SQL Context 缺失,抛出一项 `HIGH` 风险:*"SQL 没有对应的 SQL Context;下游 skill 无法保证字段 / join 的正确性。"* 然后继续做结构性检查。
+
+## 严重程度模型(Severity Model)
+
+| 等级 | 含义 | 处置 |
 |---|---|---|
-| `HIGH` | Will produce wrong numbers, fail to run, or violate policy | Block execution. Must fix. |
-| `MEDIUM` | Likely correct, but inefficient, fragile, or risky at scale | Fix before production. OK in dev. |
-| `LOW` | Style / best-practice nit | Optional fix. |
+| `HIGH` | 会导致数字错误、执行失败或违反策略 | 阻断执行,必须修。 |
+| `MEDIUM` | 大概率对,但低效、脆弱或大规模场景有风险 | 上线前修,开发环境可放过。 |
+| `LOW` | 风格 / 最佳实践小瑕疵 | 可选修。 |

-Emit findings in this order: `HIGH` → `MEDIUM` → `LOW`. A single `HIGH` finding makes the overall review verdict `FAIL`.
+按以下顺序输出发现项:`HIGH` → `MEDIUM` → `LOW`。任一 `HIGH` 让整体评审结论为 `FAIL`。

-## Workflow
+## 工作流(Workflow)

 ```dot
 digraph sql_review {
-    "Parse SQL (logical, not just regex)" [shape=box];
-    "Cross-check aliases, columns, join keys against SQL Context" [shape=box];
-    "Detect multi-statement or forbidden verbs (guardrail)" [shape=box];
-    "Analyze join cardinality & inflation risk" [shape=box];
-    "Detect repeated aggregation on same measure" [shape=box];
-    "Detect full table scan & missing partition filter" [shape=box];
-    "Verify GROUP BY matches SELECT" [shape=box];
-    "Check null handling on join keys & measures" [shape=box];
-    "Check field types vs operations" [shape=box];
-    "Aggregate findings into verdict" [shape=box];
+    "解析 SQL(逻辑层面,不只是正则)" [shape=box];
+    "对照 SQL Context 交叉校验 alias、字段、join key" [shape=box];
+    "检测多语句 / 禁用 verb(防护栏)" [shape=box];
+    "分析 join 基数与膨胀风险" [shape=box];
+    "检测同一度量的重复聚合" [shape=box];
+    "检测全表扫描与缺失分区过滤" [shape=box];
+    "校验 GROUP BY 与 SELECT 一致" [shape=box];
+    "检查 join key 与度量的空值处理" [shape=box];
+    "检查字段类型与运算的匹配" [shape=box];
+    "汇总成评审结论" [shape=box];

-    "Parse SQL (logical, not just regex)" -> "Cross-check aliases, columns, join keys against SQL Context";
-    "Cross-check aliases, columns, join keys against SQL Context" -> "Detect multi-statement or forbidden verbs (guardrail)";
-    "Detect multi-statement or forbidden verbs (guardrail)" -> "Analyze join cardinality & inflation risk";
-    "Analyze join cardinality & inflation risk" -> "Detect repeated aggregation on same measure";
-    "Detect repeated aggregation on same measure" -> "Detect full table scan & missing partition filter";
-    "Detect full table scan & missing partition filter" -> "Verify GROUP BY matches SELECT";
-    "Verify GROUP BY matches SELECT" -> "Check null handling on join keys & measures";
-    "Check null handling on join keys & measures" -> "Check field types vs operations";
-    "Check field types vs operations" -> "Aggregate findings into verdict";
+    "解析 SQL(逻辑层面,不只是正则)" -> "对照 SQL Context 交叉校验 alias、字段、join key";
+    "对照 SQL Context 交叉校验 alias、字段、join key" -> "检测多语句 / 禁用 verb(防护栏)";
+    "检测多语句 / 禁用 verb(防护栏)" -> "分析 join 基数与膨胀风险";
+    "分析 join 基数与膨胀风险" -> "检测同一度量的重复聚合";
+    "检测同一度量的重复聚合" -> "检测全表扫描与缺失分区过滤";
+    "检测全表扫描与缺失分区过滤" -> "校验 GROUP BY 与 SELECT 一致";
+    "校验 GROUP BY 与 SELECT 一致" -> "检查 join key 与度量的空值处理";
+    "检查 join key 与度量的空值处理" -> "检查字段类型与运算的匹配";
+    "检查字段类型与运算的匹配" -> "汇总成评审结论";
 }
 ```

-## Checks (run all)
+## 检查项(Checks,全部执行)

-1. **Reference integrity** — every `<alias>.<column>` in the SQL must exist in the SQL Context. Mismatches are `HIGH`.
-2. **Join key match** — every `ON` predicate must match a frozen join from the SQL Context. Invented joins are `HIGH`.
-3. **Forbidden verbs / multi-statement** — defer to `pyspark-sql-guardrails`. Any DDL/DML/maintenance verb, or any `;`-stacked statement, is `HIGH`.
-4. **Join cardinality risk** — for every join, if the SQL Context marks cardinality as `1:N` or `N:N` and there is no dedup, raise `MEDIUM` (1:N) or `HIGH` (N:N).
-5. **Repeated aggregation** — if the same measure is aggregated twice (e.g. once in a subquery, once outside) without intentional re-aggregation, raise `HIGH`.
-6. **Partition / time filter** — for any fact table with a known time field, missing `WHERE` on that field is `HIGH` (full scan risk).
-7. **GROUP BY completeness** — every selected non-aggregated column must be in `GROUP BY`. Mismatch is `HIGH`.
-8. **Null handling** — nullable join keys with `INNER JOIN` silently drop rows (`MEDIUM`); nullable measures aggregated without `COALESCE` can produce `NULL` that breaks downstream `WHERE ... > 0` (`MEDIUM`).
-9. **Type mismatch** — string columns used in numeric aggregation without cast (`MEDIUM`); date string compared to `TIMESTAMP` literal (`MEDIUM`).
-10. **Spark best practice** — `COUNT(*)` vs `COUNT(1)` is a style nit (`LOW`); `SELECT *` in production is `MEDIUM`; missing `DISTRIBUTE BY` / `CLUSTER BY` for skewed joins is `LOW` to `MEDIUM`.
+1. **引用完整性(Reference integrity)** — SQL 中的每个 `<alias>.<column>` 都必须存在于 SQL Context。不匹配即 `HIGH`
+2. **Join key 匹配** — 每个 `ON` 谓词都必须对得上 SQL Context 中已固化的 join。凭空发明的 join 算 `HIGH`
+3. **禁用 verb / 多语句** — 交给 `pyspark-sql-guardrails`。任何 DDL/DML/维护 verb,或任何 `;` 堆叠的语句,均为 `HIGH`
+4. **Join 基数风险** — 对每个 join,如果 SQL Context 标记基数为 `1:N` 或 `N:N` 且没有去重,分别抛 `MEDIUM`(1:N)或 `HIGH`(N:N)
+5. **重复聚合** — 同一度量被聚合两次(例如子查询里一次、外层再一次)且非有意重新聚合,抛 `HIGH`
+6. **分区 / 时间过滤** — 对任何有已知时间字段的事实表,缺少对该字段的 `WHERE` 过滤即为 `HIGH`(全表扫描风险)
+7. **GROUP BY 完整性** — SELECT 中每个未聚合的列都必须出现在 `GROUP BY` 中。不匹配为 `HIGH`
+8. **空值处理** — 可空 join key 上做 `INNER JOIN` 会静默丢行(`MEDIUM`);可空度量未用 `COALESCE` 聚合,可能产出 `NULL` 进而破坏下游 `WHERE ... > 0`(`MEDIUM`)
+9. **类型不匹配** — 字符串列未做 cast 就参与数值聚合(`MEDIUM`);日期字符串和 `TIMESTAMP` 字面量比较(`MEDIUM`)
+10. **Spark 最佳实践** — `COUNT(*)` vs `COUNT(1)` 属于风格(`LOW`);生产环境用 `SELECT *` 算 `MEDIUM`;数据倾斜 join 缺 `DISTRIBUTE BY` / `CLUSTER BY` 算 `LOW` 到 `MEDIUM`

-## Output Contract
+## 输出契约(Output Contract)

 ```yaml
 sql_review:
  verdict: <PASS | FAIL>
  highest_severity: <HIGH | MEDIUM | LOW>
-  summary: "<one-sentence summary>"
+  summary: "<一句话总结>"
  findings:
    - id: F1
      severity: HIGH
      check: reference_integrity
-      location: "<line/column or SQL fragment>"
-      message: "<what is wrong>"
-      fix: "<concrete suggested change>"
+      location: "<行/列号或 SQL 片段>"
+      message: "<错在哪>"
+      fix: "<具体的建议修改>"
    - id: F2
      severity: MEDIUM
      check: join_cardinality
@@ -104,21 +106,22 @@ sql_review:
    - "Reviewed by: sql-review v1"
 ```

-## Forbidden Behaviors
+## 禁止行为(Forbidden Behaviors)

-| Rationalization | Reality |
+| 自我说服 | 现实 |
 |---|---|
-| "The SQL looks fine, no need to cross-check the context" | A column can look fine and still not exist in the source table. Cross-check is mandatory. |
-| "I'll fix the SQL while reviewing" | The reviewer reports. The author fixes. Separation of concerns. |
-| "No SQL Context was provided, so I'll skip field checks" | Missing context is itself a `HIGH` finding. Never silently skip. |
-| "It's just a dev query, skip partition check" | The check is cheap. Skipping it produces silent full scans. |
-| "1:N joins are normal, no risk" | 1:N is fine *if* the grain is preserved. The reviewer must verify, not assume. |
-| "Regex is enough to check GROUP BY" | GROUP BY membership is structural. Use a parser, not regex. |
+| "SQL 看起来没问题,不用对一遍 context" | 字段可能"看起来对"但源表里根本没有,必须做交叉校验。 |
+| "我顺手在评审时把 SQL 修了" | 评审只报告,修是作者的事。职责分离。 |
+| "没给 SQL Context,那字段检查就跳过" | 缺 context 本身就是 `HIGH`,绝不能悄悄跳过。 |
+| "只是开发查询,分区检查跳过" | 这一项成本极低,跳过就会留下静默的全表扫描。 |
+| "1:N join 是常态,没风险" | 在保留粒度的前提下 1:N 是 OK 的,但评审必须验证,不能假设。 |
+| "用正则就够了,能查 GROUP BY" | GROUP BY 成员关系是结构性的,要用解析器,不要用正则。 |

-## Completion Criteria
+## 完成判定(Completion Criteria)

-A review is complete when:
- All 10 checks have been evaluated
- Every `HIGH` finding has a `fix`
- `verdict` is `PASS` iff `highest_severity` is `MEDIUM` or `LOW`
- The output YAML can be pasted into a PR comment without edits
+评审完成的条件:
+
+- 全部 10 项检查都已评估
+- 每一条 `HIGH` 风险都有 `fix`
+- 当且仅当 `highest_severity` 为 `MEDIUM` 或 `LOW` 时,`verdict` 才为 `PASS`
+- 输出的 YAML 可直接贴到 PR 评论,无需修改