diff --git a/skills/logic-planner/SKILL.md b/skills/logic-planner/SKILL.md index 3ffe6ed..a1dcdaf 100644 --- a/skills/logic-planner/SKILL.md +++ b/skills/logic-planner/SKILL.md @@ -113,7 +113,7 @@ logic_plan: right: { alias: c, table: customer_info } keys: ["o.customer_id = c.customer_id"] type: left - cardinality_assumption: 1:N # 已知时填;否则写 "unknown" + cardinality_assumption: 1:N # if known; else state "unknown" - kind: transform alias: o expressions: diff --git a/skills/pyspark-sql-guardrails/SKILL.md b/skills/pyspark-sql-guardrails/SKILL.md index dafc8da..b5f8cc2 100644 --- a/skills/pyspark-sql-guardrails/SKILL.md +++ b/skills/pyspark-sql-guardrails/SKILL.md @@ -12,13 +12,13 @@ description: 在编写或评审 PySpark 脚本、Spark SQL 字符串、`spark.sq 整条流程就是两条命令 + 一个条件: ```bash -# 第 1 步 — 生成 SQL(脑中起草 / 写出来) -# 第 2 步 — 立即把 SQL 喂给校验器(在同一轮里跑) +# Step 1 — Generate the SQL (do this in your head / draft) +# Step 2 — Run the validator on it (do this IMMEDIATELY, in the same turn) echo "" | python3 skills/pyspark-sql-guardrails/scripts/validate_sql.py -# 第 3 步 — 读输出: -# PASS - first keyword: select, body length: N → 可以把 SQL 给用户看了 -# FAIL - → 停下,修 SQL,再跑一次 +# Step 3 — Read the output: +# PASS - first keyword: select, body length: N → you may now show the SQL +# FAIL - → STOP. Fix the SQL. Re-run. ``` **如果你还没把 `PASS -` 这一行贴到对话里,就不允许贴 SQL。** 校验器就是关口,SQL 站在关口的另一边,绝不能空着手闯关。 @@ -46,21 +46,21 @@ echo "" | python3 skills/pyspark-sql-guardrails/scripts/ 唯一可接受的调用形式: ```python -# 推荐:封装函数,校验+执行一步到位 +# Preferred: wrapper that validates + executes atomically safe_spark_sql(spark, sql) -# 或者:先校验,再显式执行 -assert_select_or_insert(sql) # 违规即抛错 +# Or: validate first, then execute explicitly +assert_select_or_insert(sql) # raises on violation spark.sql(sql) ``` **禁止的调用形式(出现即视为流水线失败):** ```python -spark.sql(sql) # 直接调用,未校验 -spark.sql(f"SELECT ... {user_input} ...") # f-string 直接进 spark.sql -spark.sql(config["sql"]) # 配置驱动的 SQL 未校验 -spark.sql(open("queries/xxx.sql").read()) # 从文件加载的 SQL 未校验 +spark.sql(sql) # direct, no validation +spark.sql(f"SELECT ... {user_input} ...") # f-string into spark.sql +spark.sql(config["sql"]) # config-driven without validation +spark.sql(open("queries/xxx.sql").read()) # file-loaded without validation ``` 如果 `assert_select_or_insert()` 抛错,流水线立即停止。**不要**削弱规则,**不要**用正则去剥除禁用关键字,**不要**把 SQL"改写"成看似安全的样子。要么拒绝,要么重新生成。 @@ -115,7 +115,7 @@ def assert_select_or_insert(sql: str) -> str: if not text: raise ValueError("SQL is empty") - # 允许一个可选的尾随分号,但拒绝多条堆叠的语句 + # Allow one optional trailing semicolon, but reject stacked statements. body = text[:-1].strip() if text.endswith(";") else text if ";" in body: raise ValueError("Multiple SQL statements are not allowed") @@ -138,13 +138,13 @@ def assert_select_or_insert(sql: str) -> str: def safe_spark_sql(spark, sql: str): return spark.sql(assert_select_or_insert(sql)) -# 正确 +# Good safe_spark_sql(spark, """ INSERT INTO analytics.daily_customer_snapshot SELECT * FROM staging.daily_customer_snapshot """) -# 错误:在 Spark 看到 SQL 之前就抛错 +# Bad: raises before Spark sees it safe_spark_sql(spark, "ALTER TABLE analytics.daily_customer_snapshot DROP PARTITION (dt='2026-06-10')") ``` @@ -158,11 +158,11 @@ safe_spark_sql(spark, "ALTER TABLE analytics.daily_customer_snapshot DROP PARTIT ``` skills/pyspark-sql-guardrails/ -├── SKILL.md ← 本文件 +├── SKILL.md ← this file └── scripts/ - ├── sql_guard.py ← 权威校验器(assert_select_or_insert + safe_spark_sql) - ├── validate_sql.py ← 一行 CLI:管道喂 SQL,输出 PASS/FAIL - └── test_sql_guard.py ← 18 用例的标准测试集 + ├── sql_guard.py ← canonical validator (assert_select_or_insert + safe_spark_sql) + ├── validate_sql.py ← one-line CLI: pipe SQL in, get PASS/FAIL out + └── test_sql_guard.py ← 18-case standard test set ``` ### 第 1 步 — 用内置校验器(不复制、不重写) @@ -172,7 +172,7 @@ skill 自带一个一行 CLI:`validate_sql.py`。用它。不要写不同的调 **两种把 SQL 喂给校验器的方法:** ```bash -# 方法 A(推荐,适合多行):走 stdin +# Way A (preferred for multi-line): pipe via stdin cat <<'EOF' | python3 skills/pyspark-sql-guardrails/scripts/validate_sql.py SELECT c.city AS city, @@ -183,7 +183,7 @@ WHERE o.status = 'SUCCESS' GROUP BY c.city EOF -# 方法 B(仅适合单行):作为第一个参数传入 +# Way B (single-line only): pass as first argument python3 skills/pyspark-sql-guardrails/scripts/validate_sql.py "SELECT 1 FROM dual" ``` @@ -195,7 +195,7 @@ python3 skills/pyspark-sql-guardrails/scripts/validate_sql.py "SELECT 1 FROM dua 内置的 `sql_guard.py` 和 `validate_sql.py` 一并展示: ```python -# sql_guard.py (内置) +# sql_guard.py (bundled) import re _FORBIDDEN_SQL = re.compile( @@ -227,7 +227,7 @@ def safe_spark_sql(spark, sql: str): return spark.sql(assert_select_or_insert(sql)) -# validate_sql.py (内置,精简版) +# validate_sql.py (bundled, abridged) import sys, os SKILL_DIR = os.path.dirname(os.path.abspath(__file__)) sys.path.insert(0, SKILL_DIR) @@ -265,10 +265,10 @@ PASS - first keyword: select, body length: 420 skill 自带 `test_sql_guard.py`,含 6 个 EXPECT_OK + 12 个 EXPECT_RAISE 用例。每次校验器改动,以及在 CI 中,都要在 skill 目录下跑一遍: ```bash -# 在项目根目录 +# From project root python3 skills/pyspark-sql-guardrails/scripts/test_sql_guard.py -# 或在 scripts/ 目录下 +# Or from the scripts/ directory cd skills/pyspark-sql-guardrails/scripts python3 test_sql_guard.py ``` diff --git a/skills/pyspark-sql-pipeline/SKILL.md b/skills/pyspark-sql-pipeline/SKILL.md index 6e23f0f..edb88f2 100644 --- a/skills/pyspark-sql-pipeline/SKILL.md +++ b/skills/pyspark-sql-pipeline/SKILL.md @@ -32,15 +32,15 @@ digraph pipeline { "2. metadata-validator" [shape=box]; "3. logic-planner" [shape=box]; "4. sql-context-builder" [shape=box]; - "5. SQL 生成" [shape=box]; + "5. SQL generation" [shape=box]; "6. pyspark-sql-guardrails" [shape=box]; "7. sql-review" [shape=box]; "1. requirements-analysis" -> "2. metadata-validator" [label="status=READY_FOR_VALIDATION"]; "2. metadata-validator" -> "3. logic-planner" [label="status=VALIDATED"]; "3. logic-planner" -> "4. sql-context-builder" [label="status=PLANNED"]; - "4. sql-context-builder" -> "5. SQL 生成" [label="status=READY"]; - "5. SQL 生成" -> "6. pyspark-sql-guardrails" [label="SQL 字符串"]; + "4. sql-context-builder" -> "5. SQL generation" [label="status=READY"]; + "5. SQL generation" -> "6. pyspark-sql-guardrails" [label="SQL 字符串"]; "6. pyspark-sql-guardrails" -> "7. sql-review" [label="guardrail pass"]; } ``` @@ -84,13 +84,13 @@ digraph pipeline { 每个关口应产出一条结构化消息,格式如下: ```markdown -## Pipeline paused at: <阶段名> +## Pipeline paused at: **Status:** NEED_USER_CONFIRMATION **Open questions:** -1. <问题 1> -2. <问题 2> +1. +2. **Pending options:** A / B / C / other -**To resume:** 回答以上问题后,说"继续"。 +**To resume:** answer the questions above, then say "继续" (continue). ``` 不要把某个阶段的输出当作最终结果,如果它的状态是停止状态。不要把 `pending_questions` 埋在冗长叙述里。