1.8 KiB
1.8 KiB
name, description
| name | description |
|---|---|
| pyspark-script-builder | Use after sql-review to wrap reviewed_sql into a complete PySpark script. Inputs are reviewed_sql, app_name, and script_name; ask the user for app_name or script_name if missing; preserve reviewed_sql business logic and output app_name, script_name, and pyspark_code only. |
PySpark Script Builder
Use this skill only after sql-review, when reviewed_sql is already available and the user needs it wrapped as a complete PySpark script.
Required Inputs
reviewed_sql: SQL text that has already been reviewed.app_name: Spark application name. If missing, ask the user before producing output. Do not invent or derive it.script_name: Script file name. If missing, ask the user before producing output. Do not invent or derive it.
Rules
- Preserve the business logic of
reviewed_sql. Do not rewrite tables, columns, predicates, joins, aggregations, ordering, limits, comments, or SQL semantics. - Only wrap
reviewed_sqlin the fixed PySpark script structure below. - Do not add file persistence, job creation, confirmation, cluster launch, queue/resource selection, monitoring, logs, or lifecycle steps.
- Escape
app_nameonly as needed to keep a valid Python double-quoted string. - Put
reviewed_sqlinside a raw triple-quoted string exactly in thesql = r"""..."""assignment. - Do not run the SQL or validate it again; this skill only builds script text.
Output
Return only these fields:
app_namescript_namepyspark_code
Build pyspark_code with this exact structure:
from pyspark.sql import SparkSession
APP_NAME = "<app_name>"
def main():
spark = SparkSession.builder.appName(APP_NAME).enableHiveSupport().getOrCreate()
sql = r"""<reviewed_sql>"""
spark.sql(sql)
spark.stop()
if __name__ == "__main__": main()