Files
opencode-build/skills/pyspark-script-builder/SKILL.md
T

1.8 KiB

name, description
name description
pyspark-script-builder Use after sql-review to wrap reviewed_sql into a complete PySpark script. Inputs are reviewed_sql, app_name, and script_name; ask the user for app_name or script_name if missing; preserve reviewed_sql business logic and output app_name, script_name, and pyspark_code only.

PySpark Script Builder

Use this skill only after sql-review, when reviewed_sql is already available and the user needs it wrapped as a complete PySpark script.

Required Inputs

  • reviewed_sql: SQL text that has already been reviewed.
  • app_name: Spark application name. If missing, ask the user before producing output. Do not invent or derive it.
  • script_name: Script file name. If missing, ask the user before producing output. Do not invent or derive it.

Rules

  • Preserve the business logic of reviewed_sql. Do not rewrite tables, columns, predicates, joins, aggregations, ordering, limits, comments, or SQL semantics.
  • Only wrap reviewed_sql in the fixed PySpark script structure below.
  • Do not add file persistence, job creation, confirmation, cluster launch, queue/resource selection, monitoring, logs, or lifecycle steps.
  • Escape app_name only as needed to keep a valid Python double-quoted string.
  • Put reviewed_sql inside a raw triple-quoted string exactly in the sql = r"""...""" assignment.
  • Do not run the SQL or validate it again; this skill only builds script text.

Output

Return only these fields:

  • app_name
  • script_name
  • pyspark_code

Build pyspark_code with this exact structure:

from pyspark.sql import SparkSession

APP_NAME = "<app_name>"

def main():
    spark = SparkSession.builder.appName(APP_NAME).enableHiveSupport().getOrCreate()
    sql = r"""<reviewed_sql>"""
    spark.sql(sql)
    spark.stop()

if __name__ == "__main__": main()