Phase 5 Batch 4: analyzer 规则引擎 + 3 个高层 Tool
- internal/analyzer: 纯函数规则引擎
- Finding / Severity / Thresholds / Input / StageMetric / ExecutorMetric
- 3 规则: data_skew (max/min ratio), gc_pressure (GC/CPU ratio),
bottleneck (shuffle read+write GB)
- 严重度阶梯: warning / critical (阈值 ×2)
- Analyze 入口数据驱动 for ... range
- 11 个单测 (3 规则 × normal/warning/critical + 零除防御)
- fetch_spark_metrics: SHS 业务封装, format=summary 触发 analyzer
- fetch_cluster_env: RM /cluster/info + /cluster/metrics 聚合
- analyze_spark_log: RM 日志降级链 + LLM-Ready Prompt 拼装
- prompt 结构: cluster 元信息 + log source + log tail +
heuristic findings + suggested LLM analysis
- findings 是 first-pass filter, LLM 做最终判断
- deps.go: +AnalyzerThresholds
- main.go: 注入 cfg 的 3 个阈值
- 端到端实测: analyze_spark_log 完整 prompt 渲染
完成 11 个 Tool 表面 (list_clusters / spark_submit / 4 RM /
fetch_spark_metrics / fetch_cluster_env / analyze_spark_log /
fetch_url / upload_file)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,47 @@
|
||||
package analyzer
|
||||
|
||||
// Severity is the heuristic finding severity.
|
||||
type Severity string
|
||||
|
||||
const (
|
||||
SeverityInfo Severity = "info"
|
||||
SeverityWarning Severity = "warning"
|
||||
SeverityCritical Severity = "critical"
|
||||
)
|
||||
|
||||
// Finding is a single triggered heuristic rule.
|
||||
type Finding struct {
|
||||
Rule string `json:"rule"`
|
||||
Severity Severity `json:"severity"`
|
||||
Stage string `json:"stage,omitempty"`
|
||||
Evidence map[string]any `json:"evidence"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// Thresholds holds analyzer thresholds injected at runtime.
|
||||
type Thresholds struct {
|
||||
DataSkewRatio float64
|
||||
GCPressureRatio float64
|
||||
BottleneckShuffleGB float64
|
||||
}
|
||||
|
||||
// Input is the data passed to the heuristic rules.
|
||||
type Input struct {
|
||||
StageMetrics map[string]StageMetric `json:"stage_metrics"`
|
||||
ExecutorMetrics map[string]ExecutorMetric `json:"executor_metrics"`
|
||||
}
|
||||
|
||||
// StageMetric contains per-stage measurements.
|
||||
type StageMetric struct {
|
||||
DurationMS int64 `json:"duration_ms"`
|
||||
ShuffleReadBytes int64 `json:"shuffle_read_bytes"`
|
||||
ShuffleWriteBytes int64 `json:"shuffle_write_bytes"`
|
||||
MaxPartitionBytes int64 `json:"max_partition_bytes"`
|
||||
MinPartitionBytes int64 `json:"min_partition_bytes"`
|
||||
}
|
||||
|
||||
// ExecutorMetric contains per-executor measurements.
|
||||
type ExecutorMetric struct {
|
||||
GCTimeMS int64 `json:"gc_time_ms"`
|
||||
CPUTimeMS int64 `json:"cpu_time_ms"`
|
||||
}
|
||||
Reference in New Issue
Block a user