Cost review
The job: where the money goes, per job, per pipeline and per table, and what to do about it. It proposes, it stops and deletes nothing.
Input: the period, the last month by default.
Steps
- Read the billing and usage records of the period: compute per job and pipeline, storage per table.
- The top 10 jobs by cost, and what each one produces.
- The cost per run of each scheduled pipeline, against the previous period.
- The storage bill: what the storage costs per month, and the part spent on tables nobody reads. The list of those tables comes from the big-review skill.
- Idle spend: clusters and warehouses that ran with nothing to do, and jobs that ran for nobody.
- The spikes: which day, which job, and which change caused it.
- Proposals, each with the saving per month and the risk, in order of the saving.
- Give the result as one web page.
Output
One HTML file: the cost of the period on top, the top jobs, the cost per run, the storage bill, the idle spend, the spikes, and the proposals last.
Reads
- the billing and usage records, the run records, the query logs, the catalog
Writes
- the web page only
Never
- Never deletes a table, never stops a job, never changes a cluster: it proposes.


