Model selection by speed, quality, and cost

Speed, quality, and cost are not three independent labels on a model. They interact across the full path from a brief to an accepted asset. A quick generation that requires four prompt rewrites may have a slower turnaround than a longer first run that passes review. A low nominal generation price can become expensive when failures are frequent. High visual detail may add no value if the output misses the required composition.

A useful comparison therefore measures a production loop rather than an interface event. Define the deliverable, run the same brief through a small shortlist, and record generation, review, correction, and rejection. The models directory provides current routes for a shortlist. The comparisons hub explains why the input and acceptance standard must remain fixed.

Decision criteria

Define quality as fitness for delivery. Break it into brief compliance, subject integrity, visual defects, style suitability, and editability. Define speed as elapsed time from starting a run to obtaining an approved result, including queueing, review, and necessary corrections. Define cost as total consumed value across accepted and rejected attempts, plus material human correction time when that time matters to the production. These definitions prevent a single attractive sample or fast progress indicator from deciding the result.

Set the priority order from the job. A same-day social draft may tolerate small defects in exchange for rapid iteration. A product image may require shape fidelity even when generation takes longer. A storyboard may prioritize volume and composition over polished texture. Assign weights before testing, and state any hard threshold, such as a maximum delivery time or a defect that always causes rejection. A model that violates a hard threshold cannot compensate with a high score elsewhere.

Inputs

Use one representative brief rather than an artificially easy benchmark. Preserve the exact prompt, references, aspect ratio, output count, and any exposed controls. Decide in advance how many attempts each candidate receives. A fixed attempt budget matters because unlimited retries favor high-variance systems. If settings do not map cleanly between models, record the mismatch and use the closest production-relevant configuration instead of pretending the comparison is identical.

Create a row for every attempt. Record start and completion time, review time, pass or fail, rejection category, and any correction needed. Avoid claiming precision the interface does not support; a simple elapsed-time measure is enough when applied consistently. Keep the raw outputs beside the sheet. Also record the date and account context because current model routes, access conditions, and commercial terms can change after the evaluation.

Failure modes

The first failure is timing generation alone. This excludes prompt preparation, queueing, download, inspection, and repair, even though those steps consume the schedule. The second is dividing price by generated files rather than usable files. Rejected outputs are part of the work. The third is allowing each candidate a different prompt refinement process. Tailoring can be a later optimization, but it obscures the baseline comparison.

Quality scoring also fails when reviewers use undefined impressions such as “better” or “more cinematic.” Replace those labels with observable requirements. Did the subject remain intact? Was the camera angle correct? Could the asset be cropped to delivery size? Finally, avoid extrapolating from one subject. A model that handles a landscape well may behave differently with typography, hands, products, or complex motion. Use a second stress brief when the decision carries material cost.

Limitations

No benchmark remains current indefinitely. Service load, model versions, routing, interfaces, and credit terms may change. Measurements from one account, region, or day should be presented as a dated observation rather than a universal speed or price claim. This guide deliberately provides no permanent ranking. It supplies a measurement structure that can be repeated when the production conditions change.

Human correction time is also difficult to compare across teams. An experienced editor may repair a defect quickly while another workflow cannot. Record the role and approximate intervention instead of converting every minute into a false universal price. Legal review, rights clearance, and brand approval sit outside the generation score but may still determine whether an output is usable. Include them in the schedule when the assignment requires them.

Next actions

Choose two or three candidates and write a weighted scorecard with one hard rejection rule. Run an equal number of attempts from the same source package. Measure the complete elapsed loop and classify each output before changing the prompt. Calculate the acceptance rate, median time to an accepted result, and consumed value per accepted result using the terms visible at the time of testing. Preserve the date with the record.

If one model wins on quality but misses the schedule, test whether a smaller output, simpler brief, or earlier preview step changes the decision. If a faster model produces more rejects, test one controlled prompt revision rather than immediately increasing the attempt budget. Move the selected procedure into a documented workflow with the input, settings, scorecard, and refresh trigger. The final choice should explain which constraint it serves and what evidence would cause the team to reconsider it.