The hidden costs of translation accuracy

01 / MOTIVATION

The question

How much translation quality is traded for faster and less emissions-intensive inference when models are distilled or quantized?

02 / METHOD

The approach

NLLB-200 3.3B and distilled 600M models are benchmarked for English-to-French, Hindi, and Kannada translation, including FP16, INT8, and INT4 variants.

Evaluation combines Flores+ BLEU scores, inference latency, CodeCarbon estimates, and human accuracy and fluency ratings for translations of 100 conversational sentences.

03 / EVIDENCE

Findings & scope

The distilled FP32 model showed 71–78% lower latency and 63–65% lower estimated emissions, with a loss of 1.3–3.5 BLEU points in the reported comparisons.

INT4 translations retained mean human scores above 4.3/5 across all three languages. Emissions are estimates tied to the experimental setup; the findings describe these models, tasks, and measurement conditions.