Food Image Recognition API

Which endpoint for which photo

All four operations take one multipart/form-data field named image. The server sniffs the bytes, not the filename: JPEG, PNG or WebP, up to 8 MB, or the upload is rejected before any model runs. Photos are auto-rotated from EXIF and resized to 1024 px on the longest edge, so a full-resolution capture only costs upload time.

POST /v1/vision/barcode decodes EAN-13, EAN-8, UPC-A, UPC-E and Code-128 with plain computer vision and looks the code up exactly like GET /foods/barcode/{code}. The 200 body is { barcode, product } with the full catalog record (id, name_default, brand, nutrition_summary and the rest). No language model runs, so it costs 0 credits and is metered like a data call.

The other three are AI calls. /vision/product reads a front of pack into extracted { name, brand, quantity, lang, confidence } plus ranked matches. /vision/label transcribes a nutrition panel into per-100 g numbers, ingredients_text, allergens and serving. /vision/meal returns items[] of recognized dishes with portion estimates. Each response carries request_id and cached.

• A request without the image field is a 422 Validation Error with details[].field = "image".

Reading a /vision/product response

extracted.lang is the language read on the pack (bg for a Cyrillic label) and drives a language-aware catalog search. confidence runs 0 to 1 and gates that search: the catalog is only queried when confidence is at least 0.3 and a name was read; otherwise matches is empty.

matches holds up to 5 products, best first, each { score, product } in the same full FoodItem shape as a Food Database API record. score derives from confidence and rank, so the top match carries the confidence itself and each next entry scores lower. Keep matches[0].product.id; ids are stable across re-imports.

A miss looks like this staging response to a small product photo: {"matches":[],"extracted":{"lang":"","name":"","brand":"","quantity":"","confidence":0},"request_id":"bcca8231-f335-4468-81bf-e587b614d9a4","cached":true}. Empty strings and confidence 0 mean no name could be read; cached: true means the same bytes had been seen before, and the call was still charged 3 credits.

Labels and meals: read versus estimated

/vision/label is transcription, not inference. energy_kcal, fat_g, saturated_fat_g, carbs_g, sugars_g, fiber_g, proteins_g and salt_g are read as printed, and anything not on the label comes back null, never guessed. A readable barcode in the same photo is returned as extracted.barcode and looked up: a hit fills product with the full FoodItem, otherwise product is null.

/vision/meal is the opposite, and its disclaimer field says so. Each item has dish, portion_g_est and confidence, plus match when the dish resolved to a USDA generic food: id, name_default, per_100g { energy_kcal, proteins_g, carbs_g, fat_g } and estimated_kcal, which is per_100g.energy_kcal times portion_g_est over 100. estimated_total_kcal sums the items that have one and is null when none does. Matching uses English generic foods, so match.name_default is English whatever the photo.

• Show the disclaimer text to end users of meal estimates; a dish with match: null still costs the 5 credits.

Credits, refunds and the vision rate limit

Credits are deducted before inference: 3 for product, 4 for label, 5 for meal. The monthly allowance is shared by all keys of the account (Free 100, Starter 5,000, Pro 15,000, Enterprise 40,000) and resets on the first of the UTC month. AI responses carry X-Credits-Limit, X-Credits-Remaining and X-Credits-Reset; GET /v1/credits returns the balance for free.

Running out is a 402, not a 429: { "error": "Insufficient credits", "message": "This call costs 3 credits and your free plan's monthly allowance of 100 is exhausted. Credits reset on 2026-10-01." }. Refunds happen only when the request never reached an answer: rejected image (413 or 422), missing field (422), provider failure (502, message ends "You were not charged.") or vision disabled (503). Cache hits are charged full price on purpose.

AI calls have their own per-second limit, with no burst: 1/s on Free and Starter, 2/s on Pro, 5/s on Enterprise per key, and three times that across the account. Responses carry X-RateLimit-Vision-Limit and X-RateLimit-Vision-Remaining; exceeding it is a 429 with Retry-After: 1 and no charge, because the limiter runs before the deduction.

The scan flow and the photo

Cheapest first. Decode the barcode on-device and call the Barcode Lookup API, or let /vision/barcode decode it for 0 credits. Only when there is no barcode, or the lookup returns 404, send the front of the pack to /vision/product. With matches[0].product.id, fetch the per-100 g breakdown from the Nutrition API or a plain-language verdict from the AI Food Analysis API.

Photo quality decides confidence. For /vision/product frame the front of the pack with name, brand and pack size readable and nothing else in shot. For /vision/label lay the panel flat, fill the frame with the nutrition table and the ingredients paragraph, and include the barcode if it fits. For /vision/barcode the server's own 422 message is the rule: fill the frame with the barcode, avoid glare, keep it in focus.

• Retry a 502 after a short wait; you were refunded. Do not retry a low-confidence 200; retake instead.