Images Handling Specifications

Goals

Safely identify image properties and metadata without memory exhaustion.
Accurately extract text (OCR) and visual elements (Object Detection/Description).
Perform necessary pre-processing (resize, normalize, crop) for downstream tasks.
Handle multi-frame or high-resolution images efficiently.

Identify Properties: Use lightweight libraries (e.g., PIL/Pillow) to get format, size (width/height), and mode (RGB, RGBA, CMYK).
Check File Size: If the image is exceptionally large (e.g., >20MB or >100MP), consider downsampling or tiling before full processing.
Metadata/EXIF Extraction:
- Read EXIF data for orientation, GPS tags, and timestamps.
- Correction: Automatically apply EXIF orientation to ensure the image is "upright" before visual analysis.

Vision Analysis:
- Use multimodal vision models to describe scenes, identify objects, and detect activities.
- For complex images (e.g., infographics, UI screenshots), guide the model to focus on specific regions.
OCR (Optical Character Recognition):
- If text is detected, specify whether to extract "raw text" or "structured data" (like forms/tables).
- Handle low-contrast or noisy backgrounds by applying pre-filters (grayscale, binarization).
Format Conversion: Convert non-standard formats (e.g., HEIC, TIFF) to standard formats (JPEG/PNG) if tools require it.

Tiling: For ultra-high-res images (e.g., satellite maps, medical scans), split into overlapping tiles to avoid missing small details.
Batching: Process multiple images using generators to keep memory usage stable.
Alpha Channel: Be mindful of transparency (PNG/WebP); decide whether to discard it or composite against a solid background (e.g., white).

Safety First: Validate that the file is a genuine image (not a renamed malicious script).
Graceful Failure: Handle corrupted files, truncated downloads, or unsupported formats with descriptive error logs.
Efficiency: Avoid unnecessary re-encoding (e.g., multiple JPEG saves) to prevent "generation loss" or artifacts.
Process images individually or in small batches to prevent system crashes
Consider memory usage when working with large or high-resolution images
Resource Management: Close file pointers or use context managers (with Image.open(...) as img:) to prevent memory leaks.