Gemma 3 27B vs Qwen3.8 Flash

Gemma 3 27B, announced on March 12, 2025, is the largest open-weight model in Google DeepMind’s Gemma 3 family. With around 27 billion parameters, it is multimodal—accepting both text and images as input and producing text outputs. It supports a 128,000-token context window and typically generates up to ~8,192 tokens, enabling it to process multi-page documents, extended conversations, or large batches of images in a single prompt.

The model is instruction-tuned in its “-it” variants for chat, reasoning, and summarization use cases, and it supports structured outputs and function calling. It is multilingual, covering over 140 languages. Deployment is flexible: the full BF16 model requires ~46 GB of VRAM, but quantization-aware training (QAT) versions in 8-bit or 4-bit reduce the footprint significantly, allowing more accessible use outside large-scale clusters. While it delivers stronger reasoning and multimodal performance than smaller Gemma models, it remains lighter and more open than proprietary systems, making it well-suited for research, development, and fine-tuned applications.

Qwen3.8-Flash is a multimodal mixture-of-experts model from the Qwen team at Alibaba, and the production counterpart of the open-weight Qwen3.8-Flash-Next preview that introduces the architecture intended for the Qwen4 family. The main model carries 125 billion parameters alongside a separate 51 billion parameter N-gram embedding table, while activating roughly 6 billion parameters per token. It accepts interleaved image and text input and returns text, handling 262,144 tokens of context natively with extension to 1,000,000 tokens using YaRN. The production configuration runs with the 1M context window by default and adds built-in tool support.

Four architectural changes separate it from earlier Qwen releases: hybrid attention that pairs Gated DeltaNet for history compression with Qwen Sparse Attention, which uses a lightweight indexer to select micro-blocks of context; a Gated Residual scheme; N-gram embeddings; and training with the Muon optimizer, refined around orthogonalization accuracy and the division of parameters between Muon and AdamW. Qwen reports training cost around one ninth that of Qwen3.7-Plus, with QSA attention kernels measured up to 7.6 times faster in prefill and 4.9 times faster in decode at 1M-token context. Reported scores include 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 84.5 on AndroidWorld and 95.7 on MathVision.