Browsing: inference

The platform adds support for Meta’s sparse-architecture model, offering granular pricing and flexible deployment options for high-context workloads.

The platform introduces specialized pricing and deployment options for Meta’s latest large-scale model, emphasizing sparse activation and long-context capabilities.

The new FAI-ARA240-M module integrates discrete NPU acceleration with NXP i.MX 95 platforms to enable high-throughput, multi-modal inference at the industrial edge.

OpenAI reports significant performance gains from its custom Jalapeño inference chip, utilizing AI-assisted design to optimize hardware for agentic model workloads.

French startup Kog is attempting to maximize inference throughput on existing datacenter hardware through deep-level software optimization. The firm aims to bypass traditional bottlenecks by reverse-engineering GPU memory access patterns.

The new model architecture shifts the focus of enterprise AI deployment toward operational efficiency and high-volume cost optimization.

The new library enables native browser-based machine learning by compiling C++ inference kernels to WebAssembly and leveraging WebGPU for hardware acceleration.

OpenAI is moving to design its own specialized inference hardware to reduce reliance on third-party GPU suppliers. The project aims to optimize model performance through custom silicon architecture.

The ALCF is deploying a scalable, on-premises inference framework to provide DOE researchers with direct access to foundation models for high-performance computing.