TornadoVM has released version 7.3, transitioning its heterogeneous GPU programming framework to the Foreign Function & Memory (FFM) API while removing dependencies on the experimental JVM Compiler Interface. The update, detailed by InfoQ, allows Java developers to offload compute-heavy tasks to hardware backends, including Nvidia GPUs and Apple Metal, using a unified metadata path across JDK 21, 25, 26, and 27.
Architectural Shifts in TornadoVM 7.3
The framework has moved away from the Java Native Interface (JNI), adopting the FFM API to manage memory between the Java heap and accelerators. According to daily.dev, the project now sources all compiler metadata through a combined layer of reflection, ASM, and Unsafe operations. This change creates a standardized execution path for developers, regardless of the JDK distribution they use, such as OpenJDK, GraalVM, Temurin, or Zulu. By dropping the JVMCI dependency, the team aims to simplify the deployment of Java-based machine learning workloads that require high-performance hardware acceleration.
Deepened Nvidia Ecosystem Integration
Version 7.3 expands the framework’s capability to interact with Nvidia hardware. Developers can now utilize the cuTile programming model via a new TileContext API. The update also enables direct calls to high-performance CUDA libraries, specifically cuBLAS and CUTLASS, from within TornadoVM TaskGraphs. InfoQ notes that developers can choose to swap JIT-generated kernels for these library calls or chain them together to optimize performance. Similar hybrid API functionality has been introduced for Apple’s MLX library, and the framework now supports multi-GPU execution.
Deployment of jitLLM Inference Engine
Alongside the core framework update, the TornadoVM team launched jitLLM v2.0. This Java-based AI inference engine is designed to compete with tools like llama.cpp and Ollama. The engine is packaged as a single JAR file and is built specifically on the TornadoVM infrastructure. Both the core framework and the new inference engine are now being commercialized through Beestream.ai, a spin-off organization established by the project’s original developers.
Integration with Java Data Frameworks
TornadoVM maintains compatibility with broader Java ecosystems, including Spring AI, Quarkus, Flink, and Lang4j. In the case of Apache Flink, developers can use the Table API to define queries, with a configuration flag signaling the framework to check for available GPU resources at the task manager level. If hardware is detected, the framework analyzes the intermediate representation of the query to determine if it is suitable for GPU offloading, prioritizing data-parallel workloads like loops that lack inter-iteration dependencies.
Frequently Asked Questions
What is the primary benefit of the FFM API transition?
Replacing JNI with the Foreign Function & Memory API allows TornadoVM to provide a more stable and uniform path for compiler metadata across modern JDK versions. This change simplifies maintenance and ensures broader compatibility for developers using JDK 21 and later.
How does TornadoVM handle local versus server development?
The framework follows a "Write Once, Run Anywhere" model. Developers can write and test code locally on a laptop using available hardware, then deploy that same code to servers equipped with Nvidia GPUs to achieve performance scaling without rewriting the underlying logic.
What types of workloads are best suited for TornadoVM?
The framework is designed for data-parallel workloads. Tasks such as loops without dependencies between iterations are the most effective candidates for acceleration, as these can be distributed across multiple cores on a GPU.
Is jitLLM a standalone product?
Yes, jitLLM v2.0 is a Java AI inference engine that functions as a direct competitor to existing tools like llama.cpp. While it is built on top of TornadoVM, it is distributed as a single JAR file and managed through the commercial entity Beestream.ai.
Keep reading