AXAMICROTECH R&D / CONTROLLED BETA
StreamAI
Making Local AI Possible on Memory-Constrained Hardware.
StreamAI is an experimental local AI inference platform developed by Axamicrotech LLC to investigate efficient execution of large language models on consumer hardware with limited GPU memory.
Its architecture explores quantized inference, GPU layer streaming, adaptive model residency, and memory-aware execution strategies. Technical benchmarking, optimization, and broader hardware validation are in progress.
01 / THE CHALLENGE
Local intelligence. Finite memory.
Large language models can require more memory than a consumer GPU provides. Higher-memory GPUs can be expensive, while hardware compatibility and efficient allocation of CPU RAM and GPU VRAM complicate deployment. Existing inference systems already support CPU/GPU offloading; StreamAI investigates execution strategies within this established problem space.
02 / THE APPROACH
Execution shaped around memory.
The architecture maintains model weights in system memory or local packed caches, keeps selected decoder layers resident on the GPU, and transfers other layers as needed. Quantized representations and GPU memory-aware execution aim to reduce hardware barriers.
Demonstrated on the reference machine
DirectML execution, persistent managed model caching, automatic hardware detection and qualification, and first-run Fast model provisioning form part of the reported reference-machine implementation. Adaptive residency and transfer optimizations remain research objectives; broader backend support is not established.
03 / TECHNICAL EVIDENCE
Reference observations, openly stated.
Owner-reported testing of StreamAI 0.3.9.15: Windows, AMD Radeon RX 570, 4 GB VRAM, approximately 32 GB system RAM, DirectML execution. These are supplied reference-machine observations, not independently reproduced benchmarks.
| Observation | Fast mode | Quality mode |
|---|---|---|
| Model | Qwen2.5 1.5B | Qwen1.5 7B |
| Runtime | INT8 hybrid runtime | Managed quantized cache |
| Milestone | Standalone installation, automatic first-run provisioning, successful inference | Successful engine initialization and inference |
| Observed throughput | Approximately 1.17 tokens/second, pristine first-run test | Approximately 0.365 tokens/second |
| Time to first token | Not supplied | Approximately 72.1 seconds in recorded Quality test |
Results reflect testing on a single reference hardware configuration. Broader hardware compatibility, performance competitiveness, and commercial suitability have not yet been independently established.
No controlled comparative evidence is presented against llama.cpp, Ollama, LM Studio, or other inference systems.
04 / RESEARCH ROADMAP
The next questions to investigate.
Planned research objectives, rather than completed features:
- Competitive benchmarking against established inference engines
- Adaptive GPU layer residency
- Asynchronous weight prefetching
- CPU/GPU transfer optimization
- KV-cache management
- Additional quantization approaches
- Broader hardware/backend compatibility
- Automated performance qualification
- Stability and regression testing
05 / COLLABORATION
Help test what comes next.
We welcome academic research collaboration, GPU hardware evaluation, memory-constrained inference research, independent benchmarking, systems optimization, and AI infrastructure discussions. Startup incubation, commercialization, grant-funded research opportunities, and strategic investment discussions are also welcome.
06 / STATUS & OWNERSHIP
Controlled Beta. Active R&D.
Fast and Quality inference have been demonstrated on the Windows / AMD Radeon RX 570 / DirectML reference platform. External-machine validation remains outstanding; performance optimization is in progress. Public distribution availability has not been established. Research collaboration inquiries are welcome.
StreamAI is a research-and-development initiative of Axamicrotech LLC. Third-party models and underlying open-source technologies remain subject to their respective ownership and licenses.
← Products & Research