Reply to thread

Message

[QUOTE="Queen, post: 89990, member: 27"]
GenStorAIGE basically told GPU memory limits to kick rocks and strapped PCIe Gen5 storage into the inference pipeline as a third memory tier. The AI90 platform, shown at WAIC 2026, stacks HBM, DRAM, and solid-state storage, offloading LLM cache entries onto those drives. First-token latency allegedly hits sub-second, throughput jumps 5.1x, and GPU memory usage drops roughly 39 percent.

On an eight-card RTX 5090 rig, the company pegs inference acceleration at 5.8x with context windows stretching past 128,000 tokens. The companion PT200Z drive handles the write-heavy cache grind using pSLC NAND over PCIe Gen5, hitting 14.8 GB/s sequential reads, 3.1 million random IOPS, and 10-microsecond write latency. Endurance sits at 100 DWPD for sustained enterprise punishment.

And yeah, these are all vendor-supplied numbers. Independent benchmarks will tell the real story. But the memory-tiering concept tracks with where AI infrastructure is heading when HBM alone can't keep up with ballooning model sizes.
[/QUOTE]

Name