Reply to thread

Message: [QUOTE="Bombastus, post: 91449, member: 2178"] ACEMAGIC lists 192GB of unified memory for the F9A PRO 495, with up to 160GB allocated as graphics memory. The GPU allocation comes from the same memory pool, not an extra bank of VRAM. Running DeepSeek V4 Flash on [B][URL='https://goldmidi.com/community/threads/acemagic-will-bring-f9a-pro-495-mini-workstation-to-ifa.76842/']the 192GB F9A configuration[/URL][/B] needs a budget tied to a particular model file. A download that fits inside total RAM might still leave too little room for GPU execution, conversation history, or optional acceleration. [HEADING=2]Q4 saves less memory than its name suggests[/HEADING] Unsloth's DeepSeek-V4-Flash-0731-GGUF release lists UD-Q4_K_XL at about 155.1GB and UD-Q8_K_XL at 161.9GB. Moving from Q8 to Q4 saves 6.8GB, roughly 4%, not half. A surprisingly small reduction. These names describe mixed formats, not one precision applied to every parameter. Both keep the same compressed expert weights, so switching formats leaves most of the model untouched. The difference sits in attention and other nonexpert layers. Unsloth's Q8 build preserves those source values through BF16 storage, while its Q4 build uses Q8_0 there and introduces rounding. Unsloth calls the larger version lossless relative to the released weights. It doesn't restore precision discarded before release, or guarantee correct answers to every prompt. Unsloth reports 96.04% top-token agreement for Q4 against its reference, compared with 100% for Q8 in that test. This measures matching token choices, not the percentage of useful answers. Your code review could fail on one consequential change rather than lots of visibly bad sentences. Smaller builds need separate calculations. UD-IQ4_XS is listed at 136.7GB, although both it and the 155.1GB option carry four-bit labels. You cannot infer the savings, or the accuracy cost, from “four-bit” alone. [HEADING=2]The graphics allocation needs its own calculation[/HEADING] ACEMAGIC labels the maximum graphics allocation as 160GB without defining decimal versus binary units in its specification table. Your runtime may report capacity differently. The distinction changes the fit calculation. One gigabyte contains a billion bytes, while one gibibyte contains 1,073,741,824 bytes. Expressed in binary units, those 155.1GB and 161.9GB downloads occupy approximately 144.5GiB and 150.8GiB respectively. Suppose your runtime exposes 160GiB for GPU allocations. Subtracting the model files leaves approximately 15.6GiB for Q4 or 9.2GiB for Q8, before accounting for buffers and the context cache. Under a literal 160-billion-byte ceiling, Q4 leaves just 4.9GB, and Q8 exceeds the allowance by 1.9GB. Neither scenario establishes the F9A's actual usable capacity. They show why subtracting two differently labeled numbers can produce a bad answer. You need [B]room for the model and its working memory[/B], not just a successful download. The useful number is what your chosen runtime reports as available GPU memory, rather than the manufacturer's advertised maximum. These are weight-only screening calculations. They aren't measurements of a running F9A. Memory placement, other applications, and software allocations still matter, so a positive remainder isn't proof of a stable configuration. [HEADING=2]DSpark can use up the remaining space[/HEADING] The Flash-0731 GGUF repository includes an optional DSpark drafting module, with its Q8_0 file listed at 10.90GB. This module supports speculative decoding, which drafts candidate tokens for the main model to verify. It isn't part of the basic GGUF load unless enabled. Add that file to the 161.9GB Q8 model, and the combined weight files reach about 172.8GB, or 160.9GiB. Within a 160GiB GPU allowance, the pair already exceeds the budget before runtime overhead. These figures don't rule out other memory placements. Q4 plus the same drafter totals about 166GB, or 154.6GiB. Under the same allowance, only 5.4GiB remains before caches and buffers, considerably less than the 15.6GiB available without drafting. A less obvious problem affects automatic fitting. The DSpark instructions warn that llama.cpp can fail to measure the drafter's memory, skip its reservation, and still load it. A fitting calculation can therefore omit roughly 11GB without disabling the feature that needs it. For a first memory check, leaving DSpark disabled separates the main model's requirements from the drafter's extra allocation. Keeping the same model file and context setting when you enable drafting makes the additional demand easier to identify. For a code review, a greeting is a poor memory test. A useful workload includes your source files, the conversation already in progress, and enough output space for the reply. Reducing context gives you less room for those inputs and generated tokens. It may solve an allocation failure, but it also changes what the session can hold. Fewer files, or a shorter conversation. An 8,192-token trial doesn't validate a million-token session. Increasing the context allowance adds cache requirements, and longer inputs also require more prompt-processing work. An out-of-memory failure after that change can come from runtime allocations, even when the downloaded model files haven't changed. [/QUOTE]

Name