Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

    • brucethemoose@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      ·
      edit-2
      18 hours ago

      Not anymore. Not with hybrid offloading, where the GPU handles dense tensors and the CPU only runs the sparse MoEs. I’m running a 300B model on a single 3090, and its faster than I can read.

      You just need to use the right framework, and the right model.

      I’d suggest trying ik_llama.cpp and a MoE like one of these: https://huggingface.co/models?other=ik_llama.cpp&sort=modified&search=35B

      And speculative decoding like DFlash or MTP (which you can also get specific models for).

      EDIT: Wrong link.

      • atzanteol@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        3
        ·
        17 hours ago

        I’ll check that out - speed isn’t my biggest issue so much as coding performance… The qwen 3.5 model I was using can write code, but it’s… Meh? Like sometimes it doesn’t even compile.

        I did try tweaking llama.cpp to do some cpu offloading and it does seem to allow for much larger contexts at a modest performance loss. I’ll check out larger models.

        • brucethemoose@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          16 hours ago

          CPU offloading is too slow unless you use a hybrid MoE model, with the --n-cpu-moe parameter, specifically.

          This only offloads “sparse” parts of the model to the CPU, which take up a lot of RAM but are very compute-lite to run. In practice, thats most of the size of modern MoE LLMs.

          • robber@lemmy.ml
            link
            fedilink
            English
            arrow-up
            1
            ·
            5 hours ago

            Since implementation of the --fit parameter and its relatives, and --fit on becoming the default, llama.cpp intelligently decides what to offload. For me, it made --n-cpu-moe obsolete.

            • brucethemoose@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              4 hours ago

              Mostly, yeah.

              Sometimes it’s better to “cut it close,” with (for instance) a 27B model that’s nearly OOMing your VRAM fully offloaded, but you know will be fine in regular use without too many programs open.

              In my case, with MiMo 2.5, it fills both my CPU and GPU RAM rather completely, so it’s best to set a static value so I don’t swap CPU RAM, and don’t OOM on the GPU either.