SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 17 hours agoDo you host your own AI?message-squaremessage-square150linkfedilinkarrow-up1124file-text
arrow-up1124message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 17 hours agomessage-square150linkfedilinkfile-text
minus-squareDomi@lemmy.secnd.melinkfedilinkEnglisharrow-up2·1 hour agoAbout 200 t/s prompt processing and 10-20 t/s with MTP. Greatly depends on the task, predictable things like code generates at 18-20 t/s. Creative writing more like 10-17 t/s.
minus-squareSuspiciousCarrot78@aussie.zoneOPlinkfedilinkEnglisharrow-up1·28 minutes agoDamn - I thought strix would do a bit better than that, for how much it costs.
About 200 t/s prompt processing and 10-20 t/s with MTP.
Greatly depends on the task, predictable things like code generates at 18-20 t/s. Creative writing more like 10-17 t/s.
Damn - I thought strix would do a bit better than that, for how much it costs.