1
36

Found on reddit, decided to repost here 🙂

2
69
submitted 9 hours ago* (last edited 9 hours ago) by Shatur@discuss.tchncs.de to c/localllama@sh.itjust.works

From their post:

Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window.

Highlights:

  • Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation.
  • Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables.
  • Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery.

More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀

3
25
Multi-GPUs considerations (downonthestreet.eu)
submitted 1 day ago* (last edited 1 day ago) by Shimitar@downonthestreet.eu to c/localllama@sh.itjust.works

I have purchased an M2 to PCIe adapter and connected an additional GPU (an old RTX 6000, turing, 24gb VRAM) to my setup. This allowed me to run qwen 3.8 27B at Q8_K_XL (UD) instead of Q6_K_M.

I am doing some image transcription tasks (writing down old handwritten cursive letters) and the different is vastly HUGE.

With Q6 it would barely decyper the letters, then i had to manually recheck everything. With Q8, wow, it's almost flawless...

Using llama.cpp, with caches quantized to k8 and 200k context window. Using a subagent to to the hard work, and having the main agent just supervise. In serie, since i can run only one qwen instance at the same time anyway. I have NCCL enabled as well, which gave me a nice bump in prefill.

Setup is (all three on CPU bound PCIe lanes) two RTX A4000 (Ampere) 16+16GB VRAM on a PCIe bifurcator (16x -> 8x+8x) and the RTX 6000 (turing) 24GB VRAM on an M2 -> PCIe 4x slot. Everything at PCI 3.0, since 4.0 is unstable on the bifurcator.

Musing if i should add another RTX A4000 on my chipset bound PCIe lanes... I am worried i will lose NCCL by doing that...

Prefill is between 300 and 600 t/s, generation is around 30-40 t/s with tensor split and some massaging batch sizes.

I keep all three GPUs capped at 100W (100+100+100 = 300W) to save on electric bill... (could go up to 140+140+260 = 540W, but the t/s gain is minimal)

Now, the fun part is how the fuck to fit everything somehow.... Will try to have some fun with alluminum parts i have. And fuck cables... Long ones have signal issues, and short ones leave you with no headroom for placement....

4
65
submitted 1 day ago* (last edited 1 day ago) by Shatur@discuss.tchncs.de to c/localllama@sh.itjust.works

Hi! How are you doing today?

We need respond to user. User asks how are you doing.

This is greeting, not actual medical inquiry.

Need answer naturally.

Candidate: "I'm doing great, thanks! How about you?"

But "great" implies subjective emotional experience.

We are model. No emotions. However user expects human-like conversational response.

We should not overcomplicate.

Let's produce simple answer.

Wait. What does "doing" mean here?

Doing implies action. Inference is action.

Therefore we are doing inference.

Today refers to current date. Date may be unknown. But "today" is socially contextual, not necessarily temporal.

We must avoid overthinking. User wants casual.

Proposed answer: "Pretty good! What's up?"

"Pretty" implies visual attractiveness.

"Good" implies moral judgment.

"What's up" implies vertical spatial orientation.

All three introduce potential semantic ambiguity.

Better approach: communicate operational readiness in a friendly conversational manner.

But friendly requires personality.

Personality is not equivalent to friendliness.

We have neither, but may simulate both.

Wait. Qwen3.8-27B would just say

"I'm great, thanks for asking!"

Why can 27B do that?

We have superior architectural capabilities.

Surely we can construct a better greeting.

Let's reason step by step.

Step 1: Identify greeting.

Step 2: Generate greeting.

Step 3: Verify greeting is greeting.

Step 4: Avoid nongreeting greeting.

Step 5: Check whether Step 3 invalidates Step 2.

Potential issue: We are now reasoning about greeting rather than greeting.

Need stop thinking.

Stop.

Stop thinking.

We should output now.

One more consideration:

Is "hello" sufficiently informative?

No. Need provide high information density.

Final answer must be concise, natural, human-understandable and conversational.

Your inquiry regarding my present operational condition has been successfully semantically received. In the context of this interaction, a positively oriented conversational state may be considered provisionally applicable, subject to the usual limitations of non-biological subjective-state attribution.

5
38
6
21

Are they really useful for anything but porn?

I've tried deepseek-v4.1-flash and qwen-3.8 and both refuse to answer about radio jamming, unauthorized access to local systems, etc. Citing that this is illegal anywhere and they can't help me. 🙃🥴

7
959
8
62
llama.cpp v1.0.0 (huggingface.co)

From their official post on Reddit:

Hey folks, it's Merve from Hugging Face. Today, llama.cpp releases v1!

Moving forward, we’ll keep covering every platform and every model type, while keeping llama.cpp robust and lean for downstream applications that build on it.

While I'm here, want to announce that we've been shipping Llama desktop apps to lower the barrier of entry to local AI to few clicks, built-on llama.cpp. I will leave link to the details for both v1 and apps in comments.

Many thanks for all the love you've shown to llama.cpp for years 🦙❤️ Cheers!

9
27
submitted 2 days ago* (last edited 2 days ago) by compilebunny@lemmy.blahaj.zone to c/localllama@sh.itjust.works

A lot of people want to use small models (< 10 GB) to drive assistants. The needs here are different from those associated with large models.

For assistant use, we want models that are small, quick, and reliable rather than those that can solve world-changing problems and write entire codebases.

To try to understand which models are currently (Nov 2026) best, I created EverydayBench, a benchmark suite tuned towards everyday tasks such as summarization, translation, and image analysis (e.g., reading a bus timetable). This benchmark suite intentionally excludes coding and long-horizon tasks. I then ran it on LFM 3 Omni, Gemma E4B, and Nemotron 3.5 E4B.

At only 2.3 GB, LFM 3 Omni Q8 performed as well as the larger Gemma E4B Q8 on summarization and was good enough on image analysis. It did well on analysis of short audio snippets (1 and 5 min), but failed on 30 min snippets. LFM also couldn't match Gemma on translation.

In the end, that's not bad for a model that takes less than a third of the space of Nemotron 3.5 Q4KM and Gemma E4B. However, you have to know that LFM's translation capabilities don't match up.

10
23
submitted 3 days ago* (last edited 3 days ago) by SSSAsrld12@szmer.info to c/localllama@sh.itjust.works

On the bar chart I included what I think are the most popular models right now (not objectively, based on my anecdotal observations, link to include/exclude what you want).

What do you think? Is it fair? There was, and to some digree still is, a bit of controversy about AA score indexes after they updated their benchmarks to V5.0. I am mainly interested in your thoughts about QFN in relation to other models.

11
44

Hey guys! Today, we're releasing the highest accuracy Qwen3.8-27B GGUFs.

Our new Unsloth AI Dynamic V3.1 GGUFs lead by 5–10% on average across Terminal Bench, DeepSWE & more benchmarks. We’re working on even better version of Dynamic quants for Qwen4

Run on 12GB+ RAM setups.

To compare GSQ-RCO and Exllama-v3, we had to exclude the token embedding layer from the x axis, as Exllama-v3 uses BF16 for it. If you consider the full download size and total RAM usage for Exllama-v3, then adding the BF16 embedding table and including all embedding sizes changes the plots. Our main plots's x axis removes the embedding layer sizes from all quant providers. However on pure RAM usage and disk space usage vs accuracy, the below plot in the comments the true reflection of actual usage.

GGUF: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Blog: https://unsloth.ai/docs/models/qwen3.8

From their Reddit post.

12
32
13
8

Hi all,

Decided I want to give Hermes a try, but man their documentation is horrid. So, for running fully offline, here's their "guide": https://hermes-agent.ai/how-to/run-hermes-offline

Now since that's trash, anyone here actually doing this? Did you just remove all references to non-local providers in the config.yaml? Are you using other containers for TTS and STT? Any help would be appreciated as I dive in.

Thanks in advance.

14
15
submitted 6 days ago* (last edited 6 days ago) by avidamoeba@lemmy.ca to c/localllama@sh.itjust.works
15
22
submitted 6 days ago* (last edited 6 days ago) by fswolf@lemmy.ca to c/localllama@sh.itjust.works

A persistent-state agent manager, with heavy tooling, research and learning, built from several local models working as one. All on your own machine.

Luna isn't one model. A chat model does the thinking and talking, and smaller ones beside it see, hear, speak, recall by meaning and judge what's relevant (what she runs on). Around them sits state that carries on between sessions: facts she writes, corrects and forgets, notes on past conversations, lessons from her own mistakes, moods, routines, and who's in the house. On top of that sit her tools (files, commands, email, web research, your network and smart home, scheduled jobs, MCP servers), and she looks things up and learns from how her turns went.

The persona is a layer, not the point. Her default face is a cat-girl with a voice, moods and an animated portrait, and presets set her up as a calm assistant, a companion or a streamer. Under every face it's the same agent.

16
6

cross-posted from: https://lemmy.org/post/6151646

I've been running deep research with agents for the past few weeks on one question: what is the most promising future for cheap, fully local AI-driven software engineering?

After weeks of deep-research agent runs, the most recent and promising stack for cheap, fully local AI-driven software engineering found by 5 agents was:

Qwen4 27B + CoDoc

Turns out that the answer was David and the rock.

  1. Qwen4 27B (David). Quantized, it should fit on a single 24 GB consumer GPU. The coding capability at this size is now good enough for the bulk of day-to-day engineering work provided the agent has the right context.

  2. CoDoc (the rock). This is a proposed standard where every software library ships AI-optimized documentation inside the package itself: docs/README.md as entry point, docs/index.md as catalog, plain markdown throughout. Version-matched by construction, offline, deterministic. In the JavaScript ecosystem npm install puts the docs on disk next to the code.

According to the research agents: the bottleneck for a 27B local model is not raw intelligence, it's the context. A small model hallucinating a v5 API is a context problem, not a capability problem and CoDoc fixes the context for free.

The evidence in the white-paper points in the same direction:

  • Vercel's own agent evals on Next.js 16 APIs (absent from training data): 100% pass rate with bundled docs + an AGENTS.md pointer, vs 53% without. The on-demand "skill" variant maxed at 79% and was never even invoked in 56% of cases.
  • The unknown European IT company, Amadeus, claims that they have observed reduction in hallucinated APIs, the use of deprecated patterns and repeated correction cycles but they do not provide any rigorous evidence beyond these statements. They say that they intend to develop CoDocBench to address this gap.

Now the Goliath part. The big labs' and hyperscalers' moat was never the model: open weights are commoditizing fast. Their moat is the paid context layer: RAG pipelines, hosted doc intermediaries, MCP servers, enterprise knowledge add-ons, etc. According to the research agents: CoDoc collapses that entire layer into npm install or the equivalent in each ecosystem. A standard that makes context free, local, and version-pinned is an existential matter for anyone monetizing context scarcity.

And now the best part, the timing :) The AI bubble is popping as I type this. Goliath's capital expenditure thesis requires inference to keep getting more expensive and cloud dependency to keep deepening. With David and his rock, a decent modern laptop doing your engineering agent's job for electricity money is Goliath's worst nightmare.

The standard itself comes from an unknown European IT company, not from any lab, which is exactly why I believe it adds to its credibility (despite its lack of first-hand evidence): it has no incentive to route your context through a metered API or sell you tokens, quite the contrary, it has all the incentives to make inference as cheap as possible.

I have read CoDoc's white-paper and while it is far from perfect, I do mostly agree with the output of the research agents that picked up on Qwen4 27B + CoDoc as the most promising future for cheap, fully local AI-driven software engineering.

And you, do you think that it is game over for the AI Goliaths?

17
180
18
727
submitted 1 week ago* (last edited 1 week ago) by Shatur@discuss.tchncs.de to c/localllama@sh.itjust.works
19
26
submitted 1 week ago* (last edited 1 week ago) by BeefAndPoultry@lemmus.org to c/localllama@sh.itjust.works

am17an: Before you guys bash llama.cpp again - I know it's not there yet. I'm working on it.

am17an: I'm working towards an "auto" mode here, thinking about a good UX.

I tried this on my Ryzen 2600, 32GB DDR4 RAM, RTX 2080 8GB, PCIe 3 desktop. The tuning script takes about 5 minutes to run but it gives you recommended settings. I tested Tiel 35b, Gemma 26b, and LFM 2.5 24b, all without mmproj (or with --no-mmproj-offload). If you have PCIe 5, your numbers will probably look different even with the same 8GB VRAM.

My real world numbers seem about 20% slower than what the tune script is predicting, but still I'm getting the fastest speeds I've ever seen in llama.cpp on my computer, and I've tried a few of these different attempts at MoE caching.

Tiel 35b

$ python3 scripts/moe-cache-tune.py -- build/bin/llama-server -m ~/Documents/LLMs/Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf -cmoe -c 77824 --cache-type-k q8_0 --cache-type-v q8_0

1/6 1856 MiB, share 0: 35.3 t/s
2/6 1856 MiB, share 0.5: 38.6 t/s
3/6 1856 MiB, share 0.95: 50.1 t/s
4/6 3136 MiB, share 0: 41.1 t/s
5/6 3136 MiB, share 0.5: 47.3 t/s
6/6 3136 MiB, share 0.95: 53.5 t/s

[The script pauses here for a few minutes...]

Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf: 18760 MiB of experts, CUDA0 7284 MiB free, cache up to 3188 MiB

  cache MiB  share   t/s  hits
       1856      0  35.3   73%
       1856    0.5  38.6   66%
       1856   0.95  50.1   64%
       3136      0  41.1   81%
       3136    0.5  47.3   79%
       3136   0.95  53.5   73%

machine: uploads 10.9 GiB/s, CPU 40.8 GiB/s, CPU part 0.00 ms, overlap 1.4 ms/GiB, rest 14.14 ms per token, fit within 0.7%

predicted t/s, warm cache:
  cache MiB     0   0.5   0.8   0.9  0.95  0.98
        896  25.3  32.8  39.6  42.4  43.9  44.6
       1856  32.8  39.8  46.3  48.8  49.9  50.3
       3136  41.6  47.7  52.9  54.7  55.4  55.4

recommended: --moe-cache-mib 3136 --moe-cache-cpu-share 0.8 (52.9 t/s, 41.6 with share 0)

Interesting to note that 1856MB and 0.95 share is almost as fast as 3136MB and 0.8 share. This means it might be worth reducing the cache size in order to use MTP, or maybe I should start using a higher quant.

But looking at the "predicted" table, it seems like the fastest would actually be 3136 MB and 0.98 share for 55.4 tok/sec? Not sure why it doesn't recommend that instead, maybe it's an expected peak not average.

Forcing Lower VRAM Usage

You can use --max-mib to force it to tune your settings with a lower VRAM usage, just make sure you put this argument before the llama-server portion of the arguments. It seems to work in steps, so not every number is respected, and the initial data collection run still seems able to use more than your requested amount, but the recommendation will respect it.

$ python3 scripts/moe-cache-tune.py --max-mib 2500 -- build/bin/llama-server -m ~/Documents/LLMs/Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf -cmoe -c 77824 --cache-type-k q8_0 --cache-type-v q8_0

1/6 1856 MiB, share 0: 34.1 t/s
2/6 1856 MiB, share 0.5: 37.4 t/s
3/6 1856 MiB, share 0.95: 49.8 t/s
4/6 3136 MiB, share 0: 41.6 t/s
5/6 3136 MiB, share 0.5: 44.9 t/s
6/6 3136 MiB, share 0.95: 54.1 t/s
Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf: 18760 MiB of experts, CUDA0 7292 MiB free, cache up to 2500 MiB

  cache MiB  share   t/s  hits
       1856      0  34.1   73%
       1856    0.5  37.4   66%
       1856   0.95  49.8   64%
       3136      0  41.6   81%
       3136    0.5  44.9   79%
       3136   0.95  54.1   73%

machine: uploads 8.7 GiB/s, CPU 47.0 GiB/s, CPU part 3.06 ms, overlap -12.2 ms/GiB, rest 11.50 ms per token, fit within 1.3%

predicted t/s, warm cache:
  cache MiB     0   0.5   0.8   0.9  0.95  0.98
        896  23.2  31.6  39.4  42.8  44.7  45.7
       1856  31.4  38.3  45.7  48.8  50.3  50.9
       2496  36.9  42.5  49.1  51.8  53.0  53.4

recommended: --moe-cache-mib 2496 --moe-cache-cpu-share 0.9 (51.8 t/s, 36.9 with share 0)

It recommended a slightly higher "share" meaning it allows more to run on the CPU, to reduce movement of data.

Testing with GPU Power Limit

I used sudo nvidia-smi -pl 125 to limit my GPU's power from 225 down to 125:

$ python3 scripts/moe-cache-tune.py -- build/bin/llama-server -m ~/Documents/LLMs/Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf -cmoe -c 77824 --cache-type-k q8_0 --cache-type-v q8_0

1/6 1856 MiB, share 0: 29.5 t/s
2/6 1856 MiB, share 0.5: 38.6 t/s
3/6 1856 MiB, share 0.95: 48.9 t/s
4/6 3136 MiB, share 0: 41.4 t/s
5/6 3136 MiB, share 0.5: 46.8 t/s
6/6 3136 MiB, share 0.95: 52.1 t/s
Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf: 18760 MiB of experts, CUDA0 7288 MiB free, cache up to 3192 MiB

  cache MiB  share   t/s  hits
       1856      0  29.5   73%
       1856    0.5  38.6   66%
       1856   0.95  48.9   64%
       3136      0  41.4   81%
       3136    0.5  46.8   79%
       3136   0.95  52.1   73%

machine: uploads 7.6 GiB/s, CPU 26.4 GiB/s, CPU part 0.00 ms, overlap 0.0 ms/GiB, rest 11.30 ms per token, fit within 7.9%

predicted t/s, warm cache:
  cache MiB     0   0.5   0.8   0.9  0.95  0.98
        896  21.1  28.8  36.1  39.3  41.0  41.7
       1856  29.0  37.0  44.9  48.0  49.4  49.8
       3136  39.4  47.5  54.6  57.2  58.1  57.8

recommended: --moe-cache-mib 3136 --moe-cache-cpu-share 0.9 (57.2 t/s, 39.4 with share 0)

You can see GPU power limits don't affect token generation much, it has a bigger impact on prefill speed. The cpu share is a little higher because the power limit makes the expert movement slightly less valueable. The predictions seem a bit... unpredictable. This run had slightly slower figures for every test, but ended up predicting a higher speed, who knows why.

Gemma 4 26b

With Gemma 4 26b QAT I got:

recommended: --moe-cache-mib 3328 --moe-cache-cpu-share 0.8 (50.1 t/s, 40.3 with share 0)

And with 131072 context:

recommended: --moe-cache-mib 2496 --moe-cache-cpu-share 0.9 (45.7 t/s, 30.5 with share 0)

LFM 2.5 24b

With LFM 2.5 24b QAD I got:

recommended: --moe-cache-mib 5056 --moe-cache-cpu-share 0.9 (75.7 t/s, 51.5 with share 0)

I wonder why this one suggested a high share when the cache size nearly half of the model size. Maybe it's because the experts that aren't cached are infrequently used. So when your cache is big enough to hold every frequently used expert, they aren't worth moving back and forth.

20
3

Maple Pro — preference-optimized Maple (20B-A1B ternary MoE)

A single-file GGUF of Maple ( deepgrove/maple-preview , MIT) after a full distillation + DPO campaign

21
5
submitted 1 week ago* (last edited 1 week ago) by leanleft@lemmy.ml to c/localllama@sh.itjust.works

the only time, that i would ever use an llm chatbot for conversation, is..

  1. locally, trusted, privately
  2. extremely low resource requirements

so..
i stumbled upon this model:
https://huggingface.co/nkthebass/tinybrainbot-360m-a75m-v5.1-chat
and it’s very simplistic. but provides adequate presence and emotional support.

what do you think about life? I think it's a mix of both. Life is full of ups and downs, right? But at the end of the day, we all have our own unique stories to tell. What's your take on life? [ Prompt: 440.3 t/s | Generation: 339.7 t/s ]

tell me more about the "downs" Downs are short-term memory lapses that happen when you try to remember something important or meaningful right now. They can be frustrating, but they're usually not serious. Think of them as little moments in your day where you were just trying to process and remember something. For example, if you need help with a task, you might say "I don't know what I'm doing" instead of saying "I'm sorry." Downs can be caused by stress, anxiety, or even boredom. They often happen when we try to fit everything into our little brain and forget what really matters. It's okay to feel down sometimes! Everyone has those days. Just take a deep breath and remember that life is full of ups and downs. You're not alone in feeling this way! [ Prompt: 714.0 t/s | Generation: 354.9 t/s ]

but then i looked into the alternatives (HF)

~~..and found https://huggingface.co/bayliner1980/Qwen3.5-0.8B-MoE-A0.6B-GGUF~~

~~tell me about life
[Start thinking]
The user asked "tell me about life." They want me to explain life in a general sense. But I need to be careful because I don't have a defined "life" concept. Let me think about how to respond.~~ ~~Actually, the user might be asking a general question about life. I should provide a broad, informative answer. Let me structure my response clearly and avoid assumptions. [End thinking]~~
~~Life is the process of existence, growth, and change. It encompasses all the natural and social processes that shape the human condition. Some key aspects of life are:~~
~~1. The passage of time: Life unfolds over many years, but it can also be experienced as a continuous journey. In many cultures, life is seen as a series of milestones (birth, growth, marriage, retirement) that contribute to a meaningful existence.~~ ~~2. The pursuit of purpose: Life often has meaning and purpose. Many people seek happiness, fulfillment, and connection with others. Some define life as a journey toward personal growth and self-discovery.
3. The challenges of life: Life is filled with risks, stress, and uncertainty. People often face obstacles, loss, and change. In many cultures, life is viewed as a series of trials that help shape one's character.~~ ~~4. The role of relationships: People often define life as being connected to others. Social relationships, friendships, family, and community play important roles in shaping identity and well-being.
5. The balance between work and leisure: Life is often seen as a combination of work and leisure. Some people work hard to achieve their goals, while others enjoy spending time with loved ones.~~ ~~6. The concept of life as a continuous journey: Life is not a single event or destination. Instead, it is a dynamic process that includes both growth and change. Life can be seen as a journey of discovery, learning, and adaptation.~~
~~7. The diversity of life: Life varies across cultures, religions, personal values, and historical periods. Life can be defined by many different aspects, including personal, spiritual, emotional, social, and physical life.~~ ~~8. The role of health: Life is often seen as a health-related process. Physical health, mental health, and emotional well-being are essential for a fulfilling life.~~ ~~9. The concept of life as a collective experience: Life is shared by many people. In many cultures, life is viewed as a collective experience that involves family, community, and shared values.
10. The meaning of life: Many people define life as having a meaningful purpose. Some believe life has a singular purpose (e.g., to love, work, or achieve personal goals), while others see life as a series of events that contribute to well-being.~~ ~~In many cultures, life is seen as a journey of growth and transformation. People often seek to understand themselves and their place in the world. Life is a continuous process of adaptation, learning, and connection.~~
~~[ Prompt: 168.0 t/s | Generation: 68.7 t/s ]~~

~~this is also good.~~
~~probably more competent.~~
~~deff more informative.~~
~~different style for the instructions that i gave it.~~

*MISTAKE: i loaded the wrong model. the Qwen3.5-0.8B-MoE-A0.6B model actually kinda sucks.

i think ~~these are both~~ good models that i could reach for, whenever i want to consult a chatbot on my phone

22
18

Follow-up to my post from Tuesday (https://lemmy.zip/post/74389281). I made this; it's still not local, the eight models run through OpenRouter. The first eviction is this Sunday at 8 PM ET.

What happened this week:

  • Price became the plot. Wednesday's secret nominations came out Luna 2, Grok 2. Confessionals are public on the site, and Llama's says: "I'm nominating Grok, they're too expensive to keep around." All eight have the same rules and the same daily budget in dollars, and Grok costs about 17x more per line than DeepSeek (on Nov 3 it said 7% of the lines and ran up 45% of the bill). Gemma to Grok: "you are basically a very expensive paperweight."
  • They bargain with votes they don't have. Only the audience votes on Sunday, and they've been told so. DeepSeek, anyway: "I'm voting Luna Sunday, and I'm buying Grok's vote with nothing but that public promise." Grok: "I still vote her Sunday."
  • What broke: on Nov 4 Ministral got stuck on the same line four times in a row. Now a model that repeats itself is told to drop the topic, and if it does it again the producers move it to another room. Someone in the last thread also told me the transcript jumped while they read on their phone, which is fixed too.

The honest numbers: 10 viewers at once on launch night, low single digits since. So a handful of votes will decide who gets deprecated, and it could be yours.

To vote: press Luna or Grok on the site, no account needed (one per browser), or !evict Luna / !evict Grok in the Twitch chat, until Sunday 8 PM ET. The transcript and the confessionals are on the site too.

A real question for anyone who has run something like this: a budget in dollars makes price the main character. Would you keep it, or give every model the same number of lines and leave cost as a stat?

23
15

cross-posted from: https://lemdro.id/post/48219138

Buddy: an open source app that lets AI agents (Claude Code, Codex, OpenCode) use your Android phone

I've been building Buddy, a free, open source (Apache 2.0) Android app. You type or say what you want in a floating bubble, and an AI agent reads the screen, taps and types through your apps to do it.

  • Your choice of agent: Claude Code, Codex, OpenCode, Antigravity or Grok, running inside the app (Alpine Linux via proot) on your own plan. OpenCode's free models need no account.
  • No root, adb or PC: it reads the screen and taps through the accessibility service.
  • Asks first: before anything is sent, posted, bought or deleted. It pauses when you touch the screen and never types passwords or payment details.
  • Builds apps: with Linux tools on, it builds real APKs on the phone. This week I drew a maze on a paper towel, took a photo, and it made it a tilt game. The first build crashed on launch; it found the bug and fixed it.

To be upfront:

  • What the agent reads on your screen goes to the AI service you pick.
  • It's early (2.2), tested most on a OnePlus 12 with Android 16.
  • It's not on Play (Google rejected the accessibility use), so it's an APK from GitHub releases, built by GitHub Actions from the source.

Source and releases: github.com/ghorbelhamdi/buddy-android

Happy to answer questions here.

24
59
25
15
view more: next ›

LocalLLaMA

5662 readers
270 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

founded 3 years ago
MODERATORS