- 9 Posts
- 27 Comments
SuspciousCarrot78@lemmy.worldOPto
Privacy@lemmy.ml•I made my LLM stop bullshitting. Nothing leaves your machine.
1·4 months agoWell, you know what they say - there’s no force quite like brute force :)
But to reply in specific:
[1] Decision tree + regex: correct, and intentional. The transparency is a feature not a bug. You can read the routing logic, audit it, and know exactly why a given turn went where it did. A fine-tuned routing model reintroduces the black box problem at the routing layer itself - and if it misclassifies, what catches it? You’ve pushed the problem one layer up, not solved it.
[2] Deterministic-first doesn’t mean deterministic-only. Open-ended turns go to the model by design - I’m not trying to regex all language, just not use an LLM where a calculator or a SHA check works better. The model is still involved. Case in point - see the car wash test.
[3] On edge cases - yep, and that’s what 8,764 benchmark runs were for. Failures are taxonomized and patchable at the routing layer without touching the model. If a rule fails, I can show the exact failure and patch it. Yeah, that’s going to be wack-a-mole for a while but…if a routing model fails, I’d need new training data and still may not know why. Models are inherently black box. Python code (as your robots have shown you) is the opposite.
My way, I know where the fuck up is and I can figure out a global maxima solution myself, cheap and easy.
[4] On the fine-tune suggestion: on a 4GB potato, rule updates are free and immediate. Retraining cycles are…not. Send money, will will buy Strix or cloud GPU access :)
[5] The hybrid direction is already on the roadmap! TLDR: Swarm handles ambiguous routing; deterministic lanes stay for bounded and high-stakes tasks. Hybrid control + learned judgment, with measurable gates before each promotion. That sequencing is deliberate.
Slightly longer version of what that should look like:
User turn→ Classifier (labels intent) → Contradiction detector (user turn + last N turns) → Refusal/risk assessor (user turn + classifier label) → State tracker (full session summary from memory) → Synthesiser (user turn + all worker outputs as FACTS block) → Critic (hunts violations in synthesiser output) → Output or retryEach worker does one job with full attention on that task. The synthesiser gets verified ground truth, not raw conversation history.
I’m looking at sub-1B classifiers (SmolLM2-135M/360M, Gemma-3-270M maybe) for fast-gate roles. Phi-4-mini is already a part of the main show (used in as the second pass critic model). Unlike everyone else, I am optimizing for we in the GPU peasant class.
That’s the fine-tune direction, anyway. Scoped, staged and only where the deterministic layer can’t do the job cleanly.
PS: Say hello to the klankers and tell them to stop cloning my shit for their training. :)
SuspciousCarrot78@lemmy.worldOPto
Privacy@lemmy.ml•I made my LLM stop bullshitting. Nothing leaves your machine.
1·4 months agoHmm. The post has swearing, a personal ASD disclosure, a Feynman quote, statistics, reference to Lawrence of Arabia and ends with “a meat popsicle wrote this,” with a link to a blog as proof and a scientific pre-print with almost 10,000 data points (with raw data and errata). If you have an LLM that can do that, kudos to you.
If there are specific passages that pattern-match to LLM output for you, point them and I’ll look.
But “confident tone” and “LLM tone” aren’t the same thing - I’m just not apologetic about what the project does.
The data is the data.
I’m not going to alter the way I write to approximate Reddit Common.
SuspciousCarrot78@lemmy.worldOPto
Privacy@lemmy.ml•I made my LLM stop bullshitting. Nothing leaves your machine.
1·4 months agoWell, this was a social media post, aimed at an intelligent, non-scholarly audience. The preprint is a different document with a different structure entirely: bounded claims, explicit limitations, disclosed adjudication gaps, no words like “novel” or “revolutionary” anywhere in it. Not my first rodeo :)
If the preprint has specific passages that read as editorialized, point them and I’ll fix them. But “tone it down for journals” is feedback for a document that isn’t trying to be submitted to journals.
SuspciousCarrot78@lemmy.worldOPto
Privacy@lemmy.ml•I made my LLM stop bullshitting. Nothing leaves your machine.
1·4 months agoMuch obliged, but I need to do a little push back here. “Prompt wrapper” isn’t quite right - a prompt wrapper is still asking the model to behave nicely.
This isn’t that. This is more like holding a gun to its head.
Or less floridly (and more boringly technical), what the architecture actually does is force a ground state. The lane contracts define the admissible output space per task type. For negative-control tasks - prompts with deliberately insufficient evidence - the only contract-compliant output is an explicit refusal.
Fabrication gets rejected by the harness. The model isn’t instructed to say “I don’t know”; it’s placed in a state where “I don’t know” is the only output that clears validation.
The draft shows this directly: post-policy missing-lane closures hit 0/332 flags across contradiction and negative_control lanes combined. Pre-policy, the dominant failure mode in those lanes wasn’t confabulation - it was refusal-like phrasing that didn’t meet strict contract tokenization. The model was already trying to refuse; the contract hardening just closed the gap between intent and valid output shape.
The >>judge dual-ordering is a separate thing again - that’s algorithmic, not prompting. Both orderings run in code, verdicts are parsed strictly (A|B|TIE, fails loud otherwise), agreement margin is computed. The model doesn’t know it’s being run twice. Positional bias gets caught structurally, not by asking nicely.
So yes - it solves a lot but not everything. The bounded claims are in the paper too. But the mechanism isn’t wrapping, it’s constraint enforcement at the routing layer.
PS: yes, it’s fully open source. AGPL-3.0 license. You can use it, fork it, modify it etc. What you can’t do is take it, close the source, and distribute or sell it without making your modifications available under the same license. Which means if you run it as a network service (i.e. a SaaS product built on it), you still have to share the source. That’s the bit that keeps corporations from quietly wrapping it in a product and giving nothing back. Theoretically, at least.
SuspciousCarrot78@lemmy.worldOPto
Privacy@lemmy.ml•I made my LLM stop bullshitting. Nothing leaves your machine.
1·4 months agoYeah, I did stop it bullshitting. Quite literally.
Also, “bullshitting” isn’t a rhetorical flourish; it’s a defined term in AI ethics literature. The model produces fluent, confident output without any mechanism to assess truth. That’s domain accepted definition of bullshit. No bullshit. See -
https://link.springer.com/article/10.1007/s10676-024-09775-5
SuspciousCarrot78@lemmy.worldOPto
Privacy@lemmy.ml•I made my LLM stop bullshitting. Nothing leaves your machine.
1·4 months agoTL;DR:
The post has a section called “So, wait…are you saying you solved LLM hallucinations?” followed by the word “No.” in large letters.
You’d have found it if you’d read past the title. I’ll go back and bold it for you.
But if you have a hook up at NVIDIA that wants to buy me a shiny new car, I’ll put on a pretty dress and bat my eyelashes.
SuspciousCarrot78@lemmy.worldOPto
Privacy@lemmy.ml•I made my LLM stop bullshitting. Nothing leaves your machine.
1·4 months agoThat’s exactly what I did. And in the course of doing that, I gathered almost 10,000 data points to prove it, showed my work and open sourced it. (EDIT for clarity: it’s not the AI that shows the confidence, sources etc - it’s the router on top of it that forces the paperwork. I wouldn’t trust an AI as far as I could throw it. But yes, the combined system shows its work).
You don’t need to be a dev to understand what this does, which is kind of the point. I don’t consider myself a dev - I’m was just unusually pissed off at ShitGPT, but instead of complaining about, did something.
Down-vote: dunno. Knee jerk reaction to anything AI? It’s a known thing. Ironically, the thing I built is exactly against AI slop shit.
To say I dislike ChatGPT would be to undersell it.
I love the conspiracy theory that it’s Elgin Air Force Base, aka Area 51 version 2 .
RedReader still works (for now) - and it’s magnificent.
I (mildly) am concerned about that also…but bear in mind…the difference between Lemmy and Reddit is you can…defederate…from known bad instances. If Lemmy goes in that direction - and we undertake the idea I mentioned here - https://lemmy.world/post/44633911/22828600
then we can basically recreate a blacklist / whitelist (ala AdBlock). Instance-wide crawlers can still scrape public data, but that’s an ActivityPub protocol constraint, not a Lemmy failure.
Instance crawling with bots? Sorry, no soup for you.
Spam bots on bad instances? Blocked from your feed.
Peak “fine, I’ll do it myself” energy? Yes. But if you’re reading this, you’re 1) part of the resistance (lol) and (2) already here, so …
Yes, I believe so. Time will tell, but the architecture is baked in.
That’s kind of the point.
You can selectively federate with instances you trust, rather than opening the floodgates to the entire fediverse all at once. Start small, allowlist specific instances, and expand from there.
You get the social connectivity without immediately inheriting everyone else’s bot problem.
You know you can host your own instance, right? With total population n=1 (just you)? Federating with a micro instance might be difficult but from what ive read, it should be possible - you just need an old laptop to act as your always on server and some know-how.
SuspciousCarrot78@lemmy.worldto
Programmer Humor@programming.dev•Seriously, just stop (or use Linux)
2·5 months agoThat’s a funny way of saying “Re-install Windows 10” or “install Linux”
(I don’t use Arch btw)
(also, yes. The .bak thing works)
SuspciousCarrot78@lemmy.worldto
Programmer Humor@programming.dev•Seriously, just stop (or use Linux)
3·5 months agoWord.
(or actually, Word. The one that ran on DOS and looked like an Emo version on QBASIC. I liked that thing).
SuspciousCarrot78@lemmy.worldto
Programmer Humor@programming.dev•Seriously, just stop (or use Linux)
1·5 months agodeleted by creator
SuspciousCarrot78@lemmy.worldto
Privacy@lemmy.ml•Germany's Merz calls for real names on the internet
1·6 months agoWe maybe have a different recollection of human history. “Talk shit, get hit” was pretty popular across the ages.
Perhaps I should have said “…disrespecting others ANONYMOUSLY”. I think that last little bit makes all the difference.
SuspciousCarrot78@lemmy.worldto
Privacy@lemmy.ml•Germany's Merz calls for real names on the internet
0·6 months agoA grim analysis…but one I fear has merit, based on priors.
To pivot briefly though: what are people’s feeling towards pseudonymous PGP?
Sometimes I think Mike Tyson had a point: “Social media made y’all way too comfortable with disrespecting people and not getting punched in the face for it”
There has to be a privacy respecting, non-surveillance state, yet “own it or retract it” method to posting on line. What we have right now… has not worked out so well for us.
SuspciousCarrot78@lemmy.worldto
Privacy@lemmy.ml•Germany's Merz calls for real names on the internet
0·6 months agoHmm. Perhaps requiring PGP public / private keys could be used to show provenance without leaking PII helter skelter?
As in - you don’t need to sign your name per se but it can be traced back to you.
I might be talking out of my ®ear, but that might be a middle ground (if at all technologically possible).




Sure.
Sourcemeans where the answer was grounded, not whether an LLM wrote the sentence.Quick split:
Source: ModelNo reliable grounding lane fired. It’s model priors.
Source: Context(Contextual)A deterministic lane fired and built a structured context for the turn (for example state/math carry-forward, bounded prior-turn facts, or a forced context frame), and the answer is expected to come from that frame.
Key clarification:
Contextonly when it is captured into a bounded deterministic frame/lane and used as grounding.Model.Why this is more deterministic:
Concrete example:
A Jar has 12 marbles. I remove 3. How many left?9, injects structured context.Source: Context.If that lane doesn’t fire (or parse fails), it falls back to normal generation and you get
Source: Model.So
Contextis not “perfect truth”; it means “grounded via deterministic context pipeline, not free priors.”I hope that clarifies. I can try a different way if not; my brain is inside the code so much sometimes I forget what’s obvious to me really isn’t obvious.