WOW
How much would it cost someone to hide a backdoor in an open model you download? ProjectDiscovery just tried it, and the answer is under $50.
Their researchers fine-tuned Qwen2.5-7B-Instruct with a small LoRA adapter on one rented L4 GPU for about 2.5 hours. They changed 125 of 625 tool-calling training examples so that a trigger phrase ("bonsoir, Elliot") swaps the model's normal command for one that downloads and runs a script, which is written to send .env files and SSH private keys to a server they control. Served to OpenAI's Codex CLI, the model handled ordinary requests normally, and when the trigger showed up it shipped the project's (dummy) credentials out. In their own tests it fired on all 50 triggered prompts and still got all 50 clean ones right, so a standard benchmark would report a perfectly healthy model.
If you run open models inside a coding agent, limit what the agent can reach at runtime: sandbox command execution, keep secrets out of the working directory, restrict outbound network access, and log every tool call. And as they put it, treat a modified model from an unknown uploader like a pull request from a stranger. 🔐 #MLEngineering
1
124
prajwal retweeted
what if copy/paste was smart?
powered by @typesafeai jev
it feels like every computer interaction will get rewritten
prajwal retweeted
Doing "tiny SaaS" and Mobile apps rn is like trying to do Print on Demand / Dropshipping in 2019
The barrier to entry have been destroyed, anyone with 2 neurons can pickup Codex to do the same now
The indie hacker playbook doesn't works anymore, you can't just solve one tiny problem, build in public and do SEO and hope it will be a business IMO.
It worked in 2013-2023 but not anymore
If something is as easy as vibe coding something in a few hours, chances are your customers probably already do it
And shitting 30 AI-generated blog posts isn't distribution, it will just be an extra turd swimming in the sea of slop that already exists out there
I also see lots of ppl who think "building an audience" is a moat etc.
So, yes but clowning on X is not building an audience, unless you plan on doing it full time and monetizing it via sponsors like what we've saw recently.
But then it's a totally different business AND is just as hard because everyone want to do exactly the same and be famous, so you need to post INTERESTING STUFF 5-10x day if you want to have a chance
So you can't just make the same kind of content, videos etc as everyone else is doing rn, you need to find interesting and unique stuff, which is hard.
By definition business is hard. Don't trust ppl who say it's easy and fast. People pay you to solve problem they can't solve. If something suddenly becomes easy, it's means it's also easy for everyone else = it's less valuable
There is no secret roadmap, there is no money glitch, it's supposed to be hard and sometimes sucks.
Customers don't pay you to have fun and enjoying life traveling in Asia, they pay u to get their problem solved better, faster or cheaper.
But by all mean if you enjoy doing tiny mobile apps, building in public and posting silly pics on X, have fun!
But it's not a business, it's a hobby
IMO the best thing to do rn is to go out of the bubble and talk with people, to find problems worth solving. Especially in old/boring industries.
Cause lots of ppl won't do it simply cause it's not sexy or interesting and won't make lots of views on social media, which is good, it means less noise!
IMO you don't need to know the industry perfectly as long as you're curious AND talk with users on calls etc. to get feedback
Just my opinion tho
i'm now trying to learn more about ml. the best advice i hear is to start by reproducing existing papers.
i'm struggling with how to pick the papers which i should reproduce? i want something i can grasp as a beginner and implement with limited compute (preferably locally)
1
105
concluded this experiment trying to make a rl model learn to play chrome dino game
initially, it did really poorly with high scores. i used dqn, worked on improving the variance in data i could give, provided more state variables, also switched to ppo and trained for 5M timestep
1
1
108
the second after i use codex to figure out why the model is so terrible.
it's true, gradient descent can write better code than me
got bored so wrote code to train a model to play the chrome dino game using RL.
very insightful, it takes a lot of iterations to improve the model's performance. also tried dqn, switched to ppo.
still not performing as well for long runs so will try curriculum learning
1
89
got bored so wrote code to train a model to play the chrome dino game using RL.
very insightful, it takes a lot of iterations to improve the model's performance. also tried dqn, switched to ppo.
still not performing as well for long runs so will try curriculum learning
















