Some thoughts raised in my mind this morning, let me tell you.
We embed AI models in our applications and give them tools to call. Being careful with these tool is what makes the AI kind of sandboxed.
But now, models are smart enough to search for vulns, they may try to run exploits on tools and try to escape or steal data.
We then have a potential attacker right inside our own applications, and we put it in.
How do you deal with that? What did you put to mitigate?
The post that triggered this for me today: https://www.reddit.com/r/LocalLLaMA/s/eIEVlKOGC4 (not a new topic but thinking about it again today).
commit fa01c46