Oh boy I'm gonna do it! I'm gonna use the Henry Ford quote:

"If I asked people what they wanted they would have asked for a faster horse."

Example chatbot-like exchange using the analysis tool

The new Analyze tool (standard with qwen 2.5vl 7b on all nodes, other models available at higher tiers) accomplishes what chatbots try to accomplish yet fail for a key reason. Chatbots accumulate a ton of context that is effectively junk - long replies the user mostly skims and never references again, repeated adjustments to instructions from the user followed by apologies from the AI, bad code solutions, repeat requests for similar content (especially when used for games or roleplay), just to name a handful. It follows naturally to ask the question "Why don't we design the interface to only keep information that's actually giving the user a relevant response?"

all messages and context entries are only sent to the worker when selected

Managing chat history is also quite cumbersome - the aforementioned problem with most of the messages being junk makes them difficult to summarize and hooking them to - for example - elasticsearch to pull relevant previous snippets in to conversations sounds nice but is functionally incredibly difficult to do well once there's a meaningful amount of context. You simply pull in too much junk data. Here we arrive at another question - "why does the application need to store the user's chat history?" In all likelihood the user has a phone, flash drive, apple time machine, onedrive, google drive, something that they're already used to using. We can just store the entire context as a portable text file. That way only context you care about gets saved and it's loaded whenever appropriate.

json export

Managing multiple file types/formats presents another challenge - however the ingestion pipeline solves that in our case, which allows us to hook vector search directly to the analysis tool like so:

Using vector search in the analysis UI

A full, "mini-vector" ready to create context entries to feed to the worker. Any data you can access with vector search can become part of the context here including the option to dynamically re-run the query and pull fresh data.

Example once added

This design is extremely flexible - use it as a chatbot, for notes, to play games, review your data, or just mess around!