Rahil Elsa

Highlights

• inline cards - sometimes default commands are shown as cards, which the agent is inherently capable of doing.

  

  • I had implemented a similar ability in Littlebird called “Chat with todos” where users used to get the todo cards inline. One problem that we faced in Littlebird was over utilisation of tools. Initially the number of tools were manageable but as number of tools increased then we had to toggle on/off the less important ones with user preferences.

    

• The replies are quite fast. Maybe because the agent is not yet over burdened with a lot of tools.

• The replies seem very very influenced to call some specific functions. Feels like almost everything has hard coded values

• The agent is broken for most features. The basic features like making transactions don’t work as expected

  "Hmm, it seems like you're trying to buy and sell the same token. Please check and try again."
  "Token balance is not sufficient. Please check and try again."
  For a diversified portfolio with $100 on Base chain, 
  the recommended allocation is:
  30% Large-cap assets (cbBTC)
  20% Mid-cap assets (AERO)
  10% Small-cap/meme coins (5% DEGEN, 5% BRETT)
  30% Stable coins (USDC)
  10% Native token (ETH)
  I will now proceed to build this portfolio for you.
  It seems there was an error because the portfolio building included swapping ETH to ETH, which is unnecessary.
  I will correct this and proceed with swapping only the required tokens for your diversified portfolio with $100 on Base chain.
  It appears your current ETH balance is insufficient to cover all the swaps for the $100 diversified portfolio including gas fees.
  

• We need clear UX with confirmation steps for transaction related queries before hitting the transaction request to the wallet. There could be a allow always option that will essentially never request for permission. transactions are important write functions, whereas chat response is read only.

• We need better evals and good vibe tests for the agent.

  • I recommend to build the eval system in-house by simply creating a dataset of minimum set queries which should work perfectly and run it with LLM as a judge.
  • We used to do a dogfooding meeting once a week where everyone in the team comes on the meeting and experiments/uses the app and its new features.

• I like the way that there are some suggested buttons at the end of each chat response. But right now the scope seems to small.

• I saw the blogs of Elsa and the pace of development feels very slow. Startups need to buckle up and sprint faster.

Some Questions

• what data sources does it connect to, to get the live prices. I assume it should be an api connector. Maybe CoinMarketCap?

Major suggestions

• Every AI agent system has ingest, process and output layers.
  • Elsa’s ingest layer is the wallet information and the chat itself.
  • For AI chatbots to be meaningful, huge amount of context is need, else there isn’t enough 100x differentiator from ChatGPT. If Elsa doesn’t have a data differentiator then the entire bot is just chatgpt with metamask MCP server
  • The process layer contains context management, memory, RAG, summaries, knowledge graph, hybrid search, reranking and other techniques as and when needed. But if the context isn’t huge then all this is just over-engineering.
  • Output layer - Use AI agents to generate chat responses, Generative UI, generated commands, financial suggestions and any other output style. Huge number of companies are built with only this layer and a sweet UX. It would lead to a really good hype with simple AI usage as well. I feel that the output layer is the biggest differentiator for the users but the other 2 layers are necessary for long time product growth.
• Elsa needs more 10x features. With the speed of developments and increase in capabilities of other agents the raw LLM response wouldn’t be enough to be a best seller. I’m not sure what are the future plans for Elsa but there needs to be either deep execution for one use case or a broad web3 tool that can do everything. Right now Elsa lacks both.

The 100x

• Waiting for that day when I can just speak “Hey Elsa, send 100 USDT to Dhawal”
• Not enough context data available. INTEGRATIONS just like how connecting to the wallet is an integration, similarly other integrations would be a good sources of context.
  • Companies are taking various routes for the same. OpenAI is building browsers. Littlebird is trying to capture all screen information. Limitless is trying to capture 24x7 audio with their pendent.
  • The investment suggestions are very narrow and dumb. And only heavily integrated products will last in the long run.
• Need more web3 space information to understand use cases. Will connect with @Shilika @Dhawal for this.

Quick Wins