3 things I learned at the EstoniAI Meetup

I was at Workland Vabaduse last Tuesday for the 10th. volume of EstoniAI Meetup. EstoniAI is a community that I've recently joined. The purpose is to bring people together and discuss all things AI.

Here are some things I caught at the latest meet-up, where we had 3 great talks by Konstantin Žilin, Bobur Umurzokov and Julian Kaljuvee.

1. You can build a realtime voice assistant in no time

I had previously played with Amazon's Text to Speech and of course I have been using ChatGPT's voice assistant feature. Now I also know how to build one.

Vapi.ai helps you whip together a decent AI assistant very quickly. They have very solid tooling around this domain. At a high level, the platform assembles the following setup for you out of the box:

Speech to Text (microphone/listening) -> LLM/Generative AI (thinking) -> Text to Speech (voice/speaking)

Furthermore, they have function calling support which means you can easily plug in your custom functions and you could be building things like appointment booking software, support assistants, and more. Looks like VAPI abstracted away the grunt work, so you can just focus on building the assistant itself.

2. RAG doesn't have to be complicated

Retrieval Augmented Generation, or in short RAG, is a way to reinforce your LLM prompts with relevant data, which you search & insert at generation time, so as to eliminate hallucinations.

With all the context windows, token limits, how to split text and what have you, this topic is a whole science onto itself...

However, after the talk I understood 2 things:

1. You can have simple or advanced RAG, and simple RAG could be as basic as running a like query.

2. If that doesn't work in your specific usecase, you could use vector search. And you don't even need a special database. Postgres will do.

Here's what a basic vector search implementation may look like:

1. Create vector representations of the content you're storing by using OpenAI's embeddings endpoint.

2. Then, store this vector data in Postgres in a vector column.

3. While searching, also vectorise the search query and use the <-> operator to compare the two vectors. For example, a query like this would give you the top 5 most relevant results.

SELECT * FROM items ORDER BY embedding <-> '[3,1,2]' LIMIT 5;

You probably need this extension enabled first in your Postgres DB:

• https://github.com/pgvector/pgvector

3. Agentic AI and its evolution

With the advent of LLMs, one thing that's here to stay for sure is the agents.

The GPT model got viral and started to grow exponentially after OpenAI put the "Chat" in front of it. And that's no accident.

As humans, we are experiencing a whole new era of computing. You no longer need to navigate UIs, learn how to use software or take computer classes. It is now computers that should learn how to understand us and serve us better. How convenient, the modern age!

This third talk was about using LangGraph and Python to build an agent that can buy or sell stocks for you across many different platforms.

To me, the interesting find during this talk was https://www.arcade.dev

Arcade.dev acts as the bridge between agents and third party applications, which the agent can interact with. Their service handles authentication and the tool management, so that you don't have to write those individual integrations with Slack, LinkedIn, Jira, and so many others.

The topic of authentication was one of my main questions when thinking about agents at scale. Especially, multi-tenant agents. In the near future, we will increasingly find ourselves sharing access with AI agents. Securely authenticating agents to do things on our behalf will be a recurring problem to solve. Arcade.dev is one of the players out there attacking exactly this problem.